Conceptualization of the Smart Factory started with introducing the Industry 4.0 paradigm and its nine pillars, which it stands. The paradigm itself is automation and robot-centric focused, which means less and less involvement of the humans on the manufacturing shop floor. However, even robots and simulation aspects of the factories are the most crucial aspects; Industry 4.0 still focuses on the Augmented and Virtual Reality (AR and VR input methods for the human operators, making the smooth transition to the Industry 5.0 concept a human-centric. Although VR/AR is still being enabled and widely used in the Human-Robot Interaction (HRI) research aspect, the heavy headset is limited in the observation field of view. The input methods, such as headsets, have voice and gesture recognition; however, those are mainly limited by factory noise and cameras pointing to the human hands. These headsets constrain the use of smart wearables to a given boundary inside the factory environment. A Shape Memory Alloy (SMA) based haptic glove with discrete data outputs from the kinaesthetic analysis of the hand bending can remove the need for gesture recognition. The paper proposes a modular framework using the SMA-based Haptic Gloves in the Smart Manufacturing environment. These gloves, without additional wearables, can enable interactions with heavy machinery, screens, and all other assets of the industrial area, even with holographic. In this paper, the authors aim to prose the context, design, and framework with the chosen use-cases mainly based on the robotic system applications in the Technological University of the Shannon: Midlands Midwest (TUS: MMW), Ireland, and Tallinn University of Technology (TalTech), Estonia.