Refined Human–Computer Interaction: Enhancing Efficiency and Collaboration

Optimized human and computer interaction aims to enhance communication and collaboration between humans and machines by creating interfaces that streamline interaction and minimize user cognitive load and physical effort. This paper explores the feasibility of utilizing hand gestures as a means to interact with computers and Internet of Things (IoT) devices, employing a monocular camera system and machine learning algorithms. The study consists of two parts: software interaction and interaction with IoT devices within a private network. A customized architecture has been developed, leveraging Google's Mediapipe Hands library to detect hand gestures, coupled with a machine learning model for gesture classification. The classification output is translated into real values for controlling various components, encompassing ranges, categories, positions, or continuous values. This technology enables users to regulate parameters such as brightness, volume, and cursor movement, as well as perform tasks like software manipulation and control of IoT devices like relays and robot clippers. The model demonstrated satisfactory accuracy in controlling components within games and virtual reality environments.