Hybrid affective computing—keyboard, mouse and touch screen: from review to experiment

Emotions play an important role in human interactions. They can be integrated into the computer system to make human–computer interaction (HCI) more effective. Affective computing is an innovative computational modeling and detecting user’s emotions to optimize system responses in HCI. However, there is a trade-off between recognition accuracy and real-time performance in some of the methods such as processing the facial expressions, human voice and body gestures. Other methods lack efficiency and usability in real-world applications such as natural language processing and electroencephalography signals. To accomplish a reliable, usable and high-performance system, this paper proposes an intelligent hybrid approach to recognize users’ emotions by using easily accessible and low computational cost input devices including keyboard, mouse (touch pad: single touch) and touch screen display (single touch). Using the proposed approach, the system is developed and trained in a supervised mode by artificial neural network and support vector machine (SVM) techniques. The result shows an increase in accuracy of 6 % (93.20 %) by SVM in comparison with the currently existing methods. It is a significant contribution to show new directions of future research in emotion recognition, user modeling and emotional intelligence.