8. Human Hand Motion Analysis with Multisensory Information

In order to study and analyse human hand motions which contain multimodal information, a generalised framework integrating multiple sensors is proposed and consists of modules of sensor integration, signal preprocessing, correlation study of sensory information and motion identification. Three types of sensors are integrated to simultaneously capture the finger angle trajectories, the hand contact forces and the forearm electromyography (EMG) signals. To facilitate the rapid acquisition of human hand tasks, methods to automatically synchronise and segment manipulation primitives are developed in the signal preprocessing module. Correlations of the sensory information are studied by using FEC introduced in Chap. 6 and demonstrate there exist significant relationships between muscle signals and finger trajectories and between muscle signals and contact forces. In addition, recognising different hand grasps and manipulations based on the EMG signals is investigated by using FGMMs presented in Chap. 5 and results of comparative experiments show FGMMs outperform GMMs and Support Vector Machine (SVM) with a higher recognition rate. The proposed framework integrating the state-of-the-art sensor technology with the developed algorithms provides researchers a versatile and adaptable platform for human hand motion analysis and has potential applications especially in robotic hand or prosthetic hand control and Human Computer Interaction (HCI).