An IMU-compensated skeletal tracking system using Kinect for the upper limb

The Kinect device is being increasingly used in conjunction with rehabilitative actions. However, the use of Kinect as a skeletal tracking system requires several further modifications and technological breakthroughs. This study used inertial measurement units (IMUs) to complement skeletal tracking with the Kinect. The IMUs were used to compensate for errors in calculating shoulder and elbow joint angles detected by the Kinect device while the patients performed rehabilitation movements. Thirty normal participants were recruited, and their shoulder and elbow joint angles were recorded and calculated using the Kinect and IMUs while they moved during movement games. If movement with a larger measuring error was detected, the measurement was directed to the IMU to calculate the angle and calibrate the angles measured by the Kinect device. The mean percent errors of the Kinect measurements with respect to the IMU measurement at the shoulder joint during shoulder flexion and rotation at 90° of shoulder flexion were 15.08 ± 4.13 and 26.00 ± 7.41%, respectively. The mean percent errors of the Kinect measurements with respect to the IMU measurements at the elbow joint during shoulder flexion, shoulder rotation at 90° of shoulder abduction, and shoulder rotation at 90° of shoulder flexion were 12.92 ± 2.43, 17.75 ± 4.91, and 23.3 ± 7.01%, respectively. The mean percent errors for the participants’ shoulders in Game 2 and Game 3 were 15.47 ± 4.88 and 28.13 ± 8.51%, respectively, and the mean percent errors of the participants’ elbows in Game 3 were 55.62 ± 13.74%. The proposed method to calibrate the angles detected using the Kinect have a greater mean accuracy rate (84.58%) and a higher processing rate (10 ms/frame) than traditional methods that use only Kinect or IMUs. The proposed system increases the accuracy of movement detected by the Kinect device, and this increases the processing rate of the IMUs, thereby improving clinical practicality.