Human Avatar Robotic Puppeteering (HARP)

The Human Avatar Robotic Puppeteering (HARP) project is focused on studying whether homologous puppeteering (controlling via mimicry) is an effective control principle for robots, given minimal user training. This project aims to develop a practical implementation at low-cost. The HARP project is a three-joint robotic crane that is capable of grasping objects. In order to control via puppeteering, the system tracks the user’s hand moving in free space in real time as an avatar. This implementation relies on a Microsoft Kinect ($150) and a Leap Motion sensor ($100). This low-cost prototype is a proof-of-concept for a natural interface between user and robot, allowing gestures to be the method of communication rather than the traditional button-and-switch method. The system uses the Xbox Kinect to track the hand in reference to a known point on a table. This position is mapped using the imaging camera sensor in the Kinect, and an inverse kinematic algorithm is used to translate that position to the joint-angles for the crane. The grasping of the user’s hand is sensed with the Leap Motion sensor. A key contribution to the research field is the blending of two different gesture based sensor systems to form a robust control interface.