Robot Learning of Everyday Object Manipulation Using Kinect

In this paper, we provide a solution of teaching a robot to perform tasks through human demonstration. On the one hand, we consider many everyday complex task manipulations made up of manipulation primitives, consisting of a series of sequential rotations and translations. On the other hand, we design demonstration primitives which decompose a demonstrated task. We use Kinect sensor to locate some hot points’ position of a teacher’s hand during demonstration, in order to gain the axes of those rotations and translations which will support the manipulation primitives that will contribute to building task descriptors. Based on this, we quote a descriptor to represent this manipulation primitive. We also teach the robot where to start and where to end once a manipulation primitive is operating via recording the special two points’ coordinates in a real scene while demonstration, which could reinforce robots’ learning to those effective and sequential demonstrations and shorten the learning time. One manipulation primitive corresponds with one demonstration primitive. Two kinds of primitives will be connected to the axes of trajectory. After that, we perform experiments to discuss the universality of this framework.