Learning an end-to-end spatial grasp generation and refinement algorithm from simulation

Novel object grasping is an important technology for robot manipulation in unstructured environments. For most of current works, a grasp sampling process is required to obtain grasp candidates, combined with a local feature extractor using deep learning. However, this pipeline is time–cost, especially when grasp points are sparse such as at the edge of a bowl. To tackle this problem, our algorithm takes the whole sparse point clouds as the input and requires no sampling or search process. Our work is combined with two steps. The first step is to predict poses, categories and scores (qualities) based on a SPH3D-GCN network. The second step is an iterative grasp pose refinement, which is to refine the best grasp generated in the first step. The whole weight sizes for these two steps are only about 0.81M and 0.52M, which takes about 73 ms for a whole prediction process including an iterative grasp pose refinement using a GeForce 840M GPU. Moreover, to generate training data of multi-object scene, a single-object dataset (79 objects from YCB object set, 23.7k grasps) and a multi-object dataset (20k point clouds with annotations and masks) combined with thin structures grasp planning are generated. Our experiment shows our work gets 76.67% success rate and 94.44% completion rate, which performs better than current state-of-the-art works.