A Graph-Based Deep Reinforcement Learning Approach to Grasping Fully Occluded Objects

Grasping in cluttered scenes is an important issue in robotic manipulation. The cooperation of grasping and pushing actions based on reinforcement learning is an effective means to obtain the target object when it is completely blocked or there is no suitable grasping position around it. When exploring invisible objects, many existing methods depend excessively on model design and redundant grasping actions. We propose a graph-based deep reinforcement learning model to efficiently explore invisible objects and improve the performance for cooperative grasping and pushing tasks. Our model first extracts the state features and then estimates the Q value with different graph Q-Nets according to whether the target object is found. The graph-based Q-learning model contains an encoder, a graph reasoning module and a decoder. The encoder is used to integrate the state features such that the features of one region include those of other regions. The graph reasoning module captures the internal relationships of features between different regions through graph convolution networks. The decoder maps the features transformed by reasoning to the original state features. Our method achieves a 100% success rate in the task of exploring the target object and a success rate of more than 90% in the task of grasping and pushing cooperatively in simulation experiment, which performs better than many existing state-of-the-art methods. Our method is an effective means to help robots obtain completely occluded objects by grasping and pushing cooperation in the cluttered scenes. The verification experiment on the real robot further shows the generalization and practicability of our proposed model.