RSDCN: A Road Semantic Guided Sparse Depth Completion Network

Laser radar (Lidar) plays an indispensable role in lots of security critical applications such as autonomous driving. However, the high sparsity and non-uniformity nature of the raw laser data brings large difficulties to reliable 3D scene understanding. Traditional depth completion methods suffer from the highly ill-conditioned nature of the problem. A novel end-to-end road semantic guided depth completion neural network with a special designed Asymmetric Multiscale Convolution (AMC) structure is proposed in this paper. The whole network is composed of two parts: semantic part and depth completion part. The semantic part is constructed by an image-Lidar joint segmentation sub-network which produces semantic masks (ground or object) to the following network. The depth completion part is composed of a series of AMC convolution structure. By combining the semantic masks and treating the ground and non-ground objects separately, the proposed AMC structure can well fit the depth distribution pattern implied in road scene. The experiments carried on both synthesized and real datasets demonstrate that our method can effectively improve the accuracy of depth completion results.