SatMVS: A Novel 3D Reconstruction Pipeline for Remote Sensing Satellite Imagery

Recently, 3D reconstruction based on satellite imagery has been a hot topic in the remote sensing community. Its output called the digital surface model (DSM) can be widely used in urban planning, military navigation, and so on. Nowadays, almost all satellite image 3D reconstruction pipelines are based on traditional stereo matching algorithms which have low accuracy and long runtime. In contrast, the neural networks based on multi-view stereo (MVS) have shown great reconstruction performance in the computer vision community. To transfer the advanced MVS neural networks to the remote sensing community, we propose a novel 3D reconstruction pipeline called SatMVS. First, the input satellite images and their rational polynomial camera parameters (RPC) are cropped into small tiles according to the designated output DSM region. Second, the RPC parameters are converted to the projection matrix for the homography transform which is the core step in MVS neural networks. Third, the advanced MVS neural network is applied to estimate height maps from satellite images. At last, all inferred height maps from small tiles are converted to 3D points in Universal Transverse Mercator (UTM) coordinate system and fused to get the final complete DSM. In order to train and test SatMVS, we build a novel satellite imagery 3D reconstruction dataset called SatMVS3D dataset, which contains satellite images, RPC parameters, and height map ground truth that covers about 3km². The experimental results on the SatMVS3D dataset demonstrate that our proposed pipeline can provide robust reconstruction performance.