Motion removal for reliable RGB-D SLAM in dynamic environments

Highlights

• An on-line RGB-D data-based motion removal approach is proposed.

• The approach does not require prior knowledge from moving objects.

• The approach improves RGB-D SLAM in various dynamic scenarios.

Abstract

RGB-D data-based Simultaneous Localization and Mapping (RGB-D SLAM) aims to concurrently estimate robot poses and reconstruct traversed environments using RGB-D sensors. Many effective and impressive RGB-D SLAM algorithms have been proposed over the past years. However, virtually all the RGB-D SLAM systems developed so far rely on the static-world assumption. This is because the SLAM performance is prone to be degraded by the moving objects in dynamic environments. In this paper, we propose a novel RGB-D data-based motion removal approach to address this problem. The approach is on-line and does not require prior-known moving-object information, such as semantics or visual appearances. We integrate the approach into the front end of an RGB-D SLAM system. It acts as a pre-processing stage to filter out data that are associated with moving objects. Experimental results demonstrate that our approach is able to improve RGB-D SLAM in various challenging scenarios.

Introduction

Simultaneous Localization and Mapping (SLAM) is a fundamental step for many robotic applications. It concurrently estimates robot poses and reconstructs traversed environment models. Many effective SLAM algorithms using visual sensors, such as monocular cameras [1], stereo cameras [2] and RGB-D cameras [3], have been proposed over the past years. Related technologies, such as the augmented reality [4] and autonomous driving [5], have benefited from the development of the SLAM technology. It is worth noting that the advent of the RGB-D cameras has changed the computer vision world [6]. They provide colored point clouds with real-scale distance information, which greatly benefits the dense 3-D environment reconstruction. Many impressive RGB-D SLAM systems have been developed [[7], [8], [9], [10], [11], [12]] in recent years, and most of them adopt the graph optimization framework. We refer readers to this survey [13] to get an overview of the progress on the graph SLAM algorithm.

However, virtually all the current RGB-D SLAM algorithms are proposed under the static-world assumption. It requires that there is no moving object existing in the environment during the traversal of robots. The data associations in the SLAM front end can be hindered by moving objects. With incorrect data associations fed into the SLAM back end, the graph optimization process could be severely jeopardized, which finally leads to a catastrophic failure for the localization and mapping processes. Thus, the technology of RGB-D SLAM is still vulnerable in dynamic environments.

The data associations in the SLAM front end consists of two components, namely, the short-term data association and the long-term data association [13]. The short-term data association determines adjacent pose estimations, while the long-term one has an impact on the loop detection. Take as an example the sparse feature-based RGB-D SLAM. Standard robust estimators, such as the RANdom SAmple Consensus (RANSAC) algorithm [14], are usually employed in the SLAM front end to reject outlier feature associations. However, it is hard to reliably reject outliers when moving objects are not trivial in the camera field of view. In such a case, the outliers are unavoidably used for computing the robot poses, which makes the pose estimation erroneous. When a robot returns to a previously visited place where moving objects have gone away, the loop detection would be confused by matching the same scene but with different visual appearances. If we eliminate moving objects at the first exploration of a place, we can compare the image frames using just the static feature points, which would lead to a much more reliable loop detection result. Moreover, we can merely use the static feature points to get accurate robot pose estimations. Therefore, eliminating moving objects is able to reduce the incorrect data associations, which is critical to improve the SLAM performance.

In this paper, we develop a novel RGB-D data-based motion removal approach to address the problem of RGB-D SLAM in dynamic environments. We refer to the dense pixel-wisely moving-object segmentation as motion removal. Our approach serves as a pre-processing stage to filter out data that are associated with moving objects. With our approach, incorrect data associations can be greatly reduced in the SLAM front end.

Fig. 1 qualitatively compares the resulting point-cloud maps produced by the ORB-SLAM system [11] and the system integrated with our motion removal approach in a dynamic environment. In this scenario, two persons are playing with a basketball in an office room, where the moving objects are the two persons and the basketball. We perform the SLAM algorithm in this environment using a hand-held Asus Xtion RGB-D camera, which is moved in a circle-like trajectory in the scene. Comparing Fig. 1 a with b, we could see that the quality of the resulting map is substantially degraded. Almost no object or scene structure is correctly aligned. This is mainly caused by the jeopardized camera pose estimations due to the incorrect data associations. Moreover, moving objects are recorded as spurious objects in the resulting map, which makes the map virtually useless for future applications, such as the map-based navigation. The point-cloud map built with our motion removal approach is displayed in Fig. 1 c and d. The models for the desks, monitors, wall, floor and scene structure are correctly built. Virtually no point from the moving objects is recorded in the map. The holes caused by motion removal in the point-cloud map are complemented by the frame fusion with redundant scans. Fig. 1 clearly illustrates the negative effect caused by moving objects in dynamic environments. It should be noted that the tested environment shown in Fig. 1 is a small-scale environment. We believe that using such an example would be sufficient to illustrate the negative impact caused by moving objects. This is because large-scale environments are generally more challenging for SLAM algorithms. By performing the experiments in such a small-scale environment, the ORB-SLAM system has already given the unsatisfied performance. The performance in large-scale dynamic environments would not be better than this. The main contributions of this paper are summarized as follows:

• We propose a novel RGB-D data-based on-line motion removal approach. A foreground model is built and updated incrementally. No prior information of moving objects, such as semantics or visual appearances, is needed.

• We explain why we use motion removal to address the problem of RGB-D SLAM in dynamic environments. The experimental results confirm that the robustness of RGB-D SLAM can be increased with motion removal.

• We integrate our motion removal approach with an RGB-D SLAM system. Evaluations and method comparisons are performed with the widely used TUM RGB-D benchmark dataset [15].

The remainder of this paper is organized as follows. Section 2 reviews the related work. Section 3 formulates the problem of RGB-D SLAM in dynamic environments and explains why we use motion removal to address this problem. Sections 4 The approach overview, 5 The proposed approach give the overview and the details of our approach, respectively. The experimental results are presented in Section 6. We conclude this paper and discuss the future work in the last section.