Introduction
-
A LiDAR-based DNN which infers an absolute gravity direction which does not contain the robot’s own acceleration nor vibration is proposed. Unlike camera-based methods, the DNN can estimate it regardless of day or night.
-
The DNN inference is integrated with a gyroscope for real-time estimation.
-
A data transformation and augmentation method for the LiDAR and IMU data is proposed. Converting the 3D point cloud into a 2D depth image reduces the computation time in the DNN.
-
Pre-training the DNN with large synthetic data before fine-tuning it with real data makes the learning efficient, while the related work [16] uses only real one.
DNN estimating gravity vector
Coordinate definition
Dataset collection
Data preprocessing
LiDAR data transformation
IMU data transformation
Network
Loss function
Optimization
EKF-based real-time estimation
Prediction process
Update process
Validation
Static validation of DNN
Method list
-
LiDAR DNN (ours): ‘LiDAR DNN (ours)’ denotes the proposed method described in the section above.
-
Statistics: ‘Statistics’ denotes a method using the average of the label vectors as outputs for all samples, which means \(\sum _{\iota =0}^{\#D} \varvec{g}_\iota\) is used for estimating attitudes of all samples. Computing the error of this method is equivalent to calculating the standard deviation of the dataset. This method is regarded as the baseline in this study.
Training
id# | Environment | #Samples | Usage | |
---|---|---|---|---|
1 | Sim. | AirSim’s | 10000 | Training |
2 | ---"--- | ‘Neighborhood’ | 1000 | Test |
3 | Real | Area-I | 1941 | Fine-tuning |
4 | ---"--- | Area-II (daytime) | 443 | Test |
5 | ---"--- | Area-II (nighttime) | 447 | Test |
6 | ---"--- | Slope | 327 | Test |
MSE [\(\mathrm {m^2/s^4}\)] | Training (#1) | Test (#2) |
---|---|---|
LiDAR DNN (ours) | 0.0028 | 0.0015 |
Camera DNN | 0.0014 | 0.0033 |
Fine-tuning
MSE [\(\mathrm {m^2/s^4}\)] | Training | Test | |
---|---|---|---|
(#3) | (#4+#5) | (#6) | |
LiDAR DNN (ours) | 0.0040 | 0.0041 | 0.0337 |
Camera DNN | 0.0023 | 0.0095 | 0.0239 |
Attitude estimation
MAE [deg] {Var. [\(\mathrm {deg^2}\)]} | Dataset# | |
---|---|---|
1 | 2 | |
LiDAR DNN (ours) | 1.82 {9.87} | 1.86 {11.41} |
Camera DNN | 3.57 {24.98} | 3.18 {19.83} |
Statistics | 22.91 {71.54} | 22.82 {71.74} |
MAE [deg] {Var. [\(\mathrm {deg^2}\)]} | Dataset# | ||||
---|---|---|---|---|---|
3 | 4 | 5 | 6 | ||
Before fine-tuning | LiDAR DNN (ours) | 11.68 {108.84} | 13.71 {105.20} | 11.29 {77.18} | 24.31 {157.94} |
---"--- | Camera DNN | 6.67 {25.92} | 6.42 {22.90} | 8.72 {83.84} | 14.95 {38.24} |
After fine-tuning | LiDAR DNN (ours) | 4.08 {6.88} | 5.27 {8.65} | 5.72 {11.80} | 16.60 {61.97} |
---"--- | Camera DNN | 4.82 {18.13} | 4.83 {16.03} | 5.83 {55.13} | 14.28 {33.51} |
Statistics | 23.61 {96.13} | 22.28 {98.97} | 27.33 {86.82} | 15.91 {85.83} |
Validation of real-time estimation in simulator
Method list
-
Gyro: ‘Gyro’ denotes an estimation method integrating angular velocity from a gyroscope.
-
Gyro+Acc: ‘Gyro+Acc’ denotes an EKF-based estimation method integrating angular velocity and linear acceleration from an IMU.
-
Gyro+NDT: ‘Gyro+NDT’ denotes NDT SLAM [4] using 32 layers of LiDAR. Angular velocity from a gyroscope, linear velocity of ground truth, and the NDT output are integrated in an EKF. Note that linear velocity of the ground truth is available because the environment is a simulator.
-
DNN: ‘DNN’ denotes a method using the proposed DNN directly without EKF.
-
Gyro+DNN (ours): ‘Gyro+DNN (ours)’ denotes the proposed method described in the section above.
Experimental conditions
Experimental results
Roll [deg] | Pitch [deg] | |
---|---|---|
Gyro | 36.786 | 28.473 |
Gyro+Acc | 6.451 | 5.387 |
Gyro+NDT | 32.514 | 23.995 |
Gyro+DGSphere [11] | 9.272 | 7.534 |
DNN | 3.148 | 1.748 |
Gyro+DNN (ours) | 2.865 | 1.973 |
Validation of real-time estimation in real world
Indoor experiment with motion capture
Roll [deg] | Pitch [deg] | |
---|---|---|
Gyro | 6.012 | 5.100 |
Gyro+Acc | 2.509 | 1.648 |
Gyro+DGSphere | 4.272 | 3.147 |
DNN | 6.153 | 3.494 |
Gyro+DNN (ours) | 2.506 | 1.854 |
Outdoor experiment
Roll [deg] | Pitch [deg] | |
---|---|---|
Gyro | +5.268 | −5.047 |
Gyro+Acc | −0.269 | +0.303 |
Gyro+DNN (ours) | −1.347 | −0.654 |