1 Introduction
-
We have demonstrated a complete urban map data processing pipeline, which annotate all 3D LiDAR points with semantic labels. Our method is made efficient by combining fast rule-based processing for building and street surface segmentation and super-voxel-based feature extraction and classification for remaining map elements (cars, pedestrians, trees and traffic signs).
-
We propose two back ends for semantically labelled urban 3D map data that exemplify two important applications: (i) 3D urban map visualization and (ii) semantic segmentation of 2D street view images by backprojection of the 3D labels.
-
Parameters of the different processing stages have clear physical and intuitive meaning, and therefore they are easy to set for novel data or optimize by cross-validation over certain ranges. We have made extensive experiments on larger datasets, and moreover, optimal parameter settings are cross-validated against labelled datasets. Experimental results verify superior accuracy and efficiency of our method as compared to the existing works on three difficult datasets.
2 Related work
3 Proposed methodology
3.1 Rule-based segmentation of simple structures
3.1.1 Road surfaces
3.1.2 Building facades
3.2 Boosted decision tree detector for super-voxel features
3.2.1 Super voxels
3.2.2 Super-voxel classification
Feature | Motivation |
---|---|
Geometric features
| |
Area | Small versus large objects |
Edge ratio | Maximum and minimum edges |
Max edge | Longest dimension |
Covariance | overall shape |
Location and orientation features
| |
Height above road | |
Distance to street | Horizontal distance to the car GPS |
Normal angle | With respect to the surface orientation |
3D and photometric texture features
| |
Mean intensity | Overall reflection property of the voxel |
Density | Density of the points |
Planarity | Average distance to the best fitted plane |
4 Applications
4.1 Visualization of 3D urban maps
4.1.1 Building facade rendering
4.1.2 Rendering object models
4.2 Semantic segmentation in 2D
5 Experiments
Method | Building | Road | Tr. sign | Car | Class AVE ACCY |
---|---|---|---|---|---|
Aijazi et al. [1] | 0.914 | 0.901 | 0.710 | 0.900 | 0.856 |
Serna et al. [33] | 0.986 | 0.940 | 0.000 | 0.950 | 0.719 |
Our |
0.991
|
0.950
|
0.841
|
0.982
|
0.941
|
Measure | Method | Tree | Car | Tr. sign | Pedestrian | Fence | Building | Class AVE ACCY |
---|---|---|---|---|---|---|---|---|
Precision | Lai and Fox [21] | 0.83 | 0.91 |
0.80
| 0.41 | 0.61 | 0.86 | 0.73 |
Our |
0.89
|
0.95
| 0.72 |
0.88
|
0.85
|
0.95
|
0.87
| |
F-score | Lai and Fox [21] | 0.76 | 0.79 |
0.69
| 0.47 | 0.42 | 0.91 | 0.67 |
Our |
0.85
|
0.93
|
0.69
|
0.88
|
0.80
|
0.95
|
0.85
|
5.1 Datasets
5.2 Urban 3D segmentation and classification
Building | Road | River | Car | Tree | Tr. sign | Pedestrian | |
---|---|---|---|---|---|---|---|
Building | 0.885 | 0.083 | 0.000 | 0.000 | 0.115 | 0.000 | 0.000 |
Road | 0.041 | 0.958 | 0.000 | 0.003 | 0.015 | 0.001 | 0.000 |
River | 0.000 | 0.249 | 0.733 | 0.000 | 0.000 | 0.000 | 0.000 |
Car | 0.000 | 0.018 | 0.000 | 0.847 | 0.000 | 0.007 | 0.000 |
Tree | 0.004 | 0.001 | 0.000 | 0.000 | 0.897 | 0.007 | 0.000 |
Tr. sign | 0.000 | 0.002 | 0.000 | 0.000 | 0.113 | 0.735 | 0.002 |
Pedestrian | 0.000 | 0.049 | 0.000 | 0.000 | 0.008 | 0.000 | 0.782 |
Our method | # of voxels | Comp. time (mins) | Overall accuracy |
---|---|---|---|
w rule-based | 32,891 | 46 | 86% |
w/o rule-based | 246,548 | 291 | 75% |
Sky | Building | Road | Tree | Car | Tr. sign | Pedestrian | River | |
---|---|---|---|---|---|---|---|---|
Sky | 0.960 | 0.020 | 0.000 | 0.020 | 0.000 | 0.000 | 0.000 | 0.000 |
Building | 0.030 | 0.870 | 0.024 | 0.075 | 0.000 | 0.000 | 0.000 | 0.000 |
Road | 0.000 | 0.015 | 0.920 | 0.000 | 0.065 | 0.000 | 0.000 | 0.000 |
Tree | 0.000 | 0.280 | 0.080 | 0.640 | 0.000 | 0.000 | 0.000 | 0.000 |
Car | 0.050 | 0.000 | 0.250 | 0.020 | 0.680 | 0.000 | 0.000 | 0.000 |
Tr. sign | 0.010 | 0.280 | 0.090 | 0.000 | 0.000 | 0.370 | 0.250 | 0.000 |
Pedestrian | 0.010 | 0.340 | 0.020 | 0.020 | 0.000 | 0.330 | 0.280 | 0.000 |
River | 0.000 | 0.050 | 0.250 | 0.050 | 0.000 | 0.000 | 0.000 | 0.650 |