Top

Autonomous Robots

Published in:

28-12-2022

Urban localization based on aerial imagery by correcting projection distortion

Authors: Jonghwi Kim, Yonghoon Cho, Jinwhan Kim

Published in: Autonomous Robots | Issue 3/2023

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

This study proposes a vehicle localization method that fuses aerial maps and LiDAR measurements in urban canyon environments. The building outlines from an aerial image can be used as appropriate features for matching with the LiDAR data for localization. However, distortions caused by scaled orthographic projection of aerial maps are commonly observed in the images of metropolitan areas, which may significantly degrade the matching and resulting localization performance. In this study, a novel method for correcting such distortions is proposed and used for the vehicle localization by matching the corrected map and LiDAR measurements. Instance and semantic segmentation algorithms were used to distinguish individual buildings and generate corrected outlines of the buildings. A particle filter is applied to determine the pose of the vehicle based on the mutual information between the map and LiDAR measurements. The performance of the proposed algorithm was verified using a dataset obtained in urban areas.

previous article An efficient RRT-based motion planning algorithm for autonomous underwater vehicles under cylindrical sampling constraints

next article Provident vehicle detection at night for advanced driver assistance systems

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Alharake, A., Bresson, G., Merriaux, P., Vauchey, V., & Savatier, X. (2019). Urban localization inside cadastral maps using a likelihood field representation. In 2019 IEEE intelligent transportation systems conference (ITSC) (pp. 1329–1335). https://doi.org/10.1109/itsc.2019.8917303

Audebert, N., Le Saux, B., & Lefèvre, S. (2018). Beyond RGB: Very high resolution urban remote sensing with multimodal deep networks. ISPRS Journal of Photogrammetry and Remote Sensing, 140, 20–32. https://doi.org/10.1016/j.isprsjprs.2017.11.011CrossRef

Badrinarayanan, V., Kendall, A., & Cipolla, R. (2017). Segnet: A deep convolutional encoder-decoder architecture for image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 39(12), 2481–2495.CrossRef

Cai, S., Guo, Y., Khan, S., Hu, J., & Wen, G. (2019). Ground-to-aerial image geolocalization with a hard exemplar reweighting triplet loss. In Proceedings of the IEEE international conference on computer vision (pp. 8391–8400). https://doi.org/10.1109/iccv.2019.00848

Chen, H., Zhang, K., Xiao, W., Sheng, Y., Cheng, L., Zhou, W., & Zhang, S. (2021). Building change detection in very high-resolution remote sensing image based on pseudo-orthorectification. International Journal of Remote Sensing, 42(7), 2686–2705. https://doi.org/10.1080/01431161.2020.1862437CrossRef

Chu, H., Mei, H., Bansal, M., & Walter, M. R. (2015). Accurate vision-based vehicle localization using satellite imagery. arXiv preprint arXiv:1510.09171

Cover, T. M., & Thomas, J. A. (2006). Elements of information theory (2nd ed., p. 13). John Wiley & Sons. https://doi.org/10.1002/047174882XCrossRefMATH

de Paula Veronese, L., de Aguiar, E., Nascimento, R. C., Guivant, J., Cheein, F. A. A., De Souza, A. F., & Oliveira-Santos, T. (2015). Reemission and satellite aerial maps applied to vehicle localization on urban environments. In 2015 IEEE/RSJ international conference on intelligent robots and systems (IROS) (pp. 4285–4290). https://doi.org/10.1109/iros.2015.7353984

D’Errico, J. (2019). A suite of minimal bounding objects. https://kr.mathworks.com/ matlabcentral/fileexchange/34767.

de Veronese, L. P., Auat-Cheein, F., Mutz, F., Oliveira-Santos, T., Guivant, J. E., de Aguiar, E., & De Souza, A. F. (2020). Evaluating the limits of a LiDAR for an autonomous driving localization. IEEE Transactions on Intelligent Transportation Systems. https://doi.org/10.1109/tits.2020.2971054CrossRef

Fang, Y., Wang, C., Yao, W., Zhao, X., Zhao, H., & Zha, H. (2019). On-road vehicle tracking using part-based particle filter. IEEE Transactions on Intelligent Transportation Systems, 20(12), 4538–4552.CrossRef

Floros, G., Van Der Zander, B., & Leibe, B. (2013). Openstreetslam: Global vehicle localization using openstreetmaps. In 2013 IEEE International conference on robotics and automation (pp. 1054–1059). https://doi.org/10.1109/icra.2013.6630703

Früh, C., & Zakhor, A. (2004). An automated method for large-scale, ground-based city model acquisition. International Journal of Computer Vision, 60(1), 5–24. https://doi.org/10.1023/b:visi.0000027787.82851.b6CrossRef

Fu, M., Zhu, M., Yang, Y., Song, W., & Wang, M. (2020). LiDAR-based vehicle localization on the satellite image via a neural network. Robotics and Autonomous Systems. https://doi.org/10.1016/j.robot.2020.103519CrossRef

Guiasu, S. (1977). Information theory with applications (Vol. 202). McGraw-Hill.MATH

Han, J., Cho, Y., Kim, J., Kim, J., Son, N.-S., & Kim, S. Y. (2020). Autonomous collision detection and avoidance for ARAGON USV: Development and field tests. Journal of Field Robotics, 37(6), 987–1002. https://doi.org/10.1002/rob.21935CrossRef

Han, J., Kim, J., & Shim, D. H. (2018). Precise localization and mapping in indoor parking structures via parameterized SLAM. IEEE Transactions on Intelligent Transportation Systems, 20(12), 4415–4426. https://doi.org/10.1109/tits.2018.2885341CrossRef

Han, J., Park, J., Kim, T., & Kim, J. (2015). Precision navigation and mapping under bridges with an unmanned surface vehicle. Autonomous Robots, 38(4), 349–362. https://doi.org/10.1007/s10514-015-9419-2CrossRef

He, K., Gkioxari, G., Dollár, P., & Girshick, R. (2017). Mask R-CNN. In Proceedings of the IEEE international conference on computer vision (pp. 2961–2969). https://doi.org/10.1109/iccv.2017.322

He, K., Zhang, X., Ren, S., & Sun, J. (2016). Deep residual learning for image recognition. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 770–778). https://doi.org/10.1109/cvpr.2016.90

Hu, H., Zhu, J., Wirges, S., & Lauer, M. (2019). Localization in aerial imagery with grid maps using LocGAN. In 2019 IEEE intelligent transportation systems conference (ITSC) (pp. 2860–2865). https://doi.org/10.1109/itsc.2019.8917236

Hu, S., & Lee, G. H. (2020). Image-based geolocalization using satellite imagery. International Journal of Computer Vision, 128(5), 1205–1219. https://doi.org/10.1007/s11263-019-01186-0CrossRef

Javanmardi, M., Javanmardi, E., Gu, Y., & Kamijo, S. (2017). Towards high-definition 3D urban mapping: Road feature-based registration of mobile mapping systems and aerial imagery. Remote Sensing, 9(10), 975. https://doi.org/10.3390/rs9100975CrossRef

Jende, P., Nex, F., Gerke, M., & Vosselman, G. (2018). A fully automatic approach to register mobile mapping and airborne imagery to support the correction of platform trajectories in GNSS-denied urban areas. ISPRS Journal of Photogrammetry and Remote Sensing, 141, 86–99. https://doi.org/10.1016/j.isprsjprs.2018.04.017CrossRef

Jeong, J., Cho, Y., Shin, Y.-S., Roh, H., & Kim, A. (2019). Complex urban dataset with multi-level sensors from highly diverse urban environments. The International Journal of Robotics Research. https://doi.org/10.1177/0278364919843996CrossRef

Kampffmeyer, M., Salberg, A.-B., & Jenssen, R. (2016). Semantic segmentation of small objects and modeling of uncertainty in urban remote sensing images using deep convolutional neural networks. In IEEE conference on computer vision and pattern recognition workshops (pp. 1–9). https://doi.org/10.1109/cvprw.2016.90

Käslin, R., Fankhauser, P., Stumm, E., Taylor, Z., Mueggler, E., Delmerico, J., & Hutter, M. (2016). Collaborative localization of aerial and ground robots through elevation maps. In IEEE international symposium on safety, security, and rescue robotics (pp. 284–290). https://doi.org/10.1109/ssrr.2016.7784317

Kim, D.-K., & Walter, M. R. (2017). Satellite image-based localization via learned embeddings. In IEEE international conference on robotics and automation (pp. 2073–2080). https://doi.org/10.1109/icra.2017.7989239

Kim, J., Cho, Y., & Kim, J. (2018). Vehicle localization in urban environment using a 2d online map with building outlines. In 2018 15th International conference on ubiquitous robots (UR) (pp. 586–590). https://doi.org/10.1109/urai.2018.8441821

Kim, J., & Kim, J. (2019). Fusing lidar data and aerial imagery with perspective correction for precise localization in urban canyons. In 2019 IEEE/RSJ international conference on intelligent robots and systems (IROS) (pp. 5298–5303). https://doi.org/10.1109/iros40897.2019.8967711

Kümmerle, R., Steder, B., Dornhege, C., Kleiner, A., Grisetti, G., & Burgard, W. (2011). Large scale graph-based SLAM using aerial images as prior information. Autonomous Robots, 30(1), 25–39. https://doi.org/10.1007/s10514-010-9204-1CrossRef

Landsiedel, C., & Wollherr, D. (2017). Global localization of 3d point clouds in building outline maps of urban outdoor environments. International Journal of Intelligent Robotics and Applications, 1(4), 429–441. https://doi.org/10.1007/s41315-017-0038-2CrossRef

Lin, T.-Y., Dollár, P., Girshick, R., He, K., Hariharan, B., & Belongie, S. (2017). Feature pyramid networks for object detection. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 2117–2125). https://doi.org/10.1109/cvpr.2017.106

Liu, L., & Li, H. (2019). Lending orientation to neural networks for cross-view geolocalization. In Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (pp. 5624–5633). https://doi.org/10.1109/cvpr.2019.00577

NGII (2019). National geographic information institute. www.ngii.go.kr

Roh, H., Jeong, J., Cho, Y., & Kim, A. (2016). Accurate mobile urban mapping via digitalmap-based SLAM. Sensors, 16(8), 1315. https://doi.org/10.3390/s16081315CrossRef

Roh, H., Jeong, J., & Kim, A. (2017). Aerial image based heading correction for large scale SLAM in an urban canyon. IEEE Robotics and Automation Letters, 2(4), 2232–2239. https://doi.org/10.1109/lra.2017.2725439CrossRef

Ruchti, P., Steder, B., Ruhnke, M., & Burgard, W. (2015). Localization on openstreetmap data using a 3d laser scanner. In 2015 IEEE international conference on robotics and automation (ICRA) (pp. 5260–5265). https://doi.org/10.1109/icra.2015.7139932

Senlet, T., & Elgammal, A. (2011). A framework for global vehicle localization using stereo images and satellite and road maps. In IEEE international conference on computer vision workshops (pp. 2034–2041). https://doi.org/10.1109/iccvw.2011.6130498

Simonyan, K., & Zisserman, A. (2014). Very deep convolutional networks for largescale image recognition. arXiv preprint arXiv:1409.1556

Tang, T. Y., De Martini, D., Wu, S., & Newman, P. (2020). Self-supervised localisation between range sensors and overhead imagery. arXiv preprint arXiv:2006.02108

Tian, Y., Chen, C., & Shah, M. (2017). Cross-view image matching for geo-localization in urban environments. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 3608–3616). https://doi.org/10.1109/cvpr.2017.216

Viswanathan, A., Pires, B. R., & Huber, D. (2016). Vision-based robot localization across seasons and in remote locations. In 2016 IEEE international conference on robotics and automation (ICRA) (pp. 4815–4821). https://doi.org/10.1109/icra.2016.7487685

Vora, A., Agarwal, S., Pandey, G., & McBride, J. (2020). Aerial imagery based LIDAR localization for autonomous vehicles. arXiv preprint arXiv:2003.11192

Vysotska, O., & Stachniss, C. (2016). Exploiting building information from publicly available maps in graph-based slam. In 2016 IEEE/RSJ international conference on intelligent robots and systems (IROS) (pp. 4511–4516). https://doi.org/10.1109/iros.2016.7759664

Wang, X., Vozar, S., & Olson, E. (2017). Flag: Feature-based localization between air and ground. In 2017 IEEE international conference on robotics and automation (ICRA) (pp. 3178–3184). https://doi.org/10.1109/icra.2017.7989360

Yoo, T.-S., Kim, M.-H., Yoon, S.-I., & Kim, D.-J. (2019). Design of tightly coupled INS/DVL/RPM integrated navigation system. Journal of Ocean Engineering and Technology, 33(5), 470–478. https://doi.org/10.26748/ksoe.2019.085CrossRef

Zhang, K., Sheng, Y., Wang, M., & Fu, S. (2018). An enhanced multi-view vertical line locus matching algorithm of object space ground primitives based on positioning consistency for aerial and space images. ISPRS Journal of Photogrammetry and Remote Sensing, 139, 241–254. https://doi.org/10.1016/j.isprsjprs.2018.03.017CrossRef

Title: Urban localization based on aerial imagery by correcting projection distortion
Authors: Jonghwi Kim
Yonghoon Cho
Jinwhan Kim
Publication date: 28-12-2022
Publisher: Springer US
Published in: Autonomous Robots / Issue 3/2023
Print ISSN: 0929-5593
Electronic ISSN: 1573-7527
DOI: https://doi.org/10.1007/s10514-022-10082-5

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Other articles of this Issue 3/2023

Provident vehicle detection at night for advanced driver assistance systems

Correction to: Adaptive submodular inverse reinforcement learning for spatial search and map exploration

Adaptive optimal controller design for an unbalanced UAV with slung load

An efficient RRT-based motion planning algorithm for autonomous underwater vehicles under cylindrical sampling constraints