Skip to main content
Erschienen in: Neural Computing and Applications 21/2021

17.05.2021 | Original Article

Deep learning-based visual crack detection using Google Street View images

verfasst von: Mohsen Maniat, Charles V. Camp, Ali R. Kashani

Erschienen in: Neural Computing and Applications | Ausgabe 21/2021

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

In this study, the utility of using Google Street View (GSV) for evaluating the quality of pavement is investigated. A convolutional neural network (CNN) is developed to perform image classification on GSV pavement images. Pavement images are extracted from GSV and then divided into smaller image patches to form data sets. Each image patch is visually classified into different categories of pavement cracks based on the standard practice. A comparative study of pavement quality assessment is conducted between the results of the CNN classified image patches obtained from GSV and those from a sophisticated commercial visual inspection company. The result of the comparison indicates the feasibility and effectiveness of using GSV images for pavement evaluation. The trained network is then tested on a new data set. This study shows that the designed CNN helps classify the pavement images into different defined crack categories.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Literatur
2.
Zurück zum Zitat Phares BM, Rolander DD, Graybeal BA, Washer GA (2001) Reliability of visual bridge inspection turner-fairbank highway research center. Federal Highway Administration 64(5):22–29 Phares BM, Rolander DD, Graybeal BA, Washer GA (2001) Reliability of visual bridge inspection turner-fairbank highway research center. Federal Highway Administration 64(5):22–29
3.
Zurück zum Zitat Lenz H, Weichers B (2008) Applications of specialized visual inspection techniques on nuclear components Lenz H, Weichers B (2008) Applications of specialized visual inspection techniques on nuclear components
5.
Zurück zum Zitat Xu Y, Brownjohn JM (2018) Review of machine-vision based methodologies for displacement measurement in civil structures. J Civ Struct Heal Monit 8(1):91–110CrossRef Xu Y, Brownjohn JM (2018) Review of machine-vision based methodologies for displacement measurement in civil structures. J Civ Struct Heal Monit 8(1):91–110CrossRef
6.
Zurück zum Zitat Jahanshahi MR, Masri SF, Sukhatme GS (2011) Multi-image stitching and scene reconstruction for evaluating defect evolution in structures. Struct Health Monit 10(6):643–657CrossRef Jahanshahi MR, Masri SF, Sukhatme GS (2011) Multi-image stitching and scene reconstruction for evaluating defect evolution in structures. Struct Health Monit 10(6):643–657CrossRef
7.
Zurück zum Zitat Lee BJ, Shin DH, Seo JW, Jung JD, Lee JY (2011, June) Intelligent bridge inspection using remote controlled robot and image processing technique. In: International Symposium on Automation and Robotics in Construction (ISARC), Seoul, Korea, pp 1426–1431 Lee BJ, Shin DH, Seo JW, Jung JD, Lee JY (2011, June) Intelligent bridge inspection using remote controlled robot and image processing technique. In: International Symposium on Automation and Robotics in Construction (ISARC), Seoul, Korea, pp 1426–1431
8.
Zurück zum Zitat Lattanzi D, Miller G (2017) Review of robotic infrastructure inspection systems. J Infrastruct Syst 23(3):04017004CrossRef Lattanzi D, Miller G (2017) Review of robotic infrastructure inspection systems. J Infrastruct Syst 23(3):04017004CrossRef
9.
Zurück zum Zitat Dorafshan S, Maguire M (2018) Bridge inspection: Human performance, unmanned aerial systems and automation. J Civ Struct Heal Monit 8(3):443–476CrossRef Dorafshan S, Maguire M (2018) Bridge inspection: Human performance, unmanned aerial systems and automation. J Civ Struct Heal Monit 8(3):443–476CrossRef
10.
Zurück zum Zitat Cafiso S, Graziano AD, Battiato S (2006, October) Evaluation of pavement surface distress using digital image collection and analysis. In: Seventh international congress on advances in civil engineering, pp 1–10 Cafiso S, Graziano AD, Battiato S (2006, October) Evaluation of pavement surface distress using digital image collection and analysis. In: Seventh international congress on advances in civil engineering, pp 1–10
11.
Zurück zum Zitat Gavilán M, Balcones D, Marcos O, Llorca DF, Sotelo MA, Parra I, Ocaña M, Aliseda YP, Amírola A (2011) Adaptive road crack detection system by pavement classification. Sensors 11(10):9628–9657CrossRef Gavilán M, Balcones D, Marcos O, Llorca DF, Sotelo MA, Parra I, Ocaña M, Aliseda YP, Amírola A (2011) Adaptive road crack detection system by pavement classification. Sensors 11(10):9628–9657CrossRef
12.
Zurück zum Zitat Lopes G, Ribeiro AF, Sillero N, Gonçalves-Seco L, Silva C, Franch M, Trigueiros P (2016) High resolution trichromatic road surface scanning with a line scan camera and light emitting diode lighting for road-kill detection. Sensors 16(4):558CrossRef Lopes G, Ribeiro AF, Sillero N, Gonçalves-Seco L, Silva C, Franch M, Trigueiros P (2016) High resolution trichromatic road surface scanning with a line scan camera and light emitting diode lighting for road-kill detection. Sensors 16(4):558CrossRef
13.
Zurück zum Zitat Zhang L, Yang F, Zhang YD, Zhu YJ (2016, September) Road crack detection using deep convolutional neural network. In: 2016 IEEE international conference on image processing (ICIP), pp 3708–3712 Zhang L, Yang F, Zhang YD, Zhu YJ (2016, September) Road crack detection using deep convolutional neural network. In: 2016 IEEE international conference on image processing (ICIP), pp 3708–3712
14.
Zurück zum Zitat Maeda H, Sekimoto Y, Seto T, Kashiyama T, Omata H (2018) Road damage detection using deep neural networks with images captured through a smartphone. arXiv preprint arXiv:1801.09454 Maeda H, Sekimoto Y, Seto T, Kashiyama T, Omata H (2018) Road damage detection using deep neural networks with images captured through a smartphone. arXiv preprint arXiv:​1801.​09454
15.
Zurück zum Zitat Varadharajan S, Jose S, Sharma K, Wander L, Mertz C (2014, March) Vision for road inspection. In: IEEE winter conference on applications of computer vision, pp 115–122 Varadharajan S, Jose S, Sharma K, Wander L, Mertz C (2014, March) Vision for road inspection. In: IEEE winter conference on applications of computer vision, pp 115–122
16.
Zurück zum Zitat Lee S, Ha J, Zokhirova M, Moon H, Lee J (2018) Background information of deep learning for structural engineering. Archives of Computational Methods in Engineering 25(1):121–129MathSciNetCrossRef Lee S, Ha J, Zokhirova M, Moon H, Lee J (2018) Background information of deep learning for structural engineering. Archives of Computational Methods in Engineering 25(1):121–129MathSciNetCrossRef
17.
Zurück zum Zitat Liu R, Yang B, Zio E, Chen X (2018) Artificial intelligence for fault diagnosis of rotating machinery: a review. Mech Syst Signal Process 108:33–47CrossRef Liu R, Yang B, Zio E, Chen X (2018) Artificial intelligence for fault diagnosis of rotating machinery: a review. Mech Syst Signal Process 108:33–47CrossRef
18.
Zurück zum Zitat LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444CrossRef LeCun Y, Bengio Y, Hinton G (2015) Deep learning. Nature 521(7553):436–444CrossRef
19.
Zurück zum Zitat Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105 Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Advances in neural information processing systems, pp 1097–1105
20.
Zurück zum Zitat Deng J, Dong W, Socher R, Li L, Li K, Fei-Fei L (2009, June) Imagenet: a large-scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition, pp 248–255 Deng J, Dong W, Socher R, Li L, Li K, Fei-Fei L (2009, June) Imagenet: a large-scale hierarchical image database. In: 2009 IEEE conference on computer vision and pattern recognition, pp 248–255
21.
Zurück zum Zitat Cha YJ, Choi W, Büyüköztürk O (2017) Deep learning-based crack damage detection using convolutional neural networks. Computer-Aided Civil and Infrastructure Engineering 32(5):361–378CrossRef Cha YJ, Choi W, Büyüköztürk O (2017) Deep learning-based crack damage detection using convolutional neural networks. Computer-Aided Civil and Infrastructure Engineering 32(5):361–378CrossRef
22.
Zurück zum Zitat Cha YJ, Choi W, Suh G, Mahmoudkhani S, Büyüköztürk O (2018) Autonomous structural visual inspection using region-based deep learning for detecting multiple damage types. Computer-Aided Civil and Infrastructure Engineering 33(9):731–747CrossRef Cha YJ, Choi W, Suh G, Mahmoudkhani S, Büyüköztürk O (2018) Autonomous structural visual inspection using region-based deep learning for detecting multiple damage types. Computer-Aided Civil and Infrastructure Engineering 33(9):731–747CrossRef
23.
Zurück zum Zitat Huang HW, Li QT, Zhang DM (2018) Deep learning based image recognition for crack and leakage defects of metro shield tunnel. Tunneling and Underground Space Technology 77:166–176CrossRef Huang HW, Li QT, Zhang DM (2018) Deep learning based image recognition for crack and leakage defects of metro shield tunnel. Tunneling and Underground Space Technology 77:166–176CrossRef
24.
Zurück zum Zitat Chen FC, Jahanshahi MR (2017) NB-CNN: Deep learning-based crack detection using convolutional neural network and Naïve Bayes data fusion. IEEE Trans Industr Electron 65(5):4392–4400CrossRef Chen FC, Jahanshahi MR (2017) NB-CNN: Deep learning-based crack detection using convolutional neural network and Naïve Bayes data fusion. IEEE Trans Industr Electron 65(5):4392–4400CrossRef
25.
Zurück zum Zitat Zhang A, Wang KC, Li B, Yang E, Dai X, Peng Y, Fei Y, Liu Y, Li JQ, Chen C (2017) Automated pixel-level pavement crack detection on 3D asphalt surfaces using a deep-learning network. Computer-Aided Civil and Infrastructure Engineering 32(10):805–819CrossRef Zhang A, Wang KC, Li B, Yang E, Dai X, Peng Y, Fei Y, Liu Y, Li JQ, Chen C (2017) Automated pixel-level pavement crack detection on 3D asphalt surfaces using a deep-learning network. Computer-Aided Civil and Infrastructure Engineering 32(10):805–819CrossRef
26.
Zurück zum Zitat Zhang L, Yang F, Zhang YD, Zhu YJ (2016) Road crack detection using deep convolutional neural network. In: 2016 IEEE international conference on image processing (ICIP), pp 3708–3712 Zhang L, Yang F, Zhang YD, Zhu YJ (2016) Road crack detection using deep convolutional neural network. In: 2016 IEEE international conference on image processing (ICIP), pp 3708–3712
27.
Zurück zum Zitat Eisenbach M, Stricker R, Debes K, Gross HM (2017) Crack detection with an interactive and adaptive video inspection system. Arbeitsgruppentagung Infrastrukturmanagement, pp 94–103 Eisenbach M, Stricker R, Debes K, Gross HM (2017) Crack detection with an interactive and adaptive video inspection system. Arbeitsgruppentagung Infrastrukturmanagement, pp 94–103
28.
Zurück zum Zitat Pauly L, Hogg D, Fuentes R, Peel H (2017, July) Deeper networks for pavement crack detection. In: Proceedings of the 34th ISARC, pp 479–485 Pauly L, Hogg D, Fuentes R, Peel H (2017, July) Deeper networks for pavement crack detection. In: Proceedings of the 34th ISARC, pp 479–485
33.
Zurück zum Zitat Shapiro A (2018) Street-level: google street view’s abstraction by datafication. New Media Soc 20(3):1201–1219CrossRef Shapiro A (2018) Street-level: google street view’s abstraction by datafication. New Media Soc 20(3):1201–1219CrossRef
34.
Zurück zum Zitat Rundle AG, Bader MD, Richards CA, Neckerman KM, Teitler JO (2011) Using google street view to audit neighborhood environments. Am J Prev Med 40(1):94–100CrossRef Rundle AG, Bader MD, Richards CA, Neckerman KM, Teitler JO (2011) Using google street view to audit neighborhood environments. Am J Prev Med 40(1):94–100CrossRef
35.
Zurück zum Zitat Torii A, Havlena M, Pajdla T (2009, September) From google street view to 3d city models. In: 2009 IEEE 12th international conference on computer vision workshops, ICCV Workshops, pp 2188–2195 Torii A, Havlena M, Pajdla T (2009, September) From google street view to 3d city models. In: 2009 IEEE 12th international conference on computer vision workshops, ICCV Workshops, pp 2188–2195
36.
Zurück zum Zitat Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:1409.1556 Simonyan K, Zisserman A (2014) Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv:​1409.​1556
37.
Zurück zum Zitat LeCun Y, Boser BE, Denker JS, Henderson D, Howard RE, Hubbard WE, Jackel LD (1990) Handwritten digit recognition with a back-propagation network. In: Advances in neural information processing systems, pp 396–404 LeCun Y, Boser BE, Denker JS, Henderson D, Howard RE, Hubbard WE, Jackel LD (1990) Handwritten digit recognition with a back-propagation network. In: Advances in neural information processing systems, pp 396–404
39.
Zurück zum Zitat Reddi SJ, Kale S, Kumar S (2019) On the convergence of adam and beyond. arXiv preprint arXiv:1904.09237. Bengio Y. practical recommendations for gradient-based training of deep architectures. Neural Netw. Tricks Trade. Springer 2012:437–478 Reddi SJ, Kale S, Kumar S (2019) On the convergence of adam and beyond. arXiv preprint arXiv:1904.09237. Bengio Y. practical recommendations for gradient-based training of deep architectures. Neural Netw. Tricks Trade. Springer 2012:437–478
42.
Zurück zum Zitat Canziani A, Paszke A, Culurciello E (2016) An analysis of deep neural network models for practical applications. arXiv:1605.07678 Canziani A, Paszke A, Culurciello E (2016) An analysis of deep neural network models for practical applications. arXiv:​1605.​07678
43.
Zurück zum Zitat Xiao J, Quan L (2009, October) Multiple view semantic segmentation for street view images. In: 2009 IEEE 12th international conference on computer vision, pp 686–693 Xiao J, Quan L (2009, October) Multiple view semantic segmentation for street view images. In: 2009 IEEE 12th international conference on computer vision, pp 686–693
44.
Zurück zum Zitat Jae Lee Y, Efros AA, Hebert M (2013) Style-aware mid-level representation for discovering visual connections in space and time. In: Proceedings of the IEEE international conference on computer vision, pp 1857–1864 Jae Lee Y, Efros AA, Hebert M (2013) Style-aware mid-level representation for discovering visual connections in space and time. In: Proceedings of the IEEE international conference on computer vision, pp 1857–1864
45.
Zurück zum Zitat Goodfellow IJ, Bulatov Y, Ibarz J, Arnoud S, Shet V (2013) Multi-digit number recognition from street view imagery using deep convolutional neural networks. arXiv:1312.6082 Goodfellow IJ, Bulatov Y, Ibarz J, Arnoud S, Shet V (2013) Multi-digit number recognition from street view imagery using deep convolutional neural networks. arXiv:​1312.​6082
46.
Zurück zum Zitat Zamir AR, Shah M (2010) Accurate image localization based on google maps street view. In: European conference on computer vision. Springer, Berlin, Heidelberg Zamir AR, Shah M (2010) Accurate image localization based on google maps street view. In: European conference on computer vision. Springer, Berlin, Heidelberg
48.
Zurück zum Zitat He H, Garcia EA (2009) Learning from imbalanced data. IEEE Trans Knowl Data Eng 21(9):1263–1284CrossRef He H, Garcia EA (2009) Learning from imbalanced data. IEEE Trans Knowl Data Eng 21(9):1263–1284CrossRef
51.
Zurück zum Zitat Mikołajczyk A, Grochowski M (2018, May) Data augmentation for improving deep learning in image classification problem. In: 2018 international interdisciplinary Ph.D. workshop (IIPhDW), pp 117–122 Mikołajczyk A, Grochowski M (2018, May) Data augmentation for improving deep learning in image classification problem. In: 2018 international interdisciplinary Ph.D. workshop (IIPhDW), pp 117–122
52.
Zurück zum Zitat Abdi H, Williams LJ (2010) Principal component analysis. Wiley interdisciplinary reviews: computational statistics 2(4):433–459CrossRef Abdi H, Williams LJ (2010) Principal component analysis. Wiley interdisciplinary reviews: computational statistics 2(4):433–459CrossRef
Metadaten
Titel
Deep learning-based visual crack detection using Google Street View images
verfasst von
Mohsen Maniat
Charles V. Camp
Ali R. Kashani
Publikationsdatum
17.05.2021
Verlag
Springer London
Erschienen in
Neural Computing and Applications / Ausgabe 21/2021
Print ISSN: 0941-0643
Elektronische ISSN: 1433-3058
DOI
https://doi.org/10.1007/s00521-021-06098-0

Weitere Artikel der Ausgabe 21/2021

Neural Computing and Applications 21/2021 Zur Ausgabe

Premium Partner