Top

Published in:

2022 | OriginalPaper | Chapter

6. Object Recognition Methods in a Built Environment

Authors : Josip Stjepandić, Markus Sommer

Published in: DigiTwin: An Approach for Production Process Optimization in a Built Environment

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Recognition of an object from a point cloud, image or video is an important task in computer vision which plays a crucial role in many real-world applications. The challenges involved in object recognition, aiming at locating object instances from a large number of predefined categories in collections (images, video or, model library), are multi-model, multi-pose, complicated background, occlusion, and depth variations. In the past few years numerous methods were developed to tackle these challenges and have reported remarkable progress for 3D objects. However, suitable methods of object recognition are needed to achieve added value in built environment. Suitable acquisition methods are also necessary to compensate the impact of darkness, dirt, and occlusion. This chapter provides a comprehensive overview of the recent advances in 3D object recognition of indoor objects using Convolutional Neural Networks (CNN). Methodology for object recognition, approaches for point cloud generation, and test bases are presented. The comparison of main recognition methods based on methods of geometric shape descriptor and supervised learning and their strengths and weakness are also included. The focus lies on the specific requirements and constrains in an industrial environment like tight assembly, light, dirt, occlusion, or incomplete data sets. Finally, a recommendation for use of existing CNN framework for implementation of an automatic object recognition procedure is given.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

previous chapter Machine Learning in Manufacturing in the Era of Industry 4.0

next chapter Data Quality Management for Interoperability

Liu L, Ouyang W, Wang X, Fieguth P, Chen J, Liu X, Pietikäinen M (2020) Deep learning for generic object detection: a survey. Int J Comput Vision 128(2):261–318. https://doi.org/10.1007/s11263-019-01247-4CrossRef

Wognum N, Bil C, Elgh F, Peruzzini M, Stjepandić J, Verhagen WJC (2019) Transdisciplinary systems engineering: implications, challenges and research agenda. Int J Agile Syst Manage 12(1):58–89. https://doi.org/10.1504/IJASM.2019.098728

Ríos J, Mas Morate F, Oliva M, Hernández JC (2016) Framework to support the aircraft digital counterpart concept with an industrial design view. Int J Agile Syst Manage 9(3):212–231. https://doi.org/10.1504/IJASM.2016.079934

Quintana B, Prieto SA, Adán A, Vázquez AS (2016) Semantic scan planning for indoor structural elements of buildings. Adv Eng Inform 30:643–659. https://doi.org/10.1016/j.aei.2016.08.003CrossRef

Zutshi A, Grilo A (2019) The emergence of digital platforms a conceptual platform architecture and impact on industrial engineering. Comput Ind Eng 136:546–555. https://doi.org/10.1016/j.cie.2019.07.027CrossRef

Bondar S, Salem B, Stjepandić J (2018) Indoor object reconstruction based on acquisition by low-cost devices. Adv Transdisciplinary Eng 7(2018):113–122. https://doi.org/10.3233/978-1-61499-898-3-113CrossRef

Geng Z, Bidanda B (2017) Review of reverse engineering systems—current state of the art. Virtual Phys Prototyping 12(2):161–172. https://doi.org/10.1080/17452759.2017.1302787CrossRef

Poza-Lujan J-L, Posadas-Yagüe J-L, Simó-Ten J-E, Blanes F (2020) Distributed architecture to integrate sensor information: object recognition for smart cities. Sensors 20:112. https://doi.org/10.3390/s20010112

Carvalho LE, von Wangenheim A (2019) 3D object recognition and classification: a systematic literature review. Pattern Anal Appl 22:1243–1292. https://doi.org/10.1007/s10044-019-00804-4MathSciNetCrossRef

10.

Ma Z, Liu S (2018) A review of 3D reconstruction techniques in civil engineering and their applications. Adv Eng Inform 37(2018):163–174. https://doi.org/10.1016/j.aei.2018.05.005CrossRef

11.

Sommer M, Stjepandić J, Stobrawa S, von Soden M (2019) Automatic generation of digital twin based on scanning and object recognition. Adv Transdisciplinary Eng 10:645–654. https://doi.org/10.3233/ATDE190174CrossRef

12.

Sommer M, Stjepandić J, Stobrawa S, von Soden M (2020) Automated generation of a digital twin of a manufacturing system by using scan and convolutional neural networks. Adv Transdisciplinary Eng 12:363–372. https://doi.org/10.3233/ATDE200095CrossRef

13.

Bondar S, Ruppert C, Stjepandić J (2014) Ensuring data quality beyond change management in virtual enterprise. Int J Agile Syst Manage 7(3/4):304–323. https://doi.org/10.1504/IJASM.2014.065346

14.

Ostrosi E, Stjepandić J, Fukuda S, Kurth M (2014) Modularity: new trends for product platform strategy support in concurrent engineering. Adv Transdisciplinary Eng 1:414–423. https://doi.org/10.3233/978-1-61499-440-4-414CrossRef

15.

Sommer M, Stjepandić J, Stobrawa S, von Soden M (2020) Improvement of factory planning by automated generation of a Digital Twin. Adv Transdisciplinary Eng 12:645–654. https://doi.org/10.3233/ATDE190174CrossRef

16.

Singh RD, Mittal A, Bhatia RK (2019) 3D convolutional neural network for object recognition: a review. Multimedia Tools Appl 78:15951–15995. https://doi.org/10.1007/s11042-018-6912-6CrossRef

17.

Rangel JC, Martínez-Gómez J, Romero-González C, García-Varea I, Cazorla M (2018) Semi-supervised 3D object recognition through CNN labelling. Appl Soft Comput J 65:603–613. https://doi.org/10.1016/j.asoc.2018.02.005

18.

Xiao Y, Ma Y, Zhou M, Zhang J (2018) Deep multi-scale learning on point sets for 3D object recognition. Commun Comput Inform Sci 875:341–348. https://doi.org/10.1007/978-981-13-1702-6_34CrossRef

19.

Fathi H, Dai F, Lourakis M (2015) Automated as-built 3D reconstruction of civil infrastructure using computer vision: achievements, opportunities, and challenges. Adv Eng Inform 29(2):149–161. https://doi.org/10.1016/j.aei.2015.01.012CrossRef

20.

Liu Li, Ouyang W, Wang X, Fieguth P, Chen J, Liu X, Pietikäinen M (2020) Deep learning for generic object detection: a survey. Int J Comput Vision 128:261–318. https://doi.org/10.1007/s11263-019-01247-4CrossRef

21.

Zhao X, Ilieş HT (2017) Learned 3D shape descriptors for classifying 3D point cloud models. Comput Aided Des Appl 14(4):507–515. https://doi.org/10.1080/16864360.2016.1257192CrossRef

22.

Ghodrati H, Hamza B (2017) Nonrigid 3D shape retrieval using deep auto-encoders. Appl Intell 47:44–61. https://doi.org/10.1007/s10489-016-0880-1

23.

Kamble K, Kulkarni H, Patil J, Sukhatankar S (2018) Object recognition through smartphone using deep learning techniques. In: 2nd international conference on soft computing systems, ICSCS 2018, vol 837. Communications in Computer and Information Science, pp 242–249. https://doi.org/10.1007/978-981-13-1936-5_27

24.

Liu D, Cui Y, Chen Y, Zhang J, Fan B (2020) Video object detection for autonomous driving: Motion-aid feature calibration. Neurocomputing 409:1–11. https://doi.org/10.1016/j.neucom.2020.05.027CrossRef

25.

Ding X, Luo Y, Li Q, Cheng Y, Cai G, Munnoch R, Xue D, Qingying Yu, Zheng X, Wang B (2018) Prior knowledge-based deep learning method for indoor object recognition. Syst Sci Control Eng 6(1):249–257. https://doi.org/10.1080/21642583.2018.1482477CrossRef

26.

Wu Z, Song S, Khosla A, Yu F (2015) 3D ShapeNets: a deep representation for volumetric shapes. In: IEEE conference on computer vision and pattern recognition, pp 1912–1920. https://doi.org/10.1109/CVPR.2015.7298801

27.

Salem B, Stjepandić J, Stobrawa S (2019) Assessment of methods for industrial indoor object recognition. Adv Transdisciplinary Eng 10:390–399. https://doi.org/10.3233/ATDE190145CrossRef

28.

Princeton Modelnet Repository. https://modelnet.cs.princeton.edu/#. Accessed 30 Sep 2020

29.

Chang AX, Funkhouser T, Guibas L, Hanrahan P, Huang Q, Li Z, Savarese S, Savva M, Song S, Su H, Xiao J, Yi L, Yu F (2020) ShapeNet: an information-rich 3D model repository. https://arxiv.org/abs/1512.03012. Accessed 30 Sep 2020

30.

Quadros A (2013) Representing 3D shape in sparse range images for urban object classification. Ph.D. thesis, The University of Sydney

31.

Lehtola VV, Kaartinen H, Nüchter A, Kaijaluoto R, Kukko A, Litkey P, Honkavaara E, Rosnell T, Vaaja MT, Virtanen J-P, Kurkela M, El Issaoui A, Zhu L, Jaakkola A, Hyyppä J (2017) Comparison of the selected state-of-the-art 3D indoor scanning and point cloud generation methods. Remote Sensing 9:796. https://doi.org/10.3390/rs9080796CrossRef

32.

Rebolj D, Pučko Z, Čuš Babič N, Bizjak M, Mongus D (2017) Point cloud quality requirements for Scan-vs-BIM based automated construction progress monitoring. Autom Construct 84:323–334. https://doi.org/10.1016/j.autcon.2017.09.021

33.

Dominguez M, Dhamdhere R, Petkar A, Jain S, Sah S, Ptucha R (2018) General-purpose deep point cloud feature extractor. In: 2018 IEEE winter conference on applications of computer vision, pp 1972–1981. https://doi.org/10.1109/WACV.2018.00218

34.

Nguyen A, Le B (2013) 3D point cloud segmentation: a survey. In: 2013 IEEE 6th international conference on robotics, automation and mechatronics (RAM). https://doi.org/10.1109/RAM.2013.6758588

35.

Zhang Le, Sun J, Zheng Q (2018) 3D point cloud recognition based on a multi-view convolutional neural network. Sensors 18:3681. https://doi.org/10.3390/s18113681CrossRef

36.

Liu X, Lu Y, Wu T, Yuan T (2018) An improved local descriptor based object recognition in cluttered 3D point clouds. Int J Comput Commun Control 13(2):221–234. https://doi.org/10.15837/ijccc.2018.2.3010CrossRef

37.

Zhang W, Su S, Wang B, Hong Q, Sun L (2020) Local k-NNs pattern in omni-direction graph convolution neural network for 3D point clouds. Neurocomputing 413:487–498. https://doi.org/10.1016/j.neucom.2020.06.095CrossRef

38.

Wang K, Kim M-K (2019) Applications of 3D point cloud data in the construction industry: a fifteen-year review from 2004 to 2018. Adv Eng Inform 39:306–319. https://doi.org/10.1016/j.aei.2019.02.007

39.

Sommer M, Stjepandić J, Stobrawa S, von Soden M (2021) Automated generation of a digital twin in manufacturing for a built environment using scan and object detection. J Indus Inform Integr XX (in press)

40.

Pătrăucean V, Armeni I, Nahangi M, Yeung J, Brilakis I, Haas C (2015) State of research in automatic as-built modelling. Adv Eng Inform 29(2):162–171. https://doi.org/10.1016/j.aei.2015.01.001CrossRef

41.

Chen J, Fang Y, Cho Y (2018) Performance evaluation of 3D descriptors for object recognition in construction applications. Autom Construct 86:44–52. https://doi.org/10.1016/j.autcon.2017.10.033

42.

Brock A, Lim T, Ritchie JM, Weston N (2016) Generative and discriminative voxel modeling with convolutional neural networks. 3D Deep Learning Workshop at NIPS 2016. http://3ddl.cs.princeton.edu/2016/papers/Brock_et_al.pdf. Accessed 12 June 2019

43.

Qi CR, Su H, Mo K, Guibas LJ (2017) PointNet: deep learning on point sets for 3D classification and segmentation. In: Proceedings of the IEEE conference on computer vision and pattern recognition, Honolulu, HI, USA, 21–26 July 2017

44.

Maturana D, Scherer S (2015) VoxNet: a 3D convolutional neural network for real-time object recognition. In: IEEE international conference on intelligent robots and systems, 2015-December, 7353481, pp 922–928. https://doi.org/10.1109/IROS.2015.7353481

45.

Zhou Y, Tuzel O (2018) VoxelNet: end-to-end learning for point cloud based 3D object detection. In: The IEEE conference on computer vision and pattern recognition (CVPR), pp 4490–4499. https://doi.org/10.1109/CVPR.2018.00472

46.

Zhi S, Liu Y, Li X, Guo Y (2017) LightNet: a lightweight 3D convolutional neural network for real-time 3D object recognition. In: Eurographics workshop on 3D object retrieval, pp 9–16. https://doi.org/10.2312/3dor.20171046

47.

Hegde V, Zadeh R (2019) FusionNet: 3D object classification using multiple data representations. In: 3D deep learning workshop at NIPS 2016. https://stanford.edu/~rezab/papers/fusionnet.pdf

48.

Wang C, Cheng M, Sohel F, Bennamoun M, Li J (2019) NormalNet: a voxel-based CNN for 3D object classification and retrieval. Neurocomputing 323:139–147. https://doi.org/10.1016/j.neucom.2018.09.075CrossRef

49.

Czerniawski T (2020) Fernanda Leite, (2020) Automated digital modeling of existing buildings: a review of visual object recognition methods. Autom Construct 113:103131. https://doi.org/10.1016/j.autcon.2020.103131CrossRef

50.

Garcia-Garcia A, Garcia-Rodriguez J, Orts-Escolano S, Oprea S, Gomez-Donoso F, Cazorla M (2017) A study of the effect of noise and occlusion on the accuracy of convolutional neural networks applied to 3D object recognition. Comput Vision Image Understand 164:124–134. https://doi.org/10.1016/j.cviu.2017.06.006

51.

Ye Y, Chen H, Zhang C, Hao X, Zhang Z (2020) SARPNET: shape attention regional proposal network for liDAR-based 3D object detection. Neurocomputing 379:53–63. https://doi.org/10.1016/j.neucom.2019.09.086CrossRef

52.

Guo Y, Liu Y, Oerlemans A, Lao S, Wu S, Lew MS (2016) Deep learning for visual understanding: a review. Neurocomputing 187:27–48. https://doi.org/10.1016/j.neucom.2015.09.116CrossRef

53.

Kuhn O, Liese H, Stjepandić J (2011) Methodology for knowledge-based engineering template update. In: Cavallucci D, Guio R, Cascini G (eds) Building innovation pipelines through computer-aided innovation. Springer-Verlag, Berlin Heidelberg, pp 178–191. https://doi.org/10.1007/978-3-642-22182-8_14

54.

Dekhtiar J, Durupt A, Bricogne M, Eynard B, Rowson H, Kiritsis D (2018) Deep learning for big data applications in CAD and PLM—Research review, opportunities and case study. Comput Ind 100:227–243. https://doi.org/10.1016/j.compind.2018.04.005CrossRef

Title: Object Recognition Methods in a Built Environment
Authors: Josip Stjepandić
Markus Sommer
Publisher: Springer International Publishing
Book: DigiTwin: An Approach for Production Process Optimization in a Built Environment
Print ISBN: 978-3-030-77538-4

Electronic ISBN: 978-3-030-77539-1

Copyright Year: 2022
DOI: https://doi.org/10.1007/978-3-030-77539-1_6

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Premium Partners