Top

Published in:

2022 | OriginalPaper | Chapter

8. Object Recognition Findings in a Built Environment

Authors : Josip Stjepandić, Sergej Bondar, Wjatscheslaw Korol

Published in: DigiTwin: An Approach for Production Process Optimization in a Built Environment

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

In order to prepare the point cloud for a feature-based process of planning and simulation, the objects it contains must first be recognized and a CAD-based structure must be created. The great challenge in every process chain is to implement the desired function with minimal effort and losses. As one of the alternatives, the object recognition is selected to implement the layout planning of a factory in 3D. In this chapter, it is demonstrated how to embed the object recognition in the process of virtual 3D layout planning in a built environment as well as which findings and results can be expected. The aim of this chapter is to investigate and evaluate the usefulness of a realistic 3D virtual factory model in factory layout planning primarily for Digital Twin based on object recognition. This is addressed by a practical study of how existing methods for data acquisition and processing can be concatenated and, subsequently, applied under real industrial constraints and conditions. During this study, realistic 3D layout models are created using point clouds acquired by commercial terrestrial laser scanner and prepared for object recognition with Convolutional Neural Networks considering the strict data quality requirements. The selection of models was discussed and the results were evaluated in industrial workshops with engineers involved in the layout planning and machine operators that will work within the production system. Seamless, robust, (semi-)automatic workflow of primarily standard, modular components with low user assistance was of particular interest. This chapter is concluded with the discussion of the achieved results, the solution alternatives, and the present approaches how to extend the utilization, improve, and simplify the entire process.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

previous chapter Data Quality Management for Interoperability

next chapter Design of Simulation Models

Eriksson A, Sedelius E, Berglund J, Johansson B (2018) Virtual factory layouts from 3D laser scanning—a novel framework to define solid model requirements. Procedia CIRP 76:36–41. https://doi.org/10.1016/j.procir.2018.01.013CrossRef

Besbes M, Zolghadri M, Costa Affonso R, Masmoudi F, Haddar M (2020) 3D facility layout problem. J Intell Manuf. https://doi.org/10.1007/s10845-020-01603-zCrossRef

Lindskog E, Berglund J, Vallhagen J, Johansson B (2016) Layout planning and geometry analysis using 3D laser scanning in production system redesign. Procedia CIRP 44:126–131. https://doi.org/10.1016/j.procir.2016.02.118CrossRef

Tolio T, Sacco M, Terkaj W, Urgoa M (2013) Virtual factory: an integrated framework for manufacturing systems design and analysis. Procedia CIRP 7:25–30. https://doi.org/10.1016/j.procir.2013.05.005CrossRef

Iqbal M, Hashmi MSJ (2001) Design and analysis of a virtual factory layout. J Mater Process Technol 118(1–3):403–410. https://doi.org/10.1016/S0924-0136(01)00908-6CrossRef

Kuhn O, Liese H, Stjepandić (2011) Methodology for knowledge-based engineering tem-plate update. In: Cavallucci D, Guio R, Cascini G (eds) Building innovation pipelines through computer-aided innovation. Springer, Berlin Heidelberg, pp 178–191. https://doi.org/10.1007/978-3-642-22182-8_14

Nåfors D, Lindskog E, Berglund J, Gong L, Johansson B, Vallhagen J (2017) Realistic virtual models for factory layout planning. In: IEEE 2017 winter simulation conference (WSC). https://doi.org/10.1109/WSC.2017.8248107

Li CT, Cheng JCP, Chen K (2020) Top 10 technologies for indoor positioning on construction sites. Autom Constr 118:103309. https://doi.org/10.1016/j.autcon.2020.103309.

Geng Z, Bidanda B (2017) Review of reverse engineering systems—current state of the art. Virtual Phys Prototyp 12(2):161–172. https://doi.org/10.1080/17452759.2017.1302787CrossRef

10.

Šagi G, Lulić Z, Mahalec I (2015) Reverse engineering. In: Stjepandić J et al (eds) Concurrent engineering in the 21st century: foundations, developments and challenges. Springer International Switzerland, pp 319–353. https://doi.org/10.1007/978-3-319-13776-6_12

11.

Wang K, Kim M-K, Applications of 3D point cloud data in the construction industry: a fifteen-year review from (2004) to 2018. Adv Eng Inform 39(2019):306–319. https://doi.org/10.1016/j.aei.2019.02.007

12.

Lu Q, Chen L, Li S, Pitta M (2020) Semi-automatic geometric digital twinning for existing buildings based on images and CAD drawings. Autom Constr 115:103183. https://doi.org/10.1016/j.autcon.2020.103183.

13.

Ma Z, Liu S (2018) A review of 3D reconstruction techniques in civil engineering and their applications. Adv Eng Inform 37:163–174. https://doi.org/10.1016/j.aei.2018.05.005CrossRef

14.

Bondar S, Salem B, Stjepandić J (2018) Indoor Object reconstruction based on acquisition by low-cost devices. Adv Transdiscipl Eng 7(2018):113–122. https://doi.org/10.3233/978-1-61499-898-3-113CrossRef

15.

Pučko Z, Šuman N, Rebolj D (2018) Automated continuous construction progress monitoring using multiple workplace real time 3D scans. Adv Eng Inform 38:27–40. https://doi.org/10.1016/j.aei.2018.06.001CrossRef

16.

Fathi H, Dai F, Lourakis M (2015) Automated as-built 3D reconstruction of civil infrastructure using computer vision: achievements, opportunities, and challenges. Adv Eng Inform 29(2):149–161. https://doi.org/10.1016/j.aei.2015.01.012CrossRef

17.

Ríos J, Mas Morate F, Oliva M, Hernández JC (2016) Framework to support the aircraft digital counterpart concept with an industrial design view. Int J Agile Syst Manage 9(3):212–231. https://doi.org/10.1504/IJASM.2016.079934.

18.

Adamenko D, Kunnen S, Pluhnau R, Loibl A, Nagarajah A (2020) Review and comparison of the methods of designing the Digital Twin. Procedia CIRP 91:27–32. https://doi.org/10.1016/j.procir.2020.02.146CrossRef

19.

Fukuda S, Lulić Z, Stjepandić J (2013) FDMU-functional spatial experience beyond DMU? In: Proceedings of the 20th ISPE international conference on concurrent engineering, CE 2013, IOS Press, Amsterdam, pp 431–440. https://doi.org/10.3233/978-1-61499-302-5-431

20.

Biahmou A, Emmer C, Pfouga A, Stjepandić J (2016) Digital master as an enabler for industry 4.0. Adv Transdiscipl Eng 4:672–681. https://doi.org/10.3233/978-1-61499-703-0-672

21.

Pfouga A, Stjepandić J (2018) Leveraging 3D geometric knowledge in the product lifecycle based on industrial standards. J Comput Des Eng 5(1):54–67. https://doi.org/10.1016/j.jcde.2017.11.002CrossRef

22.

Bondar S, Ruppert C, Stjepandić J (2014) Ensuring data quality beyond change management in virtual enterprise. Int J Agile Syst Manage 7(3/4):304–323. https://doi.org/10.1504/IJASM.2014.065346

23.

Princeton Modelnet Repository. https://modelnet.cs.princeton.edu/#. Accessed 30 Nov 2020

24.

Salem B, Stjepandić J, Stobrawa S (2019) Assessment of methods for industrial indoor object recognition. Adv Transdiscipl Eng 10:390–399. https://doi.org/10.3233/ATDE190145CrossRef

25.

Emmer C, Fröhlich A, Stjepandic J (2013) Advanced engineering visualization with standardized 3D formats. In: IFIP advances in information and communication technology, vol 409. Springer, Berlin Heidelberg, pp 584–595. https://doi.org/10.1007/978-3-642-41501-2_58

26.

www.opendesc.com. Accessed 16 November 2020

27.

Ostrosi E, Stjepandić J, Fukuda S, Kurth M (2014) Modularity: New trends for product platform strategy support in concurrent engineering. Adv Transdiscipl Eng 1:414–423. https://doi.org/10.3233/978-1-61499-440-4-414CrossRef

28.

Sommer M, Stjepandić J, Stobrawa S, von Soden M (2021) Automated generation of a digital twin in manufacturing for a built environment using scan and object detection. J Ind Inf Integr (in press)

29.

Ochmann S, Vock R, Klein R (2019) Automatic reconstruction of fully volumetric 3D building models from oriented point clouds. ISPRS J Photogramm Remote Sens 151(2019):251–262. https://doi.org/10.1016/j.isprsjprs.2019.03.017CrossRef

30.

Rusu RB, Cousins S (2011) 3D is here: Point Cloud Library (PCL). In: 2011 IEEE international conference on robotics and automation, ICRA 2011, Article number 5980567. https://doi.org/10.1109/ICRA.2011.5980567

31.

Lehtola VV, Kaartinen H, Nüchter A, Kaijaluoto R, Kukko A, Litkey P, Honkavaara E, Rosnell T, Vaaja MT, Virtanen J-P, Kurkela M, El Issaoui A, Zhu L, Jaakkola A, Hyyppä J (2017) Comparison of the selected state-of-the-art 3D indoor scanning and point cloud generation methods. Remote Sens 9:796. https://doi.org/10.3390/rs9080796CrossRef

32.

Moon S, Chung S, Kwon J, Seo J (2019) Shin, comparison and utilization of point cloud generated from photogrammetry and laser scanning: 3D world model for smart heavy equipment planning. Autom Constr 98:322–331. https://doi.org/10.1016/j.autcon.2018.07.020CrossRef

33.

Garcia-Garcia A, Garcia-Rodriguez J, Orts-Escolano S, Oprea S, Gomez-Donoso F, Cazorla M (2017) A study of the effect of noise and occlusion on the accuracy of convolutional neural networks applied to 3D object recognition. Comput Vis Image Underst 164:124–134. https://doi.org/10.1016/j.cviu.2017.06.006CrossRef

34.

Melcher D, Küster B, Stonis M, Overmeyer L (2019) Fabrik-und Produktionsplanung im digitalen Modell durch automatisierte Punktwolkenverarbeitung. In: Logist J Proceed 2019. https://doi.org/10.2195/lj_Proc_melcher_de_201912_01

35.

Rebolj D, Pučko Z, Čuš Babič N, Bizjak M, Mongus D (2017) Point cloud quality requirements for Scan-vs-BIM based automated construction progress monitoring. Autom Constr 84:323–334. https://doi.org/10.1016/j.autcon.2017.09.021CrossRef

36.

Bicocchi N, Cabri G, Mandreoli F, Mecella M (2019) Dynamic digital factories for agile supply chains: an architectural approach. J Ind Inf Integr 15:111–121. https://doi.org/10.1016/j.jii.2019.02.001CrossRef

37.

Bondar S, Shammaa A, Stjepandić J, Tashiro K (2015) Advances in parameterized CAD feature translation. Adv Transdiscipl Eng 2:615–624. https://doi.org/10.3233/978-1-61499-544-9-615CrossRef

38.

Yang W, Fu C, Yan X, Chen Z (2020) A knowledge-based system for quality analysis in model-based design. J Intell Manuf 24. https://doi.org/10.1007/s10845-020-01535-8

39.

Gürdür D, El-khoury J, Nyberg M (2019) Methodology for linked enterprise data quality assessment through information visualizations. J Ind Inf Integr 15 September 2019, pp 191–200. https://doi.org/10.1016/j.jii.2018.11.002

40.

Sun J, Hiekata K, Yamato H, Nakagaki N, Sugawara A (2014) Virtualization and automation of curved shell plates manufacturing plan design process for knowledge elicitation. Int J Agile Syst Manage 7(3/4):282–303. https://doi.org/10.1504/IJASM.2014.065348CrossRef

41.

Schuh G, Rebentisch E, Riesener M, Ipers T, Tönnes C, Jank MH (2019) Data quality program management for digital shadows of products. Procedia CIRP 86:43–48. https://doi.org/10.1016/j.procir.2020.01.027CrossRef

42.

Kawashima K, Kanai S, H. (2014) As-built modeling of piping system from terrestrial laser-scanned point clouds using normal-based region growing. J Comput Des Eng 1(2013):13–26. https://doi.org/10.7315/JCDE.2014.002CrossRef

43.

Agapaki E, Brilakis I (2020) CLOI-NET: class segmentation of industrial facilities’ point cloud datasets. Adv Eng Inf 45:101121. https://doi.org/10.1016/j.aei.2020.101121

44.

Inyoung Oh, Hee Ko K (2020) Automated recognition of 3D pipelines from point clouds. Vis Comput. https://doi.org/10.1007/s00371-020-01872-yCrossRef

45.

Quintana B, Prieto SA, Adán A, Vázquez AS (2016) Semantic scan planning for indoor structural elements of buildings. Adv Eng Inform 30:643–659. https://doi.org/10.1016/j.aei.2016.08.003CrossRef

46.

Czerniawski T, Leite F (2020) Automated digital modeling of existing buildings: a review of visual object recognition methods. Autom Constr 113. https://doi.org/10.1016/j.autcon.2020.103131

47.

Dekhtiar J, Durupt A, Bricogne M, Eynard B, Rowson H, Kiritsis D (2018) Deep learning for big data applications in CAD and PLM—research review, opportunities and case study. Comput Ind 100:227–243. https://doi.org/10.1016/j.compind.2018.04.005

48.

Liu D, Cui Y, Chen Y, Zhang J, Fan B (2020) Video object detection for autonomous driving: motion-aid feature calibration. Neurocomputing 409:1–11. https://doi.org/10.1016/j.neucom.2020.05.027CrossRef

49.

Otero R, Lagüel S, Garrido I, Arias P (2020) Mobile indoor mapping technologies: a review. Autom Constr 120:103399. https://doi.org/10.1016/j.autcon.2020.103399

50.

da Silva Serapião Leal G, Guédria W, Panetto H (2019) An ontology for interoperability assessment: a systemic approach. J Ind Inf Integr 16:100100. https://doi.org/10.1016/j.jii.2019.07.001

51.

El-khoury J, Berezovskyi A, Nyberg A (2019) An industrial evaluation of data access techniques for the interoperability of engineering software tools. J Ind Inf Integr 15(2019):58–68. https://doi.org/10.1016/j.jii.2019.04.004CrossRef

52.

Wagner L (2020) Kein Digital Twin ohne digitale Durchgängigkeit. Zeitschrift für den wirtschaftlichen Fabrikbetrieb, Band 115, April 2020, pp 43–45. https://doi.org/10.3139/104.112315

53.

Wognum N, Bil C, Elgh F, Peruzzini M, Stjepandić J, Verhagen WJC (2019) Transdisciplinary systems engineering: implications, challenges and research agenda. Int J Agile Syst Manag 12(1):58–89. https://doi.org/10.1504/IJASM.2019.098728

54.

Ruppert T, Abonyi J (2020) Integration of real-time locating systems into digital twins. J Ind Inf Integr 20. https://doi.org/10.1016/j.jii.2020.100174

55.

Xue F, Lu W, Chen K, Webster CJ (2019) BIM reconstruction from 3D point clouds: a semantic registration approach based on multimodal optimization and architectural design knowledge. Adv Eng Inf 42. Article number 100965. https://doi.org/10.1016/j.aei.2019.100965

56.

Ladj A, Wang Z, Meski O, Belkadi F, Ritou M, Da Cunha C (2020) A knowledge-based Digital Shadow for machining industry in a Digital Twin perspective. J Manuf Syst. https://doi.org/10.1016/j.jmsy.2020.07.018CrossRef

57.

Shen H, Bednarz T, Nguyen H, Feng F, Wyel T, Hoek PJ, Lo EHS (2019) Information visualisation methods and techniques: state-of-the-art and future directions. J Ind Inf Integr 16:100102. https://doi.org/10.1016/j.jii.2019.07.003

58.

Bondar S, Potjewijd L, Stjepandić J (2013) Globalized OEM and tier-1 processes at SKF. In: Stjepandic et al (eds) Concurrent engineering approaches for sustainable product development in a multi-disciplinary environment. Springer, London, pp789–800. https://doi.org/10.1007/978-1-4471-4426-7_67

Title: Object Recognition Findings in a Built Environment
Authors: Josip Stjepandić
Sergej Bondar
Wjatscheslaw Korol
Publisher: Springer International Publishing
Book: DigiTwin: An Approach for Production Process Optimization in a Built Environment
Print ISBN: 978-3-030-77538-4

Electronic ISBN: 978-3-030-77539-1

Copyright Year: 2022
DOI: https://doi.org/10.1007/978-3-030-77539-1_8

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Premium Partners