Skip to main content
Top
Published in:

2023 | OriginalPaper | Chapter

An Integrated Visualization Framework to Enhance Human–Robot Collaboration in Facility Management

Authors : Yonglin Fu, Junjie Chen, Yipeng Pan, Weisheng Lu

Published in: Proceedings of the 27th International Symposium on Advancement of Construction Management and Real Estate

Publisher: Springer Nature Singapore

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

The global pandemic has sparked the popularity of robots in facility management tasks (FMTs) such as floor cleaning and disinfection. This trend brings many task scenarios where humans and robots need to cooperate with each other. Effective human–robot collaboration (HRC) relies on precise communication of the robots’ intentions (e.g., to move to a position, or to grasp an object), so that their human counterparts can adapt their behaviors/actions accordingly. However, little has been known on how this can be done in FMTs. Visualization technologies have potential to enhance HRC by communicating the robot intention in a visualized manner. This research aims to develop a framework that integrates the latest visualization technologies, e.g., building information modelling (BIM) and augmented reality (AR), to enhance HRC in facility management. The framework includes two complementary modules: (a) a remote monitoring module (RMM) that can remotely transmit and visualize robot information in a Web-based BIM to inform decision-making, and (b) an onsite collaboration module (OCM) that augments human co-workers with real-time robot intention to allow effective cooperation. Experiments were conducted to validate the proposed framework in typical FMTs. Results show that the integrated visualization framework can intuitively and unambiguously convey robots’ intentions to their human counterparts, significantly improving the performance of HRC. Future research is suggested to complement the framework with a reverse mechanism to effectively convey human intentions to robots.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Shen, Y., et al.: Robots under COVID-19 pandemic: A comprehensive survey. Ieee Access 9, 1590–1615 (2020)CrossRef Shen, Y., et al.: Robots under COVID-19 pandemic: A comprehensive survey. Ieee Access 9, 1590–1615 (2020)CrossRef
2.
go back to reference Wang, X.V., Wang, L.: A literature survey of the robotic technologies during the COVID-19 pandemic. J. Manuf. Sys. 60, 823–836 (2021)CrossRef Wang, X.V., Wang, L.: A literature survey of the robotic technologies during the COVID-19 pandemic. J. Manuf. Sys. 60, 823–836 (2021)CrossRef
3.
go back to reference Beanne, M., Brynjolfsson, E.. Working with robots in a post-pandemic world. MIT Sloan Manage. Rev. 62(2) (2020) Beanne, M., Brynjolfsson, E.. Working with robots in a post-pandemic world. MIT Sloan Manage. Rev. 62(2) (2020)
4.
go back to reference Cresswell, K., Sheikh, A.: Can disinfection robots reduce the risk of transmission of SARS-CoV-2 in health care and educational settings? J. Med. Inter. Res. 22(9), e20896 (2020) Cresswell, K., Sheikh, A.: Can disinfection robots reduce the risk of transmission of SARS-CoV-2 in health care and educational settings? J. Med. Inter. Res. 22(9), e20896 (2020)
5.
go back to reference You, S., Kim, J.H., Lee, S., Kamat, V., Robert, L.P., Jr.: Enhancing perceived safety in human–robot collaborative construction using immersive virtual environments. Automation in Construction 96, 161–170 (2018)CrossRef You, S., Kim, J.H., Lee, S., Kamat, V., Robert, L.P., Jr.: Enhancing perceived safety in human–robot collaborative construction using immersive virtual environments. Automation in Construction 96, 161–170 (2018)CrossRef
6.
go back to reference Luperto, M., et al.: What is my robot doing? remote supervision to support robots for older adults independent living: a field study. In: 2021 European Conference on Mobile Robots (ECMR), pp. 1–7. IEEE (2021) Luperto, M., et al.: What is my robot doing? remote supervision to support robots for older adults independent living: a field study. In: 2021 European Conference on Mobile Robots (ECMR), pp. 1–7. IEEE (2021)
7.
go back to reference Bauer, A., Wollherr, D., Buss, M.: Human–robot collaboration: a survey. Int. J. Humano. Robot. 5(01), 47–66 (2008)CrossRef Bauer, A., Wollherr, D., Buss, M.: Human–robot collaboration: a survey. Int. J. Humano. Robot. 5(01), 47–66 (2008)CrossRef
8.
go back to reference Ajoudani, A., Zanchettin, A.M., Ivaldi, S., Albu-Schäffer, A., Kosuge, K., Khatib, O.: Progress and prospects of the human–robot collaboration. Autonomous Robots 42(5), 957–975 (2018)CrossRef Ajoudani, A., Zanchettin, A.M., Ivaldi, S., Albu-Schäffer, A., Kosuge, K., Khatib, O.: Progress and prospects of the human–robot collaboration. Autonomous Robots 42(5), 957–975 (2018)CrossRef
9.
go back to reference Hjorth, S., Chrysostomou, D.: Human–robot collaboration in industrial environments: A literature review on non-destructive disassembly. Robot. Comp.-Integr. Manuf. 73, 102208 (2022)CrossRef Hjorth, S., Chrysostomou, D.: Human–robot collaboration in industrial environments: A literature review on non-destructive disassembly. Robot. Comp.-Integr. Manuf. 73, 102208 (2022)CrossRef
10.
go back to reference Liu, R., Issa, R.R.A.: 3D visualization of sub-surface pipelines in connection with the building utilities: Integrating GIS and BIM for facility management. In: Computing in Civil Engineering, pp. 341–348 (2012) Liu, R., Issa, R.R.A.: 3D visualization of sub-surface pipelines in connection with the building utilities: Integrating GIS and BIM for facility management. In: Computing in Civil Engineering, pp. 341–348 (2012)
11.
go back to reference Baek, F., Ha, I., Kim, H.: Augmented reality system for facility management using image-based indoor localization. Automation in construction 99, 18–26 (2019)CrossRef Baek, F., Ha, I., Kim, H.: Augmented reality system for facility management using image-based indoor localization. Automation in construction 99, 18–26 (2019)CrossRef
12.
go back to reference Chen, J., Li, S., Lu, W.: Align to locate: Registering photogrammetric point clouds to BIM for robust indoor localization. Building and Environment 209, 108675 (2022)CrossRef Chen, J., Li, S., Lu, W.: Align to locate: Registering photogrammetric point clouds to BIM for robust indoor localization. Building and Environment 209, 108675 (2022)CrossRef
14.
go back to reference Yin, J., Apuroop, K.G.S., Tamilselvam, Y.K., Mohan, R.E., Ramalingam, B., Le, A.V.: Table cleaning task by human support robot using deep learning technique. Sensors 20(6), 1698 (2020)CrossRef Yin, J., Apuroop, K.G.S., Tamilselvam, Y.K., Mohan, R.E., Ramalingam, B., Le, A.V.: Table cleaning task by human support robot using deep learning technique. Sensors 20(6), 1698 (2020)CrossRef
15.
go back to reference Syahrian, N.M., Risma, P., Dewi, T.: Vision-based pipe monitoring robot for crack detection using canny edge detection method as an image processing technique. Kinetik: Game Technology, Information System, Computer Network, Computing, Electronics, and Control, 243–250 (2017) Syahrian, N.M., Risma, P., Dewi, T.: Vision-based pipe monitoring robot for crack detection using canny edge detection method as an image processing technique. Kinetik: Game Technology, Information System, Computer Network, Computing, Electronics, and Control, 243–250 (2017)
16.
go back to reference Sycara, K., Sukthankar, G.: Literature review of teamwork models. Robot. Ins. Carneg. Mell. Univ. 31, 31 (2006) Sycara, K., Sukthankar, G.: Literature review of teamwork models. Robot. Ins. Carneg. Mell. Univ. 31, 31 (2006)
17.
go back to reference Coovert, M.D., Lee, T., Shindev, I., Sun, Y.: Spatial augmented reality as a method for a mobile robot to communicate intended movement. Comp. Hum. Behav. 34, 241–248 (2014)CrossRef Coovert, M.D., Lee, T., Shindev, I., Sun, Y.: Spatial augmented reality as a method for a mobile robot to communicate intended movement. Comp. Hum. Behav. 34, 241–248 (2014)CrossRef
18.
go back to reference Schaefer, K.E., Straub, E.R., Chen, J.Y., Putney, J., Evans, A.W., III.: Communicating intent to develop shared situation awareness and engender trust in human-agent teams. Cognit. Sys. Res. 46, 26–39 (2017)CrossRef Schaefer, K.E., Straub, E.R., Chen, J.Y., Putney, J., Evans, A.W., III.: Communicating intent to develop shared situation awareness and engender trust in human-agent teams. Cognit. Sys. Res. 46, 26–39 (2017)CrossRef
19.
go back to reference Chang, K.M., Dzeng, R.J., Wu, Y.J.: An automated IoT visualization BIM platform for decision support in facilities management. Applied Sciences 8(7), 1086 (2018)CrossRef Chang, K.M., Dzeng, R.J., Wu, Y.J.: An automated IoT visualization BIM platform for decision support in facilities management. Applied Sciences 8(7), 1086 (2018)CrossRef
20.
go back to reference Natephra, W., Motamedi, A.: Live data visualization of IoT sensors using augmented reality (AR) and BIM. In: 36th International Symposium on Automation and Robotics in Construction (ISARC 2019) (2019, May) Natephra, W., Motamedi, A.: Live data visualization of IoT sensors using augmented reality (AR) and BIM. In: 36th International Symposium on Automation and Robotics in Construction (ISARC 2019) (2019, May)
21.
go back to reference Chen, J., Li, S., Liu, D., Lu, W.: Indoor camera pose estimation via style-transfer 3D models. Comp.-Aided Civil and Infrastr. Eng. 37(3), 335–353 (2022)CrossRef Chen, J., Li, S., Liu, D., Lu, W.: Indoor camera pose estimation via style-transfer 3D models. Comp.-Aided Civil and Infrastr. Eng. 37(3), 335–353 (2022)CrossRef
Metadata
Title
An Integrated Visualization Framework to Enhance Human–Robot Collaboration in Facility Management
Authors
Yonglin Fu
Junjie Chen
Yipeng Pan
Weisheng Lu
Copyright Year
2023
Publisher
Springer Nature Singapore
DOI
https://doi.org/10.1007/978-981-99-3626-7_1