Skip to main content

2020 | OriginalPaper | Buchkapitel

2. Augmented Reality for Interactive Robot Control

verfasst von : Levi Manring, John Pederson, Dillon Potts, Beth Boardman, David Mascarenas, Troy Harden, Alessandro Cattaneo

Erschienen in: Special Topics in Structural Dynamics & Experimental Techniques, Volume 5

Verlag: Springer International Publishing

Aktivieren Sie unsere intelligente Suche um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Robots are widely used to support mission-critical, high-risk and complex operations. Human supervision and remote robot control are often required to operate robots in unpredictable and changing scenarios. Often, robots are controlled remotely by technicians via joystick interfaces which require training and experience to operate. To improve robot usage and practicality, we propose using augmented reality (AR) to create a more intuitive, less training-intensive means of controlling robots than traditional joystick control. AR is a creative platform for developing robot control systems, because AR combines the real world (the environment around the user, the physical robot, etc.) with the digital world (holograms, digital displays, etc.); it can even interpret physical gestures, such as pinching two fingers.
In this research, a Microsoft Hololens headset is used to create an AR environment to control a Yaskawa Motoman SIA5D robot. The control process begins with the user placing an interactable holographic robot in 3D space. The user can then select between two control methods: manual control and automatic control. In manual control, the user can move the end effector of the holographic robot and the physical robot will respond immediately. In automatic control, the user can move the end effector of the holographic robot to a desired location, view a holographic preview of the motion, and select execute if the motion plan is satisfactory. In this preview mode, the user is able to preview both the motion of the robot and the torques experienced by the joints of the manipulator. This gives the user additional feedback on the planned motion. In this project we succeeded in creating an AR control system that makes controlling a robotic manipulator intuitive and effective.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Kim, S.L., Suk, H.J., Kang, J.H., Jung, J.M., Laine, T.H., Westlin, J.: Using unity 3D to facilitate mobile augmented reality game development. In: 2014 IEEE World Forum on Internet of Things (WF-IoT), pp. 21–26. IEEE, Hoboken (2014)CrossRef Kim, S.L., Suk, H.J., Kang, J.H., Jung, J.M., Laine, T.H., Westlin, J.: Using unity 3D to facilitate mobile augmented reality game development. In: 2014 IEEE World Forum on Internet of Things (WF-IoT), pp. 21–26. IEEE, Hoboken (2014)CrossRef
2.
Zurück zum Zitat Lee, A., Lee, J.-H., Kim, J.: Data-driven kinematic control for robotic spatial augmented reality system with loose kinematic specifications. ETRI J. 38(2), 337–346 (2016)CrossRef Lee, A., Lee, J.-H., Kim, J.: Data-driven kinematic control for robotic spatial augmented reality system with loose kinematic specifications. ETRI J. 38(2), 337–346 (2016)CrossRef
3.
Zurück zum Zitat Kuriya, R., Tsujimura, T., Izumi, K.: Augmented reality robot navigation using infrared marker. In: Robot and Human Interactive Communication (RO-MAN), 2015 24th IEEE International Symposium on, pp. 450–455. IEEE, Hoboken (2015) Kuriya, R., Tsujimura, T., Izumi, K.: Augmented reality robot navigation using infrared marker. In: Robot and Human Interactive Communication (RO-MAN), 2015 24th IEEE International Symposium on, pp. 450–455. IEEE, Hoboken (2015)
4.
Zurück zum Zitat Zeng, H., Wang, Y., Wu, C., Song, A., Liu, J., Ji, P., Xu, B., Zhu, L., Li, H., Wen, P.: Closed-loop hybrid gaze brain-machine interface based robotic arm control with augmented reality feedback. Front. Neurorobot. 11, 60 (2017)CrossRef Zeng, H., Wang, Y., Wu, C., Song, A., Liu, J., Ji, P., Xu, B., Zhu, L., Li, H., Wen, P.: Closed-loop hybrid gaze brain-machine interface based robotic arm control with augmented reality feedback. Front. Neurorobot. 11, 60 (2017)CrossRef
5.
Zurück zum Zitat Yew, A.W.W., Ong, S.K., Nee, A.Y.C.: Immersive augmented reality environment for the teleoperation of maintenance robots. Procedia CIRP. 61, 305–310 (2017)CrossRef Yew, A.W.W., Ong, S.K., Nee, A.Y.C.: Immersive augmented reality environment for the teleoperation of maintenance robots. Procedia CIRP. 61, 305–310 (2017)CrossRef
6.
Zurück zum Zitat Hashimoto, S., Ishida, A., Inami, M., Igarashi, T.: Touchme: an augmented reality based remote robot manipulation. In: 21st International Conference on Artificial Reality and Telexistence, Proceedings of ICAT2011. Osaka University, Osaka (2011) Hashimoto, S., Ishida, A., Inami, M., Igarashi, T.: Touchme: an augmented reality based remote robot manipulation. In: 21st International Conference on Artificial Reality and Telexistence, Proceedings of ICAT2011. Osaka University, Osaka (2011)
7.
Zurück zum Zitat Fang, H.C., Ong, S.K., Nee, A.Y.C.: Interactive robot trajectory planning and simulation using augmented reality. Robot. Comput. Integr. Manuf. 28(2), 227–237 (2012)CrossRef Fang, H.C., Ong, S.K., Nee, A.Y.C.: Interactive robot trajectory planning and simulation using augmented reality. Robot. Comput. Integr. Manuf. 28(2), 227–237 (2012)CrossRef
8.
Zurück zum Zitat Fang, H.C., Ong, S.K., Nee, A.Y.C.: Orientation planning of robot end-effector using augmented reality. Int. J. Adv. Manuf. Technol. 67(9–12), 2033–2049 (2013)CrossRef Fang, H.C., Ong, S.K., Nee, A.Y.C.: Orientation planning of robot end-effector using augmented reality. Int. J. Adv. Manuf. Technol. 67(9–12), 2033–2049 (2013)CrossRef
9.
10.
Zurück zum Zitat Zhao, Z., Huang, P., Lu, Z., Liu, Z.: Augmented reality for enhancing tele-robotic system with force feedback. Robot. Auton. Syst. 96, 93–101 (2017)CrossRef Zhao, Z., Huang, P., Lu, Z., Liu, Z.: Augmented reality for enhancing tele-robotic system with force feedback. Robot. Auton. Syst. 96, 93–101 (2017)CrossRef
11.
Zurück zum Zitat Chen, R.-J., Lin, H.-W., Chang, Y.-H., Wu, C.-T., Lee, S.-T.: Development of an augmented reality force feedback virtual surgery training platform. Int J. Autom. Smart Technol. 1(1), 41–51 (2011)CrossRef Chen, R.-J., Lin, H.-W., Chang, Y.-H., Wu, C.-T., Lee, S.-T.: Development of an augmented reality force feedback virtual surgery training platform. Int J. Autom. Smart Technol. 1(1), 41–51 (2011)CrossRef
12.
Zurück zum Zitat Culbertson, H., Kuchenbecker, K.J.: Ungrounded haptic augmented reality system for displaying roughness and friction. IEEE-ASME Trans. Mech. 22(4), 1839–1849 (2017)CrossRef Culbertson, H., Kuchenbecker, K.J.: Ungrounded haptic augmented reality system for displaying roughness and friction. IEEE-ASME Trans. Mech. 22(4), 1839–1849 (2017)CrossRef
13.
Zurück zum Zitat Lee, S., Lucas, N.P., Darin Ellis, R., Pandya, A.: Development and human factors analysis of an augmented reality interface for multi-robot tele-operation and control. In: Unmanned Systems Technology XIV, vol. 8387, p. 83870N. International Society for Optics and Photonics, Baltimore, Maryland (2012)CrossRef Lee, S., Lucas, N.P., Darin Ellis, R., Pandya, A.: Development and human factors analysis of an augmented reality interface for multi-robot tele-operation and control. In: Unmanned Systems Technology XIV, vol. 8387, p. 83870N. International Society for Optics and Photonics, Baltimore, Maryland (2012)CrossRef
14.
Zurück zum Zitat Hu, Y., Meng, W.: Rosunitysim: Development and experimentation of a real-time simulator for multi-unmanned aerial vehicle local planning. Simulation. 92(10), 931–944 (2016)CrossRef Hu, Y., Meng, W.: Rosunitysim: Development and experimentation of a real-time simulator for multi-unmanned aerial vehicle local planning. Simulation. 92(10), 931–944 (2016)CrossRef
15.
Zurück zum Zitat Mizuchi, Y., Inamura, T.: Cloud-based multimodal human-robot interaction simulator utilizing ros and unity frameworks. In: System Integration (SII), 2017 IEEE/SICE International Symposium on, pp. 948–955. IEEE, Hoboken (2017)CrossRef Mizuchi, Y., Inamura, T.: Cloud-based multimodal human-robot interaction simulator utilizing ros and unity frameworks. In: System Integration (SII), 2017 IEEE/SICE International Symposium on, pp. 948–955. IEEE, Hoboken (2017)CrossRef
16.
Zurück zum Zitat Codd-Downey, R., Mojiri Forooshani, P., Speers, A., Wang, H., Jenkin, M.: From ROS to unity: leveraging robot and virtual environment middleware for immersive teleoperation. In: Information and Automation (ICIA), 2014 IEEE International Conference on, pp. 932–936. IEEE, Hoboken (2014)CrossRef Codd-Downey, R., Mojiri Forooshani, P., Speers, A., Wang, H., Jenkin, M.: From ROS to unity: leveraging robot and virtual environment middleware for immersive teleoperation. In: Information and Automation (ICIA), 2014 IEEE International Conference on, pp. 932–936. IEEE, Hoboken (2014)CrossRef
17.
Zurück zum Zitat Sita, E., Horvath, C.M., Thomessen, T., Korondi, P., Pipe, A.G.: ROS-Unity3D based system for monitoring of an industrial robotic process. In: 2017 IEEE/SICE International Symposium on System Integration (SII). IEEE, Hoboken (2018) Sita, E., Horvath, C.M., Thomessen, T., Korondi, P., Pipe, A.G.: ROS-Unity3D based system for monitoring of an industrial robotic process. In: 2017 IEEE/SICE International Symposium on System Integration (SII). IEEE, Hoboken (2018)
Metadaten
Titel
Augmented Reality for Interactive Robot Control
verfasst von
Levi Manring
John Pederson
Dillon Potts
Beth Boardman
David Mascarenas
Troy Harden
Alessandro Cattaneo
Copyright-Jahr
2020
DOI
https://doi.org/10.1007/978-3-030-12243-0_2

    Marktübersichten

    Die im Laufe eines Jahres in der „adhäsion“ veröffentlichten Marktübersichten helfen Anwendern verschiedenster Branchen, sich einen gezielten Überblick über Lieferantenangebote zu verschaffen.