Skip to main content

2017 | OriginalPaper | Buchkapitel

Exploring Design Opportunities for Intelligent Worker Assistance: A New Approach Using Projetion-Based AR and a Novel Hand-Tracking Algorithm

verfasst von : Sebastian Büttner, Oliver Sand, Carsten Röcker

Erschienen in: Ambient Intelligence

Verlag: Springer International Publishing

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

This paper presents a prototype of an intelligent assistive system for workers in stationary manual assembly using projection-based augmented reality (AR) and intelligent hand tracking. By using depth cameras, the system can track the hands of the user and makes the user aware of wrong picking actions or errors in the assembly process. The system automatically adapts the digital projection-based overlay according to the current work situation. The main research contribution of our work is the presentation of a novel hand-tracking algorithm. In addition, we present the results of an user study of the system that shows the challenges and opportunities of our system and the hand-tracking algorithm in particular. We assume that our results will inform the future design of assistive systems in manual assembly.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Bannat, A., Wallhoff, F., Rigoll, G., Friesdorf, F., Bubb, H., Stork, S., Müller, H., Schubö, A., Wiesbeck, M., Zäh, M.F., et al.: Towards optimal worker assistance: a framework for adaptive selection and presentation of assembly instructions. In: Proceedings of the 1st international Workshop on Cognition for Technical Systems, CoTeSys (2008) Bannat, A., Wallhoff, F., Rigoll, G., Friesdorf, F., Bubb, H., Stork, S., Müller, H., Schubö, A., Wiesbeck, M., Zäh, M.F., et al.: Towards optimal worker assistance: a framework for adaptive selection and presentation of assembly instructions. In: Proceedings of the 1st international Workshop on Cognition for Technical Systems, CoTeSys (2008)
2.
Zurück zum Zitat Billinghurst, M., Hakkarainen, M., Woodward, C.: Augmented assembly using a mobile phone. In: Proceedings of the 7th International Conference on Mobile and Ubiquitous Multimedia, pp. 84–87. ACM (2008) Billinghurst, M., Hakkarainen, M., Woodward, C.: Augmented assembly using a mobile phone. In: Proceedings of the 7th International Conference on Mobile and Ubiquitous Multimedia, pp. 84–87. ACM (2008)
3.
Zurück zum Zitat Büttner, S., Funk, M., Sand, O., Röcker, C.: Using head-mounted displays and in-situ projection for assistive systems-a comparison. In: Proceedings of the 9th ACM International Conference on PErvasive Technologies Related to Assistive Environments, vol. 8. ACM (2016) Büttner, S., Funk, M., Sand, O., Röcker, C.: Using head-mounted displays and in-situ projection for assistive systems-a comparison. In: Proceedings of the 9th ACM International Conference on PErvasive Technologies Related to Assistive Environments, vol. 8. ACM (2016)
4.
Zurück zum Zitat Büttner, S., Sand, O., Röcker, C.: Extending the design space in industrial manufacturing through mobile projection. In: Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct, pp. 1130–1133. ACM (2015) Büttner, S., Sand, O., Röcker, C.: Extending the design space in industrial manufacturing through mobile projection. In: Proceedings of the 17th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct, pp. 1130–1133. ACM (2015)
5.
Zurück zum Zitat Dipietro, L., Sabatini, A.M., Dario, P.: A survey of glove-based systems and their applications. IEEE Trans. Syst. Man Cybern. Part C (Appl. Rev.) 38(4), 461–482 (2008)CrossRef Dipietro, L., Sabatini, A.M., Dario, P.: A survey of glove-based systems and their applications. IEEE Trans. Syst. Man Cybern. Part C (Appl. Rev.) 38(4), 461–482 (2008)CrossRef
6.
Zurück zum Zitat Funk, M., Bächler, A., Bächler, L., Korn, O., Krieger, C., Heidenreich, T., Schmidt, A.: Comparing projected in-situ feedback at the manual assembly workplace with impaired workers. In: Proceedings of the 8th ACM International Conference on PErvasive Technologies Related to Assistive Environments, p. 1. ACM (2015) Funk, M., Bächler, A., Bächler, L., Korn, O., Krieger, C., Heidenreich, T., Schmidt, A.: Comparing projected in-situ feedback at the manual assembly workplace with impaired workers. In: Proceedings of the 8th ACM International Conference on PErvasive Technologies Related to Assistive Environments, p. 1. ACM (2015)
7.
Zurück zum Zitat Funk, M., Mayer, S., Schmidt, A.: Using in-situ projection to support cognitively impaired workers at the workplace. In: Proceedings of the 17th International ACM SIGACCESS Conference on Computers and Accessibility, pp. 185–192. ACM (2015) Funk, M., Mayer, S., Schmidt, A.: Using in-situ projection to support cognitively impaired workers at the workplace. In: Proceedings of the 17th International ACM SIGACCESS Conference on Computers and Accessibility, pp. 185–192. ACM (2015)
8.
Zurück zum Zitat Grubert, J., Hamacher, D., Mecke, R., Böckelmann, I., Schega, L., Huckauf, A., Urbina, M., Schenk, M., doil, F., Tümler, J.: Extended investigations of user-related issues in mobile industrial AR. In: 2010 9th IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp. 229–230. IEEE (2010) Grubert, J., Hamacher, D., Mecke, R., Böckelmann, I., Schega, L., Huckauf, A., Urbina, M., Schenk, M., doil, F., Tümler, J.: Extended investigations of user-related issues in mobile industrial AR. In: 2010 9th IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp. 229–230. IEEE (2010)
9.
Zurück zum Zitat Guo, A., Raghu, S., Xie, X., Ismail, S., Luo, X., Simoneau, J., Gilliland, S., Baumann, H., Southern, C., Starner, T.: A comparison of order picking assisted by head-up display (HUD), cart-mounted display (CMD), light, and paper pick list. In: Proceedings of the 2014 ACM International Symposium on Wearable Computers, pp. 71–78. ACM (2014) Guo, A., Raghu, S., Xie, X., Ismail, S., Luo, X., Simoneau, J., Gilliland, S., Baumann, H., Southern, C., Starner, T.: A comparison of order picking assisted by head-up display (HUD), cart-mounted display (CMD), light, and paper pick list. In: Proceedings of the 2014 ACM International Symposium on Wearable Computers, pp. 71–78. ACM (2014)
10.
Zurück zum Zitat Hart, S.G., Staveland, L.E.: Development of NASA-TLX (Task Load Index): results of empirical and theoretical research. Adv. Psychol. 52, 139–183 (1988)CrossRef Hart, S.G., Staveland, L.E.: Development of NASA-TLX (Task Load Index): results of empirical and theoretical research. Adv. Psychol. 52, 139–183 (1988)CrossRef
11.
Zurück zum Zitat Kirch, M., Poenicke, O.: Using the RFID wristband for automatic identification in manual processes-the RFID wristband in the automotive industry. In: 2014 European Conference on Smart Objects, Systems and Technologies (Smart SysTech), pp. 1–7. IEEE (2014) Kirch, M., Poenicke, O.: Using the RFID wristband for automatic identification in manual processes-the RFID wristband in the automotive industry. In: 2014 European Conference on Smart Objects, Systems and Technologies (Smart SysTech), pp. 1–7. IEEE (2014)
12.
Zurück zum Zitat Lasi, H., Fettke, P., Kemper, H.G., Feld, T., Hoffmann, M.: Industry 4.0. business and information. Syst. Eng. 6(4), 239 (2014) Lasi, H., Fettke, P., Kemper, H.G., Feld, T., Hoffmann, M.: Industry 4.0. business and information. Syst. Eng. 6(4), 239 (2014)
13.
Zurück zum Zitat Paelke, V., Röcker, C., Koch, N., Flatt, H., Büttner, S.: User interfaces for cyber-physical systems. at-Automatisierungstechnik 63(10), 833–843 (2015)CrossRef Paelke, V., Röcker, C., Koch, N., Flatt, H., Büttner, S.: User interfaces for cyber-physical systems. at-Automatisierungstechnik 63(10), 833–843 (2015)CrossRef
14.
Zurück zum Zitat Scharstein, D., Szeliski, R.: High-accuracy stereo depth maps using structured light. In: Proceedings of the 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. I-195. IEEE (2003) Scharstein, D., Szeliski, R.: High-accuracy stereo depth maps using structured light. In: Proceedings of the 2003 IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1, pp. I-195. IEEE (2003)
15.
Zurück zum Zitat Schwerdtfeger, B., Klinker, G.: Supporting order picking with augmented reality. In: Proceedings of the 7th IEEE/ACM international Symposium on Mixed and Augmented Reality, pp. 91–94. IEEE Computer Society (2008) Schwerdtfeger, B., Klinker, G.: Supporting order picking with augmented reality. In: Proceedings of the 7th IEEE/ACM international Symposium on Mixed and Augmented Reality, pp. 91–94. IEEE Computer Society (2008)
16.
Zurück zum Zitat Schwerdtfeger, B., Reif, R., Gunthner, W.A., Klinker, G., Hamacher, D., Schega, L., Bockelmann, I., doil, F., Tumler, J.: Pick-by-vision: a first stress test. In: 8th IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2009, pp. 115–124. IEEE (2009) Schwerdtfeger, B., Reif, R., Gunthner, W.A., Klinker, G., Hamacher, D., Schega, L., Bockelmann, I., doil, F., Tumler, J.: Pick-by-vision: a first stress test. In: 8th IEEE International Symposium on Mixed and Augmented Reality, ISMAR 2009, pp. 115–124. IEEE (2009)
17.
Zurück zum Zitat Tang, A., Owen, C., Biocca, F., Mou, W.: Comparative effectiveness of augmented reality in object assembly. In: Proceedings of the SIGCHI conference on Human factors in computing systems, pp. 73–80. ACM (2003) Tang, A., Owen, C., Biocca, F., Mou, W.: Comparative effectiveness of augmented reality in object assembly. In: Proceedings of the SIGCHI conference on Human factors in computing systems, pp. 73–80. ACM (2003)
18.
Zurück zum Zitat Zheng, X.S., Foucault, C., Matos da Silva, P., Dasari, S., Yang, T., Goose, S.: Eye-wearable technology for machine maintenance: effects of display position and hands-free operation. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, pp. 2125–2134. ACM (2015) Zheng, X.S., Foucault, C., Matos da Silva, P., Dasari, S., Yang, T., Goose, S.: Eye-wearable technology for machine maintenance: effects of display position and hands-free operation. In: Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems, pp. 2125–2134. ACM (2015)
Metadaten
Titel
Exploring Design Opportunities for Intelligent Worker Assistance: A New Approach Using Projetion-Based AR and a Novel Hand-Tracking Algorithm
verfasst von
Sebastian Büttner
Oliver Sand
Carsten Röcker
Copyright-Jahr
2017
DOI
https://doi.org/10.1007/978-3-319-56997-0_3