nach oben

Computer Supported Cooperative Work (CSCW)

Erschienen in:

01.12.2015

Design and Implementation of TeleAdvisor: a Projection-Based Augmented Reality System for Remote Collaboration

verfasst von: Pavel Gurevich, Joel Lanir, Benjamin Cohen

Erschienen in: Computer Supported Cooperative Work (CSCW) | Ausgabe 6/2015

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

TeleAdvisor is a versatile projection-based augmented reality system designed for remote collaboration. It allows a remote expert to naturally guide a local user in need of assistance in carrying out physical tasks around real-world objects. The system consists of a small projector and two cameras mounted on top of a tele-operated robotic arm at the worker’s side, and an interface to view the camera stream, control the point-of-view and gesture using projected annotations at the remote expert’s side. TeleAdvisor provides a hands-free, mobile, low-cost solution that supports gesturing by the remote expert while minimizing the cognitive overhead of the local worker. We describe the challenges, design considerations and implementation details of the two phases of the TeleAdvisor prototype, as well as its evaluation and deployment at an industrial manufacturing center. We summarize our understandings from our experiences during the project and discuss the general implications for design of augmented reality remote collaboration systems.

Vorheriger Artikel Collaboration in Augmented Reality

Nächster Artikel The Effect of View Independence in a Collaborative AR System

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

For MicroVision ShowWX+HDMI these are especially noticeable in top left and right corners.

The second video stream is transmitted for the sake of depth map calculation at the Helper’s workstation.

This caused several servos to burn out.

Alem, Leila, Franco Tecchia, and Weidong Huang, (2011). HandsOnVideo: Towards a Gesture based Mobile AR System for Remote Collaboration. In Recent Trends of Mobile Collaborative Augmented Reality Systems. New York: Springer, pp. 135–148. doi:10.1007/978-1-4419-9845-3

Anderson, Richard J., Crystal Hoyer, Steven A. Wolfman, and Ruth Anderson. (2004). A study of digital ink in lecture presentation. In Proceedings of the SIGCHI conference on Human factors in computing systems. New York: ACM Press, pp. 567–574. doi: 10.1145/985692.985764

Azuma, Ronald. (1997). A survey of augmented reality. Presence: Teleoperators and Virtual Environments, vol. 6, no. 4, pp. 355–385. doi:10.1.1.30.4999CrossRef

Azuma, Ronald, Yohan Baillot, Reinhold Behringer, Steven Feiner, Simon Julier, and Blair MacIntyre (2001). Recent advances in augmented reality. IEEE Computer Graphics and Applications, vol. 21, no. 6, pp. 34–47. doi:10.1109/38.963459 CrossRef

Baird, Kevin M., and Woodrow Barfield (1999). Evaluating the effectiveness of augmented reality displays for a manual assembly task. Virtual Reality. vol 4, no. 4, pp. 250–259. doi:10.1007/BF01421808 CrossRef

Bauer, Martin, Gerd Kortuem, and Zary Segall. (1999). Where are you pointing at? A study of remote collaboration in a wearable videoconference system. In Third International Symposium on Wearable Computers, San-Fransisco, CA, 18–19 October, 1999. Washington, DC: IEEE computer society, pp. 151–158. doi:10.1109/ISWC.1999.806696

Billinghurst, Mark, and Hirokazu Kato (2002). Collaborative augmented reality. Communications of the ACM. vol. 45, no. 7, pp. 64–70. doi:10.1145/514236.514265 CrossRef

Birchfield, Stan, and Carlo Tomasi (1998). A pixel dissimilarity measure that is insensitive to image sampling. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol 20, no. 4, pp. 401–406. doi:10.1109/34.677269 CrossRef

Birnholtz, Jeremy, Abhishek Ranjan, and Ravin Balakrishnan (2010). Providing Dynamic Visual Information for Collaborative Tasks: Experiments With Automatic Camera Control. Human-Computer Interaction. vol. 25, no. 4, pp. 261–287. doi:10.1080/07370024.2010.500146 CrossRef

Bradski, Gary, and Adrian Kaehler (2008). Learning OpenCV: Computer Vision with the OpenCV Library. Sebastopol, CA: OReilly Media Inc. doi:10.1109/MRA.2009.933612

Brown, Michael, Aditi Majumder, and Ruigang Yang (2005). Camera-based calibration techniques for seamless multiprojector displays. IEEE Transactions on Visualization and Computer Graphics, vol. 11, no. 2, pp. 193–206CrossRef

Chen, Jessie YC, Ellen C. Haas, and Michael J. Barnes (2007). Human performance issues and user interface design for teleoperated robots. Systems, Man, and Cybernetics, Part C: Applications and Reviews, IEEE Transactions on, vol. 37, no. 6, pp. 1231–1245. doi:10.1109/TSMCC.2007.905819

Darken, Rudolph P., and Barry Peterson (2001). Spatial Orientation, wayfinding and representation. In K. M. Stanney (Ed.): Handbook of Virtual Environment Technology. Mahwah, New Jersey: Lawrence Erlbaum Associates, pp. 493–518

Flagg, Matthew, and James M. Rehg (2006). Projector-Guided Painting. In Proceedings of the 19th Annual ACM Symposium on User Interface Software and Technology (UIST 2006). New York: ACM Press. pp. 235–244. doi:10.1145/1166253.1166290

Fong, Terrence, and Charles Thorpe. (2001). Vehicle teleoperation interfaces. Autonomous Robots, vol. 11, no. 1., pp 9–18. doi:10.1023/A:1011295826834 CrossRefMATH

Fussell, Susan R., Leslie D. Setlock, and Robert E. Kraut. (2003). Effects of head-mounted and scene-oriented video systems on remote collaboration on physical tasks. In Proceedings of the Conference on Human Factors in Computing Systems - CHI′03, pp. 513–520. New-York: ACM Press. doi:10.1145/642611.642701

Fussell, Susan R., Leslie D. Setlock, Jie Yang, Jiazhi Ou, Elizabeth Mauer, and Adam DI Kramer. (2004). Gestures Over Video Streams to Support Remote Collaboration on Physical Tasks. Human-Computer Interaction. vol. 19, no. 3, pp. 273–309. doi:10.1207/s15327051hci1903_3

Gauglitz, Steffen, Cha Lee, Matthew Turk, and Tobias Höllerer. (2012). Integrating the physical environment into mobile remote collaboration. Proceedings of the 14th International Conference on Human-Computer Interaction with Mobile Devices and Services - MobileHCI′12. New-York: ACM Press. pp. 241–250. doi:10.1145/2371574.2371610

Gauglitz, Steffen, Benjamin Nuernberger, Matthew Turk, and Tobias Höllerer (2014). World-stabilized annotations and virtual scene navigation for remote collaboration. Proceedings of the 27th annual ACM symposium on User interface software and technology. New-York: ACM Press. pp. 449–459.

Gergle, Darren, Robert E. Kraut, and Susan R. Fussell. (2013). Using Visual Information for Grounding and Awareness in Collaborative Tasks. Human-Computer Interaction, vol. 28, no. 1, pp. 1–39. doi:10.1080/07370024.2012.678246

Gurevich, Pavel, Joel Lanir, Benjamin Cohen, and Ran Stone. (2012). TeleAdvisor: a versatile augmented reality tool for remote assistance. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. New-York: ACM Press. pp. 619–622. doi:10.1145/2207676.2207763

Hartley, Richard, and Andrew Zisserman (2003). Multiple view geometry in computer vision. New York: Cambridge university press.

Henderson, Steven J., and Steven Feiner. (2009). Evaluating the benefits of augmented reality for task localization in maintenance of an armored personnel carrier turret. In Science and Technology Proceedings - IEEE 2009 International Symposium on Mixed and Augmented Reality, ISMAR 2009. Washington DC: IEEE Computing Society, pp. 135–144. doi:10.1109/ISMAR.2009.5336486

Henderson, Steven J., and Steven Feiner (2011). Exploring the benefits of augmented reality documentation for maintenance and repair. IEEE Transactions on Visualization and Computer Graphics, vol 17, no. 10, pp. 1355–1368. doi:10.1109/TVCG.2010.245 CrossRef

Hirschmüller, Heiko (2008). Stereo processing by semiglobal matching and mutual information. IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 30, no. 2., pp. 328–341. doi:10.1109/TPAMI.2007.1166 CrossRef

Huang, Weidong, and Leila Alem (2011). Supporting hand gestures in mobile remote collaboration: a usability evaluation. In Proceedings of the 25th BCS Conference on Human-Computer Interaction. Swinton, UK: British Computer Society. pp. 211–216.

Huang, Weidong, Leila Alem, and Franco Tecchia. (2013). HandsIn3D: supporting remote guidance with immersive virtual environments. In Human-Computer Interaction--INTERACT 2013 lecture notes in computer science. vol. 8117, pp. 70–77. Berlin: Springer

Ishii, Hiroshi, Eran Ben-Joseph, John Underkoffler, Luke Yeung, Dan Chak, Zahra Kanji, and Ben Piper. (2002). Augmented urban planning workbench: overlaying drawings, physical models and digital simulation. In Proceedings of Ismar′02.The first International Symposium on Mixed and Augmented Reality. Washington DC: IEEE Computer Society, pp. 203–211 doi:10.1109/ISMAR.2002.1115090

Junuzovic, Sasa, Kori Inkpen, Tom Blank, and Anoop Gupta. (2012). IllumiShare: sharing any surface. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. New York: ACM Press, pp. 1919–1928. doi:10.1145/2207676.2208333

Kirk, David, and Danae Stanton Fraser (2006). Comparing remote gesture technologies for supporting collaborative physical tasks. In Proceedings of the SIGCHI conference on Human Factors in computing systems. New York: ACM Press, pp. 1191–1200.

Kirk, David, Andy Crabtree, and Tom Rodden (2005). Ways of the Hands. In Proceedings of the ninth european conference on Computer-Supported Cooperative Work 18–22 September, Paris, France. pp. 1–22. Springer Netherlands.

Kirk, David, Tom Rodden, and Danaë Stanton Fraser (2007). Turn it THIS way: grounding collaborative action with remote gestures. In Proceedings of ACM CHI 2007 Conference on Human Factors in Computing Systems. New York: ACM Press, pp. 1039–1048. doi:10.1145/1240624.1240782

Kraut, Robert E., Susan R. Fussell, and Jane Siegel (2003). Visual Information as a Conversational Resource in Collaborative Physical Tasks. Human-Computer Interaction. vol. 18, no. 1, pp. 13–59. doi:10.1207/S15327051HCI1812_2 CrossRef

Kurata, Takeshi, Nobuchika Sakata, Masakatsu Kourogi, Hideaki Kuzuoka, and Mark Billinghurst. (2004). Remote collaboration using a shoulder-worn active camera/laser. Eighth International Symposium on Wearable Computers, vol 1. Washington DC: IEEE Computer Society, pp. 62–69. doi:10.1109/ISWC.2004.37

Kuzuoka, Hideaki, Shinya Oyama, Keiichi Yamazaki, Kenji Suzuki, and Mamoru Mitsuishi. (2000). GestureMan: a mobile robot that embodies a remote instructor’s actions. Proceedings of the ACM conference on Computer Supprted Cooperative Work. New York: ACM Press, pp. 155–162. doi:10.1145/358916.358986

Lanir, Joel, Ran Stone, Benjamin Cohen, and Pavel Gurevich (2013). Ownership and Control of Point of View in Remote Assistance. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems - CHI′13. New York: ACM Press, pp. 2243–2252. doi:10.1145/2470654.2481309

Linder, Natan, and Pattie Maes. (2010). LuminAR: Portable Robotic Augmented Reality Interface Design and Prototype. In Adjunct proceedings of the 23nd annual ACM symposium on User interface software and technology. New York: ACM Press, pp. 395–396. doi:10.1145/1866218.1866237

Machino, Tamotsu, Satoshi Iwaki, Hiroaki Kawata, Yoshimasa Yanagihara, Yoshito Nanjo, and Ken-ichiro Shimokura (2006). Remote-collaboration system using mobile robot with camera and projector. In Proceedings - IEEE International Conference on Robotics and Automation 2006. Washington DC: IEEE Computer Society, pp. 4063–4068. doi:10.1109/ROBOT.2006.1642326

Majumder, Aditi, Zhu He, Herman Towles, and Greg Welch. (2000). Achieving color uniformity across multi-projector displays. Proceedings Visualization 2000. VIS 2000. Washington DC: IEEE Computer Society, pp 117–124. doi:10.1109/VISUAL.2000.885684

Medioni, Gerard, and Ramakant Nevatia (1985). Segment-based stereo matching. Computer Vision, Graphics, and Image Processing. vol 31, no 1., pp. 2–18. doi:10.1016/S0734-189X(85)80073-6 CrossRef

Mistry, Pranav, Pattie Maes, and Liyan Chang (2009). WUW - Wear Ur World: A Wearable Gestural Interface. In CHI′09 Extended Abstracts on Human Factors in Computing Systems. New York: ACM Press, pp. 4111–4116. doi:10.1145/1520340.1520626

Motokawa, Yoichi, and Hideo Saito (2007). Support system for guitar playing using augmented reality display. In Proceedings - ISMAR 2006: Fifth IEEE and ACM International Symposium on Mixed and Augmented Reality. Washington DC: IEEE Computer Society, pp. 243–244. doi:10.1109/ISMAR.2006.297825

Murphy, Robin Roberson (2004). Human-robot interaction in rescue robotics. IEEE Transactions on Systems, Man and Cybernetics Part C: Applications and Reviews. vol 34. no. 2, pp 138–153. doi:10.1109/TSMCC.2004.826267 CrossRef

Neumann, Ulrich, and Anthony Majoros (1998). Cognitive, performance, and systems issues for augmented reality applications in manufacturing and maintenance. Proceedings. IEEE 1998 Virtual Reality Annual International Symposium. Washington DC: IEEE Computer Society, pp. 4–11. doi:10.1109/VRAIS.1998.658416

O’Neill, Jacki, Stefania Castellani, Frederic Roulland, Nicolas Hairon, Cornell Juliano, and Liwei Dai (2011). From ethnographic study to mixed reality: a remote collaborative troubleshooting system. In Proceedings of the ACM 2011 conference on Computer supported cooperative work. New York: ACM Press, pp. 225–234. doi:10.1145/1958824.1958859

Ou, Jiazhi, Susan R. Fussell, Xilin Chen, Leslie D. Setlock, and Jie Yang (2003). Gestural communication over video stream: supporting multimodal interaction for remote collaborative physical tasks. In Proceedings of the 5th international conference on Multimodal interfaces. New York: ACM Press, pp. 242–249.

Ranjan, Abhishek, Jeremy P. Birnholtz, and Ravin Balakrishnan (2007). Dynamic shared visual spaces: experimenting with automatic camera control in a remote repair task. In Proceedings of ACM CHI 2007 Conference on Human Factors in Computing Systems. New York: ACM Press, pp. 1177–1186. doi:10.1145/1240624.1240802

Raskar, Ramesh, Michael S. Brown, Ruigang Yang, Wei-Chao Chen, Greg Welch, Herman Towles, Brent Scales, and Henry Fuchs (1999). Multi-projector displays using camera-based registration. In Visualization′99. Proceedings. Washington DC: IEEE Computer Society, pp. 161–522.

Sheridan, Thomas B. (2002). Humans and Automation: System Design and Research Issues. New York: John Wiley & Sons.

Sodhi, Rajinder, Hrvoje Benko, and Andrew Wilson (2012). Lightguide: projected visualizations for hand movement guidance. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems. New York: ACM Press, pp. 179–188. doi:10.1145/2207676.2207702

Tang, Anthony, Carman Neustaedter, and Saul Greenberg (2006). VideoArms: Embodiments for Mixed Presence Groupware. In People and Computers XX - Engage. London: Springer, pp. 85–102. doi:10.1007/978-1-84628-664-3_8

Tecchia, Franco, Leila Alem, and Weidong Huang (2012). 3D helping hands: a gesture based MR system for remote collaboration. Proceedings of the 11th ACM SIGGRAPH International Conference on Virtual-Reality Continuum and Its Applications in Industry - VRCAI′12. New York: ACM Press, pp. 323–328. doi:10.1145/2407516.2407590

Tsai, Roger Y., and Reimar K. Lenz. (1988). Real time versatile robotics hand/eye calibration using 3D machine vision. In Robotics and Automation, 1988 Proceedings., IEEE International Conference on. Washington DC: IEEE Computer Society, pp. 554–561.

Wellner, Pierre (1993). Interacting with paper on the DigitalDesk. Communications of the ACM. vol. 36, no. 7, pp. 87–96. doi:10.1145/159544.159630 CrossRef

Woods, David D., James Tittle, Magnus Feil, and Axel Roesler (2004). Envisioning human-robot coordination in future operations. IEEE Transactions on Systems, Man and Cybernetics Part C: Applications and Reviews, vol. 34, no. 2, pp. 210–218. doi:10.1109/TSMCC.2004.826272 CrossRef

Zhang, Zhengyou, Rachid Deriche, Olivier Faugeras, and Quang-Tuan Luong (1995). A robust technique for matching two uncalibrated images through the recovery of the unknown epipolar geometry. Artificial intelligence, vol. 78, no 1, pp. 87–119.CrossRef

Titel: Design and Implementation of TeleAdvisor: a Projection-Based Augmented Reality System for Remote Collaboration
verfasst von: Pavel Gurevich
Joel Lanir
Benjamin Cohen
Publikationsdatum: 01.12.2015
Verlag: Springer Netherlands
Erschienen in: Computer Supported Cooperative Work (CSCW) / Ausgabe 6/2015
Print ISSN: 0925-9724
Elektronische ISSN: 1573-7551
DOI: https://doi.org/10.1007/s10606-015-9232-7

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Weitere Artikel der Ausgabe 6/2015

Using Projected Light for Mobile Remote Guidance

Providing Information on the Spot: Using Augmented Reality for Situational Awareness in the Security Domain

The Effect of View Independence in a Collaborative AR System

Collaboration in Augmented Reality

Premium Partner