Steffen Gauglitz
ABSTRACT
We describe a framework and prototype implementation for unobtrusive mobile remote collaboration on tasks that involve the physical environment. Our system uses the Augmented Reality paradigm and model-free, markerless visual tracking to facilitate decoupled, live updated views of the environment and world-stabilized annotations while supporting a moving camera and unknown, unprepared environments. In order to evaluate our concept and prototype, we conducted a user study with 48 participants in which a remote expert instructed a local user to operate a mock-up airplane cockpit. Users performed significantly better with our prototype (40.8 tasks completed on average) as well as with static annotations (37.3) than without annotations (28.9). 79% of the users preferred our prototype despite noticeably imperfect tracking.
Supplemental Material
- Alem, L., Tecchia, F., and Huang, W. HandsOnVideo: Towards a gesture based mobile AR system for remote collaboration. In Recent Trends of Mobile Collaborative Augmented Reality Systems, L. Alem and W. Huang, Eds. Springer New York, 2011, 135--148.Google Scholar
Cross Ref
- Baudisch, P., and Rosenholtz, R. Halo: a technique for visualizing off-screen objects. In Proc. ACM SIGCHI Conf. on Human Factors in Computing Systems (CHI), 2003, 481--488. Google ScholarDigital Library
- Bauer, M., Kortuem, G., and Segall, Z. "Where are you pointing at?" a study of remote collaboration in a wearable videoconference system. In Proc. Intl. Symp. on Wearable Computers (ISWC), 1999, 151--158. Google ScholarDigital Library
- Billinghurst, M., Bowskill, J., Jessop, M., and Morphett, J. A wearable spatial conferencing space. In Proc. Intl. Symp. on Wearable Computers (ISWC), 1998, 76--83. Google ScholarDigital Library
- Billinghurst, M., Weghorst, S., and Furness, T. Shared space: An augmented reality approach for computer supported collaborative work. Virtual Reality 3 (1998), 25--36.Google ScholarDigital Library
- Butz, A., Höllerer, T., Feiner, S., MacIntyre, B., and Beshers, C. Enveloping users and computers in a collaborative 3D augmented reality. In Proc. IEEE/ACM Intl. Workshop on Augmented Reality, 1999, 35--44. Google ScholarDigital Library
- Chastine, J., Nagel, K., Zhu, Y., and Hudachek-Buswell, M. Studies on the effectiveness of virtual pointers in collaborative augmented reality. In Proc. IEEE Symp. on 3D User Interfaces (3DUI), 2008, 117--124. Google ScholarDigital Library
- Fussell, S. R., Setlock, L. D., Yang, J., Ou, J., Mauer, E., and Kramer, A. D. I. Gestures over video streams to support remote collaboration on physical tasks. Hum.-Comput. Interact. 19 (Sept. 2004), 273--309. Google ScholarDigital Library
- Gauglitz, S., Foschini, L., Turk, M., and Höllerer, T. Efficiently selecting spatially distributed keypoints for visual tracking. In Proc. IEEE Intl. Conf. on Image Processing (ICIP), 2011.Google ScholarCross Ref
- Gergle, D., Kraut, R., and Fussell, S. Action as language in a shared visual space. In Proc. ACM Conf. on Computer Supported Cooperative Work, 2004, 487--496. Google ScholarDigital Library
- Gurevich, P., Lanir, J., Cohen, B., and Stone, R. TeleAdvisor: a versatile augmented reality tool for remote assistance. In Proc. ACM SIGCHI Conf. on Human Factors in Computing Systems, 2012, 619--622. Google ScholarDigital Library
- Gustafson, S., Baudisch, P., Gutwin, C., and Irani, P. Wedge: clutter-free visualization of off-screen locations. In Proc. ACM SIGCHI Conf. on Human Factors in Computing Systems, 2008, 787--796. Google ScholarDigital Library
- Gutwin, C., and Penner, R. Improving interpretation of remote gestures with telepointer traces. In Proc. ACM Conf. on Computer Supported Cooperative Work, 2002, 49--57. Google ScholarDigital Library
- Güven, S., Feiner, S., and Oda, O. Mobile augmented reality interaction techniques for authoring situated media on-site. In Proc. IEEE/ACM Intl. Symp. on Mixed and Augmented Reality (ISMAR), 2006, 235--236. Google ScholarDigital Library
- Hayne, S., Pendergast, M., and Greenberg, S. Implementing gesturing with cursors in group support systems. J. Manag. Informat. Syst. (1993), 43--61. Google ScholarDigital Library
- Hindmarsh, J., Fraser, M., Heath, C., Benford, S., and Greenhalgh, C. Fragmented interaction: establishing mutual orientation in virtual environments. In Proc. ACM Conf. on Computer Supported Cooperative Work, 1998, 217--226. Google ScholarDigital Library
- Kirk, D., and Stanton Fraser, D. Comparing remote gesture technologies for supporting collaborative physical tasks. In Proc. SIGCHI Conf. on Human Factors in Computing Systems (CHI), 2006, 1191--1200. Google ScholarDigital Library
- Kirk, D. S. Turn It This Way: Remote Gesturing in Video-Mediated Communication. PhD thesis, University of Nottingham, 2006.Google Scholar
- Klein, G., and Murray, D. Improving the agility of keyframe-based SLAM. In Proc. European Conf. on Computer Vision (ECCV), 2008, 802--815. Google ScholarDigital Library
- Kraut, R., Miller, M., and Siegel, J. Collaboration in performance of physical tasks: Effects on outcomes and communication. In Proc. ACM Conf. on Computer Supported Cooperative Work, 1996, 57--66. Google ScholarDigital Library
- Kurata, T., Sakata, N., Kourogi, M., Kuzuoka, H., and Billinghurst, M. Remote collaboration using a shoulder-worn active camera/laser. In Proc. Intl. Symp. on Wearable Computers (ISWC), 2004, 62--69. Google ScholarDigital Library
- Kurillo, G., Bajcsy, R., Nahrsted, K., and Kreylos, O. Immersive 3D environment for remote collaboration and training of physical activities. In Proc. IEEE Virtual Reality, 2008, 269--270.Google Scholar
- Kuzuoka, H., Oyama, S., Yamazaki, K., Suzuki, K., and Mitsuishi, M. Gestureman: a mobile robot that embodies a remote instructor's actions. In Proc. ACM Conf. on Computer Supported Cooperative Work, 2000, 155--162. Google ScholarDigital Library
- Ladikos, A., Benhimane, S., Appel, M., and Navab, N. Model-free markerless tracking for remote support in unknown environments. In Proc. Intl. Conf. on Computer Vision Theory and Applications, 2008.Google Scholar
- Lee, T., and Höllerer, T. Viewpoint stabilization for live collaborative video augmentations. In Proc. IEEE/ACM Intl. Symp. on Mixed and Augmented Reality, 2006, 241--242. Google ScholarDigital Library
- Mortensen, J., Vinayagamoorthy, V., Slater, M., Steed, A., Lok, B., and Whitton, M. Collaboration in tele-immersive environments. In Proc. Workshop on Virtual environments, 2002, 93--101. Google ScholarDigital Library
- Newcombe, R., Lovegrove, S., and Davison, A. Dtam: Dense tracking and mapping in real-time. In Proc. IEEE Intl. Conf. on Computer Vision, 2011, 2320--2327. Google ScholarDigital Library
- Newcombe, R. A., Izadi, S., Hilliges, O., Molyneaux, D., Kim, D., Davison, A. J., Kohli, P., Shotton, J., Hodges, S., and Fitzgibbon, A. KinectFusion: Real-time dense surface mapping and tracking. In Proc. IEEE Intl. Symp. on Mixed and Augmented Reality, 2011. Google ScholarDigital Library
- Ou, J., Fussell, S. R., Chen, X., Setlock, L. D., and Yang, J. Gestural communication over video stream: supporting multimodal interaction for remote collaborative physical tasks. In Proc. Intl. Conf. on Multimodal Interfaces (ICMI), 2003, 242--249. Google ScholarDigital Library
- Regenbrecht, H., Lum, T., Kohler, P., Ott, C., Wagner, M., Wilke, W., and Mueller, E. Using augmented virtuality for remote collaboration. Presence: Teleoper. Virtual Environ. 13 (July 2004), 338--354. Google ScholarDigital Library
- Reitmayr, G., and Schmalstieg, D. Mobile collaborative augmented reality. In Proc. IEEE/ACM Intl. Symp. on Augmented Reality, 2001, 114--123. Google ScholarDigital Library
- Rosten, E., and Drummond, T. Machine learning for high-speed corner detection. In Proc. IEEE European Conf. on Computer Vision, 2006, 430--443. Google ScholarDigital Library
- Sadagic, A., Towles, H., Holden, L., Daniilidis, K., and Zeleznik, B. Tele-immersion portal: Towards an ultimate systhesis of Computer Graphics and Computer Vision Systems. In Proc. Intl. Workshop on Presence, 2001.Google Scholar
- Tang, J. C., and Minneman, S. L. VideoDraw: a video interface for collaborative drawing. ACM Trans. Inf. Syst. 9 (April 1991), 170--184. Google ScholarDigital Library
- Wagner, D., Mulloni, A., Langlotz, T., and Schmalstieg, D. Real-time panoramic mapping and tracking on mobile phones. In Proc. IEEE Virtual Reality, 2010. Google ScholarDigital Library
- Wagner, D., Schmalstieg, D., and Bischof, H. Multiple target detection and tracking with guaranteed framerates on mobile phones. In Proc. IEEE Intl. Symp. on Mixed and Augmented Reality (ISMAR), 2009, 57--64. Google ScholarDigital Library
- Wellner, P., and Freeman, S. The DoubleDigitalDesk: Shared editing of paper documents. Tech. Rep. EPC-93-108, EuroPARC, 1993.Google Scholar
Index Terms
- Integrating the physical environment into mobile remote collaboration
Recommendations
World-stabilized annotations and virtual scene navigation for remote collaboration
UIST '14: Proceedings of the 27th annual ACM symposium on User interface software and technologyWe present a system that supports an augmented shared visual space for live mobile remote collaboration on physical tasks. The remote user can explore the scene independently of the local user's current camera position and can communicate via spatial ...
In touch with the remote world: remote collaboration with augmented reality drawings and virtual navigation
VRST '14: Proceedings of the 20th ACM Symposium on Virtual Reality Software and TechnologyAugmented reality annotations and virtual scene navigation add new dimensions to remote collaboration. In this paper, we present a touchscreen interface for creating freehand drawings as world-stabilized annotations and for virtually navigating a scene ...
A prototype setup to integrate the physical environment into mobile remote collaboration
MobileHCI '12: Proceedings of the 14th international conference on Human-computer interaction with mobile devices and services companionIn the accompanying paper [1], we describe a framework and prototype implementation for unobtrusive mobile remote collaboration on tasks that involve the physical environment. Our system uses model-free, markerless visual tracking to facilitate ...
Comments