Omid Fakourfar
ABSTRACT
Recent mobile technology has provided new opportunities for creating remote assistance systems. However, mobile support systems present a particular challenge: both the camera and display are held by the user, leading to shaky video. When pointing or drawing annotations, this means that the desired target often moves, causing the gesture to lose its intended meaning. To address this problem, we investigate annotation stabilization techniques, which allow annotations to stick to their intended location. We studied two annotation systems, using three different forms of annotations, with both tablets and head-mounted displays. Our analysis suggests that stabilized annotations and head-mounted displays are only beneficial in certain situations. However, the simplest approach of automatically freezing video while drawing annotations was surprisingly effective in facilitating the completion of remote assistance tasks.
Supplemental Material
- Martin Bauer, Gerd Kortuem, and Zary Segall. 1999. "Where Are You Pointing At"? A Study of Remote Collaboration in a Wearable Videoconference System. In Proceedings of the 3rd IEEE International Symposium on Wearable Computers (ISWC '99). IEEE Computer Society, Washington, DC, USA, 151--161. Google Scholar
Digital Library
- Eric A. Bier, Maureen C. Stone, Ken Pier, William Buxton, and Tony D. DeRose. 1993. Toolglass and magic lenses. Proceedings of the 20th annual conference on Computer graphics and interactive techniques SIGGRAPH '93, ACM Press, 73--80. http://doi.org/10.1145/166117.166126 Google ScholarDigital Library
- Mark Billinghurst, Adrian Clark, and Gun Lee. 2015. A Survey of Augmented Reality. Foundations and Trends® in Human-Computer Interaction 8, 2--3: 73--272. http://doi.org/10.1561/1100000049 Google ScholarDigital Library
- Sara A. Bly, Steve R. Harrison, and Susan Irwin. 1993. Media spaces: bringing people together in a video, audio, and computing environment. Communications of the ACM 36, 1: 28--46. http://doi.org/10.1145/151233.151235 Google ScholarDigital Library
- Jed R. Brubaker, Gina Venolia, and John C. Tang. 2012. Focusing on shared experiences. Proceedings of the Designing Interactive Systems Conference on DIS '12, ACM Press, 96. http://doi.org/10.1145/2317956.2317973 Google ScholarDigital Library
- Bill Buxton. 2009. Mediaspace -- Meaningspace -- Meetingspace. In Media Space: 20+ Years of Mediated Life, S. Harrison (Ed.). Springer, London, UK, 217--231.Google Scholar
- Herbert H. Clark. 1996. Using Language. Cambridge University Press.Google Scholar
- Veronika Domova, Elina Vartiainen, and Marcus Englund. 2014. Designing a Remote Video Collaboration System for Industrial Settings. In Proceedings of the Ninth ACM International Conference on Interactive Tabletops and Surfaces (ITS '14). ACM, NY, NY, USA, 229--238. http://doi.acm.org/10.1145/2669485.2669517 Google ScholarDigital Library
- Susan R. Fussell, Leslie D. Setlock, Jie Yang, Jiazhi Ou, Elizabeth Mauer, and Adam D. I. Kramer. 2004. Gestures over video streams to support remote collaboration on physical tasks. Hum.-Comput. Interact. 19, 3 (September 2004), 273--309. http://dx.doi.org/10.1207/s15327051hci1903_3 Google ScholarDigital Library
- Susan R. Fussell, Leslie D. Setlock, and Robert E. Kraut. 2003. Effects of head-mounted and sceneoriented video systems on remote collaboration on physical tasks. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '03). ACM, NY, NY, USA, 513--520. http://doi.acm.org/10.1145/642611.642701 Google ScholarDigital Library
- Steffen Gauglitz, Cha Lee, Matthew Turk, and Tobias Höllerer. 2012. Integrating the physical environment into mobile remote collaboration. In Proceedings of the 14th international conference on Human-computer interaction with mobile devices and services (MobileHCI '12). ACM, NY, NY, USA, 241250. http://doi.acm.org/10.1145/2371574.2371610 Google ScholarDigital Library
- Steffen Gauglitz, Benjamin Nuernberger, Matthew Turk, and Tobias Höllerer. 2014. In touch with the remote world: remote collaboration with augmented reality drawings and virtual navigation. InProceedings of the 20th ACM Symposium on Virtual Reality Software and Technology (VRST '14). ACM, NY, NY, USA, 197--205. http://dx.doi.org/10.1145/2671015.2671016 Google ScholarDigital Library
- Steffen Gauglitz, Benjamin Nuernberger, Matthew Turk, and Tobias Höllerer. 2014. World-stabilized annotations and virtual scene navigation for remote collaboration. In Proceedings of the 27th annual ACM symposium on User interface software and technology (UIST '14). ACM, NY, NY, USA, 449--459. http://doi.acm.org/10.1145/2642918.2647372 Google ScholarDigital Library
- Darren Gergle, Robert E. Kraut, and Susan R. Fussell. 2004. Action as language in a shared visual space. Proceedings of the 2004 ACM conference on Computer supported cooperative work CSCW '04, ACM Press, 487. http://doi.org/10.1145/1031607.1031687 Google ScholarDigital Library
- Pavel Gurevich, Joel Lanir, Benjamin Cohen, and Ran Stone. 2012. TeleAdvisor: a versatile augmented reality tool for remote assistance. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '12). ACM, NY, NY, USA, 619--622. http://doi.acm.org/10.1145/2207676.2207763 Google ScholarDigital Library
- Carl Gutwin, Mark Roseman, and Saul Greenberg. 1996. A usability study of awareness widgets in a shared workspace groupware system. Proceedings of the 1996 ACM conference on Computer supported cooperative work CSCW '96, ACM Press, 258--267. http://doi.org/10.1145/240080.240298 Google ScholarDigital Library
- Weidong Huang and Leila Alem. 2013. HandsinAir: a wearable system for remote collaboration on physical tasks. In Proceedings of the 2013 conference on Computer supported cooperative work companion (CSCW '13). ACM, NY, NY, USA, 153--156. http://doi.acm.org/10.1145/2441955.2441994 Google ScholarDigital Library
- Hiroshi Ishii, Minoru Kobayashi, and Jonathan Grudin. 1993. Integration of interpersonal space and shared workspace: ClearBoard design and experiments. ACM Transactions on Information Systems 11, 4: 349--375. http://doi.org/10.1145/159764.159762 Google ScholarDigital Library
- Shahram Izadi, Ankur Agarwal, Antonio Criminisi, John M. Winn, Andrew Blake and Andrew W. Fitzgibbon. 2007. C-Slate: A Multi-Touch and Object Recognition System for Remote Collaboration using Horizo. In Second IEEE International Workshop on Horizontal Interactive Human-Computer Systems Tabletop 2007 October 10--12, 2007, Newport, Rhode Island, USA. pp. 3--10. http://doi.ieeecomputersociety.org/10.1109/TABLETO P.2007.5Google Scholar
- Hyungeun Jo and Sungjae Hwang. 2013. Chili: viewpoint control and on-video drawing for mobile video calls. In CHI '13 Extended Abstracts on Human Factors in Computing Systems (CHI EA '13). ACM, NY, NY, USA, 1425--1430. http://doi.acm.org/10.1145/2468356.2468610 Google ScholarDigital Library
- Steven Johnson, Madeleine Gibson, and Bilge Mutlu. 2015. Handheld or Handsfree?: Remote Collaboration via Lightweight Head-Mounted Displays and Handheld Devices. In Proceedings of the 18th ACM Conference on Computer Supported Cooperative Work & Social Computing (CSCW '15). ACM, NY, NY, USA, 1825--1836. http://doi.acm.org/10.1145/2675133.2675176 Google ScholarDigital Library
- Brennan Jones, Anna Witcraft, Scott Bateman, Carman Neustaedter, and Anthony Tang. 2015. Mechanics of Camera Work in Mobile Video Collaboration. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, NY, NY, USA, 957--966. http://doi.acm.org/10.1145/2702123.2702345 Google ScholarDigital Library
- Brigitte Jordan and Austin Henderson. 1995. Interaction Analysis: Foundations and Practice. The Journal of the Learning Sciences, 39--103.Google ScholarCross Ref
- Shunichi Kasahara and Jun Rekimoto. 2014. JackIn: integrating first-person view with out-of-body vision generation for human-human augmentation. In Proceedings of the 5th Augmented Human International Conference (AH '14). ACM, NY, NY, USA, Article 46, 8 pages. http://doi.acm.org/10.1145/2582051.2582097 Google ScholarDigital Library
- Seungwon Kim, Gun A. Lee, Sangtae Ha, Nobuchika Sakata and Mark Billinghurst. 2015. Automatically Freezing Live Video for Annotation during Remote Collaboration. In CHI '15 Extended Abstracts on Human Factors in Computing Systems (CHI EA '15). ACM, NY, NY, USA, 1669--1674. http://dx.doi.org/10.1145/2702613.2732838 Google ScholarDigital Library
- David Kirk, Tom Rodden, and Danaë Stanton Fraser. 2007. Turn it this way. Proceedings of the SIGCHI conference on Human factors in computing systems CHI '07, ACM Press, 1039. http://doi.org/10.1145/1240624.1240782Google ScholarDigital Library
- David Kirk and Danae Stanton Fraser. 2006. Comparing remote gesture technologies for supporting collaborative physical tasks. Proceedings of the SIGCHI conference on Human Factors in computing systems CHI '06, ACM Press, 1191. http://doi.org/10.1145/1124772.1124951 Google ScholarDigital Library
- Hideaki Kuzuoka, Shinya Oyama, Keiichi Yamazaki, Kenji Suzuki, and Mamoru Mitsuishi. 2000. GestureMan. Proceedings of the 2000 ACM conference on Computer supported cooperative work CSCW '00, ACM Press, 155--162. http://doi.org/10.1145/358916.358986 Google ScholarDigital Library
- Joel Lanir, Ran Stone, Benjamin Cohen, and Pavel Gurevich. 2013. Ownership and control of point of view in remote assistance. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13). ACM, NY, NY, USA, 22432252. http://doi.acm.org/10.1145/2470654.2481309 Google ScholarDigital Library
- Doo Young Lee and Mark R. Lehto. 2013. User acceptance of youtube for procedural learning: an extension of the technology acceptance model. Computers & Education 61: 193--208. Google ScholarDigital Library
- Paul Luff, Christian Heath, Hideaki Kuzuoka, Jon Hindmarsh, Keiichi Yamazaki, and Shinya Oyama. 2003. Fractured Ecologies: Creating Environments for Collaboration. Human-Computer Interaction 18, 1: 51--84. http://doi.org/10.1207/S15327051HCI1812_3 Google ScholarDigital Library
- Paul K. Luff, Naomi Yamashita, Hideaki Kuzuoka, and Christian Heath. 2015. Flexible Ecologies And Incongruent Locations. Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems CHI '15, ACM Press, 877--886. http://doi.org/10.1145/2702123.2702286 Google ScholarDigital Library
- James Norris, Holger M. Schnädelbach, and Paul K. Luff. 2013. Putting things in focus: establishing coorientation through video in context. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13). ACM, NY, NY, USA, 1329--1338. http://doi.acm.org/10.1145/2470654.2466174 Google ScholarDigital Library
- Jiazhi Ou, Xilin Chen, Susan R. Fussell, and Jie Yang. 2003. DOVE. Proceedings of the eleventh ACM international conference on Multimedia MULTIMEDIA '03, ACM Press, 100. http://doi.org/10.1145/957013.957034Google Scholar
- Abhishek Ranjan, Jeremy P. Birnholtz, and Ravin Balakrishnan. 2007. Dynamic shared visual spaces: experimenting with automatic camera control in a remote repair task. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '07). ACM, NY, NY, USA, 1177--1186. http://doi.acm.org/10.1145/1240624.1240802 Google ScholarDigital Library
- Kim Seungwon, Gun A. Lee, and Nobuchika Sakata. 2013. Comparing pointing and drawing for remote collaboration. In 2013 IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp.1--6, Oct. 2013. http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnum ber=6671833&isnumber=6671745Google ScholarCross Ref
- Kim Seungwon, Gun A. Lee, Nobuchika Sakata, Andreas Dunser, Elina Vartiainen, and Mark Billinghurst. 2013. Study of Augmented Gesture Communication Cues and View Sharing in Remote Collaboration. In IEEE International Symposium on Mixed and Augmented Reality (ISMAR), pp.261--262, Oct. 2013. http://ieeexplore.ieee.org/stamp/stamp.jsp?tp=&arnum ber=6671795&isnumber=6671745Google Scholar
- Rajinder S. Sodhi, Brett R. Jones, David Forsyth, Brian P. Bailey, and Giuliano Maciocci. 2013. BeThere: 3D mobile collaboration with spatial input. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI '13). ACM, NY, NY, USA, 179--188. http://doi.acm.org/10.1145/2470654.2470679 Google ScholarDigital Library
- Matthew Tait and Mark Billinghurst. 2015. The Effect of View Independence in a Collaborative AR System. Computer Supported Cooperative Work (CSCW). http://doi.org/10.1007/s10606-015-9231-8 Google ScholarDigital Library
- John C. Tang and Scott L. Minneman. 1991. Videodraw: a video interface for collaborative drawing. ACM Transactions on Information Systems 9, 2: 170--184. http://doi.org/10.1145/123078.128729 Google ScholarDigital Library
- Anthony Tang, Carman Newstaedter and Saul Greenberg. 2006. VideoArms: Embodiments for Mixed Presence Groupware. In Proceedings of the 20th British HCI Group Annual Conference (HCI 2006). London, UK, 85--102.Google Scholar
- Xianjun Sam Zheng, Cedric Foucault, Patrik Matos da Silva, Siddharth Dasari, Tao Yang, and Stuart Goose. 2015. Eye-Wearable Technology for Machine Maintenance: Effects of Display Position and Handsfree Operation. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (CHI '15). ACM, NY, NY, USA, 2125--2134. http://doi.acm.org/10.1145/2702123.2702305 Google ScholarDigital Library
Index Terms
- Stabilized Annotations for Mobile Remote Assistance
Recommendations
World-stabilized annotations and virtual scene navigation for remote collaboration
UIST '14: Proceedings of the 27th annual ACM symposium on User interface software and technologyWe present a system that supports an augmented shared visual space for live mobile remote collaboration on physical tasks. The remote user can explore the scene independently of the local user's current camera position and can communicate via spatial ...
Mini-Me: An Adaptive Avatar for Mixed Reality Remote Collaboration
CHI '18: Proceedings of the 2018 CHI Conference on Human Factors in Computing SystemsWe present Mini-Me, an adaptive avatar for enhancing Mixed Reality (MR) remote collaboration between a local Augmented Reality (AR) user and a remote Virtual Reality (VR) user. The Mini-Me avatar represents the VR user's gaze direction and body gestures ...
TeleAdvisor: a versatile augmented reality tool for remote assistance
CHI '12: Proceedings of the SIGCHI Conference on Human Factors in Computing SystemsTeleAdvisor is a novel solution designed to support remote assistance tasks in many real-world scenarios. It consists of a video camera and a small projector mounted at the end of a tele-operated robotic arm. This enables a remote helper to view and ...
Comments