Steven J. Henderson
Steven Feiner
ABSTRACT
We present Opportunistic Controls, a class of user interaction techniques for augmented reality (AR) applications that support gesturing on, and receiving feedback from, otherwise unused affordances already present in the domain environment. Opportunistic Controls leverage characteristics of these affordances to provide passive haptics that ease gesture input, simplify gesture recognition, and provide tangible feedback to the user. 3D widgets are tightly coupled with affordances to provide visual feedback and hints about the functionality of the control. For example, a set of buttons is mapped to existing tactile features on domain objects. We describe examples of Opportunistic Controls that we have designed and implemented using optical marker tracking, combined with appearance-based gesture recognition. We present the results of a user study in which participants performed a simulated maintenance inspection of an aircraft engine using a set of virtual buttons implemented both as Opportunistic Controls and using simpler passive haptics. Opportunistic Controls allowed participants to complete their tasks significantly faster and were preferred over the baseline technique.
Supplemental Material
- Da Pam 738--751, 1992. Functional Users Manual for The Army Maintenance Management System - Aviation (TAMMS-A). Washington D.C: U.S. Army.Google Scholar
- Blaskó, G. and Feiner, S. 2004. An interaction system for watch computers using tactile guidance and bidirectional segmented strokes. Proc. ISWC 2004. 120--123. Google ScholarDigital Library
- Bleser, G. and Stricker, D. 2008. Advanced tracking through efficient image processing and visual-inertial sensor fusion. Proc. IEEE Virtual Reality Conf. (VR '08), 137--144.Google Scholar
- Brooks, F. P., Ouh-Young, M., Batter, J. J. and Kilpatrick, P. J. 1990. Project GROPE-Haptic displays for scientific visualization. Proc. 17th Annual Conf. on Comp. Graphics and Interactive Techniques, 177--185. Google ScholarDigital Library
- Buxton, W., Hill, R. and Rowley, P. 1985. Issues and techniques in touch-sensitive tablet input, Proc. 12th Annual Conf. on Comp. Graphics and Interactive Techniques, 215--224. Google ScholarDigital Library
- Conner, B. D., Snibbe, S. S., Herndon, K. P., Robbins, D. C., Zeleznik, R. C. and Dam, A. V. 1992. Three-Dimensional Widgets. Proc. 1992 Symp. on Interactive 3D Graphics, 183--188. Google ScholarDigital Library
- Fiala, M. L. 2005. ARTag, a fiducial marker system using digital techniques. Proc. 2005 IEEE Computer Society Conf. on Comp. Vision and Pattern Recognition (CVPR'05) - Volume 2, 590--596. Google ScholarDigital Library
- Fishkin, K. P. 2004. A taxonomy for and analysis of tangible interfaces, Personal Ubiquitous Computing 8, 5, 347--358. Google ScholarDigital Library
- Gibson, J. 1986. The Ecological Approach to Visual Perception. Hillsdale, N.J.: Lawrence Erlbaum Associates.Google Scholar
- Hinckley, K., Pausch, R., Goble, J. C. and Kassell, N. F. 1994. Passive real-world interface props for neurosurgical visualization. Proc. SIGCHI Conf. on Human Factors in Computing Systems, 452--458. Google ScholarDigital Library
- Insko, B. E., Meehan, M. J., Whitton, M. C. and Frederick P. Brooks, J. 2001. Passive haptics significantly enhances virtual environments. Technical Report 0-493-17286-6. The University of North Carolina at Chapel Hill. Google ScholarDigital Library
- Ishii, H. and Ullmer, B. 1997. Tangible Bits: Towards seamless interfaces between people, bits and atoms. Proc. SIGCHI Conf. on Human Factors in Comp. Sys. 234--241. Google ScholarDigital Library
- Kjeldsen, R. and Kender, J. 1996. Finding skin in color images. Proc. 2nd International Conf. on Automatic Face and Gesture Recognition (FG '96), 312. Google ScholarDigital Library
- Klein, G. and Murray, D. 2007. Parallel tracking and mapping for small AR workspaces. Proc. International Symp. on Mixed and Augmented Reality (ISMAR'07). Google ScholarDigital Library
- Lindeman, R. W., Sibert, J. L. and Hahn, J. K. 1999. Hand-held windows: towards effective 2D interaction in immersive virtual environments. Proc. IEEE Virtual Reality Conference, 205--212. Google ScholarDigital Library
- Murray-Smith, R., Williamson, J., Hughes, S. and Quaade, T. 2008. Stane: Synthesized surfaces for tactile input. Proc. of the Twenty-sixth annual SIGCHI Conf. on Human Factors in Comp. Systems, 1299--1302. Google ScholarDigital Library
- Norman, D. 1988. The Psychology of Everyday Things. New York: Basic Books.Google Scholar
- Roeber, H., Bacus, J. and Tomasi, C. 2003. Typing in thin air: The Canesta projection keyboard - a new method of interaction with electronic devices. CHI '03 Extended Abstracts on Human Factors in Computing Systems, 712--713. Google ScholarDigital Library
- 3DV Systems, 2008. http://www.3dvsystems.comGoogle Scholar
- Szalavari, Z. and Gervautz, M. 1997. The personal interaction panel - a two-handed interface for augmented reality Computer Graphics Forum 16, 3, 335--346.Google Scholar
- Tomasi, C., Rafii, A. and Torunoglu, I. 2003. Full-size projection keyboard for handheld devices, Communications of the ACM 46, 7, 70--75. Google ScholarDigital Library
- Weimer, D. and Ganapathy, S. K. 1989. A synthetic visual environment with hand gesturing and voice input. Proc. SIGCHI Conf. on Human Factors in Comp. Systems, 235--240. Google ScholarDigital Library
Index Terms
- Opportunistic controls: leveraging natural affordances as tangible user interfaces for augmented reality
Recommendations
Head-mounted display with mid-air tactile feedback
VRST '15: Proceedings of the 21st ACM Symposium on Virtual Reality Software and TechnologyVirtual and physical worlds are merging. Currently users of head-mounted displays cannot have unobtrusive tactile feedback while touching virtual objects. We present a mid-air tactile feedback system for head-mounted displays. Our prototype uses the ...
3D Tabletop AR: A Comparison of Mid-Air, Touch and Touch+Mid-Air Interaction
AVI '20: Proceedings of the International Conference on Advanced Visual InterfacesThis paper contributes a first comparative study of three techniques for selecting 3D objects anchored to the table in tabletop Augmented Reality (AR). The impetus for this study is that touch interaction makes more sense when the targeted objects are ...
Flyables: Exploring 3D Interaction Spaces for Levitating Tangibles
TEI '18: Proceedings of the Twelfth International Conference on Tangible, Embedded, and Embodied InteractionRecent advances in technology and miniaturization allow the building of self-levitating tangible interfaces. This includes flying tangibles, which extend the mid-air interaction space from 2D to 3D. While a number of theoretical concepts about ...
Comments