Robert J.K. Jacob
Audrey Girouard
Leanne M. Hirshfield
Michael S. Horn
Orit Shaer
Erin Treacy Solovey
Jamie Zigelbaum
ABSTRACT
We are in the midst of an explosion of emerging human-computer interaction techniques that redefine our understanding of both computers and interaction. We propose the notion of Reality-Based Interaction (RBI) as a unifying concept that ties together a large subset of these emerging interaction styles. Based on this concept of RBI, we provide a framework that can be used to understand, compare, and relate current paths of recent HCI research as well as to analyze specific interaction designs. We believe that viewing interaction through the lens of RBI provides insights for design and uncovers gaps or opportunities for future research.
- Abowd, G.D. and Dix, A.J., Integrating Status and Event Phenomena in Formal Specifications of Interactive Systems. in Proc. SIGSOFT 1994, (1994), Addison-Wesley/ACM Press. Google Scholar
Digital Library
- Agarawala, A. and Balakrishnan, R. Keepin' it real: pushing the desktop metaphor with physics, piles and the pen Proc. CHI2006, ACM Press, 2006. Google ScholarDigital Library
- Ängeslevä, J., Oakley, I., Hughes, S. and O'Modhrain, S., Body Mnemonics Portable device interaction design concept. in UIST, (2003).Google Scholar
- Apted, T., Kay, J. and Quigley, A. Tabletop sharing of digital photographs for the elderly Proc. of SIGCHI, ACM Press, 2006. Google ScholarDigital Library
- Bae, S.-H., Kobayash, T., Kijima, R. and Kim, W.-S., Tangible NURBS-curve manipulation techniques using graspable handles on a large display. in Proc. UIST 2004, (2004), ACM Press, 81--90. Google ScholarDigital Library
- Beaudouin-Lafon, M. Instrumental Interaction: An Interaction Model for Designing Post-WIMP User Interfaces Proc. CHI 2000, Addison-Wesley/ACM Press, 2000, 446--453. Google ScholarDigital Library
- Beeharee, A. and Steed, A. Exploiting real world knowledge in ubiquitous applications. Personal and Ubiquitous Computing, 11. 429--437. Google ScholarDigital Library
- Bellotti, V., Back, M., Edwards, W.K., Grinter, R.E., Henderson, A. and Lopes, C. Making Sense of Sensing Systems: Five Questions for Designers and Researchers Proc. CHI 2002, ACM Press, 2002, 415--422. Google ScholarDigital Library
- Benford, S., Schnadelbach, H., Koleva, B., Anastasi, R., Greenhalgh, C., Rodden, T., Green, J., Ghali, A., Pridmore, T., Gaver, B., Boucher, A., Walker, B., Pennington, S., Schmidt, A., Gellersen, H. and Steed, A. Expected, sensed, and desired: A framework for designing sensing--based interaction. ACM Transactions Computer-Human Interaction, 12 (1). 3--30. Google ScholarDigital Library
- Bowman, D.A., Kruijff, E., LaViola, J.J. and Poupyrev, I. 3D User Interfaces: Theory and Practice. Addison-Wesley/Pearson Education, 2004. Google ScholarDigital Library
- Buur, J., Jensen, M.V. and Djajadiningrat, T., Please touch tangible UIs: Hands-only scenarios and video action walls: novel methods for tangible user interaction design. in Proc. DIS'04, (2004). Google ScholarDigital Library
- Buxton, B. http://www.billbuxton.com/multitouchOverview.html, 2007.Google Scholar
- Christou, G. Towards a new method for the evaluation of reality based interaction CHI '07 extended abstracts, ACM, San Jose, CA, USA, 2007. Google ScholarDigital Library
- Coutrix, C. and Nigay, L. Mixed reality: a model of mixed interaction. Proceedings of the working conference on Advanced visual interfaces. Google ScholarDigital Library
- Dickie, C., Vertegaal, R., Sohn, C. and Cheng, D. eyeLook: using attention to facilitate mobile media consumption Proc. UIST 2005, ACM Press, 2005. Google ScholarDigital Library
- Dourish, P. Where The Action Is: The Foundations of Embodied Interaction, MIT Press, Cambridge, Mass., 2001. Google ScholarCross Ref
- Dubois, E. and Gray, P. A Design-Oriented Information-Flow Refinement of the ASUR Interaction Model. Proceedings of EHCI 2007.Google Scholar
- Fishkin, K.P. A Taxonomy for and Analysis of Tangible Interfaces. Personal and Ubiquitous Computing, 8 (5). 347 -- 358. Google ScholarDigital Library
- Fishkin, K.P., Moran, T.P. and Harrison, B.L. Embodied User Interfaces: Toward Invisible User Interfaces Proc. of EHCI'98, 1998. Google ScholarDigital Library
- Forlines, C. and Shen, C., Touch: DTLens: multi-user tabletop spatial data exploration. in Proc. UIST 2005, (2005). Google ScholarDigital Library
- Hayes, P.J. The second naive physics manifesto Cognitive Science Technical Report URCS-10, University of Rochester, 1983.Google Scholar
- Hornecker, E. and Buur, J., Getting a Grip on Tangible Interaction: A Framework on Physical Space and Social Interaction. in Proc. of CHI 2006, (2006), ACM, 437--446. Google ScholarDigital Library
- Hurtienne, J. and Israel, J.H. Image schemas and their metaphorical extensions: intuitive patterns for tangible interaction Proc. TEI 2007, ACM Press, 2007. Google ScholarDigital Library
- Hutchins, E.L., Hollan, J.D. and Norman, D.A. Direct Manipulation Interfaces. Draper, D.A.N.a.S.W. ed. User Centered System Design: New Perspectives on Human-computer Interaction, Lawrence Erlbaum, Hillsdale, N.J., 1986, 87--124.Google Scholar
- Jackson, R.L. and Fagan, E., Collaboration and learning within immersive virtual reality. in Proc. Collaborative Virtual Environments, (2000), ACM Press, 83 -- 92. Google ScholarDigital Library
- Jacob, R.J.K., Girouard, A., Hirshfield, L.M., Horn, M.S., Shaer, O., Solovey, E.T. and Zigelbaum, J., Reality-Based Interaction: Unifying the New Generation of Interaction Styles (Work in Progress paper). in CHI 2007, (2007), 2465--2470. Google ScholarDigital Library
- Jacob, R.J.K., Girouard, A., Horn, M., Miller, L., Shaer, O., Treacy, E. and Zigelbaum, J. What Is the Next Generation of Human--Computer Interaction? interactions, 2007. Google ScholarDigital Library
- Kalanithi, J. Connectibles: Tangible Social Networking, MS Thesis, MIT Media Lab, Cambridge, 2007, 132pp.Google Scholar
- Klemmer, S.R., Hartmann, B. and Takayama, L., How bodies matter: five themes for interaction design. in Proc. DIS 2006, (2006), ACM Press, 140--149. Google ScholarDigital Library
- MacLean, A., Young, R.M., Bellotti, V.M.E. and Moran, T.P. Questions, Options, and Criteria: Elements of Design Space Analysis. Human-Computer Interaction, 6. 201--250. Google ScholarDigital Library
- Maeda, J. The Laws of Simplicity (Simplicity: Design, Technology, Business, Life). The MIT Press, 2006. Google ScholarDigital Library
- Merrill, D. and Maes, P., Augmenting Looking, Pointing and Reaching Gestures to Enhance the Searching and Browsing of Physical Objects. in Proc. Pervasive'07, (2007). Google ScholarDigital Library
- Milgram, P. and Kishino, F., Augmented reality: A class of displays on the reality-virtuality continuum. in SPIE, (1994), 282--292.Google Scholar
- Mohler, B.J., Thompson, W.B., Creem-Regehr, S.H., Willemsen, P., Herbert L. Pick, Jr. and Rieser, J.J. Calibration of locomotion resulting from visual motion in a treadmill-based virtual environment. ACM Trans. Appl. Percept., 4 (1). 4. Google ScholarDigital Library
- Nielsen, J. Noncommand User Interfaces Comm. ACM, 1993, 83--99. Google ScholarDigital Library
- Poupyrev, I., Newton--Dunn, H. and Bau, O. D20: interaction with multifaceted display devices CHI '06 extended abstracts, ACM Press, 2006. Google ScholarDigital Library
- Rohrer, T. Metaphors We Compute By: Bringing Magic into Interface Design, Center for the Cognitive Science of Metaphor, Philosophy Department, University of Oregon, 1995.Google Scholar
- Second Life. http://secondlife.com/.Google Scholar
- Shneiderman, B. Direct Manipulation. A Step Beyond Programming Languages. IEEE Transactions on Computers, 16 (8). 57. Google ScholarDigital Library
- Shneiderman, B. Leonardo's Laptop: Human Needs and the New Computing Technologies, MIT Press, Cambridge, Mass., 2002. Google ScholarDigital Library
- Shneiderman, B. Why Not Make Interfaces Better Than 3D Reality? IEEE Computer Graphics and Applications, 2003, 12--15. Google ScholarDigital Library
- Slater, M., Usoh, M. and Steed, A. Taking steps: the influence of a walking technique on presence in virtual reality. ACM Trans. Comput.-Hum. Interact., 2 (3). 201--219. Google ScholarDigital Library
- Smith, J.D., Vertegaal, R. and Sohn, C., ViewPointer: lightweight calibration-free eye tracking for ubiquitous handsfree deixis. in Proc. UIST 2005, (2005). Google ScholarDigital Library
- Streitz, N.A., Tandler, P., Muller-Tomfelde, C. and Konomi, S. Roomware: Toward the Next Generation of Human-Computer Interaction Based on an Integrated Design of Real and Virtual Worlds. Carroll, J.M. ed. Human-Computer Interaction in the New Millenium, Addison-Wesley/ACM Press, Reading, Mass., 2001.Google Scholar
- Tanriverdi, V. A Virtual Reality Interface Design (VRID) Model and Methodology, Tufts University, 2001.Google Scholar
- Tanriverdi, V. and Jacob, R.J.K., Interacting with Eye Movements in Virtual Environments. in Proc. CHI 2000, (2000), Addison-Wesley/ACM Press, 265--272. Google ScholarDigital Library
- Ullmer, B. and Ishii, H. Emerging Frameworks for Tangible User Interfaces. Carroll, J.M. ed. Human-Computer Interaction in the New Millenium, Addison-Wesley/ACM Press, Reading, Mass., 2001.Google Scholar
- Ullmer, B., Ishii, H. and Jacob, R.J.K. Token+Constraint Systems for Tangible Interaction with Digital Information ACM TOCHI, 2005, 81--118. Google ScholarDigital Library
- Underkoffler, J. and Ishii, H. Urp: A Luminous-Tangible Workbench for Urban Planning and Design Proc. CHI'99, Addison-Wesley/ACM Press, 1999, 386--393. Google ScholarDigital Library
- Usoh, M., Arthur, K., Whitton, M.C., Bastos, R., Steed, A., Slater, M. and Brooks, F.P. Walking > walking-in-place > flying, in virtual environments Proc. SIGGRAPH'99, Addison-Wesley/ACM Press, 1999, 359--364. Google ScholarDigital Library
- Van Dam, A. Post-WIMP user interfaces. Commun. ACM, 40 (2). 63--67. Google ScholarDigital Library
- Vinson, N.G. Design guidelines for landmarks to support navigation in virtual environments Proc CHI, ACM Press, 1999, 278--285. Google ScholarDigital Library
- Vogel, D. and Balakrishnan, R., Distant freehand pointing and clicking on very large, high resolution displays. in Proc. UIST 2005, (2005). Google ScholarDigital Library
- Warren, J. A Wii Workout: When Videogames Hurt Wall Street Journal Online, 2006.Google Scholar
- Weiser, M. The Computer for the Twenty-first Century Scientific American, 1991, 94--104.Google Scholar
- Zigelbaum, J., Horn, M., Shaer, O. and Jacob, R., The tangible video editor: collaborative video editing with active tokens. in Proc. TEI '07, (2007), ACM, 43--46. Google ScholarDigital Library
Index Terms
- Reality-based interaction: a framework for post-WIMP interfaces
Recommendations
Instrumental interaction: an interaction model for designing post-WIMP user interfaces
CHI '00: Proceedings of the SIGCHI conference on Human Factors in Computing SystemsThis article introduces a new interaction model called Instrumental Interaction that extends and generalizes the principles of direct manipulation. It covers existing interaction styles, including traditional WIMP interfaces, as well as new interaction ...
Reality-based interaction: unifying the new generation of interaction styles
CHI EA '07: CHI '07 Extended Abstracts on Human Factors in Computing SystemsWe are in the midst of an explosion of emerging human-computer interaction techniques that have redefined our understanding of both computers and interaction. We propose the notion of Reality-Based Interaction (RBI) as a unifying concept that ties ...
Body-centric interaction techniques for very large wall displays
NordiCHI '10: Proceedings of the 6th Nordic Conference on Human-Computer Interaction: Extending BoundariesWe examine the design space of interaction techniques for very large wall displays by drawing from existing theory and practice for reality-based interfaces and whole-body interfaces. We also apply insights drawn from research in psychology about the ...
Comments