Andrew Bragdon
Robert Zeleznik
ABSTRACT
GestureBar is a novel, approachable UI for learning gestural interactions that enables a walk-up-and-use experience which is in the same class as standard menu and toolbar interfaces. GestureBar leverages the familiar, clean look of a common toolbar, but in place of executing commands, richly discloses how to execute commands with gestures, through animated images, detail tips and an out-of-document practice area. GestureBar's simple design is also general enough for use with any recognition technique and for integration with standard, non-gestural UI components. We evaluate GestureBar in a formal experiment showing that users can perform complex, ecologically valid tasks in a purely gestural system without training, introduction, or prior gesture experience when using GestureBar, discovering and learning a high percentage of the gestures needed to perform the tasks optimally, and significantly outperforming a state of the art crib sheet. The relative contribution of the major design elements of GestureBar is also explored. A second experiment shows that GestureBar is preferred to a basic crib sheet and two enhanced crib sheet variations.
Supplemental Material
- Alvarado, C. Sketch Recognition User Interfaces: Guidelines for Design and Development. In Proc. of AAAI Fall Symposium on Intelligent Pen-based Interfaces, (2004).Google Scholar
- Alvarado, C. and Davis, R. SketchREAD: a multi-domain sketch recognition engine. In Proc. of UIST'04, 23--32. Google ScholarDigital Library
- Bau, O., and Mackay, W. OctoPocus: A Dynamic Guide for Learning Gesture-Based Command Sets. UIST'08, 37--46. Google ScholarDigital Library
- Buxton, W. Chunking and phrasing and the design of human-computer dialogues. IFIP World Computer Congress'86, 475--480.Google Scholar
- Buxton, W., Fiume, E., Hill, R., Lee, A., Woo, C. Continuous Hand-Gesture Driven Input. Graphics Interface '83, 191--195.Google Scholar
- Forsberg, A, Holden, L., Miller, T., and Zeleznik, R. The Music Notepad, Brown University (2005).Google Scholar
- Grossman, T., Dragicevic, P. and Balakrishnan, R. Strategies for Accelerating Online Learning of Hotkeys. CHI'07, 137--144. Google ScholarDigital Library
- Grossman, T., Hinckley, K., Baudisch, P., Agrawala, M., Balakrishnan, R. Hover Widgets: Using the Tracking State to Extend the Capabilities of Pen-Operated Devices. CHI'06, 861--870. Google ScholarDigital Library
- Hinckley, K., Baudisch, P., Ramos, G., Guimbretiere, F. Design and Analysis of Delimiters for Selection-Action Pen Gesture Phrases in Scriboli. In Proc. of CHI'05, 453--460. Google ScholarDigital Library
- Hinckley, K., Zhao, S., Sarin, R., Baudisch, P., Cutrell, Ed., Shilman, M., Tan, D. InkSeine: In Situ Search for Active Note Taking. In Proc. of CHI (2007), 251--260. Google ScholarDigital Library
- Holm, S. A Simple Sequentially Rejective Multiple Test Procedure. Scandinavian Journal of Statistics, 6 (1979), 60--65.Google Scholar
- Hong, J. and Landay, J. SATIN: A Toolkit for Informal Ink-Based Applications. In Proc. of UIST (2000), 63--72. Google ScholarDigital Library
- Hse, H. and Newton, A. Recognition and beautification of multi-stroke symbols in digital ink. Computers&Graphics, 29, 4 (August 2005), 533--546. Google ScholarDigital Library
- Igarashi, T., Matsuoka, S., Kawachiya, S., and Tanaka, H. Interactive Beautification: A Technique for Rapid Geometric Design. In Proc. of UIST (2007), 105--114. Google ScholarDigital Library
- Kurtenbach, G., Moran, T. P. and Buxton, W. Contextual Animation of Gestural Commands. Graphics Interface '94, 83--90.Google Scholar
- Long, C., Landay, J., Rowe, L., and Michiels, J. Visual Similarity of Pen Gestures. In Proc. of CHI'00, 360--367. Google ScholarDigital Library
- Mankoff, J., Hudson, S. and Abowd, G. Providing Integrated Toolkit-level Suport for Ambiguity in Recognition-based Interfaces. CHI'00, 368--375. Google ScholarDigital Library
- Microsoft, Inc. Office 2007. 2006.Google Scholar
- Microsoft, Inc. Windows Vista. 2006.Google Scholar
- Mouse Gestures Add-on for the Mozilla Firefox Web Browser. http://www.mousegestures.org/.Google Scholar
- Palm, Inc. Graffiti character recognizer.Google Scholar
- Polson, P. and Lewis, C. Theory-Based Design for Easily Learned Interfaces. Human-Computer Interaction, 5, 2 (June 1990), 191--220. Google ScholarDigital Library
- Rubine, D. Specifying Gestures by Example. In Proc. of SIGGRAPH'91, 329--337. Google ScholarDigital Library
- Tapia, M., and Kurtenbach, G. Some Design Refinements and Principles on the Appearance and Behavior of Marking Menus. UIST'95, 189--195.25. Google ScholarDigital Library
- Wobbrock, J., Wilson, A., and Li, Y. Gestures without Libraries, Toolkits or Training: A $1 Recognizer for User Interface Prototypes. In Proc. of UIST (2007), 159--168. Google ScholarDigital Library
- Zeleznik, R., Bragdon, A., Liu, C., and Forsberg, A. Lineogrammer: Creating Diagrams by Drawing. UIST '08, 161--170. Google ScholarDigital Library
- Zeleznik, R., and Miller, T. Fluid Inking: Augmenting the Medium of Free-Form Inking with Gestures. In Proceedings of Graphics Interface (2006), 155--162. Google ScholarDigital Library
- Zhao, S., Agrawala, M. and Hinckley, K. Zone and Polygon Menus: Using Relative Position to Increase the Breadth of Multi-stroke Marking Menus. CHI (2006), 1077--1086. Google ScholarDigital Library
Index Terms
- GestureBar: improving the approachability of gesture-based interfaces
Recommendations
Pen + touch = new tools
UIST '10: Proceedings of the 23nd annual ACM symposium on User interface software and technologyWe describe techniques for direct pen+touch input. We observe people's manual behaviors with physical paper and notebooks. These serve as the foundation for a prototype Microsoft Surface application, centered on note-taking and scrapbooking of ...
Gesture play: motivating online gesture learning with fun, positive reinforcement and physical metaphors
ITS '10: ACM International Conference on Interactive Tabletops and SurfacesLearning a set of gestures requires a non-trivial investment of time from novice users. We propose a novel approach based on positive reinforcement for motivating the online learning of multi-touch gestures: introducing simple, game-like elements to ...
PhantomPen: virtualization of pen head for digital drawing free from pen occlusion & visual parallax
UIST '12: Proceedings of the 25th annual ACM symposium on User interface software and technologyWe present PhantomPen, a direct pen input device whose pen head is virtualized onto the tablet display surface and visually connected to a graspable pen barrel in order to achieve digital drawing free from pen occlusion and visual parallax. As the pen ...
Comments