P. O. Kristensson
L. C. Denby
ABSTRACT
We present a technique that enables continuous recognition and visualization of pen strokes and touch-screen gestures. We describe an incremental recognition algorithm that provides probability distributions over template classes as a function of users' partial or complete stroke articulations. We show that this algorithm can predict users' intended template classes with high accuracy on several different datasets. We use the algorithm to design two new visualizations that reveal various aspects of the recognition process to users. We then demonstrate how these visualizations can help users to understand how the recognition process interprets their input and how interactions between different template classes affect recognition outcomes.
- {AB10} Appert C., Bau O.: Scale detection for a priori gesture recognition. In Proceedings of the 28th ACM Conference on Human Factors in Computing Systems (2010), ACM Press, pp. 879--882. 1, 3, 7 Google Scholar
Digital Library
- {AN00} Arvo J., Novins K.: Fluid sketches: continuous recognition and morphing of simple hand-drawn shapes. In Proceedings of the 13th Annual ACM Symposium on User Interface Software and Technology (2000), ACM Press, pp. 73--80. 1 Google ScholarDigital Library
- {AN06} Arvo J., Novins K.: Fluid sketching of directed graphs. In Proceedings of the 7th Australasian User Interface Conference (2006), Australian Computer Society, Inc., pp. 81--86. 1 Google ScholarDigital Library
- {BM08} Bau O., Mackay W.: Octopocus: a dynamic guide for learning gesture-based command sets. In Proceedings of the 21st Annual ACM Symposium on User Interface Software and Technology (2008), ACM Press, pp. 37--46. 1, 3, 7 Google ScholarDigital Library
- {BZW*09} Bragdon A., Zeleznik R., Williamson B., Miller T., LaViola Jr. J. J.: Gesturebar: improving the approachability of gesture-based interfaces. In Proceedings of the 27th ACM Conference on Human Factors in Computing Systems (2009), ACM Press, pp. 2269--2278. 1 Google ScholarDigital Library
- {GR93} Goldberg D., Richardson D.: Touch-typing with a stylus. In Proceedings of the 17th ACM Conference on Human Factors in Computing Systems (1993), ACM Press, pp. 80--87. 2 Google ScholarDigital Library
- {JLR99} Jr. A. L., Landay J., Rowe L.: Implications for a gesture design tool. In Proceedings of the 17th ACM Conference on Human Factors in Computing Systems (1999), ACM Press, pp. 40--47. 1, 8 Google ScholarDigital Library
- {Kri09} Kristensson P. O.: Five challenges for intelligent text entry methods. AI Magazine 30, 4 (2009), 85--94. 1Google Scholar
- {KZ04} Kristensson P. O., Zhai S.: SHARK2: a large vocabulary shorthand writing system for pen-based computers. In Proceedings of the 17th Annual ACM Symposium on User Interface Software and Technology (2004), ACM Press, pp. 43--52. 1, 2, 3, 4, 7, 8 Google ScholarDigital Library
- {KZ07} Kristensson P. O., Zhai S.: Command strokes with and without preview: using pen gestures on keyboard for command selection. In Proceedings of the 25th ACM Conference on Human Factors in Computing Systems (2007), ACM Press, pp. 1137--1146. 1, 7 Google ScholarDigital Library
- {Li09} Li Y.: Beyond pinch and flick: Enriching mobile gesture interaction. IEEE Computer 42 (2009), 87--89. 1 Google ScholarDigital Library
- {Li10} Li Y.: Protractor: a fast and accurate gesture recognizer. In Proceedings of the 28th ACM Conference on Human Factors in Computing Systems (2010), ACM Press, pp. 2169--2172. 1 Google ScholarDigital Library
- {NY95} Niblack W., Yin J.: A pseudo-distance measure for 2d shapes based on turning angle. In Proceedings of the 2nd International Conference on Image Processing (1995), IEEE Press, pp. 352--355. 3, 7 Google ScholarDigital Library
- {Rub91} Rubine D.: Specifying gestures by example. In Proceedings of the 18th Annual Conference on Computer Graphics and Interactive Techniques (1991), ACM Press, pp. 329--337. 1 Google ScholarDigital Library
- {TF99} Tseng S., Fogg B.: Credibility and computing technology. Communications of the ACM 42 (1999), 39--44. 1 Google ScholarDigital Library
- {WAWL07} Wobbrock J., A. D. Wilson A., Li Y.: Gestures without libraries, toolkits or training: a $1 recognizer for user interface prototypes. In Proceedings of the 20th Annual ACM Symposium on User Interface Software and Technology (2007), ACM Press, pp. 159--168. 1, 3, 4, 6, 7 Google ScholarDigital Library
- {ZK03} Zhai S., Kristensson P. O.: Shorthand writing on stylus keyboard. In Proceedings of the 21st ACM Conference on Human Factors in Computing Systems (2003), ACM Press, pp. 97--104. 1, 2, 7, 8 Google ScholarDigital Library
- {ZM06} Zeleznik R., Miller T.: Fluid inking: augmenting the medium of free-form inking with gestures. In Proceedings of Graphics Interface 2006 (2006), Canadian Information Processing Society, pp. 155--162. 1 Google ScholarDigital Library
Index Terms
- Continuous recognition and visualization of pen strokes and touch-screen gestures
Recommendations
Grips and gestures on a multi-touch pen
CHI '11: Proceedings of the SIGCHI Conference on Human Factors in Computing SystemsThis paper explores the interaction possibilities enabled when the barrel of a digital pen is augmented with a multi-touch sensor. We present a novel multi-touch pen (MTPen) prototype and discuss its alternate uses beyond those of a standard stylus, ...
Investigating multi-touch and pen gestures for diagram editing on interactive surfaces
ITS '09: Proceedings of the ACM International Conference on Interactive Tabletops and SurfacesCreating and editing large graphs and node-link diagrams are crucial activities in many application areas. For them, we consider multi-touch and pen input on interactive surfaces as very promising. This fundamental work presents a user study ...
Pen + touch = new tools
UIST '10: Proceedings of the 23nd annual ACM symposium on User interface software and technologyWe describe techniques for direct pen+touch input. We observe people's manual behaviors with physical paper and notebooks. These serve as the foundation for a prototype Microsoft Surface application, centered on note-taking and scrapbooking of ...
Comments