skip to main content
10.1145/2642918.2647379acmconferencesArticle/Chapter ViewAbstractPublication PagesuistConference Proceedingsconference-collections
research-article

Sensing techniques for tablet+stylus interaction

Published:05 October 2014Publication History

ABSTRACT

We explore grip and motion sensing to afford new techniques that leverage how users naturally manipulate tablet and stylus devices during pen + touch interaction. We can detect whether the user holds the pen in a writing grip or tucked between his fingers. We can distinguish bare-handed inputs, such as drag and pinch gestures produced by the nonpreferred hand, from touch gestures produced by the hand holding the pen, which necessarily impart a detectable motion signal to the stylus. We can sense which hand grips the tablet, and determine the screen's relative orientation to the pen. By selectively combining these signals and using them to complement one another, we can tailor interaction to the context, such as by ignoring unintentional touch inputs while writing, or supporting contextually-appropriate tools such as a magnifier for detailed stroke work that appears when the user pinches with the pen tucked between his fingers. These and other techniques can be used to impart new, previously unanticipated subtleties to pen + touch interaction on tablets.

Skip Supplemental Material Section

Supplemental Material

uistf2557-file3.wmv

wmv

77.9 MB

References

  1. Annett, M., et al. The Pen is Mightier: Understanding Stylus Behaviour While Inking on Tablets. Graphics Interface (GI 2014). Google ScholarGoogle ScholarDigital LibraryDigital Library
  2. Annett, M., et al. Is it Intended or Unintended? Palm Rejection during Direct Pen Interaction. (Currently submitted for review), 2014.Google ScholarGoogle Scholar
  3. Becchio, C., et al. Grasping intentions: from thought experiments to emprical evidence. Frontiers in Human Neuroscience, 2012. 6(117).Google ScholarGoogle ScholarCross RefCross Ref
  4. Bi, X., et al. An Exploration of Pen Rolling. UIST'08.Google ScholarGoogle Scholar
  5. Brandl, P., et al. Occlusion-aware menu design for digital tabletops. CHI'09 Extended Abstracts. Google ScholarGoogle ScholarDigital LibraryDigital Library
  6. Buxton, W. Integrating the Periphery and Context: A New Taxonomy of Telematics. Proc. Graphics Interface '95.Google ScholarGoogle Scholar
  7. Cheng, L.-P., et al. iRotate grasp: automatic screen rotation based on grasp of mobile devices. UIST Adjunct Proc. '12. Google ScholarGoogle ScholarDigital LibraryDigital Library
  8. Cheng, L.-P., et al. iGrasp: grasp-based adaptive keyboard for mobile devices. CHI '13 Extended Abstracts. Google ScholarGoogle ScholarDigital LibraryDigital Library
  9. Chun Yat Li, F., et al. The 1Line Keyboard: A QWERTY Layout in a Single Line. UIST'11.Google ScholarGoogle Scholar
  10. FiftyThree, I. A closer look at zoom. http://tinyurl.com/mxqdshz.Google ScholarGoogle Scholar
  11. FiftyThree, I. Pencil (for Paper). http://www.fiftythree.com/pencil.Google ScholarGoogle Scholar
  12. Goel, M., et al. GripSense: Using Built-In Sensors to Detect Hand Posture and Pressure on Commodity Mobile Phones. UIST'12. Google ScholarGoogle ScholarDigital LibraryDigital Library
  13. Guiard, Y., Millerat, F., Writing Postures in Left-Handers: Inverters are Hand-Crossers. Neuropsychologia, 1984. 22(5): p. 535--538.Google ScholarGoogle Scholar
  14. Harrison, B., et al. Squeeze me, hold me, tilt me! An exploration of manipulative user interfaces. CHI'98. Google ScholarGoogle ScholarDigital LibraryDigital Library
  15. Hasan, K., et al. A-coord input: coordinating auxiliary input streams for augmenting contextual pen-based interactions. CHI '12. Google ScholarGoogle ScholarDigital LibraryDigital Library
  16. Hinckley, K. Synchronous Gestures for Multiple Users and Computers. UIST'03. Google ScholarGoogle ScholarDigital LibraryDigital Library
  17. Hinckley, K., et al. Motion and Context Sensing Techniques for Pen Computing. Proc. Graphics Interface 2013 (GI'13). Google ScholarGoogle ScholarDigital LibraryDigital Library
  18. Hinckley, K., et al. Codex: A Dual-Screen Tablet Computer. CHI'09. Google ScholarGoogle ScholarDigital LibraryDigital Library
  19. Hinckley, K., Pierce, J., Sinclair, M., Horvitz, E. Sensing techniques for mobile interaction. UIST 2000. Google ScholarGoogle ScholarDigital LibraryDigital Library
  20. Hinckley, K., Song, H. Sensor Synaesthesia: Touch in Motion, and Motion in Touch. CHI 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  21. Hinckley, K., et al. Pen + Touch = New Tools. UIST 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  22. Holmquist, L., et al. Smart-Its Friends: A Technique for Users to Easily Establish Connections between Smart Artifacts. Ubicomp'01. Google ScholarGoogle ScholarDigital LibraryDigital Library
  23. Hudson, S., et al. Whack Gestures: Inexact and Inattentive Interaction with Mobile Devices. TEI 2010. Google ScholarGoogle ScholarDigital LibraryDigital Library
  24. Kim, K.-E., et al. Hand Grip Pattern Recognition for Mobile User Interfaces. Proceedings of AAAI/IAAI-2006: Innovative Appl. Google ScholarGoogle ScholarDigital LibraryDigital Library
  25. Kurtenbach, G., et al. The design of a GUI paradigm based on tablets, two-hands, and transparency. CHI'97. Google ScholarGoogle ScholarDigital LibraryDigital Library
  26. Lee, D., et al. PhantomPen: virtualization of pen head for digital drawing free from pen occlusion & visual parallax. UIST'12. Google ScholarGoogle ScholarDigital LibraryDigital Library
  27. Lester, J., et al. Are You With Me? Using Accelerometers to Determine if Two Devices are Carried. Pervasive'04.Google ScholarGoogle Scholar
  28. Liu, S., Guimbretiere, F. FlexAura: A Flexible Near-Surface Range Sensor. UIST '12. Google ScholarGoogle ScholarDigital LibraryDigital Library
  29. Luff, P., Heath, C. Mobility in collaboration. CSCW '98. Google ScholarGoogle ScholarDigital LibraryDigital Library
  30. Mackenzie, C., Iberall, T., The Grasping Hand. Advances in Psychology 104. 1994, Amsterdam: North Holland.Google ScholarGoogle Scholar
  31. Mahoney, R., Nonlinear Complementary Filters on the Special Orthogonal Group. IEEE Trans. on Automatic Control, 2008. 53(5).Google ScholarGoogle Scholar
  32. Marquardt, N., et al. Cross-Device Interaction via Micro-mobility and F-formations. UIST '12. 2012. Cambridge, MA. Google ScholarGoogle ScholarDigital LibraryDigital Library
  33. Mason, A., MacKenzie, C., Grip forces when passing an object to a partner. Exp. Brain Res, 2005(163): p. 173--187.Google ScholarGoogle Scholar
  34. Matulic, F., Norrie, M. Supporting active reading on pen and touchoperated tabletops. Proc. Int'l Working Conf. on Advanced Visual Interfaces (AVI'12). 2012. Capri Island, Italy. Google ScholarGoogle ScholarDigital LibraryDigital Library
  35. Moleskine Journal (iPad app). 2014. http://tinyurl.com/kqhtomz.Google ScholarGoogle Scholar
  36. Penultimate by Evernote (iPad app). 2014. http://tinyurl.com/kx82c8a.Google ScholarGoogle Scholar
  37. Premerlani, W., Bizard, P. Direction Cosine Matrix IMU: Theory. 2009. http://gentlenav.googlecode.com/files/DCMDraft2.pdf.Google ScholarGoogle Scholar
  38. Rofouei, M., et al. Your Phone or Mine? Fusing Body, Touch and Device Sensing for MultiUser Device-Display Interaction. CHI 2012. Google ScholarGoogle ScholarDigital LibraryDigital Library
  39. Roudaut, A., et al. TimeTilt: Using Sensor-Based Gestures to Travel through Multiple Applications on a Mobile Device. Interact '09. Google ScholarGoogle ScholarDigital LibraryDigital Library
  40. Schmidt, A., et al. Advanced interaction in context. Handheld and Ubiquitous Computing (HUC'99). Google ScholarGoogle ScholarDigital LibraryDigital Library
  41. Schmidt, D., et al. PhoneTouch: A Technique for Direct Phone Interaction on Surfaces. UIST'10. Google ScholarGoogle ScholarDigital LibraryDigital Library
  42. Schwarz, J., et al. Probabilistic Palm Rejection Using Spatiotemporal Touch Features and Iterative Classification. CHI'14. Google ScholarGoogle ScholarDigital LibraryDigital Library
  43. Selin, A., Pencil Grip: A descriptive Model and Four Empirical Studies. 2003: Akademi University Press.Google ScholarGoogle Scholar
  44. Sellen, A., Shaw., K. How Knowledge Workers Use the Web. CHI'02. Google ScholarGoogle ScholarDigital LibraryDigital Library
  45. Siio, I., Tsujita, H. Mobile interaction using paperweight metaphor. UIST '06. Google ScholarGoogle ScholarDigital LibraryDigital Library
  46. Song, H., et al. Grips and Gestures on a Multi-Touch Pen. CHI'11. Google ScholarGoogle ScholarDigital LibraryDigital Library
  47. Sun, M., et al. Enhancing Naturalness of Pen-and-Tablet Drawing through Context Sensing. Interactive Tabletops and Surfaces (ITS '11). Google ScholarGoogle ScholarDigital LibraryDigital Library
  48. Suzuki, Y., et al. Stylus Enhancement to Enrich Interaction with Computers. HCII 2007, Part II. 2007: Springer-Verlag. Google ScholarGoogle ScholarDigital LibraryDigital Library
  49. Taylor, B., Bove Jr., V. Graspables: Grasp-Recognition as a User Interface. CHI'09. Google ScholarGoogle ScholarDigital LibraryDigital Library
  50. Tian, F., et al. Tilt Menu: Using the 3D Orientation Information of Pen Devices to Extend the Selection Capability. CHI'08. Google ScholarGoogle ScholarDigital LibraryDigital Library
  51. Vogel, D., Casiez, G. Conte: Multimodal Input Inspired by an Artist's Crayon. UIST 2011. Google ScholarGoogle ScholarDigital LibraryDigital Library
  52. Wagner, J., et al. BiTouch and BiPad: Designing Bimanual Interaction for Hand-held Tablets. CHI'12. Google ScholarGoogle ScholarDigital LibraryDigital Library
  53. Wimmer, R., Boring, S. HandSense Discriminating Different Ways of Grasping and Holding a Tangible User Interface. TEI '09. Google ScholarGoogle ScholarDigital LibraryDigital Library
  54. Xin, Y., et al. Natural Use Profiles for the Pen: An Empirical Exploration of Pressure, Tilt, and Azimuth. CHI'12. Google ScholarGoogle ScholarDigital LibraryDigital Library
  55. Yoon, D., Chen, N., Guimbretiere, F. TextTearing: Expanding Whitespace for Digital Ink Annotation. UIST'13. Google ScholarGoogle ScholarDigital LibraryDigital Library

Index Terms

  1. Sensing techniques for tablet+stylus interaction

    Recommendations

    Comments

    Login options

    Check if you have access through your login credentials or your institution to get full access on this article.

    Sign in
    • Published in

      cover image ACM Conferences
      UIST '14: Proceedings of the 27th annual ACM symposium on User interface software and technology
      October 2014
      722 pages
      ISBN:9781450330695
      DOI:10.1145/2642918

      Copyright © 2014 ACM

      Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from [email protected].

      Publisher

      Association for Computing Machinery

      New York, NY, United States

      Publication History

      • Published: 5 October 2014

      Permissions

      Request permissions about this article.

      Request Permissions

      Check for updates

      Qualifiers

      • research-article

      Acceptance Rates

      UIST '14 Paper Acceptance Rate74of333submissions,22%Overall Acceptance Rate842of3,967submissions,21%

      Upcoming Conference

      UIST '24

    PDF Format

    View or Download as a PDF file.

    PDF

    eReader

    View online with eReader.

    eReader