Shahzad Malik
ABSTRACT
This paper presents the Visual Touchpad, a low-cost vision-based input device that allows for fluid two-handed interactions with desktop PCs, laptops, public kiosks, or large wall displays. Two downward-pointing cameras are attached above a planar surface, and a stereo hand tracking system provides the 3D positions of a user's fingertips on and above the plane. Thus the planar surface can be used as a multi-point touch-sensitive device, but with the added ability to also detect hand gestures hovering above the surface. Additionally, the hand tracker not only provides positional information for the fingertips but also finger orientations. A variety of one and two-handed multi-finger gestural interaction techniques are then presented that exploit the affordances of the hand tracker. Further, by segmenting the hand regions from the video images and then augmenting them transparently into a graphical interface, our system provides a compelling direct manipulation experience without the need for more expensive tabletop displays or touch-screens, and with significantly less self-occlusion.
- Buxton, W. (1992). Telepresence: integrating shared task and person spaces. In Proceedings of Graphics Interface, pp. 123--129.]] Google Scholar
Digital Library
- Chen, X., Koike, H., Nakanishi, Y., Oka, K., Sato, Y. (2002). Two-handed drawing on augmented desk system. In Proceedings of Advanced Visual Interfaces (AVI). pp. 219--222.]] Google ScholarDigital Library
- Corso, J., Burschka, D., Hager, G. (2003). The 4D Touchpad: Unencumbered HCI with VICs. In Proceedings of CVPR-HCI.]]Google ScholarCross Ref
- Faugeras, O., Luong, Q. (2001). The Geometry of Multiple Images. The MIT Press.]] Google ScholarDigital Library
- Guiard, Y. (1987). Asymmetric Divison of Labor in Human Skilled Bimanual Action: The Kinematic Chain as a Model. Journal of Motor Behavior, 19(4). pp. 486--517.]]Google ScholarCross Ref
- Hardenberg, C., Berard, F. (2001). Bare-hand Human-computer Interaction. In Proceedings of ACM Workshop on Perceptive User Interfaces (PUI).]] Google ScholarDigital Library
- Ishii, H., Kobayashi, M. (1992). ClearBoard: a seamless medium for shared drawing and conversation with eye contact. In Proceedings of ACM CHI Conference. pp. 525--532.]] Google ScholarDigital Library
- Kabbash, P., Buxton, W., Sellen, A. (1994). Two-Handed Input in a Compound Task. In Proceedings of ACM CHI Conference. pp. 417--423.]] Google ScholarDigital Library
- Kolsch, M., Turk, M. (2002). Keyboards without Keyboards: A Survey of Virtual Keyboards. Technical Report 2002-21. University of California, Santa Barbara.]]Google Scholar
- Krueger, M. (1991). VIDEOPLACE and the Interface of the Future. In The Art of Human Computer Interface Design. Addison Wesley, Menlo Park, CA. pp. 417--422.]]Google Scholar
- Kurtenbach, G., Fitzmaurice, G., Baudel, T., Buxton, B. (1997). The Design of a GUI Paradigm based on Tablets, Two-hands, and Transparency. In Proceedings of ACM CHI Conference. pp. 35--42.]] Google ScholarDigital Library
- MacCormick, J., Isard, M. (2000). Partitioned sampling, articulated objects, and interface-quality hand tracking. In Proceedings of the European Conference on Computer Vision (ECCV), Volume 2. pp. 3--19.]] Google ScholarDigital Library
- Mysliwiec, T. (1994). FingerMouse: A Freehand Computer Pointing Interface. Technical Report VISLab-94-001, University of Illinois Chicago.]]Google Scholar
- Oka, K., Sato, Y., & Koike, H. (2002). Real-time tracking of multiple fingertips and gesture recognition for augmented desk interface systems. In Proceedings of IEEE Conference on Automatic Face and Gesture Recognition (FG). pp. 429--434.]] Google ScholarDigital Library
- Rekimoto, J. (2002). SmartSkin: An Infrastructure for Freehand Manipulation on Interactive Surfaces. In Proceedings of ACM CHI Conference. pp. 113--120.]] Google ScholarDigital Library
- Ringel, M., Berg, H., Jin, Y., and Winograd, T. (2001). Barehands: Implement-Free Interaction with a Wall-Mounted Display. In Proceedings of ACM CHI Conference Extended Abstracts. pp. 367--368.]] Google ScholarDigital Library
- Roussel, N. (2001). Exploring new uses of video with videoSpace. In Proceedings of the IFIP Conference on Engineering for HCI, Volume 2254 of Lecture Notes in Computer Science, Springer. pp. 73--90.]] Google ScholarDigital Library
- Segen, J., & Kumar, S. (1998). GestureVR: Vision-based 3D Hand Interface for Spatial Interaction. In Proceedings of the Sixth ACM International Conference on Multimedia. pp. 455--464.]] Google ScholarDigital Library
- Stotts, D., Smith, J., and Jen, D. (2003). The Vis-a-Vid Transparent Video FaceTop. In Proceedings of ACM UIST. pp. 57--58.]]Google Scholar
- Tang, J. & Minneman, S. (1991). Videowhiteboard: video shadows to support remote collaboration. In Proceedings of ACM CHI Conference. pp. 315--322.]] Google ScholarDigital Library
- Trucco, E., Verri, A. (1998). Introductory Techniques for 3-D Computer Vision. Prentice-Hall.]] Google ScholarDigital Library
- Ukita, N., Kidode, M. (2004). Wearable Virtual Tablet: Fingertip Drawing on a Portable Plane-Object using an Active-Infrared Camera. In Proceedings of the International Conference on Intelligent User Interfaces. pp. 169--176.]] Google ScholarDigital Library
- Wellner, P. (1993). Interacting with Paper on the DigitalDesk. Communications of the ACM, 36(7), July 1993. pp. 86--96.]] Google ScholarDigital Library
- Wu, M., Balakrishnan, R. (2003). Multi-finger and Whole Hand Gestural Interaction Techniques for Multi-User Tabletop Displays. In Proceedings ACM UIST. pp. 193--202.]] Google ScholarDigital Library
- Yee, K. (2004). Two-Handed Interaction on a Tablet Display. In Proceedings of ACM CHI Extended Abstracts. pp. 1493--1496.]] Google ScholarDigital Library
- Zhang, Z., Wu, Y., Shan, Y., & Shafer, S. (2001) Visual Panel: Virtual Mouse, Keyboard, and 3D Controller with an Ordinary Piece of Paper. In Proceedings of ACM Workshop on Perceptive User Interfaces (PUI).]] Google ScholarDigital Library
Index Terms
- Visual touchpad: a two-handed gestural input device
Recommendations
Interacting with large displays from a distance with vision-tracked multi-finger gestural input
UIST '05: Proceedings of the 18th annual ACM symposium on User interface software and technologyWe explore the idea of using vision-based hand tracking over a constrained tabletop surface area to perform multi-finger and whole-hand gestural interactions with large displays from a distance. We develop bimanual techniques to support a variety of ...
The tactile touchpad
CHI EA '97: CHI '97 Extended Abstracts on Human Factors in Computing SystemsA prototype touchpad with embedded tactile feedback is described. Tactile feedback allows the touchpad to mimic the operation of a mouse for basic transactions such as clicking, double-clicking, and dragging. A button click is achieved by increasing the ...
Visual panel: virtual mouse, keyboard and 3D controller with an ordinary piece of paper
PUI '01: Proceedings of the 2001 workshop on Perceptive user interfacesThis paper presents a vision-based interface system, VISUAL PANEL, which employs an arbitrary quadrangle-shaped panel (e.g., an ordinary piece of paper) and a tip pointer (e.g., fingertip) as an intuitive, wireless and mobile input device. The system ...
Comments