Abstract
This article describes a design and implementation of an augmented desk system, named EnhancedDesk, which smoothly integrates paper and digital information on a desk. The system provides users an intelligent environment that automatically retrieves and displays digital information corresponding to the real objects (e.g., books) on the desk by using computer vision. The system also provides users direct manipulation of digital information by using the users' own hands and fingers for more natural and more intuitive interaction. Based on the experiments with our first prototype system, some critical issues on augmented desk systems were identified when trying to pursue rapid and fine recognition of hands and fingers. To overcome these issues, we developed a novel method for realtime finger tracking on an augmented desk system by introducing a infrared camera, pattern matching with normalized correlation, and a pan-tilt camera. We then show an interface prototype on EnhancedDesk. It is an application to a computer-supported learning environment, named Interactive Textbook. The system shows how effective the integration of paper and digital information is and how natural and intuitive direct manipulation of digital information with users' hands and fingers is.
- ARAI, T., AUST,D.,AND HUDSON, S. 1997. Paperlink: A technique for hyperlinking from real paper to electronic content. In Proceedings of the ACM Conference on Human Factors in Computing Systems (CHI'97).]] Google Scholar
- ARAI, T., MACHII, K., AND KUZUNUKI, S. 1995. Retrieving electronic documents with realworld objects on interacivedesk. In Proceedings of the ACMSymposium on User Interface Software and Technology (UIST'95). 37-38.]] Google Scholar
- BOLT, R. A. 1984. The Human Interface. Lifetime Learning Publications, Belmont, CA.]]Google Scholar
- FITZMAURICE, G., ISHII, H., AND BUXTON, W. 1995. Bricks: Laying the foundations for graspable user interfaces. In Proceedings of the ACM Conference on Human Factors in Computing System (CHI'95). 442-449.]] Google Scholar
- FREEMAN,W.T.,ANDERSON,D.B.,BEARDSLEY, P. A., DODGE,C.N.,ROTH, M., WEISSMAN,C.D.,AND YERAZUNIS, W. S. 1998. Computer vision for interactive computer graphics. IEEE Comput. Graph. Appl. 42-53.]] Google Scholar
- ISHII,H.AND ULLMER, B. 1997. Tangible bits: Towards seamless interface between people, bits and atoms. In Proceedings of the ACM Conference on Human Factors in Computing System (CHI'97). 234-241.]] Google Scholar
- ISHII, H., WISNESKI, C., ORBANES, J., CHUN,B.,AND PARADISO, J. 1999. Pingpongplus: Design of an athletic-tangible interface for computer-supported cooperative play. In Proceedings of the ACM Conference on Human Factors in Computing System (CHI'99). 394-401.]] Google Scholar
- KOBAYASHI,M.AND KOIKE, H. 1997. InforFloor Project of Haishi Exhibition at NTT Inter Communication Center in Tokyo. http://www.ntticc.or.jp/special/utopia/index e.html.]]Google Scholar
- KOBAYASHI,M.AND KOIKE, H. 1998. Enhanceddesk: Integrating paper documents and digital documents. n Proceedings of 1998 Asia Pacific Computer Human Interaction (APCHI'98). IEEE CS, 57-62.]] Google Scholar
- KOBAYASHI, Y., KITAJIMA, K., NISHI,T.,AND KOIKE, H. 1999. Visionplus7 Table Project at International Conference, Visionplus7. http://www.visionplus7.com/.]]Google Scholar
- KOIKE, H., SATO, Y., KOBAYASHI, Y., TOBITA, H., AND KOBAYASHI, M. 2000. Interactive Textbook and interactive Venn diagram: Natural and intuitive interface on Augmented Desk System. In Proceedings of Human Factors in Computing Systems (CHI'2000). ACM, 121-128.]] Google Scholar
- KRUGER, M. 1991. Artificial Reality, 2nd ed. Addison-Wesley, Reading, MA.]]Google Scholar
- MACKAY, W. 1993. Augmenting reality: Adding computational dimensions to paper. Commun. ACM 36, 7, 96-97.]] Google Scholar
- NISHI, T., SATO,Y.,AND KOIKE, H. 2001. Snaplink: Interactive object registration and recognition for augmented desk interface. In Proceedings of IFIP International Conference on Human- Computer Interaction (INTERACT'01). IOS Press, 240-246.]] Google Scholar
- REKIMOTO, J. 1998. Matrix: A realtime object identification and registration method for augmented reality. In Proceedings 1998 Asia Pacific Computer Human Interaction (APCHI'98).]] Google Scholar
- REKIMOTO, J., OKA, M., MATSUSHITA,N.,AND KOIKE, H. 1998. Holowall: Interactive digital surfaces. In Proceedings of the Conference on SIGGRAPH 98: Conference abstracts and applications. 108.]] Google Scholar
- REKIMOTO,J.AND SAITO, M. 1999. Augmented surfaces: A spatially continuous work space for hybrid computing environments. In Proceedings of the ACM Conference on Human Factors in Computing System (CHI'99). 378-385.]] Google Scholar
- ROBINSON, P. 1997. Animated paper documents. In Proceedings of HCI'97(21B). 655-658.]]Google Scholar
- SATO, Y., KOBAYASHI,Y.,AND KOIKE, H. 2000. Fast tracking of hands and fingertips in infrared images for augmented desk interface. In Proceedings of IEEE Automatic Face and Gesture Recognition (FG'2000). IEEE/CS, 462-467.]] Google Scholar
- TAKADA,T.AND KOIKE, H. 1998. Chabudai project at Bauhaus Museum in Berlin. http://www.vogue.is.uec.ac.jp/zetaka/Public/kenky-u/table/berlintable.html.]]Google Scholar
- TURK, M. 1998. Moving from GUIs to PUIs. In Proceedings of Fourth Symposium on Intelligent Information Media.]]Google Scholar
- UNDERKOFFLER,J.AND ISHII, H. 1998. Illuminating light: An optical design tool with a luminoustangible interface. In Proceedings of the ACMConference on Human Factors in Computing System (CHI'98). 542-549.]] Google Scholar
- UNDERKOFFLER,J.AND ISHII, H. 1999. Urp: A luminous-tangible workbench for urban planning and design. In Proceedings of the ACMConference on Human Factors in Computing System (CHI'99). 386-393.]] Google Scholar
- WELLNER, P. 1993. Interacting with the paper on the DigitalDesk. Commun. ACM 36, 7, 87-96.]] Google Scholar
Index Terms
- Integrating paper and digital information on EnhancedDesk: a method for realtime finger tracking on an augmented desk system
Recommendations
Two-handed drawing on augmented desk system
AVI '02: Proceedings of the Working Conference on Advanced Visual InterfacesThis paper describes a two-handed drawing tool developed on our augmented desk system. Using our real-time finger tracking method, a user can draw and manipulate objects interactively by his/her own finger/hand. Based on the former work on two-handed ...
Visual touchpad: a two-handed gestural input device
ICMI '04: Proceedings of the 6th international conference on Multimodal interfacesThis paper presents the Visual Touchpad, a low-cost vision-based input device that allows for fluid two-handed interactions with desktop PCs, laptops, public kiosks, or large wall displays. Two downward-pointing cameras are attached above a planar ...
A gesture control system for intuitive 3D interaction with virtual objects
VRCAI 08We present a system for interacting with 3D objects in a 3D virtual environment. Using the notion that a typical head-mounted display (HMD) does not cover the user's entire face, we use a fiducial marker placed on the HMD to locate the user's exposed ...
Reviews
Jose M. Ramirez
The authors describe this research as an application for a computer-supported learning environment, but from my point of view, this is a very sophisticated pattern recognition system, focused on real-time finger tracking. EnhancedDesk is an integration of cameras and sensors that allows users to interact with a computer graphic simulation corresponding to the subject presented in their reading material. The reading material, a book, is prepared with 2D matrix codes that trigger the presentation of graphics. This is called an interactive textbook. The real contribution of this paper is the system integration it describes, which appears to give robust results in the finger tracking task under different lighting conditions. The system may be useful as a starting point in the path to obtain a general-purpose, analog-digital integrated environment. The related work presented is quite limited and the conclusions are weak: there is no formal evaluation of the system’s performance to give the reader an idea of how far it is from a real-world system. However, the references are well chosen, and the illustrations are very clear. The section “Design and Implementation of EnhancedDesk,” is particularly clear and fluid. Online Computing Reviews Service
Access critical reviews of Computing literature here
Become a reviewer for Computing Reviews.
Comments