Shaun K. Kane
Daniel Avrahami
Jacob O. Wobbrock
ABSTRACT
We present Bonfire, a self-contained mobile computing system that uses two laptop-mounted laser micro-projectors to project an interactive display space to either side of a laptop keyboard. Coupled with each micro-projector is a camera to enable hand gesture tracking, object recognition, and information transfer within the projected space. Thus, Bonfire is neither a pure laptop system nor a pure tabletop system, but an integration of the two into one new nomadic computing platform. This integration (1) enables observing the periphery and responding appropriately, e.g., to the casual placement of objects within its field of view, (2) enables integration between physical and digital objects via computer vision, (3) provides a horizontal surface in tandem with the usual vertical laptop display, allowing direct pointing and gestures, and (4) enlarges the input/output space to enrich existing applications. We describe Bonfire's architecture, and offer scenarios that highlight Bonfire's advantages. We also include lessons learned and insights for further development and use.
Supplemental Material
- Agarawala, A. and Balakrishnan, R. (2006) Keepin' it real: pushing the desktop metaphor with physics, piles and the pen. Proc. CHI '06. New York: ACM Press, 1283--1292. Google Scholar
Digital Library
- Arai, T., Machii, K. and Kuzunuki, S. (1995) Retrieving electronic documents with real-world objects on InteractiveDESK. Proc. UIST '95. New York: ACM Press, 37--38. Google ScholarDigital Library
- Beardsley, P., Baar, J.V., Raskar, R. and Forlines, C. (2005) Interaction using a handheld projector. IEEE Computer Graphics and Applications, 25 (1), 39--43. Google ScholarDigital Library
- Berry, L., Bartram, L. and Booth, K.S. (2005) Role-based control of shared application views. Proc. UIST '05. New York: ACM Press, 23--32. Google ScholarDigital Library
- Blasko, G., Feiner, S. and Coriand, F. (2005) Exploring interaction with a simulated wrist-worn projection display. Proc. ISWC '09. Washington, DC: IEEE Computer Society, 2--9. Google ScholarDigital Library
- Butler, A., Izadi, S. and Hodges, S. (2008) SideSight: multi-"touch" interaction around small devices. Proc. UIST '08. NewYork: ACM Press, 201--204. Google ScholarDigital Library
- Cadiz, J.J., Venolia, G., Jancke, G. and Gupta, A. (2002) Designing and deploying an information awareness interface. Proc. CSCW '02. New York: ACM Press, 314--323. Google ScholarDigital Library
- Cao, X. and Balakrishnan, R. (2006) Interacting with dynamically defined information spaces using a handheld projector and a pen. Proc. UIST '06. New York: ACM Press, 225--234. Google ScholarDigital Library
- Cao, X., Forlines, C. and Balakrishnan, R. (2007) Multi-user interaction using handheld projectors. Proc. UIST '07. New York: ACM Press, 43--52. Google ScholarDigital Library
- Cohen, C.J., Beach, G.J. and Foulk, G. (2001) A basic hand gesture control system for PC applications. Proc. AIPR '01. Washington, DC: IEEE Computer Society, 74. Google ScholarDigital Library
- Dietz, P. and Leigh, D. (2001) DiamondTouch: a multi-user touch technology. Proc. UIST '01. New York: ACM Press, 219--226. Google ScholarDigital Library
- Greaves, A., Hang, A. and Rukzio, E. (2008) Evaluation of picture browsing using a projector phone. Proc. MobileHCI '08. New York: ACM Press, 527--530. Google ScholarDigital Library
- Grudin, J. (2001) Partitioning digital worlds: focal and peripheral awareness in multiple monitor use. Proc. CHI '01. New York: ACM Press, 458--465. Google ScholarDigital Library
- Hang, A., Rukzio, E. and Greaves, A. (2008) Projector phone: a study of using mobile phones with integrated projector for interaction with maps. Proc. MobileHCI '08. New York: ACM Press, 207--216. Google ScholarDigital Library
- Harrison, C. and Hudson, S.E. (2008) Scratch input: creating large, inexpensive, unpowered and mobile finger input surfaces. Proc. UIST '08. New York: ACM Press, 205--208. Google ScholarDigital Library
- Hosoi, K., Dao, V.N., Mori, A. and Sugimoto, M. (2007) CoGAME: manipulation using a handheld projector. Proc. SIGGRAPH '07 Emerging Technologies. New York: ACM Press. Google ScholarDigital Library
- Koike, H., Sato, Y. and Kobayashi, Y. (2001) Integrating paper and digital information on EnhancedDesk: a method for realtime finger tracking on an augmented desk system. ACM Transactions on Computer-Human Interaction, 8 (4), 307--322. Google ScholarDigital Library
- Letessier, J. and Bérard, F. (2004) Visual tracking of bare fingers for interactive surfaces. Proc. UIST '04. New York: ACM Press, 119--122. Google ScholarDigital Library
- MacIntyre, B., Mynatt, E.D., Voida, S., Hansen, K.M., Tullio, J. and Corso, G.M. (2001) Support for multitasking and background awareness using interactive peripheral displays. Proc. UIST '01. New York: ACM Press, 41--50. Google ScholarDigital Library
- Manresa, C., Varona, J., Mas, R. and Perales, F. (2005) Hand tracking and gesture recognition for human-computer interaction. Electronic Letters on Computer Vision and Image Analysis, 5 (3), 96--104.Google ScholarCross Ref
- Mistry, M., P., Maes, P. and Chang, L. (2009) WUW -- Wear Ur World -- a wearable gestural interface. Proc. CHI '09 Extended Abstracts. ACM Press, 4111--4116. Google ScholarDigital Library
- Olwal, A. and Wilson, A.D. (2008) SurfaceFusion: unobtrusive tracking of everyday objects in tangible user interfaces. Proc. GI '08. Toronto: Canadian Information Processing Society, 235--242. Google ScholarDigital Library
- Pinhanez, C., Kjeldsen, R., Tang, L., Levas, A., Podlaseck, M., Sukaviriya, N. and Pingali, G. (2003) Creating touch-screens anywhere with interactive projected displays. Proc. MULTIMEDIA '03. New York: ACM Press, 460--461. Google ScholarDigital Library
- Pinhanez, C.S. (2001) The Everywhere Displays Projector: a device to create ubiquitous graphical interfaces. Proc. UbiComp'01. London: Springer-Verlag, 315--331. Google ScholarDigital Library
- Raskar, R., Baar, J.v., Beardsley, P., Willwacher, T., Rao, S. and Forlines, C. (2003) iLamps: geometrically aware and self-configuring projectors. Proc. SIGGRAPH '03. New York: ACM Press, 809--818. Google ScholarDigital Library
- Raskar, R., Beardsley, P., van Baar, J., Wang, Y., Dietz, P., Lee, J., Leigh, D. and Willwacher, T. (2004) RFIG lamps: interacting with a self-describing world via photosensing wireless tags and projectors. Proc. SIGGRAPH '04. New York: ACM Press, 406--415. Google ScholarDigital Library
- Rekimoto, J. (2002) SmartSkin: an infrastructure for freehand manipulation on interactive surfaces. Proc. CHI '02. New York: ACM Press, 113--120. Google ScholarDigital Library
- Rekimoto, J. and Saitoh, M. (1999) Augmented surfaces: a spatially continuous work space for hybrid computing environments. Proc. CHI '99. New York: ACM Press, 378--385. Google ScholarDigital Library
- Shen, C., Everitt, K. and Ryall, K. (2003) UbiTable: impromptu face-to-face collaboration on horizontal interactive surfaces. Proc. UbiComp '03. Heidelberg: Springer Berlin, 281--288.Google ScholarCross Ref
- Song, H., Grossman, T., Fitzmaurice, G., Guimbretiere, F., Khan, A., Attar, R. and Kurtenbach, G. (2009) PenLight: combining a mobile projector and a digital pen for dynamic visual overlay. Proc. CHI '09. New York: ACM Press, 143--152. Google ScholarDigital Library
- Stauffer, C. and Grimson, W.E.L. (1999) Adaptive background mixture models for real-time tracking. Proc. CVPR '99. Washington, DC: IEEE Computer Society, 246--252.Google Scholar
- Sugimoto, M., Miyahara, K., Inoue, H. and Tsunesada, Y. (2005) Hotaru: intuitive manipulation techniques for projected displays of mobile devices. Proc. INTERACT '05. Heidelberg: Springer Berlin, 57--688. Google ScholarDigital Library
- Sukaviriya, N., Kjeldsen, R., Pinhanez, C., Tang, L., Levas, A., Pingali, G. and Podlaseck, M. (2004) A portable system for anywhere interactions. Proc. CHI '04 Extended Abstracts. ACM Press, 789--790. Google ScholarDigital Library
- Sukthankar, R., Stockton, R.G. and Mullin, M.D. (2001) Smarter presentations: exploiting homography in camera-projector systems. Proc. ICCV '01. Washington, DC: IEEE Computing Society, 247--253.Google ScholarCross Ref
- Swain, M.J. and Ballard, D.H. (1991) Color indexing. International Journal of Computer Vision, 7 (1), 11--32. Google ScholarDigital Library
- Tognazzini, B. (1994) The "Starfire" video prototype project: a case history. Proc. CHI '94. New York: ACM Press, 99--105. Google ScholarDigital Library
- Ullmer, B. and Ishii, H. (1997) The metaDESK: Models and prototypes for tangible user interfaces. Proc. UIST '97. New York, ACM Press, 223--232. Google ScholarDigital Library
- Want, R., Fishkin, K., Gujar, A., and Harrison, B. (1999). Bridging Physical and Virtual Worlds with Electronic Tags. (1999). Proc. CHI '99, Pittsburgh, PA. New York: ACM Press, 370--377. Google ScholarDigital Library
- Weiser, M. (1999) The computer for the 21st century. Scientific American (September 1991), 94--104.Google Scholar
- Wellner, P. (1991) The DigitalDesk calculator: tangible manipulation on a desk top display. Proc. UIST '91. New York: ACM Press, 27--33. Google ScholarDigital Library
- Wellner, P. (1993) Interacting with paper on the DigitalDesk. Communications of the ACM, 36 (7), 87--96. Google ScholarDigital Library
- Wigdor, D., Jiang, H., Forlines, C., Borkin, M. and Shen, C. (2009) WeSpace: the design development and deployment of a walk-up and share multi-surface visual collaboration system. Proc. CHI'09. ACM Press, 1237--1246. Google ScholarDigital Library
- Wilson, A.D. (2005) PlayAnywhere: a compact interactive tabletop projection-vision system. Proc. UIST '05. New York: ACM Press, 83--92. Google ScholarDigital Library
- Wilson, A.D., Izadi, S., Hilliges, O., Garcia-Mendoza, A. and Kirk, D. (2008) Bringing physics to the surface. Proc. UIST '08. New York: ACM Press, 67--76. Google ScholarDigital Library
- Wilson, A.D. and Sarin, R. (2007) BlueTable: connecting wireless mobile devices on interactive surfaces using vision-based handshaking. Proc. GI '07. New York: ACM Press, 119--125. Google ScholarDigital Library
- Wobbrock, J.O., Morris, M.R. and Wilson, A.D. (2009) User-defined gestures for surface computing. Proc. CHI '09. New York: ACM Press, 1083--1092. Google ScholarDigital Library
- Wu, M. and Balakrishnan, R. (2003) Multi-finger and whole hand gestural interaction techniques for multi-user tabletop displays. Proc. UIST '03. New York: ACM Press, 193--202. Google ScholarDigital Library
- Yang, R. and Welch, G. (2001) Automatic and continuous projector display surface calibration using everyday imagery. Proc. WSCG '01.Google Scholar
- Zhang, Z., Wu, Y., Shan, Y. and Shafer, S. (2001) Visual panel: virtual mouse, keyboard and 3D controller with an ordinary piece of paper. Proc. PUI '01. New York: ACM Press, 1--8. Google ScholarDigital Library
- Zhou, H., Xie, L. and Fang, X. (2007) Visual Mouse: SIFT Detection and PCA Recognition. Proc. CISW '07. Washington, DC: IEEE Computer Society, 263--266. Google ScholarDigital Library
- Ziola, R., Kellar, M. and Inkpen, K. (2007) DeskJockey: exploiting passive surfaces to display peripheral information. Proc. INTERACT '07. Heidelberg: Springer Berlin, 447--460. Google ScholarDigital Library
Index Terms
- Bonfire: a nomadic system for hybrid laptop-tabletop interaction
Recommendations
Pen + touch = new tools
UIST '10: Proceedings of the 23nd annual ACM symposium on User interface software and technologyWe describe techniques for direct pen+touch input. We observe people's manual behaviors with physical paper and notebooks. These serve as the foundation for a prototype Microsoft Surface application, centered on note-taking and scrapbooking of ...
Understanding Multi-touch Manipulation for Surface Computing
INTERACT '09: Proceedings of the 12th IFIP TC 13 International Conference on Human-Computer Interaction: Part IITwo-handed, multi-touch surface computing provides a scope for interactions that are closer analogues to physical interactions than classical windowed interfaces. The design of natural and intuitive gestures is a difficult problem as we do not know how ...
Eugenie: gestural and tangible interaction with active tokens for bio-design
UIST '14 Adjunct: Adjunct Proceedings of the 27th Annual ACM Symposium on User Interface Software and TechnologyWe present a case study of a tangible user interface that implements novel interaction techniques for the construction of complex queries in large data sets. Our interface, Eugenie, utilizes gestural interaction with active physical tokens and a multi-...
Comments