Matei Negulescu
ABSTRACT
The form factor of mobile devices remains small while their computing power grows at an accelerated rate. Prior work has explored expanding the output space by leveraging free displays in the environment. However, existing solutions often do not scale. In this paper we discuss Open Project, an end-to-end framework that allows a user to "project" a native mobile application onto a display using a phone camera, leveraging interaction spaces ranging from a PC monitor to a public wall-sized display. Any display becomes projectable instantaneously by simply accessing the lightweight Open Project server via a web browser. By distributing computation load onto each projecting mobile device, our framework easily scales for hosting many projection sessions and devices simultaneously. Our performance experiments and user studies indicated that Open Project supported a variety of useful collaborative, sharing scenarios and performed reliably in diverse settings.
Supplemental Material
- Arthur, R. and Olsen,Jr., D.R. XICE windowing toolkit: Seamless display annexation. ACM Trans. Computer-Human Interaction. 18, 3 (2011), 14:1--14:46. Google Scholar
Digital Library
- Ballagas, R., Rohs, M., and Sheridan, J.G. Sweep and point and shoot: phonecam-based interactions for large public displays. CHI '05 Extended Abstracts, 2005, 1200--1203. Google ScholarDigital Library
- Baur, D., Boring, S., and Feiner, S. Virtual projection: exploring optical projection as a metaphor for multi-device interaction. Proc. of CHI'12, ACM (2012), 1693--1702. Google ScholarDigital Library
- Beardsley, P., van Baar, J., Raskar, R., and Forlines, C. Interaction using a handheld projector. Computer Graphics and Applications, IEEE 25, 1 (2005), 39 --43. Google ScholarDigital Library
- Blasko, G., Feiner, S., and Coriand, F. Exploring Interaction with a Simulated Wrist-Worn Projection Display. Proc. of Wearable Computers, IEEE Computer Society (2005), 2--9. Google ScholarDigital Library
- Boring, S., Jurmu, M., and Butz, A. Scroll, tilt or move it: using mobile phones to continuously control pointers on large public displays. Proc. of OzCHI'09, 2009, 161--168. Google ScholarDigital Library
- Bragdon, A., DeLine, R., Hinckley, K., and Morris, M.R. Code space: touch + air gesture hybrid interactions for supporting developer meetings. Proc. of ITS'11, 2011, 212--22. Google ScholarDigital Library
- Cao, X. and Balakrishnan, R. Interacting with dynamically defined information spaces using a handheld projector and a pen. ACM Press (2006), 225. Google ScholarDigital Library
- Cao, X., Forlines, C., and Balakrishnan, R. Multi-user interaction using handheld projectors. Proc. of UIST'07, ACM (2007), 43--52. Google ScholarDigital Library
- Chang, T.-H. and Li, Y. Deep shot: a framework for migrating tasks across devices using mobile phone cameras. Proc. of CHI'11, ACM (2011), 2163--2172. Google ScholarDigital Library
- Fischler, M.A. and Bolles, R.C. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24, 6 (1981), 381--395. Google ScholarDigital Library
- Greenberg, S. and Rounding, M. The notification collage: posting information to public and personal displays. Proc. of CHI'01, ACM (2001), 514--521. Google ScholarDigital Library
- Herbert, L., Pears, N., Jackson, D., and Olivier, P. Mobile Device and Intelligent Display Interaction via Scale-invariant Image Feature Matching. PECCS, 2011, 207--214.Google Scholar
- Izadi, S., Brignull, H., Rodden, T., Rogers, Y., and Underwood, M. Dynamo: a public interactive surface supporting the cooperative sharing and exchange of media. Proc. of UIST'03, ACM (2003), 159--168. Google ScholarDigital Library
- Molyneaux, D., Izadi, S., Kim, D., et al. Interactive Environment-Aware Handheld Projectors for Pervasive Computing Spaces. Pervasive, (2012), 197--215. Google ScholarDigital Library
- Rekimoto, J. and Saitoh, M. Augmented surfaces: a spatially continuous work space for hybrid computing environments. Proc. of CHI'99, ACM (1999), 378--385. Google ScholarDigital Library
- Remondino, F. and Fraser, C. Digital camera calibration methods: considerations and comparisons. Photogrammetry, Remote Sensing and Spatial Information Sciences, (2006).Google Scholar
- Rohs, M. Real-world interaction with camera phones. Proc. of UCS'05, Springer-Verlag (2005), 74--89. Google ScholarDigital Library
- Shen, C., Everitt, K., and Ryall, K. UbiTable: Impromptu Face-to-Face Collaboration on Horizontal Interactive Surfaces. Proc. of UbiComp 2003, (2003), 281--288.Google Scholar
- Smith, G.M. and schraefel, m. c. The radial scroll tool: scrolling support for stylus- or touch-based document navigation. Proc. of UIST'04, ACM (2004), 53--56. Google ScholarDigital Library
- Tan, D.S., Meyers, B., and Czerwinski, M. WinCuts: manipulating arbitrary window regions for more effective use of screen space. CHI'04 extended abstracts, 2004, 1525--1528. Google ScholarDigital Library
- Wang, J., Zhai, S., and Canny, J. Camera phone based motion sensing: interaction techniques, applications and performance study. Proc. of UIST'06, ACM (2006), 101--110. Google ScholarDigital Library
- Wigdor, D., Jiang, H., Forlines, C., Borkin, M., and Shen, C. WeSpace: the design development and deployment of a walk-up and share multi-surface visual collaboration system. Proc. of CHI'09, ACM (2009), 1237--1246. Google ScholarDigital Library
- Wilson, A.D. and Benko, H. Combining multiple depth cameras and projectors for interactions on, above and between surfaces. Proc. of UIST'10, ACM (2010), 273--282. Google ScholarDigital Library
- OpenCV. http://opencv.org/.Google Scholar
- zxing - 1D/2D barcode image processing library. http://code.google.com/p/zxing/.Google Scholar
- Apache Tomcat http://tomcat.apache.org/.Google Scholar
Index Terms
- Open project: a lightweight framework for remote sharing of mobile applications
Recommendations
Touch & Interact: touch-based interaction with a tourist application
MobileHCI '08: Proceedings of the 10th international conference on Human computer interaction with mobile devices and servicesTouch & Interact is an interaction technique which combines mobile phones and public displays. The motivation for the project is to overcome the intrinsic output limitations of mobile phones. Touch & Interact extends the phone output to a public display ...
Impact of mobile tangible slider design and its reachability on pointing performance
IHM '16: Actes de la 28ième conference francophone sur l'Interaction Homme-MachineTouch interaction is almost omnipresent on mobile devices and technological advances have made screens bigger. The users can then interact with more content displayed on screen but it is more difficult for the users to reach all the parts of the screen ...
Touch & interact: touch-based interaction of mobile phones with displays
MobileHCI '08: Proceedings of the 10th international conference on Human computer interaction with mobile devices and servicesThe limited screen size and resolution of current mobile devices can still be problematic for map, multimedia and browsing applications. In this paper we present Touch & Interact: an interaction technique in which a mobile phone is able to touch a ...
Comments