Neel Joshi
ABSTRACT
We present a touch-free interface for viewing large imagery on mobile devices. In particular, we focus on viewing paradigms for 360 degree panoramas, parallax image sequences, and long multi-perspective panoramas. We describe a sensor fusion methodology that combines face tracking using a front-facing camera with gyroscope data to produce a robust signal that defines the viewer's 3D position relative to the display. The gyroscopic data provides both low-latency feedback and allows extrapolation of the face position beyond the the field-of-view of the front-facing camera. We also demonstrate a hybrid position and rate control that uses the viewer's 3D position to drive exploration of very large image spaces. We report on the efficacy of the hybrid control vs. position only control through a user study.
Supplemental Material
- Casiez, G., Vogel, D., Pan, Q., and Chaillou, C. Rubberedge: reducing clutching by combining position and rate control with elastic feedback. In Proceedings of the 20th annual ACM symposium on User interface software and technology, UIST '07, ACM (New York, NY, USA, 2007), 129--138. Google Scholar
Digital Library
- Eriksson, E., Hansen, T. R., and Lykke-Olesen, A. Movement-based interaction in camera spaces: a conceptual framework. Personal Ubiquitous Comput. 11 (December 2007), 621--632. Google ScholarDigital Library
- Hannuksela, J., Sangi, P., Turtinen, M., and Heikkilä, J. Face tracking for spatially aware mobile user interfaces. In Proceedings of the 3rd international conference on Image and Signal Processing, ICISP '08, Springer-Verlag (Berlin, Heidelberg, 2008), 405--412. Google ScholarDigital Library
- Hansen, T. R., Eriksson, E., and Lykke-Olesen, A. Use your head: exploring face tracking for mobile interaction. In CHI '06 extended abstracts on Human factors in computing systems, CHI EA '06, ACM (New York, NY, USA, 2006), 845--850. Google ScholarDigital Library
- Hinckley, K. Input technologies and techniques. In The Human-Computer Iteraction Handbook, A. Sears and J. A. Jacko, Eds. Addison Wesley, 2008, 161--176.Google Scholar
- Hinckley, K., Cutrell, E., Bathiche, S., and Muss, T. Quantitative analysis of scrolling techniques. In Proceedings of the SIGCHI conference on Human factors in computing systems: Changing our world, changing ourselves, CHI '02, ACM (New York, NY, USA, 2002), 65--72. Google ScholarDigital Library
- Hua, G., Yang, T.-Y., and Vasireddy, S. Peye: toward a visual motion based perceptual interface for mobile devices. In Proceedings of the 2007 IEEE international conference on Human-computer interaction, HCI'07, Springer-Verlag (Berlin, Heidelberg, 2007), 39--48. Google ScholarDigital Library
- Igarashi, T., and Hinckley, K. Speed-dependent automatic zooming for browsing large documents. In Proceedings of the 13th annual ACM symposium on User interface software and technology, UIST '00, ACM (New York, NY, USA, 2000), 139--148. Google ScholarDigital Library
- Ishak, E. W., and Feiner, S. K. Content-aware scrolling. In Proceedings of the 19th annual ACM symposium on User interface software and technology, UIST '06, ACM (New York, NY, USA, 2006), 155--158. Google ScholarDigital Library
- Karlson, A. K., and Bederson, B. B. Understanding single-handed mobile device interaction. Tech. rep., HCIL-2006-02, 2006.Google Scholar
- Karlson, A. K., Bederson, B. B., and Contreras-Vidal, J. L. Understanding One-Handed Use of Mobile Devices. In Handbook of Research on User Interface Design and Evaluation for Mobile Technology, J. Lumsden, Ed. Information Science Reference, 2008, ch. VI, 86--101.Google Scholar
- Kopf, J., Chen, B., Szeliski, R., and Cohen, M. Street slide: browsing street level imagery. ACM Trans. Graph. 29 (July 2010), 96:1--96:8. Google ScholarDigital Library
- Kumar, M., and Winograd, T. Gaze-enhanced scrolling techniques. In Proceedings of the 20th annual ACM symposium on User interface software and technology, UIST '07, ACM (New York, NY, USA, 2007), 213--216. Google ScholarDigital Library
- Olwal, A., Feiner, S., and Heyman, S. Rubbing and tapping for precise and rapid selection on touch-screen displays. In Proceeding of the twenty-sixth annual SIGCHI conference on Human factors in computing systems, CHI '08, ACM (New York, NY, USA, 2008), 295--304. Google ScholarDigital Library
- Premerlani, W., and Bizard, P. Direction cosine matrix imu: Theory. http://gentlenav.googlecode.com/files/DCMDraft2.pdf.Google Scholar
- Roudaut, A., Huot, S., and Lecolinet, E. Taptap and magstick: improving one-handed target acquisition on small touch-screens. In Proceedings of the working conference on Advanced visual interfaces, AVI '08, ACM (New York, NY, USA, 2008), 146--153. Google ScholarDigital Library
- Roudaut, A., Lecolinet, E., and Guiard, Y. Microrolls: expanding touch-screen input vocabulary by distinguishing rolls vs. slides of the thumb. In Proceedings of the 27th international conference on Human factors in computing systems, CHI '09, ACM (New York, NY, USA, 2009), 927--936. Google ScholarDigital Library
- Smith, G. M., and Schraefel, M. C. The radial scroll tool: scrolling support for stylus- or touch-based document navigation. In Proceedings of the 17th annual ACM symposium on User interface software and technology, UIST '04, ACM (New York, NY, USA, 2004), 53--56. Google ScholarDigital Library
- Tsang, M., Fitzmzurice, G. W., Kurtenbach, G., Khan, A., and Buxton, B. Boom chameleon: simultaneous capture of 3d viewpoint, voice and gesture annotations on a spatially-aware display. ACM Trans. Graph. 22 (July 2003), 698--698. Google ScholarDigital Library
- Viola, P., and Jones, M. J. Robust real-time face detection. Int. J. Comput. Vision 57 (May 2004), 137--154. Google ScholarDigital Library
- Zheng, K. C., Colburn, A., Agarwala, A., Agrawala, M., Salesin, D., Curless, B., and Cohen, M. F. Parallax photography: creating 3d cinematic effects from stills. In Proceedings of Graphics Interface 2009, GI '09, Canadian Information Processing Society (Toronto, Ont., Canada, Canada, 2009), 111--118. Google ScholarDigital Library
Index Terms
- Looking at you: fused gyro and face tracking for viewing large imagery on mobile devices
Recommendations
Catadioptric Stereo Using Planar Mirrors
By using mirror reflections of a scene, stereo images can be captured with a single camera (catadioptric stereo). In addition to simplifying data acquisition single camera stereo provides both geometric and radiometric advantages over traditional two ...
A 6-DOF ARTag-based tracking system
This paper presents a system to track position and orientation of a generic mobile device equipped with a camera using a set of variable size fiducial markers. The system provides six degrees-of-freedom (6-DOF) by tracking fiducial markers through the ...
Joint Depth and Color Camera Calibration with Distortion Correction
We present an algorithm that simultaneously calibrates two color cameras, a depth camera, and the relative pose between them. The method is designed to have three key features: accurate, practical, and applicable to a wide range of sensors. The method ...
Comments