ABSTRACT
Continuous, real-time tracking of eye gaze is valuable in a variety of scenarios including hands-free interaction with the physical world, detection of unsafe behaviors, leveraging visual context for advertising, life logging, and others. While eye tracking is commonly used in clinical trials and user studies, it has not bridged the gap to everyday consumer use. The challenge is that a real-time eye tracker is a power-hungry and computation-intensive device which requires continuous sensing of the eye using an imager running at many tens of frames per second, and continuous processing of the image stream using sophisticated gaze estimation algorithms. Our key contribution is the design of an eye tracker that dramatically reduces the sensing and computation needs for eye tracking, thereby achieving orders of magnitude reductions in power consumption and form-factor. The key idea is that eye images are extremely redundant, therefore we can estimate gaze by using a small subset of carefully chosen pixels per frame. We instantiate this idea in a prototype hardware platform equipped with a low-power image sensor that provides random access to pixel values, a low-power ARM Cortex M3 microcontroller, and a bluetooth radio to communicate with a mobile phone. The sparse pixel-based gaze estimation algorithm is a multi-layer neural network learned using a state-of-the-art sparsity-inducing regularization function that minimizes the gaze prediction error while simultaneously minimizing the number of pixels used. Our results show that we can operate at roughly 70mW of power, while continuously estimating eye gaze at the rate of 30 Hz with errors of roughly 3 degrees.
- A neural-based remote eye gaze tracker under natural head motion. Computer Methods and Programs in Biomedicine, 92(1):66--78, 2008. Google ScholarDigital Library
- Applied Science Laboratories. NeXtGeneration Mobile Eye: Mobile Eye XG. http://www.asleyetracking.com/Site/Portals/0/MobileEyeXGwireless.pdf, 2013. Online; accessed April 7, 2013.Google Scholar
- S. Baluja and D. Pomerleau. Non-intrusive gaze tracking using artificial neural networks. Technical report, Pittsburgh, PA, USA, 1994. Google ScholarDigital Library
- C. M. Bishop. Neural networks for pattern recognition. Oxford university press, 1995. Google ScholarDigital Library
- D. Cheng and R. Vertegaal. An eye for an eye: a performance evaluation comparison of the lc technologies and tobii eye trackers. In Eye Tracking Research & Application: Proceedings of the 2004 symposium on Eye tracking research & applications, volume 22, pages 61--61, 2004. Google ScholarDigital Library
- A. T. Duchowski. Eye Tracking Methodology: Theory and Practice. Springer-Verlag New York, Inc., Secaucus, NJ, USA, 2007. Google ScholarDigital Library
- D. W. Hansen and Q. Ji. In the eye of the beholder: A survey of models for eyes and gaze. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 32(3):478--500, 2010. Google ScholarDigital Library
- T. Hastie, R. Tibshirani, J. Friedman, and J. Franklin. The elements of statistical learning: data mining, inference and prediction. The Mathematical Intelligencer, 27(2):83--85, 2005.Google ScholarCross Ref
- K. Holmqvist, M. Nyström, R. Andersson, R. Dewhurst, H. Jarodzka, and J. Van de Weijer. Eye tracking: A comprehensive guide to methods and measures. OUP Oxford, 2011.Google Scholar
- Invensense-9150. MPU-9150 Nine-Axis (Gyro + Accelerometer + Compass) MEMS MotionTracking Device. http: //www.invensense.com/mems/gyro/mpu9150.html, 2013.Google Scholar
- iShadow. iShadow Videos. http://sensors.cs.umass.edu/projects/eyeglass/, 2013.Google Scholar
- Y. Ishiguro, A. Mujibiya, T. Miyaki, and J. Rekimoto. Aide eyes: eye activity sensing for daily life. In Proceedings of the 1st Augmented Human International Conference, page 25. ACM, 2010. Google ScholarDigital Library
- D. Li, J. Babcock, and D. J. Parkhurst. openeyes: a low-cost head-mounted eye-tracking solution. In Proceedings of the 2006 symposium on Eye tracking research & applications, pages 95--100. ACM, 2006. Google ScholarDigital Library
- C. Morimoto and M. Mimica. Eye gaze tracking techniques for interactive applications. Computer Vision and Image Understanding, 98(1):4--24, 2005. Google ScholarDigital Library
- J. Nocedal and S. J. Wright. Numerical optimization. Springer Science+ Business Media, 2006.Google Scholar
- V. Rantanen, T. Vanhala, O. Tuisku, P. Niemenlehto, J. Verho, V. Surakka, M. Juhola, and J. Lekkala. A wearable, wireless gaze tracker with integrated selection command source for human-computer interaction. Information Technology in Biomedicine, IEEE Transactions on, 15(5):795--801, 2011. Google ScholarDigital Library
- D. E. Rumelhart, G. E. Hinton, and R. J. Williams. Learning representations by back-propagating errors. Nature, 323(Oct):533--536+, 1986.Google Scholar
- W. Sewell and O. Komogortsev. Real-time eye gaze tracking with an unmodified commodity webcam employing a neural network. In CHI'10 Extended Abstracts on Human Factors in Computing Systems, pages 3739--3744. ACM, 2010. Google ScholarDigital Library
- STM32. STM32 32-bit ARM Cortex MCUs. http: //www.st.com/web/en/catalog/mmc/FM141/SC1169, 2013.Google Scholar
- Stonyman. Stonyman Vision Chip. http://centeye.com/products/stonyman-vision-chip-breakout-board/, 2013.Google Scholar
- K.-H. Tan, D. Kriegman, and N. Ahuja. Appearance-based eye gaze estimation. In Proceedings of the Sixth IEEE Workshop on Applications of Computer Vision, 2002., pages 191--195, 2002. Google ScholarDigital Library
- R. Tibshirani. Regression shrinkage and selection via the lasso. Journal of the Royal Statistical Society. Series B (Methodological), pages 267--288, 1996.Google ScholarCross Ref
- Tobii. Tobii EyeX Controller. http://www.tobii.com/eye experience/, 2013.Google Scholar
- Tobii Technology. Tobii Glasses Eye Tracker. Online, 2013. Online; accessed April 7, 2013.Google Scholar
- L. Young and D. Sheena. Survey of eye movement recording methods. Behavior Research Methods, 7(5):397--429, 1975.Google ScholarCross Ref
- M. Yuan and Y. Lin. Model selection and estimation in regression with grouped variables. Journal of the Royal Statistical Society: Series B (Statistical Methodology), 68(1):49--67, 2006.Google Scholar
Index Terms
- iShadow: design of a wearable, real-time mobile gaze tracker
Recommendations
CIDER: Enabling Robustness-Power Tradeoffs on a Computational Eyeglass
MobiCom '15: Proceedings of the 21st Annual International Conference on Mobile Computing and NetworkingThe human eye offers a fascinating window into an individual's health, cognitive attention, and decision making, but we lack the ability to continually measure these parameters in the natural environment. The challenges lie in: a) handling the ...
iShadow: the computational eyeglass system
ETRA '14: Proceedings of the Symposium on Eye Tracking Research and ApplicationsContinuous, real-time tracking of eye gaze is valuable in a variety of scenarios including hands-free interaction with the physical world, detection of unsafe behaviors, leveraging visual context for advertising, life logging, and others. While eye ...
Analysing EOG signal features for the discrimination of eye movements with wearable devices
PETMEI '11: Proceedings of the 1st international workshop on pervasive eye tracking & mobile eye-based interactionEye tracking research in human-computer interaction and experimental psychology traditionally focuses on stationary devices and a small number of common eye movements. The advent of pervasive eye tracking promises new applications, such as eye-based ...
Comments