Eshed Ohn-Bar
Cuong Tran
ABSTRACT
We present a real-time vision-based system that discriminates hand gestures performed by in-vehicle front-row seat occupants for accessing the infotainment system. The hand gesture-based visual user interface may be more natural and intuitive to the user than the current tactile interaction interface. Consequently, it may encourage a gaze-free interaction, which can alleviate driver distraction without limiting the user's infotainment experience. The system uses visible and depth images of the dashboard and center-console area in the vehicle. The first step in the algorithm uses the representation of the image area given by a modified histogram-of-oriented-gradients descriptor and a support vector machine (SVM) to classify whether the driver, passenger, or no one is interacting with the region of interest. The second step extracts gesture characteristics from temporal dynamics of the features derived in the initial step, which are then inputted to a SVM in order to perform gesture classification from a set of six classes of hand gestures. The rate of correct user classification into one of the three classes is 97.9% on average. Average hand gesture classification rates for the driver and passenger using color and depth input are above 94%. These rates were achieved on in-vehicle collected data over varying illumination conditions and human subjects. This approach demonstrates the feasibility of the hand gesture-based in-vehicle visual user interface.
- Chang, C.-C., and Lin, C.-J. LIBSVM: A library for support vector machines. ACM Transactions on Intelligent Systems and Technology 2 (2011), 27:1--27:27. Software available at http://www.csie.ntu.edu.tw/cjlin/libsvm. Google Scholar
Digital Library
- Cheng, S. Y., and Trivedi, M. M. Vision-based infotainment user determination by hand recognition for driver assistance. IEEE Transactions on Intelligent Transportation Systems 11, 3 (September 2010), 759--764. Google ScholarDigital Library
- Dalal, N., and Triggs, B. Histograms of oriented gradients for human detection. In Proc. IEEE CVPR (2005), vol. 1, pp. 886--893. Google ScholarDigital Library
- Doshi, A., Cheng, S. Y., and Trivedi, M. M. A novel active heads-up display for driver assistance. IEEE Transactions on Systems, Man, and Cybernetics, Part B 39, 1 (2009), 85--93. Google ScholarDigital Library
- Drews, F. A., Yazdani, H., Godfrey, C. N., Cooper, J. M., and Strayer, D. L. Text messaging during simulated driving. The Journal of the Human Factors and Ergonomics Society, 51 (January 2011), 762--770.Google ScholarCross Ref
- Ecker, R., Broy, V., Hertzschuch, K., and Butz, A. Visual cues supporting direct touch gesture interaction with in-vehicle information systems. In Proceedings of the 2nd International Conference on Automotive User Interfaces and Interactive Vehicular Applications (New York, NY, USA, 2010), AutomotiveUI '10, ACM, pp. 80--87. Google ScholarDigital Library
- Harter, J. J. E., Scharenbroch, G. K., Fultz, W. W., Griffin, D. P., and Witt, G. J. User discrimination control of vehicle infotainment system, u.s. patent 6 668 221.Google Scholar
- Horrey, W. J. Assesing the effects of in-vehicle tasks on driving performance. Ergonomics in Design, 19 (December 2011), 4--7.Google ScholarCross Ref
- Jahn, G., Krems, J. F., and Gelau, C. Skill aquisition while operating in-vehicle information systems: Interface design determines the level of safety-relevant distractions. Human Factors and Ergonomics Society, 51 (June 2009), 136--151.Google ScholarCross Ref
- Kolsch, M., and Turk, M. Analysis of rotational robustness of hand detection with viola jones method. In ICPR (2004), pp. 107--110. Google ScholarDigital Library
- Lee, J. D., Roberts, S. C., Hoffman, J. D., and Angell, L. S. Scrolling and driving: How an mp3 player and its aftermarket controller affect driving performance and visual behavior. The Journal of the Human Factors and Ergonomics Society (January 2012).Google Scholar
- Perez, M. A. Safety implications of infotainment system use in naturalistic driving. Work: A Journal of Prevention, Assessment and Rehabilitation 41 (2012), 4200--4204.Google Scholar
- Pitts, M. J., Williams, M. A., Wellings, T., and Attridge, A. Assessing subjective response to haptic feedback in automotive touchscreens. In Proceedings of the 1st International Conference on Automotive User Interfaces and Interactive Vehicular Applications (New York, NY, USA, 2009), AutomotiveUI '09, ACM, pp. 11--18. Google ScholarDigital Library
- Richter, H., Ecker, R., Deisler, C., and Butz, A. Haptouch and the 2+1 state model: potentials of haptic feedback on touch based in-vehicle information systems. In Proceedings of the 2nd International Conference on Automotive User Interfaces and Interactive Vehicular Applications (New York, NY, USA, 2010), AutomotiveUI '10, ACM, pp. 72--79. Google ScholarDigital Library
- Riener, A., and Wintersberger, P. Natural, intuitive finger-based input as a means of in-vehicle information system operation. In Automotive User Interfaces and Interactive Vehicular Applications (December 2011), pp. 159--166. Google ScholarDigital Library
- Tran, C., and Trivedi, M. Driver assistance for "keeping hands on the wheel and eyes on the road". In Vehicular Electronics and Safety (ICVES), 2009 IEEE International Conference on (nov. 2009), pp. 97--101.Google ScholarCross Ref
- Tran, C., and Trivedi, M. M. Vision for driver assistance: Looking at people in a vehicle. In Visual Analysis of Humans, T. B. Moeslund, A. Hilton, V. Kruger, and L. Sigal, Eds. Springer London, 2011, pp. 597--614.Google ScholarCross Ref
- Trivedi, M. M., and Cheng, S. Y. Holistic sensing and active displays for intelligent driver support systems. Computer 40, 5 (May 2007), 60--68. Google ScholarDigital Library
- Trivedi, M. M., Gandhi, T., and McCall, J. Looking-in and looking-out of a vehicle: Computer-vision-based enhanced vehicle safety. IEEE Trans. Intell. Transp. Syst. 8, 1 (Mar 2007), 108--120. Google ScholarDigital Library
- Williamson, A. R., Young, K. L., Navarro, J., and Lenne, M. G. Music selection using a touch screen interface: effect of auditory and visual feedback on driving and usability. International Journal of Vehicle Design 57, 4 (2011), 391--404.Google ScholarCross Ref
Index Terms
- Hand gesture-based visual user interface for infotainment
Recommendations
Vision-based infotainment user determination by hand recognition for driver assistance
We present a novel real-time computer-vision system that robustly discriminates which of the front-row seat occupants is accessing the infotainment controls. The knowledge of who is the user--that is, driver, passenger, or no one--can alleviate driver ...
Finger identification and hand gesture recognition techniques for natural user interface
APCHI '13: Proceedings of the 11th Asia Pacific Conference on Computer Human InteractionThe natural user interface using hand gesture have been popular field in Human-Computer-Interaction(HCI). Many research papers have been proposed in this field. They proposed vision-based, glove-based and depth-based approach for hand gesture ...
A Survey on Hand Gesture Recognition
CSA '13: Proceedings of the 2013 International Conference on Computer Sciences and ApplicationsHand gesture recognition has become one of the key techniques of human-computer interaction (HCI). Many researchers are devoted in this field. In this paper, firstly the history of hand gesture recognition is discussed and the technical difficulties are ...
Comments