Skip to main content
Top

2019 | OriginalPaper | Chapter

Detecting Driver Drowsiness in Real Time Through Deep Learning Based Object Detection

Authors : Muhammad Faique Shakeel, Nabit A. Bajwa, Ahmad Muhammad Anwaar, Anabia Sohail, Asifullah Khan, Haroon-ur-Rashid

Published in: Advances in Computational Intelligence

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Vehicle accidents due to drowsiness in drivers take thousands of lives each year worldwide. This fact clearly exhibits a need for a drowsiness detection application that can help prevent such accidents and ultimately save lives. In this work, we propose a novel deep learning methodology based on Convolutional Neural Networks (CNN) to tackle this problem. The proposed methodology treats drowsiness detection as an object detection task, and from an incoming video stream of a driver, detects and localizes open and closed eyes. MobileNet CNN architecture with Single Shot Multibox Detector (SSD) is used for this task of object detection. A separate algorithm is then used to detect driver drowsiness based on the output from the MobileNet-SSD architecture. In order to train the MobileNet-SSD Network a custom dataset of about 6000 images was compiled and labeled with the objects face, eye open and eye closed. Out of these, 350 images were randomly separated and used to test the trained model. The trained model was evaluated on the test dataset using the PASCAL VOC metric and achieved a Mean Average Precision (mAP) of 0.84 on these categories. The proposed methodology, while maintaining reasonable accuracy, is also computationally efficient and cost effective, as it can process an incoming video stream in real time on a standalone mobile device without the need of expensive hardware support. It can easily be deployed on cheap embedded devices in vehicles, such as the Raspberry Pi 3 or a mobile smartphone.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
4.
go back to reference Saini, V., Saini, R.: Driver drowsiness detection system and techniques: a review. Int. J. Comput. Sci. Inf. Technol. 5, 4245–4249 (2014) Saini, V., Saini, R.: Driver drowsiness detection system and techniques: a review. Int. J. Comput. Sci. Inf. Technol. 5, 4245–4249 (2014)
5.
go back to reference Bhatt, P.P., Trivedi, J.A.: Various methods for driver drowsiness detection: an overview. Int. J. Comput. Sci. Eng. 9, 70–74 (2017) Bhatt, P.P., Trivedi, J.A.: Various methods for driver drowsiness detection: an overview. Int. J. Comput. Sci. Eng. 9, 70–74 (2017)
6.
go back to reference Chieh, T.C., Mustafa, M.M., Hussain, A., Hendi, S.F., Majlis, B.Y.: Development of vehicle driver drowsiness detection system using Electrooculogram (EOG). In: 1st International Conference on Computers, Communications, and Signal Processing With Special Track on Biomedical Engineering (CCSP), Kuala Lumpur, Malaysia, pp. 165–168 (2005) Chieh, T.C., Mustafa, M.M., Hussain, A., Hendi, S.F., Majlis, B.Y.: Development of vehicle driver drowsiness detection system using Electrooculogram (EOG). In: 1st International Conference on Computers, Communications, and Signal Processing With Special Track on Biomedical Engineering (CCSP), Kuala Lumpur, Malaysia, pp. 165–168 (2005)
7.
go back to reference Takei, Y., Furukawa, Y.: Estimate of driver’s fatigue through steering motion. In: 2005 IEEE International Conference on Systems, Man and Cybernetics, vol. 2, pp. 1–6 (2005) Takei, Y., Furukawa, Y.: Estimate of driver’s fatigue through steering motion. In: 2005 IEEE International Conference on Systems, Man and Cybernetics, vol. 2, pp. 1–6 (2005)
8.
go back to reference Lee, A.: Comparing deep neural networks and traditional vision algorithms in mobile robotics. Swart. Coll. (2015) Lee, A.: Comparing deep neural networks and traditional vision algorithms in mobile robotics. Swart. Coll. (2015)
9.
go back to reference Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)CrossRef Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)CrossRef
10.
go back to reference Lazebnik, S., Schmid, C., Ponce, J.: Beyond bags of features: spatial pyramid matching for recognizing natural scene categories, pp. 2169–2178. IEEE Computer Society (2006) Lazebnik, S., Schmid, C., Ponce, J.: Beyond bags of features: spatial pyramid matching for recognizing natural scene categories, pp. 2169–2178. IEEE Computer Society (2006)
11.
go back to reference Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: Conference on Computer Vision and Pattern Recognition (2005) Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: Conference on Computer Vision and Pattern Recognition (2005)
13.
go back to reference Bergasa, L.M., Nuevo, J.: Real-time system for monitoring driver vigilance. In: Proceedings of the IEEE International Symposium on Industrial Electronics, vol. III, pp. 1303–1308 (2005) Bergasa, L.M., Nuevo, J.: Real-time system for monitoring driver vigilance. In: Proceedings of the IEEE International Symposium on Industrial Electronics, vol. III, pp. 1303–1308 (2005)
14.
go back to reference Suzuki, M., Yamamoto, N., Yamamoto, O., Nakano, T., Yamamoto, S.: Measurement of driver’s consciousness by image processing - a method for presuming driver’s drowsiness by eye-blinks coping with individual differences. In: Conference Proceedings - International Conference on Systems, Man and Cybernetics, vol. 4, pp. 2891–2896 (2007) Suzuki, M., Yamamoto, N., Yamamoto, O., Nakano, T., Yamamoto, S.: Measurement of driver’s consciousness by image processing - a method for presuming driver’s drowsiness by eye-blinks coping with individual differences. In: Conference Proceedings - International Conference on Systems, Man and Cybernetics, vol. 4, pp. 2891–2896 (2007)
16.
go back to reference Fan, C.: Driver fatigue detection based. In: Proceedings of the 2004 IEEE International Conference on Networking, Sensing and Control, pp. 7–12 (2004) Fan, C.: Driver fatigue detection based. In: Proceedings of the 2004 IEEE International Conference on Networking, Sensing and Control, pp. 7–12 (2004)
17.
go back to reference Abtahi, S., Hariri, B., Shirmohammadi, S.: Driver drowsiness monitoring based on yawning detection. In: IEEE International Instrumentation and Measurement Technology Conference, Binjiang, Hangzhou, China (2011) Abtahi, S., Hariri, B., Shirmohammadi, S.: Driver drowsiness monitoring based on yawning detection. In: IEEE International Instrumentation and Measurement Technology Conference, Binjiang, Hangzhou, China (2011)
18.
go back to reference Danisman, T., Bilasco, I.M., Djeraba, C., Ihaddadene, N.: Drowsy driver detection system using eye blink patterns. In: Proceedings of the 2010 International Conference on Machine and Web Intelligence, ICMWI 2010, pp. 230–233 (2010) Danisman, T., Bilasco, I.M., Djeraba, C., Ihaddadene, N.: Drowsy driver detection system using eye blink patterns. In: Proceedings of the 2010 International Conference on Machine and Web Intelligence, ICMWI 2010, pp. 230–233 (2010)
19.
go back to reference Bronte, S., Bergasa, L.M., Almaz, J., Yebes, J.: Vision-based drowsiness detector for real driving conditions (2012) Bronte, S., Bergasa, L.M., Almaz, J., Yebes, J.: Vision-based drowsiness detector for real driving conditions (2012)
20.
go back to reference Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Neural Information Processing Systems (2012) Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Neural Information Processing Systems (2012)
21.
go back to reference He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Conference on Computer Vision and Pattern Recognition (2016) He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Conference on Computer Vision and Pattern Recognition (2016)
22.
go back to reference Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. In: Neural Information Processing Systems, pp. 1–14 (2015) Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. In: Neural Information Processing Systems, pp. 1–14 (2015)
23.
go back to reference Shelhamer, E., Long, J., Darrell, T.: Fully convolutional networks for semantic segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 39, 640–651 (2017)CrossRef Shelhamer, E., Long, J., Darrell, T.: Fully convolutional networks for semantic segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 39, 640–651 (2017)CrossRef
24.
go back to reference Dwivedi, K., Biswaranjan, K., Sethi, A.: Drowsy driver detection using representation learning. In: Souvenir 2014 IEEE International Advanced Computing Conference, IACC 2014, pp. 995–999 (2014) Dwivedi, K., Biswaranjan, K., Sethi, A.: Drowsy driver detection using representation learning. In: Souvenir 2014 IEEE International Advanced Computing Conference, IACC 2014, pp. 995–999 (2014)
25.
go back to reference Reddy, B., Kim, Y.-H., Yun, S., Seo, C., Jang, J.: Real-time driver drowsiness detection for embedded system using model compression of deep neural networks. In: Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 438–445 (2017) Reddy, B., Kim, Y.-H., Yun, S., Seo, C., Jang, J.: Real-time driver drowsiness detection for embedded system using model compression of deep neural networks. In: Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 438–445 (2017)
27.
go back to reference Jabbar, R., Al-Khalifa, K., Kharbeche, M., Alhajyaseen, W., Jafari, M., Jiang, S.: Real-time driver drowsiness detection for android application using deep neural networks techniques. Procedia Comput. Sci. 130, 400–407 (2018)CrossRef Jabbar, R., Al-Khalifa, K., Kharbeche, M., Alhajyaseen, W., Jafari, M., Jiang, S.: Real-time driver drowsiness detection for android application using deep neural networks techniques. Procedia Comput. Sci. 130, 400–407 (2018)CrossRef
28.
go back to reference Lyu, J., Yuan, Z., Chen, D.: Long-term multi-granularity deep framework for driver drowsiness detection (2018) Lyu, J., Yuan, Z., Chen, D.: Long-term multi-granularity deep framework for driver drowsiness detection (2018)
29.
go back to reference Howard, A.G., et al.: MobileNets: efficient convolutional neural networks for mobile vision applications (2017) Howard, A.G., et al.: MobileNets: efficient convolutional neural networks for mobile vision applications (2017)
31.
go back to reference Huang, J., et al.: Speed/accuracy trade-offs for modern convolutional object detectors. In: Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, pp. 3296–3305, January 2017 Huang, J., et al.: Speed/accuracy trade-offs for modern convolutional object detectors. In: Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, pp. 3296–3305, January 2017
32.
go back to reference Schiffman, H.R.: Sensation and Perception: An Integrated Approach. Wiley, New York (2000) Schiffman, H.R.: Sensation and Perception: An Integrated Approach. Wiley, New York (2000)
33.
go back to reference Jain, V., Learned-Miller, E.: FDDB: a benchmark for face detection in unconstrained settings. UM-CS-2010-009 (2010) Jain, V., Learned-Miller, E.: FDDB: a benchmark for face detection in unconstrained settings. UM-CS-2010-009 (2010)
34.
go back to reference Abtahi, S., Omidyeganeh, M., Shirmohammadi, S., Hariri, B.: YawDD. In: Proceedings of the 5th ACM Multimedia Systems Conference on - MMSys 2014, pp. 24–28. ACM Press, New York (2014) Abtahi, S., Omidyeganeh, M., Shirmohammadi, S., Hariri, B.: YawDD. In: Proceedings of the 5th ACM Multimedia Systems Conference on - MMSys 2014, pp. 24–28. ACM Press, New York (2014)
36.
go back to reference Pan, S.J., Fellow, Q.Y.: A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22, 1345–1359 (2010)CrossRef Pan, S.J., Fellow, Q.Y.: A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22, 1345–1359 (2010)CrossRef
37.
go back to reference Oquab, M., Bottou, L., Laptev, I., Sivic, J.: Learning and transferring mid-level image representations using convolutional neural networks. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1717–1724 (2014) Oquab, M., Bottou, L., Laptev, I., Sivic, J.: Learning and transferring mid-level image representations using convolutional neural networks. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1717–1724 (2014)
39.
go back to reference Everingham, M., et al.: The PASCAL Visual Object Classes (VOC) challenge. Int. J. Comput. Vis. 88, 303–338 (2010)CrossRef Everingham, M., et al.: The PASCAL Visual Object Classes (VOC) challenge. Int. J. Comput. Vis. 88, 303–338 (2010)CrossRef
42.
go back to reference Krishnamoorthi, R.: Quantizing deep convolutional networks for efficient inference: a whitepaper (2018) Krishnamoorthi, R.: Quantizing deep convolutional networks for efficient inference: a whitepaper (2018)
Metadata
Title
Detecting Driver Drowsiness in Real Time Through Deep Learning Based Object Detection
Authors
Muhammad Faique Shakeel
Nabit A. Bajwa
Ahmad Muhammad Anwaar
Anabia Sohail
Asifullah Khan
Haroon-ur-Rashid
Copyright Year
2019
DOI
https://doi.org/10.1007/978-3-030-20521-8_24

Premium Partner