Skip to main content
Top
Published in: Soft Computing 2/2021

05-08-2020 | Methodologies and Application

Machine learning techniques to identify mind-wandering and predict hazard response time in fully immersive driving simulation

Authors: John Beninger, Andrew Hamilton-Wright, Heather E. K. Walker, Lana M. Trick

Published in: Soft Computing | Issue 2/2021

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

This work presents machine learning-based techniques for detecting mind-wandering and predicting hazard response time in driving using only easily measurable driving performance data (speed, horizontal and frontal acceleration, lane gap, and brake pressure). Such classifiers are relevant as research tools in the driving simulation community. We present a simple method, and a feature extraction-based method, of representing time-series driving performance data that both support machine learning-based predictions. We use the two types of representations to compare the effectiveness of support vector machines, random forest, and multi-layer perceptrons on data from 117 drives performed by 39 participants during a previous study in the high-fidelity driving simulator at the University of Guelph. Classification of mind-wandering and prediction of hazard response time was successful when compared to baseline measures. Specifically, random forest methods were most effective in both types of prediction and feature extraction supported the strongest random forest prediction of hazard response time. A discussion of the reasoning for this is included. While this has previously been studied using a single screen experimental setup, to our knowledge this is the first driving pattern-based classification of mind-wandering in a fully immersive driving simulator.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Footnotes
1
SciKit-learn version 0.20.3, Numpy version 1.16.2, Pandas version 0.24.2.
 
2
Scipy version 1.2.1.
 
Literature
go back to reference Albert DA, Ouimet MC, Jarret J, Cloutier MS, Paquette M, Badeau N, Brown TG (2018) Linking mind wandering tendency to risky driving in young male drivers. Accid Anal Prev 111:125–132CrossRef Albert DA, Ouimet MC, Jarret J, Cloutier MS, Paquette M, Badeau N, Brown TG (2018) Linking mind wandering tendency to risky driving in young male drivers. Accid Anal Prev 111:125–132CrossRef
go back to reference Baldwin CL, Roberts DM, Barragan D, Lee JD, Lerner N, Higgins JS (2017) Detecting and quantifying mind wandering during simulated driving. Front Hum Neurosci 11:406CrossRef Baldwin CL, Roberts DM, Barragan D, Lee JD, Lerner N, Higgins JS (2017) Detecting and quantifying mind wandering during simulated driving. Front Hum Neurosci 11:406CrossRef
go back to reference Bergasa LM, Nuevo J, Sotelo MA, Barea R, Lopez ME (2006) Real-time system for monitoring driver vigilance. IEEE Trans Intell Transp Syst 7(1):63–77CrossRef Bergasa LM, Nuevo J, Sotelo MA, Barea R, Lopez ME (2006) Real-time system for monitoring driver vigilance. IEEE Trans Intell Transp Syst 7(1):63–77CrossRef
go back to reference Blanchard N, Bixler R, Joyce T, D’Mello S (2014) Automated physiological-based detection of mind wandering during learning. In: International conference on intelligent tutoring systems. pp 55–60, Springer, Berlin Blanchard N, Bixler R, Joyce T, D’Mello S (2014) Automated physiological-based detection of mind wandering during learning. In: International conference on intelligent tutoring systems. pp 55–60, Springer, Berlin
go back to reference Burian SE, Liguori A, Robinson JH (2002) Effects of alcohol on risk-taking during simulated driving. Hum Psychopharmacol Clin Exp 17(3):141–150CrossRef Burian SE, Liguori A, Robinson JH (2002) Effects of alcohol on risk-taking during simulated driving. Hum Psychopharmacol Clin Exp 17(3):141–150CrossRef
go back to reference Christoff K, Gordon AM, Smallwood J, Smith R, Schooler JW (2009) Experience sampling during fMRI reveals default network and executive system contributions to mind wandering. Proc Nat Acad Sci 106(21):8719–8724CrossRef Christoff K, Gordon AM, Smallwood J, Smith R, Schooler JW (2009) Experience sampling during fMRI reveals default network and executive system contributions to mind wandering. Proc Nat Acad Sci 106(21):8719–8724CrossRef
go back to reference Dong Y, Hu Z, Uchimura K, Murayama N (2011) Driver inattention monitoring system for intelligent vehicles: a review. IEEE Trans Intell Transp Syst 12(2):596–614CrossRef Dong Y, Hu Z, Uchimura K, Murayama N (2011) Driver inattention monitoring system for intelligent vehicles: a review. IEEE Trans Intell Transp Syst 12(2):596–614CrossRef
go back to reference Durantin G, Dehais F, Delorme A (2015) Characterization of mind wandering using fNIRS. Front Syst Neurosci 9:45CrossRef Durantin G, Dehais F, Delorme A (2015) Characterization of mind wandering using fNIRS. Front Syst Neurosci 9:45CrossRef
go back to reference Eskandarian A, Sayed R, Delaigue P, Mortazavi A, Blum J (2007) Advanced driver fatigue research. Tech. rep., George Washington University National Crash Analysis Center, affiliated with: Federal Motor Carrier Safety Administration Eskandarian A, Sayed R, Delaigue P, Mortazavi A, Blum J (2007) Advanced driver fatigue research. Tech. rep., George Washington University National Crash Analysis Center, affiliated with: Federal Motor Carrier Safety Administration
go back to reference Filio D, Dony L, Gonzalez D, Oliver M (2017) Comparison of wrap around screens and hmds on a driver’s response to an unexpected pedestrian crossing using simulator vehicle parameters Filio D, Dony L, Gonzalez D, Oliver M (2017) Comparison of wrap around screens and hmds on a driver’s response to an unexpected pedestrian crossing using simulator vehicle parameters
go back to reference Fisher DL, Rizzo M, Caird J, Lee JD (2011) Handbook of driving simulation for engineering, medicine, and psychology. CRC Press, Boca RatonCrossRef Fisher DL, Rizzo M, Caird J, Lee JD (2011) Handbook of driving simulation for engineering, medicine, and psychology. CRC Press, Boca RatonCrossRef
go back to reference Freund Y, Schapire RE (1997) A decision-theoretic generalization of on-line learning and an application to boosting. J Comput Syst Sci 55(1):119–139MathSciNetCrossRef Freund Y, Schapire RE (1997) A decision-theoretic generalization of on-line learning and an application to boosting. J Comput Syst Sci 55(1):119–139MathSciNetCrossRef
go back to reference Galéra C, Orriols L, M’Bailara K, Laborey M, Contrand B, Ribéreau-Gayon R, Masson F, Bakiri S, Gabaude C, Fort A et al (2012) Mind wandering and driving: responsibility case-control study. BMJ 345:e8105CrossRef Galéra C, Orriols L, M’Bailara K, Laborey M, Contrand B, Ribéreau-Gayon R, Masson F, Bakiri S, Gabaude C, Fort A et al (2012) Mind wandering and driving: responsibility case-control study. BMJ 345:e8105CrossRef
go back to reference He J, Becic E, Lee YC, McCarley JS (2011) Mind wandering behind the wheel: performance and oculomotor correlates. Hum Factors 53(1):13–21CrossRef He J, Becic E, Lee YC, McCarley JS (2011) Mind wandering behind the wheel: performance and oculomotor correlates. Hum Factors 53(1):13–21CrossRef
go back to reference Killingsworth MA, Gilbert DT (2010) A wandering mind is an unhappy mind. Science 330(6006):932–932CrossRef Killingsworth MA, Gilbert DT (2010) A wandering mind is an unhappy mind. Science 330(6006):932–932CrossRef
go back to reference Liang Y, Lee JD Reyes ML (2007a) Nonintrusive detection of driver cognitive distraction in real time using bayesian networks. Transportation research record 2018(1):1–8 Liang Y, Lee JD Reyes ML (2007a) Nonintrusive detection of driver cognitive distraction in real time using bayesian networks. Transportation research record 2018(1):1–8
go back to reference Liang Y, Reyes ML, Lee JD (2007b) Real-time detection of driver cognitive distraction using support vector machines. IEEE Trans Intell Transp Syst 8(2):340–350CrossRef Liang Y, Reyes ML, Lee JD (2007b) Real-time detection of driver cognitive distraction using support vector machines. IEEE Trans Intell Transp Syst 8(2):340–350CrossRef
go back to reference Liu DC, Nocedal J (1989) On the limited memory bfgs method for large scale optimization. Math Program 45(1–3):503–528MathSciNetCrossRef Liu DC, Nocedal J (1989) On the limited memory bfgs method for large scale optimization. Math Program 45(1–3):503–528MathSciNetCrossRef
go back to reference Lochner MJ, Trick LM (2014) Multiple-object tracking while driving: the multiple-vehicle tracking task. Atten Percept Psychophys 76(8):2326–2345CrossRef Lochner MJ, Trick LM (2014) Multiple-object tracking while driving: the multiple-vehicle tracking task. Atten Percept Psychophys 76(8):2326–2345CrossRef
go back to reference McDonald AD, Lee JD, Schwarz C, Brown TL (2014) Steering in a random forest: Ensemble learning for detecting drowsiness-related lane departures. Hum Factors 56(5):986–998CrossRef McDonald AD, Lee JD, Schwarz C, Brown TL (2014) Steering in a random forest: Ensemble learning for detecting drowsiness-related lane departures. Hum Factors 56(5):986–998CrossRef
go back to reference Mrazek MD, Franklin MS, Phillips DT, Baird B, Schooler JW (2013) Mindfulness training improves working memory capacity and gre performance while reducing mind wandering. Psychol Sci 24(5):776–781CrossRef Mrazek MD, Franklin MS, Phillips DT, Baird B, Schooler JW (2013) Mindfulness training improves working memory capacity and gre performance while reducing mind wandering. Psychol Sci 24(5):776–781CrossRef
go back to reference Nowosielski RJ, Trick LM, Toxopeus R (2018) Good distractions: testing the effects of listening to an audiobook on driving performance in simple and complex road environments. Accid Anal Prev 111:202–209CrossRef Nowosielski RJ, Trick LM, Toxopeus R (2018) Good distractions: testing the effects of listening to an audiobook on driving performance in simple and complex road environments. Accid Anal Prev 111:202–209CrossRef
go back to reference Owens DA, Tyrrell RA (1999) Effects of luminance, blur, and age on nighttime visual guidance: A test of the selective degradation hypothesis. Journal of Experimental Psychology: Applied 5(2):115 Owens DA, Tyrrell RA (1999) Effects of luminance, blur, and age on nighttime visual guidance: A test of the selective degradation hypothesis. Journal of Experimental Psychology: Applied 5(2):115
go back to reference Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V, Vanderplas J, Passos A, Cournapeau D, Brucher M, Perrot M, Duchesnay E (2011) Scikit-learn: machine learning in Python. J Mach Learn Res 12:2825–2830MathSciNetMATH Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V, Vanderplas J, Passos A, Cournapeau D, Brucher M, Perrot M, Duchesnay E (2011) Scikit-learn: machine learning in Python. J Mach Learn Res 12:2825–2830MathSciNetMATH
go back to reference Raileanu LE, Stoffel K (2004) Theoretical comparison between the gini index and information gain criteria. Ann Math Artif Intell 41(1):77–93MathSciNetCrossRef Raileanu LE, Stoffel K (2004) Theoretical comparison between the gini index and information gain criteria. Ann Math Artif Intell 41(1):77–93MathSciNetCrossRef
go back to reference Razali NM, Wah YB et al (2011) Power comparisons of Shapiro–Wilk, Kolmogorov–Smirnov, Lilliefors and Anderson–Darling tests. J Stat Model Anal 2(1):21–33 Razali NM, Wah YB et al (2011) Power comparisons of Shapiro–Wilk, Kolmogorov–Smirnov, Lilliefors and Anderson–Darling tests. J Stat Model Anal 2(1):21–33
go back to reference Sathyanarayana A, Boyraz P, Hansen JH (2008) Driver behavior analysis and route recognition by hidden markov models. In: 2008 IEEE International Conference on Vehicular Electronics and Safety, IEEE, pp 276–281 Sathyanarayana A, Boyraz P, Hansen JH (2008) Driver behavior analysis and route recognition by hidden markov models. In: 2008 IEEE International Conference on Vehicular Electronics and Safety, IEEE, pp 276–281
go back to reference Sewell RA, Poling J, Sofuoglu M (2009) The effect of cannabis compared with alcohol on driving. Am J Addict 18(3):185–193CrossRef Sewell RA, Poling J, Sofuoglu M (2009) The effect of cannabis compared with alcohol on driving. Am J Addict 18(3):185–193CrossRef
go back to reference Sharma DG, Tanev I, Shimohara K (2016) Automatic classification of driving conditions for the detection of driver-induced steering oscillation. In: Proceedings of The Second International Conference on Electronics and Software Science (ICESS 2016), pp 88–95 Sharma DG, Tanev I, Shimohara K (2016) Automatic classification of driving conditions for the detection of driver-induced steering oscillation. In: Proceedings of The Second International Conference on Electronics and Software Science (ICESS 2016), pp 88–95
go back to reference Summala H, Nieminen T, Punto M (1996) Maintaining lane position with peripheral vision during in-vehicle tasks. Hum Factors 38(3):442–451CrossRef Summala H, Nieminen T, Punto M (1996) Maintaining lane position with peripheral vision during in-vehicle tasks. Hum Factors 38(3):442–451CrossRef
go back to reference Trick L, Toxopeus R, Rodd H, Nowosielski R, Khan A (2016) Using technology for road safety: Distraction monitoring and real time transition from driving assistance to automation function. Tech. rep., University of Guelph, in Association with Carleton University Trick L, Toxopeus R, Rodd H, Nowosielski R, Khan A (2016) Using technology for road safety: Distraction monitoring and real time transition from driving assistance to automation function. Tech. rep., University of Guelph, in Association with Carleton University
go back to reference Verwey WB, Zaidel DM (2000) Predicting drowsiness accidents from personal attributes, eye blinks and ongoing driving behaviour. Personal Individ Differ 28(1):123–142CrossRef Verwey WB, Zaidel DM (2000) Predicting drowsiness accidents from personal attributes, eye blinks and ongoing driving behaviour. Personal Individ Differ 28(1):123–142CrossRef
go back to reference Walker HE, Trick LM (2018) Mind-wandering while driving: the impact of fatigue, task length, and sustained attention abilities. Trans Res Part F Traffic Psychol Behav 59:81–97CrossRef Walker HE, Trick LM (2018) Mind-wandering while driving: the impact of fatigue, task length, and sustained attention abilities. Trans Res Part F Traffic Psychol Behav 59:81–97CrossRef
go back to reference Walker HE, Trick LM (2019) How the emotional content of roadside images affect driver attention and performance. Saf Sci 115:121–130CrossRef Walker HE, Trick LM (2019) How the emotional content of roadside images affect driver attention and performance. Saf Sci 115:121–130CrossRef
go back to reference Wickens CD (2002) Multiple resources and performance prediction. Theor Issues Ergon Sci 3(2):159–177CrossRef Wickens CD (2002) Multiple resources and performance prediction. Theor Issues Ergon Sci 3(2):159–177CrossRef
go back to reference Xu B, Wang N, Chen T, Li M (2015) Empirical evaluation of rectified activations in convolutional network. arXiv preprint arXiv:1505.00853 Xu B, Wang N, Chen T, Li M (2015) Empirical evaluation of rectified activations in convolutional network. arXiv preprint arXiv:​1505.​00853
go back to reference Yanko MR, Spalek TM (2013) Route familiarity breeds inattention: a driving simulator study. Accid Anal Prev 57:80–86CrossRef Yanko MR, Spalek TM (2013) Route familiarity breeds inattention: a driving simulator study. Accid Anal Prev 57:80–86CrossRef
go back to reference Yanko MR, Spalek TM (2014) Driving with the wandering mind: the effect that mind-wandering has on driving performance. Hum Factors 56(2):260–269CrossRef Yanko MR, Spalek TM (2014) Driving with the wandering mind: the effect that mind-wandering has on driving performance. Hum Factors 56(2):260–269CrossRef
go back to reference Yeo MV, Li X, Shen K, Wilder-Smith EP (2009) Can SVM be used for automatic EEG detection of drowsiness during car driving? Saf Sci 47(1):115–124CrossRef Yeo MV, Li X, Shen K, Wilder-Smith EP (2009) Can SVM be used for automatic EEG detection of drowsiness during car driving? Saf Sci 47(1):115–124CrossRef
go back to reference Zhang Y, Kumada T (2018) Automatic detection of mind wandering in a simulated driving task with behavioral measures. PLoS ONE 13(11):e0207092CrossRef Zhang Y, Kumada T (2018) Automatic detection of mind wandering in a simulated driving task with behavioral measures. PLoS ONE 13(11):e0207092CrossRef
Metadata
Title
Machine learning techniques to identify mind-wandering and predict hazard response time in fully immersive driving simulation
Authors
John Beninger
Andrew Hamilton-Wright
Heather E. K. Walker
Lana M. Trick
Publication date
05-08-2020
Publisher
Springer Berlin Heidelberg
Published in
Soft Computing / Issue 2/2021
Print ISSN: 1432-7643
Electronic ISSN: 1433-7479
DOI
https://doi.org/10.1007/s00500-020-05217-8

Other articles of this Issue 2/2021

Soft Computing 2/2021 Go to the issue

Premium Partner