Skip to main content
Erschienen in: Soft Computing 2/2021

05.08.2020 | Methodologies and Application

Machine learning techniques to identify mind-wandering and predict hazard response time in fully immersive driving simulation

verfasst von: John Beninger, Andrew Hamilton-Wright, Heather E. K. Walker, Lana M. Trick

Erschienen in: Soft Computing | Ausgabe 2/2021

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

This work presents machine learning-based techniques for detecting mind-wandering and predicting hazard response time in driving using only easily measurable driving performance data (speed, horizontal and frontal acceleration, lane gap, and brake pressure). Such classifiers are relevant as research tools in the driving simulation community. We present a simple method, and a feature extraction-based method, of representing time-series driving performance data that both support machine learning-based predictions. We use the two types of representations to compare the effectiveness of support vector machines, random forest, and multi-layer perceptrons on data from 117 drives performed by 39 participants during a previous study in the high-fidelity driving simulator at the University of Guelph. Classification of mind-wandering and prediction of hazard response time was successful when compared to baseline measures. Specifically, random forest methods were most effective in both types of prediction and feature extraction supported the strongest random forest prediction of hazard response time. A discussion of the reasoning for this is included. While this has previously been studied using a single screen experimental setup, to our knowledge this is the first driving pattern-based classification of mind-wandering in a fully immersive driving simulator.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Fußnoten
1
SciKit-learn version 0.20.3, Numpy version 1.16.2, Pandas version 0.24.2.
 
2
Scipy version 1.2.1.
 
Literatur
Zurück zum Zitat Albert DA, Ouimet MC, Jarret J, Cloutier MS, Paquette M, Badeau N, Brown TG (2018) Linking mind wandering tendency to risky driving in young male drivers. Accid Anal Prev 111:125–132CrossRef Albert DA, Ouimet MC, Jarret J, Cloutier MS, Paquette M, Badeau N, Brown TG (2018) Linking mind wandering tendency to risky driving in young male drivers. Accid Anal Prev 111:125–132CrossRef
Zurück zum Zitat Baldwin CL, Roberts DM, Barragan D, Lee JD, Lerner N, Higgins JS (2017) Detecting and quantifying mind wandering during simulated driving. Front Hum Neurosci 11:406CrossRef Baldwin CL, Roberts DM, Barragan D, Lee JD, Lerner N, Higgins JS (2017) Detecting and quantifying mind wandering during simulated driving. Front Hum Neurosci 11:406CrossRef
Zurück zum Zitat Bergasa LM, Nuevo J, Sotelo MA, Barea R, Lopez ME (2006) Real-time system for monitoring driver vigilance. IEEE Trans Intell Transp Syst 7(1):63–77CrossRef Bergasa LM, Nuevo J, Sotelo MA, Barea R, Lopez ME (2006) Real-time system for monitoring driver vigilance. IEEE Trans Intell Transp Syst 7(1):63–77CrossRef
Zurück zum Zitat Blanchard N, Bixler R, Joyce T, D’Mello S (2014) Automated physiological-based detection of mind wandering during learning. In: International conference on intelligent tutoring systems. pp 55–60, Springer, Berlin Blanchard N, Bixler R, Joyce T, D’Mello S (2014) Automated physiological-based detection of mind wandering during learning. In: International conference on intelligent tutoring systems. pp 55–60, Springer, Berlin
Zurück zum Zitat Burian SE, Liguori A, Robinson JH (2002) Effects of alcohol on risk-taking during simulated driving. Hum Psychopharmacol Clin Exp 17(3):141–150CrossRef Burian SE, Liguori A, Robinson JH (2002) Effects of alcohol on risk-taking during simulated driving. Hum Psychopharmacol Clin Exp 17(3):141–150CrossRef
Zurück zum Zitat Christoff K, Gordon AM, Smallwood J, Smith R, Schooler JW (2009) Experience sampling during fMRI reveals default network and executive system contributions to mind wandering. Proc Nat Acad Sci 106(21):8719–8724CrossRef Christoff K, Gordon AM, Smallwood J, Smith R, Schooler JW (2009) Experience sampling during fMRI reveals default network and executive system contributions to mind wandering. Proc Nat Acad Sci 106(21):8719–8724CrossRef
Zurück zum Zitat Dong Y, Hu Z, Uchimura K, Murayama N (2011) Driver inattention monitoring system for intelligent vehicles: a review. IEEE Trans Intell Transp Syst 12(2):596–614CrossRef Dong Y, Hu Z, Uchimura K, Murayama N (2011) Driver inattention monitoring system for intelligent vehicles: a review. IEEE Trans Intell Transp Syst 12(2):596–614CrossRef
Zurück zum Zitat Durantin G, Dehais F, Delorme A (2015) Characterization of mind wandering using fNIRS. Front Syst Neurosci 9:45CrossRef Durantin G, Dehais F, Delorme A (2015) Characterization of mind wandering using fNIRS. Front Syst Neurosci 9:45CrossRef
Zurück zum Zitat Eskandarian A, Sayed R, Delaigue P, Mortazavi A, Blum J (2007) Advanced driver fatigue research. Tech. rep., George Washington University National Crash Analysis Center, affiliated with: Federal Motor Carrier Safety Administration Eskandarian A, Sayed R, Delaigue P, Mortazavi A, Blum J (2007) Advanced driver fatigue research. Tech. rep., George Washington University National Crash Analysis Center, affiliated with: Federal Motor Carrier Safety Administration
Zurück zum Zitat Filio D, Dony L, Gonzalez D, Oliver M (2017) Comparison of wrap around screens and hmds on a driver’s response to an unexpected pedestrian crossing using simulator vehicle parameters Filio D, Dony L, Gonzalez D, Oliver M (2017) Comparison of wrap around screens and hmds on a driver’s response to an unexpected pedestrian crossing using simulator vehicle parameters
Zurück zum Zitat Fisher DL, Rizzo M, Caird J, Lee JD (2011) Handbook of driving simulation for engineering, medicine, and psychology. CRC Press, Boca RatonCrossRef Fisher DL, Rizzo M, Caird J, Lee JD (2011) Handbook of driving simulation for engineering, medicine, and psychology. CRC Press, Boca RatonCrossRef
Zurück zum Zitat Freund Y, Schapire RE (1997) A decision-theoretic generalization of on-line learning and an application to boosting. J Comput Syst Sci 55(1):119–139MathSciNetCrossRef Freund Y, Schapire RE (1997) A decision-theoretic generalization of on-line learning and an application to boosting. J Comput Syst Sci 55(1):119–139MathSciNetCrossRef
Zurück zum Zitat Galéra C, Orriols L, M’Bailara K, Laborey M, Contrand B, Ribéreau-Gayon R, Masson F, Bakiri S, Gabaude C, Fort A et al (2012) Mind wandering and driving: responsibility case-control study. BMJ 345:e8105CrossRef Galéra C, Orriols L, M’Bailara K, Laborey M, Contrand B, Ribéreau-Gayon R, Masson F, Bakiri S, Gabaude C, Fort A et al (2012) Mind wandering and driving: responsibility case-control study. BMJ 345:e8105CrossRef
Zurück zum Zitat He J, Becic E, Lee YC, McCarley JS (2011) Mind wandering behind the wheel: performance and oculomotor correlates. Hum Factors 53(1):13–21CrossRef He J, Becic E, Lee YC, McCarley JS (2011) Mind wandering behind the wheel: performance and oculomotor correlates. Hum Factors 53(1):13–21CrossRef
Zurück zum Zitat Killingsworth MA, Gilbert DT (2010) A wandering mind is an unhappy mind. Science 330(6006):932–932CrossRef Killingsworth MA, Gilbert DT (2010) A wandering mind is an unhappy mind. Science 330(6006):932–932CrossRef
Zurück zum Zitat Liang Y, Lee JD Reyes ML (2007a) Nonintrusive detection of driver cognitive distraction in real time using bayesian networks. Transportation research record 2018(1):1–8 Liang Y, Lee JD Reyes ML (2007a) Nonintrusive detection of driver cognitive distraction in real time using bayesian networks. Transportation research record 2018(1):1–8
Zurück zum Zitat Liang Y, Reyes ML, Lee JD (2007b) Real-time detection of driver cognitive distraction using support vector machines. IEEE Trans Intell Transp Syst 8(2):340–350CrossRef Liang Y, Reyes ML, Lee JD (2007b) Real-time detection of driver cognitive distraction using support vector machines. IEEE Trans Intell Transp Syst 8(2):340–350CrossRef
Zurück zum Zitat Liu DC, Nocedal J (1989) On the limited memory bfgs method for large scale optimization. Math Program 45(1–3):503–528MathSciNetCrossRef Liu DC, Nocedal J (1989) On the limited memory bfgs method for large scale optimization. Math Program 45(1–3):503–528MathSciNetCrossRef
Zurück zum Zitat Lochner MJ, Trick LM (2014) Multiple-object tracking while driving: the multiple-vehicle tracking task. Atten Percept Psychophys 76(8):2326–2345CrossRef Lochner MJ, Trick LM (2014) Multiple-object tracking while driving: the multiple-vehicle tracking task. Atten Percept Psychophys 76(8):2326–2345CrossRef
Zurück zum Zitat McDonald AD, Lee JD, Schwarz C, Brown TL (2014) Steering in a random forest: Ensemble learning for detecting drowsiness-related lane departures. Hum Factors 56(5):986–998CrossRef McDonald AD, Lee JD, Schwarz C, Brown TL (2014) Steering in a random forest: Ensemble learning for detecting drowsiness-related lane departures. Hum Factors 56(5):986–998CrossRef
Zurück zum Zitat Mrazek MD, Franklin MS, Phillips DT, Baird B, Schooler JW (2013) Mindfulness training improves working memory capacity and gre performance while reducing mind wandering. Psychol Sci 24(5):776–781CrossRef Mrazek MD, Franklin MS, Phillips DT, Baird B, Schooler JW (2013) Mindfulness training improves working memory capacity and gre performance while reducing mind wandering. Psychol Sci 24(5):776–781CrossRef
Zurück zum Zitat Nowosielski RJ, Trick LM, Toxopeus R (2018) Good distractions: testing the effects of listening to an audiobook on driving performance in simple and complex road environments. Accid Anal Prev 111:202–209CrossRef Nowosielski RJ, Trick LM, Toxopeus R (2018) Good distractions: testing the effects of listening to an audiobook on driving performance in simple and complex road environments. Accid Anal Prev 111:202–209CrossRef
Zurück zum Zitat Owens DA, Tyrrell RA (1999) Effects of luminance, blur, and age on nighttime visual guidance: A test of the selective degradation hypothesis. Journal of Experimental Psychology: Applied 5(2):115 Owens DA, Tyrrell RA (1999) Effects of luminance, blur, and age on nighttime visual guidance: A test of the selective degradation hypothesis. Journal of Experimental Psychology: Applied 5(2):115
Zurück zum Zitat Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V, Vanderplas J, Passos A, Cournapeau D, Brucher M, Perrot M, Duchesnay E (2011) Scikit-learn: machine learning in Python. J Mach Learn Res 12:2825–2830MathSciNetMATH Pedregosa F, Varoquaux G, Gramfort A, Michel V, Thirion B, Grisel O, Blondel M, Prettenhofer P, Weiss R, Dubourg V, Vanderplas J, Passos A, Cournapeau D, Brucher M, Perrot M, Duchesnay E (2011) Scikit-learn: machine learning in Python. J Mach Learn Res 12:2825–2830MathSciNetMATH
Zurück zum Zitat Raileanu LE, Stoffel K (2004) Theoretical comparison between the gini index and information gain criteria. Ann Math Artif Intell 41(1):77–93MathSciNetCrossRef Raileanu LE, Stoffel K (2004) Theoretical comparison between the gini index and information gain criteria. Ann Math Artif Intell 41(1):77–93MathSciNetCrossRef
Zurück zum Zitat Razali NM, Wah YB et al (2011) Power comparisons of Shapiro–Wilk, Kolmogorov–Smirnov, Lilliefors and Anderson–Darling tests. J Stat Model Anal 2(1):21–33 Razali NM, Wah YB et al (2011) Power comparisons of Shapiro–Wilk, Kolmogorov–Smirnov, Lilliefors and Anderson–Darling tests. J Stat Model Anal 2(1):21–33
Zurück zum Zitat Sathyanarayana A, Boyraz P, Hansen JH (2008) Driver behavior analysis and route recognition by hidden markov models. In: 2008 IEEE International Conference on Vehicular Electronics and Safety, IEEE, pp 276–281 Sathyanarayana A, Boyraz P, Hansen JH (2008) Driver behavior analysis and route recognition by hidden markov models. In: 2008 IEEE International Conference on Vehicular Electronics and Safety, IEEE, pp 276–281
Zurück zum Zitat Sewell RA, Poling J, Sofuoglu M (2009) The effect of cannabis compared with alcohol on driving. Am J Addict 18(3):185–193CrossRef Sewell RA, Poling J, Sofuoglu M (2009) The effect of cannabis compared with alcohol on driving. Am J Addict 18(3):185–193CrossRef
Zurück zum Zitat Sharma DG, Tanev I, Shimohara K (2016) Automatic classification of driving conditions for the detection of driver-induced steering oscillation. In: Proceedings of The Second International Conference on Electronics and Software Science (ICESS 2016), pp 88–95 Sharma DG, Tanev I, Shimohara K (2016) Automatic classification of driving conditions for the detection of driver-induced steering oscillation. In: Proceedings of The Second International Conference on Electronics and Software Science (ICESS 2016), pp 88–95
Zurück zum Zitat Summala H, Nieminen T, Punto M (1996) Maintaining lane position with peripheral vision during in-vehicle tasks. Hum Factors 38(3):442–451CrossRef Summala H, Nieminen T, Punto M (1996) Maintaining lane position with peripheral vision during in-vehicle tasks. Hum Factors 38(3):442–451CrossRef
Zurück zum Zitat Trick L, Toxopeus R, Rodd H, Nowosielski R, Khan A (2016) Using technology for road safety: Distraction monitoring and real time transition from driving assistance to automation function. Tech. rep., University of Guelph, in Association with Carleton University Trick L, Toxopeus R, Rodd H, Nowosielski R, Khan A (2016) Using technology for road safety: Distraction monitoring and real time transition from driving assistance to automation function. Tech. rep., University of Guelph, in Association with Carleton University
Zurück zum Zitat Verwey WB, Zaidel DM (2000) Predicting drowsiness accidents from personal attributes, eye blinks and ongoing driving behaviour. Personal Individ Differ 28(1):123–142CrossRef Verwey WB, Zaidel DM (2000) Predicting drowsiness accidents from personal attributes, eye blinks and ongoing driving behaviour. Personal Individ Differ 28(1):123–142CrossRef
Zurück zum Zitat Walker HE, Trick LM (2018) Mind-wandering while driving: the impact of fatigue, task length, and sustained attention abilities. Trans Res Part F Traffic Psychol Behav 59:81–97CrossRef Walker HE, Trick LM (2018) Mind-wandering while driving: the impact of fatigue, task length, and sustained attention abilities. Trans Res Part F Traffic Psychol Behav 59:81–97CrossRef
Zurück zum Zitat Walker HE, Trick LM (2019) How the emotional content of roadside images affect driver attention and performance. Saf Sci 115:121–130CrossRef Walker HE, Trick LM (2019) How the emotional content of roadside images affect driver attention and performance. Saf Sci 115:121–130CrossRef
Zurück zum Zitat Wickens CD (2002) Multiple resources and performance prediction. Theor Issues Ergon Sci 3(2):159–177CrossRef Wickens CD (2002) Multiple resources and performance prediction. Theor Issues Ergon Sci 3(2):159–177CrossRef
Zurück zum Zitat Xu B, Wang N, Chen T, Li M (2015) Empirical evaluation of rectified activations in convolutional network. arXiv preprint arXiv:1505.00853 Xu B, Wang N, Chen T, Li M (2015) Empirical evaluation of rectified activations in convolutional network. arXiv preprint arXiv:​1505.​00853
Zurück zum Zitat Yanko MR, Spalek TM (2013) Route familiarity breeds inattention: a driving simulator study. Accid Anal Prev 57:80–86CrossRef Yanko MR, Spalek TM (2013) Route familiarity breeds inattention: a driving simulator study. Accid Anal Prev 57:80–86CrossRef
Zurück zum Zitat Yanko MR, Spalek TM (2014) Driving with the wandering mind: the effect that mind-wandering has on driving performance. Hum Factors 56(2):260–269CrossRef Yanko MR, Spalek TM (2014) Driving with the wandering mind: the effect that mind-wandering has on driving performance. Hum Factors 56(2):260–269CrossRef
Zurück zum Zitat Yeo MV, Li X, Shen K, Wilder-Smith EP (2009) Can SVM be used for automatic EEG detection of drowsiness during car driving? Saf Sci 47(1):115–124CrossRef Yeo MV, Li X, Shen K, Wilder-Smith EP (2009) Can SVM be used for automatic EEG detection of drowsiness during car driving? Saf Sci 47(1):115–124CrossRef
Zurück zum Zitat Zhang Y, Kumada T (2018) Automatic detection of mind wandering in a simulated driving task with behavioral measures. PLoS ONE 13(11):e0207092CrossRef Zhang Y, Kumada T (2018) Automatic detection of mind wandering in a simulated driving task with behavioral measures. PLoS ONE 13(11):e0207092CrossRef
Metadaten
Titel
Machine learning techniques to identify mind-wandering and predict hazard response time in fully immersive driving simulation
verfasst von
John Beninger
Andrew Hamilton-Wright
Heather E. K. Walker
Lana M. Trick
Publikationsdatum
05.08.2020
Verlag
Springer Berlin Heidelberg
Erschienen in
Soft Computing / Ausgabe 2/2021
Print ISSN: 1432-7643
Elektronische ISSN: 1433-7479
DOI
https://doi.org/10.1007/s00500-020-05217-8

Weitere Artikel der Ausgabe 2/2021

Soft Computing 2/2021 Zur Ausgabe

Premium Partner