Skip to main content

2017 | OriginalPaper | Buchkapitel

The SenseEmotion Database: A Multimodal Database for the Development and Systematic Validation of an Automatic Pain- and Emotion-Recognition System

verfasst von : Maria Velana, Sascha Gruss, Georg Layher, Patrick Thiam, Yan Zhang, Daniel Schork, Viktor Kessler, Sascha Meudt, Heiko Neumann, Jonghwa Kim, Friedhelm Schwenker, Elisabeth André, Harald C. Traue, Steffen Walter

Erschienen in: Multimodal Pattern Recognition of Social Signals in Human-Computer-Interaction

Verlag: Springer International Publishing

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

In our modern industrial society the group of the older (generation 65+) is constantly growing. Many subjects of this group are severely affected by their health and are suffering from disability and pain. The problem with chronic illness and pain is that it lowers the patient’s quality of life, and therefore accurate pain assessment is needed to facilitate effective pain management and treatment. In the future, automatic pain monitoring may enable health care professionals to assess and manage pain in a more and more objective way. To this end, the goal of our SenseEmotion project is to develop automatic pain- and emotion-recognition systems for successful assessment and effective personalized management of pain, particularly for the generation 65+. In this paper the recently created SenseEmotion Database for pain- vs. emotion-recognition is presented. Data of 45 healthy subjects is collected to this database. For each subject approximately 30 min of multimodal sensory data has been recorded. For a comprehensive understanding of pain and affect three rather different modalities of data are included in this study: biopotentials, camera images of the facial region, and, for the first time, audio signals. Heat stimulation is applied to elicit pain, and affective image stimuli accompanied by sound stimuli are used for the elicitation of emotional states.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Fußnoten
1
The IAPS identification numbers for the pleasant images were: 1650, 2216, 4311, 4611, 4658, 4659, 4664, 4676, 4677, 4690, 4694, 4695, 4800, 4810, 5460, 5470, 5626, 5629, 7502, 8030, 8080,8178, 8179, 8180, 8185, 8186, 8191, 8193, 8210, 8251, 8300, 8340, 8341, 8370, 8499, 8501. The identification numbers for the unpleasant images were: 1050, 1052, 1113, 1120, 1201, 1525, 1932, 2811, 3150, 3250, 3400, 3500, 5972, 6021, 6022, 6210, 6212, 6260, 6312, 6315, 6415, 6510, 6530, 6550, 6570, 6821, 8480, 8485, 9050, 9250, 9254, 9300, 9600, 9620, 9622, 9902, 9910, 9921. The identification number for the neutral images were: 5471, 5731, 6150, 7002, 7009, 7025, 7030, 7034, 7035, 7036, 7038, 7040, 7041, 7050, 7052, 7053, 7055, 7056, 7057, 7059, 7090, 7100, 7130, 7140, 7150, 7161, 7170, 7185, 7233, 7235, 7493, 7500, 7547, 7705. Mean valence and arousal ratings across image sets were: pleasant images (valance: M = 6.94, arousal: M = 6.40), unpleasant images (valance: M = 2.72, arousal: M = 6.42), and neutral images (valance: M = 5.03, arousal: M = 2.96).
 
2
The EmoPicS identification numbers for the pleasant images were: 006, 008, 028, 043, 050, 052, 053, 055, 056, 057, 058, 059, 061, 062, 063, 064, 065, 066, 067, 069, 070, 071, 075, 078. The identification numbers for the unpleasant images were: 207, 210, 211, 213, 214, 216, 219, 222, 229, 231, 232, 235, 238, 244, 250, 251, 252, 254, 321, 325, 326, 329. The identification numbers for the neutral images were: 123, 125, 127, 185, 188, 195, 196, 277, 281, 301, 302, 318, 335, 341, 342, 349, 354, 356, 365, 368, 371, 372, 373, 374, 376, 377. Mean valence and arousal ratings across image sets were: pleasant images (valance: M = 6.94, arousal: M = 5.57), unpleasant images (valance: M = 2.61, arousal: M = 6.26), and neutral images (valance: M = 5.02, arousal: M = 2.84).
 
Literatur
1.
Zurück zum Zitat Boiten, F.A., Frijda, N.H., Wientjes, C.J.E.: Emotions and respiratory patterns: review and critical analysis. Int. J. Psychophysiol. Official J. Int. Organ. Psychophysiol. 17(2), 103–128 (1994)CrossRef Boiten, F.A., Frijda, N.H., Wientjes, C.J.E.: Emotions and respiratory patterns: review and critical analysis. Int. J. Psychophysiol. Official J. Int. Organ. Psychophysiol. 17(2), 103–128 (1994)CrossRef
2.
Zurück zum Zitat Bradley, M.M., Codispoti, M., Cuthbert, B.N., Lang, P.J.: Emotion and motivation I: defensive and appetitive reactions in picture processing. Emotion (Washington, D.C.) 1(3), 276–298 (2001) Bradley, M.M., Codispoti, M., Cuthbert, B.N., Lang, P.J.: Emotion and motivation I: defensive and appetitive reactions in picture processing. Emotion (Washington, D.C.) 1(3), 276–298 (2001)
3.
Zurück zum Zitat Bradley, M.M., Lang, P.J.: Measuring emotion: the self-assessment manikin and the semantic differential. J. Behav. Ther. Exp. Psychiatry 25(1), 49–59 (1994)CrossRef Bradley, M.M., Lang, P.J.: Measuring emotion: the self-assessment manikin and the semantic differential. J. Behav. Ther. Exp. Psychiatry 25(1), 49–59 (1994)CrossRef
4.
Zurück zum Zitat Greenwald, M.K., Cook, E.W., Lang, P.J.: Affective judgment and psychophysiological response: dimensional covariation in the evaluation of pictorial stimuli. J. Psychophysiol. 3, 51–64 (1989) Greenwald, M.K., Cook, E.W., Lang, P.J.: Affective judgment and psychophysiological response: dimensional covariation in the evaluation of pictorial stimuli. J. Psychophysiol. 3, 51–64 (1989)
5.
Zurück zum Zitat Gruss, S.: Schmerzerkennung anhand psychophysiologischer Signale mithilfe maschineller Lerner. Dissertation, Universität Ulm (2015) Gruss, S.: Schmerzerkennung anhand psychophysiologischer Signale mithilfe maschineller Lerner. Dissertation, Universität Ulm (2015)
6.
Zurück zum Zitat Gruss, S., Treister, R., Werner, P., Traue, H.C., Crawcour, S., Andrade, A., Walter, S.: Pain intensity recognition rates via biopotential feature patterns with support vector machines. PLoS ONE 10(10), 1–14 (2015)CrossRef Gruss, S., Treister, R., Werner, P., Traue, H.C., Crawcour, S., Andrade, A., Walter, S.: Pain intensity recognition rates via biopotential feature patterns with support vector machines. PLoS ONE 10(10), 1–14 (2015)CrossRef
7.
Zurück zum Zitat Haag, A., Goronzy, S., Schaich, P., Williams, J.: Emotion recognition using bio-sensors: first steps towards an automatic system. In: André, E., Dybkjær, L., Minker, W., Heisterkamp, P. (eds.) ADS 2004. LNCS, vol. 3068, pp. 36–48. Springer, Heidelberg (2004). doi:10.1007/978-3-540-24842-2_4 CrossRef Haag, A., Goronzy, S., Schaich, P., Williams, J.: Emotion recognition using bio-sensors: first steps towards an automatic system. In: André, E., Dybkjær, L., Minker, W., Heisterkamp, P. (eds.) ADS 2004. LNCS, vol. 3068, pp. 36–48. Springer, Heidelberg (2004). doi:10.​1007/​978-3-540-24842-2_​4 CrossRef
8.
Zurück zum Zitat Hammal, Z., Cohn, J.F.: Automatic detection of pain intensity. In: Proceedings of the 14th ACM International Conference on Multimodal Interaction, ICMI 2012, pp. 47–52. ACM, New York (2012) Hammal, Z., Cohn, J.F.: Automatic detection of pain intensity. In: Proceedings of the 14th ACM International Conference on Multimodal Interaction, ICMI 2012, pp. 47–52. ACM, New York (2012)
9.
Zurück zum Zitat Healey, J.: Physiological sensing of emotion. In: Calvo, R., D’Mello, S., Gratch, J., Kappas, A., (eds.) The Oxford Handbook of Affective Computing, pp. 204–216. Oxford University Press, New York (2015) Healey, J.: Physiological sensing of emotion. In: Calvo, R., D’Mello, S., Gratch, J., Kappas, A., (eds.) The Oxford Handbook of Affective Computing, pp. 204–216. Oxford University Press, New York (2015)
10.
Zurück zum Zitat Jensen, C., Vasseljen, O., Westgaard, R.H.: The influence of electrode position on bipolar surface electromyogram recordings of the upper trapezius muscle. Eur. J. Appl. Physiol. 67(3), 266–273 (1993)CrossRef Jensen, C., Vasseljen, O., Westgaard, R.H.: The influence of electrode position on bipolar surface electromyogram recordings of the upper trapezius muscle. Eur. J. Appl. Physiol. 67(3), 266–273 (1993)CrossRef
11.
Zurück zum Zitat Kächele, M., Amirian, M., Thiam, P., Werner, P., Walter, S., Palm, G., Schwenker, F.: Adaptive confidence learning for the personalization of pain intensity estimation systems. Evolving Syst. 8(1), 71–83 (2017)CrossRef Kächele, M., Amirian, M., Thiam, P., Werner, P., Walter, S., Palm, G., Schwenker, F.: Adaptive confidence learning for the personalization of pain intensity estimation systems. Evolving Syst. 8(1), 71–83 (2017)CrossRef
12.
Zurück zum Zitat Kehlet, H.: Acute pain control and accelerated postoperative surgical recovery. Surg. Clin. North Am. 79(2), 431–443 (1999) Kehlet, H.: Acute pain control and accelerated postoperative surgical recovery. Surg. Clin. North Am. 79(2), 431–443 (1999)
13.
Zurück zum Zitat Kim, J., André, E.: Emotion recognition based on physiological changes in music listening. IEEE Trans. Pattern Anal. Mach. Intell. 30(12), 2067–2083 (2008)CrossRef Kim, J., André, E.: Emotion recognition based on physiological changes in music listening. IEEE Trans. Pattern Anal. Mach. Intell. 30(12), 2067–2083 (2008)CrossRef
14.
Zurück zum Zitat Lang, P.J.: Behavioral treatment and bio-behavioral assessment: Computer applications. In: Sidowski, J.B., Johnson, J.H., Williams, T.A. (eds.) Technology in Mental Health Care Delivery Systems, pp. 119–137. Ablex Publishing, Norwood (1980) Lang, P.J.: Behavioral treatment and bio-behavioral assessment: Computer applications. In: Sidowski, J.B., Johnson, J.H., Williams, T.A. (eds.) Technology in Mental Health Care Delivery Systems, pp. 119–137. Ablex Publishing, Norwood (1980)
15.
Zurück zum Zitat Lang, P.J.: The emotion probe: studies of motivation and attention. Am. Psychol. 50(5), 372–85 (1995)CrossRef Lang, P.J.: The emotion probe: studies of motivation and attention. Am. Psychol. 50(5), 372–85 (1995)CrossRef
16.
Zurück zum Zitat Lang, P.J., Bradley, M.M., Cuthbert, B.N.: International affective picture system (IAPS): Affective ratings of pictures and instruction manual. Technical report A-8, University of Florida, Gainesville, FL (2008) Lang, P.J., Bradley, M.M., Cuthbert, B.N.: International affective picture system (IAPS): Affective ratings of pictures and instruction manual. Technical report A-8, University of Florida, Gainesville, FL (2008)
17.
Zurück zum Zitat Lang, P.J., Greenwald, M.K., Bradley, M.M., Hamm, A.O.: Looking at pictures: affective, facial, visceral, and behavioral reactions. Psychophysiology 30(3), 261–273 (1993)CrossRef Lang, P.J., Greenwald, M.K., Bradley, M.M., Hamm, A.O.: Looking at pictures: affective, facial, visceral, and behavioral reactions. Psychophysiology 30(3), 261–273 (1993)CrossRef
18.
Zurück zum Zitat Lautenbacher, S.: Schmerzmessung. In: Basler, H.D., Franz, C., Kröner-Herwig, B., Rehfisch, H.P. (eds.) Psychologische Schmerztherapie, pp. 271–288. Springer, Berlin (2004) Lautenbacher, S.: Schmerzmessung. In: Basler, H.D., Franz, C., Kröner-Herwig, B., Rehfisch, H.P. (eds.) Psychologische Schmerztherapie, pp. 271–288. Springer, Berlin (2004)
19.
Zurück zum Zitat Limbrecht-Ecklundt, K., Werner, P., Traue, H.C., Walter, S.: Mimische Aktivität differenzierter Schmerzintensitäten. Korrelation der Merkmale von Facial Action Coding System und Elektromyografie. Der. Schmerz 30(3), 248–256 (2016)CrossRef Limbrecht-Ecklundt, K., Werner, P., Traue, H.C., Walter, S.: Mimische Aktivität differenzierter Schmerzintensitäten. Korrelation der Merkmale von Facial Action Coding System und Elektromyografie. Der. Schmerz 30(3), 248–256 (2016)CrossRef
20.
Zurück zum Zitat McQuay, H., Moore, A., Justins, D.: Treating acute pain in hospital. Br. Med. J. 314(7093), 1531–1535 (1997)CrossRef McQuay, H., Moore, A., Justins, D.: Treating acute pain in hospital. Br. Med. J. 314(7093), 1531–1535 (1997)CrossRef
21.
Zurück zum Zitat Meagher, M.W., Arnau, R.C., Rhudy, J.L.: Pain and emotion: effects of affective picture modulation. Psychosom. Med. 63(1), 79–90 (2001)CrossRef Meagher, M.W., Arnau, R.C., Rhudy, J.L.: Pain and emotion: effects of affective picture modulation. Psychosom. Med. 63(1), 79–90 (2001)CrossRef
22.
Zurück zum Zitat Medoc advanced medical systems (2009) Medoc advanced medical systems (2009)
23.
Zurück zum Zitat Rhudy, J.L., Williams, A.E., McCabe, K.M., Nguyen, M.A., Rambo, P.: Affective modulation of nociception at spinal and supraspinal levels. Psychophysiology 42(5), 579–587 (2005) Rhudy, J.L., Williams, A.E., McCabe, K.M., Nguyen, M.A., Rambo, P.: Affective modulation of nociception at spinal and supraspinal levels. Psychophysiology 42(5), 579–587 (2005)
24.
25.
Zurück zum Zitat Tan, J.W., Andrade, A.O., Li, H., Walter, S., Hrabal, D., Rukavina, S., Limbrecht-Ecklundt, K., Hoffman, H., Traue, H.C.: Recognition of intensive valence and arousal affective states via facial electromyographic activity in young and senior adults. PLoS ONE 11(1), 1–14 (2016) Tan, J.W., Andrade, A.O., Li, H., Walter, S., Hrabal, D., Rukavina, S., Limbrecht-Ecklundt, K., Hoffman, H., Traue, H.C.: Recognition of intensive valence and arousal affective states via facial electromyographic activity in young and senior adults. PLoS ONE 11(1), 1–14 (2016)
26.
Zurück zum Zitat Wagner, J., Lingenfelser, F., Baur, T., Damian, I., Kistler, F., André, E.: The social signal interpretation (SSI) framework: multimodal signal processing and recognition in real-time. In Proceedings of the 21st ACM International Conference on Multimedia - MM 2013, pp. 831–834. ACM Press, New York (2013) Wagner, J., Lingenfelser, F., Baur, T., Damian, I., Kistler, F., André, E.: The social signal interpretation (SSI) framework: multimodal signal processing and recognition in real-time. In Proceedings of the 21st ACM International Conference on Multimedia - MM 2013, pp. 831–834. ACM Press, New York (2013)
27.
Zurück zum Zitat Walter, S., Gruss, S., Ehleiter, H., Tan, J., Traue, H.C., Crawcour, S., Werner, P., Al-Hamadi, A., Andrade, A.O., Moreira da Silva, G.: The BioVid heat pain database - data for the advancement and systematic validation of an automated pain recognition system. In: 2013 IEEE International Conference on Cybernetics (CYBCONF), pp. 128–131. IEEE, June 2013 Walter, S., Gruss, S., Ehleiter, H., Tan, J., Traue, H.C., Crawcour, S., Werner, P., Al-Hamadi, A., Andrade, A.O., Moreira da Silva, G.: The BioVid heat pain database - data for the advancement and systematic validation of an automated pain recognition system. In: 2013 IEEE International Conference on Cybernetics (CYBCONF), pp. 128–131. IEEE, June 2013
28.
Zurück zum Zitat Walter, S., Gruss, S., Limbrecht-Ecklundt, K., Traue, H.C., Werner, P., Al-Hamadi, A., Diniz, N., Moreira, G., Andrade, A.O.: Automatic pain quantification using autonomic parameters. Psychol. Neurosci. 7(3), 363–380 (2014)CrossRef Walter, S., Gruss, S., Limbrecht-Ecklundt, K., Traue, H.C., Werner, P., Al-Hamadi, A., Diniz, N., Moreira, G., Andrade, A.O.: Automatic pain quantification using autonomic parameters. Psychol. Neurosci. 7(3), 363–380 (2014)CrossRef
29.
Zurück zum Zitat Werner, P., Al-Hamadi, A., Limbrecht-Ecklundt, K., Walter, S., Gruss, S., Traue, H.C.: Automatic pain assessment with facial activity descriptors. IEEE Trans. Affect. Comput. 99, 1–14 (2016)CrossRef Werner, P., Al-Hamadi, A., Limbrecht-Ecklundt, K., Walter, S., Gruss, S., Traue, H.C.: Automatic pain assessment with facial activity descriptors. IEEE Trans. Affect. Comput. 99, 1–14 (2016)CrossRef
30.
Zurück zum Zitat Wessa, M., Kanske, P., Neumeister, P., Bode, K., Heissler, J., Schönfelder, S.: EmoPics: Subjektive und psychophysiologische Evaluation neuen Bildmaterials für die klinisch-bio-psychologische Forschung. Zeitschrift für Klinische Psychologie und Psychotherapie 39(Suppl. 1/11), 77 (2010) Wessa, M., Kanske, P., Neumeister, P., Bode, K., Heissler, J., Schönfelder, S.: EmoPics: Subjektive und psychophysiologische Evaluation neuen Bildmaterials für die klinisch-bio-psychologische Forschung. Zeitschrift für Klinische Psychologie und Psychotherapie 39(Suppl. 1/11), 77 (2010)
31.
Zurück zum Zitat Wewers, M.E., Lowe, N.K.: A critical review of visual analogue scales in the measurement of clinical phenomena. Res. Nurs. Health 13(4), 227–236 (1990)CrossRef Wewers, M.E., Lowe, N.K.: A critical review of visual analogue scales in the measurement of clinical phenomena. Res. Nurs. Health 13(4), 227–236 (1990)CrossRef
32.
Zurück zum Zitat Zwakhalen, S.M.G., Hamers, J.P.H., Abu-Saad, H.H., Berger, M.P.F.: Pain in elderly people with severe dementia: a systematic review of behavioural pain assessment tools. BMC Geriatr. 6(3), 1–15 (2006) Zwakhalen, S.M.G., Hamers, J.P.H., Abu-Saad, H.H., Berger, M.P.F.: Pain in elderly people with severe dementia: a systematic review of behavioural pain assessment tools. BMC Geriatr. 6(3), 1–15 (2006)
Metadaten
Titel
The SenseEmotion Database: A Multimodal Database for the Development and Systematic Validation of an Automatic Pain- and Emotion-Recognition System
verfasst von
Maria Velana
Sascha Gruss
Georg Layher
Patrick Thiam
Yan Zhang
Daniel Schork
Viktor Kessler
Sascha Meudt
Heiko Neumann
Jonghwa Kim
Friedhelm Schwenker
Elisabeth André
Harald C. Traue
Steffen Walter
Copyright-Jahr
2017
DOI
https://doi.org/10.1007/978-3-319-59259-6_11