Skip to main content

2017 | Supplement | Buchkapitel

On the Diffusion Process for Heart Rate Estimation from Face Videos Under Realistic Conditions

verfasst von : Christian S. Pilz, Jarek Krajewski, Vladimir Blazek

Erschienen in: Pattern Recognition

Verlag: Springer International Publishing

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

This work addresses the problem of estimating heart rate from face videos under real conditions using a model based on the recursive inference problem that leverages the local invariance of the heart rate. The proposed solution is based on the canonical state space representation of an Itō process and a Wiener velocity model. Empirical results yield to excellent real-time and estimation performance of heart rate in presence of disturbing factors, like rigid head motion, talking and facial expressions under natural illumination conditions making the process of heart rate estimation from face videos applicable in a much broader sense. To facilitate comparisons and to support research we made the code and data for reproducing the results public available.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Fußnoten
1
We also reimplemented other methods [14, 25, 41], since their code is not available. Unfortunately, we obtained worse results.
 
Literatur
1.
Zurück zum Zitat Arnold, I.: Ordinary Differential Equations. MIT Press, Cambridge (1973)MATH Arnold, I.: Ordinary Differential Equations. MIT Press, Cambridge (1973)MATH
2.
Zurück zum Zitat Bland, J., Altman, D.: Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 327(8476), 307–310 (1986)CrossRef Bland, J., Altman, D.: Statistical methods for assessing agreement between two methods of clinical measurement. Lancet 327(8476), 307–310 (1986)CrossRef
3.
Zurück zum Zitat Blanik, N., Blazek, C., Pereira, C., Blazek, V., Leonhardt, S.: Wearable photoplethysmographic sensors: past and present. In: Proceedings of the SPIE 9034, Medical Imaging: Image Processing (2014) Blanik, N., Blazek, C., Pereira, C., Blazek, V., Leonhardt, S.: Wearable photoplethysmographic sensors: past and present. In: Proceedings of the SPIE 9034, Medical Imaging: Image Processing (2014)
4.
Zurück zum Zitat Blazek, C., Hülsbusch, M.: Assessment of allergic skin reactions and their hemodynamical quantification using photoplethysmography imaging. In: Proceedings of 11th International Symposium CNVD, Computer-Aided Noninvasive Vascular Diagnostics, vol. 3, 85–90 (2005) Blazek, C., Hülsbusch, M.: Assessment of allergic skin reactions and their hemodynamical quantification using photoplethysmography imaging. In: Proceedings of 11th International Symposium CNVD, Computer-Aided Noninvasive Vascular Diagnostics, vol. 3, 85–90 (2005)
5.
Zurück zum Zitat Blazek, V.: Optoelektronische Erfassung und rechnerunterstützte Analyse der Mikrozirkulations-Rhythmik. Biomed. Techn. 30(1), 121–122 (1985)CrossRef Blazek, V.: Optoelektronische Erfassung und rechnerunterstützte Analyse der Mikrozirkulations-Rhythmik. Biomed. Techn. 30(1), 121–122 (1985)CrossRef
6.
Zurück zum Zitat Blazek, V., Blanik, N., Blazek, C., Paul, M., Pereira, C., Koeny, M., Venema, B., Leonhardt, S.: Active and passive optical imaging modality for unobtrusive cardiorespiratory monitoring and facial expressions assessment. Assessment. Anesth Analg. 124, 104–119 (2017)CrossRef Blazek, V., Blanik, N., Blazek, C., Paul, M., Pereira, C., Koeny, M., Venema, B., Leonhardt, S.: Active and passive optical imaging modality for unobtrusive cardiorespiratory monitoring and facial expressions assessment. Assessment. Anesth Analg. 124, 104–119 (2017)CrossRef
7.
Zurück zum Zitat Bloom, H., Bar-Shalom, Y.: The interacting multiple model algorithm for systems with markovian switching coefficients. IEEE Trans. Autom. Control 33(8), 780–783 (1988)CrossRefMATH Bloom, H., Bar-Shalom, Y.: The interacting multiple model algorithm for systems with markovian switching coefficients. IEEE Trans. Autom. Control 33(8), 780–783 (1988)CrossRefMATH
8.
9.
Zurück zum Zitat Cox, D.: Some statistical methods connected with series of events. J. Roy. Stat. Soc. 17(2), 129–164 (1950)MathSciNetMATH Cox, D.: Some statistical methods connected with series of events. J. Roy. Stat. Soc. 17(2), 129–164 (1950)MathSciNetMATH
10.
Zurück zum Zitat Durbin, J., Koopman, S.: Time Series Analysis by State Space Methods. Oxford University Press, Oxford (2001)MATH Durbin, J., Koopman, S.: Time Series Analysis by State Space Methods. Oxford University Press, Oxford (2001)MATH
11.
Zurück zum Zitat Feynman, R., Leighton, R., Sands, M.: The Feynman Lectures on Physics, vol. 1. Addison-Wesley, Boston (1963). Chap. 21MATH Feynman, R., Leighton, R., Sands, M.: The Feynman Lectures on Physics, vol. 1. Addison-Wesley, Boston (1963). Chap. 21MATH
12.
Zurück zum Zitat Gray, R., Neuhoff, D.: Quantization. IEEE Trans. Inf. Theory 44(6), 2325–2383 (1998)CrossRefMATH Gray, R., Neuhoff, D.: Quantization. IEEE Trans. Inf. Theory 44(6), 2325–2383 (1998)CrossRefMATH
13.
Zurück zum Zitat Grewal, M., Andrews, A.: Kalman Filtering Theory and Practice Using Matlab. Wiley Interscience, Hoboken (2001)MATH Grewal, M., Andrews, A.: Kalman Filtering Theory and Practice Using Matlab. Wiley Interscience, Hoboken (2001)MATH
14.
Zurück zum Zitat de Haan, G., Jeanne, V.: Robust pulse-rate from chrominance-based rppg. IEEE Trans. Biomed. Eng. 60(10), 2878–2886 (2014)CrossRef de Haan, G., Jeanne, V.: Robust pulse-rate from chrominance-based rppg. IEEE Trans. Biomed. Eng. 60(10), 2878–2886 (2014)CrossRef
15.
Zurück zum Zitat de Haan, G., van Leest, A.: Improved motion robustness of remote-ppg by using the blood volume pulse signature. Physiol. Meas. 3(9), 1913–1926 (2014)CrossRef de Haan, G., van Leest, A.: Improved motion robustness of remote-ppg by using the blood volume pulse signature. Physiol. Meas. 3(9), 1913–1926 (2014)CrossRef
16.
Zurück zum Zitat Hertzman, A.: Photoelectric plethysmography of the fingers and toes in man. Exp. Biol. Med. 37(3), 529–534 (1937)CrossRef Hertzman, A.: Photoelectric plethysmography of the fingers and toes in man. Exp. Biol. Med. 37(3), 529–534 (1937)CrossRef
17.
Zurück zum Zitat Hülsbusch, M.: A functional imaging technique for opto-electronic assessment of skin perfusion. Ph.D. thesis, RWTH Aachen University (2008) Hülsbusch, M.: A functional imaging technique for opto-electronic assessment of skin perfusion. Ph.D. thesis, RWTH Aachen University (2008)
18.
Zurück zum Zitat Itô, K.: On Stochastic Differential Equations, vol. 4. Memoris of The American Mathematical Society (1951) Itô, K.: On Stochastic Differential Equations, vol. 4. Memoris of The American Mathematical Society (1951)
19.
Zurück zum Zitat Jazwinski, A.: Stochastic Processes and Filtering Theory. Academic Press, New York (1970)MATH Jazwinski, A.: Stochastic Processes and Filtering Theory. Academic Press, New York (1970)MATH
20.
Zurück zum Zitat Jones, R.H.: Fitting multivariate models to unequally spaced data. In: Parzen, E. (ed.) Time Series Analysis of Irregularly Observed Data. LNS, vol. 25, pp. 158–188. Springer, New York (1984). doi:10.1007/978-1-4684-9403-7_8 CrossRef Jones, R.H.: Fitting multivariate models to unequally spaced data. In: Parzen, E. (ed.) Time Series Analysis of Irregularly Observed Data. LNS, vol. 25, pp. 158–188. Springer, New York (1984). doi:10.​1007/​978-1-4684-9403-7_​8 CrossRef
21.
Zurück zum Zitat Kalman, R., Bucy, R.: New results in linear filtering and prediction theory. Trans. ASME-J. Basic Eng. 83, 95–108 (1961)MathSciNetCrossRef Kalman, R., Bucy, R.: New results in linear filtering and prediction theory. Trans. ASME-J. Basic Eng. 83, 95–108 (1961)MathSciNetCrossRef
22.
Zurück zum Zitat Khintchine, A.: Korrelationstheorie der stationären stochastischen Prozesse. Springer-Mathematische Annalen 109, 604–615 (1934)MathSciNetCrossRefMATH Khintchine, A.: Korrelationstheorie der stationären stochastischen Prozesse. Springer-Mathematische Annalen 109, 604–615 (1934)MathSciNetCrossRefMATH
23.
Zurück zum Zitat Lam, A., Kuno, Y.: Robust heart rate measurement from video using select random patches. In: IEEE International Conference on Computer Vision, pp. 3640–3648 (2015) Lam, A., Kuno, Y.: Robust heart rate measurement from video using select random patches. In: IEEE International Conference on Computer Vision, pp. 3640–3648 (2015)
24.
Zurück zum Zitat Lewandowska, M., Ruminski, J., Kocejko, T., Nowak, J.: Measuring pulse rate with a webcam - a non-contact method for evaluating cardiac activity. In: Proceedings of the FedCSIS, Szczecin, Poland, pp. 405–410 (2011) Lewandowska, M., Ruminski, J., Kocejko, T., Nowak, J.: Measuring pulse rate with a webcam - a non-contact method for evaluating cardiac activity. In: Proceedings of the FedCSIS, Szczecin, Poland, pp. 405–410 (2011)
25.
Zurück zum Zitat Li, X., Chen, J., Zhao, G., Pietikinen, M.: Remote heart rate measurement from face videos under realistic situations. In: IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH (2014) Li, X., Chen, J., Zhao, G., Pietikinen, M.: Remote heart rate measurement from face videos under realistic situations. In: IEEE Conference on Computer Vision and Pattern Recognition, Columbus, OH (2014)
26.
Zurück zum Zitat Lomb, N.: Least-squares frequency analysis of unequally spaced data. Astrophys. Space Sci. 39(2), 447–462 (1976)CrossRef Lomb, N.: Least-squares frequency analysis of unequally spaced data. Astrophys. Space Sci. 39(2), 447–462 (1976)CrossRef
27.
Zurück zum Zitat Makhnin, O.: Filtering for some stochastic processes with discrete observations. Ph.D. thesis, Department of Statistics and Probability, Michigan State University (2002) Makhnin, O.: Filtering for some stochastic processes with discrete observations. Ph.D. thesis, Department of Statistics and Probability, Michigan State University (2002)
28.
Zurück zum Zitat McDuff, D., Gontarek, S., Picard, R.: Remote measurement of cognitive stress via heart rate variability. In: 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 2957–2960 (2014) McDuff, D., Gontarek, S., Picard, R.: Remote measurement of cognitive stress via heart rate variability. In: 36th Annual International Conference of the IEEE Engineering in Medicine and Biology Society, pp. 2957–2960 (2014)
29.
Zurück zum Zitat Moço, A., Stuijk, S., de Haan, G.: Ballistocardiographic artifacts in PPG imaging. IEEE Trans. Biomed. Eng. 63(9), 1804–1811 (2015)CrossRef Moço, A., Stuijk, S., de Haan, G.: Ballistocardiographic artifacts in PPG imaging. IEEE Trans. Biomed. Eng. 63(9), 1804–1811 (2015)CrossRef
30.
Zurück zum Zitat Moço, A., Stuijk, S., de Haan, G.: Motion robust PPG-imaging through color channel mapping. Biomed. Opt. Express 7, 1737–1754 (2016)CrossRef Moço, A., Stuijk, S., de Haan, G.: Motion robust PPG-imaging through color channel mapping. Biomed. Opt. Express 7, 1737–1754 (2016)CrossRef
31.
Zurück zum Zitat Molitor, H., Knaizuk, M.: A new bloodless method for continuous recording of peripheral change. J. Pharmacol. Exp. Theret. 27, 5–16 (1936) Molitor, H., Knaizuk, M.: A new bloodless method for continuous recording of peripheral change. J. Pharmacol. Exp. Theret. 27, 5–16 (1936)
32.
33.
Zurück zum Zitat Oliver, B., Pierce, J., Shannon, C.: The philosophy of PCM. Proc. IRE 36, 1324–1331 (1948)CrossRef Oliver, B., Pierce, J., Shannon, C.: The philosophy of PCM. Proc. IRE 36, 1324–1331 (1948)CrossRef
34.
Zurück zum Zitat Osman, A., Turcot, J., Kaliouby, R.E.: Supervised learning approach to remote heart rate estimation from facial videos. In: 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition, pp. 1–6 (2015) Osman, A., Turcot, J., Kaliouby, R.E.: Supervised learning approach to remote heart rate estimation from facial videos. In: 11th IEEE International Conference and Workshops on Automatic Face and Gesture Recognition, pp. 1–6 (2015)
35.
Zurück zum Zitat Poh, M., McDuff, J., Picard, R.: Non-contact, automated cardiac pulse measurements using video imaging and blind source separation. Opt. Express 18(10), 10762–10774 (2010)CrossRef Poh, M., McDuff, J., Picard, R.: Non-contact, automated cardiac pulse measurements using video imaging and blind source separation. Opt. Express 18(10), 10762–10774 (2010)CrossRef
36.
Zurück zum Zitat Ramirez, G., Fuentes, O., Crites, S., Jimenez, M., Ordonez, J.: Color analysis of facial skin: detection of emotional state. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 474–479 (2014) Ramirez, G., Fuentes, O., Crites, S., Jimenez, M., Ordonez, J.: Color analysis of facial skin: detection of emotional state. In: IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 474–479 (2014)
37.
Zurück zum Zitat Särkkä, S.: Recursive Bayesian inference on stochastic differential equations. Ph.D. thesis, Helsinki University of Technology (2006) Särkkä, S.: Recursive Bayesian inference on stochastic differential equations. Ph.D. thesis, Helsinki University of Technology (2006)
38.
Zurück zum Zitat Särkkä, S., Solin, A., Nummenmaa, A., Vehtari, T., Vanni, F.L.: Dynamic retrospective filtering of physiological noise in BOLD fMRI. NeuroImage 60(2), 1517–1527 (2012)CrossRef Särkkä, S., Solin, A., Nummenmaa, A., Vehtari, T., Vanni, F.L.: Dynamic retrospective filtering of physiological noise in BOLD fMRI. NeuroImage 60(2), 1517–1527 (2012)CrossRef
39.
Zurück zum Zitat Scargle, J.: Studies in astronomical time series analysis. II - statistical aspects of spectral analysis of unevenly spaced data. Astrophys. J. 263(1), 835–853 (1982)CrossRef Scargle, J.: Studies in astronomical time series analysis. II - statistical aspects of spectral analysis of unevenly spaced data. Astrophys. J. 263(1), 835–853 (1982)CrossRef
40.
Zurück zum Zitat Teplov, V.: Blood pulsation imaging. Ph.D. thesis, Department of Applied Physics, University of Eastern Finland (2014) Teplov, V.: Blood pulsation imaging. Ph.D. thesis, Department of Applied Physics, University of Eastern Finland (2014)
41.
Zurück zum Zitat Tulyakov, S., Pineda, X.A., Ricci, E., Yin, L., Cohn, J., Sebe, N.: Self-adaptive matrix completion for heart rate estimation from face videos under realistic conditions. In: Computer Vision and Pattern Recognition (2016) Tulyakov, S., Pineda, X.A., Ricci, E., Yin, L., Cohn, J., Sebe, N.: Self-adaptive matrix completion for heart rate estimation from face videos under realistic conditions. In: Computer Vision and Pattern Recognition (2016)
42.
Zurück zum Zitat Verkruysse, W., Svaasand, L., Nelson, J.: Remote plethysmographic imaging using ambient light. Opt. Express 16(26), 21434–21445 (2008)CrossRef Verkruysse, W., Svaasand, L., Nelson, J.: Remote plethysmographic imaging using ambient light. Opt. Express 16(26), 21434–21445 (2008)CrossRef
43.
Zurück zum Zitat Viola, P., Jones, M.: Robust real-time object detection. Int. J. Comput. Vis. 57, 137–154 (2001)CrossRef Viola, P., Jones, M.: Robust real-time object detection. Int. J. Comput. Vis. 57, 137–154 (2001)CrossRef
44.
Zurück zum Zitat Wang, W., Stuijk, S., de Haan, G.: A novel algorithm for remote photoplethysmography: spatial subspace rotation. IEEE Trans. Biomed. Eng. 63(9), 1974–1984 (2015)CrossRef Wang, W., Stuijk, S., de Haan, G.: A novel algorithm for remote photoplethysmography: spatial subspace rotation. IEEE Trans. Biomed. Eng. 63(9), 1974–1984 (2015)CrossRef
45.
Zurück zum Zitat Wiener, N.: The average of an analytical functional and the brownian movement. Proc. Nat. Acad. Sci. USA 7(1), 294–298 (1921)MathSciNetCrossRef Wiener, N.: The average of an analytical functional and the brownian movement. Proc. Nat. Acad. Sci. USA 7(1), 294–298 (1921)MathSciNetCrossRef
47.
Zurück zum Zitat Zakai, M.: On the optimal filtering of diffusion processes. Zeitschrift für Wahrscheinlichkeitstheorie und Verwandte Gebiete 11(3), 230–243 (1969)MathSciNetCrossRefMATH Zakai, M.: On the optimal filtering of diffusion processes. Zeitschrift für Wahrscheinlichkeitstheorie und Verwandte Gebiete 11(3), 230–243 (1969)MathSciNetCrossRefMATH
Metadaten
Titel
On the Diffusion Process for Heart Rate Estimation from Face Videos Under Realistic Conditions
verfasst von
Christian S. Pilz
Jarek Krajewski
Vladimir Blazek
Copyright-Jahr
2017
DOI
https://doi.org/10.1007/978-3-319-66709-6_29

Premium Partner