Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Published in:
Application of Image Classification for Fine-Grained Nudity Detection Read chapter Read first chapter

2019 | OriginalPaper | Chapter

Heart Rate Based Face Synthesis for Pulse Estimation

Authors : Umur Aybars Ciftci, Lijun Yin

Published in: Advances in Visual Computing

Publisher: Springer International Publishing

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

With the technological advancements in non-invasive heart rate (HR) detection, it becomes more feasible to estimate heart rate using commodity digital cameras. However, achieving high accuracy in HR estimation still remains a challenge. One of the bottlenecks is the lack of sufficient facial videos annotated with corresponding HR signals. In order to prevent this bottleneck, we propose to create videos enriched with different HR values from existing data sets with an attempt to increase the data size in a controllable manner. This paper presents a new method to generate facial videos with various heart rate values through a video synthesis procedure. Our method involves the synthesis of heart beat effects from skin colors of a face. New face video is generated with various heart rate values while taking identity information into account. The quality of the synthetic videos is evaluated by comparing to the original ground truth videos at the pixel level as well as by computing their differentiability across the synthetic face videos. Furthermore, the usability of the new data is assessed through the application of HR estimation from remote video approaches.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now
previous chapter Jitter-Free Registration for Unmanned Aerial Vehicle Videos
next chapter Light-Weight Novel View Synthesis for Casual Multiview Photography
Literature
1.
Aarts, L.A., Jeanne, V., Cleary, J.P., Lieber, C., Nelson, J.S., Oetomo, S.B., Verkruysse, W.: Non-contact heart rate monitoring utilizing camera photoplethysmography in the neonatal intensive care unit-a pilot study. Early Human Dev. 89(12), 943–948 (2013)CrossRef
2.
Al-Ali, A., Diab, M.K., Coverston, R., Maurer, G., Schmidt, J., Schulz, C.: Pulse oximetry sensor, 9 October 2007, patent 7,280,858
3.
Baltrusaitis, T., Robinson, P., Morency, L.P.: Openface: an open source facial behavior analysis toolkit. In: WACV, pp. 1–10. IEEE Computer Society (2016)
4.
Bao, S.D., Zhang, Y.T., Shen, L.F.: Physiological signal based entity authentication for body area sensor networks and mobile healthcare systems. In: IEEE Engineering in Medicine and Biology (2006)
5.
6.
Bobbia, S., Benezeth, Y., Dubois, J.: Remote photoplethysmography based on implicit living skin tissue segmentation. In: IEEE ICPR, pp. 361–365 (2016)
7.
Chekmenev, S.Y., Rara, H., Farag, A.A.: Non-contact, wavelet-based measurement of vital signs using thermal imaging. In: The 1st International Conference on Graphics, Vision, and Image Processing (GVIP), pp. 107–112 (2005)
8.
Chollet, F.: Xception: deep learning with depthwise separable convolutions. In: IEEE CVPR (2017)
9.
De Haan, G., Jeanne, V.: Robust pulse rate from chrominance-based rPPG. IEEE Trans. Biomed. Eng. 60(10), 2878–2886 (2013)CrossRef
10.
Feng, L., Po, L., Xu, X., Li, Y., Ma, R.: Motion-resistant remote imaging photoplethysmography based on the optical properties of skin. IEEE Trans. Circuits Syst. Video Technol. 25(5), 879–891 (2015)CrossRef
11.
Gibert, G., D’Alessandro, D., Lance, F.: Face detection method based on photoplethysmography. In: IEEE International Conference on Advanced Video and Signal Based Surveillance (AVSS) (2013)
12.
Gordon, J.W.: Certain molar movements of the human body produced by the circulation of the blood. J. Anat. Physiol. 11(Pt 3), 533–536 (1877)
13.
Hassan, M., et al.: Heart rate estimation using facial video: a review. Biomed. Signal Process. Control 38, 346–360 (2017)CrossRef
14.
Hsu, G., Ambikapathi, A., Chen, M.: Deep learning with time-frequency representation for pulse estimation from facial videos. In: 2017 IEEE International Joint Conference on Biometrics (IJCB), pp. 383–389, October 2017
15.
Hsu, Y., Lin, Y., Hsu, W.: Learning-based heart rate detection from remote photoplethysmography features. In: 2014 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 4433–4437 (2014)
16.
Inan, O.T., et al.: Ballistocardiography and seismocardiography: a review of recent advances. IEEE J. Biomed. Health Inf. 19(4), 1414–1427 (2015)CrossRef
17.
Jose, A.D., Collison, D.: The normal range and determinants of the intrinsic heart rate in man. Cardiovasc. Res. 4(2), 160–167 (1970)CrossRef
18.
Kakria, P., Tripathi, N.K., Kitipawang, P.: A real-time health monitoring system for remote cardiac patients using smartphone and wearable sensors. Int. J. Telemed. Appl. 2015, 373474 (2015)
19.
Kranjec, J., Beguš, S., Geršak, G., Drnovšek, J.: Non-contact heart rate and heart rate variability measurements: a review. Biomed. Sig. Process. Control 13, 102–112 (2014)CrossRef
20.
Kumar, M., Veeraraghavan, A., Sabharwal, A.: DistancePPG: robust non-contact vital signs monitoring using a camera. Biomed. Optics Express. 6(5), 1565–1588 (2015)CrossRef
21.
Lam, A., Kuno, Y.: Robust heart rate measurement from video using select random patches. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3640–3648 (2015)
22.
Lawson, G., Belcher, R., Dawes, G., Redman, C.: A comparison of ultrasound (with autocorrelation) and direct electrocardiogram fetal heart rate detector systems. Am. J. Obstet. Gynecol. 147(6), 721–722 (1983)CrossRef
23.
Lee, A., Kim, Y.: Photoplethysmography as a form of biometric authentication. In: IEEE SENSORS, pp. 1–2 (2015)
24.
25.
Lewandowska, M., Rumiński, J., Kocejko, T., Nowak, J.: Measuring pulse rate with a webcam a non-contact method for evaluating cardiac activity. In: IEEE Federated Conference on Computer Science and Information Systems (FedCSIS) (2011)
26.
Li, X., Chen, J., Zhao, G., Pietikäinen, M.: Remote heart rate measurement from face videos under realistic situations. In: 2014 IEEE Conference on Computer Vision and Pattern Recognition, pp. 4264–4271, June 2014
27.
Lu, G., Yang, F., Taylor, J.A., Stein, J.F.: A comparison of photoplethysmography and ECG recording to analyse heart rate variability in healthy subjects. J. Med. Eng. Technol. 33(8), 634–641 (2009)CrossRef
28.
Mestha, L.K., Kyal, S., Xu, B., Lewis, L.E., Kumar, V.: Towards continuous monitoring of pulse rate in neonatal intensive care unit with a webcam. In: IEEE Engineering in Medicine and Biology Society (EMBC) (2014)
29.
Niu, X., Han, H., Shan, S., Chen, X.: SynRhythm: learning a deep heart rate estimator from general to specific. In: 2018 24th International Conference on Pattern Recognition (ICPR), pp. 3580–3585 (2018)
30.
Poh, M.Z., McDuff, D.J., Picard, R.W.: Non-contact, automated cardiac pulse measurements using video imaging and blind source separation. Opt. Express 18(10), 10762–10774 (2010)CrossRef
31.
Poh, M.Z., McDuff, D.J., Picard, R.W.: Advancements in noncontact, multiparameter physiological measurements using a webcam. IEEE Trans. Biomed. Eng. 58(1), 7–11 (2011)CrossRef
32.
Rouast, P.V., Adam, M.T., Chiong, R., Cornforth, D., Lux, E.: Remote heart rate measurement using low-cost RGB face video: a technical literature review. Front. Comput. Sci. 12(5), 858–872 (2018)CrossRef
33.
Rouast, P.V., Adam, M.T.P., Cornforth, D.J., Lux, E., Weinhardt, C.: Using contactless heart rate measurements for real-time assessment of affective states. In: Davis, F.D., Riedl, R., vom Brocke, J., Léger, P.-M., Randolph, A.B. (eds.) Information Systems and Neuroscience. LNISO, vol. 16, pp. 157–163. Springer, Cham (2017). https://​doi.​org/​10.​1007/​978-3-319-41402-7_​20CrossRef
34.
Sameni, R., Clifford, G.D.: A review of fetal ECG signal processing; issues and promising directions. Open Pacing Electrophysiol. Ther. J. 3, 4 (2010)
35.
Scalise, L., Bernacchia, N., Ercoli, I., Marchionni, P.: Heart rate measurement in neonatal patients using a webcamera. In: IEEE International Symposium on Medical Measurements and Applications (MeMeA) (2012)
36.
Sodi-Pallares, D.: Electrophysiology of the heart. Am. J. Cardiol. (1961)
37.
Soleymani, M., Lichtenauer, J., Pun, T., Pantic, M.: A multimodal database for affect recognition and implicit tagging. IEEE Trans. Affect. Comput. 3(1), 42–55 (2012)CrossRef
38.
Tulyakov, S., Alameda-Pineda, X., Ricci, E., Yin, L., Cohn, J.F., Sebe, N.: Self-adaptive matrix completion for heart rate estimation from face videos under realistic conditions. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)
39.
Verkruysse, W., Svaasand, L.O., Nelson, J.S.: Remote plethysmographic imaging using ambient light. Opt. Express 16(26), 21434–21445 (2008)CrossRef
40.
Wang, W., den Brinker, A.C., Stuijk, S., de Haan, G.: Algorithmic principles of remote PPG. IEEE Trans. Biomed. Eng. 64(7), 1479–1491 (2017)CrossRef
41.
Wang, W., Stuijk, S., De Haan, G.: Unsupervised subject detection via remote PPG. IEEE Trans. Biomed. Eng. 62(11), 2629–2637 (2015)CrossRef
42.
Wang, W., Stuijk, S., De Haan, G.: A novel algorithm for remote photoplethysmography: spatial subspace rotation. IEEE Trans. Biomed. Eng. 63(9), 1974–1984 (2016)CrossRef
43.
Wang, W., Stuijk, S., de Haan, G.: Living-skin classification via remote-PPG. IEEE Trans. Biomed. Eng. 64(12), 2781–2792 (2017)CrossRef
44.
Yan, Y., Ma, X., Yao, L., Ouyang, J.: Noncontact measurement of heart rate using facial video illuminated under natural light and signal weighted analysis. Biomed. Mater. Eng. 26(s1), S903–S909 (2015)
45.
Yu, Y.P., Kwan, B.H., Lim, C.L., Wong, S.L., Raveendran, P.: Video-based heart rate measurement using short-time Fourier transform. In: Intelligent Signal Processing and Communications Systems (ISPACS), pp. 704–707. IEEE (2013)
46.
Zhang, Z., et al.: Multimodal spontaneous emotion corpus for human behavior analysis. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3438–3446 (2016)
Metadata
Title
Heart Rate Based Face Synthesis for Pulse Estimation
Authors
Umur Aybars Ciftci
Lijun Yin
Copyright Year
2019
Book
Advances in Visual Computing

Print ISBN: 978-3-030-33719-3

Electronic ISBN: 978-3-030-33720-9

Copyright Year: 2019

DOI
https://doi.org/10.1007/978-3-030-33720-9_42

Premium Partner

    Image Credits