Skip to main content

2018 | OriginalPaper | Buchkapitel

Robust and Real-Time Face Swapping Based on Face Segmentation and CANDIDE-3

verfasst von : Haosen Wang, Dongliang Xie, Lu Wei

Erschienen in: PRICAI 2018: Trends in Artificial Intelligence

Verlag: Springer International Publishing

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Despite some successes have been made in face swapping research, face swapping is still not robust and real-time enough. In this paper, a robust and real-time method for face swapping based on face segmentation and CANDIDE-3 is proposed. We implement our method through four steps: face detection, face alignment, modelling and swapping. We test our method on three publicly available datasets and some videos, and results show that our method can effectively improve the robust and real-time performance of face swapping.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Blanz, V., Scherbaum, K., Vetter, T., Seidel, H.P.: Exchanging faces in images. In: Computer Graphics Forum, pp. 669–676 (2004)CrossRef Blanz, V., Scherbaum, K., Vetter, T., Seidel, H.P.: Exchanging faces in images. In: Computer Graphics Forum, pp. 669–676 (2004)CrossRef
2.
Zurück zum Zitat Bitouk, D., Kumar, N., Dhillon, S., Belhumeur, P., Nayar, S.K.: Face swapping: automatically replacing faces in photographs. In: ACM SIGGRAPH, p. 39 (2008) Bitouk, D., Kumar, N., Dhillon, S., Belhumeur, P., Nayar, S.K.: Face swapping: automatically replacing faces in photographs. In: ACM SIGGRAPH, p. 39 (2008)
3.
Zurück zum Zitat Yang, F., Wang, J., Shechtman, E., Bourdev, L., Metaxas, D.: Expression flow for 3D-aware face component transfer. Acm Trans. Graph. 30(4), 1–10 (2011)CrossRef Yang, F., Wang, J., Shechtman, E., Bourdev, L., Metaxas, D.: Expression flow for 3D-aware face component transfer. Acm Trans. Graph. 30(4), 1–10 (2011)CrossRef
4.
Zurück zum Zitat Lin, Y., Wang, S., Lin, Q., Tang, F.: Face swapping under large pose variations: a 3D model based approach. In: 2012 IEEE International Conference on Multimedia and Expo (ICME), pp. 333–338. IEEE (2012) Lin, Y., Wang, S., Lin, Q., Tang, F.: Face swapping under large pose variations: a 3D model based approach. In: 2012 IEEE International Conference on Multimedia and Expo (ICME), pp. 333–338. IEEE (2012)
5.
Zurück zum Zitat Nirkin, Y., Masi, I., Tran, A.T., Hassner, T., Medioni, G.: On face segmentation, face swapping, and face perception. arXiv preprint arXiv:1704.06729 (2017) Nirkin, Y., Masi, I., Tran, A.T., Hassner, T., Medioni, G.: On face segmentation, face swapping, and face perception. arXiv preprint arXiv:​1704.​06729 (2017)
6.
Zurück zum Zitat De La Hunty, M., Asthana, A., Goecke, R.: Linear facial expression transfer with active appearance models. In: 2010 20th International Conference on Pattern Recognition (ICPR), pp. 3789–3792. IEEE (2010) De La Hunty, M., Asthana, A., Goecke, R.: Linear facial expression transfer with active appearance models. In: 2010 20th International Conference on Pattern Recognition (ICPR), pp. 3789–3792. IEEE (2010)
7.
Zurück zum Zitat Zhu, J., Van Gool, L., Hoi, S.C.: Unsupervised face alignment by robust nonrigid mapping. In: 2009 IEEE 12th International Conference on Computer Vision, pp. 1265–1272. IEEE (2009) Zhu, J., Van Gool, L., Hoi, S.C.: Unsupervised face alignment by robust nonrigid mapping. In: 2009 IEEE 12th International Conference on Computer Vision, pp. 1265–1272. IEEE (2009)
8.
Zurück zum Zitat Liu, S., Yang, J., Huang, C., Yang, M.H.: Multi-objective convolutional learning for face labeling. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3451–3459 (2015) Liu, S., Yang, J., Huang, C., Yang, M.H.: Multi-objective convolutional learning for face labeling. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3451–3459 (2015)
9.
Zurück zum Zitat Qian, K., Wang, B., Chen, H.: Automatic flexible face replacement with no auxiliary data. Comput. Graph. 45, 64–74 (2014)CrossRef Qian, K., Wang, B., Chen, H.: Automatic flexible face replacement with no auxiliary data. Comput. Graph. 45, 64–74 (2014)CrossRef
10.
Zurück zum Zitat King, D.E.: Dlib-ml: a machine learning toolkit. J. Mach. Learn. Res. 10, 1755–1758 (2009) King, D.E.: Dlib-ml: a machine learning toolkit. J. Mach. Learn. Res. 10, 1755–1758 (2009)
11.
Zurück zum Zitat Kazemi, V., Sullivan, J.: One millisecond face alignment with an ensemble of regression trees. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1867–1874 (2014) Kazemi, V., Sullivan, J.: One millisecond face alignment with an ensemble of regression trees. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1867–1874 (2014)
12.
Zurück zum Zitat Nguyen, A., Simard-Meilleur, A., Berthiaume, C., Godbout, R., Mottron, L.: Head circumference in canadian male adults: development of a normalized chart. Int. J. Morphol. 30(4), 1474–1480 (2012)CrossRef Nguyen, A., Simard-Meilleur, A., Berthiaume, C., Godbout, R., Mottron, L.: Head circumference in canadian male adults: development of a normalized chart. Int. J. Morphol. 30(4), 1474–1480 (2012)CrossRef
13.
Zurück zum Zitat Ahlberg, J.: CANDIDE-3 - an updated parameterised face. Rinsho Byori Jpn. J. Clin. Pathol. 48(3), 385–388 (2001) Ahlberg, J.: CANDIDE-3 - an updated parameterised face. Rinsho Byori Jpn. J. Clin. Pathol. 48(3), 385–388 (2001)
14.
Zurück zum Zitat Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Computer Vision and Pattern Recognition, pp. 3431–3440 (2015) Long, J., Shelhamer, E., Darrell, T.: Fully convolutional networks for semantic segmentation. In: Computer Vision and Pattern Recognition, pp. 3431–3440 (2015)
15.
Zurück zum Zitat Burgos-Artizzu, X.P., Perona, P., Dollar, P.: Robust face landmark estimation under occlusion. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1513–1520 (2013) Burgos-Artizzu, X.P., Perona, P., Dollar, P.: Robust face landmark estimation under occlusion. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1513–1520 (2013)
16.
Zurück zum Zitat Sagonas, C., Tzimiropoulos, G., Zafeiriou, S., Pantic, M.: 300 faces in-the-wild challenge: the first facial landmark localization challenge. In: IEEE International Conference on Computer Vision Workshops, pp. 397–403 (2014) Sagonas, C., Tzimiropoulos, G., Zafeiriou, S., Pantic, M.: 300 faces in-the-wild challenge: the first facial landmark localization challenge. In: IEEE International Conference on Computer Vision Workshops, pp. 397–403 (2014)
17.
Zurück zum Zitat Learned-Miller, E., Huang, G.B., Roychowdhury, A., Li, H., Hua, G.: Labeled Faces in the Wild: A Survey. Springer International Publishing, Heidelberg (2016) Learned-Miller, E., Huang, G.B., Roychowdhury, A., Li, H., Hua, G.: Labeled Faces in the Wild: A Survey. Springer International Publishing, Heidelberg (2016)
18.
Zurück zum Zitat Reinhard, E., Adhikhmin, M., Gooch, B., Shirley, P.: Color transfer between images. IEEE Comput. Graph. Appl. 21(5), 34–41 (2001)CrossRef Reinhard, E., Adhikhmin, M., Gooch, B., Shirley, P.: Color transfer between images. IEEE Comput. Graph. Appl. 21(5), 34–41 (2001)CrossRef
19.
Zurück zum Zitat Jia, Y., et al.: Caffe: convolutional architecture for fast feature embedding. In: Proceedings of the 22nd ACM International Conference on Multimedia, pp. 675–678. ACM (2014) Jia, Y., et al.: Caffe: convolutional architecture for fast feature embedding. In: Proceedings of the 22nd ACM International Conference on Multimedia, pp. 675–678. ACM (2014)
20.
Zurück zum Zitat Bradski, G.: The opencv library. Dr. Dobbs J. Softw. Tools Prof. Program. 25(11), 120–123 (2000) Bradski, G.: The opencv library. Dr. Dobbs J. Softw. Tools Prof. Program. 25(11), 120–123 (2000)
21.
Zurück zum Zitat Jia, X., Yang, H., Chan, K.P., Patras, I.: Structured semi-supervised forest for facial landmarks localization with face mask reasoning. In: BMVC (2014) Jia, X., Yang, H., Chan, K.P., Patras, I.: Structured semi-supervised forest for facial landmarks localization with face mask reasoning. In: BMVC (2014)
22.
Zurück zum Zitat Ghiasi, G., Fowlkes, C.: Using segmentation to predict the absence of occluded parts. In: British Machine Vision Conference, pp. 22.1–22.12 (2015) Ghiasi, G., Fowlkes, C.: Using segmentation to predict the absence of occluded parts. In: British Machine Vision Conference, pp. 22.1–22.12 (2015)
Metadaten
Titel
Robust and Real-Time Face Swapping Based on Face Segmentation and CANDIDE-3
verfasst von
Haosen Wang
Dongliang Xie
Lu Wei
Copyright-Jahr
2018
DOI
https://doi.org/10.1007/978-3-319-97310-4_38

Premium Partner