Skip to main content
Top

2019 | OriginalPaper | Chapter

Face Swapping for Solving Collateral Privacy Issues in Multimedia Analytics

Author : Werner Bailer

Published in: MultiMedia Modeling

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

A wide range of components of multimedia analytics systems relies on visual content that is used for supervised (e.g., classification) and unsupervised (e.g., clustering) machine learning methods. This content may contain privacy sensitive information, e.g., show faces of persons. In many cases it is just an inevitable side-effect that persons appear in the content, and the application may not require identification – a situation which we call “collateral privacy issues”. We propose de-identification of faces in images by using a generative adversarial network to generate new face images, and use them to replace faces in the original images. We demonstrate that face swapping does not impact the performance of visual descriptor matching and extraction.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Agarwal, A., Singh, R., Vatsa, M., Noore, A.: Swapped! digital face presentation attack detection via weighted local magnitude pattern. In: 2017 IEEE International Joint Conference on Biometrics (IJCB), pp. 659–665. IEEE (2017) Agarwal, A., Singh, R., Vatsa, M., Noore, A.: Swapped! digital face presentation attack detection via weighted local magnitude pattern. In: 2017 IEEE International Joint Conference on Biometrics (IJCB), pp. 659–665. IEEE (2017)
2.
go back to reference Ahonen, T., Rahtu, E., Ojansivu, V., Heikkila, J.: Recognition of blurred faces using local phase quantization. In: 19th International Conference on Pattern Recognition, ICPR 2008, pp. 1–4. IEEE (2008) Ahonen, T., Rahtu, E., Ojansivu, V., Heikkila, J.: Recognition of blurred faces using local phase quantization. In: 19th International Conference on Pattern Recognition, ICPR 2008, pp. 1–4. IEEE (2008)
3.
go back to reference Badii, A., Einig, M., Piatrik, T., et al.: Overview of the mediaeval 2013 visual privacy task. In: MediaEval (2014) Badii, A., Einig, M., Piatrik, T., et al.: Overview of the mediaeval 2013 visual privacy task. In: MediaEval (2014)
4.
go back to reference Bergeron, C., Sidaty, N., Hamidouche, W., Boyadjis, B., Le Feuvre, J., Lim, Y.: Real-time selective encryption solution based on ROI for MPEG-a visual identity management AF. In: 2017 22nd International Conference on Digital Signal Processing (DSP), pp. 1–5, Aug 2017 Bergeron, C., Sidaty, N., Hamidouche, W., Boyadjis, B., Le Feuvre, J., Lim, Y.: Real-time selective encryption solution based on ROI for MPEG-a visual identity management AF. In: 2017 22nd International Conference on Digital Signal Processing (DSP), pp. 1–5, Aug 2017
5.
go back to reference Bitouk, D., Kumar, N., Dhillon, S., Belhumeur, P., Nayar, S.K.: Face swapping: automatically replacing faces in photographs. In: ACM Transactions on Graphics (TOG), vol. 27, p. 39. ACM (2008)CrossRef Bitouk, D., Kumar, N., Dhillon, S., Belhumeur, P., Nayar, S.K.: Face swapping: automatically replacing faces in photographs. In: ACM Transactions on Graphics (TOG), vol. 27, p. 39. ACM (2008)CrossRef
6.
go back to reference Evaluation framework for compact descriptors for video analysis - search and retrieval - version 2.0. Technical report ISO/IEC JTC1/SC29/WG11/N15729 (2015) Evaluation framework for compact descriptors for video analysis - search and retrieval - version 2.0. Technical report ISO/IEC JTC1/SC29/WG11/N15729 (2015)
7.
go back to reference Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation). Official Journal of the European Union, L119:1–88, May 2016 Regulation (EU) 2016/679 of the European Parliament and of the Council of 27 April 2016 on the protection of natural persons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation). Official Journal of the European Union, L119:1–88, May 2016
8.
go back to reference Hadid, A., Nishiyama, M., Sato, Y.: Recognition of blurred faces via facial deblurring combined with blur-tolerant descriptors. In: 2010 20th International Conference on Pattern Recognition (ICPR), pp. 1160–1163. IEEE (2010) Hadid, A., Nishiyama, M., Sato, Y.: Recognition of blurred faces via facial deblurring combined with blur-tolerant descriptors. In: 2010 20th International Conference on Pattern Recognition (ICPR), pp. 1160–1163. IEEE (2010)
9.
go back to reference King, D.E.: Dlib-ml: a machine learning toolkit. J. Mach. Learn. Res. 10, 1755–1758 (2009) King, D.E.: Dlib-ml: a machine learning toolkit. J. Mach. Learn. Res. 10, 1755–1758 (2009)
10.
go back to reference Lander, K., Bruce, V., Hill, H.: Evaluating the effectiveness of pixelation and blurring on masking the identity of familiar faces. Appl. Cogn. Psychol. 15(1), 101–116 (2001)CrossRef Lander, K., Bruce, V., Hill, H.: Evaluating the effectiveness of pixelation and blurring on masking the identity of familiar faces. Appl. Cogn. Psychol. 15(1), 101–116 (2001)CrossRef
11.
go back to reference Letournel, G., Bugeau, A., Ta, V.T., Domenger, J.P.: Face de-identification with expressions preservation. In: 2015 IEEE International Conference on Image Processing (ICIP), pp. 4366–4370. IEEE (2015) Letournel, G., Bugeau, A., Ta, V.T., Domenger, J.P.: Face de-identification with expressions preservation. In: 2015 IEEE International Conference on Image Processing (ICIP), pp. 4366–4370. IEEE (2015)
12.
go back to reference Lin, J., Duan, L.-Y., Huang, Y., Luo, S., Huang, T., Gao, W.: Rate-adaptive compact fisher codes for mobile visual search. IEEE Signal Process. Lett. 21(2), 195–198 (2014)CrossRef Lin, J., Duan, L.-Y., Huang, Y., Luo, S., Huang, T., Gao, W.: Rate-adaptive compact fisher codes for mobile visual search. IEEE Signal Process. Lett. 21(2), 195–198 (2014)CrossRef
13.
go back to reference Liu, Z., Luo, P., Wang, X., Tang, X.: Deep learning face attributes in the wild. In: Proceedings of International Conference on Computer Vision (ICCV) (2015) Liu, Z., Luo, P., Wang, X., Tang, X.: Deep learning face attributes in the wild. In: Proceedings of International Conference on Computer Vision (ICCV) (2015)
14.
go back to reference Lou, Y., et al.: Compact deep invariant descriptors for video retrieval. In: Data Compression Conference (DCC), pp. 420–429, April 2017 Lou, Y., et al.: Compact deep invariant descriptors for video retrieval. In: Data Compression Conference (DCC), pp. 420–429, April 2017
15.
16.
go back to reference Mahajan, S., Chen, L.J., Tsai, T.C.: SwapItUP: a face swap application for privacy protection. In: 2017 IEEE 31st International Conference on Advanced Information Networking and Applications (AINA), pp. 46–50. IEEE (2017) Mahajan, S., Chen, L.J., Tsai, T.C.: SwapItUP: a face swap application for privacy protection. In: 2017 IEEE 31st International Conference on Advanced Information Networking and Applications (AINA), pp. 46–50. IEEE (2017)
17.
go back to reference Meng, L., Sun, Z., Collado, O.T.: Efficient approach to de-identifying faces in videos. IET Sig. Process. 11(9), 1039–1045 (2017)CrossRef Meng, L., Sun, Z., Collado, O.T.: Efficient approach to de-identifying faces in videos. IET Sig. Process. 11(9), 1039–1045 (2017)CrossRef
18.
go back to reference Nakashima, Y., Babaguchi, N., Fan, J.: Intended human object detection for automatically protecting privacy in mobile video surveillance. Multimed. Syst. 18(2), 157–173 (2012)CrossRef Nakashima, Y., Babaguchi, N., Fan, J.: Intended human object detection for automatically protecting privacy in mobile video surveillance. Multimed. Syst. 18(2), 157–173 (2012)CrossRef
19.
go back to reference Natsume, R., Yatagawa, T., Morishima, S.: RSGAN: face swapping and editing using face and hair representation in latent spaces. arXiv preprint arXiv:1804.03447 (2018) Natsume, R., Yatagawa, T., Morishima, S.: RSGAN: face swapping and editing using face and hair representation in latent spaces. arXiv preprint arXiv:​1804.​03447 (2018)
20.
go back to reference Newton, E.M., Sweeney, L., Malin, B.: Preserving privacy by de-identifying face images. IEEE Trans. Knowl. Data Eng. 17(2), 232–243 (2005)CrossRef Newton, E.M., Sweeney, L., Malin, B.: Preserving privacy by de-identifying face images. IEEE Trans. Knowl. Data Eng. 17(2), 232–243 (2005)CrossRef
21.
go back to reference Nirkin, Y., Masi, I., Tuan, A.T., Hassner, T., Medioni, G.: On face segmentation, face swapping, and face perception. In: 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition, FG 2018, pp. 98–105. IEEE (2018) Nirkin, Y., Masi, I., Tuan, A.T., Hassner, T., Medioni, G.: On face segmentation, face swapping, and face perception. In: 2018 13th IEEE International Conference on Automatic Face & Gesture Recognition, FG 2018, pp. 98–105. IEEE (2018)
22.
go back to reference Nishiyama, M., Takeshima, H., Shotton, J., Kozakaya, T., Yamaguchi, O.: Facial deblur inference to improve recognition of blurred faces. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009, pp. 1115–1122. IEEE (2009) Nishiyama, M., Takeshima, H., Shotton, J., Kozakaya, T., Yamaguchi, O.: Facial deblur inference to improve recognition of blurred faces. In: IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2009, pp. 1115–1122. IEEE (2009)
23.
go back to reference Padilla-López, J.R., Chaaraoui, A.A., Flórez-Revuelta, F.: Visual privacy protection methods: a survey. Expert. Syst. Appl. 42(9), 4177–4195 (2015)CrossRef Padilla-López, J.R., Chaaraoui, A.A., Flórez-Revuelta, F.: Visual privacy protection methods: a survey. Expert. Syst. Appl. 42(9), 4177–4195 (2015)CrossRef
24.
go back to reference Philbin, J., Chum, O., Isard, M., Sivic, J., Zisserman, A.: Object retrieval with large vocabularies and fast spatial matching. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2007) Philbin, J., Chum, O., Isard, M., Sivic, J., Zisserman, A.: Object retrieval with large vocabularies and fast spatial matching. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (2007)
25.
go back to reference Radford, A., Metz, L., Chintala, S.: Unsupervised representation learning with deep convolutional generative adversarial networks. CoRR, abs/1511.06434 (2015) Radford, A., Metz, L., Chintala, S.: Unsupervised representation learning with deep convolutional generative adversarial networks. CoRR, abs/1511.06434 (2015)
26.
go back to reference Saini, M., Atrey, P.K., Mehrotra, S., Kankanhalli, M.: W3-privacy: understanding what, when, and where inference channels in multi-camera surveillance video. Multimed. Tools Appl. 68(1), 135–158 (2014)CrossRef Saini, M., Atrey, P.K., Mehrotra, S., Kankanhalli, M.: W3-privacy: understanding what, when, and where inference channels in multi-camera surveillance video. Multimed. Tools Appl. 68(1), 135–158 (2014)CrossRef
27.
go back to reference Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. CoRR, abs/1409.1556 (2014) Simonyan, K., Zisserman, A.: Very deep convolutional networks for large-scale image recognition. CoRR, abs/1409.1556 (2014)
28.
go back to reference Zhang, K., Zhang, Z., Li, Z., Qiao, Y.: Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Sig. Process. Lett. 23(10), 1499–1503 (2016)CrossRef Zhang, K., Zhang, Z., Li, Z., Qiao, Y.: Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Sig. Process. Lett. 23(10), 1499–1503 (2016)CrossRef
29.
go back to reference Zhang, Y., Zheng, L., Thing, V.L.: Automated face swapping and its detection. In: 2017 IEEE 2nd International Conference on Signal and Image Processing (ICSIP), pp. 15–19. IEEE (2017) Zhang, Y., Zheng, L., Thing, V.L.: Automated face swapping and its detection. In: 2017 IEEE 2nd International Conference on Signal and Image Processing (ICSIP), pp. 15–19. IEEE (2017)
Metadata
Title
Face Swapping for Solving Collateral Privacy Issues in Multimedia Analytics
Author
Werner Bailer
Copyright Year
2019
DOI
https://doi.org/10.1007/978-3-030-05710-7_14