nach oben

Erschienen in:

2024 | OriginalPaper | Buchkapitel

Integration Model of Deep Forgery Video Detection Based on rPPG and Spatiotemporal Signal

verfasst von : Lujia Yang, Wenye Shu, Yongjia Wang, Zhichao Lian

Erschienen in: Green, Pervasive, and Cloud Computing

Verlag: Springer Nature Singapore

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config

KI-gestützte Suche

Aus

Abstract

With the development of deep learning, video forgery technology is becoming more and more mature, which may bring security risk and the further development of forgery detection is urgently needed. Most of the existing forgery detection technique are based on artifacts and detail features, which are greatly affected by the resolution, and its generalization ability needs to be improved. In this paper, a multi-modal fusion forgery detection model architecture based on the inherent biological signals and spatio-temporal signals in videos is proposed. In the process of forgery detection, the model first recognizes the face of the video. Subsequently, video frame extraction and rPPG signal extraction based on Green channel are performed on the video, respectively. These two data are later input into 3D and 2D convolutional neural networks to train the base learner respectively. Finally, the integration model is constructed based on stacking strategy. Sufficient experiments show that the established fusion model can cope well with low-resolution cases and has good generalization performance, achieving 93.38% and 91.57% accuracy on FF++ c23 and celeb-DF-v2 data set, respectively.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Jetzt informieren

Vorheriges Kapitel Research on Script-Based Software Component Development

Nächstes Kapitel A Design of Hybrid Transactional and Analytical Processing Database for Energy Efficient Big Data Queries

Deepfakes. Deepfakes github [EB/OL]. https://github.com/Deepfakes/facesw-ap. Accessed 08 May 2023

Zao: Zaoapp [EB/OL], 01 Dec 2019. https://zaodownload.com/download-zao-app-deepfake. Accessed 08 May 2023

Faceapp. Faceapp [EB/OL]. https://apps.apple.com/gb/app/faceapp-ai-faceedi~tor/id1180884341. Accessed 08 May 2023

FakeAPP. FakeAPP [EB/OL]. https://www.fakeapp.org. Accessed 08 May 2023

FaceSwap. FaceSwapgithub [EB/OL]. https://github.com/MarekKowalski/FaceSwap. Accessed 08 May 2023

Goodfellow, I.J., et al.: Generative adversarial nets. In: Proceedings of the 27th International Conference on Neural Information Processing Systems, pp. 2672–2680. MIT Press, Montreal, Canada (2014)

Karrast, T., Ailat, T., Laines, S., et al.: Progressive growing of GANs for improved quality, stability, andvariation. In: Proceedings of the 6th International Conference on Learning Representations. Vancouver, Canada: Open Revier.net (2018)

Karras, T., Laine, S., Aittala, M., Hellsten, J., Lehtinen, J., Aila, T.: Analyzing and improving the image quality of styleGAN. In: Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 8107–8116. IEEE, Seattle, USA (2020). https://doi.org/10.1109/CVPR42600.2020.00813

Choi, Y., Uh, Y., Yoo, J., Ha, J.W.: StarGAN v2: diverse image synthesis for multiple domains. In: Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), pp. 8185–8194. IEEE, Seattle, USA (2020). https://doi.org/10.1109/CVPR42600.2020.00821

10.

Li, L.Z., Bao, J.M., Yang, H., et al.: Advancing high fidelity identity swapping for forgery detection. In: Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5073–5082. IEEE, Seattle, USA (2020)

11.

Xu, C., Zhang, J.N., Hua, M., et al.: Region-awareface swapping. In: Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7622–7631. IEEE, New Orleans, USA (2022)

12.

Xu, Y.Y., Deng, B.L., Wang, J.L., et al.: High-resolution face swapping via latent semantics disentanglement. In: Proceedings of 2022 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 7632–7641. IEEE, New Orleans, USA (2022)

13.

Thies, J., Zollhöfer, M., Nießner, M.: Deferred neural rendering: Image synthesis using neural textures. ACM Trans. Graph. 38(4), 66 (2019)

14.

Wiles, O., Koepke, A.S., Zisserman, A.: X2Face: A network for controlling face generation using images, audio, and pose codes. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11217, pp. 690–706. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01261-8_41CrossRef

15.

Schwarzk, L.Y.Y., Niemeyer, M., et al.: Graf: generative radiance fields for 3D-aware image synthesis. In: Proceedings of the 34th International Conference on Neural Information Processing Systems, p. 1692. Curran Associates Inc., Vancouver, Canada (2020)

16.

Shen, Y.J., Gu, J.J., Tang, X.O., et al.: Interpreting the latent space of GANs for semantic face editing. In: Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 9240–9249. IEEE, Seattle, USA (2020)

17.

Chollet, F.: Xception: deep learning with depthwise separable convolutions. In: Proceedings of 2017 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1800–1807. IEEE, Honolulu, USA (2017)

18.

Tan, M.X., Le, Q.V.: Efficient net: rethinking model scaling for convolutional neural networks. In: Proceedings of the 36th International Conference on Machine Learning, pp. 6105–6114. PMLR, Long Beach, USA (2019)

19.

Li, L.Z., Bao, J.M., Zhang, T., et al.: Face X-ray for more general face forgery detection. In: Proceedings of 2020 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 5000–5009. IEEE, Seattle, USA (2020)

20.

Qian, Y., Yin, G., Sheng, L., Chen, Z., Shao, J.: Thinking in frequency: face forgery detection by mining frequency-aware clues. In: Vedaldi, A., Bischof, H., Brox, T., Frahm, J.-M. (eds.) ECCV 2020. LNCS, vol. 12357, pp. 86–103. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-58610-2_6CrossRef

21.

Li, J.M., Xie, H.T., Li, J.H., et al.: Frequency-aware discriminative feature learning supervised by single-center loss for face forgery detection. In: Proceedings of 2021 IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 6454–6463. IEEE, Nashville, USA (2021)

22.

Wang, J.K., Wu, Z.X., Ouyang, W.H., et al.: M2TR: multi-modal multi-scale transformers for deepfake detection. In: Proceedings of the 2022 International Conference on Multi media Retrieval, pp. 615–623. ACM, Newark, USA (2022)

23.

Conotter, V., Bodnari, E., Boato, G., Farid, H.: Physiologically based detection of computer generated faces in video. In: Proceedings of 2014 IEEE International Conference on Image Processing (ICIP), pp. 248–252. IEEE, Paris, France (2014). https://doi.org/10.1109/ICIP.2014.7025049

24.

Li, Y., Chang, M.C., Lyu, S.: In ictu oculi: exposing ai created fake videos by detecting eye blinking. In: 2018 IEEE International Workshop on Information Forensics and Security (WIFS), pp. 1–7. IEEE (2018)

25.

Hu, S., Li, Y., Lyu, S.: Exposing GAN-generated faces using inconsistent corneal specular highlights. In: ICASSP 2021–2021 IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP), pp. 2500–2504. IEEE (2021)

26.

Amerini, I., Galteri, L., Caldelli, R., del Bimbo, A.: Deepfake video detection through optical flow based CNN. In: 2019 IEEE/CVF International Conference on Computer Vision Workshop (ICCVW), p. 12051207. IEEE, Seoul, Korea (South) (2020). https://doi.org/10.1109/ICCVW.2019.00152

27.

Mittal, T., Bhattacharya, U., Chandra, R., Bera, A., Manocha, D.: Emotions don’ t lie: an audio-visual DeepFake detection method using affective cues [EB/OL] (2020). https://arxiv.org/pdf/2003.06711.pdf. Accessed 08 May 2023

28.

Deng, J., Guo, J., Zhou, Y., Yu, J., Kotsia, I., Zafeiriou, S.: Retinaface: single-stage dense face localisation in the wild. CoRR, abs/1905.00641 (2019)

29.

Verkruysse, W., et al.: Remote plethysmographic imaging using ambient light. Opt. Express 16(26), 21434–21445 (2008)CrossRef

30.

Ciftci, U.A., Demir, I., Yin, L.J.: FakeCatcher: detection of synthetic portrait videos using biological signals. IEEE Trans. Pattern Anal. Mach. Intell. 3029287 (2020b). https://doi.org/10.1109/TPAMI.2020.3009287

31.

Rössler, A., Cozzolino, D., Verdoliva, L., Riess, C., Thies, J., Niessner, M.: FaceForensics++: learning to detect manipulated facial images. In: Proceedings of the 2019 IEEE/ CVF International Conference on Computer Vision, pp. 1–11. IEEE, Seoul, Korea (South) (2019). https://doi.org/10.1109/ICCV.2019.00009

32.

Dolhansky, B., et al.: The DeepFake detection challenge (DFDC) dataset [EB/OL] (2020). https://arxiv.org/pdf/2006.07397.pdf. Accessed 08 May 2023

33.

Li, Y.Z., Yang, X., Sun, P., et al.: Celeb-DF (v2): a new dataset for deepfake forensics [EB/OL] (2023). https://arxiv.org/abs/1909.12962. Accessed 08 May 2023

34.

Thies, J., Zollhofer, M., Stamminger, M., Theobalt, C., Nießner, M.: Face2Face: real-time face capture and reenactment of RGB videos. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 2387–2395, June 2016

35.

Zhao, C., Lin, C., Chen, W., Li, Z.: A novel framework for remote photoplethysmography pulse extraction on compressed videos. In: The IEEE Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, June 2018

36.

Haliassos, A., Vougioukas, K., Petridis, S., et al.: Lips don’t lie: a generalisable and robust approach to face forgery detection. arXiv:2012.07657v3 [cs.CV]. 15 August 2021

37.

Li, J., Xie, H., Li, J., et al.: Frequency-aware discriminative feature learning supervised by single-center loss for face forgery detection. arXiv:2103.09096v1 [cs.CV], 16 March 2021

Titel: Integration Model of Deep Forgery Video Detection Based on rPPG and Spatiotemporal Signal
verfasst von: Lujia Yang
Wenye Shu
Yongjia Wang
Zhichao Lian
Verlag: Springer Nature Singapore
Buch: Green, Pervasive, and Cloud Computing
Print ISBN: 978-981-9998-92-0

Electronic ISBN: 978-981-9998-93-7

Copyright-Jahr: 2024
DOI: https://doi.org/10.1007/978-981-99-9893-7_9

Springer Professional

Abstract

Bitte loggen Sie sich ein, um Zugang zu Ihrer Lizenz zu erhalten.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Premium Partner