Skip to main content
Top
Published in:

01-12-2020 | Original Article

FauxWard: a graph neural network approach to fauxtography detection using social media comments

Authors: Lanyu Shang, Yang Zhang, Daniel Zhang, Dong Wang

Published in: Social Network Analysis and Mining | Issue 1/2020

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Online social media has been a popular source for people to consume and share news content. More recently, the spread of misinformation online has caused widespread concerns. In this work, we focus on a critical task of detecting fauxtography on social media where the image and associated text together convey misleading information. Many efforts have been made to mitigate misinformation online, but we found that the fauxtography problem has not been fully addressed by existing work. Solutions focusing on detecting fake images or misinformed texts alone on social media often fail to identify the misinformation delivered together by the image and the associated text of a fauxtography post. In this paper, we develop FauxWard, a novel graph convolutional neural network framework that explicitly explores the complex information extracted from a user comment network of a social media post to effectively identify fauxtography. FauxWard is content-free in the sense that it does not analyze the visual or textual contents of the post itself, which makes it robust against sophisticated fauxtography uploaders who intentionally craft image-centric posts by editing either the text or image content. We evaluate FauxWard on two real-world datasets collected from mainstream social media platforms (i.e., Reddit and Twitter). The results show that FauxWard is both effective and efficient in identifying fauxtography posts on social media.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Literature
go back to reference Bayar B, Stamm MC (2016) A deep learning approach to universal image manipulation detection using a new convolutional layer. In: Proceedings of the 4th ACM workshop on information hiding and multimedia security, pp 5–10 Bayar B, Stamm MC (2016) A deep learning approach to universal image manipulation detection using a new convolutional layer. In: Proceedings of the 4th ACM workshop on information hiding and multimedia security, pp 5–10
go back to reference Chen J, Ma T, Xiao C (2018) Fastgcn: fast learning with graph convolutional networks via importance sampling. arXiv preprint arXiv:180110247 Chen J, Ma T, Xiao C (2018) Fastgcn: fast learning with graph convolutional networks via importance sampling. arXiv preprint arXiv:​180110247
go back to reference Cooper SD (2007) A concise history of the fauxtography blogstorm in the 2006 lebanon war. Am Commun J 9 Cooper SD (2007) A concise history of the fauxtography blogstorm in the 2006 lebanon war. Am Commun J 9
go back to reference Cui L, Wang S, Lee D (2019) Same: sentiment-aware multi-modal embedding for detecting fake news. In: Proceedings of the 2019 IEEE/ACM international conference on advances in social networks analysis and mining, pp 41–48 Cui L, Wang S, Lee D (2019) Same: sentiment-aware multi-modal embedding for detecting fake news. In: Proceedings of the 2019 IEEE/ACM international conference on advances in social networks analysis and mining, pp 41–48
go back to reference Fout A, Byrd J, Shariat B, Ben-Hur A (2017) Protein interface prediction using graph convolutional networks. In: Advances in neural information processing systems, pp 6530–6539 Fout A, Byrd J, Shariat B, Ben-Hur A (2017) Protein interface prediction using graph convolutional networks. In: Advances in neural information processing systems, pp 6530–6539
go back to reference Fridrich AJ, Soukal BD, Lukáš AJ (2003) Detection of copy-move forgery in digital images. In: Proceedings of digital forensic research workshop, Citeseer Fridrich AJ, Soukal BD, Lukáš AJ (2003) Detection of copy-move forgery in digital images. In: Proceedings of digital forensic research workshop, Citeseer
go back to reference Gupta A, Lamba H, Kumaraguru P, Joshi A (2013) Faking sandy: characterizing and identifying fake images on twitter during hurricane sandy. In: Proceedings of the 22nd international conference on World Wide Web, pp 729–736 Gupta A, Lamba H, Kumaraguru P, Joshi A (2013) Faking sandy: characterizing and identifying fake images on twitter during hurricane sandy. In: Proceedings of the 22nd international conference on World Wide Web, pp 729–736
go back to reference Hamaguchi T, Oiwa H, Shimbo M, Matsumoto Y (2017) Knowledge transfer for out-of-knowledge-base entities: a graph neural network approach. arXiv preprint arXiv:170605674 Hamaguchi T, Oiwa H, Shimbo M, Matsumoto Y (2017) Knowledge transfer for out-of-knowledge-base entities: a graph neural network approach. arXiv preprint arXiv:​170605674
go back to reference Hamilton W, Ying Z, Leskovec J (2017) Inductive representation learning on large graphs. In: Advances in neural information processing systems, pp 1024–1034 Hamilton W, Ying Z, Leskovec J (2017) Inductive representation learning on large graphs. In: Advances in neural information processing systems, pp 1024–1034
go back to reference Huynh-Kha T, Le-Tien T, Ha-Viet-Uyen S, Huynh-Van K, Luong M (2016) A robust algorithm of forgery detection in copy-move and spliced images. Int J Adv Comput Sci Appl 7(3) Huynh-Kha T, Le-Tien T, Ha-Viet-Uyen S, Huynh-Van K, Luong M (2016) A robust algorithm of forgery detection in copy-move and spliced images. Int J Adv Comput Sci Appl 7(3)
go back to reference Kwak H, Lee C, Park H, Moon S (2010) What is twitter, a social network or a news media? In: Proceedings of the 19th international conference on World wide web, pp 591–600 Kwak H, Lee C, Park H, Moon S (2010) What is twitter, a social network or a news media? In: Proceedings of the 19th international conference on World wide web, pp 591–600
go back to reference Lazer DM, Baum MA, Benkler Y, Berinsky AJ, Greenhill KM, Menczer F, Metzger MJ, Nyhan B, Pennycook G, Rothschild D et al (2018) The science of fake news. Science 359(6380):1094–1096CrossRef Lazer DM, Baum MA, Benkler Y, Berinsky AJ, Greenhill KM, Menczer F, Metzger MJ, Nyhan B, Pennycook G, Rothschild D et al (2018) The science of fake news. Science 359(6380):1094–1096CrossRef
go back to reference Li R, Wang S, Zhu F, Huang J (2018) Adaptive graph convolutional neural networks. In: Thirty-second AAAI conference on artificial intelligence Li R, Wang S, Zhu F, Huang J (2018) Adaptive graph convolutional neural networks. In: Thirty-second AAAI conference on artificial intelligence
go back to reference Matern F, Riess C, Stamminger M (2019) Gradient-based illumination description for image forgery detection. IEEE Transactions on Information Forensics and Security 15:1303–1317CrossRef Matern F, Riess C, Stamminger M (2019) Gradient-based illumination description for image forgery detection. IEEE Transactions on Information Forensics and Security 15:1303–1317CrossRef
go back to reference Nguyen TH, Grishman R (2018) Graph convolutional networks with argument-aware pooling for event detection. In: Thirty-second AAAI conference on artificial intelligence Nguyen TH, Grishman R (2018) Graph convolutional networks with argument-aware pooling for event detection. In: Thirty-second AAAI conference on artificial intelligence
go back to reference Priya S, Sequeira R, Chandra J, Dandapat SK (2019) Where should one get news updates: Twitter or reddit. Online Soc Netw Media 9:17–29CrossRef Priya S, Sequeira R, Chandra J, Dandapat SK (2019) Where should one get news updates: Twitter or reddit. Online Soc Netw Media 9:17–29CrossRef
go back to reference Pun CM, Yuan XC, Bi XL (2015) Image forgery detection using adaptive oversegmentation and feature point matching. IEEE Trans Inf Forensics Secur 10(8):1705–1716CrossRef Pun CM, Yuan XC, Bi XL (2015) Image forgery detection using adaptive oversegmentation and feature point matching. IEEE Trans Inf Forensics Secur 10(8):1705–1716CrossRef
go back to reference Qian F, Gong C, Sharma K, Liu Y (2018) Neural user response generator: fake news detection with collective user intelligence. IJCAI 18:3834–3840 Qian F, Gong C, Sharma K, Liu Y (2018) Neural user response generator: fake news detection with collective user intelligence. IJCAI 18:3834–3840
go back to reference Rashid MT, Wang D (2020) Covidsens: a vision on reliable social sensing for covid-19. Artif Intell Rev, pp 1–25 Rashid MT, Wang D (2020) Covidsens: a vision on reliable social sensing for covid-19. Artif Intell Rev, pp 1–25
go back to reference Schlichtkrull M, Kipf TN, Bloem P, Van Den Berg R, Titov I, Welling M (2018) Modeling relational data with graph convolutional networks. In: European semantic web conference, Springer, Berlin, pp 593–607 Schlichtkrull M, Kipf TN, Bloem P, Van Den Berg R, Titov I, Welling M (2018) Modeling relational data with graph convolutional networks. In: European semantic web conference, Springer, Berlin, pp 593–607
go back to reference Shang L, Zhang DY, Wang M, Lai S, Wang D (2019a) Towards reliable online clickbait video detection: a content-agnostic approach. Knowl-Based Syst 182:104851CrossRef Shang L, Zhang DY, Wang M, Lai S, Wang D (2019a) Towards reliable online clickbait video detection: a content-agnostic approach. Knowl-Based Syst 182:104851CrossRef
go back to reference Shang L, Zhang DY, Wang M, Wang D (2019b) Vulnercheck: a content-agnostic detector for online hatred-vulnerable videos. In: 2019 IEEE international conference on big data (Big Data), IEEE, pp 573–582 Shang L, Zhang DY, Wang M, Wang D (2019b) Vulnercheck: a content-agnostic detector for online hatred-vulnerable videos. In: 2019 IEEE international conference on big data (Big Data), IEEE, pp 573–582
go back to reference Shu K, Sliva A, Wang S, Tang J, Liu H (2017) Fake news detection on social media: a data mining perspective. ACM SIGKDD Explor Newsl 19(1):22–36CrossRef Shu K, Sliva A, Wang S, Tang J, Liu H (2017) Fake news detection on social media: a data mining perspective. ACM SIGKDD Explor Newsl 19(1):22–36CrossRef
go back to reference Vo N, Lee K (2018) The rise of guardians: fact-checking url recommendation to combat fake news. arXiv preprint arXiv:180607516 Vo N, Lee K (2018) The rise of guardians: fact-checking url recommendation to combat fake news. arXiv preprint arXiv:​180607516
go back to reference Volkova S, Shaffer K, Jang JY, Hodas N (2017) Separating facts from fiction: linguistic models to classify suspicious and trusted news posts on twitter. In: Proceedings of the 55th annual meeting of the association for computational linguistics (volume 2: short papers), pp 647–653 Volkova S, Shaffer K, Jang JY, Hodas N (2017) Separating facts from fiction: linguistic models to classify suspicious and trusted news posts on twitter. In: Proceedings of the 55th annual meeting of the association for computational linguistics (volume 2: short papers), pp 647–653
go back to reference Wang D, Abdelzaher T, Kaplan L, Ganti R, Hu S, Liu H (2015b) Reliable social sensing with physical constraints: analytic bounds and performance evaluation. Real-Time Syst 51(6):724–762CrossRef Wang D, Abdelzaher T, Kaplan L, Ganti R, Hu S, Liu H (2015b) Reliable social sensing with physical constraints: analytic bounds and performance evaluation. Real-Time Syst 51(6):724–762CrossRef
go back to reference Wang D, Szymanski BK, Abdelzaher T, Ji H, Kaplan L (2019) The age of social sensing. Computer 52(1):36–45CrossRef Wang D, Szymanski BK, Abdelzaher T, Ji H, Kaplan L (2019) The age of social sensing. Computer 52(1):36–45CrossRef
go back to reference Wang D, Abdelzaher T, Kaplan L (2015a) Social sensing: building reliable systems on unreliable data. Morgan Kaufmann Wang D, Abdelzaher T, Kaplan L (2015a) Social sensing: building reliable systems on unreliable data. Morgan Kaufmann
go back to reference Wang D, Abdelzaher T, Kaplan L, Aggarwal CC (2013a) Recursive fact-finding: a streaming approach to truth estimation in crowdsourcing applications. In: 2013 IEEE 33rd international conference on distributed computing systems, IEEE, pp 530–539 Wang D, Abdelzaher T, Kaplan L, Aggarwal CC (2013a) Recursive fact-finding: a streaming approach to truth estimation in crowdsourcing applications. In: 2013 IEEE 33rd international conference on distributed computing systems, IEEE, pp 530–539
go back to reference Wang D, Abdelzaher T, Kaplan L, Ganti R, Hu S, Liu H (2013b) Exploitation of physical constraints for reliable social sensing. In: Real-time systems symposium (RTSS), 2013 IEEE 34th, IEEE, pp 212–223 Wang D, Abdelzaher T, Kaplan L, Ganti R, Hu S, Liu H (2013b) Exploitation of physical constraints for reliable social sensing. In: Real-time systems symposium (RTSS), 2013 IEEE 34th, IEEE, pp 212–223
go back to reference Wang D, Amin MT, Li S, Abdelzaher T, Kaplan L, Gu S, Pan C, Liu H, Aggarwal CC, Ganti R, et al. (2014) Using humans as sensors: an estimation-theoretic perspective. In: Information processing in sensor networks, IPSN-14 proceedings of the 13th international symposium on, IEEE, pp 35–46 Wang D, Amin MT, Li S, Abdelzaher T, Kaplan L, Gu S, Pan C, Liu H, Aggarwal CC, Ganti R, et al. (2014) Using humans as sensors: an estimation-theoretic perspective. In: Information processing in sensor networks, IPSN-14 proceedings of the 13th international symposium on, IEEE, pp 35–46
go back to reference Wu Z, Pan S, Chen F, Long G, Zhang C, Yu PS (2019) A comprehensive survey on graph neural networks. arXiv preprint arXiv:190100596 Wu Z, Pan S, Chen F, Long G, Zhang C, Yu PS (2019) A comprehensive survey on graph neural networks. arXiv preprint arXiv:​190100596
go back to reference Yang Y, Zheng L, Zhang J, Cui Q, Li Z, Yu PS (2018) Ti-cnn: Convolutional neural networks for fake news detection. arXiv preprint arXiv:180600749 Yang Y, Zheng L, Zhang J, Cui Q, Li Z, Yu PS (2018) Ti-cnn: Convolutional neural networks for fake news detection. arXiv preprint arXiv:​180600749
go back to reference Yao QQ, Perlmutter DD, Liu JZ (2017) What are shaping the ethical bottom line?: Identifying factors influencing young readers’ acceptance of digital news photo alteration. Telematics Inform 34(1):124–132CrossRef Yao QQ, Perlmutter DD, Liu JZ (2017) What are shaping the ethical bottom line?: Identifying factors influencing young readers’ acceptance of digital news photo alteration. Telematics Inform 34(1):124–132CrossRef
go back to reference Ying R, He R, Chen K, Eksombatchai P, Hamilton WL, Leskovec J (2018a) Graph convolutional neural networks for web-scale recommender systems. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, ACM, pp 974–983 Ying R, He R, Chen K, Eksombatchai P, Hamilton WL, Leskovec J (2018a) Graph convolutional neural networks for web-scale recommender systems. In: Proceedings of the 24th ACM SIGKDD International Conference on Knowledge Discovery & Data Mining, ACM, pp 974–983
go back to reference Ying Z, You J, Morris C, Ren X, Hamilton W, Leskovec J (2018b) Hierarchical graph representation learning with differentiable pooling. In: Advances in neural information processing systems, pp 4800–4810 Ying Z, You J, Morris C, Ren X, Hamilton W, Leskovec J (2018b) Hierarchical graph representation learning with differentiable pooling. In: Advances in neural information processing systems, pp 4800–4810
go back to reference Zhang DY, Badilla J, Zhang Y, Wang D (2018b) Towards reliable missing truth discovery in online social media sensing applications. In: 2018 IEEE/ACM international conference on advances in social networks analysis and mining (ASONAM), IEEE, pp 143–150 Zhang DY, Badilla J, Zhang Y, Wang D (2018b) Towards reliable missing truth discovery in online social media sensing applications. In: 2018 IEEE/ACM international conference on advances in social networks analysis and mining (ASONAM), IEEE, pp 143–150
go back to reference Zhang DY, Li Q, Tong H, Badilla J, Zhang Y, Wang D (2018c) Crowdsourcing-based copyright infringement detection in live video streams. In: 2018 IEEE/ACM international conference on advances in social networks analysis and mining (ASONAM), IEEE, pp 367–374 Zhang DY, Li Q, Tong H, Badilla J, Zhang Y, Wang D (2018c) Crowdsourcing-based copyright infringement detection in live video streams. In: 2018 IEEE/ACM international conference on advances in social networks analysis and mining (ASONAM), IEEE, pp 367–374
go back to reference Zhang DY, Shang L, Geng B, Lai S, Li K, Zhu H, Amin MT, Wang D (2018d) Fauxbuster: a content-free fauxtography detector using social media comments. In: 2018 IEEE international conference on big data (Big Data), IEEE, pp 891–900 Zhang DY, Shang L, Geng B, Lai S, Li K, Zhu H, Amin MT, Wang D (2018d) Fauxbuster: a content-free fauxtography detector using social media comments. In: 2018 IEEE international conference on big data (Big Data), IEEE, pp 891–900
go back to reference Zhang DY, Song L, Li Q, Zhang Y, Wang D (2018e) Streamguard: a bayesian network approach to copyright infringement detection problem in large-scale live video sharing systems. In: 2018 IEEE international conference on big data (big data), IEEE, pp 901–910 Zhang DY, Song L, Li Q, Zhang Y, Wang D (2018e) Streamguard: a bayesian network approach to copyright infringement detection problem in large-scale live video sharing systems. In: 2018 IEEE international conference on big data (big data), IEEE, pp 901–910
go back to reference Zhang DY, Wang D, Zhang Y (2017) Constraint-aware dynamic truth discovery in big data social media sensing. In: Big data (big data), 2017 IEEE international conference on, IEEE, pp 57–66 Zhang DY, Wang D, Zhang Y (2017) Constraint-aware dynamic truth discovery in big data social media sensing. In: Big data (big data), 2017 IEEE international conference on, IEEE, pp 57–66
go back to reference Zhang D, Wang D, Vance N, Zhang Y, Mike S (2018a) On scalable and robust truth discovery in big data social media sensing applications. IEEE Trans Big Data 5(2):195–208CrossRef Zhang D, Wang D, Vance N, Zhang Y, Mike S (2018a) On scalable and robust truth discovery in big data social media sensing applications. IEEE Trans Big Data 5(2):195–208CrossRef
go back to reference Zhang J, Cui L, Fu Y, Gouza FB (2018f) Fake news detection with deep diffusive network model. arXiv preprint arXiv:180508751 Zhang J, Cui L, Fu Y, Gouza FB (2018f) Fake news detection with deep diffusive network model. arXiv preprint arXiv:​180508751
go back to reference Zhang Y, Dong X, Rashid MT, Shang L, Han J, Zhang D, Wang D (2020a) Pqa-cnn: Towards perceptual quality assured single-image super-resolution in remote sensing. In: 2020 IEEE/ACM international symposium on quality of service, IEEE Zhang Y, Dong X, Rashid MT, Shang L, Han J, Zhang D, Wang D (2020a) Pqa-cnn: Towards perceptual quality assured single-image super-resolution in remote sensing. In: 2020 IEEE/ACM international symposium on quality of service, IEEE
go back to reference Zhang Y, Dong X, Shang L, Zhang D, Wang D (2020b) A multi-modal graph neural network approach to traffic risk forecasting in smart urban sensing. In: The 17th annual IEEE international conference on sensing, communication and networking, IEEE Zhang Y, Dong X, Shang L, Zhang D, Wang D (2020b) A multi-modal graph neural network approach to traffic risk forecasting in smart urban sensing. In: The 17th annual IEEE international conference on sensing, communication and networking, IEEE
go back to reference Zhang D, Vance N, Wang D (2019) When social sensing meets edge computing: vision and challenges. In: 2019 28th International conference on computer communication and networks (ICCCN), IEEE, pp 1–9 Zhang D, Vance N, Wang D (2019) When social sensing meets edge computing: vision and challenges. In: 2019 28th International conference on computer communication and networks (ICCCN), IEEE, pp 1–9
go back to reference Zhou J, Cui G, Zhang Z, Yang C, Liu Z, Wang L, Li C, Sun M (2018) Graph neural networks: a review of methods and applications. arXiv preprint arXiv:181208434 Zhou J, Cui G, Zhang Z, Yang C, Liu Z, Wang L, Li C, Sun M (2018) Graph neural networks: a review of methods and applications. arXiv preprint arXiv:​181208434
go back to reference Zhou X, Zafarani R (2018) Fake news: a survey of research, detection methods, and opportunities. arXiv preprint arXiv:181200315 Zhou X, Zafarani R (2018) Fake news: a survey of research, detection methods, and opportunities. arXiv preprint arXiv:​181200315
Metadata
Title
FauxWard: a graph neural network approach to fauxtography detection using social media comments
Authors
Lanyu Shang
Yang Zhang
Daniel Zhang
Dong Wang
Publication date
01-12-2020
Publisher
Springer Vienna
Published in
Social Network Analysis and Mining / Issue 1/2020
Print ISSN: 1869-5450
Electronic ISSN: 1869-5469
DOI
https://doi.org/10.1007/s13278-020-00689-w

Premium Partner