Top

Neural Processing Letters

Published in:

08-02-2022

How Does Bayesian Noisy Self-Supervision Defend Graph Convolutional Networks?

Authors: Jun Zhuang, Mohammad Al Hasan

Published in: Neural Processing Letters | Issue 4/2022

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

In recent years, it has been shown that, compared to other contemporary machine learning models, graph convolutional networks (GCNs) achieve superior performance on the node classification task. However, two potential issues threaten the robustness of GCNs, label scarcity and adversarial attacks. .Intensive studies aim to strengthen the robustness of GCNs from three perspectives, the self-supervision-based method, the adversarial-based method, and the detection-based method. Yet, all of the above-mentioned methods can barely handle both issues simultaneously. In this paper, we hypothesize noisy supervision as a kind of self-supervised learning method and then propose a novel Bayesian graph noisy self-supervision model, namely GraphNS, to address both issues. Extensive experiments demonstrate that GraphNS can significantly enhance node classification against both label scarcity and adversarial attacks. This enhancement proves to be generalized over four classic GCNs and is superior to the competing methods across six public graph datasets.

previous article Method of Rain Attenuation Prediction Based on Long–Short Term Memory Network

next article Optimization of Propofol Dose Estimated During Anesthesia Through Artificial Intelligence by Genetic Algorithm: Design and Clinical Assessment

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Available only for authorised users

https://www.kdd.org/kdd-cup/view/kdd-cup-2016.

Bishop CM (2006) Pattern recognition and machine learning. Springer, BerlinMATH

Bruna J, Zaremba W, Szlam A, LeCun Y (2013) Spectral networks and locally connected networks on graphs. arXiv preprint arXiv:1312.6203

Dai H, Li H, Tian T, Huang X, Wang L, Zhu J, Song L (2018) Adversarial attack on graph structured data. arXiv preprint arXiv:1806.02371

Defferrard M, Bresson X, Vandergheynst P (2016) Convolutional neural networks on graphs with fast localized spectral filtering. In: Advances in neural information processing systems, pp 3844–3852

Deng Z, Dong Y, Zhu J (2019) Batch virtual adversarial training for graph convolutional networks. arXiv preprint arXiv:1902.09192

Du B, Xinyao T, Wang Z, Zhang L, Tao D (2018) Robust graph-based semisupervised learning for noisy labeled data via maximum correntropy criterion. IEEE Trans Cybern

Du J, Zhang S, Wu G, Moura JM, Kar S (2017) Topology adaptive graph convolutional networks. arXiv preprint arXiv:1710.10370

Entezari N, Al-Sayouri SA, Darvishzadeh A, Papalexakis EE (2020) All you need is low (rank) defending against adversarial attacks on graphs. In: Proceedings of the 13th International Conference on Web Search and Data Mining

Feng F, He X, Tang J, Chua TS (2019) Graph adversarial training: Dynamically regularizing based on graph structure. IEEE Trans Knowl Data Eng

10.

Feng W, Zhang J, Dong Y, Han Y, Luan H, Xu Q, Yang Q, Kharlamov E, Tang J (2020) Graph random neural network for semi-supervised learning on graphs. In: NeurIPS’20

11.

Galke L, Vagliano I, Scherp A (2019) Can graph neural networks go “online"? an analysis of pretraining and inference. arXiv preprint arXiv:1905.06018

12.

Geisler S, Zügner D, Günnemann S (2020) Reliable graph neural networks via robust aggregation. Adv Neural Inf Process Syst, 33

13.

Goldberger J, Ben-Reuven E (2017) Training deep neural-networks using a noise adaptation layer. International Conference on Learning Representations

14.

Hamilton W, Ying Z, Leskovec J (2017) Inductive representation learning on large graphs. In: Advances in neural information processing systems, pp 1024–1034

15.

Hassani K, Khasahmadi AH (2020) Contrastive multi-view representation learning on graphs. In: International conference on machine learning, PMLR, pp 4116–4126

16.

Hu W, Liu B, Gomes J, Zitnik M, Liang P, Pande V, Leskovec J (2019a) Strategies for pre-training graph neural networks. In: International conference on learning representations

17.

Hu W, Fey M, Zitnik M, Dong Y, Ren H, Liu B, Catasta M, Leskovec J (2020a) Open graph benchmark: Datasets for machine learning on graphs. arXiv preprint arXiv:2005.00687

18.

Hu Z, Fan C, Chen T, Chang KW, Sun Y (2019b) Pre-training graph neural networks for generic structural feature extraction. arXiv preprint arXiv:1905.13728

19.

Hu Z, Dong Y, Wang K, Chang KW, Sun Y (2020b) Gpt-gnn: Generative pre-training of graph neural networks. In: Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, pp 1857–1867

20.

Hwang D, Park J, Kwon S, Kim KM, Ha JW, Kim HJ (2020) Self-supervised auxiliary learning with meta-paths for heterogeneous graphs. arXiv preprint arXiv:2007.08294

21.

Jin H, Zhang X (2019) Latent adversarial training of graph convolution networks. In: ICML workshop on learning and reasoning with graph-structured representations

22.

Jin W, Ma Y, Liu X, Tang X, Wang S, Tang J (2020) Graph structure learning for robust graph neural networks. In: Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, pp 66–74

23.

Kefato ZT, Girdzijauskas S (2021) Self-supervised graph neural networks without explicit negative sampling. arXiv preprint arXiv:2103.14958

24.

Kipf TN, Welling M (2016) Semi-supervised classification with graph convolutional networks. arXiv preprint arXiv:1609.02907

25.

Miller BA, Çamurcu M, Gomez AJ, Chan K, Eliassi-Rad T (2019) Improving robustness to attacks against vertex classification. In: MLG Workshop

26.

Misra I, Lawrence Zitnick C, Mitchell M, Girshick R (2016) Seeing through the human reporting bias: Visual classifiers from noisy human-centric labels. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 2930–2939

27.

NT H, Jin CJ, Murata T (2019) Learning graph neural networks with noisy labels. arXiv preprint arXiv:1905.01591

28.

Patrini G, Rozza A, Krishna Menon A, Nock R, Qu L (2017) Making deep neural networks robust to label noise: A loss correction approach. In: Proceedings of the IEEE conference on computer vision and pattern recognition

29.

Qiu J, Chen Q, Dong Y, Zhang J, Yang H, Ding M, Wang K, Tang J (2020) Gcc: Graph contrastive coding for graph neural network pre-training. In: Proceedings of the 26th ACM SIGKDD international conference on knowledge discovery & data mining, pp 1150–1160

30.

Qu M, Bengio Y, Tang J (2019) Gmnn: Graph markov neural networks. In: International conference on machine learning, PMLR, pp 5241–5250

31.

Reed S, Lee H, Anguelov D, Szegedy C, Erhan D, Rabinovich A (2014) Training deep neural networks on noisy labels with bootstrapping. arXiv preprint arXiv:1412.6596

32.

Rong Y, Bian Y, Xu T, Xie W, Wei Y, Huang W, Huang J (2020) Self-supervised graph transformer on large-scale molecular data. Adv Neural Inf Process Syst, 33

33.

Sen P, Namata G, Bilgic M, Getoor L, Galligher B, Eliassi-Rad T (2008) Collective classification in network data. AI magazine

34.

Shang J, Ma T, Xiao C, Sun J (2019) Pre-training of graph augmented transformers for medication recommendation. arXiv preprint arXiv:1906.00346

35.

Shchur O, Mumme M, Bojchevski A, Günnemann S (2018) Pitfalls of graph neural network evaluation. Relational Representation Learning Workshop, NeurIPS

36.

Sukhbaatar S, Bruna J, Paluri M, Bourdev L, Fergus R (2014) Training convolutional networks with noisy labels. arXiv preprint arXiv:1406.2080

37.

Sun K, Zhu Z, Lin Z (2019) Multi-stage self-supervised learning for graph convolutional networks. arXiv preprint arXiv:1902.11038

38.

Sun L, Dou Y, Yang C, Wang J, Yu PS, Li B (2018) Adversarial attack and defense on graph data: A survey. arXiv preprint arXiv:1812.10528

39.

Tang X, Li Y, Sun Y, Yao H, Mitra P, Wang S (2020) Transferring robustness for graph neural network against poisoning attacks. In: Proceedings of the 13th international conference on web search and data mining

40.

Tsitsulin A, Mottin D, Karras P, Bronstein A, Müller E (2018) Sgr: Self-supervised spectral graph representation learning. arXiv preprint arXiv:1811.06237

41.

Veličković P, Cucurull G, Casanova A, Romero A, Liò P, Bengio Y (2018) Graph attention networks. In: International conference on learning representations

42.

Wang S, Chen Z, Ni J, Yu X, Li Z, Chen H, Yu PS (2019a) Adversarial defense framework for graph neural network. arXiv preprint arXiv:1905.03679

43.

Wang X, Liu X, Hsieh CJ (2019b) Graphdefense: Towards robust graph convolutional networks. arXiv preprint arXiv:1911.04429

44.

Wu F, Zhang T, Souza Jr AHd, Fifty C, Yu T, Weinberger KQ (2019a) Simplifying graph convolutional networks. arXiv preprint arXiv:1902.07153

45.

Wu H, Wang C, Tyshetskiy Y, Docherty A, Lu K, Zhu L (2019b) Adversarial examples on graph data: Deep insights into attack and defense. arXiv preprint arXiv:1903.01610

46.

Xie Y, Xu Z, Zhang J, Wang Z, Ji S (2021) Self-supervised learning of graph neural networks: A unified review. arXiv preprint arXiv:2102.10757

47.

Xu B, Shen H, Cao Q, Qiu Y, Cheng X (2019a) Graph wavelet neural network. arXiv preprint arXiv:1904.07785

48.

Xu K, Hu W, Leskovec J, Jegelka S (2018) How powerful are graph neural networks? arXiv preprint arXiv:1810.00826

49.

Xu K, Chen H, Liu S, Chen PY, Weng TW, Hong M, Lin X (2019b) Topology attack and defense for graph neural networks: an optimization perspective. arXiv preprint arXiv:1906.04214

50.

Yao J, Wu H, Zhang Y, Tsang IW, Sun J (2019) Safeguarded dynamic label regression for noisy supervision. Proc AAAI Conf Artif Intell 33:9103–9110

51.

Yasunaga M, Liang P (2020) Graph-based, self-supervised program repair from diagnostic feedback. In: International conference on machine learning, PMLR, pp 10799–10808

52.

You Y, Chen T, Sui Y, Chen T, Wang Z, Shen Y (2020a) Graph contrastive learning with augmentations. Adv Neural Inf Process Syst, 33

53.

You Y, Chen T, Wang Z, Shen Y (2020b) When does self-supervision help graph convolutional networks? In: International conference on machine learning, PMLR, pp 10871–10880

54.

You Y, Chen T, Shen Y, Wang Z (2021) Graph contrastive learning automated. arXiv preprint arXiv:2106.07594

55.

Zhang A, Ma J (2020) Defensevgae: Defending against adversarial attacks on graph data via a variational graph autoencoder. arXiv preprint arXiv:2006.08900

56.

Zhang Y, Khan S, Coates M (2019a) Comparing and detecting adversarial attacks for graph deep learning. In: Proc. Representation Learning on Graphs and Manifolds Workshop, Int. Conf. Learning Representations, New Orleans, LA, USA

57.

Zhang Y, Pal S, Coates M, Ustebay D (2019) Bayesian graph convolutional neural networks for semi-supervised classification. Proc AAAI Conf Artif Intell 33:5829–5836

58.

Zheng C, Zong B, Cheng W, Song D, Ni J, Yu W, Chen H, Wang W (2020) Robust graph representation learning via neural sparsification. In: International conference on machine learning, PMLR, pp 11458–11468

59.

Zhong JX, Li N, Kong W, Liu S, Li TH, Li G (2019) Graph convolutional label noise cleaner: Train a plug-and-play action classifier for anomaly detection. In: Proceedings of the IEEE conference on computer vision and pattern recognition

60.

Zügner D, Günnemann S (2019) Adversarial attacks on graph neural networks via meta learning. arXiv preprint arXiv:1902.08412

61.

Zügner D, Akbarnejad A, Günnemann S (2018) Adversarial attacks on neural networks for graph data. In: Proceedings of the 24th ACM SIGKDD international conference on knowledge discovery & data mining, pp 2847–2856

Title: How Does Bayesian Noisy Self-Supervision Defend Graph Convolutional Networks?
Authors: Jun Zhuang
Mohammad Al Hasan
Publication date: 08-02-2022
Publisher: Springer US
Published in: Neural Processing Letters / Issue 4/2022
Print ISSN: 1370-4621
Electronic ISSN: 1573-773X
DOI: https://doi.org/10.1007/s11063-022-10750-8

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Springer Professional "Technik"

Springer Professional "Wirtschaft"

Other articles of this Issue 4/2022

Multimodal Orthodontic Corpus Construction Based on Semantic Tag Classification Method

Deep CNN Co-design for HEVC CU Partition Prediction on FPGA–SoC

Adaptive Synchronization-Based Approach for Finite-Time Parameters Identification of Genetic Regulatory Networks

A Targeted Universal Attack on Graph Convolutional Network by Using Fake Nodes

Robust Multi-view Classification with Sample Constraints

Semi-supervised Learning with Graph Convolutional Networks Based on Hypergraph