Top

Neural Computing and Applications

Published in:

11-08-2020 | Original Article

A noise injection strategy for graph autoencoder training

Authors: Yingfeng Wang, Biyun Xu, Myungjae Kwak, Xiaoqin Zeng

Published in: Neural Computing and Applications | Issue 10/2021

Activate our intelligent search to find suitable subject content or patents.

search-config

AI-assisted search

Off

Abstract

Graph autoencoder can map graph data into a low-dimensional space. It is a powerful graph embedding method applied in graph analytics to lower the computational cost. Researchers have developed different graph autoencoders for addressing different needs. This paper proposes a strategy based on noise injection for graph autoencoder training. This is a general training strategy that can flexibly fit most existing training algorithms. The experimental results verify this general strategy can significantly reduce overfitting and identify the noise rate setting for consistent training performance improvement.

previous article Decoding movement intent patterns based on spatiotemporal and adaptive filtering method towards active motor training in stroke rehabilitation systems

next article Research on trend analysis method of multi-series economic data based on correlation enhancement of deep learning

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 340 Zeitschriften

aus folgenden Fachgebieten:

Bauwesen + Immobilien
Business IT + Informatik
Finance + Banking
Management + Führung
Marketing + Vertrieb
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

über 67.000 Bücher
über 390 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Maschinenbau + Werkstoffe

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

über 102.000 Bücher
über 537 Zeitschriften

aus folgenden Fachgebieten:

Automobil + Motoren
Bauwesen + Immobilien
Business IT + Informatik
Elektrotechnik + Elektronik
Energie + Nachhaltigkeit
Finance + Banking
Management + Führung
Marketing + Vertrieb
Maschinenbau + Werkstoffe
Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Wang Y, Xu B, Kwak M, Zeng X (2020) A simple training strategy for graph autoencoder. In: Proceedings of the international conference on machine learning and computing (ICMLC), pp 341–345. https://doi.org/10.1145/3383972.3383985

Tahmasebi H, Ravanmehr R, Mohamadrezaei R (2020) Social movie recommender system based on deep autoencoder network using Twitter data. Neural Comput Appl. https://doi.org/10.1007/s00521-020-05085-1CrossRef

Cai H, Zheng VW, Chang KCC (2018) A comprehensive survey of graph embedding: Problems, techniques, and applications. IEEE Trans Knowl Data Eng 30:1616–1637. https://doi.org/10.1109/TKDE.2018.2807452CrossRef

Li B, Pi D (2020) Network representation learning: a systematic literature review. Neural Comput Appl. https://doi.org/10.1007/s00521-020-04908-5CrossRef

Pan S, Hu R, Fung SF et al (2020) Learning graph embedding with adversarial training methods. IEEE Trans Cybern 50:2475–2487. https://doi.org/10.1109/TCYB.2019.2932096CrossRef

Pan S, Hu R, Long G, et al (2018) Adversarially regularized graph autoencoder for graph embedding. In: Proceedings of 27th international joint conference artificial intelligence, pp 2609–2615. https://doi.org/10.1523/JNEUROSCI.1317-08.2008

Zhang D, Yin J, Zhu X, Zhang C (2018) Network Representation Learning: A Survey. IEEE Trans Big Data. https://doi.org/10.1109/tbdata.2018.2850013CrossRef

Kipf TN, Welling M (2016) Variational graph auto-encoders. In: NIPS workshop on bayesian deep learning

Wang D, Cui P, Zhu W (2016) Structural deep network embedding. In: Proceedings of the ACM SIGKDD international conference on knowledge discovery and data mining, pp 1225–1234

10.

Tu K, Cui P, Wang X, et al (2018) Deep recursive network embedding with regular equivalence. In: Proceedings of ACM SIGKDD international conference knowledge discovery data min, pp 2357–2366. https://doi.org/10.1145/3219819.3220068

11.

Samanta B, DE A, Jana G, et al (2019) NeVAE: A deep generative model for molecular graphs. In: Proceedings of the AAAI conference on artificial intelligence. pp 1110–1117

12.

Grover A, Zweig A, Ermon S (2019) Graphite: Iterative generative modeling of graphs. In: Proceedings of machine learning research. pp 2434--2444

13.

Cao S, Lu W, Xu Q (2016) Deep neural networks for learning graph representations. In: Proceedings of 30th AAAI conference on artificial intelligence, pp 1145–1152

14.

Yu W, Zheng C, Cheng W, et al (2018) Learning deep network representations with adversarially. In: Proceedings of the international conference on knowledge discovery and data mining, pp 2663–2671

15.

Goodfellow I, Pouget-Abadie J, Mirza M et al (2014) Generative adversarial nets. Adv Neural Inf Process Syst 27:2672–2680

16.

Elman JL, Zipser D (1988) Learning the hidden structure of speech. J Acoust Soc Am 83:1615–1626. https://doi.org/10.1121/1.395916CrossRef

17.

Sietsma J, Dow RJF (1991) Creating artificial neural networks that generalize. Neural Netw 4:67–79. https://doi.org/10.1016/0893-6080(91)90033-2CrossRef

18.

Holmstrom L, Koistinen P (1992) Using additive noise in back propagation training. IEEE Trans Neural Netw 3:24–38. https://doi.org/10.1109/72.105415CrossRef

19.

Skurichina M, Raudys Š, Duin RPW (2000) K-nearest neighbors directed noise injection in multilayer perceptron training. IEEE Trans Neural Netw 11:504–511. https://doi.org/10.1109/72.839019CrossRef

20.

Brown WM, Gedeon TD, Groves DI (2003) Use of noise to augment training data: a neural network method of mineral-potential mapping in regions of limited known deposit examples. Nat Resour Res 12:141–152. https://doi.org/10.1023/A:1024218913435CrossRef

21.

Matsuoka K (1992) Noise injection into inputs in back-propagation learning. IEEE Trans Syst Man Cybern 22:436–440. https://doi.org/10.1109/21.370200CrossRef

22.

Reed R, Marks RJ, Oh S (1995) Similarities of error regularization, sigmoid gain scaling, target smoothing, and training with jitter. IEEE Trans Neural Netw 6:529–538. https://doi.org/10.1109/72.377960CrossRef

23.

Bishop CM (1995) Training with noise is equivalent to Tikhonov regularization. Neural Comput 7:108–116. https://doi.org/10.1162/neco.1995.7.1.108CrossRef

24.

Grandvalet Y, Canu S, Boucheron S (1997) Noise injection: theoretical prospects. Neural Comput 9:1093–1108. https://doi.org/10.1162/neco.1997.9.5.1093CrossRef

25.

An G (1996) The Effects of adding noise during backpropagation training on a generalization performance. Neural Comput 8:643–674. https://doi.org/10.1162/neco.1996.8.3.643CrossRef

26.

Piotrowski AP, Napiorkowski JJ (2013) A comparison of methods to avoid overfitting in neural networks training in the case of catchment runoff modelling. J Hydrol 476:97–111. https://doi.org/10.1016/j.jhydrol.2012.10.019CrossRef

27.

Wright WA (1999) Bayesian approach to neural-network modeling with input uncertainty. IEEE Trans Neural Netw 10:1261–1270. https://doi.org/10.1109/72.809073CrossRef

28.

Wright WA, Ramage G, Cornford D, Nabney IT (2000) Neural network modelling with input uncertainty: theory and application. J VLSI Signal Process Syst Signal Image Video Technol 26:169–188. https://doi.org/10.1023/A:1008111920791CrossRefMATH

29.

Zhang S, Tong H, Xu J, Maciejewski R (2019) Graph convolutional networks: a comprehensive review. Comput Soc Netw 6:1–23. https://doi.org/10.1186/s40649-019-0069-yCrossRef

30.

McDowell LK, Gupta KM, Aha DW (2009) Cautious collective classification. J Mach Learn Res 10:2777–2836MathSciNetMATH

31.

Giles CL, Bollacker KD, Lawrence S (1998) CiteSeer: an automatic citation indexing system. In: Proceedings of ACM international conference digital library, pp 89–98

32.

Kingma DP, Ba JL (2015) Adam: a method for stochastic optimization. In: Proceedings of the 3rd international conference on learning representations

33.

Fawcett T (2006) An introduction to ROC analysis. Pattern Recognit Lett 27:861–874. https://doi.org/10.1016/j.patrec.2005.10.010CrossRef

34.

McClish DK (1989) Analyzing a portion of the ROC curve. Med Decis Mak 9:190–195. https://doi.org/10.1177/0272989X8900900307CrossRef

35.

Wikipedia entry for the Receiver operating characteristic. https://en.wikipedia.org/wiki/Receiver_operating_characteristic. Accessed 9 Jan 2019

Title: A noise injection strategy for graph autoencoder training
Authors: Yingfeng Wang
Biyun Xu
Myungjae Kwak
Xiaoqin Zeng
Publication date: 11-08-2020
Publisher: Springer London
Published in: Neural Computing and Applications / Issue 10/2021
Print ISSN: 0941-0643
Electronic ISSN: 1433-3058
DOI: https://doi.org/10.1007/s00521-020-05283-x

Springer Professional

Abstract

Please log in to get access to your license.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft"

Springer Professional "Technik"

Springer Professional "Wirtschaft+Technik"

Other articles of this Issue 10/2021

Modified semi-supervised affinity propagation clustering with fuzzy density fruit fly optimization

Deep reinforcement learning for permanent magnet synchronous motor speed control systems

A precise and stable machine learning algorithm: eigenvalue classification (EigenClass)

Two-branch encoding and iterative attention decoding network for semantic segmentation

Convolutional neural network with spatial pyramid pooling for hand gesture recognition

A new framework for sign language alphabet hand posture recognition using geometrical features through artificial neural network (part 1)

Premium Partner