Skip to main content
Erschienen in: Pattern Analysis and Applications 4/2023

07.10.2023 | Theoretical Advances

Adaptive frequency-based fully hyperbolic graph neural networks

verfasst von: FeiFei Wei, MingZhu Ping, KuiZhi Mei

Erschienen in: Pattern Analysis and Applications | Ausgabe 4/2023

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Graph Convolutional Networks (GCNs) have attracted broad attention from industry and academia, for their excellent expressive power in terms of modeling the irregular data, e.g., skeletal data and graph-structured data. The most effective existing model may be the fully hyperbolic graph neural network. However, it involves a large number of parameters, thus consuming considerable computing resources. In this paper, we propose a model based on adaptive frequency filter and corresponding optimizer in hyperbolic space. The adaptive frequency can learn the different frequency components of the embeddings of the nodes in graph, which adaptively adjust the beneficial signals of high-frequency and low-frequency. And the optimizer is based on a subset of the orthogonal constraint, which is dedicated for the adaptive frequency with less parameters. Moreover, our model bridges the gap of hyperbolic space and the spectral space for exploring the underlying semantics of the node and relation embeddings of graph. Consequently, our model needs only to optimize the less parameters in hyperbolic space and meanwhile prevent the distortion caused by conventional manifold GCN. Experimental results show that our method achieves substantial improvements and outperforms the state-of-the-art performance in terms of node classification.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Wang H, Xu T, Liu Q, Lian D, Chen E, Du D, Wu H, Su W (2019) MCNE: an end-to-end framework for learning multiple conditional network representations of social network. In: Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pp 1064–1072 Wang H, Xu T, Liu Q, Lian D, Chen E, Du D, Wu H, Su W (2019) MCNE: an end-to-end framework for learning multiple conditional network representations of social network. In: Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pp 1064–1072
2.
Zurück zum Zitat Ali Z, Qi G, Muhammad K, Bhattacharyya S, Ullah I, Abro W (2022) Citation recommendation employing heterogeneous bibliographic network embedding. Neural Comput Appl 34(13):10229–10242CrossRef Ali Z, Qi G, Muhammad K, Bhattacharyya S, Ullah I, Abro W (2022) Citation recommendation employing heterogeneous bibliographic network embedding. Neural Comput Appl 34(13):10229–10242CrossRef
3.
Zurück zum Zitat Zhang X-M, Liang L, Liu L, Tang M-J (2021) Graph neural networks and their current applications in bioinformatics. Front Genetics 12:690049CrossRef Zhang X-M, Liang L, Liu L, Tang M-J (2021) Graph neural networks and their current applications in bioinformatics. Front Genetics 12:690049CrossRef
4.
Zurück zum Zitat Rahevar M, Ganatra A (2023) Spatial-temporal gated graph attention network for skeleton-based action recognition. Pattern Anal Appl 26:1–11CrossRef Rahevar M, Ganatra A (2023) Spatial-temporal gated graph attention network for skeleton-based action recognition. Pattern Anal Appl 26:1–11CrossRef
5.
Zurück zum Zitat Hamilton WL, Ying R, Leskovec J (2017) Inductive representation learning on large graphs. In: Proceedings of the 31st international conference on neural information processing systems, pp 1025–1035 Hamilton WL, Ying R, Leskovec J (2017) Inductive representation learning on large graphs. In: Proceedings of the 31st international conference on neural information processing systems, pp 1025–1035
6.
Zurück zum Zitat Kipf TN, Welling M (2017) Semi-supervised classification with graph convolutional networks. In: Proceedings of the international conference on learning Kipf TN, Welling M (2017) Semi-supervised classification with graph convolutional networks. In: Proceedings of the international conference on learning
7.
Zurück zum Zitat Chen W, Fang W, Hu G, Mahoney MW (2013) On the hyperbolicity of small-world and treelike random graphs. Internet Math 9(4):434–491MathSciNetCrossRefMATH Chen W, Fang W, Hu G, Mahoney MW (2013) On the hyperbolicity of small-world and treelike random graphs. Internet Math 9(4):434–491MathSciNetCrossRefMATH
8.
Zurück zum Zitat Sarkar R (2011) Low distortion delaunay embedding of trees in hyperbolic plane. In: International symposium on graph drawing. Springer, pp. 355–366 Sarkar R (2011) Low distortion delaunay embedding of trees in hyperbolic plane. In: International symposium on graph drawing. Springer, pp. 355–366
9.
Zurück zum Zitat Chami I, Ying Z, Ré C, Leskovec J (2019) Hyperbolic graph convolutional neural networks. Adv Neural Inf Process Syst 32:4868–4879 Chami I, Ying Z, Ré C, Leskovec J (2019) Hyperbolic graph convolutional neural networks. Adv Neural Inf Process Syst 32:4868–4879
10.
Zurück zum Zitat Nickel M, Kiela D (2017) Poincaré embeddings for learning hierarchical representations. Adv Neural Inf Process Syst 30:6338–6347 Nickel M, Kiela D (2017) Poincaré embeddings for learning hierarchical representations. Adv Neural Inf Process Syst 30:6338–6347
11.
Zurück zum Zitat Krioukov D, Papadopoulos F, Kitsak M, Vahdat A, Boguná M (2010) Hyperbolic geometry of complex networks. Phys Rev E 82(3):036106MathSciNetCrossRef Krioukov D, Papadopoulos F, Kitsak M, Vahdat A, Boguná M (2010) Hyperbolic geometry of complex networks. Phys Rev E 82(3):036106MathSciNetCrossRef
12.
Zurück zum Zitat Muscoloni A, Thomas JM, Ciucci S, Bianconi G, Cannistraci CV (2017) Machine learning meets complex networks via coalescent embedding in the hyperbolic space. Nat Commun 8(1):1–19CrossRef Muscoloni A, Thomas JM, Ciucci S, Bianconi G, Cannistraci CV (2017) Machine learning meets complex networks via coalescent embedding in the hyperbolic space. Nat Commun 8(1):1–19CrossRef
13.
Zurück zum Zitat Dai J, Wu Y, Gao Z, Jia Y (2021) A hyperbolic-to-hyperbolic graph convolutional network. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 154–163 Dai J, Wu Y, Gao Z, Jia Y (2021) A hyperbolic-to-hyperbolic graph convolutional network. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 154–163
14.
Zurück zum Zitat Dong Y, Ding K, Jalaian B, Ji S, Li J (2021) AdaGNN: graph neural networks with adaptive frequency response filter. In: Proceedings of the 30th ACM international conference on information & knowledge management, pp 392–401 Dong Y, Ding K, Jalaian B, Ji S, Li J (2021) AdaGNN: graph neural networks with adaptive frequency response filter. In: Proceedings of the 30th ACM international conference on information & knowledge management, pp 392–401
15.
Zurück zum Zitat Wu F, Souza A, Zhang T, Fifty C, Yu T, Weinberger K (2019) Simplifying graph convolutional networks. In: International conference on machine learning. PMLR, pp 6861–6871 Wu F, Souza A, Zhang T, Fifty C, Yu T, Weinberger K (2019) Simplifying graph convolutional networks. In: International conference on machine learning. PMLR, pp 6861–6871
16.
Zurück zum Zitat Gori M, Monfardini G, Scarselli F (2005) A new model for learning in graph domains. In: Proceedings. 2005 IEEE international joint conference on neural networks, vol 2, pp 729–734 Gori M, Monfardini G, Scarselli F (2005) A new model for learning in graph domains. In: Proceedings. 2005 IEEE international joint conference on neural networks, vol 2, pp 729–734
17.
Zurück zum Zitat Scarselli F, Gori M, Tsoi AC, Hagenbuchner M, Monfardini G (2008) The graph neural network model. IEEE Trans Neural Netw 20(1):61–80CrossRef Scarselli F, Gori M, Tsoi AC, Hagenbuchner M, Monfardini G (2008) The graph neural network model. IEEE Trans Neural Netw 20(1):61–80CrossRef
18.
Zurück zum Zitat Bruna J, Zaremba W, Szlam A, LeCun Y (2014) Spectral networks and locally connected networks on graphs. In: Proceedings of the international conference on learning representations Bruna J, Zaremba W, Szlam A, LeCun Y (2014) Spectral networks and locally connected networks on graphs. In: Proceedings of the international conference on learning representations
19.
Zurück zum Zitat Defferrard M, Bresson X, Vandergheynst P (2016) Convolutional neural networks on graphs with fast localized spectral filtering. Adv Neural Inf Process Syst 29 Defferrard M, Bresson X, Vandergheynst P (2016) Convolutional neural networks on graphs with fast localized spectral filtering. Adv Neural Inf Process Syst 29
20.
Zurück zum Zitat Duvenaud DK, Maclaurin D, Iparraguirre J, Bombarell R, Hirzel T, Aspuru-Guzik A, Adams RP (2015) Convolutional networks on graphs for learning molecular fingerprints. Adv Neural Inf Process Syst 28 Duvenaud DK, Maclaurin D, Iparraguirre J, Bombarell R, Hirzel T, Aspuru-Guzik A, Adams RP (2015) Convolutional networks on graphs for learning molecular fingerprints. Adv Neural Inf Process Syst 28
21.
Zurück zum Zitat Gilmer J, Schoenholz SS, Riley PF, Vinyals O, Dahl GE (2017) Neural message passing for quantum chemistry. In: International conference on machine learning. PMLR, pp. 1263–1272 Gilmer J, Schoenholz SS, Riley PF, Vinyals O, Dahl GE (2017) Neural message passing for quantum chemistry. In: International conference on machine learning. PMLR, pp. 1263–1272
22.
Zurück zum Zitat Papadopoulos F, Kitsak M, Serrano M, Boguná M, Krioukov D (2012) Popularity versus similarity in growing networks. Nature 489(7417):537–540CrossRef Papadopoulos F, Kitsak M, Serrano M, Boguná M, Krioukov D (2012) Popularity versus similarity in growing networks. Nature 489(7417):537–540CrossRef
23.
Zurück zum Zitat Nickel M, Kiela D (2018) Learning continuous hierarchies in the Lorentz model of hyperbolic geometry. In: International conference on machine learning. PMLR, pp 3779–3788 Nickel M, Kiela D (2018) Learning continuous hierarchies in the Lorentz model of hyperbolic geometry. In: International conference on machine learning. PMLR, pp 3779–3788
24.
Zurück zum Zitat Balazevic I, Allen C, Hospedales T (2019) Multi-relational Poincaré graph embeddings. Adv Neural Inf Process Syst 32 Balazevic I, Allen C, Hospedales T (2019) Multi-relational Poincaré graph embeddings. Adv Neural Inf Process Syst 32
25.
Zurück zum Zitat Sun Z, Chen M, Hu W, Wang C, Dai J, Zhang W (2020) Knowledge association with hyperbolic knowledge graph embeddings. In: EMNLP, pp 5704–5716 Sun Z, Chen M, Hu W, Wang C, Dai J, Zhang W (2020) Knowledge association with hyperbolic knowledge graph embeddings. In: EMNLP, pp 5704–5716
26.
Zurück zum Zitat Khrulkov V, Mirvakhabova L, Ustinova E, Oseledets I, Lempitsky V (2020) Hyperbolic image embeddings. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 6418–6428 Khrulkov V, Mirvakhabova L, Ustinova E, Oseledets I, Lempitsky V (2020) Hyperbolic image embeddings. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 6418–6428
27.
Zurück zum Zitat Liu S, Chen J, Pan L, Ngo C-W, Chua T-S, Jiang Y-G (2020) Hyperbolic visual embedding learning for zero-shot recognition. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 9273–9281 Liu S, Chen J, Pan L, Ngo C-W, Chua T-S, Jiang Y-G (2020) Hyperbolic visual embedding learning for zero-shot recognition. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 9273–9281
28.
Zurück zum Zitat Bronstein MM, Bruna J, LeCun Y, Szlam A, Vandergheynst P (2017) Geometric deep learning: going beyond Euclidean data. IEEE Signal Process Mag 34(4):18–42CrossRef Bronstein MM, Bruna J, LeCun Y, Szlam A, Vandergheynst P (2017) Geometric deep learning: going beyond Euclidean data. IEEE Signal Process Mag 34(4):18–42CrossRef
29.
Zurück zum Zitat Liu Q, Nickel M, Kiela D (2019) Hyperbolic graph neural networks. Adv Neural Inf Process Syst 32 Liu Q, Nickel M, Kiela D (2019) Hyperbolic graph neural networks. Adv Neural Inf Process Syst 32
30.
Zurück zum Zitat Zhuang C, Ma Q (2018) Dual graph convolutional networks for graph-based semi-supervised classification. In: Proceedings of the 2018 world wide web conference, pp 499–508 Zhuang C, Ma Q (2018) Dual graph convolutional networks for graph-based semi-supervised classification. In: Proceedings of the 2018 world wide web conference, pp 499–508
31.
Zurück zum Zitat Atwood J, Towsley D (2016) Diffusion-convolutional neural networks. Adv Neural Inf Process Syst 29 Atwood J, Towsley D (2016) Diffusion-convolutional neural networks. Adv Neural Inf Process Syst 29
32.
Zurück zum Zitat Dong Y, Liu N, Jalaian B, Li J (2022) Edits: modeling and mitigating data bias for graph neural networks. In: Proceedings of the ACM web conference 2022, pp 1259–1269 Dong Y, Liu N, Jalaian B, Li J (2022) Edits: modeling and mitigating data bias for graph neural networks. In: Proceedings of the ACM web conference 2022, pp 1259–1269
33.
Zurück zum Zitat Levie R, Monti F, Bresson X, Bronstein MM (2018) CayleyNets: graph convolutional neural networks with complex rational spectral filters. IEEE Trans Signal Process 67(1):97–109MathSciNetCrossRefMATH Levie R, Monti F, Bresson X, Bronstein MM (2018) CayleyNets: graph convolutional neural networks with complex rational spectral filters. IEEE Trans Signal Process 67(1):97–109MathSciNetCrossRefMATH
34.
Zurück zum Zitat Hoang N, Maehara T, Murata T (2021) Revisiting graph neural networks: graph filtering perspective. In: 2020 25th international conference on pattern recognition (ICPR). IEEE, pp 8376–8383 Hoang N, Maehara T, Murata T (2021) Revisiting graph neural networks: graph filtering perspective. In: 2020 25th international conference on pattern recognition (ICPR). IEEE, pp 8376–8383
35.
Zurück zum Zitat Bo D, Wang X, Shi C, Shen H (2021) Beyond low-frequency information in graph convolutional networks. arXiv preprint arXiv:2101.00797 Bo D, Wang X, Shi C, Shen H (2021) Beyond low-frequency information in graph convolutional networks. arXiv preprint arXiv:​2101.​00797
36.
Zurück zum Zitat Chen Y, Fan H, Xu B, Yan Z, Kalantidis Y, Rohrbach M, Yan S, Feng J (2019) Drop an octave: reducing spatial redundancy in convolutional neural networks with octave convolution. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 3435–3444 Chen Y, Fan H, Xu B, Yan Z, Kalantidis Y, Rohrbach M, Yan S, Feng J (2019) Drop an octave: reducing spatial redundancy in convolutional neural networks with octave convolution. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 3435–3444
37.
Zurück zum Zitat Robbin JW, Salamon DA (2011) Introduction to differential geometry. ETH, Lecture Notes, preliminary version, 18 Robbin JW, Salamon DA (2011) Introduction to differential geometry. ETH, Lecture Notes, preliminary version, 18
38.
Zurück zum Zitat He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778 He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778
39.
Zurück zum Zitat Xu K, Hu W, Leskovec J, Jegelka S (2018) How powerful are graph neural networks? In: Proceedings of the 7th international conference on learning representations Xu K, Hu W, Leskovec J, Jegelka S (2018) How powerful are graph neural networks? In: Proceedings of the 7th international conference on learning representations
40.
Zurück zum Zitat Veličković P, Cucurull G, Casanova A, Romero A, Lio P, Bengio Y (2018) Graph attention networks. In: International conference on learning representations Veličković P, Cucurull G, Casanova A, Romero A, Lio P, Bengio Y (2018) Graph attention networks. In: International conference on learning representations
41.
Zurück zum Zitat Boothby WM, Boothby WM (2003) An introduction to differentiable manifolds and Riemannian geometry, revised. Gulf Professional Publishing 120 Boothby WM, Boothby WM (2003) An introduction to differentiable manifolds and Riemannian geometry, revised. Gulf Professional Publishing 120
42.
Zurück zum Zitat Fréchet M (1948) Les éléments aléatoires de nature quelconque dans un espace distancié. In: Annales de L’institut Henri Poincaré, vol 10, pp 215–310 Fréchet M (1948) Les éléments aléatoires de nature quelconque dans un espace distancié. In: Annales de L’institut Henri Poincaré, vol 10, pp 215–310
43.
Zurück zum Zitat Ungar AA (2005) Analytic hyperbolic geometry: mathematical foundations and applications. World ScientificCrossRefMATH Ungar AA (2005) Analytic hyperbolic geometry: mathematical foundations and applications. World ScientificCrossRefMATH
44.
Zurück zum Zitat Shimizu R, Mukuta Y, Harada T (2021) Hyperbolic neural networks++. In: Proceedings of the international conference on learning representations Shimizu R, Mukuta Y, Harada T (2021) Hyperbolic neural networks++. In: Proceedings of the international conference on learning representations
45.
Zurück zum Zitat Anderson RM, May RM (1992) Infectious diseases of humans: dynamics and control. Oxford University Press Anderson RM, May RM (1992) Infectious diseases of humans: dynamics and control. Oxford University Press
47.
Zurück zum Zitat Sen P, Namata G, Bilgic M, Getoor L, Galligher B, Eliassi-Rad T (2008) Collective classification in network data. AI Mag 29(3):93–93 Sen P, Namata G, Bilgic M, Getoor L, Galligher B, Eliassi-Rad T (2008) Collective classification in network data. AI Mag 29(3):93–93
48.
Zurück zum Zitat Paszke A, Gross S, Chintala S, Chanan G, Yang E, DeVito Z, Lin Z, Desmaison A, Antiga L, Lerer A (2017) Automatic differentiation in PyTorch Paszke A, Gross S, Chintala S, Chanan G, Yang E, DeVito Z, Lin Z, Desmaison A, Antiga L, Lerer A (2017) Automatic differentiation in PyTorch
49.
Zurück zum Zitat Kingma DP, Ba J (2015) Adam: a method for stochastic optimization. In: Proceedings of ICLR Kingma DP, Ba J (2015) Adam: a method for stochastic optimization. In: Proceedings of ICLR
50.
Zurück zum Zitat Van der Maaten L, Hinton G (2008) Visualizing data using t-SNE. J Mach Learn Res 9(11):2579MATH Van der Maaten L, Hinton G (2008) Visualizing data using t-SNE. J Mach Learn Res 9(11):2579MATH
51.
Zurück zum Zitat Singh R, Gill SS (2023) Edge AI: a survey. Internet of Things and Cyber-Physical Systems Singh R, Gill SS (2023) Edge AI: a survey. Internet of Things and Cyber-Physical Systems
52.
Zurück zum Zitat Elias VRM, Gogineni VC, Martins WA, Werner S (2022) Kernel regression over graphs using random Fourier features. IEEE Trans Signal Process 70:936–949MathSciNetCrossRef Elias VRM, Gogineni VC, Martins WA, Werner S (2022) Kernel regression over graphs using random Fourier features. IEEE Trans Signal Process 70:936–949MathSciNetCrossRef
53.
Zurück zum Zitat Nikhitha NK, Afzal A, Asharaf S (2021) Deep kernel machines: a survey. Pattern Anal Appl 24:537–556CrossRef Nikhitha NK, Afzal A, Asharaf S (2021) Deep kernel machines: a survey. Pattern Anal Appl 24:537–556CrossRef
Metadaten
Titel
Adaptive frequency-based fully hyperbolic graph neural networks
verfasst von
FeiFei Wei
MingZhu Ping
KuiZhi Mei
Publikationsdatum
07.10.2023
Verlag
Springer London
Erschienen in
Pattern Analysis and Applications / Ausgabe 4/2023
Print ISSN: 1433-7541
Elektronische ISSN: 1433-755X
DOI
https://doi.org/10.1007/s10044-023-01201-8

Weitere Artikel der Ausgabe 4/2023

Pattern Analysis and Applications 4/2023 Zur Ausgabe

Premium Partner