Skip to main content
Top
Published in: Pattern Analysis and Applications 4/2023

07-10-2023 | Theoretical Advances

Adaptive frequency-based fully hyperbolic graph neural networks

Authors: FeiFei Wei, MingZhu Ping, KuiZhi Mei

Published in: Pattern Analysis and Applications | Issue 4/2023

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Graph Convolutional Networks (GCNs) have attracted broad attention from industry and academia, for their excellent expressive power in terms of modeling the irregular data, e.g., skeletal data and graph-structured data. The most effective existing model may be the fully hyperbolic graph neural network. However, it involves a large number of parameters, thus consuming considerable computing resources. In this paper, we propose a model based on adaptive frequency filter and corresponding optimizer in hyperbolic space. The adaptive frequency can learn the different frequency components of the embeddings of the nodes in graph, which adaptively adjust the beneficial signals of high-frequency and low-frequency. And the optimizer is based on a subset of the orthogonal constraint, which is dedicated for the adaptive frequency with less parameters. Moreover, our model bridges the gap of hyperbolic space and the spectral space for exploring the underlying semantics of the node and relation embeddings of graph. Consequently, our model needs only to optimize the less parameters in hyperbolic space and meanwhile prevent the distortion caused by conventional manifold GCN. Experimental results show that our method achieves substantial improvements and outperforms the state-of-the-art performance in terms of node classification.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
1.
go back to reference Wang H, Xu T, Liu Q, Lian D, Chen E, Du D, Wu H, Su W (2019) MCNE: an end-to-end framework for learning multiple conditional network representations of social network. In: Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pp 1064–1072 Wang H, Xu T, Liu Q, Lian D, Chen E, Du D, Wu H, Su W (2019) MCNE: an end-to-end framework for learning multiple conditional network representations of social network. In: Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining, pp 1064–1072
2.
go back to reference Ali Z, Qi G, Muhammad K, Bhattacharyya S, Ullah I, Abro W (2022) Citation recommendation employing heterogeneous bibliographic network embedding. Neural Comput Appl 34(13):10229–10242CrossRef Ali Z, Qi G, Muhammad K, Bhattacharyya S, Ullah I, Abro W (2022) Citation recommendation employing heterogeneous bibliographic network embedding. Neural Comput Appl 34(13):10229–10242CrossRef
3.
go back to reference Zhang X-M, Liang L, Liu L, Tang M-J (2021) Graph neural networks and their current applications in bioinformatics. Front Genetics 12:690049CrossRef Zhang X-M, Liang L, Liu L, Tang M-J (2021) Graph neural networks and their current applications in bioinformatics. Front Genetics 12:690049CrossRef
4.
go back to reference Rahevar M, Ganatra A (2023) Spatial-temporal gated graph attention network for skeleton-based action recognition. Pattern Anal Appl 26:1–11CrossRef Rahevar M, Ganatra A (2023) Spatial-temporal gated graph attention network for skeleton-based action recognition. Pattern Anal Appl 26:1–11CrossRef
5.
go back to reference Hamilton WL, Ying R, Leskovec J (2017) Inductive representation learning on large graphs. In: Proceedings of the 31st international conference on neural information processing systems, pp 1025–1035 Hamilton WL, Ying R, Leskovec J (2017) Inductive representation learning on large graphs. In: Proceedings of the 31st international conference on neural information processing systems, pp 1025–1035
6.
go back to reference Kipf TN, Welling M (2017) Semi-supervised classification with graph convolutional networks. In: Proceedings of the international conference on learning Kipf TN, Welling M (2017) Semi-supervised classification with graph convolutional networks. In: Proceedings of the international conference on learning
7.
go back to reference Chen W, Fang W, Hu G, Mahoney MW (2013) On the hyperbolicity of small-world and treelike random graphs. Internet Math 9(4):434–491MathSciNetCrossRefMATH Chen W, Fang W, Hu G, Mahoney MW (2013) On the hyperbolicity of small-world and treelike random graphs. Internet Math 9(4):434–491MathSciNetCrossRefMATH
8.
go back to reference Sarkar R (2011) Low distortion delaunay embedding of trees in hyperbolic plane. In: International symposium on graph drawing. Springer, pp. 355–366 Sarkar R (2011) Low distortion delaunay embedding of trees in hyperbolic plane. In: International symposium on graph drawing. Springer, pp. 355–366
9.
go back to reference Chami I, Ying Z, Ré C, Leskovec J (2019) Hyperbolic graph convolutional neural networks. Adv Neural Inf Process Syst 32:4868–4879 Chami I, Ying Z, Ré C, Leskovec J (2019) Hyperbolic graph convolutional neural networks. Adv Neural Inf Process Syst 32:4868–4879
10.
go back to reference Nickel M, Kiela D (2017) Poincaré embeddings for learning hierarchical representations. Adv Neural Inf Process Syst 30:6338–6347 Nickel M, Kiela D (2017) Poincaré embeddings for learning hierarchical representations. Adv Neural Inf Process Syst 30:6338–6347
11.
go back to reference Krioukov D, Papadopoulos F, Kitsak M, Vahdat A, Boguná M (2010) Hyperbolic geometry of complex networks. Phys Rev E 82(3):036106MathSciNetCrossRef Krioukov D, Papadopoulos F, Kitsak M, Vahdat A, Boguná M (2010) Hyperbolic geometry of complex networks. Phys Rev E 82(3):036106MathSciNetCrossRef
12.
go back to reference Muscoloni A, Thomas JM, Ciucci S, Bianconi G, Cannistraci CV (2017) Machine learning meets complex networks via coalescent embedding in the hyperbolic space. Nat Commun 8(1):1–19CrossRef Muscoloni A, Thomas JM, Ciucci S, Bianconi G, Cannistraci CV (2017) Machine learning meets complex networks via coalescent embedding in the hyperbolic space. Nat Commun 8(1):1–19CrossRef
13.
go back to reference Dai J, Wu Y, Gao Z, Jia Y (2021) A hyperbolic-to-hyperbolic graph convolutional network. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 154–163 Dai J, Wu Y, Gao Z, Jia Y (2021) A hyperbolic-to-hyperbolic graph convolutional network. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 154–163
14.
go back to reference Dong Y, Ding K, Jalaian B, Ji S, Li J (2021) AdaGNN: graph neural networks with adaptive frequency response filter. In: Proceedings of the 30th ACM international conference on information & knowledge management, pp 392–401 Dong Y, Ding K, Jalaian B, Ji S, Li J (2021) AdaGNN: graph neural networks with adaptive frequency response filter. In: Proceedings of the 30th ACM international conference on information & knowledge management, pp 392–401
15.
go back to reference Wu F, Souza A, Zhang T, Fifty C, Yu T, Weinberger K (2019) Simplifying graph convolutional networks. In: International conference on machine learning. PMLR, pp 6861–6871 Wu F, Souza A, Zhang T, Fifty C, Yu T, Weinberger K (2019) Simplifying graph convolutional networks. In: International conference on machine learning. PMLR, pp 6861–6871
16.
go back to reference Gori M, Monfardini G, Scarselli F (2005) A new model for learning in graph domains. In: Proceedings. 2005 IEEE international joint conference on neural networks, vol 2, pp 729–734 Gori M, Monfardini G, Scarselli F (2005) A new model for learning in graph domains. In: Proceedings. 2005 IEEE international joint conference on neural networks, vol 2, pp 729–734
17.
go back to reference Scarselli F, Gori M, Tsoi AC, Hagenbuchner M, Monfardini G (2008) The graph neural network model. IEEE Trans Neural Netw 20(1):61–80CrossRef Scarselli F, Gori M, Tsoi AC, Hagenbuchner M, Monfardini G (2008) The graph neural network model. IEEE Trans Neural Netw 20(1):61–80CrossRef
18.
go back to reference Bruna J, Zaremba W, Szlam A, LeCun Y (2014) Spectral networks and locally connected networks on graphs. In: Proceedings of the international conference on learning representations Bruna J, Zaremba W, Szlam A, LeCun Y (2014) Spectral networks and locally connected networks on graphs. In: Proceedings of the international conference on learning representations
19.
go back to reference Defferrard M, Bresson X, Vandergheynst P (2016) Convolutional neural networks on graphs with fast localized spectral filtering. Adv Neural Inf Process Syst 29 Defferrard M, Bresson X, Vandergheynst P (2016) Convolutional neural networks on graphs with fast localized spectral filtering. Adv Neural Inf Process Syst 29
20.
go back to reference Duvenaud DK, Maclaurin D, Iparraguirre J, Bombarell R, Hirzel T, Aspuru-Guzik A, Adams RP (2015) Convolutional networks on graphs for learning molecular fingerprints. Adv Neural Inf Process Syst 28 Duvenaud DK, Maclaurin D, Iparraguirre J, Bombarell R, Hirzel T, Aspuru-Guzik A, Adams RP (2015) Convolutional networks on graphs for learning molecular fingerprints. Adv Neural Inf Process Syst 28
21.
go back to reference Gilmer J, Schoenholz SS, Riley PF, Vinyals O, Dahl GE (2017) Neural message passing for quantum chemistry. In: International conference on machine learning. PMLR, pp. 1263–1272 Gilmer J, Schoenholz SS, Riley PF, Vinyals O, Dahl GE (2017) Neural message passing for quantum chemistry. In: International conference on machine learning. PMLR, pp. 1263–1272
22.
go back to reference Papadopoulos F, Kitsak M, Serrano M, Boguná M, Krioukov D (2012) Popularity versus similarity in growing networks. Nature 489(7417):537–540CrossRef Papadopoulos F, Kitsak M, Serrano M, Boguná M, Krioukov D (2012) Popularity versus similarity in growing networks. Nature 489(7417):537–540CrossRef
23.
go back to reference Nickel M, Kiela D (2018) Learning continuous hierarchies in the Lorentz model of hyperbolic geometry. In: International conference on machine learning. PMLR, pp 3779–3788 Nickel M, Kiela D (2018) Learning continuous hierarchies in the Lorentz model of hyperbolic geometry. In: International conference on machine learning. PMLR, pp 3779–3788
24.
go back to reference Balazevic I, Allen C, Hospedales T (2019) Multi-relational Poincaré graph embeddings. Adv Neural Inf Process Syst 32 Balazevic I, Allen C, Hospedales T (2019) Multi-relational Poincaré graph embeddings. Adv Neural Inf Process Syst 32
25.
go back to reference Sun Z, Chen M, Hu W, Wang C, Dai J, Zhang W (2020) Knowledge association with hyperbolic knowledge graph embeddings. In: EMNLP, pp 5704–5716 Sun Z, Chen M, Hu W, Wang C, Dai J, Zhang W (2020) Knowledge association with hyperbolic knowledge graph embeddings. In: EMNLP, pp 5704–5716
26.
go back to reference Khrulkov V, Mirvakhabova L, Ustinova E, Oseledets I, Lempitsky V (2020) Hyperbolic image embeddings. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 6418–6428 Khrulkov V, Mirvakhabova L, Ustinova E, Oseledets I, Lempitsky V (2020) Hyperbolic image embeddings. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 6418–6428
27.
go back to reference Liu S, Chen J, Pan L, Ngo C-W, Chua T-S, Jiang Y-G (2020) Hyperbolic visual embedding learning for zero-shot recognition. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 9273–9281 Liu S, Chen J, Pan L, Ngo C-W, Chua T-S, Jiang Y-G (2020) Hyperbolic visual embedding learning for zero-shot recognition. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 9273–9281
28.
go back to reference Bronstein MM, Bruna J, LeCun Y, Szlam A, Vandergheynst P (2017) Geometric deep learning: going beyond Euclidean data. IEEE Signal Process Mag 34(4):18–42CrossRef Bronstein MM, Bruna J, LeCun Y, Szlam A, Vandergheynst P (2017) Geometric deep learning: going beyond Euclidean data. IEEE Signal Process Mag 34(4):18–42CrossRef
29.
go back to reference Liu Q, Nickel M, Kiela D (2019) Hyperbolic graph neural networks. Adv Neural Inf Process Syst 32 Liu Q, Nickel M, Kiela D (2019) Hyperbolic graph neural networks. Adv Neural Inf Process Syst 32
30.
go back to reference Zhuang C, Ma Q (2018) Dual graph convolutional networks for graph-based semi-supervised classification. In: Proceedings of the 2018 world wide web conference, pp 499–508 Zhuang C, Ma Q (2018) Dual graph convolutional networks for graph-based semi-supervised classification. In: Proceedings of the 2018 world wide web conference, pp 499–508
31.
go back to reference Atwood J, Towsley D (2016) Diffusion-convolutional neural networks. Adv Neural Inf Process Syst 29 Atwood J, Towsley D (2016) Diffusion-convolutional neural networks. Adv Neural Inf Process Syst 29
32.
go back to reference Dong Y, Liu N, Jalaian B, Li J (2022) Edits: modeling and mitigating data bias for graph neural networks. In: Proceedings of the ACM web conference 2022, pp 1259–1269 Dong Y, Liu N, Jalaian B, Li J (2022) Edits: modeling and mitigating data bias for graph neural networks. In: Proceedings of the ACM web conference 2022, pp 1259–1269
33.
go back to reference Levie R, Monti F, Bresson X, Bronstein MM (2018) CayleyNets: graph convolutional neural networks with complex rational spectral filters. IEEE Trans Signal Process 67(1):97–109MathSciNetCrossRefMATH Levie R, Monti F, Bresson X, Bronstein MM (2018) CayleyNets: graph convolutional neural networks with complex rational spectral filters. IEEE Trans Signal Process 67(1):97–109MathSciNetCrossRefMATH
34.
go back to reference Hoang N, Maehara T, Murata T (2021) Revisiting graph neural networks: graph filtering perspective. In: 2020 25th international conference on pattern recognition (ICPR). IEEE, pp 8376–8383 Hoang N, Maehara T, Murata T (2021) Revisiting graph neural networks: graph filtering perspective. In: 2020 25th international conference on pattern recognition (ICPR). IEEE, pp 8376–8383
35.
36.
go back to reference Chen Y, Fan H, Xu B, Yan Z, Kalantidis Y, Rohrbach M, Yan S, Feng J (2019) Drop an octave: reducing spatial redundancy in convolutional neural networks with octave convolution. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 3435–3444 Chen Y, Fan H, Xu B, Yan Z, Kalantidis Y, Rohrbach M, Yan S, Feng J (2019) Drop an octave: reducing spatial redundancy in convolutional neural networks with octave convolution. In: Proceedings of the IEEE/CVF international conference on computer vision, pp 3435–3444
37.
go back to reference Robbin JW, Salamon DA (2011) Introduction to differential geometry. ETH, Lecture Notes, preliminary version, 18 Robbin JW, Salamon DA (2011) Introduction to differential geometry. ETH, Lecture Notes, preliminary version, 18
38.
go back to reference He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778 He K, Zhang X, Ren S, Sun J (2016) Deep residual learning for image recognition. In: Proceedings of the IEEE conference on computer vision and pattern recognition, pp 770–778
39.
go back to reference Xu K, Hu W, Leskovec J, Jegelka S (2018) How powerful are graph neural networks? In: Proceedings of the 7th international conference on learning representations Xu K, Hu W, Leskovec J, Jegelka S (2018) How powerful are graph neural networks? In: Proceedings of the 7th international conference on learning representations
40.
go back to reference Veličković P, Cucurull G, Casanova A, Romero A, Lio P, Bengio Y (2018) Graph attention networks. In: International conference on learning representations Veličković P, Cucurull G, Casanova A, Romero A, Lio P, Bengio Y (2018) Graph attention networks. In: International conference on learning representations
41.
go back to reference Boothby WM, Boothby WM (2003) An introduction to differentiable manifolds and Riemannian geometry, revised. Gulf Professional Publishing 120 Boothby WM, Boothby WM (2003) An introduction to differentiable manifolds and Riemannian geometry, revised. Gulf Professional Publishing 120
42.
go back to reference Fréchet M (1948) Les éléments aléatoires de nature quelconque dans un espace distancié. In: Annales de L’institut Henri Poincaré, vol 10, pp 215–310 Fréchet M (1948) Les éléments aléatoires de nature quelconque dans un espace distancié. In: Annales de L’institut Henri Poincaré, vol 10, pp 215–310
43.
go back to reference Ungar AA (2005) Analytic hyperbolic geometry: mathematical foundations and applications. World ScientificCrossRefMATH Ungar AA (2005) Analytic hyperbolic geometry: mathematical foundations and applications. World ScientificCrossRefMATH
44.
go back to reference Shimizu R, Mukuta Y, Harada T (2021) Hyperbolic neural networks++. In: Proceedings of the international conference on learning representations Shimizu R, Mukuta Y, Harada T (2021) Hyperbolic neural networks++. In: Proceedings of the international conference on learning representations
45.
go back to reference Anderson RM, May RM (1992) Infectious diseases of humans: dynamics and control. Oxford University Press Anderson RM, May RM (1992) Infectious diseases of humans: dynamics and control. Oxford University Press
47.
go back to reference Sen P, Namata G, Bilgic M, Getoor L, Galligher B, Eliassi-Rad T (2008) Collective classification in network data. AI Mag 29(3):93–93 Sen P, Namata G, Bilgic M, Getoor L, Galligher B, Eliassi-Rad T (2008) Collective classification in network data. AI Mag 29(3):93–93
48.
go back to reference Paszke A, Gross S, Chintala S, Chanan G, Yang E, DeVito Z, Lin Z, Desmaison A, Antiga L, Lerer A (2017) Automatic differentiation in PyTorch Paszke A, Gross S, Chintala S, Chanan G, Yang E, DeVito Z, Lin Z, Desmaison A, Antiga L, Lerer A (2017) Automatic differentiation in PyTorch
49.
go back to reference Kingma DP, Ba J (2015) Adam: a method for stochastic optimization. In: Proceedings of ICLR Kingma DP, Ba J (2015) Adam: a method for stochastic optimization. In: Proceedings of ICLR
50.
go back to reference Van der Maaten L, Hinton G (2008) Visualizing data using t-SNE. J Mach Learn Res 9(11):2579MATH Van der Maaten L, Hinton G (2008) Visualizing data using t-SNE. J Mach Learn Res 9(11):2579MATH
51.
go back to reference Singh R, Gill SS (2023) Edge AI: a survey. Internet of Things and Cyber-Physical Systems Singh R, Gill SS (2023) Edge AI: a survey. Internet of Things and Cyber-Physical Systems
52.
go back to reference Elias VRM, Gogineni VC, Martins WA, Werner S (2022) Kernel regression over graphs using random Fourier features. IEEE Trans Signal Process 70:936–949MathSciNetCrossRef Elias VRM, Gogineni VC, Martins WA, Werner S (2022) Kernel regression over graphs using random Fourier features. IEEE Trans Signal Process 70:936–949MathSciNetCrossRef
53.
go back to reference Nikhitha NK, Afzal A, Asharaf S (2021) Deep kernel machines: a survey. Pattern Anal Appl 24:537–556CrossRef Nikhitha NK, Afzal A, Asharaf S (2021) Deep kernel machines: a survey. Pattern Anal Appl 24:537–556CrossRef
Metadata
Title
Adaptive frequency-based fully hyperbolic graph neural networks
Authors
FeiFei Wei
MingZhu Ping
KuiZhi Mei
Publication date
07-10-2023
Publisher
Springer London
Published in
Pattern Analysis and Applications / Issue 4/2023
Print ISSN: 1433-7541
Electronic ISSN: 1433-755X
DOI
https://doi.org/10.1007/s10044-023-01201-8

Other articles of this Issue 4/2023

Pattern Analysis and Applications 4/2023 Go to the issue

Premium Partner