Skip to main content
Top
Published in: Data Mining and Knowledge Discovery 6/2023

26-07-2023

Datasets, tasks, and training methods for large-scale hypergraph learning

Authors: Sunwoo Kim, Dongjin Lee, Yul Kim, Jungho Park, Taeho Hwang, Kijung Shin

Published in: Data Mining and Knowledge Discovery | Issue 6/2023

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Relations among multiple entities are prevalent in many fields, and hypergraphs are widely used to represent such group relations. Hence, machine learning on hypergraphs has received considerable attention, and especially much effort has been made in neural network architectures for hypergraphs (a.k.a., hypergraph neural networks). However, existing studies mostly focused on small datasets for a few single-entity-level downstream tasks and overlooked scalability issues, although most real-world group relations are large-scale. In this work, we propose new tasks, datasets, and scalable training methods for addressing these limitations. First, we introduce two pair-level hypergraph-learning tasks to formulate a wide range of real-world problems. Then, we build and publicly release two large-scale hypergraph datasets with tens of millions of nodes, rich features, and labels. After that, we propose PCL, a scalable learning method for hypergraph neural networks. To tackle scalability issues, PCL splits a given hypergraph into partitions and trains a neural network via contrastive learning. Our extensive experiments demonstrate that hypergraph neural networks can be trained for large-scale hypergraphs by PCL while outperforming 16 baseline models. Specifically, the performance is comparable, or surprisingly even better than that achieved by training hypergraph neural networks on the entire hypergraphs without partitioning.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Appendix
Available only for authorised users
Footnotes
1
For example, in an unsupervised setting without any given positive example pairs, one can additionally split edges in \(\mathcal {E}'\) to use them as positive example pairs for training.
 
2
For example, it can be considered in unsupervised settings especially when the clustering membership is strongly correlated with node features and/or topological information.
 
5
Available at https://​scholar.​google.​com/​citations?​view_​op=​top_​venues &​hl=​en &​vq=​eng. In this taxonomy, a single venue is associated with a single sub-field.
 
6
PaToH is a balanced partitioning method. It ensures that all generated partitions are of similar sizes (Çatalyürek and Aykanat 2011), specifically satisfying \(\vert \mathcal {P}_{k}^{\mathcal {V}}\vert \le \frac{(1 + \epsilon )}{\vert \mathcal {P}\vert } \sum _{i=1}^{\vert \mathcal {P}\vert } \vert \mathcal {P}_{i}^{\mathcal {V}}\vert , \ \forall k=1,\cdots ,\vert \mathcal {P}\vert\). As shown in Table 8 in Sect. 6.3.5, partitions obtained by PaToH from real-world hypergraphs are well balanced. Specifically, the standard deviation of the number of nodes in each partition is less than 0.5% of the average number of nodes per partition.
 
7
One can set K based on the available amount of space (low K takes more memory consumption in general). Note that the performance of the proposed method is not significantly affected by K, which will be demonstrated in Sect. 6.3.5.
 
8
Note that other self-supervised losses (e.g., (Addanki et al. 2021)) can be used alternatively.
 
9
Since graph encoders require a graph topology as an input, we convert original hypergraphs into graphs by Clique Expansion, described in Appendix A.2.
 
10
This is because partitioning algorithms generally assign such nodes in the same partition.
 
11
At each mini-batch (partition) of contrastive learning, we record the GPU memory usage after completing the gradient computation (spec., execute loss.backward() and check the current GPU memory allocation using torch.cuda.memory_allocated(device)). After training an encoder in every mini-batch, we calculate the average GPU memory usage for the current epoch by averaging the usage across all partitions. Finally, we compute the average GPU memory usage across all epochs.
 
12
The total contrastive training epochs are 50.
 
Literature
go back to reference Chiang WL, Liu X, Si S, et al (2019) Cluster-GCN: An efficient algorithm for training deep and large graph convolutional networks. In: Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining (KDD), pp 257–266, https://doi.org/10.1145/3292500.3330925 Chiang WL, Liu X, Si S, et al (2019) Cluster-GCN: An efficient algorithm for training deep and large graph convolutional networks. In: Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery & data mining (KDD), pp 257–266, https://​doi.​org/​10.​1145/​3292500.​3330925
go back to reference Hwang H, Lee S, Park C, et al (2022) AHP: learning to negative sample for hyperedge prediction. In: Proceedings of the 45th International ACM SIGIR conference on research and development in information retrieval (SIGIR), pp 2237–2242, https://doi.org/10.1145/3477495.3531836 Hwang H, Lee S, Park C, et al (2022) AHP: learning to negative sample for hyperedge prediction. In: Proceedings of the 45th International ACM SIGIR conference on research and development in information retrieval (SIGIR), pp 2237–2242, https://​doi.​org/​10.​1145/​3477495.​3531836
go back to reference Nair V, Hinton GE (2010) Rectified linear units improve restricted boltzmann machines. In: Proceedings of the 27th international conference on machine learning (ICML-10), pp 807–814 Nair V, Hinton GE (2010) Rectified linear units improve restricted boltzmann machines. In: Proceedings of the 27th international conference on machine learning (ICML-10), pp 807–814
Metadata
Title
Datasets, tasks, and training methods for large-scale hypergraph learning
Authors
Sunwoo Kim
Dongjin Lee
Yul Kim
Jungho Park
Taeho Hwang
Kijung Shin
Publication date
26-07-2023
Publisher
Springer US
Published in
Data Mining and Knowledge Discovery / Issue 6/2023
Print ISSN: 1384-5810
Electronic ISSN: 1573-756X
DOI
https://doi.org/10.1007/s10618-023-00952-6

Other articles of this Issue 6/2023

Data Mining and Knowledge Discovery 6/2023 Go to the issue

Premium Partner