Skip to main content

10.04.2024 | Regular Paper

Deep graph clustering via mutual information maximization and mixture model

verfasst von: Maedeh Ahmadi, Mehran Safayani, Abdolreza Mirzaei

Erschienen in: Knowledge and Information Systems

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Attributed graph clustering or community detection which learns to cluster the nodes of a graph is a challenging task in graph analysis. Recently contrastive learning has shown significant results in various unsupervised graph learning tasks. In spite of the success of graph contrastive learning methods in self-supervised graph learning, using them for graph clustering is not well explored. In this paper, we introduce a contrastive learning framework for learning clustering-friendly node embedding. We propose Gaussian mixture information maximization which utilizes a mutual information maximization approach for node embedding. Meanwhile, in order to have a clustering-friendly embedding space, it imposes a mixture of Gaussians distribution on this space. The parameters of the contrastive node embedding model and the mixture distribution are optimized jointly in a unified framework. Experiments show that our clustering-directed embedding space can enhance clustering performance in comparison with the case where community structure of the graph is ignored during node representation learning. The results on real-world datasets demonstrate the effectiveness of our method in community detection.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Anhänge
Nur mit Berechtigung zugänglich
Literatur
1.
Zurück zum Zitat Yang J, McAuley J, Leskovec J (2013) Community detection in networks with node attributes. In: 2013 IEEE 13th international conference on data mining. IEEE, pp 1151–1156 Yang J, McAuley J, Leskovec J (2013) Community detection in networks with node attributes. In: 2013 IEEE 13th international conference on data mining. IEEE, pp 1151–1156
2.
Zurück zum Zitat Chen P, Redner S (2010) Community structure of the physical review citation network. J Informetr 4(3):278–290CrossRef Chen P, Redner S (2010) Community structure of the physical review citation network. J Informetr 4(3):278–290CrossRef
3.
Zurück zum Zitat Nicolini C, Bordier C, Bifone A (2017) Community detection in weighted brain connectivity networks beyond the resolution limit. Neuroimage 146:28–39CrossRef Nicolini C, Bordier C, Bifone A (2017) Community detection in weighted brain connectivity networks beyond the resolution limit. Neuroimage 146:28–39CrossRef
4.
Zurück zum Zitat Chen J, Yuan B (2006) Detecting functional modules in the yeast protein-protein interaction network. Bioinformatics 22(18):2283–2290CrossRef Chen J, Yuan B (2006) Detecting functional modules in the yeast protein-protein interaction network. Bioinformatics 22(18):2283–2290CrossRef
5.
Zurück zum Zitat Wang X, Cui P, Wang J, Pei J, Zhu W, Yang S (2017) Community preserving network embedding. In: Thirty-first AAAI conference on artificial intelligence Wang X, Cui P, Wang J, Pei J, Zhu W, Yang S (2017) Community preserving network embedding. In: Thirty-first AAAI conference on artificial intelligence
6.
Zurück zum Zitat Yang J, Leskovec J (2013) Overlapping community detection at scale: a nonnegative matrix factorization approach. In: Proceedings of the sixth ACM international conference on web search and data mining, pp 587–596 Yang J, Leskovec J (2013) Overlapping community detection at scale: a nonnegative matrix factorization approach. In: Proceedings of the sixth ACM international conference on web search and data mining, pp 587–596
7.
Zurück zum Zitat Pan S, Hu R, Fung S-F, Long G, Jiang J, Zhang C (2019) Learning graph embedding with adversarial training methods. IEEE Trans Cybern 50(6):2475–2487CrossRef Pan S, Hu R, Fung S-F, Long G, Jiang J, Zhang C (2019) Learning graph embedding with adversarial training methods. IEEE Trans Cybern 50(6):2475–2487CrossRef
8.
Zurück zum Zitat Wang C, Pan S, Hu R, Long G, Jiang J, Zhang C (2019) Attributed graph clustering: a deep attentional embedding approach. In: Proceedings of the 28th international joint conference on artificial intelligence, pp 3670–3676 Wang C, Pan S, Hu R, Long G, Jiang J, Zhang C (2019) Attributed graph clustering: a deep attentional embedding approach. In: Proceedings of the 28th international joint conference on artificial intelligence, pp 3670–3676
10.
Zurück zum Zitat Wang C, Pan S, Long G, Zhu X, Jiang J (2017) Mgae: marginalized graph autoencoder for graph clustering. In: Proceedings of the 2017 ACM on conference on information and knowledge management, pp 889–898 Wang C, Pan S, Long G, Zhu X, Jiang J (2017) Mgae: marginalized graph autoencoder for graph clustering. In: Proceedings of the 2017 ACM on conference on information and knowledge management, pp 889–898
11.
Zurück zum Zitat Zhang X, Liu H, Li Q, Wu X-M (2019) Attributed graph clustering via adaptive graph convolution. In: Proceedings of the 28th international joint conference on artificial intelligence, pp 4327–4333 Zhang X, Liu H, Li Q, Wu X-M (2019) Attributed graph clustering via adaptive graph convolution. In: Proceedings of the 28th international joint conference on artificial intelligence, pp 4327–4333
12.
Zurück zum Zitat Sun F-Y, Hoffman J, Verma V, Tang J (2020) Infograph: unsupervised and semi-supervised graph-level representation learning via mutual information maximization. In: International conference on learning representations Sun F-Y, Hoffman J, Verma V, Tang J (2020) Infograph: unsupervised and semi-supervised graph-level representation learning via mutual information maximization. In: International conference on learning representations
13.
Zurück zum Zitat Velickovic P, Fedus W, Hamilton WL, Liò P, Bengio Y, Hjelm RD (2019) Deep graph infomax. ICLR (Poster) 2(3):4 Velickovic P, Fedus W, Hamilton WL, Liò P, Bengio Y, Hjelm RD (2019) Deep graph infomax. ICLR (Poster) 2(3):4
14.
Zurück zum Zitat Hassani K, Khasahmadi AH (2020) Contrastive multi-view representation learning on graphs. In: International conference on machine learning. PMLR, pp 4116–4126 Hassani K, Khasahmadi AH (2020) Contrastive multi-view representation learning on graphs. In: International conference on machine learning. PMLR, pp 4116–4126
15.
Zurück zum Zitat Jiao Y, Xiong Y, Zhang J, Zhang Y, Zhang T, Zhu Y (2022) Scalable self-supervised graph representation learning via enhancing and contrasting subgraphs. Knowl Inf Syst 64(1):235–260CrossRef Jiao Y, Xiong Y, Zhang J, Zhang Y, Zhang T, Zhu Y (2022) Scalable self-supervised graph representation learning via enhancing and contrasting subgraphs. Knowl Inf Syst 64(1):235–260CrossRef
16.
Zurück zum Zitat Liu Y, Yang X, Zhou S, Liu X, Wang S, Liang K, Tu W, Li L (2023) Simple contrastive graph clustering. IEEE Trans Neural Netw Learn Syst Liu Y, Yang X, Zhou S, Liu X, Wang S, Liang K, Tu W, Li L (2023) Simple contrastive graph clustering. IEEE Trans Neural Netw Learn Syst
17.
Zurück zum Zitat Jiang Z, Zheng Y, Tan H, Tang B, Zhou H (2016) Variational deep embedding: An unsupervised and generative approach to clustering. arXiv:1611.05148 Jiang Z, Zheng Y, Tan H, Tang B, Zhou H (2016) Variational deep embedding: An unsupervised and generative approach to clustering. arXiv:​1611.​05148
18.
Zurück zum Zitat Uğur Y, Arvanitakis G, Zaidi A (2020) Variational information bottleneck for unsupervised clustering: deep gaussian mixture embedding. Entropy 22(2):213MathSciNetCrossRef Uğur Y, Arvanitakis G, Zaidi A (2020) Variational information bottleneck for unsupervised clustering: deep gaussian mixture embedding. Entropy 22(2):213MathSciNetCrossRef
20.
Zurück zum Zitat Klicpera J, Weißenberger S, Günnemann S (2019) Diffusion improves graph learning. In: Proceedings of the 33rd international conference on neural information processing systems, pp 13366–13378 Klicpera J, Weißenberger S, Günnemann S (2019) Diffusion improves graph learning. In: Proceedings of the 33rd international conference on neural information processing systems, pp 13366–13378
21.
Zurück zum Zitat Newman ME, Girvan M (2004) Finding and evaluating community structure in networks. Phys Rev E 69(2):026113CrossRef Newman ME, Girvan M (2004) Finding and evaluating community structure in networks. Phys Rev E 69(2):026113CrossRef
22.
Zurück zum Zitat Perozzi B, Al-Rfou R, Skiena S (2014) Deepwalk: Online learning of social representations. In: Proceedings of the 20th ACM SIGKDD international conference on knowledge discovery and data mining, pp 701–710 Perozzi B, Al-Rfou R, Skiena S (2014) Deepwalk: Online learning of social representations. In: Proceedings of the 20th ACM SIGKDD international conference on knowledge discovery and data mining, pp 701–710
23.
Zurück zum Zitat Grover A, Leskovec J (2016) node2vec: Scalable feature learning for networks. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp 855–864 Grover A, Leskovec J (2016) node2vec: Scalable feature learning for networks. In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp 855–864
24.
Zurück zum Zitat Le Q, Mikolov T (2014) Distributed representations of sentences and documents. In: International conference on machine learning. PMLR, pp 1188–1196 Le Q, Mikolov T (2014) Distributed representations of sentences and documents. In: International conference on machine learning. PMLR, pp 1188–1196
26.
Zurück zum Zitat Veličković P, Cucurull G, Casanova A, Romero A, Liò P, Bengio Y (2018) Graph attention networks. In: International conference on learning representations Veličković P, Cucurull G, Casanova A, Romero A, Liò P, Bengio Y (2018) Graph attention networks. In: International conference on learning representations
27.
Zurück zum Zitat Xu K, Hu W, Leskovec J, Jegelka S (2019) How powerful are graph neural networks? In: International conference on learning representations Xu K, Hu W, Leskovec J, Jegelka S (2019) How powerful are graph neural networks? In: International conference on learning representations
28.
Zurück zum Zitat You Y, Chen T, Sui Y, Chen T, Wang Z, Shen Y (2020) Graph contrastive learning with augmentations. Adv Neural Inf Process Syst 33:5812–5823 You Y, Chen T, Sui Y, Chen T, Wang Z, Shen Y (2020) Graph contrastive learning with augmentations. Adv Neural Inf Process Syst 33:5812–5823
29.
Zurück zum Zitat Bachman P, Hjelm RD, Buchwalter W (2019) Learning representations by maximizing mutual information across views. Adv Neural Inf Process Syst 32 Bachman P, Hjelm RD, Buchwalter W (2019) Learning representations by maximizing mutual information across views. Adv Neural Inf Process Syst 32
30.
Zurück zum Zitat Chen T, Kornblith S, Norouzi M, Hinton G (2020) A simple framework for contrastive learning of visual representations. In: International conference on machine learning. PMLR, pp 1597–1607 Chen T, Kornblith S, Norouzi M, Hinton G (2020) A simple framework for contrastive learning of visual representations. In: International conference on machine learning. PMLR, pp 1597–1607
34.
Zurück zum Zitat Chang J, Blei D (2009) Relational topic models for document networks. In: Artificial intelligence and statistics. PMLR, pp 81–88 Chang J, Blei D (2009) Relational topic models for document networks. In: Artificial intelligence and statistics. PMLR, pp 81–88
35.
Zurück zum Zitat Pei Y, Chakraborty N, Sycara K (2015) Nonnegative matrix tri-factorization with graph regularization for community detection in social networks. In: Twenty-fourth international joint conference on artificial intelligence Pei Y, Chakraborty N, Sycara K (2015) Nonnegative matrix tri-factorization with graph regularization for community detection in social networks. In: Twenty-fourth international joint conference on artificial intelligence
36.
Zurück zum Zitat Wang X, Jin D, Cao X, Yang L, Zhang W (2016) Semantic community identification in large attribute networks. In: Proceedings of the AAAI conference on artificial intelligence 30 Wang X, Jin D, Cao X, Yang L, Zhang W (2016) Semantic community identification in large attribute networks. In: Proceedings of the AAAI conference on artificial intelligence 30
37.
Zurück zum Zitat Xie J, Girshick R, Farhadi A (2016) Unsupervised deep embedding for clustering analysis. In: International conference on machine learning. PMLR, pp 478–487 Xie J, Girshick R, Farhadi A (2016) Unsupervised deep embedding for clustering analysis. In: International conference on machine learning. PMLR, pp 478–487
39.
Zurück zum Zitat Sun F-Y, Qu M, Hoffmann J, Huang C-W, Tang J (2019) vgraph: A generative model for joint community detection and node representation learning. Adv Neural Inf Process Syst 32 Sun F-Y, Qu M, Hoffmann J, Huang C-W, Tang J (2019) vgraph: A generative model for joint community detection and node representation learning. Adv Neural Inf Process Syst 32
41.
Zurück zum Zitat Zhang H, Li P, Zhang R, Li X (2022) Embedding graph auto-encoder for graph clustering. IEEE Trans Neural Netw Learn Syst Zhang H, Li P, Zhang R, Li X (2022) Embedding graph auto-encoder for graph clustering. IEEE Trans Neural Netw Learn Syst
42.
Zurück zum Zitat Guo L, Dai Q (2022) Graph clustering via variational graph embedding. Pattern Recognit 122:108334CrossRef Guo L, Dai Q (2022) Graph clustering via variational graph embedding. Pattern Recognit 122:108334CrossRef
43.
Zurück zum Zitat Yang X, Liu Y, Zhou S, Wang S, Tu W, Zheng Q, Liu X, Fang L, Zhu E (2023) Cluster-guided contrastive graph clustering network. In: Proceedings of the AAAI conference on artificial intelligence, 37 Yang X, Liu Y, Zhou S, Wang S, Tu W, Zheng Q, Liu X, Fang L, Zhu E (2023) Cluster-guided contrastive graph clustering network. In: Proceedings of the AAAI conference on artificial intelligence, 37
44.
Zurück zum Zitat Yang X, Tan C, Liu Y, Liang K, Wang S, Zhou S, Xia J, Li SZ, Liu X, Zhu E (2023) Convert: contrastive graph clustering with reliable augmentation. In: Proceedings of the 31th ACM international conference on multimedia Yang X, Tan C, Liu Y, Liang K, Wang S, Zhou S, Xia J, Li SZ, Liu X, Zhu E (2023) Convert: contrastive graph clustering with reliable augmentation. In: Proceedings of the 31th ACM international conference on multimedia
46.
Zurück zum Zitat Li Q, Han Z, Wu X-M (2018) Deeper insights into graph convolutional networks for semi-supervised learning. In: Thirty-second AAAI conference on artificial intelligence Li Q, Han Z, Wu X-M (2018) Deeper insights into graph convolutional networks for semi-supervised learning. In: Thirty-second AAAI conference on artificial intelligence
47.
Zurück zum Zitat Page L, Brin S, Motwani R, Winograd T (1999) The pagerank citation ranking: bringing order to the web. Technical report, Stanford InfoLab Page L, Brin S, Motwani R, Winograd T (1999) The pagerank citation ranking: bringing order to the web. Technical report, Stanford InfoLab
48.
Zurück zum Zitat Dempster AP, Laird NM, Rubin DB (1977) Maximum likelihood from incomplete data via the EM algorithm. J R Stat Soc: Ser B (Methodol) 39(1):1–22MathSciNet Dempster AP, Laird NM, Rubin DB (1977) Maximum likelihood from incomplete data via the EM algorithm. J R Stat Soc: Ser B (Methodol) 39(1):1–22MathSciNet
49.
Zurück zum Zitat Andersen R, Chung F, Lang K (2006) Local graph partitioning using pagerank vectors. In: 2006 47th annual IEEE symposium on foundations of computer science (FOCS’06). IEEE, pp 475–486 Andersen R, Chung F, Lang K (2006) Local graph partitioning using pagerank vectors. In: 2006 47th annual IEEE symposium on foundations of computer science (FOCS’06). IEEE, pp 475–486
50.
Zurück zum Zitat Sen P, Namata G, Bilgic M, Getoor L, Galligher B, Eliassi-Rad T (2008) Collective classification in network data. AI Mag 29(3):93–93 Sen P, Namata G, Bilgic M, Getoor L, Galligher B, Eliassi-Rad T (2008) Collective classification in network data. AI Mag 29(3):93–93
51.
Zurück zum Zitat Yang C, Liu Z, Zhao D, Sun M, Chang E (2015) Network representation learning with rich text information. In: Twenty-fourth international joint conference on artificial intelligence Yang C, Liu Z, Zhao D, Sun M, Chang E (2015) Network representation learning with rich text information. In: Twenty-fourth international joint conference on artificial intelligence
52.
Zurück zum Zitat Wang X, Ji H, Shi C, Wang B, Ye Y, Cui P, Yu PS (2019) Heterogeneous graph attention network. In: The world wide web conference, pp 2022–2032 Wang X, Ji H, Shi C, Wang B, Ye Y, Cui P, Yu PS (2019) Heterogeneous graph attention network. In: The world wide web conference, pp 2022–2032
53.
Zurück zum Zitat Li J, Hu X, Tang J, Liu H (2015) Unsupervised streaming feature selection in social media. In: Proceedings of the 24th ACM international on conference on information and knowledge management, pp 1041–1050 Li J, Hu X, Tang J, Liu H (2015) Unsupervised streaming feature selection in social media. In: Proceedings of the 24th ACM international on conference on information and knowledge management, pp 1041–1050
55.
Zurück zum Zitat Ng A, Jordan M, Weiss Y (2002) On spectral clustering: analysis and an algorithm. Adv Neural Inf Process Syst 14 Ng A, Jordan M, Weiss Y (2002) On spectral clustering: analysis and an algorithm. Adv Neural Inf Process Syst 14
56.
Zurück zum Zitat Tian F, Gao B, Cui Q, Chen E, Liu T-Y (2014) Learning deep representations for graph clustering. In: Proceedings of the AAAI conference on artificial intelligence 28 Tian F, Gao B, Cui Q, Chen E, Liu T-Y (2014) Learning deep representations for graph clustering. In: Proceedings of the AAAI conference on artificial intelligence 28
57.
Zurück zum Zitat Cao S, Lu W, Xu Q (2016) Deep neural networks for learning graph representations. In: Proceedings of the AAAI conference on artificial intelligence 30 Cao S, Lu W, Xu Q (2016) Deep neural networks for learning graph representations. In: Proceedings of the AAAI conference on artificial intelligence 30
58.
Zurück zum Zitat Yang C, Liu Z, Zhao D, Sun M, Chang E (2015) Network representation learning with rich text information. In: Twenty-fourth international joint conference on artificial intelligence Yang C, Liu Z, Zhao D, Sun M, Chang E (2015) Network representation learning with rich text information. In: Twenty-fourth international joint conference on artificial intelligence
59.
Zurück zum Zitat Li J, Yu J, Li J, Zhang H, Zhao K, Rong Y, Cheng H, Huang J (2020) Dirichlet graph variational autoencoder. Adv Neural Inf Process Syst 33:5274–5283 Li J, Yu J, Li J, Zhang H, Zhao K, Rong Y, Cheng H, Huang J (2020) Dirichlet graph variational autoencoder. Adv Neural Inf Process Syst 33:5274–5283
60.
Zurück zum Zitat Zhang T, Xiong Y, Zhang J, Zhang Y, Jiao Y, Zhu Y (2020) CommDGI: community detection oriented deep graph infomax. In: Proceedings of the 29th ACM international conference on information and knowledge management, pp 1843–1852 Zhang T, Xiong Y, Zhang J, Zhang Y, Jiao Y, Zhu Y (2020) CommDGI: community detection oriented deep graph infomax. In: Proceedings of the 29th ACM international conference on information and knowledge management, pp 1843–1852
61.
Zurück zum Zitat Zhang X, Liu H, Wu X-M, Zhang X, Liu X (2021) Spectral embedding network for attributed graph clustering. Neural Netw 142:388–396CrossRef Zhang X, Liu H, Wu X-M, Zhang X, Liu X (2021) Spectral embedding network for attributed graph clustering. Neural Netw 142:388–396CrossRef
62.
Zurück zum Zitat Zheng S, Zhu Z, Zhang X, Liu Z, Cheng J, Zhao Y (2020) Distribution-induced bidirectional generative adversarial network for graph representation learning. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 7224–7233 Zheng S, Zhu Z, Zhang X, Liu Z, Cheng J, Zhao Y (2020) Distribution-induced bidirectional generative adversarial network for graph representation learning. In: Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, pp 7224–7233
63.
Zurück zum Zitat Reynolds DA (2009) Gaussian mixture models. Encycl Biom 741:659–663 Reynolds DA (2009) Gaussian mixture models. Encycl Biom 741:659–663
64.
Zurück zum Zitat Van Der Maaten L (2014) Accelerating t-SNE using tree-based algorithms. J Mach Learn Res 15(1):3221–3245MathSciNet Van Der Maaten L (2014) Accelerating t-SNE using tree-based algorithms. J Mach Learn Res 15(1):3221–3245MathSciNet
Metadaten
Titel
Deep graph clustering via mutual information maximization and mixture model
verfasst von
Maedeh Ahmadi
Mehran Safayani
Abdolreza Mirzaei
Publikationsdatum
10.04.2024
Verlag
Springer London
Erschienen in
Knowledge and Information Systems
Print ISSN: 0219-1377
Elektronische ISSN: 0219-3116
DOI
https://doi.org/10.1007/s10115-024-02097-4

Premium Partner