Skip to main content
Erschienen in: International Journal of Machine Learning and Cybernetics 4/2024

21.10.2023 | Original Article

An interpretable neural network for robustly determining the location and number of cluster centers

verfasst von: Xuetao Xie, Yi-Fei Pu, Huaqing Zhang, Jacek Mańdziuk, El-Sayed M. El-Alfy, Jian Wang

Erschienen in: International Journal of Machine Learning and Cybernetics | Ausgabe 4/2024

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

K-means is a clustering method with an interpretable mechanism. However, its clustering results are significantly affected by the location of the initial cluster centers. More importantly, for it and its improved versions, it is extremely hard to adaptively determine the number of cluster centers. In contrast, ordinary neural networks have powerful information representation ability but lack interpretability. Moreover, to the best of our knowledge, the use of interpretable neural networks to determine the number of cluster centers of K-means is absent. This paper proposes K-meaNet that combines the interpretable mechanism of K-means and the powerful information representation ability of neural networks. For the neural network in K-meaNet, its inputs, weights, and mathematical expressions of each layer have clear meanings. During training, if one cluster center is critical, the value of one of the weights in the neural network, the gate, corresponding to this cluster center will increase. At the same time, the position of this cluster center will be close to the ideal cluster center. Besides, the location of the cluster center(s) and the value(s) of the corresponding gate(s) will not change significantly. This endows K-meaNet with the ability to adaptively determine the location and number of cluster centers compared with K-means and its improved versions. Moreover, this adaptive ability is robust to the location of the initial cluster centers, the number of the initial cluster centers, and the number of features. On six synthetic datasets and three real datasets, numerical experiments verify that K-meaNet can adaptively determine the number of cluster centers and is robust to the location of the initial cluster centers, the number of the initial cluster centers, and the number of features.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Weitere Produktempfehlungen anzeigen
Anhänge
Nur mit Berechtigung zugänglich
Literatur
1.
Zurück zum Zitat Ahmed M, Seraj R, Islam SMS (2020) The k-means algorithm: A comprehensive survey and performance evaluation. Electronics 9:1295CrossRef Ahmed M, Seraj R, Islam SMS (2020) The k-means algorithm: A comprehensive survey and performance evaluation. Electronics 9:1295CrossRef
3.
Zurück zum Zitat Ester M, Kriegel H. P, Sander J, Xu X (1996)“A density-based algorithm for discovering clusters in large spatial databases with noise. In Proceedings of the 2nd KDD. AAAI Press Ester M, Kriegel H. P, Sander J, Xu X (1996)“A density-based algorithm for discovering clusters in large spatial databases with noise. In Proceedings of the 2nd KDD. AAAI Press
4.
Zurück zum Zitat Dong S, Xia Y, Peng T (2021) Network abnormal traffic detection model based on semi-supervised deep reinforcement learning. IEEE Trans Netw Serv Manag 18(4):4197–4212CrossRef Dong S, Xia Y, Peng T (2021) Network abnormal traffic detection model based on semi-supervised deep reinforcement learning. IEEE Trans Netw Serv Manag 18(4):4197–4212CrossRef
5.
Zurück zum Zitat Wang H, Cheng R, Zhou J, Tao L, Kwan HK (2022) Multistage model for robust face alignment using deep neural networks. Cogn Comput 14:1123–1139CrossRef Wang H, Cheng R, Zhou J, Tao L, Kwan HK (2022) Multistage model for robust face alignment using deep neural networks. Cogn Comput 14:1123–1139CrossRef
6.
Zurück zum Zitat Li F, Gao D, Yang Y, Zhu J (2023) Small target deep convolution recognition algorithm based on improved YOLOv4. Int J Mach Learn Cybern 14:387–394CrossRef Li F, Gao D, Yang Y, Zhu J (2023) Small target deep convolution recognition algorithm based on improved YOLOv4. Int J Mach Learn Cybern 14:387–394CrossRef
7.
Zurück zum Zitat Zhang Y, Mańdziuk J, Quek CH, Goh BW (2017) Curvature-based method for determining the number of clusters. Inf Sci 415–416:414–428CrossRef Zhang Y, Mańdziuk J, Quek CH, Goh BW (2017) Curvature-based method for determining the number of clusters. Inf Sci 415–416:414–428CrossRef
8.
Zurück zum Zitat Liu Q, Wu H, Xu Z (2021) Consensus model based on probability K-means clustering algorithm for large scale group decision making. Int J Mach Learn Cybern 12:1609–1626CrossRefADS Liu Q, Wu H, Xu Z (2021) Consensus model based on probability K-means clustering algorithm for large scale group decision making. Int J Mach Learn Cybern 12:1609–1626CrossRefADS
9.
Zurück zum Zitat Biswas TK, Giri K, Roy S (2023) ECKM: An improved K-means clustering based on computational geometry. Expert Syst Appl 212:118862CrossRef Biswas TK, Giri K, Roy S (2023) ECKM: An improved K-means clustering based on computational geometry. Expert Syst Appl 212:118862CrossRef
10.
Zurück zum Zitat Hu H, Liu J, Zhang X, Fang M (2023) An effective and adaptable K-means algorithm for big data cluster analysis. Pattern Recognit 139:109404CrossRef Hu H, Liu J, Zhang X, Fang M (2023) An effective and adaptable K-means algorithm for big data cluster analysis. Pattern Recognit 139:109404CrossRef
11.
Zurück zum Zitat Liu L, Li P, Chu M, Liu S (2023) Robust nonparallel support vector machine with privileged information for pattern recognition. Int J Mach Learn Cybern 14:1465–1482CrossRef Liu L, Li P, Chu M, Liu S (2023) Robust nonparallel support vector machine with privileged information for pattern recognition. Int J Mach Learn Cybern 14:1465–1482CrossRef
12.
Zurück zum Zitat Tanveer M, Gupta T, Shah M, Richhariya B (2021) Sparse twin support vector clustering using pinball loss. IEEE J Biomed Health Inf 25(10):3776–3783CrossRef Tanveer M, Gupta T, Shah M, Richhariya B (2021) Sparse twin support vector clustering using pinball loss. IEEE J Biomed Health Inf 25(10):3776–3783CrossRef
13.
Zurück zum Zitat Tanveer M, Gupta T, Shah M (2021) Pinball loss twin support vector clustering. ACM Trans Multimed Comput Commun Appl 17(2s):1–23CrossRef Tanveer M, Gupta T, Shah M (2021) Pinball loss twin support vector clustering. ACM Trans Multimed Comput Commun Appl 17(2s):1–23CrossRef
14.
Zurück zum Zitat Tanveer M, Tabish M, Jangir J (2022) Sparse pinball twin bounded support vector clustering. IEEE Trans Comput Soc Syst 9(6):1820–1829CrossRef Tanveer M, Tabish M, Jangir J (2022) Sparse pinball twin bounded support vector clustering. IEEE Trans Comput Soc Syst 9(6):1820–1829CrossRef
15.
Zurück zum Zitat Demuth HB, Beale MH, De Jésus O, Hagan MT (2014) Neural network design. Martin Hagan, Stillwater, Oklahoma, USA Demuth HB, Beale MH, De Jésus O, Hagan MT (2014) Neural network design. Martin Hagan, Stillwater, Oklahoma, USA
16.
Zurück zum Zitat Larochelle H, Bengio Y, Louradour J, Lamblin P (2009) Exploring strategies for training deep neural networks. J Mach Learn Res 10(1):1–40 Larochelle H, Bengio Y, Louradour J, Lamblin P (2009) Exploring strategies for training deep neural networks. J Mach Learn Res 10(1):1–40
17.
Zurück zum Zitat Xie X, Li Z, Pu YF, Wang J, Zhang W, Wen Y (2023) A fractional filter based on reinforcement learning for effective tracking under impulsive noise. Neurocomputing 516:155–168CrossRef Xie X, Li Z, Pu YF, Wang J, Zhang W, Wen Y (2023) A fractional filter based on reinforcement learning for effective tracking under impulsive noise. Neurocomputing 516:155–168CrossRef
20.
Zurück zum Zitat Caron M, Bojanowski P, Joulin A, Douze M (2018) Deep clustering for unsupervised learning of visual features. In: European conference on computer vision Caron M, Bojanowski P, Joulin A, Douze M (2018) Deep clustering for unsupervised learning of visual features. In: European conference on computer vision
21.
Zurück zum Zitat Dang Z, Deng C, Yang X, Wei K, Huang H (2021) Nearest neighbor matching for deep clustering. In: IEEE/CVF conference on computer vision and pattern recognition Dang Z, Deng C, Yang X, Wei K, Huang H (2021) Nearest neighbor matching for deep clustering. In: IEEE/CVF conference on computer vision and pattern recognition
22.
Zurück zum Zitat Xu J, Ren Y, Li G, Pan L, Zhu C, Xu Z (2021) Deep embedded multi-view clustering with collaborative training. Inf Sci 573:279–290MathSciNetCrossRef Xu J, Ren Y, Li G, Pan L, Zhu C, Xu Z (2021) Deep embedded multi-view clustering with collaborative training. Inf Sci 573:279–290MathSciNetCrossRef
23.
Zurück zum Zitat Özgül OF, Bardak B, Tan M (2021) A convolutional deep clustering framework for gene expression time series. IEEE ACM Trans Comput Biol Bioinform 18(6):2198–2207CrossRefPubMed Özgül OF, Bardak B, Tan M (2021) A convolutional deep clustering framework for gene expression time series. IEEE ACM Trans Comput Biol Bioinform 18(6):2198–2207CrossRefPubMed
24.
Zurück zum Zitat Cai J, Fan J, Guo W, Wang S, Zhang Y, Zhang Z (2022) Efficient deep embedded subspace clustering. In: IEEE/CVF conference on computer vision and pattern recognition Cai J, Fan J, Guo W, Wang S, Zhang Y, Zhang Z (2022) Efficient deep embedded subspace clustering. In: IEEE/CVF conference on computer vision and pattern recognition
25.
Zurück zum Zitat Cai J, Wang S, Xu C, Guo W (2022) Unsupervised deep clustering via contractive feature representation and focal loss. Pattern Recognit 123:108386CrossRef Cai J, Wang S, Xu C, Guo W (2022) Unsupervised deep clustering via contractive feature representation and focal loss. Pattern Recognit 123:108386CrossRef
26.
Zurück zum Zitat Li S, Yuan M, Chen J, Hu Z (2022) AdaDC: adaptive deep clustering for unsupervised domain adaptation in person re-identification. IEEE Trans Circuits Syst Video Technol 32(6):3825–3838CrossRef Li S, Yuan M, Chen J, Hu Z (2022) AdaDC: adaptive deep clustering for unsupervised domain adaptation in person re-identification. IEEE Trans Circuits Syst Video Technol 32(6):3825–3838CrossRef
27.
Zurück zum Zitat Wang J, Wu B, Ren Z, Zhang H, Zhou Y (2023) Multi-scale deep multi-view subspace clustering with self-weighting fusion and structure preserving. Expert Syst Appl 213:119031CrossRef Wang J, Wu B, Ren Z, Zhang H, Zhou Y (2023) Multi-scale deep multi-view subspace clustering with self-weighting fusion and structure preserving. Expert Syst Appl 213:119031CrossRef
28.
Zurück zum Zitat Wang Y, Chang D, Fu Z, Zhao Y (2023) Learning a bi-directional discriminative representation for deep clustering. Pattern Recogn 137:109237CrossRef Wang Y, Chang D, Fu Z, Zhao Y (2023) Learning a bi-directional discriminative representation for deep clustering. Pattern Recogn 137:109237CrossRef
29.
Zurück zum Zitat Wang T, Zhang X, Lan L, Luo Z (2023) Local-to-global deep clustering on approximate Uniform manifold. IEEE Trans Knowl Data Eng 35(5):5035–5046 Wang T, Zhang X, Lan L, Luo Z (2023) Local-to-global deep clustering on approximate Uniform manifold. IEEE Trans Knowl Data Eng 35(5):5035–5046
31.
32.
33.
Zurück zum Zitat Tang Z et al (2019) Interpretable classification of Alzheimer’s disease pathologies with a convolutional neural network pipeline. Nat Commun 10(1):1–14MathSciNetADS Tang Z et al (2019) Interpretable classification of Alzheimer’s disease pathologies with a convolutional neural network pipeline. Nat Commun 10(1):1–14MathSciNetADS
34.
Zurück zum Zitat Samek W, Montavon G, Lapuschkin S, Anders CJ, Müller KR (2021) Explaining deep neural networks and beyond: a review of methods and applications. Proc IEEE 109(3):247–278CrossRef Samek W, Montavon G, Lapuschkin S, Anders CJ, Müller KR (2021) Explaining deep neural networks and beyond: a review of methods and applications. Proc IEEE 109(3):247–278CrossRef
35.
Zurück zum Zitat Peng X, Li Y, Tsang IW, Zhu H, Lv J, Zhou JT (2022) XAI beyond classification: interpretable neural clustering. J Mach Learn Res 23(6):1–28MathSciNet Peng X, Li Y, Tsang IW, Zhu H, Lv J, Zhou JT (2022) XAI beyond classification: interpretable neural clustering. J Mach Learn Res 23(6):1–28MathSciNet
36.
Zurück zum Zitat Yu L, Zhang Z, Xie X, Chen H, Wang J (2019) Unsupervised feature selection using RBF autoencoder. Int Symp Neural Netw 11554:48–57 Yu L, Zhang Z, Xie X, Chen H, Wang J (2019) Unsupervised feature selection using RBF autoencoder. Int Symp Neural Netw 11554:48–57
37.
Zurück zum Zitat Ma L, Wang X, Zhou Y (2022) Observer and command-filter-based adaptive neural network control algorithms for nonlinear multi-agent systems with input delay. Cogn Comput 14:814–827CrossRef Ma L, Wang X, Zhou Y (2022) Observer and command-filter-based adaptive neural network control algorithms for nonlinear multi-agent systems with input delay. Cogn Comput 14:814–827CrossRef
38.
Zurück zum Zitat Wang K, Yan C, Yuan X, Wang Y, Liu C (2022) A reduced nonstationary discrete convolution kernel for multimode process monitoring. Int J Mach Learn Cybern 13:3711–3725CrossRef Wang K, Yan C, Yuan X, Wang Y, Liu C (2022) A reduced nonstationary discrete convolution kernel for multimode process monitoring. Int J Mach Learn Cybern 13:3711–3725CrossRef
39.
Zurück zum Zitat Gao T, Zhang Z, Chang Q, Xie X, Ren P, Wang J (2019) Conjugate gradient-based Takagi–Sugeno fuzzy neural network parameter identification and its convergence analysis. Neurocomputing 364:168–181CrossRef Gao T, Zhang Z, Chang Q, Xie X, Ren P, Wang J (2019) Conjugate gradient-based Takagi–Sugeno fuzzy neural network parameter identification and its convergence analysis. Neurocomputing 364:168–181CrossRef
40.
Zurück zum Zitat Wang J, Chang Q, Gao T, Zhang K, Pal NR (2022) Sensitivity analysis of Takagi–Sugeno fuzzy neural network. Inf Sci 582:725–749CrossRef Wang J, Chang Q, Gao T, Zhang K, Pal NR (2022) Sensitivity analysis of Takagi–Sugeno fuzzy neural network. Inf Sci 582:725–749CrossRef
43.
Zurück zum Zitat Xie X, Zhang H, Wang J, Chang Q, Wang J, Pal NR (2020) Learning optimized structure of neural networks by hidden node pruning with \(L_ {1}\) regularization. IEEE Trans Cybern 50(3):1333–1346CrossRefPubMed Xie X, Zhang H, Wang J, Chang Q, Wang J, Pal NR (2020) Learning optimized structure of neural networks by hidden node pruning with \(L_ {1}\) regularization. IEEE Trans Cybern 50(3):1333–1346CrossRefPubMed
44.
Zurück zum Zitat Dau HA et al (2019) The UCR time series archive. IEEE CAA J Autom Sin 6(6):1293–1305CrossRef Dau HA et al (2019) The UCR time series archive. IEEE CAA J Autom Sin 6(6):1293–1305CrossRef
46.
Zurück zum Zitat Park HS, Jun CH (2009) A simple and fast algorithm for K-medoids clustering. Expert Syst Appl 36:3336–3341CrossRef Park HS, Jun CH (2009) A simple and fast algorithm for K-medoids clustering. Expert Syst Appl 36:3336–3341CrossRef
Metadaten
Titel
An interpretable neural network for robustly determining the location and number of cluster centers
verfasst von
Xuetao Xie
Yi-Fei Pu
Huaqing Zhang
Jacek Mańdziuk
El-Sayed M. El-Alfy
Jian Wang
Publikationsdatum
21.10.2023
Verlag
Springer Berlin Heidelberg
Erschienen in
International Journal of Machine Learning and Cybernetics / Ausgabe 4/2024
Print ISSN: 1868-8071
Elektronische ISSN: 1868-808X
DOI
https://doi.org/10.1007/s13042-023-01978-4

Weitere Artikel der Ausgabe 4/2024

International Journal of Machine Learning and Cybernetics 4/2024 Zur Ausgabe

Neuer Inhalt