Skip to main content
Erschienen in: Neural Processing Letters 3/2020

20.05.2020

Using Locality Preserving Projections to Improve the Performance of Kernel Clustering

verfasst von: Mengmeng Zhan, Guangquan Lu, Guoqiu Wen, Leyuan Zhang, Lin Wu

Erschienen in: Neural Processing Letters | Ausgabe 3/2020

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Many clustering methods may have poor performance when the data structure is complex (i.e., the data has an aspheric shape or non-linear relationship). Inspired by this view, we proposed a clustering model which combines kernel function and Locality Preserving Projections (LPP) together. Specifically, we map original data into the high-dimensional feature space according to the idea of kernel function. Secondly, it is feasible to explore the local structure of data in clustering tasks. LPP is used to preserve the original local structure information of data to improve the validity of the clustering model. Finally, some outliers are often included in real data, so we embedded sparse regularization items in the model to adjust feature weights and remove outliers. In addition, we design a simple iterative optimization method to solve the final objective function and show the convergence of the optimization method in the experimental part. The experimental analysis of ten public data sets showed that our proposed method has better efficiency and performance in clustering tasks than existing clustering methods.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
2.
Zurück zum Zitat Baudat G, Anouar F (2000) Generalized discriminant analysis using a kernel approach. Neural Comput 12(10):2385–2404CrossRef Baudat G, Anouar F (2000) Generalized discriminant analysis using a kernel approach. Neural Comput 12(10):2385–2404CrossRef
3.
Zurück zum Zitat Ben-Hur A, Horn D, Siegelmann HT, Vapnik V (2001) Support vector clustering. J Mach Learn Res 2(2):125–137MATH Ben-Hur A, Horn D, Siegelmann HT, Vapnik V (2001) Support vector clustering. J Mach Learn Res 2(2):125–137MATH
4.
Zurück zum Zitat Bourgain J, Dirksen S, Nelson J (2015) Toward a unified theory of sparse dimensionality reduction in euclidean space. Geom Funct Anal 25(4):1009–1088MathSciNetMATHCrossRef Bourgain J, Dirksen S, Nelson J (2015) Toward a unified theory of sparse dimensionality reduction in euclidean space. Geom Funct Anal 25(4):1009–1088MathSciNetMATHCrossRef
5.
Zurück zum Zitat Camastra F, Verri A (2005) A novel kernel method for clustering. IEEE Trans Pattern Anal Mach Intell 27(5):801–805CrossRef Camastra F, Verri A (2005) A novel kernel method for clustering. IEEE Trans Pattern Anal Mach Intell 27(5):801–805CrossRef
6.
Zurück zum Zitat Chen S, Zhao H, Kong M, Luo B (2007) 2d-lpp: a two-dimensional extension of locality preserving projections. Neurocomputing 70(4):912–921CrossRef Chen S, Zhao H, Kong M, Luo B (2007) 2d-lpp: a two-dimensional extension of locality preserving projections. Neurocomputing 70(4):912–921CrossRef
7.
Zurück zum Zitat Chiang JH, Hao PY (2003) A new kernel-based fuzzy clustering approach: support vector clustering with cell growing. IEEE Trans Fuzzy Syst 11(4):518–527CrossRef Chiang JH, Hao PY (2003) A new kernel-based fuzzy clustering approach: support vector clustering with cell growing. IEEE Trans Fuzzy Syst 11(4):518–527CrossRef
8.
Zurück zum Zitat Dhanachandra N, Manglem K, Chanu YJ (2015) Image segmentation using k -means clustering algorithm and subtractive clustering algorithm. Proc Comput Sci 54:764–771CrossRef Dhanachandra N, Manglem K, Chanu YJ (2015) Image segmentation using k -means clustering algorithm and subtractive clustering algorithm. Proc Comput Sci 54:764–771CrossRef
10.
Zurück zum Zitat Graves D, Pedrycz W (2010) Kernel-based fuzzy clustering and fuzzy clustering: a comparative experimental study. Fuzzy Sets Syst 161(4):522–543MathSciNetCrossRef Graves D, Pedrycz W (2010) Kernel-based fuzzy clustering and fuzzy clustering: a comparative experimental study. Fuzzy Sets Syst 161(4):522–543MathSciNetCrossRef
11.
Zurück zum Zitat Gui J, Sun Z, Ji S, Tao D, Tan T (2017) Feature selection based on structured sparsity: a comprehensive study. IEEE Trans Neural Netw Learn Syst 28(7):1490–1507MathSciNetCrossRef Gui J, Sun Z, Ji S, Tao D, Tan T (2017) Feature selection based on structured sparsity: a comprehensive study. IEEE Trans Neural Netw Learn Syst 28(7):1490–1507MathSciNetCrossRef
12.
Zurück zum Zitat Halkidi M, Batistakis Y, Vazirgiannis M (2001) On clustering validation techniques. J Intell Inf Syst 17(2–3):107–145MATHCrossRef Halkidi M, Batistakis Y, Vazirgiannis M (2001) On clustering validation techniques. J Intell Inf Syst 17(2–3):107–145MATHCrossRef
13.
Zurück zum Zitat Hartigan JA, Wong MA (1979) Algorithm as 136: a k-means clustering algorithm. J Roy Stat Soc 28(1):100–108MATH Hartigan JA, Wong MA (1979) Algorithm as 136: a k-means clustering algorithm. J Roy Stat Soc 28(1):100–108MATH
16.
Zurück zum Zitat Inokuchi R, Miyamoto S (2004) LVQ clustering and SOM using a kernel function. In: 2004 IEEE international conference on fuzzy systems (IEEE cat. no. 04CH37542), vol 3. IEEE, pp 1497–1500 Inokuchi R, Miyamoto S (2004) LVQ clustering and SOM using a kernel function. In: 2004 IEEE international conference on fuzzy systems (IEEE cat. no. 04CH37542), vol 3. IEEE, pp 1497–1500
17.
Zurück zum Zitat Jain AK, Dubes RC et al (1988) Algorithms for clustering data, vol 6. Prentice Hall, Englewood CliffsMATH Jain AK, Dubes RC et al (1988) Algorithms for clustering data, vol 6. Prentice Hall, Englewood CliffsMATH
19.
Zurück zum Zitat Kalousis A, Prados J, Hilario M (2007) Stability of feature selection algorithms: a study on high-dimensional spaces. Knowl Inf Syst 12(1):95–116CrossRef Kalousis A, Prados J, Hilario M (2007) Stability of feature selection algorithms: a study on high-dimensional spaces. Knowl Inf Syst 12(1):95–116CrossRef
20.
Zurück zum Zitat Kannan SR, Sathya A, Sathya A, Sathya A (2011) Robust kernel FCM in segmentation of breast medical images. Expert Syst Appl Int J 38(4):4382–4389CrossRef Kannan SR, Sathya A, Sathya A, Sathya A (2011) Robust kernel FCM in segmentation of breast medical images. Expert Syst Appl Int J 38(4):4382–4389CrossRef
21.
Zurück zum Zitat Lei Z, Zhu S (2007) Face recognition based on orthogonal discriminant locality preserving projections. Neurocomputing 70(7):1543–1546 Lei Z, Zhu S (2007) Face recognition based on orthogonal discriminant locality preserving projections. Neurocomputing 70(7):1543–1546
22.
Zurück zum Zitat Li Z, Zhang Z, Qin J, Zhang Z, Shao L (2019) Discriminative fisher embedding dictionary learning algorithm for object recognition. IEEE Trans Neural Netw Learn Syst 99(1):1–14 Li Z, Zhang Z, Qin J, Zhang Z, Shao L (2019) Discriminative fisher embedding dictionary learning algorithm for object recognition. IEEE Trans Neural Netw Learn Syst 99(1):1–14
23.
Zurück zum Zitat Liu Y, Jiao L, Shang F (2013) An efficient matrix factorization based low-rank representation for subspace clustering. Pattern Recogn 46(1):284–292MATHCrossRef Liu Y, Jiao L, Shang F (2013) An efficient matrix factorization based low-rank representation for subspace clustering. Pattern Recogn 46(1):284–292MATHCrossRef
25.
Zurück zum Zitat Müller KR, Mika S, Rätsch G, Tsuda K, Schölkopf B (2001) An introduction to kernel-based learning algorithms. IEEE Trans Neural Netw 12(2):181CrossRef Müller KR, Mika S, Rätsch G, Tsuda K, Schölkopf B (2001) An introduction to kernel-based learning algorithms. IEEE Trans Neural Netw 12(2):181CrossRef
26.
27.
Zurück zum Zitat Ng AY, Jordan MI, Weiss Y (2001) On spectral clustering: analysis and an algorithm. Proc Nips 14:849–856 Ng AY, Jordan MI, Weiss Y (2001) On spectral clustering: analysis and an algorithm. Proc Nips 14:849–856
28.
Zurück zum Zitat Nie F, Rui Z, Xuelong LI (2017) A generalized power iteration method for solving quadratic problem on the stiefel manifold. Sci China (Information Sciences) 60(11):112101MathSciNetCrossRef Nie F, Rui Z, Xuelong LI (2017) A generalized power iteration method for solving quadratic problem on the stiefel manifold. Sci China (Information Sciences) 60(11):112101MathSciNetCrossRef
29.
Zurück zum Zitat Nie F, Xu D, Tsang IW, Zhang C (2009) Spectral embedded clustering. In: International joint conference on artificial intelligence Nie F, Xu D, Tsang IW, Zhang C (2009) Spectral embedded clustering. In: International joint conference on artificial intelligence
30.
Zurück zum Zitat Noble WS (2006) What is a support vector machine? Nat Biotechnol 24(12):1565CrossRef Noble WS (2006) What is a support vector machine? Nat Biotechnol 24(12):1565CrossRef
31.
Zurück zum Zitat Patel VM, Van Nguyen H, Vidal R (2013) Latent space sparse subspace clustering. In: Proceedings of the IEEE international conference on computer vision, pp 225–232 Patel VM, Van Nguyen H, Vidal R (2013) Latent space sparse subspace clustering. In: Proceedings of the IEEE international conference on computer vision, pp 225–232
32.
Zurück zum Zitat Ren X, Elkishky A, Wang C, Tao F, Voss CR, Ji H, Han J (2015) Clustype: effective entity recognition and typing by relation phrase-based clustering. In: Kdd 2015, pp 995–1004 Ren X, Elkishky A, Wang C, Tao F, Voss CR, Ji H, Han J (2015) Clustype: effective entity recognition and typing by relation phrase-based clustering. In: Kdd 2015, pp 995–1004
33.
Zurück zum Zitat Roweis ST, Saul LK (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500):2323–2326CrossRef Roweis ST, Saul LK (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500):2323–2326CrossRef
34.
Zurück zum Zitat Sánchez A VD (2003) Advanced support vector machines and kernel methods. Neurocomputing 55(1–2):5–20CrossRef Sánchez A VD (2003) Advanced support vector machines and kernel methods. Neurocomputing 55(1–2):5–20CrossRef
35.
Zurück zum Zitat Santos JM, Embrechts M (2009) On the use of the adjusted rand index as a metric for evaluating supervised classification. In: International conference on artificial neural networks. Springer, pp 175–184 Santos JM, Embrechts M (2009) On the use of the adjusted rand index as a metric for evaluating supervised classification. In: International conference on artificial neural networks. Springer, pp 175–184
36.
37.
Zurück zum Zitat Schölkopf B, Smola A, Müller KR (1998) Nonlinear component analysis as a kernel eigenvalue problem. Neural Comput 10(5):1299–1319CrossRef Schölkopf B, Smola A, Müller KR (1998) Nonlinear component analysis as a kernel eigenvalue problem. Neural Comput 10(5):1299–1319CrossRef
38.
Zurück zum Zitat Wang B, Yang Y, Xu X, Hanjalic A, Shen HT (2017) Adversarial cross-modal retrieval. In: Proceedings of the 2017 ACM on multimedia conference, pp 154–162 Wang B, Yang Y, Xu X, Hanjalic A, Shen HT (2017) Adversarial cross-modal retrieval. In: Proceedings of the 2017 ACM on multimedia conference, pp 154–162
39.
Zurück zum Zitat Wang S, Nie F, Chang X, Li X, Sheng QZ, Yao L (2016) Uncovering locally discriminative structure for feature analysis. In: Joint European conference on machine learning and knowledge discovery in databases. Springer, pp 281–295 Wang S, Nie F, Chang X, Li X, Sheng QZ, Yao L (2016) Uncovering locally discriminative structure for feature analysis. In: Joint European conference on machine learning and knowledge discovery in databases. Springer, pp 281–295
40.
Zurück zum Zitat Wang W, Liu B, Guan H, Zhou Y, Xia S (2015) Spectral embedded clustering algorithm based on kernel function. J Comput Appl 35(3):761–765,810 Wang W, Liu B, Guan H, Zhou Y, Xia S (2015) Spectral embedded clustering algorithm based on kernel function. J Comput Appl 35(3):761–765,810
42.
Zurück zum Zitat Xie GS, Zhang Z, Liu L, Zhu F, Zhang XY, Shao L, Li X (2019) SRSC: selective, robust, and supervised constrained feature representation for image classification. IEEE Trans Neural Netw Learn Syst 99(1):1–14 Xie GS, Zhang Z, Liu L, Zhu F, Zhang XY, Shao L, Li X (2019) SRSC: selective, robust, and supervised constrained feature representation for image classification. IEEE Trans Neural Netw Learn Syst 99(1):1–14
43.
Zurück zum Zitat Zhan M, Lu G, Wen G, Zhang L, Wu L (2019) A clustering algorithm via kernel function and locality preserving projections. In: IEEE symposium series on computational intelligence Zhan M, Lu G, Wen G, Zhang L, Wu L (2019) A clustering algorithm via kernel function and locality preserving projections. In: IEEE symposium series on computational intelligence
44.
Zurück zum Zitat Zhang DQ, Chen SC (2003) Clustering incomplete data using kernel-based fuzzy c-means algorithm. Neural Process Lett 18(3):155–162CrossRef Zhang DQ, Chen SC (2003) Clustering incomplete data using kernel-based fuzzy c-means algorithm. Neural Process Lett 18(3):155–162CrossRef
45.
Zurück zum Zitat Zhang S, Wong HS (2010) ARIMP: a generalized adjusted rand index for cluster ensembles. In: 2010 20th international conference on pattern recognition. IEEE, pp 778–781 Zhang S, Wong HS (2010) ARIMP: a generalized adjusted rand index for cluster ensembles. In: 2010 20th international conference on pattern recognition. IEEE, pp 778–781
46.
Zurück zum Zitat Zhang Z, Lai Z, Huang Z, Wong W, Xie G, Liu L, Shao L (2019) Scalable supervised asymmetric hashing with semantic and latent factor embedding. IEEE Trans Image Process 28(10):4803–4818MathSciNetMATHCrossRef Zhang Z, Lai Z, Huang Z, Wong W, Xie G, Liu L, Shao L (2019) Scalable supervised asymmetric hashing with semantic and latent factor embedding. IEEE Trans Image Process 28(10):4803–4818MathSciNetMATHCrossRef
47.
Zurück zum Zitat Zhang Z, Liu L, Shen F, Shen HT, Shao L (2019) Binary multi-view clustering. IEEE Trans Pattern Anal Mach Intell 41(7):1774–1782CrossRef Zhang Z, Liu L, Shen F, Shen HT, Shao L (2019) Binary multi-view clustering. IEEE Trans Pattern Anal Mach Intell 41(7):1774–1782CrossRef
48.
Zurück zum Zitat Zheng Q, Liu Z (2016) Research on improved normalized cut spectral clustering algorithm. In: 2016 Chinese control and decision conference (CCDC). IEEE, pp 1981–1984 Zheng Q, Liu Z (2016) Research on improved normalized cut spectral clustering algorithm. In: 2016 Chinese control and decision conference (CCDC). IEEE, pp 1981–1984
49.
Zurück zum Zitat Zheng W, Zhu X, Zhu Y, Hu R, Lei C (2018) Dynamic graph learning for spectral feature selection. Multimed. Tools Appl. 77(22):29739–29755CrossRef Zheng W, Zhu X, Zhu Y, Hu R, Lei C (2018) Dynamic graph learning for spectral feature selection. Multimed. Tools Appl. 77(22):29739–29755CrossRef
51.
Zurück zum Zitat Zhu X, Li X, Zhang S, Xu Z, Yu L, Wang C (2017) Graph PCA hashing for similarity search. IEEE Trans Multimed 19(9):2033–2044CrossRef Zhu X, Li X, Zhang S, Xu Z, Yu L, Wang C (2017) Graph PCA hashing for similarity search. IEEE Trans Multimed 19(9):2033–2044CrossRef
53.
Zurück zum Zitat Zhu X, Zhang S, Hu R, He W, Lei C, Zhu P (2019) One-step multi-view spectral clustering. IEEE Trans Knowl Data Eng 31(10):2022–2034CrossRef Zhu X, Zhang S, Hu R, He W, Lei C, Zhu P (2019) One-step multi-view spectral clustering. IEEE Trans Knowl Data Eng 31(10):2022–2034CrossRef
54.
Zurück zum Zitat Zhu X, Zhang S, Li Y, Zhang J, Yang L, Fang Y (2019) Low-rank sparse subspace for spectral clustering. IEEE Trans Knowl Data Eng 31(8):1532–1543CrossRef Zhu X, Zhang S, Li Y, Zhang J, Yang L, Fang Y (2019) Low-rank sparse subspace for spectral clustering. IEEE Trans Knowl Data Eng 31(8):1532–1543CrossRef
Metadaten
Titel
Using Locality Preserving Projections to Improve the Performance of Kernel Clustering
verfasst von
Mengmeng Zhan
Guangquan Lu
Guoqiu Wen
Leyuan Zhang
Lin Wu
Publikationsdatum
20.05.2020
Verlag
Springer US
Erschienen in
Neural Processing Letters / Ausgabe 3/2020
Print ISSN: 1370-4621
Elektronische ISSN: 1573-773X
DOI
https://doi.org/10.1007/s11063-020-10252-5

Weitere Artikel der Ausgabe 3/2020

Neural Processing Letters 3/2020 Zur Ausgabe

Neuer Inhalt