Skip to main content
Erschienen in:
Buchtitelbild

2016 | OriginalPaper | Buchkapitel

1. Discovering Cluster Dynamics Using Kernel Spectral Methods

verfasst von : Rocco Langone, Raghvendra Mall, Joos Vandewalle, Johan A. K. Suykens

Erschienen in: Complex Systems and Networks

Verlag: Springer Berlin Heidelberg

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Networks represent patterns of interactions between components of complex systems present in nature, science, technology and society. Furthermore, graph theory allows to perform insightful analysis for different kinds of data by representing the instances as nodes of a weighted network, where the weights characterize similarity between the data points. In this chapter we describe a number of algorithms to perform cluster analysis, that is finding groups of similar items (called clusters or communities) and understand their evolution over time. These algorithms are designed in a kernel-based framework: the original data are mapped into an high dimensional feature space; linear models are designed in this space; complex nonlinear relationships between the data in the original input space can then be detected. Applications like fault detection in industrial machines, community detection of static and evolving networks, image segmentation, incremental time-series clustering and text clustering are considered.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Fußnoten
1
If \(V = I\), problem (1.3) is equivalent to a kernel PCA formulation [2224].
 
3
A matlab implementation of the KSC algorithm is available at: http://​www.​esat.​kuleuven.​be/​stadius/​ADB/​alzate/​softwareKSClab.​php.
 
5
The software to generate the word clouds visualization is available at: http://​www.​wordle.​net/​.
 
6
A matlab implementation of the IKSC algorithm is available at: http://​www.​esat.​kuleuven.​be/​stadius/​ADB/​langone/​softwareIKSClab.​php.
 
Literatur
1.
Zurück zum Zitat Chung, F.R.K.: Spectral Graph Theory. American Mathematical Society, Providence (1997)MATH Chung, F.R.K.: Spectral Graph Theory. American Mathematical Society, Providence (1997)MATH
2.
Zurück zum Zitat Shi, J., Malik, J.: Normalized cuts and image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 22(8), 888–905 (2000)CrossRef Shi, J., Malik, J.: Normalized cuts and image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 22(8), 888–905 (2000)CrossRef
3.
Zurück zum Zitat Ng, A.Y., Jordan, M.I., Weiss, Y.: On spectral clustering: analysis and an algorithm. Advances in Neural Information Processing Systems, vol. 14, pp. 849–856. MIT Press, Cambridge (2002) Ng, A.Y., Jordan, M.I., Weiss, Y.: On spectral clustering: analysis and an algorithm. Advances in Neural Information Processing Systems, vol. 14, pp. 849–856. MIT Press, Cambridge (2002)
5.
Zurück zum Zitat Fowlkes, C., Belongie, S., Chung, F., Malik, J.: Spectral grouping using the Nyström method. IEEE Trans. Pattern Anal. Mach. Intell. 26(2), 214–225 (2004)CrossRefMATH Fowlkes, C., Belongie, S., Chung, F., Malik, J.: Spectral grouping using the Nyström method. IEEE Trans. Pattern Anal. Mach. Intell. 26(2), 214–225 (2004)CrossRefMATH
6.
Zurück zum Zitat Williams, C.K.I., Seeger, M.: Using the Nyström method to speed up kernel machines. Advances in Neural Information Processing Systems, vol. 13. MIT Press, Cambridge (2001) Williams, C.K.I., Seeger, M.: Using the Nyström method to speed up kernel machines. Advances in Neural Information Processing Systems, vol. 13. MIT Press, Cambridge (2001)
7.
Zurück zum Zitat Lin, F., Cohen, William, W.: Power iteration clustering. In: ICML, pp. 655–662 (2010) Lin, F., Cohen, William, W.: Power iteration clustering. In: ICML, pp. 655–662 (2010)
8.
Zurück zum Zitat Ning, H., Xu, W., Chi, Y., Gong, Y., Huang, T.S.: Incremental spectral clustering by efficiently updating the eigen-system. Pattern Recognit. 43(1), 113–127 (2010)CrossRefMATH Ning, H., Xu, W., Chi, Y., Gong, Y., Huang, T.S.: Incremental spectral clustering by efficiently updating the eigen-system. Pattern Recognit. 43(1), 113–127 (2010)CrossRefMATH
9.
10.
Zurück zum Zitat Frederix, K., Van Marc, B.: Sparse spectral clustering method based on the incomplete Cholesky decomposition. J. Comput. Appl. Math. 237(1), 145–161 (2013)MathSciNetCrossRef Frederix, K., Van Marc, B.: Sparse spectral clustering method based on the incomplete Cholesky decomposition. J. Comput. Appl. Math. 237(1), 145–161 (2013)MathSciNetCrossRef
11.
Zurück zum Zitat Alzate, C., Suykens, J.A.K.: Multiway spectral clustering with out-of-sample extensions through weighted kernel PCA. IEEE Trans. Pattern Anal. Mach. Intell. 32(2), 335–347 (2010). (ESAT-SISTA, K.U. Leuven)CrossRef Alzate, C., Suykens, J.A.K.: Multiway spectral clustering with out-of-sample extensions through weighted kernel PCA. IEEE Trans. Pattern Anal. Mach. Intell. 32(2), 335–347 (2010). (ESAT-SISTA, K.U. Leuven)CrossRef
12.
Zurück zum Zitat Suykens, J.A.K., Van Gestel, T., De Brabanter, J., De Moor, B., Vandewalle, J.: Least Squares Support Vector Machines. World Scientific, Singapore (2002)CrossRefMATH Suykens, J.A.K., Van Gestel, T., De Brabanter, J., De Moor, B., Vandewalle, J.: Least Squares Support Vector Machines. World Scientific, Singapore (2002)CrossRefMATH
13.
Zurück zum Zitat Chakrabarti, D., Kumar, R., Tomkins, A.: Evolutionary clustering. In: Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery And Data Mining, pp. 554–560. ACM, New York (2006) Chakrabarti, D., Kumar, R., Tomkins, A.: Evolutionary clustering. In: Proceedings of the 12th ACM SIGKDD International Conference on Knowledge Discovery And Data Mining, pp. 554–560. ACM, New York (2006)
14.
Zurück zum Zitat Chi, Y., Song, X., Zhou, D., Hino, K., Tseng, B.L.: Evolutionary spectral clustering by incorporating temporal smoothness. In: KDD’07, pp. 153–162 (2007) Chi, Y., Song, X., Zhou, D., Hino, K., Tseng, B.L.: Evolutionary spectral clustering by incorporating temporal smoothness. In: KDD’07, pp. 153–162 (2007)
15.
Zurück zum Zitat Xu, K.S., Kliger, M., Hero, III., Alfred, O.: Adaptive evolutionary clustering. Data Min. Knowl. Discov. 1–33 (2013) Xu, K.S., Kliger, M., Hero, III., Alfred, O.: Adaptive evolutionary clustering. Data Min. Knowl. Discov. 1–33 (2013)
16.
Zurück zum Zitat Mucha, P.J., Richardson, T., Macon, K., Porter, M.A., Onnela, J.P.: Community structure in time-dependent, multiscale, and multiplex networks. Science 328(5980), 876–878 (2010)MathSciNetCrossRef Mucha, P.J., Richardson, T., Macon, K., Porter, M.A., Onnela, J.P.: Community structure in time-dependent, multiscale, and multiplex networks. Science 328(5980), 876–878 (2010)MathSciNetCrossRef
17.
Zurück zum Zitat Chakraborty, S., Nagwani, N.K.: Analysis and study of incremental K-means clustering algorithm. High Perform. Archit. Grid Comput. 169, 338–341 (2011)CrossRef Chakraborty, S., Nagwani, N.K.: Analysis and study of incremental K-means clustering algorithm. High Perform. Archit. Grid Comput. 169, 338–341 (2011)CrossRef
18.
Zurück zum Zitat Rocco, L., Carlos, A., Suykens, J.A.K.: Kernel spectral clustering with memory effect. Phys. A: Stat. Mech. Appl. 392(10), 2588–2606 (2013)CrossRefMATH Rocco, L., Carlos, A., Suykens, J.A.K.: Kernel spectral clustering with memory effect. Phys. A: Stat. Mech. Appl. 392(10), 2588–2606 (2013)CrossRefMATH
19.
Zurück zum Zitat Rocco, L., Suykens, J.A.K.: Community detection using kernel spectral clustering with memory. J. Phys. Conf. Ser. 410(1), 012100 (2013) Rocco, L., Suykens, J.A.K.: Community detection using kernel spectral clustering with memory. J. Phys. Conf. Ser. 410(1), 012100 (2013)
20.
Zurück zum Zitat Langone, R., Mall, R., Suykens, J.A.K.: Clustering data over time using kernel spectral clustering with memory. In: SSCI (CIDM) (2014) Langone, R., Mall, R., Suykens, J.A.K.: Clustering data over time using kernel spectral clustering with memory. In: SSCI (CIDM) (2014)
21.
Zurück zum Zitat Langone, R., Agudelo, O.M., De Moor, B., Suykens, J.A.K.: Incremental kernel spectral clustering for online learning of non-stationary data. Neurocomputing 139, 246–260 (2014)CrossRef Langone, R., Agudelo, O.M., De Moor, B., Suykens, J.A.K.: Incremental kernel spectral clustering for online learning of non-stationary data. Neurocomputing 139, 246–260 (2014)CrossRef
22.
Zurück zum Zitat Suykens, J.A.K., Van Gestel, T., Vandewalle, J., De Moor, B.: A support vector machine formulation to PCA analysis and its kernel version. IEEE Trans. Neural Netw. 14(2), 447–450 (2003)CrossRef Suykens, J.A.K., Van Gestel, T., Vandewalle, J., De Moor, B.: A support vector machine formulation to PCA analysis and its kernel version. IEEE Trans. Neural Netw. 14(2), 447–450 (2003)CrossRef
23.
Zurück zum Zitat Schölkopf, B., Smola, A.J., Müller, K.R.: Nonlinear component analysis as a kernel eigenvalue problem. Neural Comput. 10, 1299–1319 (1998)CrossRef Schölkopf, B., Smola, A.J., Müller, K.R.: Nonlinear component analysis as a kernel eigenvalue problem. Neural Comput. 10, 1299–1319 (1998)CrossRef
24.
Zurück zum Zitat Mika, S., Schölkopf, B., Smola, A.J., Müller, K.R., Scholz, M., Rätsch, G.: Kernel PCA and de-noising in feature spaces. Advances in Neural Information Processing Systems, vol. 11. MIT Press, Cambridge (1999) Mika, S., Schölkopf, B., Smola, A.J., Müller, K.R., Scholz, M., Rätsch, G.: Kernel PCA and de-noising in feature spaces. Advances in Neural Information Processing Systems, vol. 11. MIT Press, Cambridge (1999)
25.
Zurück zum Zitat Puzicha, J., Hofmann, T., Buhmann, J.: Non-parametric similarity measures for unsupervised texture segmentation and image retrieval. Comput. Vis. Pattern Recognit. 267–272 (1997) Puzicha, J., Hofmann, T., Buhmann, J.: Non-parametric similarity measures for unsupervised texture segmentation and image retrieval. Comput. Vis. Pattern Recognit. 267–272 (1997)
26.
Zurück zum Zitat Warren, L.T.: Clustering of time series data—a survey. Pattern Recognit. 38(11), 1857–1874 (2005)CrossRefMATH Warren, L.T.: Clustering of time series data—a survey. Pattern Recognit. 38(11), 1857–1874 (2005)CrossRefMATH
27.
Zurück zum Zitat Meila, M., Shi, J.: A random walks view of spectral segmentation. In: Artificial Intelligence and Statistics AISTATS (2001) Meila, M., Shi, J.: A random walks view of spectral segmentation. In: Artificial Intelligence and Statistics AISTATS (2001)
28.
Zurück zum Zitat Meila, M., Shi, J.: Learning segmentation by random walks. Advances in Neural Information Processing Systems, vol. 13. MIT Press, Cambridge (2001) Meila, M., Shi, J.: Learning segmentation by random walks. Advances in Neural Information Processing Systems, vol. 13. MIT Press, Cambridge (2001)
29.
Zurück zum Zitat Delvenne, J.C., Yaliraki, S.N., Barahona, M.: Stability of graph communities across time scales. Proc. Natl. Acad. Sci. 107(29), 12755–12760 (2010)CrossRef Delvenne, J.C., Yaliraki, S.N., Barahona, M.: Stability of graph communities across time scales. Proc. Natl. Acad. Sci. 107(29), 12755–12760 (2010)CrossRef
30.
Zurück zum Zitat Langone, R., Mall, R., Suykens, J.A.K.: Soft Kernel spectral clustering. In: Proceedings of the International Joint Conference on Neural Networks (IJCNN 2013), pp. 1028–1035 (2013) Langone, R., Mall, R., Suykens, J.A.K.: Soft Kernel spectral clustering. In: Proceedings of the International Joint Conference on Neural Networks (IJCNN 2013), pp. 1028–1035 (2013)
31.
Zurück zum Zitat Mall, R., Langone, R., Suykens, J.A.K.: Kernel spectral clustering for big data networks. Entropy (Special Issue on Big Data) 15(5), 1567–1586 (2013)MathSciNet Mall, R., Langone, R., Suykens, J.A.K.: Kernel spectral clustering for big data networks. Entropy (Special Issue on Big Data) 15(5), 1567–1586 (2013)MathSciNet
32.
Zurück zum Zitat Langone, R., Alzate, C., Suykens, J.A.K.: Modularity-based model selection for kernel spectral clustering. In: Proceedings of the International Joint Conference on Neural Networks (IJCNN 2011), pp. 1849–1856 (2011) Langone, R., Alzate, C., Suykens, J.A.K.: Modularity-based model selection for kernel spectral clustering. In: Proceedings of the International Joint Conference on Neural Networks (IJCNN 2011), pp. 1849–1856 (2011)
33.
Zurück zum Zitat Langone, R., Alzate, C., Suykens, J.A.K.: Kernel spectral clustering for community detection in complex networks. In: Proceedings of the International Joint Conference on Neural Networks (IJCNN 2012), pp. 2596–2603 (2012) Langone, R., Alzate, C., Suykens, J.A.K.: Kernel spectral clustering for community detection in complex networks. In: Proceedings of the International Joint Conference on Neural Networks (IJCNN 2012), pp. 2596–2603 (2012)
34.
Zurück zum Zitat Newman, M.E.J.: Modularity and community structure in networks. Proc. Natl. Acad. Sci. USA 103(23), 8577–8582 (2006)CrossRef Newman, M.E.J.: Modularity and community structure in networks. Proc. Natl. Acad. Sci. USA 103(23), 8577–8582 (2006)CrossRef
35.
Zurück zum Zitat Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Proceedings of the 8th International Conference on Computer Vision, vol. 2, pp. 416–423 (2001) Martin, D., Fowlkes, C., Tal, D., Malik, J.: A database of human segmented natural images and its application to evaluating segmentation algorithms and measuring ecological statistics. In: Proceedings of the 8th International Conference on Computer Vision, vol. 2, pp. 416–423 (2001)
36.
Zurück zum Zitat Alzate, C., Suykens, A.K.J.: Hierarchical kernel spectral clustering. Neural Netw. 35, 21–30 (2012)CrossRefMATH Alzate, C., Suykens, A.K.J.: Hierarchical kernel spectral clustering. Neural Netw. 35, 21–30 (2012)CrossRefMATH
37.
Zurück zum Zitat Alzate, C., Sinn, M.: Improved electricity load forecasting via kernel spectral clustering of smart meters. In: ICDM, pp. 943–948 (2013) Alzate, C., Sinn, M.: Improved electricity load forecasting via kernel spectral clustering of smart meters. In: ICDM, pp. 943–948 (2013)
38.
Zurück zum Zitat Langone, R., Alzate, C., De Ketelaere, B., Suykens, J.A.K.: Kernel spectral clustering for predicting maintenance of industrial machines. In: IEEE Symposium Series on Computational Intelligence and data mining SSCI (CIDM) 2013, pp. 39–45 (2013) Langone, R., Alzate, C., De Ketelaere, B., Suykens, J.A.K.: Kernel spectral clustering for predicting maintenance of industrial machines. In: IEEE Symposium Series on Computational Intelligence and data mining SSCI (CIDM) 2013, pp. 39–45 (2013)
39.
Zurück zum Zitat Langone, R., Alzate, C., De Ketelaere, B., Vlasselaer, J., Meert, W., Suykens, J.A.K.: LS-SVM based spectral clustering and regression for predicting maintenance of industrial machines. Eng. Appl. Artif. Intell. 37, 268–278 (2015)CrossRef Langone, R., Alzate, C., De Ketelaere, B., Vlasselaer, J., Meert, W., Suykens, J.A.K.: LS-SVM based spectral clustering and regression for predicting maintenance of industrial machines. Eng. Appl. Artif. Intell. 37, 268–278 (2015)CrossRef
40.
Zurück zum Zitat Mall, R., Langone, R., Suykens, J.A.K.: Multilevel hierarchical kernel spectral clustering for real-life large scale complex networks. PLoS ONE. Public Libr. Sci. 9(6), e99966 (2014)CrossRef Mall, R., Langone, R., Suykens, J.A.K.: Multilevel hierarchical kernel spectral clustering for real-life large scale complex networks. PLoS ONE. Public Libr. Sci. 9(6), e99966 (2014)CrossRef
41.
Zurück zum Zitat McAuley, J., Jure, L.: Discovering social circles in ego networks. TKDD 8(1), 4 (2014)CrossRef McAuley, J., Jure, L.: Discovering social circles in ego networks. TKDD 8(1), 4 (2014)CrossRef
42.
Zurück zum Zitat Eagle, N., Pentland, A.S., Lazer, D.: Inferring social network structure using mobile phone data. PNAS 106(1), 15274–15278 (2009)CrossRef Eagle, N., Pentland, A.S., Lazer, D.: Inferring social network structure using mobile phone data. PNAS 106(1), 15274–15278 (2009)CrossRef
43.
Zurück zum Zitat Derek, G., Padraig C.: Spectral co-clustering for dynamic bipartite graphs. In: Proceedings of the 1st Workshop on Dynamic Networks and Knowledge Discovery, Barcelona, Spain (2010) Derek, G., Padraig C.: Spectral co-clustering for dynamic bipartite graphs. In: Proceedings of the 1st Workshop on Dynamic Networks and Knowledge Discovery, Barcelona, Spain (2010)
44.
Zurück zum Zitat Guha, S., Meyerson, A., Mishra, N., Motwani, R., O’Callaghan, L.: Clustering data streams: theory and practice. IEEE Trans. Knowl. Data Eng. 15(3), 515–528 (2003)CrossRef Guha, S., Meyerson, A., Mishra, N., Motwani, R., O’Callaghan, L.: Clustering data streams: theory and practice. IEEE Trans. Knowl. Data Eng. 15(3), 515–528 (2003)CrossRef
45.
Zurück zum Zitat Aggarwal, C.C., Han, J., Wang, J., Yu, P.S.: A framework for clustering evolving data streams. In: Proceedings of the 29th International Conference on Very Large Data Bases, vol. 29, VLDB ’03, pp. 81–92 (2003) Aggarwal, C.C., Han, J., Wang, J., Yu, P.S.: A framework for clustering evolving data streams. In: Proceedings of the 29th International Conference on Very Large Data Bases, vol. 29, VLDB ’03, pp. 81–92 (2003)
46.
Zurück zum Zitat Can, F.: Incremental clustering for dynamic information processing. ACM Trans. Inf. Syst. 11(2), 143–164 (1993)MathSciNetCrossRef Can, F.: Incremental clustering for dynamic information processing. ACM Trans. Inf. Syst. 11(2), 143–164 (1993)MathSciNetCrossRef
47.
Zurück zum Zitat Gupta, C., Grossman, R.L.: GenIc: a single-pass generalized incremental algorithm for clustering. In: SDM. SIAM (2004) Gupta, C., Grossman, R.L.: GenIc: a single-pass generalized incremental algorithm for clustering. In: SDM. SIAM (2004)
48.
Zurück zum Zitat Ning, H., Xu, W., Chi, Y., Gong, Y., Huang, T.S.: Incremental spectral clustering with application to monitoring of evolving blog communities. In: SDM (2007) Ning, H., Xu, W., Chi, Y., Gong, Y., Huang, T.S.: Incremental spectral clustering with application to monitoring of evolving blog communities. In: SDM (2007)
49.
Zurück zum Zitat Alzate, C., Suykens, J.A.K.: Out-of-Sample eigenvectors in kernel spectral clustering. In: Proceedings of the International Joint Conference on Neural Networks (IJCNN 2011), pp. 2349–2356 (2011) Alzate, C., Suykens, J.A.K.: Out-of-Sample eigenvectors in kernel spectral clustering. In: Proceedings of the International Joint Conference on Neural Networks (IJCNN 2011), pp. 2349–2356 (2011)
Metadaten
Titel
Discovering Cluster Dynamics Using Kernel Spectral Methods
verfasst von
Rocco Langone
Raghvendra Mall
Joos Vandewalle
Johan A. K. Suykens
Copyright-Jahr
2016
Verlag
Springer Berlin Heidelberg
DOI
https://doi.org/10.1007/978-3-662-47824-0_1

Premium Partner