Skip to main content

2018 | OriginalPaper | Buchkapitel

Tensor Learning in Multi-view Kernel PCA

verfasst von : Lynn Houthuys, Johan A. K. Suykens

Erschienen in: Artificial Neural Networks and Machine Learning – ICANN 2018

Verlag: Springer International Publishing

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

In many real-life applications data can be described through multiple representations, or views. Multi-view learning aims at combining the information from all views, in order to obtain a better performance. Most well-known multi-view methods optimize some form of correlation between two views, while in many applications there are three or more views available. This is usually tackled by optimizing the correlations pairwise. However, this ignores the higher-order correlations that could only be discovered when exploring all views simultaneously. This paper proposes novel multi-view Kernel PCA models. By introducing a model tensor, the proposed models aim to include the higher-order correlations between all views. The paper further explores the use of these models as multi-view dimensionality reduction techniques and shows experimental results on several real-life datasets. These experiments demonstrate the merit of the proposed methods.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Fußnoten
1
To calculate the NMI, and hence asses the performance, the labels of the dataset are used. However, notice that they are never used in the training or validation phase of KM, KSC or the proposed multi-view KPCA models.
 
Literatur
1.
Zurück zum Zitat Alzate, C., Suykens, J.A.K.: Multiway spectral clustering with out-of-sample extensions through weighted kernel PCA. IEEE Trans. Pattern Anal. Mach. Intell. 32(2), 335–347 (2010)CrossRef Alzate, C., Suykens, J.A.K.: Multiway spectral clustering with out-of-sample extensions through weighted kernel PCA. IEEE Trans. Pattern Anal. Mach. Intell. 32(2), 335–347 (2010)CrossRef
2.
Zurück zum Zitat Andrew, G., Arora, R., Bilmes, J., Livescu, K.: Deep canonical correlation analysis. In: ICML, pp. 1247–1255 (2013) Andrew, G., Arora, R., Bilmes, J., Livescu, K.: Deep canonical correlation analysis. In: ICML, pp. 1247–1255 (2013)
3.
Zurück zum Zitat Bekker, A., Shalhon, M., Greenspan, H., Goldberger, J.: Multi-view probabilistic classification of breast microcalcifications. IEEE Trans. Med. Imaging 35(2), 645–653 (2016)CrossRef Bekker, A., Shalhon, M., Greenspan, H., Goldberger, J.: Multi-view probabilistic classification of breast microcalcifications. IEEE Trans. Med. Imaging 35(2), 645–653 (2016)CrossRef
4.
Zurück zum Zitat Blum, A., Mitchell, T.: Combining labeled and unlabeled data with co-training. In: COLT, pp. 92–100 (1998) Blum, A., Mitchell, T.: Combining labeled and unlabeled data with co-training. In: COLT, pp. 92–100 (1998)
5.
Zurück zum Zitat Davies, D.L., Bouldin, D.W.: A cluster separation measure. IEEE Trans. Pattern Anal. Mach. Intell. 1(2), 224–227 (1979)CrossRef Davies, D.L., Bouldin, D.W.: A cluster separation measure. IEEE Trans. Pattern Anal. Mach. Intell. 1(2), 224–227 (1979)CrossRef
6.
Zurück zum Zitat Foster, D.P., Kakade, S.M., Zhang, T.: Multi-view dimensionality reduction via canonical correlation analysis. Toyota Technical Institute-Chicago (2008) Foster, D.P., Kakade, S.M., Zhang, T.: Multi-view dimensionality reduction via canonical correlation analysis. Toyota Technical Institute-Chicago (2008)
8.
Zurück zum Zitat Han, Y., Wu, F., Tao, D., Shao, J., Zhuang, Y., Jiang, J.: Sparse unsupervised dimensionality reduction for multiple view data. IEEE Trans. Circ. Syst. Video Technol. 22(10), 1485–1496 (2012)CrossRef Han, Y., Wu, F., Tao, D., Shao, J., Zhuang, Y., Jiang, J.: Sparse unsupervised dimensionality reduction for multiple view data. IEEE Trans. Circ. Syst. Video Technol. 22(10), 1485–1496 (2012)CrossRef
9.
Zurück zum Zitat Hinton, G.E.: What kind of a graphical model is the brain? In: Proceedings of the 19th International Joint Conference on Artificial Intelligence, IJCAI 2005, pp. 1765–1775. Morgan Kaufmann Publishers Inc., San Francisco (2005) Hinton, G.E.: What kind of a graphical model is the brain? In: Proceedings of the 19th International Joint Conference on Artificial Intelligence, IJCAI 2005, pp. 1765–1775. Morgan Kaufmann Publishers Inc., San Francisco (2005)
10.
Zurück zum Zitat Hotelling, H.: Relations between two sets of variates. Biometrica 28, 321–377 (1936)CrossRef Hotelling, H.: Relations between two sets of variates. Biometrica 28, 321–377 (1936)CrossRef
11.
Zurück zum Zitat Houthuys, L., Langone, R., Suykens, J.A.K.: Multi-view least squares support vector machines classification. Neurocomputing 282, 78–88 (2018)CrossRef Houthuys, L., Langone, R., Suykens, J.A.K.: Multi-view least squares support vector machines classification. Neurocomputing 282, 78–88 (2018)CrossRef
13.
Zurück zum Zitat Kidron, E., Schechner, Y.Y., Elad, M.: Pixels that sound. In: CVPR, vol. 1, pp. 88–95 (2005) Kidron, E., Schechner, Y.Y., Elad, M.: Pixels that sound. In: CVPR, vol. 1, pp. 88–95 (2005)
14.
Zurück zum Zitat Kolenda, T., Hansen, L.K., Larsen, J., Winther, O.: Independent component analysis for understanding multimedia content. In: IEEE Workshop on Neural Networks for Signal Processing, vol. 12, pp. 757–766 (2002) Kolenda, T., Hansen, L.K., Larsen, J., Winther, O.: Independent component analysis for understanding multimedia content. In: IEEE Workshop on Neural Networks for Signal Processing, vol. 12, pp. 757–766 (2002)
15.
Zurück zum Zitat Kushmerick, N.: Learning to remove internet advertisements. In: AGENTS 1999, pp. 175–181 (1999) Kushmerick, N.: Learning to remove internet advertisements. In: AGENTS 1999, pp. 175–181 (1999)
16.
Zurück zum Zitat Luo, Y., Tao, D., Ramamohanarao, K., Xu, C., Wen, Y.: Tensor canonical correlation analysis for multi-view dimension reduction. IEEE Trans. Knowl. Data Eng. 27(11), 3111–3124 (2015)CrossRef Luo, Y., Tao, D., Ramamohanarao, K., Xu, C., Wen, Y.: Tensor canonical correlation analysis for multi-view dimension reduction. IEEE Trans. Knowl. Data Eng. 27(11), 3111–3124 (2015)CrossRef
17.
Zurück zum Zitat Lyon, R.F., Rehn, M., Bengio, S., Walters, T.C., Chechik, G.: Sound retrieval and ranking using sparse auditory representations. Neural Comput. 22(9), 2390–2416 (2010)CrossRef Lyon, R.F., Rehn, M., Bengio, S., Walters, T.C., Chechik, G.: Sound retrieval and ranking using sparse auditory representations. Neural Comput. 22(9), 2390–2416 (2010)CrossRef
18.
Zurück zum Zitat Macqueen, J.: Some methods for classification and analysis of multivariate observations. In: Berkeley Symposium on Mathematical Statistics and Probability, pp. 281–297 (1967) Macqueen, J.: Some methods for classification and analysis of multivariate observations. In: Berkeley Symposium on Mathematical Statistics and Probability, pp. 281–297 (1967)
19.
Zurück zum Zitat Madani, O., Georg, M., Ross, D.A.: On using nearly-independent feature families for high precision and confidence. Mach. Learn. 92, 457–477 (2013)MathSciNetCrossRef Madani, O., Georg, M., Ross, D.A.: On using nearly-independent feature families for high precision and confidence. Mach. Learn. 92, 457–477 (2013)MathSciNetCrossRef
20.
Zurück zum Zitat Mercer, J.: Functions of positive and negative type, and their connection with the theory of integral equations. Philos. Trans. R. Soc. London. Ser. A Contain. Pap. Math. Phys. Character 209, 415–446 (1909)CrossRef Mercer, J.: Functions of positive and negative type, and their connection with the theory of integral equations. Philos. Trans. R. Soc. London. Ser. A Contain. Pap. Math. Phys. Character 209, 415–446 (1909)CrossRef
21.
Zurück zum Zitat Schölkopf, B., Smola, A., Müller, K.R.: Nonlinear component analysis as a kernel eigenvalue problem. Neural Comput. 10(5), 1299–1319 (1998)CrossRef Schölkopf, B., Smola, A., Müller, K.R.: Nonlinear component analysis as a kernel eigenvalue problem. Neural Comput. 10(5), 1299–1319 (1998)CrossRef
22.
Zurück zum Zitat Signoretto, M., Tran Dinh, Q., De Lathauwer, L., Suykens, J.A.K.: Learning with tensors: a framework based on convex optimization and spectral regularization. Mach. Learn. 94, 303–351 (2014)MathSciNetCrossRef Signoretto, M., Tran Dinh, Q., De Lathauwer, L., Suykens, J.A.K.: Learning with tensors: a framework based on convex optimization and spectral regularization. Mach. Learn. 94, 303–351 (2014)MathSciNetCrossRef
23.
Zurück zum Zitat Strehl, A., Ghosh, J.: Cluster ensembles - a knowledge reuse framework for combining multiple partitions. J. Mach. Learn. Res. 3, 583–617 (2002)MathSciNetMATH Strehl, A., Ghosh, J.: Cluster ensembles - a knowledge reuse framework for combining multiple partitions. J. Mach. Learn. Res. 3, 583–617 (2002)MathSciNetMATH
24.
Zurück zum Zitat Suykens, J.A.K.: Deep restricted kernel machines using conjugate feature duality. Neural Comput. 29(8), 2123–2163 (2017)CrossRef Suykens, J.A.K.: Deep restricted kernel machines using conjugate feature duality. Neural Comput. 29(8), 2123–2163 (2017)CrossRef
25.
Zurück zum Zitat Suykens, J.A.K., Van Gestel, T., De Brabanter, J., De Moor, B., Vandewalle, J.: Least Squares Support Vector Machines. World Scientific, Singapore (2002)CrossRef Suykens, J.A.K., Van Gestel, T., De Brabanter, J., De Moor, B., Vandewalle, J.: Least Squares Support Vector Machines. World Scientific, Singapore (2002)CrossRef
26.
Zurück zum Zitat Suykens, J.A.K., Van Gestel, T., Vandewalle, J., De Moor, B.: A support vector machine formulation to PCA analysis and its kernel version. IEEE Trans. Neural Netw. 14(2), 447–450 (2003)CrossRef Suykens, J.A.K., Van Gestel, T., Vandewalle, J., De Moor, B.: A support vector machine formulation to PCA analysis and its kernel version. IEEE Trans. Neural Netw. 14(2), 447–450 (2003)CrossRef
27.
Zurück zum Zitat Wimalawarne, K., Sugiyama, M., Tomioka, R.: Multitask learning meets tensor factorization: Task imputation via convex optimization. In: NIPS, vol. 4, pp. 2825–2833 (2014) Wimalawarne, K., Sugiyama, M., Tomioka, R.: Multitask learning meets tensor factorization: Task imputation via convex optimization. In: NIPS, vol. 4, pp. 2825–2833 (2014)
29.
Zurück zum Zitat Yang, W., Toderici, G.: Discriminative tag learning on Youtube videos with latent sub-tags. In: CVPR, pp. 3217–3224 (2011) Yang, W., Toderici, G.: Discriminative tag learning on Youtube videos with latent sub-tags. In: CVPR, pp. 3217–3224 (2011)
Metadaten
Titel
Tensor Learning in Multi-view Kernel PCA
verfasst von
Lynn Houthuys
Johan A. K. Suykens
Copyright-Jahr
2018
DOI
https://doi.org/10.1007/978-3-030-01421-6_21