Skip to main content
Top

2017 | OriginalPaper | Chapter

Feature Extraction for Incomplete Data via Low-rank Tucker Decomposition

Authors : Qiquan Shi, Yiu-ming Cheung, Qibin Zhao

Published in: Machine Learning and Knowledge Discovery in Databases

Publisher: Springer International Publishing

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Extracting features from incomplete tensors is a challenging task which is not well explored. Due to the data with missing entries, existing feature extraction methods are not applicable. Although tensor completion techniques can estimate the missing entries well, they focus on data recovery and do not consider the relationships among tensor samples for effective feature extraction. To solve this problem of feature extraction for incomplete data, we propose an unsupervised method, TDVM, which incorporates low-rank T ucker D ecomposition with feature V ariance M aximization in a unified framework. Based on Tucker decomposition, we impose nuclear norm regularization on the core tensors while minimizing reconstruction errors, and meanwhile maximize the variance of core tensors (i.e., extracted features). Here, the relationships among tensor samples are explored via variance maximization while estimating the missing entries. We thus can simultaneously obtain lower-dimensional core tensors and informative features directly from observed entries. The alternating direction method of multipliers approach is utilized to solve the optimization objective. We evaluate the features extracted from two real data with different missing entries for face recognition tasks. Experimental results illustrate the superior performance of our method with a significant improvement over the state-of-the-art methods.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literature
2.
go back to reference Tucker, L.R.: Implications of factor analysis of three-way matrices for measurement of change. Prob. Meas. Change 15, 122–137 (1963) Tucker, L.R.: Implications of factor analysis of three-way matrices for measurement of change. Prob. Meas. Change 15, 122–137 (1963)
3.
go back to reference Lu, H., Plataniotis, K.N., Venetsanopoulos, A.N.: MPCA: multilinear principal component analysis of tensor objects. IEEE Trans. Neural Netw. 19(1), 18–39 (2008)CrossRef Lu, H., Plataniotis, K.N., Venetsanopoulos, A.N.: MPCA: multilinear principal component analysis of tensor objects. IEEE Trans. Neural Netw. 19(1), 18–39 (2008)CrossRef
4.
go back to reference Lu, J., Tan, Y.-P., Wang, G.: Discriminative multimanifold analysis for face recognition from a single training sample per person. IEEE Trans. Pattern Anal. Mach. Intell. 35(1), 39–51 (2013)CrossRef Lu, J., Tan, Y.-P., Wang, G.: Discriminative multimanifold analysis for face recognition from a single training sample per person. IEEE Trans. Pattern Anal. Mach. Intell. 35(1), 39–51 (2013)CrossRef
5.
go back to reference Shi, Q., Lu, H.: Semi-orthogonal multilinear PCA with relaxed start. In: International Conference on Joint Conference on Artificial Intelligence (2015) Shi, Q., Lu, H.: Semi-orthogonal multilinear PCA with relaxed start. In: International Conference on Joint Conference on Artificial Intelligence (2015)
7.
go back to reference Li, X., Ng, M.K., Cong, G., Ye, Y., Wu, Q.: MR-NTD: manifold regularization nonnegative tucker decomposition for tensor data dimension reduction and representation. IEEE Trans. Neural Netw. Learn. Syst. 28(8), 1787–1800 (2017). IEEEMathSciNetCrossRef Li, X., Ng, M.K., Cong, G., Ye, Y., Wu, Q.: MR-NTD: manifold regularization nonnegative tucker decomposition for tensor data dimension reduction and representation. IEEE Trans. Neural Netw. Learn. Syst. 28(8), 1787–1800 (2017). IEEEMathSciNetCrossRef
8.
go back to reference Jolliffe, I.T.: Principal Component Analysis (2nd edn), In: Springer Serires in Statistics (2002) Jolliffe, I.T.: Principal Component Analysis (2nd edn), In: Springer Serires in Statistics (2002)
9.
go back to reference Lu, C., Feng, J., Chen, Y., Liu, W., Lin, Z., Yan, S.: Tensor robust principal component analysis: exact recovery of corrupted low-rank tensors via convex optimization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5249–5257 (2016) Lu, C., Feng, J., Chen, Y., Liu, W., Lin, Z., Yan, S.: Tensor robust principal component analysis: exact recovery of corrupted low-rank tensors via convex optimization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5249–5257 (2016)
10.
go back to reference Peng, Y., Ganesh, A., Wright, J., Wenli, X., Ma, Y.: Rasl: robust alignment by sparse and low-rank decomposition for linearly correlated images. IEEE Trans. Pattern Anal. Mach. Intell. 34(11), 2233–2246 (2012)CrossRef Peng, Y., Ganesh, A., Wright, J., Wenli, X., Ma, Y.: Rasl: robust alignment by sparse and low-rank decomposition for linearly correlated images. IEEE Trans. Pattern Anal. Mach. Intell. 34(11), 2233–2246 (2012)CrossRef
11.
go back to reference Liu, G., Lin, Z., Yan, S., Sun, J., Yu, Y., Ma, Y.: Robust recovery of subspace structures by low-rank representation. IEEE Trans. Pattern Anal. Mach. Intell. 35(1), 171–184 (2013)CrossRef Liu, G., Lin, Z., Yan, S., Sun, J., Yu, Y., Ma, Y.: Robust recovery of subspace structures by low-rank representation. IEEE Trans. Pattern Anal. Mach. Intell. 35(1), 171–184 (2013)CrossRef
12.
go back to reference Acar, E., Dunlavy, D.M., Kolda, T.G., Mørup, M.: Scalable tensor factorizations for incomplete data. Chemometr. Intell. Lab. Syst. 106(1), 41–56 (2011)CrossRef Acar, E., Dunlavy, D.M., Kolda, T.G., Mørup, M.: Scalable tensor factorizations for incomplete data. Chemometr. Intell. Lab. Syst. 106(1), 41–56 (2011)CrossRef
13.
go back to reference Williams, D., Liao, X., Xue, Y., Carin, L., Krishnapuram, B.: On classification with incomplete data. IEEE Trans. Pattern Anal. Mach. Intell. 29(3), 427–436 (2007). IEEECrossRef Williams, D., Liao, X., Xue, Y., Carin, L., Krishnapuram, B.: On classification with incomplete data. IEEE Trans. Pattern Anal. Mach. Intell. 29(3), 427–436 (2007). IEEECrossRef
14.
go back to reference Williams, D., Liao, X., Xue, Y., Carin, L.: Incomplete-data classification using logistic regression. In: Proceedings of the 22nd International Conference on Machine Learning, pp. 972–979. ACM (2005) Williams, D., Liao, X., Xue, Y., Carin, L.: Incomplete-data classification using logistic regression. In: Proceedings of the 22nd International Conference on Machine Learning, pp. 972–979. ACM (2005)
15.
go back to reference Guleryuz, O.G.: Nonlinear approximation based image recovery using adaptive sparse reconstructions and iterated denoising-part I: theory. IEEE Trans. Image Process. 15(3), 539–554 (2006)CrossRef Guleryuz, O.G.: Nonlinear approximation based image recovery using adaptive sparse reconstructions and iterated denoising-part I: theory. IEEE Trans. Image Process. 15(3), 539–554 (2006)CrossRef
16.
go back to reference Hazan, E., Livni, R., Mansour, Y.: Classification with low rank and missing data. In: Proceedings of the 32nd International Conference on Machine Learning, pp. 257–266 (2015) Hazan, E., Livni, R., Mansour, Y.: Classification with low rank and missing data. In: Proceedings of the 32nd International Conference on Machine Learning, pp. 257–266 (2015)
17.
go back to reference Liu, J., Musialski, P., Wonka, P., Ye, J.: Tensor completion for estimating missing values in visual data. IEEE Trans. Pattern Anal. Mach. Intell. 35(1), 208–220 (2013)CrossRef Liu, J., Musialski, P., Wonka, P., Ye, J.: Tensor completion for estimating missing values in visual data. IEEE Trans. Pattern Anal. Mach. Intell. 35(1), 208–220 (2013)CrossRef
18.
go back to reference Hu, W., Tao, D., Zhang, W., Xie, Y., Yang, Y.: The twist tensor nuclear norm for video completion. IEEE Trans. Neural Netw. Learn. Syst. 28(12), 2961–2973 (2017). IEEECrossRef Hu, W., Tao, D., Zhang, W., Xie, Y., Yang, Y.: The twist tensor nuclear norm for video completion. IEEE Trans. Neural Netw. Learn. Syst. 28(12), 2961–2973 (2017). IEEECrossRef
19.
go back to reference Liu, Y., Shang, F., Fan, W., Cheng, J., Cheng, H.: Generalized higher-order orthogonal iteration for tensor decomposition and completion. In: Advances in Neural Information Processing Systems, pp. 1763–1771 (2014) Liu, Y., Shang, F., Fan, W., Cheng, J., Cheng, H.: Generalized higher-order orthogonal iteration for tensor decomposition and completion. In: Advances in Neural Information Processing Systems, pp. 1763–1771 (2014)
20.
go back to reference Zhou, G., Cichocki, A., Zhao, Q., Xie, S.: Efficient nonnegative tucker decompositions: algorithms and uniqueness. IEEE Trans. Image Process. 24(12), 4990–5003 (2015)MathSciNetCrossRef Zhou, G., Cichocki, A., Zhao, Q., Xie, S.: Efficient nonnegative tucker decompositions: algorithms and uniqueness. IEEE Trans. Image Process. 24(12), 4990–5003 (2015)MathSciNetCrossRef
21.
go back to reference Jia, C., Zhong, G., Fu, Y.: Low-rank tensor learning with discriminant analysis for action classification and image recovery. In: Proceedings of the International Conference Artificial Intelligence, pp. 1228–1234. AAAI Press (2014) Jia, C., Zhong, G., Fu, Y.: Low-rank tensor learning with discriminant analysis for action classification and image recovery. In: Proceedings of the International Conference Artificial Intelligence, pp. 1228–1234. AAAI Press (2014)
22.
23.
go back to reference De Lathauwer, L., De Moor, B., Vandewalle, J.: A multilinear singular value decomposition. SIAM J. Matrix Anal. Appl. 21(4), 1253–1278 (2000)MathSciNetCrossRefMATH De Lathauwer, L., De Moor, B., Vandewalle, J.: A multilinear singular value decomposition. SIAM J. Matrix Anal. Appl. 21(4), 1253–1278 (2000)MathSciNetCrossRefMATH
24.
go back to reference Boyd, S.: Alternating direction method of multipliers. In: NIPS Workshop on Optimization and Machine Learning (2011) Boyd, S.: Alternating direction method of multipliers. In: NIPS Workshop on Optimization and Machine Learning (2011)
25.
go back to reference Lu, H., Plataniotis, K.N., Venetsanopoulos, A.: Multilinear Subspace Learning: Dimensionality Reduction of Multidimensional Data. CRC Press, Boca Raton (2013) Lu, H., Plataniotis, K.N., Venetsanopoulos, A.: Multilinear Subspace Learning: Dimensionality Reduction of Multidimensional Data. CRC Press, Boca Raton (2013)
26.
go back to reference Wright, J., Yang, A.Y., Ganesh, A., Sastry, S.S., Ma, Y.: Robust face recognition via sparse representation. IEEE Trans. Pattern Anal. Mach. Intell. 31(2), 210–227 (2009)CrossRef Wright, J., Yang, A.Y., Ganesh, A., Sastry, S.S., Ma, Y.: Robust face recognition via sparse representation. IEEE Trans. Pattern Anal. Mach. Intell. 31(2), 210–227 (2009)CrossRef
27.
go back to reference Liu, G., Yan, S.: Latent low-rank representation for subspace segmentation and feature extraction. In: Proceedings of the IEEE Conference on Computer Vision, pp. 1615–1622. IEEE (2011) Liu, G., Yan, S.: Latent low-rank representation for subspace segmentation and feature extraction. In: Proceedings of the IEEE Conference on Computer Vision, pp. 1615–1622. IEEE (2011)
28.
go back to reference Elhamifar, E., Vidal, R.: Sparse subspace clustering: algorithm, theory, and applications. IEEE Trans. Pattern Anal. Mach. Intell. 35(11), 2765–2781 (2013)CrossRef Elhamifar, E., Vidal, R.: Sparse subspace clustering: algorithm, theory, and applications. IEEE Trans. Pattern Anal. Mach. Intell. 35(11), 2765–2781 (2013)CrossRef
29.
go back to reference Cai, J.-F., Candès, E.J., Shen, Z.: A singular value thresholding algorithm for matrix completion. SIAM J. Optim. 20(4), 1956–1982 (2010)MathSciNetCrossRefMATH Cai, J.-F., Candès, E.J., Shen, Z.: A singular value thresholding algorithm for matrix completion. SIAM J. Optim. 20(4), 1956–1982 (2010)MathSciNetCrossRefMATH
30.
go back to reference Shang, F., Liu, Y., Cheng, J.: Generalized higher-order tensor decomposition via parallel ADMM. In: Proceedings AAAI Conference on Artificial Intelligence, pp. 1279–1285. AAAI Press (2014) Shang, F., Liu, Y., Cheng, J.: Generalized higher-order tensor decomposition via parallel ADMM. In: Proceedings AAAI Conference on Artificial Intelligence, pp. 1279–1285. AAAI Press (2014)
31.
go back to reference Higham, N., Papadimitriou, P.: Matrix procrustes problems. Rapport Technique, University of Manchester (1995) Higham, N., Papadimitriou, P.: Matrix procrustes problems. Rapport Technique, University of Manchester (1995)
32.
go back to reference Liu, Y., Shang, F., Fan, W., Cheng, J., Cheng, H.: Generalized higher order orthogonal iteration for tensor learning and decomposition. IEEE Trans. Neural Netw. Learn. Syst. 27(12), 2551–2563 (2016)MathSciNetCrossRef Liu, Y., Shang, F., Fan, W., Cheng, J., Cheng, H.: Generalized higher order orthogonal iteration for tensor learning and decomposition. IEEE Trans. Neural Netw. Learn. Syst. 27(12), 2551–2563 (2016)MathSciNetCrossRef
33.
go back to reference LeCun, Y., Cortes, C., Burges, C.J.C.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998). IEEECrossRef LeCun, Y., Cortes, C., Burges, C.J.C.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998). IEEECrossRef
34.
go back to reference Phillips, P.J., Moon, H., Rizvi, S.A., Rauss, P.J.: The FERET evaluation methodology for face-recognition algorithms. IEEE Trans. Pattern Anal. Mach. Intell. 22(10), 1090–1104 (2000)CrossRef Phillips, P.J., Moon, H., Rizvi, S.A., Rauss, P.J.: The FERET evaluation methodology for face-recognition algorithms. IEEE Trans. Pattern Anal. Mach. Intell. 22(10), 1090–1104 (2000)CrossRef
35.
go back to reference Lee, K.C., Ho, J., Kriegman, D.: Acquiring linear subspaces for face recognition under variable lighting. IEEE Trans. Pattern Anal. Mach. Intell. 27(5), 684–698 (2005)CrossRef Lee, K.C., Ho, J., Kriegman, D.: Acquiring linear subspaces for face recognition under variable lighting. IEEE Trans. Pattern Anal. Mach. Intell. 27(5), 684–698 (2005)CrossRef
Metadata
Title
Feature Extraction for Incomplete Data via Low-rank Tucker Decomposition
Authors
Qiquan Shi
Yiu-ming Cheung
Qibin Zhao
Copyright Year
2017
DOI
https://doi.org/10.1007/978-3-319-71249-9_34

Premium Partner