Skip to main content
Top
Published in: International Journal of Machine Learning and Cybernetics 10/2020

17-04-2020 | Original Article

Accelerated inexact matrix completion algorithm via closed-form q-thresholding \((q = 1/2, 2/3)\) operator

Authors: Zhi Wang, Chao Gao, Xiaohu Luo, Ming Tang, Jianjun Wang, Wu Chen

Published in: International Journal of Machine Learning and Cybernetics | Issue 10/2020

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

\(l_{q}\) (\(0< q < 1\)) regularization is a dominating strategy for matrix completion problems. The main goal of nonconvex \(l_{q}\) regularization based algorithm is to find a so-called low-rank solution.Unfortunately, most existing algorithms suffer from full singular value decomposition (SVD), and thus become inefficient for large-scale matrix completion problems. To alleviate this limitation, in this paper we propose an accelerated inexact algorithm to handle such problem. The key idea is to employ the closed-form q-thresholding (\(q = 1/2, 2/3\)) operator to approximate the rank of a matrix. The power method and the special “sparse plus low-rank” structure of the matrix iterates are adopted to allow efficient SVD. Besides, we employ Nesterov’s accelerated gradient method and continuation technique to further accelerate the convergence speed of our proposed algorithm. A convergence analysis shows that the sequence \(\{X_{t}\}\) generated by our proposed algorithm is bounded and has at least one accumulation point. Extensive experiments have been conducted to study its recovery performance on synthetic data, image recovery and recommendation problems. All results demonstrate that our proposed algorithm is able to achieve comparable recovery performance, while being faster and more efficient than state-of-the-art methods.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Show more products
Literature
1.
go back to reference Rennie J, Srebro N (2005) Fast maximum margin matrix factorization for collaborative prediction. In: Proceedings of the 22nd international conference on machine learning (ICML-05), pp 713–719 Rennie J, Srebro N (2005) Fast maximum margin matrix factorization for collaborative prediction. In: Proceedings of the 22nd international conference on machine learning (ICML-05), pp 713–719
2.
go back to reference Koren Y (2008) Factorization meets the neighborhood: A multifaceted collaborative filtering model. In: Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining, pp 426–434 Koren Y (2008) Factorization meets the neighborhood: A multifaceted collaborative filtering model. In: Proceedings of the 14th ACM SIGKDD international conference on Knowledge discovery and data mining, pp 426–434
3.
go back to reference Li X, Wang Z, Gao C, Shi L (2017) Reasoning human emotional responses from large-scale social and public media. Appl Math Comput 310:128–193MathSciNetMATH Li X, Wang Z, Gao C, Shi L (2017) Reasoning human emotional responses from large-scale social and public media. Appl Math Comput 310:128–193MathSciNetMATH
4.
go back to reference Peng X, Zhang Y, Tang H (2016) A unified framework for representation-based subspace clustering of out-of-sample and large-scale data. IEEE Trans Neural Netw Learn Syst 27(12):2499–2512MathSciNetCrossRef Peng X, Zhang Y, Tang H (2016) A unified framework for representation-based subspace clustering of out-of-sample and large-scale data. IEEE Trans Neural Netw Learn Syst 27(12):2499–2512MathSciNetCrossRef
5.
go back to reference Yang L, Nie F, Gao Q (2018) Nuclear-norm based semi-supervised multiple labels learning. Neurocomputing 275:940–947CrossRef Yang L, Nie F, Gao Q (2018) Nuclear-norm based semi-supervised multiple labels learning. Neurocomputing 275:940–947CrossRef
6.
go back to reference Yang L, Gao Q, Li J, Han J, Shao L (2018) Zero shot learning via low-rank embedded semantic autoencoder. In: Proceedings of the international joint conference on artificial intelligence, pp 2490–2496 Yang L, Gao Q, Li J, Han J, Shao L (2018) Zero shot learning via low-rank embedded semantic autoencoder. In: Proceedings of the international joint conference on artificial intelligence, pp 2490–2496
7.
go back to reference Cabral R, De la Torre F, Costeira JP, Bernardino A (2015) Matrix completion for weakly-supervised multi-label image classification. IEEE Trans Pattern Anal Mach Intell 37(1):121–135CrossRef Cabral R, De la Torre F, Costeira JP, Bernardino A (2015) Matrix completion for weakly-supervised multi-label image classification. IEEE Trans Pattern Anal Mach Intell 37(1):121–135CrossRef
8.
go back to reference Yang L, Shan C, Gao Q, Gao X, Han J, Cui R (2019) Hyperspectral image denoising via minimizing the partial sum of singular values and superpixel segmentation. Neurocomputing 330:465–482CrossRef Yang L, Shan C, Gao Q, Gao X, Han J, Cui R (2019) Hyperspectral image denoising via minimizing the partial sum of singular values and superpixel segmentation. Neurocomputing 330:465–482CrossRef
9.
go back to reference Recht B, Fazel M, Parrilo PA (2010) Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization. SIAM Rev 52(3):471–501MathSciNetCrossRef Recht B, Fazel M, Parrilo PA (2010) Guaranteed minimum-rank solutions of linear matrix equations via nuclear norm minimization. SIAM Rev 52(3):471–501MathSciNetCrossRef
10.
11.
go back to reference Fazel M (2002) Matrix rank minimization with applications. Ph.D. thesis, Stanford University Fazel M (2002) Matrix rank minimization with applications. Ph.D. thesis, Stanford University
12.
go back to reference Cai J-F, Candès EJ, Shen Z (2010) A singular value thresholding algorithm for matrix completion. SIAM J Optim 20(4):1956–1982MathSciNetCrossRef Cai J-F, Candès EJ, Shen Z (2010) A singular value thresholding algorithm for matrix completion. SIAM J Optim 20(4):1956–1982MathSciNetCrossRef
13.
go back to reference Toh K-C, Yun S (2010) An accelerated proximal gradient algorithm for nuclear norm regularized linear least squares problems. Pac J Optim 6(3):615–640MathSciNetMATH Toh K-C, Yun S (2010) An accelerated proximal gradient algorithm for nuclear norm regularized linear least squares problems. Pac J Optim 6(3):615–640MathSciNetMATH
14.
go back to reference Ma S, Goldfarb D, Chen L (2011) Fixed point and Bregman iterative methods for matrix rank minimization. Math Program 128(1–2):321–353MathSciNetCrossRef Ma S, Goldfarb D, Chen L (2011) Fixed point and Bregman iterative methods for matrix rank minimization. Math Program 128(1–2):321–353MathSciNetCrossRef
15.
go back to reference Mazumder R, Hastie T, Tibshirani R (2010) Spectral regularization algorithms for learning large incomplete matrices. J Mach Learn Res 11:2287–2322MathSciNetMATH Mazumder R, Hastie T, Tibshirani R (2010) Spectral regularization algorithms for learning large incomplete matrices. J Mach Learn Res 11:2287–2322MathSciNetMATH
16.
go back to reference Yao Q, Kwok JT (2015) Accelerated inexact soft-impute for fast largescale matrix completion. In: Proceedings of the international joint conference on artificial intelligence, pp 4002–4008 Yao Q, Kwok JT (2015) Accelerated inexact soft-impute for fast largescale matrix completion. In: Proceedings of the international joint conference on artificial intelligence, pp 4002–4008
17.
go back to reference Fan J, Li R (2001) Variable selection via nonconcave penalized likelihood and its Oracle properties. J Am Stat Assoc 96:1348–1361MathSciNetCrossRef Fan J, Li R (2001) Variable selection via nonconcave penalized likelihood and its Oracle properties. J Am Stat Assoc 96:1348–1361MathSciNetCrossRef
18.
go back to reference Zhang T (2010) Analysis of multi-stage convex relaxation for sparse regularization. J Mach Learn Res 11:1081–1107MathSciNetMATH Zhang T (2010) Analysis of multi-stage convex relaxation for sparse regularization. J Mach Learn Res 11:1081–1107MathSciNetMATH
19.
go back to reference Candès EJ, Wakin MB, Boyd SP (2008) Enhancing sparsity by reweighted \(l_{1}\) minimization. J Fourier Anal Appl 14(5–6):877–905MathSciNetCrossRef Candès EJ, Wakin MB, Boyd SP (2008) Enhancing sparsity by reweighted \(l_{1}\) minimization. J Fourier Anal Appl 14(5–6):877–905MathSciNetCrossRef
20.
go back to reference Hu Y, Zhang D, Ye J, Li X, He X (2013) Fast and accurate matrix completion via truncated nuclear norm regularization. IEEE Trans Pattern Anal Mach Intell 35(19):2117–2130CrossRef Hu Y, Zhang D, Ye J, Li X, He X (2013) Fast and accurate matrix completion via truncated nuclear norm regularization. IEEE Trans Pattern Anal Mach Intell 35(19):2117–2130CrossRef
21.
22.
go back to reference Marjanovic G, Solo V (2012) On \(l_{q}\) optimization and matrix completion. IEEE Trans Signal Process 60(11):5714–5724MathSciNetCrossRef Marjanovic G, Solo V (2012) On \(l_{q}\) optimization and matrix completion. IEEE Trans Signal Process 60(11):5714–5724MathSciNetCrossRef
23.
go back to reference Abu Arqub O, Abo-Hammour Z (2014) Numerical solution of systems of second-order boundary value problems using continuous genetic algorithm. Inf Sci 279:396–415MathSciNetCrossRef Abu Arqub O, Abo-Hammour Z (2014) Numerical solution of systems of second-order boundary value problems using continuous genetic algorithm. Inf Sci 279:396–415MathSciNetCrossRef
24.
go back to reference Abu Arqub O, AL-Smadi M, Momani S, Hayat M (2016) Numerical solutions of fuzzy differential equtions using reproducing kernel Hilbert space method. Soft Comput 20(8):3283–3302CrossRef Abu Arqub O, AL-Smadi M, Momani S, Hayat M (2016) Numerical solutions of fuzzy differential equtions using reproducing kernel Hilbert space method. Soft Comput 20(8):3283–3302CrossRef
25.
go back to reference Abu Arqub O, AL-Smadi M, Momani S, Hayat M (2017) Application of reproducing kernel algorithm for solving second-order, two-point fuzzy boundary value problems. Soft Comput 21(23):7191–7206CrossRef Abu Arqub O, AL-Smadi M, Momani S, Hayat M (2017) Application of reproducing kernel algorithm for solving second-order, two-point fuzzy boundary value problems. Soft Comput 21(23):7191–7206CrossRef
26.
go back to reference Abu Arqub O (2017) Adaptation of reproducing kernel algorithm for solving fuzzy Fredholm–Volterra integrodifferential equations. Neural Comput Appl 28(7):1591–1610CrossRef Abu Arqub O (2017) Adaptation of reproducing kernel algorithm for solving fuzzy Fredholm–Volterra integrodifferential equations. Neural Comput Appl 28(7):1591–1610CrossRef
27.
go back to reference Xu Z, Chang X, Xu F, Zhang H (2012) \(L_{1/2}\) regularization: a thresholding representation theory and a fast solver. IEEE Trans Neural Netw Learn Syst 23(7):1013–1027CrossRef Xu Z, Chang X, Xu F, Zhang H (2012) \(L_{1/2}\) regularization: a thresholding representation theory and a fast solver. IEEE Trans Neural Netw Learn Syst 23(7):1013–1027CrossRef
28.
go back to reference Cao W, Sun J, Xu Z (2013) Fast image deconvolution using closed-form thresholding formulas of \(l_{q}(q = 1/2, 2/3)\) regularization. J Vis Commun Image R 24(1):31–41CrossRef Cao W, Sun J, Xu Z (2013) Fast image deconvolution using closed-form thresholding formulas of \(l_{q}(q = 1/2, 2/3)\) regularization. J Vis Commun Image R 24(1):31–41CrossRef
29.
go back to reference Peng D, Xiu N, Yu J (2017) \(S_{1/2}\) regularization methods and fixed point algorithms for affine rank minimization problems. Comput Optim Appl 67:543–569MathSciNetCrossRef Peng D, Xiu N, Yu J (2017) \(S_{1/2}\) regularization methods and fixed point algorithms for affine rank minimization problems. Comput Optim Appl 67:543–569MathSciNetCrossRef
30.
go back to reference Wang Z, Wang W, Wang J, Chen S (2019) Fast and efficient algorithm for matrix completion via closed-form 2/3-thresholding operator. Neurocomputing 330:212–222CrossRef Wang Z, Wang W, Wang J, Chen S (2019) Fast and efficient algorithm for matrix completion via closed-form 2/3-thresholding operator. Neurocomputing 330:212–222CrossRef
31.
go back to reference Qian W, Cao F (2019) Adaptive algorithms for low-rank and sparse matrix recovery with truncated nuclear norm. Int J Mach Learn Cyb 10(6):1341–1355CrossRef Qian W, Cao F (2019) Adaptive algorithms for low-rank and sparse matrix recovery with truncated nuclear norm. Int J Mach Learn Cyb 10(6):1341–1355CrossRef
33.
go back to reference Li H, Lin Z (2015) Accelerated proximal gradient methods for noncovex programming. In: Proceedings of the advances in neural information processing systems, pp 379–387 Li H, Lin Z (2015) Accelerated proximal gradient methods for noncovex programming. In: Proceedings of the advances in neural information processing systems, pp 379–387
34.
go back to reference Yao Q, Kwok J (2019) Accelerated and inexact soft-impute for large-scale matrix and tensor completion. IEEE Trans Knowl Data En 31(9):1665–1679CrossRef Yao Q, Kwok J (2019) Accelerated and inexact soft-impute for large-scale matrix and tensor completion. IEEE Trans Knowl Data En 31(9):1665–1679CrossRef
35.
go back to reference Yao Q, Kwok J, Wang T, Liu T (2019) Large-scale low-rank matrix learning with nonconvex regularizers. IEEE Trans Pattern Anal Mach Intell 41(11):2628–2643CrossRef Yao Q, Kwok J, Wang T, Liu T (2019) Large-scale low-rank matrix learning with nonconvex regularizers. IEEE Trans Pattern Anal Mach Intell 41(11):2628–2643CrossRef
36.
go back to reference Gu B, Wang D, Huo Z, Huang H (2018) Inexact proximal gradient methods for non-convex and non-smooth optimization. In: AAAI conference on artificial intelligence Gu B, Wang D, Huo Z, Huang H (2018) Inexact proximal gradient methods for non-convex and non-smooth optimization. In: AAAI conference on artificial intelligence
37.
go back to reference Yao Q, Kwok J, Gao F, Chen W, Liu T (2017) Efficient inexact proximal gradient algorithm for nonconvex problems. In: Proceedings of the international joint conference on artificial intelligence, pp 3308–3314 Yao Q, Kwok J, Gao F, Chen W, Liu T (2017) Efficient inexact proximal gradient algorithm for nonconvex problems. In: Proceedings of the international joint conference on artificial intelligence, pp 3308–3314
38.
go back to reference Oh T, Matsushita Y, Tai Y, Kweon I (2018) Fast randomized singular value thresholding for low-rank optimization. IEEE Trans Pattern Anal Mach Intell 40(2):376–391CrossRef Oh T, Matsushita Y, Tai Y, Kweon I (2018) Fast randomized singular value thresholding for low-rank optimization. IEEE Trans Pattern Anal Mach Intell 40(2):376–391CrossRef
39.
go back to reference Halko N, Martinsson P-G, Tropp J (2011) Finding structure with randomness: probabilistic algorithms for constructing approximate matrix decompositions. SIAM Rev 53(2):1805–1811MathSciNetCrossRef Halko N, Martinsson P-G, Tropp J (2011) Finding structure with randomness: probabilistic algorithms for constructing approximate matrix decompositions. SIAM Rev 53(2):1805–1811MathSciNetCrossRef
40.
go back to reference Hsieh C.-J, Olsen P (2014) Nuclear norm minimization via active subspace selection. In: Proceedings of the 31st international conference on machine learning (ICML-14), pp 575–583 Hsieh C.-J, Olsen P (2014) Nuclear norm minimization via active subspace selection. In: Proceedings of the 31st international conference on machine learning (ICML-14), pp 575–583
41.
go back to reference Larsen R (1998) Lanczos bidiagonalization with partial reorthogonalization. Department of Computer Science, Aarhus University, DAIMI PB-357 Larsen R (1998) Lanczos bidiagonalization with partial reorthogonalization. Department of Computer Science, Aarhus University, DAIMI PB-357
42.
go back to reference Gong P, Zhang C, Lu Z, Huang J, Ye J (2013) A general iterative shrinkage and tresholding algorithm for non-convex regularized optimization problems. In: Proceedings of the 30th international conference on machine learning (ICML-13), pp 37–45 Gong P, Zhang C, Lu Z, Huang J, Ye J (2013) A general iterative shrinkage and tresholding algorithm for non-convex regularized optimization problems. In: Proceedings of the 30th international conference on machine learning (ICML-13), pp 37–45
43.
go back to reference Wen Z, Yin W, Zhang Y (2012) Solving a low-rank factorization model for matrix completion by a nonlinear successive over-relaxation algorithm. Math Program Comput 44(4):333–361MathSciNetCrossRef Wen Z, Yin W, Zhang Y (2012) Solving a low-rank factorization model for matrix completion by a nonlinear successive over-relaxation algorithm. Math Program Comput 44(4):333–361MathSciNetCrossRef
44.
go back to reference Wang Z, Lai M, Lu Z, Fan W, Davulcu H, Ye J (2015) Orthogonal rank-one matrix pursuit for low rank matrix completion. SIAM J Sci Comput 37(1):A488–A514MathSciNetCrossRef Wang Z, Lai M, Lu Z, Fan W, Davulcu H, Ye J (2015) Orthogonal rank-one matrix pursuit for low rank matrix completion. SIAM J Sci Comput 37(1):A488–A514MathSciNetCrossRef
45.
go back to reference Tanner J, Wei K (2016) Low rank matrix completion by alternating steepest descent methods. Appl Comput Harmon A 40:417–420MathSciNetCrossRef Tanner J, Wei K (2016) Low rank matrix completion by alternating steepest descent methods. Appl Comput Harmon A 40:417–420MathSciNetCrossRef
46.
go back to reference Xu C, Lin Z, Zha H (2017) A unified convex surrogate for the schatten-\(p\) norm. In: AAAI conference on artificial intelligence Xu C, Lin Z, Zha H (2017) A unified convex surrogate for the schatten-\(p\) norm. In: AAAI conference on artificial intelligence
47.
go back to reference Lu C, Tang J, Yan S, Lin Z (2016) Noncovex nonsmooth low rank minimization via iteratively reweighted nuclear norm. IEEE Trans Image Process 25(2):829–839MathSciNetCrossRef Lu C, Tang J, Yan S, Lin Z (2016) Noncovex nonsmooth low rank minimization via iteratively reweighted nuclear norm. IEEE Trans Image Process 25(2):829–839MathSciNetCrossRef
48.
go back to reference Jain P, Meka R, Dhillon I (2010) Guaranteed rank minimization via singular value projection. In: Proceedings of the advances in neural information processing systems, pp 937–945 Jain P, Meka R, Dhillon I (2010) Guaranteed rank minimization via singular value projection. In: Proceedings of the advances in neural information processing systems, pp 937–945
49.
go back to reference Lai M, Xu Y, Yin W (2013) Improved iteratively rewighted least squares for unconstrained smoothed \(l_{p}\) minimization. SIAM J Numer Anal 5:927–957CrossRef Lai M, Xu Y, Yin W (2013) Improved iteratively rewighted least squares for unconstrained smoothed \(l_{p}\) minimization. SIAM J Numer Anal 5:927–957CrossRef
50.
go back to reference Hastie T, Mazumder R, Lee J, Zadeh R (2015) Matrix completion and low-rank SVD via fast alternating least squares. J Mach Learn Res 16:3367–3402MathSciNetMATH Hastie T, Mazumder R, Lee J, Zadeh R (2015) Matrix completion and low-rank SVD via fast alternating least squares. J Mach Learn Res 16:3367–3402MathSciNetMATH
51.
go back to reference Thu Q, Ghanbari M (2008) Scope of validity of PSNR in image/video quality assesment. Electron Lett 44(13):800–801CrossRef Thu Q, Ghanbari M (2008) Scope of validity of PSNR in image/video quality assesment. Electron Lett 44(13):800–801CrossRef
Metadata
Title
Accelerated inexact matrix completion algorithm via closed-form q-thresholding operator
Authors
Zhi Wang
Chao Gao
Xiaohu Luo
Ming Tang
Jianjun Wang
Wu Chen
Publication date
17-04-2020
Publisher
Springer Berlin Heidelberg
Published in
International Journal of Machine Learning and Cybernetics / Issue 10/2020
Print ISSN: 1868-8071
Electronic ISSN: 1868-808X
DOI
https://doi.org/10.1007/s13042-020-01121-7

Other articles of this Issue 10/2020

International Journal of Machine Learning and Cybernetics 10/2020 Go to the issue