Skip to main content
Erschienen in: Journal of Scientific Computing 2/2020

01.11.2020

Nonconvex Optimization for Robust Tensor Completion from Grossly Sparse Observations

verfasst von: Xueying Zhao, Minru Bai, Michael K. Ng

Erschienen in: Journal of Scientific Computing | Ausgabe 2/2020

Einloggen

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

In this paper, we consider the robust tensor completion problem for recovering a low-rank tensor from limited samples and sparsely corrupted observations, especially by impulse noise. A convex relaxation of this problem is to minimize a weighted combination of tubal nuclear norm and the \(\ell _1\)-norm data fidelity term. However, the \(\ell _1\)-norm may yield biased estimators and fail to achieve the best estimation performance. To overcome this disadvantage, we propose and develop a nonconvex model, which minimizes a weighted combination of tubal nuclear norm, the \(\ell _1\)-norm data fidelity term, and a concave smooth correction term. Further, we present a Gauss–Seidel difference of convex functions algorithm (GS-DCA) to solve the resulting optimization model by using a linearization technique. We prove that the iteration sequence generated by GS-DCA converges to the critical point of the proposed model. Furthermore, we propose an extrapolation technique of GS-DCA to improve the performance of the GS-DCA. Numerical experiments for color images, hyperspectral images, magnetic resonance imaging images and videos demonstrate that the effectiveness of the proposed method.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Attouch, H., Bolte, J., Svaiter, B.F.: Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward–backward splitting, and regularized Gauss–Seidel methods. Math. Program. 137(1–2), 91–129 (2013)MathSciNetCrossRef Attouch, H., Bolte, J., Svaiter, B.F.: Convergence of descent methods for semi-algebraic and tame problems: proximal algorithms, forward–backward splitting, and regularized Gauss–Seidel methods. Math. Program. 137(1–2), 91–129 (2013)MathSciNetCrossRef
2.
Zurück zum Zitat Ahn, M.J., Pang, J.S., Xin, J.: Difference-of-convex learning: directional stationarity, optimality, and sparsity. SIAM J. Optim. 27(3), 1637–1665 (2017)MathSciNetCrossRef Ahn, M.J., Pang, J.S., Xin, J.: Difference-of-convex learning: directional stationarity, optimality, and sparsity. SIAM J. Optim. 27(3), 1637–1665 (2017)MathSciNetCrossRef
3.
Zurück zum Zitat Bai, M.R., Zhang, X.J., Ni, G.Y., Cui, C.F.: An adaptive correction approach for tensor completion. SIAM J. Imaging Sci. 9(3), 1298–1323 (2016)MathSciNetCrossRef Bai, M.R., Zhang, X.J., Ni, G.Y., Cui, C.F.: An adaptive correction approach for tensor completion. SIAM J. Imaging Sci. 9(3), 1298–1323 (2016)MathSciNetCrossRef
4.
Zurück zum Zitat Bai, M.R., Zhang, X.J., Shao, Q.Q.: Adaptive correction procedure for TVL1 image deblurring under impulse noise. Inverse Probl. 32, 085004 (2016)MathSciNetCrossRef Bai, M.R., Zhang, X.J., Shao, Q.Q.: Adaptive correction procedure for TVL1 image deblurring under impulse noise. Inverse Probl. 32, 085004 (2016)MathSciNetCrossRef
5.
Zurück zum Zitat Bochnak, J., Coste, M., Roy, M.F.: Real Algebraic Geometry. Ergeb. Math. Grenzgeb 36. Springer, Berlin (1998)CrossRef Bochnak, J., Coste, M., Roy, M.F.: Real Algebraic Geometry. Ergeb. Math. Grenzgeb 36. Springer, Berlin (1998)CrossRef
6.
Zurück zum Zitat Bolte, J., Sabach, S., Teboulle, M.: Proximal alternating linearized minimization for nonconvex and nonsmooth problems. Math. Program. 146(1), 459–494 (2014)MathSciNetCrossRef Bolte, J., Sabach, S., Teboulle, M.: Proximal alternating linearized minimization for nonconvex and nonsmooth problems. Math. Program. 146(1), 459–494 (2014)MathSciNetCrossRef
7.
8.
Zurück zum Zitat Fan, H.Y., Chen, Y.J., Guo, Y.L., Zhang, H.Y., Kuang, G.Y.: Hyperspectral image restoration using low-rank tensor recovery. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 10, 4589–4604 (2017)CrossRef Fan, H.Y., Chen, Y.J., Guo, Y.L., Zhang, H.Y., Kuang, G.Y.: Hyperspectral image restoration using low-rank tensor recovery. IEEE J. Sel. Top. Appl. Earth Obs. Remote Sens. 10, 4589–4604 (2017)CrossRef
9.
Zurück zum Zitat Fan, J.Q., Li, R.Z.: Variable selection via nonconcave penalized likelihood and its oracle properties. J. Am. Stat. Assoc. 96(456), 1348–1360 (2001)MathSciNetCrossRef Fan, J.Q., Li, R.Z.: Variable selection via nonconcave penalized likelihood and its oracle properties. J. Am. Stat. Assoc. 96(456), 1348–1360 (2001)MathSciNetCrossRef
10.
Zurück zum Zitat Gandy, S., Recht, B., Yamada, I.: Tensor completion and low-n-rank tensor recovery via convex optimization. Inverse Probl. 27(2), 025010 (2011)MathSciNetCrossRef Gandy, S., Recht, B., Yamada, I.: Tensor completion and low-n-rank tensor recovery via convex optimization. Inverse Probl. 27(2), 025010 (2011)MathSciNetCrossRef
11.
Zurück zum Zitat Goldfarb, D., Qin, Z.W.: Robust low-rank tensor recovery: models and algorithms. SIAM J. Matrix Anal. Appl. 35(1), 225–253 (2014)MathSciNetCrossRef Goldfarb, D., Qin, Z.W.: Robust low-rank tensor recovery: models and algorithms. SIAM J. Matrix Anal. Appl. 35(1), 225–253 (2014)MathSciNetCrossRef
12.
Zurück zum Zitat Gu, G.Y., Jiang, S.H., Yang, J.F.: A TVSCAD approach for image deblurring with impulse noise. Inverse Probl. 33(12), 125008 (2017)CrossRef Gu, G.Y., Jiang, S.H., Yang, J.F.: A TVSCAD approach for image deblurring with impulse noise. Inverse Probl. 33(12), 125008 (2017)CrossRef
13.
Zurück zum Zitat Hitchcock, F.L.: The expression of a tensor or a polyadic as a sum of products. J. Math. Phys. 6, 164–189 (1927)CrossRef Hitchcock, F.L.: The expression of a tensor or a polyadic as a sum of products. J. Math. Phys. 6, 164–189 (1927)CrossRef
14.
Zurück zum Zitat Hong, M.Y., Luo, Z.Q., Razaviyayn, M.: Convergence analysis of alternating direction method of multipliers for a family of nonconvex problems. SIAM J. Optim. 26(1), 337–364 (2016)MathSciNetCrossRef Hong, M.Y., Luo, Z.Q., Razaviyayn, M.: Convergence analysis of alternating direction method of multipliers for a family of nonconvex problems. SIAM J. Optim. 26(1), 337–364 (2016)MathSciNetCrossRef
15.
Zurück zum Zitat Jiang, Q., Ng, M.: Robust low-tubal-rank tensor completion via convex optimization. In: Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, pp. 2649–2655 (2019) Jiang, Q., Ng, M.: Robust low-tubal-rank tensor completion via convex optimization. In: Proceedings of the Twenty-Eighth International Joint Conference on Artificial Intelligence, pp. 2649–2655 (2019)
16.
Zurück zum Zitat Karatzoglou, A., Amatriain, X., Baltrunas, L., Oliver, N.: Multiverse recommendation: n-dimensional tensor factorization for context-aware collaborative filtering. In: Proceedings of the fourth ACM Conference on Recommender Systems, pp. 79–86 (2010) Karatzoglou, A., Amatriain, X., Baltrunas, L., Oliver, N.: Multiverse recommendation: n-dimensional tensor factorization for context-aware collaborative filtering. In: Proceedings of the fourth ACM Conference on Recommender Systems, pp. 79–86 (2010)
17.
Zurück zum Zitat Kilmer, M.W., Braman, K., Hao, N., Hoover, R.C.: Third-order tensors as operators on matrices: a theoretical and computational framework with applications in imaging. SIAM J. Matrix Anal. Appl. 34(1), 148–172 (2013)MathSciNetCrossRef Kilmer, M.W., Braman, K., Hao, N., Hoover, R.C.: Third-order tensors as operators on matrices: a theoretical and computational framework with applications in imaging. SIAM J. Matrix Anal. Appl. 34(1), 148–172 (2013)MathSciNetCrossRef
18.
Zurück zum Zitat Kilmer, M.W., Martin, C.D.: Factorization strategies for third-order tensors. Linear Algebra Appl. 435, 641–658 (2011)MathSciNetCrossRef Kilmer, M.W., Martin, C.D.: Factorization strategies for third-order tensors. Linear Algebra Appl. 435, 641–658 (2011)MathSciNetCrossRef
19.
20.
Zurück zum Zitat Lathauwer, L.D., Moor, B.D.: From matrix to tensor: multilinear algebra and signal processing. In: McWhirter, J., Proudler, I. (eds.) Mathematics in Signal Processing IV, pp. 1–15. Clarendon Press, Oxford (1998) Lathauwer, L.D., Moor, B.D.: From matrix to tensor: multilinear algebra and signal processing. In: McWhirter, J., Proudler, I. (eds.) Mathematics in Signal Processing IV, pp. 1–15. Clarendon Press, Oxford (1998)
21.
Zurück zum Zitat Li, Y.F., Shang, K., Huang, Z.H.: Low Tucker rank tensor recovery via ADMM based on exact and inexact iteratively reweighted algorithms. J. Comput. Appl. Math. 331(1), 64–81 (2018)MathSciNetCrossRef Li, Y.F., Shang, K., Huang, Z.H.: Low Tucker rank tensor recovery via ADMM based on exact and inexact iteratively reweighted algorithms. J. Comput. Appl. Math. 331(1), 64–81 (2018)MathSciNetCrossRef
22.
Zurück zum Zitat Liu, J., Musialski, P., Wonka, P., Ye, J.: Tensor completion for estimating missing values invisual data. IEEE Trans. Pattern Anal. Mach. Intell. 35(1), 208–220 (2013)CrossRef Liu, J., Musialski, P., Wonka, P., Ye, J.: Tensor completion for estimating missing values invisual data. IEEE Trans. Pattern Anal. Mach. Intell. 35(1), 208–220 (2013)CrossRef
23.
Zurück zum Zitat Lu, C.Y., Feng, J.S., Chen, Y.D., Liu, W., Lin, Z.C., Yan, S.: Tensor robust principal component analysis with a new tensor nuclear norm. IEEE Trans. Pattern Anal. Mach. Intell. 42(4), 925–938 (2020)CrossRef Lu, C.Y., Feng, J.S., Chen, Y.D., Liu, W., Lin, Z.C., Yan, S.: Tensor robust principal component analysis with a new tensor nuclear norm. IEEE Trans. Pattern Anal. Mach. Intell. 42(4), 925–938 (2020)CrossRef
24.
Zurück zum Zitat MøRup, M.: Applications of tensor (multiway array) factorizations and decompositions in data mining. Wiley Interdiscip. Rev. Data Min. Knowl. Discov. 1(1), 24–40 (2011)CrossRef MøRup, M.: Applications of tensor (multiway array) factorizations and decompositions in data mining. Wiley Interdiscip. Rev. Data Min. Knowl. Discov. 1(1), 24–40 (2011)CrossRef
25.
Zurück zum Zitat Mu, C., Huang, B., Wright, J., Goldfarb, D.: Square deal: lower bounds and improved relaxations for tensor recovery. In: Proceedings of the 31st International Conference on Machine Learning, vol. 32, pp. 73–81 (2014) Mu, C., Huang, B., Wright, J., Goldfarb, D.: Square deal: lower bounds and improved relaxations for tensor recovery. In: Proceedings of the 31st International Conference on Machine Learning, vol. 32, pp. 73–81 (2014)
26.
Zurück zum Zitat Miao, W.M., Pan, S.H., Sun, D.F.: A rank-corrected procedure for matrix completion with fixed basis coefficients. Math. Program. 159, 289–338 (2016)MathSciNetCrossRef Miao, W.M., Pan, S.H., Sun, D.F.: A rank-corrected procedure for matrix completion with fixed basis coefficients. Math. Program. 159, 289–338 (2016)MathSciNetCrossRef
27.
28.
Zurück zum Zitat Patwardhan, K.A., Sapiro, G., Bertalmio, M.: Video inpainting under constrained camera motion. IEEE Trans. Image Process. 16(2), 545–553 (2007)MathSciNetCrossRef Patwardhan, K.A., Sapiro, G., Bertalmio, M.: Video inpainting under constrained camera motion. IEEE Trans. Image Process. 16(2), 545–553 (2007)MathSciNetCrossRef
29.
Zurück zum Zitat Rockafellar, R.T., Wets, R.J.: Variational Analysis. Springer, Berlin (1997)MATH Rockafellar, R.T., Wets, R.J.: Variational Analysis. Springer, Berlin (1997)MATH
30.
Zurück zum Zitat Romera-Paredes, B., Pontil, M.: A new convex relaxation for tensor completion. In: Proceedings of Advances in Neural Informance Processing Systems, vol. 26, pp. 2967–2975 (2013) Romera-Paredes, B., Pontil, M.: A new convex relaxation for tensor completion. In: Proceedings of Advances in Neural Informance Processing Systems, vol. 26, pp. 2967–2975 (2013)
31.
Zurück zum Zitat Salakhutdinov, R., Srebro, N.: Collaborative filtering in a non-uniform world: learning with the weighted trace norm. In: Proceedings of Advances in Neural Informance Processing Systems, vol. 23,pp. 2056–2064 (2010) Salakhutdinov, R., Srebro, N.: Collaborative filtering in a non-uniform world: learning with the weighted trace norm. In: Proceedings of Advances in Neural Informance Processing Systems, vol. 23,pp. 2056–2064 (2010)
32.
Zurück zum Zitat Sauve, A.C., Hero, A.O., Rogers, W.L., Wilderman, S.J., Clinthorne, N.H.: 3D image reconstruction for a compton spect camera model. IEEE Trans. Nucl. Sci. 46(6), 2075–2084 (1999)CrossRef Sauve, A.C., Hero, A.O., Rogers, W.L., Wilderman, S.J., Clinthorne, N.H.: 3D image reconstruction for a compton spect camera model. IEEE Trans. Nucl. Sci. 46(6), 2075–2084 (1999)CrossRef
33.
Zurück zum Zitat Semerci, O.S., Hao, N., Kilmer, M.E., Miller, E.L.: Tensor-based formulation and nuclear norm regularization for multienergy computed tomography. IEEE Trans. Image Process. 23(4), 1678–1693 (2014)MathSciNetCrossRef Semerci, O.S., Hao, N., Kilmer, M.E., Miller, E.L.: Tensor-based formulation and nuclear norm regularization for multienergy computed tomography. IEEE Trans. Image Process. 23(4), 1678–1693 (2014)MathSciNetCrossRef
34.
Zurück zum Zitat Signoretto, M., Dinh, Q.T., Lathauwer, L.D., Suykens, J.A.K.: Learning with tensors: a framework based on convex optimization and spectral regularization. Mach. Learn. 94, 303–351 (2014)MathSciNetCrossRef Signoretto, M., Dinh, Q.T., Lathauwer, L.D., Suykens, J.A.K.: Learning with tensors: a framework based on convex optimization and spectral regularization. Mach. Learn. 94, 303–351 (2014)MathSciNetCrossRef
35.
Zurück zum Zitat Sun, T., Yin, P.H., Cheng, L.Z., Jiang, H.: Alternating direction method of multipliers with difference of convex functions. Adv. Comput. Math. 44, 723–744 (2018)MathSciNetCrossRef Sun, T., Yin, P.H., Cheng, L.Z., Jiang, H.: Alternating direction method of multipliers with difference of convex functions. Adv. Comput. Math. 44, 723–744 (2018)MathSciNetCrossRef
36.
Zurück zum Zitat Tao, P.D., An, L.T.H.: Convex analysis approach to D.C. programming: theory, algorithms and applications. Acta Math. Vietnam. 22, 289–355 (1997)MathSciNetMATH Tao, P.D., An, L.T.H.: Convex analysis approach to D.C. programming: theory, algorithms and applications. Acta Math. Vietnam. 22, 289–355 (1997)MathSciNetMATH
37.
Zurück zum Zitat Tao, P.D.: Convergence of a subgradient method for computing the bound norm of matrices. Linear Algebra Appl. 62, 163–182 (1984)MathSciNetCrossRef Tao, P.D.: Convergence of a subgradient method for computing the bound norm of matrices. Linear Algebra Appl. 62, 163–182 (1984)MathSciNetCrossRef
38.
Zurück zum Zitat Tao, P.D., Bernoussi, S.E.: Algorithms for solving a class of nonconvex optimization problems. Methods of subgradients. North-Holl. Math. Stud. 129, 249–271 (1986)MathSciNetCrossRef Tao, P.D., Bernoussi, S.E.: Algorithms for solving a class of nonconvex optimization problems. Methods of subgradients. North-Holl. Math. Stud. 129, 249–271 (1986)MathSciNetCrossRef
39.
40.
Zurück zum Zitat Wang, Y., Yin, W.T., Zeng, J.S.: Global convergence of ADMM in nonconvex nonsmooth optimization. J. Sci. Comput. 78, 29–63 (2019)MathSciNetCrossRef Wang, Y., Yin, W.T., Zeng, J.S.: Global convergence of ADMM in nonconvex nonsmooth optimization. J. Sci. Comput. 78, 29–63 (2019)MathSciNetCrossRef
41.
Zurück zum Zitat Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)CrossRef Wang, Z., Bovik, A.C., Sheikh, H.R., Simoncelli, E.P.: Image quality assessment: from error visibility to structural similarity. IEEE Trans. Image Process. 13(4), 600–612 (2004)CrossRef
42.
Zurück zum Zitat Wright, J., Peng, Y., Ma, Y., Ganesh, A., Rao, S.: Robust principal component analysis: exact recovery of corrupted low-rank matrices via convex optimization. In: Proceedings of Advances in Neural Informance Processing Systems, vol. 22, pp. 2080–2088 (2009) Wright, J., Peng, Y., Ma, Y., Ganesh, A., Rao, S.: Robust principal component analysis: exact recovery of corrupted low-rank matrices via convex optimization. In: Proceedings of Advances in Neural Informance Processing Systems, vol. 22, pp. 2080–2088 (2009)
43.
Zurück zum Zitat Yang, L., Pong, T.K., Chen, X.J.: Alternating direction method of multipliers for a class of nonconvex and nonsmooth problems with applications to background/foreground extraction. SIAM J. Imaging Sci. 10(1), 74–110 (2017)MathSciNetCrossRef Yang, L., Pong, T.K., Chen, X.J.: Alternating direction method of multipliers for a class of nonconvex and nonsmooth problems with applications to background/foreground extraction. SIAM J. Imaging Sci. 10(1), 74–110 (2017)MathSciNetCrossRef
44.
Zurück zum Zitat Yang, Y.N., Feng, Y.L., Suykens, J.A.K.: Robust low-rank tensor recovery with regularized redescending m-estimator. IEEE Trans. Neural Netw. Learn. Syst. 27(9), 1933–1946 (2016)MathSciNetCrossRef Yang, Y.N., Feng, Y.L., Suykens, J.A.K.: Robust low-rank tensor recovery with regularized redescending m-estimator. IEEE Trans. Neural Netw. Learn. Syst. 27(9), 1933–1946 (2016)MathSciNetCrossRef
45.
Zurück zum Zitat Zhang, C.H.: Nearly unbiased variable selection under minimax concave penalty. Ann. Stat. 38(2), 894–942 (2010)MathSciNetCrossRef Zhang, C.H.: Nearly unbiased variable selection under minimax concave penalty. Ann. Stat. 38(2), 894–942 (2010)MathSciNetCrossRef
Metadaten
Titel
Nonconvex Optimization for Robust Tensor Completion from Grossly Sparse Observations
verfasst von
Xueying Zhao
Minru Bai
Michael K. Ng
Publikationsdatum
01.11.2020
Verlag
Springer US
Erschienen in
Journal of Scientific Computing / Ausgabe 2/2020
Print ISSN: 0885-7474
Elektronische ISSN: 1573-7691
DOI
https://doi.org/10.1007/s10915-020-01356-0

Weitere Artikel der Ausgabe 2/2020

Journal of Scientific Computing 2/2020 Zur Ausgabe

Premium Partner