Skip to main content

2020 | OriginalPaper | Buchkapitel

A Robust Fully Correntropy–Based Sparse Modeling Alternative to Dictionary Learning

Aktivieren Sie unsere intelligente Suche, um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Abstract

Correntropy is a dependence measure that goes beyond Gaussian environments and optimizations based on Minimum Squared Error (MSE). Its ability to induce a metric that is fully modulated by a single parameter makes it an attractive tool for adaptive signal processing. We propose a sparse modeling framework based on the dictionary learning technique known as K–SVD where Correntropy replaces MSE in the sparse coding and dictionary update subroutines. The former yields a robust variant of Orthogonal Matching Pursuit while the latter exploits robust Singular Value Decompositions. The result is Correntropy–based dictionary learning. The data–driven nature of the approach combines two appealing features in unsupervised learning—robustness and sparseness—without adding hyperparameters to the framework. Robust recovery of bases in synthetic data and image denoising under impulsive noise confirm the advantages of the proposed techniques.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Aharon, M., Elad, M., Bruckstein, A., et al.: K-SVD: an algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans. Sig. Process. 54(11), 4311 (2006)CrossRef Aharon, M., Elad, M., Bruckstein, A., et al.: K-SVD: an algorithm for designing overcomplete dictionaries for sparse representation. IEEE Trans. Sig. Process. 54(11), 4311 (2006)CrossRef
2.
Zurück zum Zitat Akaike, H.: Information theory and an extension of the maximum likelihood principle. In: Selected Papers of Hirotugu Akaike, pp. 199–213. Springer (1998) Akaike, H.: Information theory and an extension of the maximum likelihood principle. In: Selected Papers of Hirotugu Akaike, pp. 199–213. Springer (1998)
3.
Zurück zum Zitat Andersen, R.: Modern Methods for Robust Regression, vol. 152. Sage (2008) Andersen, R.: Modern Methods for Robust Regression, vol. 152. Sage (2008)
4.
Zurück zum Zitat Barron, A., Rissanen, J., Yu, B.: The minimum description length principle in coding and modeling. IEEE Trans. Inf. Theor. 44(6), 2743–2760 (1998)MathSciNetCrossRef Barron, A., Rissanen, J., Yu, B.: The minimum description length principle in coding and modeling. IEEE Trans. Inf. Theor. 44(6), 2743–2760 (1998)MathSciNetCrossRef
5.
Zurück zum Zitat Candès, E.J., Romberg, J., Tao, T.: Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inf. Theor. 52(2), 489–509 (2006)MathSciNetCrossRef Candès, E.J., Romberg, J., Tao, T.: Robust uncertainty principles: exact signal reconstruction from highly incomplete frequency information. IEEE Trans. Inf. Theor. 52(2), 489–509 (2006)MathSciNetCrossRef
6.
Zurück zum Zitat Donoho, D.L., Johnstone, J.M.: Ideal spatial adaptation by wavelet shrinkage. Biometrika 81(3), 425–455 (1994)MathSciNetCrossRef Donoho, D.L., Johnstone, J.M.: Ideal spatial adaptation by wavelet shrinkage. Biometrika 81(3), 425–455 (1994)MathSciNetCrossRef
7.
Zurück zum Zitat Elad, M., Aharon, M.: Image denoising via sparse and redundant representations over learned dictionaries. IEEE Trans. Image Process. 15(12), 3736–3745 (2006)MathSciNetCrossRef Elad, M., Aharon, M.: Image denoising via sparse and redundant representations over learned dictionaries. IEEE Trans. Image Process. 15(12), 3736–3745 (2006)MathSciNetCrossRef
8.
Zurück zum Zitat He, R., Hu, B.G., Zheng, W.S., Kong, X.W.: Robust principal component analysis based on maximum correntropy criterion. IEEE Trans. Image Process. 20(6), 1485–1494 (2011)MathSciNetCrossRef He, R., Hu, B.G., Zheng, W.S., Kong, X.W.: Robust principal component analysis based on maximum correntropy criterion. IEEE Trans. Image Process. 20(6), 1485–1494 (2011)MathSciNetCrossRef
9.
Zurück zum Zitat Liu, W., Pokharel, P.P., Príncipe, J.C.: Correntropy: properties and applications in non-gaussian signal processing. IEEE Trans. Sig. Process. 55(11), 5286–5298 (2007)MathSciNetCrossRef Liu, W., Pokharel, P.P., Príncipe, J.C.: Correntropy: properties and applications in non-gaussian signal processing. IEEE Trans. Sig. Process. 55(11), 5286–5298 (2007)MathSciNetCrossRef
10.
Zurück zum Zitat Loza, C.A., Principe, J.C.: A robust maximum correntropy criterion for dictionary learning. In: 2016 IEEE 26th International Workshop on Machine Learning for Signal Processing (MLSP), pp. 1–6. IEEE (2016) Loza, C.A., Principe, J.C.: A robust maximum correntropy criterion for dictionary learning. In: 2016 IEEE 26th International Workshop on Machine Learning for Signal Processing (MLSP), pp. 1–6. IEEE (2016)
11.
Zurück zum Zitat Mairal, J., Bach, F., Ponce, J., Sapiro, G., Zisserman, A.: Non-local sparse models for image restoration. In: 2009 IEEE 12th International Conference on Computer Vision, pp. 2272–2279. IEEE (2009) Mairal, J., Bach, F., Ponce, J., Sapiro, G., Zisserman, A.: Non-local sparse models for image restoration. In: 2009 IEEE 12th International Conference on Computer Vision, pp. 2272–2279. IEEE (2009)
12.
Zurück zum Zitat Mairal, J., Elad, M., Sapiro, G.: Sparse representation for color image restoration. IEEE Trans. Image Process. 17(1), 53–69 (2008)MathSciNetCrossRef Mairal, J., Elad, M., Sapiro, G.: Sparse representation for color image restoration. IEEE Trans. Image Process. 17(1), 53–69 (2008)MathSciNetCrossRef
13.
Zurück zum Zitat Mallat, S., Zhang, Z.: Matching pursuit with time-frequency dictionaries. Technical report, Courant Institute of Mathematical Sciences, New York, United States (1993) Mallat, S., Zhang, Z.: Matching pursuit with time-frequency dictionaries. Technical report, Courant Institute of Mathematical Sciences, New York, United States (1993)
14.
Zurück zum Zitat Nikolova, M., Ng, M.K.: Analysis of half-quadratic minimization methods for signal and image recovery. SIAM J. Sci. Comput. 27(3), 937–966 (2005)MathSciNetCrossRef Nikolova, M., Ng, M.K.: Analysis of half-quadratic minimization methods for signal and image recovery. SIAM J. Sci. Comput. 27(3), 937–966 (2005)MathSciNetCrossRef
15.
Zurück zum Zitat Olshausen, B.A., Field, D.J.: Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381(6583), 607 (1996)CrossRef Olshausen, B.A., Field, D.J.: Emergence of simple-cell receptive field properties by learning a sparse code for natural images. Nature 381(6583), 607 (1996)CrossRef
16.
Zurück zum Zitat Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D: Nonlinear Phenom. 60(1–4), 259–268 (1992)MathSciNetCrossRef Rudin, L.I., Osher, S., Fatemi, E.: Nonlinear total variation based noise removal algorithms. Physica D: Nonlinear Phenom. 60(1–4), 259–268 (1992)MathSciNetCrossRef
17.
Zurück zum Zitat Silverman, B.W.: Density Estimation for Statistics and Data Analysis. Routledge (2018) Silverman, B.W.: Density Estimation for Statistics and Data Analysis. Routledge (2018)
18.
Zurück zum Zitat Tibshirani, R.: Regression shrinkage and selection via the lasso. J. Roy. Stat. Soc. Series B (Methodol.) 58(1), 267–288 (1996)MathSciNetMATH Tibshirani, R.: Regression shrinkage and selection via the lasso. J. Roy. Stat. Soc. Series B (Methodol.) 58(1), 267–288 (1996)MathSciNetMATH
19.
Zurück zum Zitat Tropp, J.A., Gilbert, A.C.: Signal recovery from random measurements via orthogonal matching pursuit. IEEE Trans. Inf. Theor. 53(12), 4655–4666 (2007)MathSciNetCrossRef Tropp, J.A., Gilbert, A.C.: Signal recovery from random measurements via orthogonal matching pursuit. IEEE Trans. Inf. Theor. 53(12), 4655–4666 (2007)MathSciNetCrossRef
20.
Zurück zum Zitat Wang, Y., Tang, Y.Y., Li, L.: Correntropy matching pursuit with application to robust digit and face recognition. IEEE Trans. Cybern. 47(6), 1354–1366 (2017)CrossRef Wang, Y., Tang, Y.Y., Li, L.: Correntropy matching pursuit with application to robust digit and face recognition. IEEE Trans. Cybern. 47(6), 1354–1366 (2017)CrossRef
Metadaten
Titel
A Robust Fully Correntropy–Based Sparse Modeling Alternative to Dictionary Learning
verfasst von
Carlos A. Loza
Copyright-Jahr
2020
DOI
https://doi.org/10.1007/978-3-030-33509-0_79