Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Published in:
Three-Dimensional Model of Signal Processing in the Presynaptic Bouton of the Neuron Read chapter Read first chapter

2018 | OriginalPaper | Chapter

Direct Incorporation of \(L_1\)-Regularization into Generalized Matrix Learning Vector Quantization

Authors : Falko Lischke, Thomas Neumann, Sven Hellbach, Thomas Villmann, Hans-Joachim Böhme

Published in: Artificial Intelligence and Soft Computing

Publisher: Springer International Publishing

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

Frequently, high-dimensional features are used to represent data to be classified. This paper proposes a new approach to learn interpretable classification models from such high-dimensional data representation. To this end, we extend a popular prototype-based classification algorithm, the matrix learning vector quantization, to incorporate an enhanced feature selection objective via \(L_1\)-regularization. In contrast to previous work, we propose a framework that directly optimizes this objective using the alternating direction method of multipliers (ADMM) and manifold optimization. We evaluate our method on synthetic data and on real data for speech-based emotion recognition. Particularly, we show that our method achieves state-of-the-art results on the Berlin Database of Emotional speech and show its abilities to select relevant dimensions from the eGeMAPS set of audio features.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now
previous chapter Multi-level Aggregation in Face Recognition
next chapter Classifiers for Matrix Normal Images: Derivation and Testing
Literature
1.
Absil, P.A., Mahony, R., Sepulchre, R.: Optimization Algorithms on Matrix Manifolds. Princeton University Press, Princeton (2009)MATH
2.
Ali, H., Hariharan, M., Yaacob, S., Adom, A.H.: Facial emotion recognition using empirical mode decomposition. Expert Syst. Appl. 42(3), 1261–1277 (2015)CrossRef
3.
Beck, A., Teboulle, M.: A fast iterative shrinkage-thresholding algorithm for linear inverse problems. SIAM J. Imaging Sci. 2(1), 183–202 (2009)MathSciNetCrossRef
4.
Bi, J., Bennett, K., Embrechts, M., Breneman, C., Song, M.: Dimensionality reduction via sparse support vector machines. JMLR 3(Mar), 1229–1243 (2003)MATH
5.
Biehl, M., Hammer, B., Villmann, T.: Prototype-based models in machine learning. Wiley Interdisc. Rev.: Cogn. Sci. 7(2), 92–111 (2016)CrossRef
6.
Biehl, M., Hammer, B., Schleif, F.M., Schneider, P., Villmann, T.: Stationarity of matrix relevance learning vector quantization. Mach. Learn. Rep. 3, 1–17 (2009)
7.
Bishop, C.M.: Pattern Recognition and Machine Learning (Information Science and Statistics). Springer, New York (2006)MATH
8.
Bojer, T., Hammer, B., Schunk, D., Von Toschanowitz, K.: Relevance determination in learning vector quantization. In: Proceedings of ESANN (2001)
9.
Boyd, S., Parikh, N., Chu, E., Peleato, B., Eckstein, J.: Distributed optimization and statistical learning via the alternating direction method of multipliers. Found. Trends Mach. Learn. 3(1), 1–122 (2011)CrossRef
10.
Burkhardt, F., Paeschke, A., Rolfes, M., Sendlmeier, W., Weiss, B.: A database of German emotional speech. In: Interspeech, vol. 5, pp. 1517–1520 (2005)
11.
Chandrashekar, G., Sahin, F.: A survey on feature selection methods. Comput. Electr. Eng. 40(1), 16–28 (2014)CrossRef
12.
13.
Donoho, D.L.: For most large underdetermined systems of linear equations the minimal \(\ell \)1-norm solution is also the sparsest solution. CPAMA 59(6), 797–829 (2006)MathSciNetMATH
14.
Eyben, F., Scherer, K.R., Schuller, B.W., Sundberg, J., André, E., Busso, C., Devillers, L.Y., Epps, J., Laukka, P., Narayanan, S.S., Truong, K.P.: The Geneva minimalistic acoustic parameter set (GeMAPS) for voice research and affective computing. IEEE TAC 7(2), 190–202 (2016)
15.
Eyben, F., Weninger, F., Gross, F., Schuller, B.: Recent developments in openSMILE, the munich open-source multimedia feature extractor. In: Proceedings of the 21st ACM, pp. 835–838. ACM (2013)
16.
Hammer, B., Villmann, T.: Generalized relevance learning vector quantization. Neural Netw. 15(8), 1059–1068 (2002)CrossRef
17.
Hsu, C.W., Lin, C.J.: A comparison of methods for multiclass support vector machines. IEEE TNN 13(2), 415–425 (2002)
18.
Kaden, M., Lange, M., Nebel, D., Riedel, M., Geweniger, T., Villmann, T.: Aspects in classification learning - review of recent developments in learning vector quantization. Found. Comput. Decis. Sci. 39(2), 79–105 (2014)MathSciNetCrossRef
19.
Kanth, N.R., Saraswathi, S.: Efficient speech emotion recognition using binary support vector machines multiclass SVM. In: 2015 IEEE ICCIC, December 2015
20.
Kim, J., Truong, K.P., Englebienne, G., Evers, V.: Learning spectro-temporal features with 3D CNNs for speech emotion recognition. arXiv preprint arXiv:​1708.​05071 (2017)
21.
22.
Korkmaz, O.E., Atasoy, A.: Emotion recognition from speech signal using mel-frequency cepstral coefficients. In: 2015 9th ELECO, pp. 1254–1257, November 2015
23.
Lee, J., Tashev, I.: High-level feature representation using recurrent neural network for speech emotion recognition. In: Interspeech 2015. ISCA, September 2015
24.
Mao, Q., Dong, M., Huang, Z., Zhan, Y.: Learning salient features for speech emotion recognition using convolutional neural networks. IEEE Trans. Multimedia 16(8), 2203–2213 (2014)CrossRef
25.
Murty, K.G., Kabadi, S.N.: Some NP-complete problems in quadratic and nonlinear programming. Math. Program. 39(2), 117–129 (1987)MathSciNetCrossRef
26.
Ng, A.Y.: Feature selection, L1 vs. L2 regularization, and rotational invariance. In: Proceedings of the 21th ICML, ICML 2004, p. 78. ACM, New York (2004)
27.
Obozinski, G., Taskar, B., Jordan, M.: Multi-task feature selection. Statistics Department, UC Berkeley, Technical report 2 (2006)
28.
Ofodile, I., Kulkarni, K., Corneanu, C.A., Escalera, S., Baro, X., Hyniewska, S., Allik, J., Anbarjafari, G.: Automatic recognition of deceptive facial expressions of emotion (2017)
29.
Palo, H., Mohanty, M., Chandra, M.: Efficient feature combination techniques for emotional speech classification. IJST 19(1), 135–150 (2016)
30.
Riedel, M., Rossi, F., Kästner, M., Villmann, T.: Regularization in relevance learning vector quantization using \(l_1\)-norms. In: Verleysen, M. (ed.) Proceedings of ESANN 2013, pp. 17–22 (2013)
31.
32.
Sato, A., Yamada, K.: Generalized learning vector quantization. In: Advances in Neural Information Processing Systems, pp. 423–429 (1996)
33.
Schneider, P., Biehl, M., Hammer, B.: Adaptive relevance matrices in learning vector quantization. Neural Comput. 21(12), 3532–3561 (2009)MathSciNetCrossRef
34.
Schneider, P., Bunte, K., Stiekema, H., Hammer, B., Villmann, T., Biehl, M.: Regularization in matrix relevance learning. IEEE Trans. Neural Netw. 21(5), 831–840 (2010)CrossRef
35.
Schuller, B., Steidl, S., Batliner, A.: The INTERSPEECH 2009 emotion challenge. In: 10th Annual Conference of the ISCA (2009)
36.
Schuller, B., Steidl, S., Batliner, A., Schiel, F., Krajewski, J.: The INTERSPEECH 2011 speaker state challenge. In: 12th Annual Conference of the ISCA (2011)
37.
Schuller, B., Steidl, S., Batliner, A., et al.: The INTERSPEECH 2017 computational paralinguistics challenge: addressee, cold and snoring. In: ComParE, Interspeech 2017, pp. 3442–3446 (2017)
38.
Sinith, M.S., Aswathi, E., Deepa, T.M., Shameema, C.P., Rajan, S.: Emotion recognition from audio signals using support vector machine. In: 2015 IEEE RAICS, pp. 139–144, December 2015
39.
Tibshirani, R.: Regression shrinkage and selection via the lasso. J. Roy. Stat. Soc. Ser. B (Methodol.) 267–288 (1996)MathSciNetMATH
40.
Townsend, J., Koep, N., Weichwald, S.: Pymanopt: a python toolbox for optimization on manifolds using automatic differentiation. J. Mach. Learn. Res. 17(137), 1–5 (2016)MathSciNetMATH
41.
Villmann, T., Bohnsack, A., Kaden, M.: Can learning vector quantization be an alternative to SVM and deep learning? JAISCR 7(1), 65–81 (2017)
42.
Wang, K., An, N., Li, B.N., Zhang, Y., Li, L.: Speech emotion recognition using fourier parameters. IEEE TAC 6(1), 69–75 (2015)
43.
Wen, G., Li, H., Huang, J., Li, D., Xun, E.: Random deep belief networks for recognizing emotions from speech signals. Comput. Intell. Neurosci. 2017 (2017)CrossRef
44.
Zhang, Y., Zhang, L., Hossain, M.A.: Adaptive 3D facial action intensity estimation and emotion recognition. Expert Syst. Appl. 42(3), 1446–1464 (2015)CrossRef
45.
Zhu, J., Rosset, S., Tibshirani, R., Hastie, T.J.: 1-norm support vector machines. In: Advances in Neural Information Processing Systems, pp. 49–56 (2004)
Metadata
Title
Direct Incorporation of -Regularization into Generalized Matrix Learning Vector Quantization
Authors
Falko Lischke
Thomas Neumann
Sven Hellbach
Thomas Villmann
Hans-Joachim Böhme
Copyright Year
2018
Book
Artificial Intelligence and Soft Computing

Print ISBN: 978-3-319-91252-3

Electronic ISBN: 978-3-319-91253-0

Copyright Year: 2018

DOI
https://doi.org/10.1007/978-3-319-91253-0_61

Premium Partner

    Image Credits