Skip to main content
Disciplines
Automotive Business IT + Informatics Construction + Real Estate Electrical Engineering + Electronics Energy + Sustainability Insurance + Risk Finance + Banking Management + Leadership Marketing + Sales Mechanical Engineering + Materials
Events
DE
EN
Books
Journals
Topic Page
Marketing
Start single access now
Access for companies
EXTENDED SEARCH
Log in
Top
Published in:
Information Theory, Machine Learning, and Reproducing Kernel Hilbert Spaces Read chapter Read first chapter

2010 | OriginalPaper | Chapter

6. Classification with EEC, Divergence Measures, and Error Bounds

Authors : Deniz Erdogmus, Dongxin Xu, Kenneth Hild II

Published in: Information Theoretic Learning

Publisher: Springer New York

Log in

Activate our intelligent search to find suitable subject content or patents.

search-config
loading …

Abstract

The previous chapters provided extensive coverage of the error entropy criterion (EEC) especially in regard to minimization of the error entropy (MEE) for linear and nonlinear filtering (or regression) applications. However, the spectrum of engineering applications of adaptive systems is much broader than filtering or regression. Even looking at the subclass of supervised applications we have yet to deal with classification, which is an important application area for learning technologies. All of the practical ingredients are here to extend EEC to classification inasmuch as Chapter 5 covered the integration of EEC with the backpropagation algorithm (MEE-BP). Hence we have all the tools needed to train classifiers with MEE. We show that indeed this is the case and that the classifiers trained with MEE have performances normally better than MSE-trained classifiers. However, there are still no mathematical foundations to ascertain under what conditions EEC is optimal for classification, and further work is necessary.

Dont have a licence yet? Then find out more about our products and how to get one now:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

inform now

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

inform now
previous chapter Nonlinear Adaptive Filtering with MEE, MCC, and Applications
next chapter Clustering with ITL Principles
Literature
1.
Aczél J., Daróczy Z., On measures of information and their characterizations, Mathematics in Science and Engineering, vol. 115, Academic Press, New York, 1975.
2.
Ahmad I., Lin P., A nonparametric estimation of the entropy for absolutely continuous distributions, IEEE Trans. on Inf. Theor., 22:372–375, 1976.CrossRefMATHMathSciNet
21.
Battiti R., Using mutual information for selecting features in supervised neural net learning, IEEE Trans. Neural Netw., 5(4):537–550, July 1994.CrossRef
37.
Biem A., Katagiri S., Juang B., Pattern recognition using discriminative feature extraction, IEEE Trans. Signal Process., 45(2):500–504, Feb. 1997.CrossRef
38.
Bishop C., Neural Networks for Pattern Recognition, Clarendon Press, Oxford, 1995.
65.
68.
Deco G., Obradovic D., An Information-Theoretic Approach to Neural Computing, Springer, New York, 1996.CrossRefMATH
80.
Duda R., Hart P., Stork D., Pattern Classification and Scene Analysis. John Wiley & Sons, New York, 2nd edition, 2001.
86.
Erdogmus D., Information theoretic learning: Renyi’s entropy and its applications to adaptive systems training, Ph.D. Dissertation, University of Florida, Gainesville, 2002.
92.
Erdogmus D., Principe J., Lower and upper bounds for misclassification probability based on Renyi’s information, J. VLSI Signal Process. Syst., 37(2/3):305–317, 2004.CrossRefMATH
97.
Fano R., Transmission of Information: A Statistical Theory of Communications, MIT Press, New York, 1961.
98.
101.
Fisher R., The use of multiple measurements in taxonomic problems, Ann. Eugenics 7; 170–188, Wiley, New York, 1950.
108.
Fukunaga K., An Introduction to Statistical Pattern Recognition, Academic Press, New York, 1972
132.
145.
Hertz J., Krogh A., and Palmer R., Introduction to the Theory of Neural Computation, Addison Wesley, Readings, MA, 1991.
149.
Hild II K., Blind source separation of convolutive mixtures using Renyi’s divergence, University of Florida, Gainesville, Fall 2003.
151.
Hild II K., Erdogmus D., Torkkola K., and Principe J., Feature extraction using information-theoretic learning, IEEE Trans. Pat. Anal. Mach. Intell. 28(9):1385–1392, 2006CrossRef
195.
LeCun Y., Bottou L., Bengio Y., Haffner P., Gradient-based learning applied to document recognition, Proc. IEEE, 86(11):2278–2324, Nov. 1998.CrossRef
223.
Morejon R., An information theoretic approach to sonar automatic target recognition, Ph.D. dissertation, University of Florida, Spring 2003
253.
Principe J., Euliano N., Lefebvre C., Neural Systems: Fundamentals through Simulations, CD-ROM textbook, John Wiley, New York, 2000.
254.
Principe J., Xu D., Zhao Q., Fisher J. Learning from examples with information theoretic criteria, VLSI Signal Process. Syst., 26:61–77, 2001.CrossRef
268.
Ripley B., Pattern Recognition and Neural Networks, Cambridge University Press, New York, 1996CrossRefMATH
283.
Santos J., Alexandre L., Sa J., The error entropy minimization algorithm for neural network classification, in A. Lofti (Ed.), Int Conf. Recent Advances in Soft Computing, pp. 92–97, 2004.
298.
Silva L., Felgueiras C., Alexandre L., Sa J., Error entropy in classification problems: a univariate data analysis, Neural comput., 18(9):2036–2061, 2006.CrossRefMATHMathSciNet
299.
Silva L., Neural networks with error density risk functionals for data classification, Ph.D. Thesis, Faculdade de Engenharia, University of Porto, 2008.
309.
318.
Torkkola K., Visualizing class structure in data using mutual information, Proceedings of NNSP X, pp. 376–385, Sydney, Australia, 2000
323.
324.
Velten V., Ross T., Mossing J., Worrell S., Bryant M., Standard SAR ATR evaluation experiments using the MSTAR public release data set, research report, Wright State University, 1998.
334.
Wittner B. Denker J., Strategies for teaching layered networks classification tasks, in Neural Inf. Proc. Syst. (Ed Anderson), 850–859, Ame. Inst. Phys. 1987.
340.
Xu D., Energy, Entropy and Information Potential for Neural Computation, PhD Dissertation, University of Florida, Gainesville, 1999
349.
Zhao Q., Principe J., Brennan V., Xu D., Wang Z., Synthetic aperture radar automatic target recognition with three strategies of learning and representation, Opt. Eng., 39(5):1230–1244, 2000.CrossRef
Metadata
Title
Classification with EEC, Divergence Measures, and Error Bounds
Authors
Deniz Erdogmus
Dongxin Xu
Kenneth Hild II
Copyright Year
2010
Publisher
Springer New York
Book
Information Theoretic Learning

Print ISBN: 978-1-4419-1569-6

Electronic ISBN: 978-1-4419-1570-2

Copyright Year: 2010

DOI
https://doi.org/10.1007/978-1-4419-1570-2_6

Premium Partner

    Image Credits