Skip to main content

2010 | OriginalPaper | Buchkapitel

Entropy and Cross Entropy: Characterizations and Applications

verfasst von : C. R. Rao

Erschienen in: The Legacy of Alladi Ramakrishnan in the Mathematical Sciences

Verlag: Springer New York

Aktivieren Sie unsere intelligente Suche um passende Fachinhalte oder Patente zu finden.

search-config
loading …

Summary

The paper provides an axiomatic setup for an entropy function as a measure of diversity. A general definition of cross entropy is given and its use in solving a variety of stochastic and nonstochastic optimization problems is mentioned. A method of deriving a cross entropy function associated with a given entropy function is given.

Sie haben noch keine Lizenz? Dann Informieren Sie sich jetzt über unsere Produkte:

Springer Professional "Wirtschaft+Technik"

Online-Abonnement

Mit Springer Professional "Wirtschaft+Technik" erhalten Sie Zugriff auf:

  • über 102.000 Bücher
  • über 537 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Maschinenbau + Werkstoffe
  • Versicherung + Risiko

Jetzt Wissensvorsprung sichern!

Springer Professional "Technik"

Online-Abonnement

Mit Springer Professional "Technik" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 390 Zeitschriften

aus folgenden Fachgebieten:

  • Automobil + Motoren
  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Elektrotechnik + Elektronik
  • Energie + Nachhaltigkeit
  • Maschinenbau + Werkstoffe




 

Jetzt Wissensvorsprung sichern!

Springer Professional "Wirtschaft"

Online-Abonnement

Mit Springer Professional "Wirtschaft" erhalten Sie Zugriff auf:

  • über 67.000 Bücher
  • über 340 Zeitschriften

aus folgenden Fachgebieten:

  • Bauwesen + Immobilien
  • Business IT + Informatik
  • Finance + Banking
  • Management + Führung
  • Marketing + Vertrieb
  • Versicherung + Risiko




Jetzt Wissensvorsprung sichern!

Literatur
1.
Zurück zum Zitat Celesti, F., Dambreville, F., and Le Ladre, J.P. (2006). Optimal path planning using cross-entropy method, (fusion.carthel.com/technical − program/abstracts/303.htm). Celesti, F., Dambreville, F., and Le Ladre, J.P. (2006). Optimal path planning using cross-entropy method, (fusion.carthel.com/technical program/abstracts/303.htm).
2.
Zurück zum Zitat de-Boer, P.T., Kroese, D.P., Mannor, S., and Rubinstein, R.Y. (2005). A tutorial on cross-entropy method, Annals of Operations Research, 134, 19–67. de-Boer, P.T., Kroese, D.P., Mannor, S., and Rubinstein, R.Y. (2005). A tutorial on cross-entropy method, Annals of Operations Research, 134, 19–67.
3.
Zurück zum Zitat Habermann, S.J. (1982). Analysis of dispersion of multinomial probabilities, J. Amer. Statist. Ass., 77, 568–580.CrossRef Habermann, S.J. (1982). Analysis of dispersion of multinomial probabilities, J. Amer. Statist. Ass., 77, 568–580.CrossRef
4.
Zurück zum Zitat Havrada, J. and Charvát, F. (1967). Quantification method in classification processes: Concept of structural α-entropy, Kybernetika, 30, 30–35. Havrada, J. and Charvát, F. (1967). Quantification method in classification processes: Concept of structural α-entropy, Kybernetika, 30, 30–35.
5.
Zurück zum Zitat Homem-de-Mello, T. (2007). A study of the cross-entropy method for rare-event probability estimation, INFORMS J. on Computing, 19, 381–394.MathSciNetMATHCrossRef Homem-de-Mello, T. (2007). A study of the cross-entropy method for rare-event probability estimation, INFORMS J. on Computing, 19, 381–394.MathSciNetMATHCrossRef
7.
Zurück zum Zitat Joos, G. (1951). Theoretical Physics, Haffner, New York.MATH Joos, G. (1951). Theoretical Physics, Haffner, New York.MATH
8.
Zurück zum Zitat Kroese, D.T., Rubinstein, R.Y., and Taimre, T. (2007). Application of cross-entropy method to clustering, J. Glob Optimum, 37, 137–157.MATHCrossRef Kroese, D.T., Rubinstein, R.Y., and Taimre, T. (2007). Application of cross-entropy method to clustering, J. Glob Optimum, 37, 137–157.MATHCrossRef
10.
Zurück zum Zitat Lewontin, R.C. (1972). The apportionment of human diversity, Evolutionary Biology, 6, 381–398.CrossRef Lewontin, R.C. (1972). The apportionment of human diversity, Evolutionary Biology, 6, 381–398.CrossRef
11.
Zurück zum Zitat Mannor, S., Peleg, D., and Rubinstein, R. (2005). Proc. 22nd International Conference on Machine Learning. Mannor, S., Peleg, D., and Rubinstein, R. (2005). Proc. 22nd International Conference on Machine Learning.
12.
13.
Zurück zum Zitat Pielou, E.C. (1975). Ecological Diversity, Wiley, New York. Pielou, E.C. (1975). Ecological Diversity, Wiley, New York.
14.
Zurück zum Zitat Rao, C.R. (1973). Linear Statistical Inference and its Applications(Second edition), Wiley, New York. Rao, C.R. (1973). Linear Statistical Inference and its Applications(Second edition), Wiley, New York.
15.
Zurück zum Zitat Rao, C.R. (1982a). Diversity and dissimilarity coefficients: A unified approach, Theor. Popl. Bio, 21, 24–43.MATHCrossRef Rao, C.R. (1982a). Diversity and dissimilarity coefficients: A unified approach, Theor. Popl. Bio, 21, 24–43.MATHCrossRef
16.
Zurück zum Zitat Rao, C.R. (1982b). Diversity, its meausrement, decomposition, apportionment and analysis, Sankhya, 44, 1–21.MATH Rao, C.R. (1982b). Diversity, its meausrement, decomposition, apportionment and analysis, Sankhya, 44, 1–21.MATH
17.
Zurück zum Zitat Rao, C.R. (1982c). Gini-Simpson index of diversity: A characterization, generalization and applications, Utilitas Mathematics, 21, 273–282. Rao, C.R. (1982c). Gini-Simpson index of diversity: A characterization, generalization and applications, Utilitas Mathematics, 21, 273–282.
18.
Zurück zum Zitat Rao, C.R. (1984). Convexity properties of entropy functions and analysis of diversity, In Inequalities in Statistics and Probability, Ed. Y.L. Tong, IMS Lecture Notes, 5, 68–77. Rao, C.R. (1984). Convexity properties of entropy functions and analysis of diversity, In Inequalities in Statistics and Probability, Ed. Y.L. Tong, IMS Lecture Notes, 5, 68–77.
19.
Zurück zum Zitat Rao, C.R. (1986). Rao’s axiomatization of diversity measures, In Encyclopedia of Statistical Sciences, Vol 7, Wiley, New York, 614–617. Rao, C.R. (1986). Rao’s axiomatization of diversity measures, In Encyclopedia of Statistical Sciences, Vol 7, Wiley, New York, 614–617.
20.
Zurück zum Zitat Rao, C.R. and Nayak, T. (1985). Cross-entropy, dissimilarity measures and quadratic entropy, IEEE Transactions of Information Theory, 31, 589–593.MathSciNetMATHCrossRef Rao, C.R. and Nayak, T. (1985). Cross-entropy, dissimilarity measures and quadratic entropy, IEEE Transactions of Information Theory, 31, 589–593.MathSciNetMATHCrossRef
21.
Zurück zum Zitat Renyi, A. (1961). On measures of information and entropy, Proc. 4-th Berkeley Symposium on Math. Stat. and Prob., 547–561. Renyi, A. (1961). On measures of information and entropy, Proc. 4-th Berkeley Symposium on Math. Stat. and Prob., 547–561.
22.
Zurück zum Zitat Rubinstein, R. and Kroese, D.P. (2004). The cross entropy method: A unified approach to combinational optimization, Monte Carlo simulation and machine learning, Information Science & Statistics, Springer. Rubinstein, R. and Kroese, D.P. (2004). The cross entropy method: A unified approach to combinational optimization, Monte Carlo simulation and machine learning, Information Science & Statistics, Springer.
23.
24.
Zurück zum Zitat Shannon, C.E. (1948). A mathematical theory of communication, Bell System Technical Journal, 27, 379–423, 623–656.MathSciNet Shannon, C.E. (1948). A mathematical theory of communication, Bell System Technical Journal, 27, 379–423, 623–656.MathSciNet
26.
Zurück zum Zitat Zoltan, B.-D. (2008). Rao’s quadratic entropy as a measure of functional diversity based on multiple traits, J.Vegetation Science, 16, 533–540. Zoltan, B.-D. (2008). Rao’s quadratic entropy as a measure of functional diversity based on multiple traits, J.Vegetation Science, 16, 533–540.
Metadaten
Titel
Entropy and Cross Entropy: Characterizations and Applications
verfasst von
C. R. Rao
Copyright-Jahr
2010
Verlag
Springer New York
DOI
https://doi.org/10.1007/978-1-4419-6263-8_22