Advertisement
Published in:
2022 | OriginalPaper | Chapter
10. Information Entropy
Author : Peter Müller
Published in: Handbook of Dynamics and Probability
Publisher: Springer International Publishing
Activate our intelligent search to find suitable subject content or patents.
Select sections of text to find matching patents with Artificial Intelligence. powered by
Select sections of text to find additional relevant content using AI-assisted search. powered by
Abstract
The concepts of information content and entropy grew out of attempts to quantify randomness. Early ideas go back to Boltzmann, but the definitive step was taken by Claude Shannon (1948) in his treatise “A Mathematical Theory of Communication” which lay the groundwork for what is known today as information theory. More recent references are Mackey (2003) and Cover and Thomas (2005). Information theory studies sequences of letters or symbols from a finite alphabet. It presumes that a source produces these letters with a given probability distribution and then studies how information is degraded when storing, processing, or transmitting data.