## Swipe to navigate through the chapters of this book

Published in:

2019 | OriginalPaper | Chapter

# Information and Entropy in Physical Systems

Author : Craig S. Lent

Published in:

Publisher:

## Abstract

The Landauer Principle connects the information theoretic notion of entropy to the physics of statistical mechanics. When a physical system performs a logical operation that erases or loses information, without a copy being preserved, it must transfer a minimum amount of heat, $$k_B T \log (2)$$, to the environment. How can there be such a connection between the abstract idea of information and the concrete physical reality of heat? To address this question, we adopt the Jaynes approach of grounding statistical mechanics in the Shannon notion of entropy. Probability is a quantification of incomplete information. Entropy should not be conceived in terms of disorder, but rather as a measure on a probability distribution that characterizes the amount of missing information the distribution represents. The thermodynamic entropy is a special case of the Shannon entropy applied to a physical system in equilibrium with a heat bath so that its average energy is fixed. The thermal probability distribution is obtained by maximizing the Shannon entropy, subject to the physical constraints of the problem. It is then possible to naturally extend this description to include a physical memory device, which must be in a nonequilibrium long-lived metastable state. We can then explicitly demonstrate how the requirement for a fundamental minimum energy dissipation is tied to erasure of an unknown bit. Both classical and quantum cases are considered. We show that the classical thermodynamic entropy is in some situations best matched in quantum mechanics, not by the von Neumann entropy, but by a perhaps less familiar quantity—the quantum entropy of outcomes. The case of free expansion of an ideal quantum gas is examined in this context.
Footnotes
1
Pierre Simon Laplace, A Philosophical Essay on Probabilities, 1814.

2
This is the famous Measurement Problem in quantum mechanics. The term is immediately misleading because prior to the measurement, a quantum system does not in general have an underlying value of the measured result.

3
We will not wade into the subtler issues involved, but refer the reader to Chapter 12 of Jaynes [3]. The quantum treatment in Sect. 6 actually makes the choice of basis explicit, and therefore clarifies the question: “Ignorance with respect to what?”

4
Jaynes [3], p. 634.

5
The factor (1 − λ0) is used instead of λ0 to simplify the form of the result.

6
Myron Trebus’s thermodynamics text for engineers was an early attempt to popularize Jaynes grounding of the field on Shannon information theory.

7
Note that we assume that state indices (a series of 1’s and 0’s specifying each particular state) are chosen in an optimal way, employing a so-called Huffman code, that uses fewer bits to specify more probable states and longer bit sequences for rarer states. The average register length is the average index length weighted by the state probabilities.

8
The weak interaction responsible for the decay of the neutral B meson has been directly shown to violate time reversal symmetry. See J. P. Lees et al., “Observation of Time-Reversal Violation in the B0 Meson System,” Phys. Rev. Lett. 109, 211801 (2012). We will restrict our considerations to systems not involving B or K mesons, or the weak interaction.

9
This sort of engine does not, needless to say, violate the second law of thermodynamics or create a perpetual motion machine of the second kind.

10
“Meaningless” is, of course, a question of context—maybe it was meant as a probe to measure bath fluctuations.

11
A Hilbert space is a complex linear vector space with an inner product that produces a norm on the space. Using this norm, all Cauchy sequences of vectors in the space converge to a vector in the space.

Literature
1.
W. Porod, R. Grondin, D. Ferry, G. Porod, Phys. Rev. Lett. 52, 232 (1984) CrossRef
2.
J.D. Norton, Stud. Hist. Philos. Mod. Phys. 36B(2), 375 (2005) CrossRef
3.
E. Jaynes, Probability Theory: The Logic of Science (Cambridge University Press, Cambridge, 2003) CrossRef
4.
A. Ben-Naim, Modern Thermodynamics (World Scientific, Singapore, 2016) CrossRef
5.
A. Ben-Naim, A Farewell to Entropy: Statistical Mechanics Based on Information (World Scientific, Singapore, 2008) CrossRef
6.
T. Cover, J. Thomas, Elements of Information Theory, 2nd edn. (Wiley, Hoboken, 2006) MATH
7.
M. Trebus, Thermostatics and Thermodynamics (D. Van Nostrand, New York, 1961)
8.
G.M. Jochen Gemmer, M. Michel, Quantum Thermodynamics: Emergence of Thermodynamic Behavior Within Composite Quantum Systems, Lecture Notes in Physics, vol. 784 (Springer, Berlin, 2009) MATH
9.
I. Amlani, A.O. Orlov, G.L. Snider, C.S. Lent, G.H. Bernstein, Appl. Phys. Lett. 72(17), 2179 (1998) CrossRef
10.
J. Christie, R. Forrest, S. Corcelli, N. Wasio, R. Quardokus, R. Brown, S. Kandel, C. Lent, Y. Lu, K. Henderson, Angew. Chem. 127, 15668 (2015) CrossRef
11.
A.O. Orlov, C.S. Lent, C.C. Thorpe, G.P. Boechler, G.L. Snider, Jpn. J. Appl. Phys. 51, 06FE10m (2012)
12.
R.K. Kummamuru, A.O. Orlov, R. Ramasubramaniam, C.S. Lent, G.H. Bernstein, G.L. Snider, IEEE Trans. Electron Devices 50(9), 1906 (2003) CrossRef
13.
L. Szilard, 5, 840 (1929)
14.
15.
M. Giustina, M.A.M. Versteegh, S. Wengerowsky, J. Handsteiner, A. Hochrainer, K. Phelan, F. Steinlechner, J. Kofler, J.A. Larsson, C. Abellán, W. Amaya, V. Pruneri, M.W. Mitchell, J. Beyer, T. Gerrits, A.E. Lita, L.K. Shalm, S.W. Nam, T. Scheidl, R. Ursin, B. Wittmann, A. Zeilinger, Phys. Rev. Lett. 115, 250401 (2015). https://​doi.​org/​10.​1103/​PhysRevLett.​115.​250401. http://​link.​aps.​org/​doi/​10.​1103/​PhysRevLett.​115.​250401
16.
J. Handsteiner, A.S. Friedman, D. Rauch, J. Gallicchio, B. Liu, H. Hosp, J. Kofler, D. Bricher, M. Fink, C. Leung, A. Mark, H.T. Nguyen, I. Sanders, F. Steinlechner, R. Ursin, S. Wengerowsky, A.H. Guth, D.I. Kaiser, T. Scheidl, A. Zeilinger, Phys. Rev. Lett. 118, 060401 (2017). https://​doi.​org/​10.​1103/​PhysRevLett.​118.​060401. https://​link.​aps.​org/​doi/​10.​1103/​PhysRevLett.​118.​060401
17.
B. Hensen, H. Bernien, A.E. Dreau, A. Reiserer, N. Kalb, M.S. Blok, J. Ruitenberg, R.F.L. Vermeulen, R.N. Schouten, C. Abellan, W. Amaya, V. Pruneri, M.W. Mitchell, M. Markham, D.J. Twitchen, D. Elkouss, S. Wehner, T.H. Taminiau, R. Hanson, Nature 526, 682 (2015). https://​doi.​org/​10.​1038/​nature15759 CrossRef
18.
L.K. Shalm, E. Meyer-Scott, B.G. Christensen, P. Bierhorst, M.A. Wayne, M.J. Stevens, T. Gerrits, S. Glancy, D.R. Hamel, M.S. Allman, K.J. Coakley, S.D. Dyer, C. Hodge, A.E. Lita, V.B. Verma, C. Lambrocco, E. Tortorici, A.L. Migdall, Y. Zhang, D.R. Kumor, W.H. Farr, F. Marsili, M.D. Shaw, J.A. Stern, C. Abellán, W. Amaya, V. Pruneri, T. Jennewein, M.W. Mitchell, P.G. Kwiat, J.C. Bienfang, R.P. Mirin, E. Knill, S.W. Nam, Phys. Rev. Lett. 115, 250402 (2015). https://​doi.​org/​10.​1103/​PhysRevLett.​115.​250402. https://​link.​aps.​org/​doi/​10.​1103/​PhysRevLett.​115.​250402
19.
E. Blair, C. Lent, J. Appl. Phys. 113, 124302 (2013) CrossRef
20.
R.P. Feynman, Statistical Mechanics: A Set of Lectures (W.A. Benjamin, Reading, 1972) MATH
21.
J.P. Timler, Energy dissipation and power gain in quantum-dot cellular automata, Ph.D. thesis, University of Notre Dame, 2003
22.
E.P. Blair, S.A. Corcelli, C.S. Lent, J. Chem. Phys. 145(1), 014307 (2016) CrossRef