Laws of Information Which Govern Systems

Information theory was created for the purpose of studying the communication of messages from one point to another, and since its appearance,14 its focus has remained on the question, “how can the constraint between the two variables X (message sent) and Y (message received) be measured and maximized”? Although the theory was generalized to N dimensions,10,2 and its relation to the analysis of variance noted,9 not much use seems to have been made of the result, perhaps in part because the descriptors “N-dimensional Information Theory” or “Uncertainty Analysis” did not adequately represent what can actually be seen as the analysis of constraints in multivariable systems. In any statistically-analyzable system of several variables interacting in a lively way, some variables (or sets of them) exert effects on others. These effects are reflected statistically as non-independence of the variables involved, and it is this deviation from independence which we indicate by the term “constraint.” We prefer this term to the term “dependence” because the latter suggests dependence of XonY while the former is neutral as to direction. To the extent that the variables are not independent, they are “in communication” with one another, and information theory can be used to analyze the non-independence. In addition, the fluctuation of values taken by any variable can be viewed as a message it sends, a flow of information about itself to all other parts of the system which are “listening.” The view of systems as networks of information transfer leads to quantitative conclusions about system behavior and structure which are somewhat novel and of wide applicability.