Wonders without number: the information economy of data and its subjects

I refer here to information in the engineering sense introduced by Claude Shannon to describe the relationship between signal and noise, quantifiable in bits to measure meaningful order as opposed to entropy. Information in this sense is a physical quantity, directly related to mass, energy and space in the work of Heisenberg, Hawking and others, although these relations are not yet widely taught in school curricula. The understanding of information as surprise, relative to the human observer, was understood from the outset and explained in Shannon and Weaver’s Mathematical Theory of Communication (1949). In everyday use of computers, the quantifiable surprising-ness of information can be contrasted with redundancy, and relates to factors such as file compression (files that are more redundant and less surprising contain less information so can be compressed), and predictive text (more surprising text is harder to predict, and can only be offered by larger and more expensive language models). Information as surprise can be related to Bayes theorem, as the amount of information that is gained from an observation, by comparison to the observer’s prior knowledge. I have discussed these issues at greater length, for audiences in the humanities and social sciences, in Objective Functions: (In)humanity and Inequity in Artificial Intelligence. HAU: Journal of Ethnographic Theory 9(1) (2019), 137–146 and What does digital content mean? Umberto Eco and the Open Work. In J. Bardzell, S. Bardzell, M. Blythe (Eds). Critical Theory and Interaction Design, MIT Press (2018), pp. 167–185. A comprehensive mathematical treatment, with more detailed explanations of all the aspects alluded to in this footnote, can be found in David MacKay’s definitive text Information theory, inference and learning algorithms. Cambridge University Press (2003).