Echoes of myth and magic in the language of Artificial Intelligence

For a highly poetic rendering of our all too human tendency to liken the mind to anything but itself, including mirrors, consider the following passage by George Eliot, that crown jewel of psychological belles lettres: “It is astonishing what a different result one gets by changing the metaphor! Once call the brain an intellectual stomach, and one’s ingenious conception of the classics and geometry as ploughs and harrows seems to settle nothing. But then, it is open to someone else to follow great authorities and call the mind a sheet of white paper or a mirror, in which case one’s knowledge of the digestive process becomes quite irrelevant. It was doubtless an ingenious idea to call the camel the ship of the desert, but it would hardly lead one far in training that useful beast. O Aristotle! if you had the advantage of being “the freshest modern” instead of the greatest ancient, would you not have mingled your praise of metaphorical speech as a sign of high intelligence, with a lamentation that intelligence so rarely shows itself in speech without metaphor,—that we can so seldom declare what a thing is, except by saying it is something else?” (Eliot 1997, p. 125). For an insightful in-depth treatment of the theoretical consequences of modeling the mind as a computer see Hurtado (2017).

The poet T.S. Eliot, a friend of his youth, once described him (in a private letter) as “a great wonderful fat toad bloated with wisdom.” (Eliot 2011, p. 108).

Butler’s closing remarks in the same piece (though it is hard to discern whether they be not at least partially tongue-in-cheek) radiate such passionate neo-luddite appeal that they might well have inspired Frank Herbert (1965), one of science-fiction’s most dearly cherished authors, in his masterpiece of geopolitical and philosophical intrigue, Dune, to give the name ‘Butlerian Jihad’ to a crusade that led to a galaxy-wide ban on thinking machines: “Day by day, however, the machines are gaining ground upon us; day by day we are becoming more subservient to them; more men are daily bound down as slaves to tend them, more men are daily devoting the energies of their whole lives to the development of mechanical life. The upshot is simply a question of time, but that the time will come when the machines will hold the real supremacy over the world and its inhabitants is what no person of a truly philosophic mind can for a moment question. Our opinion is that war to the death should be instantly proclaimed against them. Every machine of every sort should be destroyed by the well-wisher of his species. Let there be no exceptions made, no quarter shown; let us at once go back to the primeval condition of the race. If it be urged that this is impossible under the present condition of human affairs, this at once proves that the mischief is already done, that our servitude has commenced in good earnest, that we have raised a race of beings whom it is beyond our power to destroy, and that we are not only enslaved but are absolutely acquiescent in our bondage.” (Butler 1863, ¶ 7)

Just as in Wiener’s, in the following passage from William James we see how the single-mindedness of machines can coexist with their endowment with minds as a cause for concern: “A machine in working order functions fatally in one way. Our consciousness calls this the right way. Take out a valve, throw a wheel out of gear or bend a pivot, and it becomes a different machine, functioning just as fatally in another way which we call the wrong way. But the machine itself knows nothing of wrong or right: matter has no ideals to pursue. A locomotive will carry its train through an open drawbridge as cheerfully as to any other destination.” (James 1879, ¶ 37)

In a symmetrical way, many qualitative researchers have, for similar reasons, adopted the techniques of their quantitative colleagues. See Musa et al. (2015).

It bears mentioning that in a volume put forth by Edge Magazine, attempting to capture the thoughts of nearly two hundred scholars and thinkers on the topic of machines that think, Freeman Dyson offers the shortest response. After declaring his general skepticism that such machines will ever come to exist, he simply adds: “If I am wrong, as I often am, any thoughts I might have about the question are irrelevant. If I am right, then the whole question is irrelevant.” (Dyson 2015, p. 47)

Although there are some differences in flavour and shading between the terms ‘extropianism’ and ‘transhumanism’ (as well as within the use of the term ‘transhumanism’ itself on the part of different writers) for the purposes of this essay we will use them interchangeably.

In addition to the socialist antecedent, Burkhead (1997, ¶ 8) offers another biblical forebear to this grand scheme: “The vision of a transhuman condition goes all the way back to Isaiah. Never again will there be in it [the new Jerusalem] an infant who lives but a few days, or an old man who does not live out his years; he who dies at a hundred will be thought a mere youth; he who fails to reach a hundred will be considered accursed.”

It must be clarified, however, that despite the existence of certain foundational texts and certain prominent figures and institutions that act as attractors, there is no real unified organization that would encompass all of those that would identify as transhumanists. Speaking of AI makers, AI researchers and, for that matter, even transhumanists, as though they were one single unified front in terms of belief and purpose is a misleading overgeneralization. A cursory perusal of the individual writings of key figures will show just how manifold the viewpoints they hold are.

Ever the masterful salesman, Kurzweil opens the article on his law with: “You will get $40 trillion just by reading this essay and understanding what it says” (2001, ¶ 2). Lest my own readers should abandon this paper and instantly flock there in pursuit of so tasty a reward, I must add, malgré moi, the spoiler that by the end of the piece he explains that: “The English word ‘you’ can be singular or plural. I meant it in the sense of ‘all of you’” (2001, ¶ 268).

Transhumanism critic HP LaLancette (2007) takes this form of reasoning to its paroxysmic logical conclusion, pointing out that the very same argument can also be used to prove that the end goal of natural selection is the creation of the toilet brush. All that is needed is to replace the relevant landmarks. Thus, the Big Bang took place 13.7 billion years ago, after which another 10 had to elapse for life on Earth to arise. The appearance of the digestive tract, however, took only a further 2,75 and from then on the sphincter showed up merely another 575 million years hence. This projection leads us to the inescapable conclusion: eventually the whole universe will turn into one giant toilet brush.

While not identical to theirs, this classification owes much clarity to Cave and Dihal’s recent typology of the “ways in which these narratives [of hope and fear] could shape [AI] technology and its impact.” (Cave and Dihal 2019, p. 74)

Not to mention that AI researchers do not merely consume sci-fi but produce it as well. To single but two prominent examples, both John McCarthy and Marvin Minsky, starring figures at the Dartmouth Conference on Artificial Intelligence, which many consider the official birthplace of the field (Kline 2011), have contributed their talents to the narrative arts. Minsky co-authored the technothriller The Turing Option (Harrison and Minsky 1992) and McCarthy (2014) penned the delightful short story The Robot and the Baby, which shows just how hard it is to prevent people from anthropomorphizing automata.

In his Foreword to the Millennial Edition of 2001: A Space Odyssey, Arthur C. Clarke reproduces a touching letter sent to him by astronaut Joseph Allen, mission specialist on the Space shuttle program: “Dear Arthur, When I was a boy, you infected me with both the writing bug and the space bug, but neglected to tell me how difficult either undertaking can be.” (Clarke 2000, p. xviii)

Carnegie Mellon (academic home of Newell and Simon) is not just any university when it comes to the history of AI. Along with Minsky’s MIT, McCarthy’s Stanford and the Stanford Research Institute, it is one of the main four centers where AI took off. Seeking to characterize their differing styles, Pamela McCorduck offered this droll analogy between AI and the garment industry: “Consider MIT haute couture, the Women’s Wear Daily of the field. No sooner do hemlines go down with enormous fanfare than they go up again, the provinces growing dizzy with trying to keep pace and usually falling behind. MIT thinks itself stylish, but outsiders have been known to call it faddish. Carnegie Mellon, on the contrary, represents old-world craftsmanship, attending to detail and using the finest materials. These qualities presumably speak for themselves in gowns you can wear to a dinner party ten years from now and never fear the seams might part. But classic can be stodgy: if Queen Elizabeth of England bought artificial intelligence, she’d surely buy at Carnegie Mellon. Stanford has two ateliers. The first is the Levis’ jeans of AI: sturdy, durable, democratic; worn by socialites and welfare clients alike; and mentioned proudly by everyone in the trade whenever questions of practicality or utility come up. The other is Nudist World, incorporating After Six; this shop is visionary about the formal wear of the future, but meanwhile remains naked. Finally, Stanford Research Institute is Seventh Avenue. Maybe those models are knock-offs, but hardly anyone can afford haute couture, and except for the jeans people, who else is going to bring AI into the real world?” (McCorduck 1979, p. 112)

Compare with Dryden’s (1913) rendering of Pygmalion’s enthrallment to his creation, as told by Ovid: Pleas’d with his Idol, he commends, admires, Adores; and last, the Thing ador’d, desires.

The three laws made their first formal appearance in Asimov’s (1942) short story Runaround. To this story, Marvin Minsky claims a deep debt: “After ‘Runaround’ appeared in the March 1942 issue of Astounding, I never stopped thinking about how minds might work. Surely we'd someday build robots that think. But how would they think and about what?” (Minsky cited in Markoff 1992, ¶18).

Contrary to what the example suggests, the goal of some AI system needs not be particularly stupid to be extremely dangerous. Stephen Omohundro has argued that even a chess-playing robot “will indeed be dangerous unless it is designed very carefully. Without special precautions, it will resist being turned off, will try to break into other machines and make copies of itself, and will try to acquire resources without regard for anyone else’s safety. These potentially harmful behaviors will occur not because they were programmed in at the start, but because of the intrinsic nature of goal driven systems.” (Omohundro 2008, p. 483)

If you can’t beat them, join them, folksy wisdom asserts, and that is precisely what Yudkowsky did from 2010 to 2015 when he wrote his acclaimed spin on the Harry Potter franchise (Yudkowsky 2015). Hailed as one of the most successful fan fictions ever written (Whelan 2015), Harry Potter and the Methods of Rationality portrays Harry as a precocious genius that unleashes the whole arsenal of scientific reasoning upon the functioning of the magic world in order to maximize his own power (and optimize the world while he's at it). Keeping in line with Geraci’s (2012, p. 40) claim that the incursions of the AI community into the realm of fiction crafting are more often than not evangelical in nature and are never written just for fun, HPMOR, as it is popularly known, is an attempt, much like Yudkowsky’s Center for Applied Rationality, to induct young talents into the practice of Bayesian thinking, which could set them on a path of preventing the emergence of hostile superintelligences.

We mean ‘tacit’ in the sense of Polanyi 1983.

A very vivid case in point is a recent flashy headline that made the rounds of social media, to the effect that Facebook had been forced to shut down some of its Artificial Intelligence agents since they had developed their own secret language and started communicating with each other to the befuddlement of their creators (Griffin 2017; Bradley 2017; Collins 2017). What actually happened, though, is that chatbots designed for interaction with humans in a negotiation setting drifted from using conventional English and the researchers simply refined their reward schema to keep them on track with language that was grammatical (Lewis et al. 2017).