Patiency is not a virtue: the design of intelligent systems and systems of ethics

The usages of terms local to this article are chosen to attempt clear communication despite introducing a perspective many people find radical. I believe that incoherence has been introduced to AI and robot ethics debates partly because some terms are made to do “double duty.” For example, conscious and intelligent have fairly clear psychological and even computational meanings, but as a confound are often assumed to be core to moral obligation. The usages I exploit here are intended as strict subsets of ordinary language usage—that is, I attempt local to this article to focus these terms’ usage down to one essential meaning. I intend to use here only the formal philosophical term moral agent to mean “something deemed responsible by a society for its actions,” only moral patient to mean “something a society deems itself responsible for preserving the well being of,” and moral subjects to be all moral agents or moral patients recognised by a society (cf. Gray and Wegner 2009; Karlsson 2002; Kant 1785; Duranti 2004, p. 454). Note that moral agency is a distinct special case of agency more generally. Agency is just the capacity to effect change, for example chemical agents cause reactions. In itself the term implies no moral status.

When I talk about something as intelligent, I mean only that it can detect contexts appropriate for expressing one of an available suite of actions. If I call something cognitive then it is both intelligent and can learn new contexts, actions, and/or associations between these (cf. Winston 1984; Carlson 1994). If I call something conscious I only imply that it can act on explicit memory—memory of individual events (Bryson 2012). Local to this paper, these are treated as psychological terms that neither require moral status nor necessarily in themselves imply any other human-like trait. Using these extremely specific definitions, we can now meaningfully discuss questions such as “Does being intelligent necessarily make one a moral agent?” This is not to neglect the many other definitions of these terms or to say that they are ‘wrong’—any word’s meaning is exactly how it is used (Caliskan et al. 2017). The purpose of these definitions is to label and advance a set of concepts specifically rallied to answer the questions of whether or when AI should be considered as—or constructed to be—a moral patient.

Ethics will mean here the entire set of behaviours that maintains a society, including by defining it. This is the most radical departure from convention presented in this section. I will discuss its ramifications below, but first I will clarify. By this definition, ethics does include components that have clear universal utility, e.g. prohibition on murder or theft. But each individual society’s ethical system will also consist of components dedicated to creating its unique identity, or carving out an ecological or economic niche. Identity may seem less essential than universal morality, but determining the boundaries of a society may be essential to its persistence. Identity is linked to autonomy, the capacity of an entity to be distinguished. For example a nation is not autonomous if it cannot control its borders (Armstrong and Read 1995; Cooke 1999). The way I am defining ethics here, the interaction between individual and societal autonomy is key. Members of a society will often produce a system of public goods—that is, goods that benefit all members, such as security or transport. Public goods may be key to a society’s autonomy, and may even define a society (Bryson et al. 2014). For altruism I will use a standard definition from biology and economics: the willingness to pay a cost (such as the cost of constructing public goods) in order to benefit others. Such willingness is adaptive in the biological sense (meaning it can be promoted by evolution) if the sum of the benefits to the beneficiaries times the relatedness of the beneficiaries to the altruist is less than or equal to the cost to the altruist (\(\text {cost}_i < \sum _j \text {benefit}_j \times \text {relatedness}_{ij}\), Hamilton’s Law, Hamilton 1964).

One critique of this definition of ethics is that normally we would like to believe in a single ethical standard against which societies are able to improve. My definition does undermine this exact formulation, but does not mean we cannot make moral comparisons between societies, only that we have to specify a metric for any specific such comparison. So for example eliminating slavery or expanding suffrage results in a more egalitarian society or a more coherent set of laws, not just a “more ethical” society. The Nazis were not unethical, they had an elaborate system of ethics that while briefly facilitating rapid expansion of power, also killed many people (including co-nationals), and attracted its own destruction. Again, I am not claiming my definitions are necessarily the right or common ways to use these words, but that they are precise and useful for the present article’s discussion.

A key aspect of my normative argument hinges also on the definition of artefact. Again, local only to this article, I limit the term artefact to objects deliberately created by moral agents. This means that things referred to as ‘artefacts’ here are also specific to the society that deigns an agent moral, and therefore only occur in species that have established concepts and norms concerning responsibility and deliberation. Artefacts are for this article discontinuous from nature; the point at which a culture develops the concepts deliberate and responsible is when this discontinuity occurs and artefacts can begin to be made.

As stated earlier, any normative recommendation requires specifying the intended societal outcome. My recommendation is not simply derived from descriptive precedent. The advent of potentially-autonomous decision-making human artefacts is novel, and requires constructing new social and ethical accommodation. Descriptive ethics may take us some way by establishing precedent, but few consider precedent sufficient or even necessary for establishing what is right. Is does not imply ought (Hume 1739).

I will work then from two fairly familiar, hopefully uncontroversial objectives:The nature of machines as artefacts means that the question of their morality is not simply a question of what moral status they deserve (Miller 2015). Rather, at the same time that we ask what moral status we ought to assign intelligent artefacts, we must also ask what moral status we ought to build those artefacts to meet. This second aspect of our concurrent, tightly-coupled responsibilities has been neglected even by those scholars who have observed the constructive nature of the first (e.g. Coeckelbergh 2010; Gunkel 2014). As I said, ought does require able—computationally intractable and indeed logically incoherent systems such as Asimov’s laws are excluded (Myers 2010; Bryson 2017). So are ecologically unsustainable objectives.

Our capacity to design an artefact defines the term, which means that obligations regarding intelligent artefacts, unlike those regarding natural entities, can be met not only through constructing the socio-ethical system but also through specification of the intelligent artefacts themselves. This fact defies the intuition of many who cannot conceive of intelligence in non-human contexts, or who conceive of it on a single, Lamarckian scale that converges to human-like. The historical correlation of language, episodic memory, and reasoning with the prototypical moral subjects—human adults—is taken as necessary or even causal, as if there were particular badges or features of human moral status that could be excised from our gestalt and still deserve the same treatment as a citizen of our society.

In an effort both to reduce this confusion, and also to consider what would be minimally disruptive per the second objective just mentioned, the next section of this article discusses not what should matter to us, but why some things do. Before considering where we might want to slot robots into our contemporary ethical frameworks and society, I start by considering ethics and moral patiency from an evolutionary perspective. I do this less to inform our intuitions than to explain them.

As with all human and other ape (Whiten and van Schaik 2007) behaviour, our ethics is rooted both in our biology and our culture. Nature is a scruffy designer with no motivation or capacity to cleanly discriminate between these two sources of behaviour, except that what must change more quickly should be represented more plasticly (Depew 2003; Hinton and Nowlan 1987). As human cultural evolution has accelerated our societies’ paces of change, increasingly our ethical norms are represented in highly plastic forms such as legislation and policy (Ostas 2001).

The problem with a system of action selection so extremely plastic as explicit decision making is that it can be subject to dithering —switching from one goal to another so rapidly that little or no progress is made on either (Humphrys 1996; Rohlfshagen and Bryson 2010). Dithering is a problem potentially faced by any autonomous actor with multiple goals that at least partially conflict and must be maintained concurrently. Conflict is often resource-based, for example visually attending to two children at one time, or needing to both sleep and work. An example of dithering in early computers was thrashing—a process of alternating between two programs on a single CPU where each required access to the majority of main memory. Poor system design could result in an operating system allocating a slice of time to each process shorter than the time it took to be read into main memory from disk, preventing either program from achieving any of its real functions. More generally, dithering implies changing goals—or even optimising processes—so frequently that more time is wasted in the transition than is gained in accomplishment.

Perhaps to avoid dithering, we as humans prefer to regulate social behaviour even in an extremely dynamic present by planting norms in a “permanent,” bedrock past, like the anchoring of tall buildings built over a swamp. For example, American law is often debated in the context of the US constitution, despite being rooted in British Common Law and therefore a constantly changing set of precedents. Ethics is often debated in the context of holy ancient texts, even when the ethical questions at hand concern contemporary matters such as abortion or robots about which there is no reference or consideration in the original documents. Societies tend to believe that basic principles are rational, fixed, and universal. Enormous changes in social order such as universal suffrage or the end of legalised human slavery are simply viewed as corrections, bringing about the originally-intended rather than a newly-improved (or worse, locally-convenient) order.

In fact our ethical structures and morality do co-evolve with our society (Waal 1996). When the value of human life relative to other resources was lower, murder was more frequent and less sanctioned, and political empowerment was less widely distributed (Johnson and Monkkonen 1996; Pinker 2012). When women can support themselves and their children independently, infidelity is viewed less harshly (Price et al. 2014). What it means to be human changes, and our ethical systems have to accommodate that change.

As I implied when defining ethics, an ethical systems will contain components addressing two problems:The first problem may underpin our psychological obsession with ingroup-outgroup dynamics. I have suggested elsewhere that a society may be defined by the public goods it creates and defends, thus the scale of a coherent economy may limit the size of a society (Bryson et al. 2014, cf. Powers et al. 2011). The second problem could however at least in theory be universal, and as such could also be a candidate for describing how AI might become a moral subject. Maintaining a society internally is also the topic of the rest of this section.

I begin by considering the most basic component of social behaviour: whether that behaviour is for or against society—pro- or anti-social. Assessing morality is not trivial, even for apparently trivial, ‘robotic’ behaviour of single cell organisms, which also behave pro- and anti-socially. For example MacLean et al. (2010) demonstrate the overall social utility of organisms behaving in a way that at first assessment seems to be obviously anti-social—free riding off of pro-social agents that manufacture costly public goods. Single-cell organisms produce a wide array of shared goods ranging from shelter to instructions for combating antibiotics (Rankin et al. 2010). MacLean et al. (2010) focus on the production of digestive enzymes by the more ‘altruistic’ of two isogenic yeast strains. Having no stomachs, yeast must excrete such enzymes outside of their bodies. The production of these enzymes is costly, requiring difficult-to-construct proteins, and the production of pre-digested food is beneficial not only to the excreting yeast but also to any other yeast in its vicinity. The production of these enzymes thus meets the common anthropological and economic definition of altruism: paying a cost to express behaviour that benefits others (Fehr and Gächter 2000).

In the case of single-cell organisms there is no ‘choice’ as to whether to be free-riding or pro-social. This is genetically determined by their strain, but the two sorts of behaviour are accessible from each other during reproduction (the construction of new individuals) via common mutations (Kitano 2004; Youk and Lim 2014). For such systems, natural selection performs the ‘action selection’ between goals by determining what proportion of which strategy lives and dies. What MacLean et al. (2010) show is that selection can operate such that the lineage as a whole benefits from mixing both strategies (cf. Akçay and Cleve 2016). The ‘altruistic’ strain in fact overproduces the public good (the digestive enzymes) at a level that would be wasteful, while the ‘free-riding’ strain of course underproduces. Thus the greatest good—the most efficient exploitation of the available resources—is achieved by the species as a whole.

Why can’t the altruistic strain evolve to produce the right level of public goods? This returns to my earlier point about rates of plasticity. The optimal amount of enzyme production is determined by available food and this will change more quickly than the physical mechanism for enzyme production in a single strain could evolve. However death and birth can be fast and cheap in single-cell organisms. A mixed population composed of multiple strategies, where the high and low producers will always over and under produce respectively, and where their proportions can be changed very rapidly, is thus an agile solution. Thus the greater good for the species is served by the ‘selfishness’ of many of its members, but would not be so served without the presence of altruists.

Human society also appears to up and down regulate investment in public goods (Bryson et al. 2014). We may increase production of public goods by calling their creation ‘good’, and associating ‘good’ with a social status that is beneficial in the socio-economic contexts where more public goods are beneficial. Meanwhile, self interest and individual learning from direct reenforcement can be relied on to motivate and maintain the countervailing population of underproducers. For human society too the ‘correct’ amount of investment may vary quickly due to shifts in socio-economic and political context. For example, national military investment may be worthwhile under threat of invasion, but investment in local businesses may be more advantageous at other times. This implies that the reduction of other’s ‘good’ behaviour can itself be of public utility in times when society benefits from more individual productivity or self-sufficiency (cf. Trivers 1971; Rosas 2012). If so, we would expect that in such contexts it may also be easier for human institutions to change their overall assessment of which public goods require investment than to change their exact rate of output for all individuals (Bryson et al. 2014).

Is does not imply ought. The roots of our ethics do not entirely determine where we should or will progress. But roots do affect our intuitions. Our intuitions towards inclusion of artefacts in our society are probably driven by the extent to which we identify with such artefacts (Bryson and Kime 2011). This goes back to the biological account for altruism given in the definitions section: we are by nature willing to pay a higher cost for those more related to us. For humans, this ‘relatedness’ seems to extend also to those whose ideas we share (Plotkin 1995; Gardner and West 2014). This would allow us to be a phenomenally agile species, rapidly generating new societies to exploit available opportunities, particularly if (as seems to be true, Coman et al. 2014) we can prompt each other to focus on particular identities in particular circumstances.

Others have proposed using our intuitions as a mechanism for determining our obligations with respect to robots and AI (Dennett 1987; Brooks 2002; Prescott 2017). Because of their origins in our evolutionary past, and the simple observation of how patiency can be attributed to plush toys (Bryson and Kime 2011), I do not trust this strategy to create coherent ethics. I do however trust those with vested interests—such as interests in selling weapons, robots, or even books—to exploit such intuitions (Bryson 2010; Bryson et al. 2017). Although established precedent is close to my second objective proposed earlier for the justification we seek for a normative recommendation, I consider picking a precedent (in-group identification) that divides as much as it unites to be unsatisfactory. Such divisions seem particularly dated given that we can expect communication technology to increase the potential size of our social group (Roughgarden et al. 2006; Bryson 2015). In the next section I turn as an alternative established source of criteria for making a normative recommendation to philosophy, which I exploit in the sections following to propose a more coherent, minimally disruptive path to situating AI in our society, and (therefore) our ethics.

“[Moral] action is an exercise of freedom and freedom is what makes morality possible.”—Johnson (2006). For millennia morality has been recognised as something uniquely human, and therefore taken as an indication of human uniqueness and even divinity (Forest 2009). But if we throw away a supernaturalist and dualistic understanding of human mind and origins, we can still maintain that human morality is at least rooted in the one incontrovertible aspect of human uniqueness—language—and our unsurpassed competence for cultural accumulation that language both exemplifies and further enables (Bryson 2008). The cultural accumulation of new concepts gives us more ideas and choices to reason over, and our accumulation of tools gives us as individuals more power to derive substantial changes to our environment from our intentions.

Some of these tools include social concepts that have proved useful fulcrums for the leverage we need to construct our complex societies. These include ‘self’, ‘society’, ‘justice’, ‘responsibility’, ‘freedom’, and ‘intention’. As asserted earlier in the section on definitions, it is at the point of the invention of these concepts that we can discriminate artificial from natural intelligence. An artefact is something for which the design is intentional. That intention—the authorship of our action—ordinarily is seen as entailing responsibility, even in the face of determinism (Fischer 1999).

If human morality depended simply on human language then our increasingly language-capable machines would be excellent candidate moral subjects. But I believe that freedom—which I take here to mean the socially-recognised capacity to exercise choice is the essential property of a moral actor (cf. Tonkens 2009; Rosas 2012). Dennett (2003) argues that human freedom is a consequence of evolving complexity beyond our own capacity to provide a better account for our behaviour than to attribute it to our own individual responsibility. This argument entails a wide variety of interesting—and not necessarily desirable—consequences. For example, as our science develops and our behaviour becomes more explicable via other means (e.g. insanity) fewer actions might be taken as moral. This principle might also be seen to encourage the irresponsible construction of opaque institutions or obfuscated source code for robots or other AI systems in order to avoid individual or institutional responsibility (Siponen 2004; Bryson et al. 2017).

I will nevertheless here be conservative and follow from Dennett’s suggestion to generalise morality beyond human ethics. Again only local to this article, I define moral actions for an individual agent to be those for which:Note that this definition captures society-specific morals as well as the individual’s role as the actor.

With this definition I again reach to the biological heritage of our present ethics by deliberately extending morality to include actions by other species which may be sanctioned by their society, or by ours. For example, non-human primates will punish individuals that violate their social norms, e.g. for being excessively brutal in punishing a subordinate (Waal 2007), for failing to vocally ‘report’ available food (Hauser 1992), or for sneaking copulation (Byrne and Whiten 1988).¹ Similarly, this definition allows us to say dogs and even cats can be good or bad when they obey or disobey human social norms they have been trained to recognise, provided they have demonstrated a capacity to select between relevant alternative behaviours, and particularly when they behave as if they expect social sanction when they select the proscribed option. I make this inclusive reach to prepare for a consideration of ethics from the perspective of a society of artefacts.

With respect to AI, there is no question that we can train or simply program machines to recognise more or less socially-acceptable actions, and to use that information to inform action selection (Cakmak et al. 2010; Riedl and Harrison 2016). So we can certainly build AI to take moral actions. But this in itself does not determine moral agency. The question is, who would be responsible for those actions? An agent that takes a moral action is not necessarily the moral agent—not necessarily the or even a locus of responsibility for that action. A robot, a child, a pet, even a plant or the wind might be an agent that alters some aspect of an environment. Children, pets, and robots may know they could have done ‘better.’ We can expect the assignment of responsibility for moral acts by intellgent artefacts to be similarly subject to debate and variation. Moral responsibility is only attributed to those a moral community has recognised as being in a position of responsibility. Households may differ in their assignment of culpability to children and pets, so may civilisations. Presently in the OECD at least, small children and pets are certainly not considered legally responsible for their actions.

My recommendation for AI (below) will be similar. However, the core observation here is that a moral community defines itself and its moral agents—not by simple assertion of an individual, but by consensus of the society formed. A growing child will demand agency, and as they grow these demands generally become both more costly to deny and safer to accede. However, children by our nature become adults, the components of our societies. We tend to build AI systems in their final form—as search engines, surveillance cameras, spell checkers, automobiles, or whatever product we are marketing. Should we produce a product to be a moral agent? Can our society sustain itself if responsibility is delegated to entities that can be specified, built, bought, and sold?

We have now reached the heart of machine responsibility. This is the point at which there is no simple descriptive solution, but rather we are looking to establish norms and laws that will lead to an ethics that is both sound and stable. We could designate to intelligent artefacts any position of responsibility we choose, and indeed several such positions are presently being considered by various legal systems (Bryson et al. 2017). To motivate my normative recommendations, allow me to ask a relevant question: Would it be moral for us to construct a machine that would of its own volition choose any but the most moral action?

This is a trick question, the key to which returns to the definition of freedom I took from Dennett. For it to be rational for us to describe an action by a machine to be “of its own volition”, we must already have sufficiently obfuscated its decision-making process such that we cannot otherwise predict its behaviour, and thus be reduced to applying sanctions to it in order for it to learn to behave in a way that our society prefers. Otherwise, if the machine acted as we intended, the responsibility would be ours just as if we had performed the action with any other tool.

Note that if we assume as I’ve asserted that ought implies can, there’s an issue of whether we can coherently and ethically produce AI that suffers from sanctions. I will return to this point below, but first I wish to discuss obfuscation. I do not consider training action selection via deep learning, reinforcement learning or any other statistical technique to be necessarily obfuscating in this sense. Even if we do not know the exact ‘meaning’ of every individual components of an internal representation, the basic principles of optimisation that underlie machine learning are well-understood and the probable outcomes known to in my mind be sufficient for moral clarity (see e.g.Wilson et al. 2016). Further, there are basic and well-established procedures for creating test suites and exploring responses to input to ensure a system incorporating machine learning or any other sort of programming, or even human judgement, is performing to a standard that we approve (Jones 2008; Chessell and Smith 2013; Dwork et al. 2012; Feldman et al. 2015). Similarly, I do not consider the fact that unexpected effects ‘emerge’ during the operation of complex systems to alter the designers’ responsibility to observe and account for such effects. Neither do present courts of law (Bryson et al. 2017).

As I asserted earlier, the step change from nature to artefact is our intentional acts of creation, for which we as moral agents are almost by definition and certainly by convention considered responsible. When executing an intentional action, deliberately blinding oneself to an outcome is not ordinarily seen as ending responsibility; rather it is termed wilful negligence. The same should hold true for not following adequate procedures to ensure transparency in the construction of intelligent artefacts (Wortham and Theodorou 2017; Bryson and Winfield 2017). Since we have perfect control over when and how a robot is created, we also have responsibility for it. Assigning responsibility to the artefact for actions we designed it to execute would be to deliberately disavow our responsibility for that design. Currently, even where we have imperfect control over something as in the case of young children, owned animals, and operated machinery, causing harm by losing control entails at least some level of responsibility to the moral agent, the legal person. If you deliberately drive into someone you commit murder, if you do it accidentally you commit manslaughter. You are responsible but the sanctions are reduced.

Why, for example, are we responsible for intelligent beings such as children, but only up to a fixed age? Because society has found this to be the best method for organising itself, though there is some dispute and therefore variability about the exact point at which the child becomes responsible. Thus legal persons are responsible for maintaining control over their dogs, cars, and children, and individuals can be held accountable for negligence and manslaughter performed by those things over which control was not sufficiently maintained (Lia 2015). Again, these laws are not based on some pure, indisputable, formal, mathematical fact—though they can be informed by science, for example of developmental psychology. Fundamentally though they reflect the best way of maintaining our society that our society has been able to both discover and agree to enforce. Subparts of our society, for example clubs or families, may institute additional rules that maintain and hopefully enhance the lives of members of these smaller societies.

Many people have suggested after Kant that failing to treat something that appears to us to be human as if it were human would be a moral wrong towards other humans, because it encourages our propensity to dehumanise (Gunkel 2017 for a recent review). While it would be both foolish and unnecessary to argue against Kant, what these arguments overlook is that there are two ways to address this problem—either by treating artefacts as human, or by making their inhumanity transparent (Theodorou et al. 2017). The only extant national-level robot ethics policies recommends the latter (Boden et al. 2011), as do I.  Another possible benefit of robot moral subjects is that, for some of us, it gives us pleasure or feeds our egos to construct objects that we owe moral status (Helmreich 1997; Bringsjord et al. 2012). Some of us also project ourselves into our creations, and see AI as a route to immortality more perfect than other forms of procreation (Goertzel 2010). Of course, that perceived identity is demonstrably false (Choe et al. 2012; Claxton 2015).

To me the only persuasive argument is that it is possible that in the long term treating intelligent artefacts as moral agents would be a simpler way to control an opaquely-complex intelligence, and that the benefits of that opaquely complex intelligence might outweigh the costs of losing some of our own moral responsibility and therefore moral status. However this necessity has yet to be demonstrated. Given the costs of abdicating responsibility, we should not abdicate based on speculation (Bryson et al. 2017).

Some are currently arguing that enforcing good practice in transparency and accountability for AI may slow the rate of progress. Empirically, it often turns out that designing a system to be transparent makes it easier to maintain and extend, so any such penalties even if they exist may only do so in the short term (Theodorou et al. 2017; Zeng et al. 2017). But even if the penalties of transparency prove real and long-lasting, we need to consider the benefits of any accelerated progress against the costs of losing human responsibility, costs which may include losing social cohesion. The principal such cost I see is the facilitation of the unnecessary abrogation of responsibility by sellers or operators of AI. For example, customers could be fooled into wasting resources needed by their children or parents on a robot, or citizens could be fooled into blaming a robot rather than a politician for unnecessary fatalities in warfare (Sharkey and Sharkey 2010; Bryson and Kime 2011; Bryson 2000). A corporation could displace responsibility for its decision to use automation rather than human employment onto the automation itself, creating a legal lacuna—a set of far poorer, purely-synthetic entities set up to be held responsible for tax and legal liability (Bryson et al. 2017). If such an entity went bankrupt or were jailed, it would disuade no one into changing their or its behaviour.

The more I study the legal aspects of this problem, the more convinced I am that academics facilitating the fantasies of trans- and post-humanists (Geraci 2010) run the risk of encouraging political and economic decisions that could seriously disrupt our ability to govern, as well as our economy. The artificial entities likely to be most advantaged are transnational corporations, and there are few effective mechanisms of government at the transnational level. Without a capacity to govern our economy we lose the collective ability to encourage arts and innovation beneficial to all. With unaccountable power and economic inequality, the majority of humanity feels and may indeed become too at-risk to innovate and contribute to their local economy and public goods. Empirically, this leads to social disruption (Atkinson 2015).

Although this argument has been overlooked by some critics (notably, Gunkel 2012; Prescott 2017), the policies I promote (e.g. Bryson 2010, 2009) have always explicitly considered the welfare of potential intelligent artefacts. I cast these as second-order moral patients. Why should we design artefacts to be in the position of competing with us for resources; of longing for higher social status (as all evolved social vertebrates do); of fearing injury, extinction, or humiliation? We are able to ensure that AI is properly and continuously backed up. We can and do build it to have no concern for social status, nor sense of purpose.

In short, we can afford to stay agnostic about whether an artefact can have qualia, because we can avoid constructing motivation systems encompassing suffering. We know we can do this because we already have. There are many proactive AI systems now, and none of them suffer. There are already machines that play go, chess, checkers, do arithmetic, refrigerate, and clean clothes better than we do, but none of these aspires to world domination. We can limit AI—or at least legally-produced commercial AI—to be as it is now, something to which no obligations are owed directly. There can therefore be no ethical costs to the AI of maintaining AI in its present status of non-suffering, unless we postulate rights of the ‘unbuilt’.

Tonkens (2009) makes a very similar point to mine concerning AI well being, which Rosas (2012) disputes. I believe the root of the conflict here is that Rosas believes morality must be rooted in social dominance structures. The definition of morality I introduced in the previous section eliminates this confound. For evolved intelligence, dominance structure may be an inevitable part of the selective process; therefore we may expect the dysphoric aspects of subjugation may also be universal in evolved beings. Certainly therefore human ethical systems, as a part of social regulation, have much to say concerning dominance. But in designed artefacts we can safely eliminate this dysphoric aspect of subservience. Even negative self assessment by a robot has no need to lead to self harm or degradation, just restraint in risk taking and a request for repairs.

In the Introduction I suggested two criteria for ethical systems: coherence, and a lack of social disruption. I can think of no coherent reason to create agents with which we should compete. Every value we have, from æsthetics to peace to winning, comes from our evolutionary origins as apes. What coherent reason can we have to ‘pass the baton’ to machines made to share and compete on the basis of these goals? Even if we take the technologically-dubious case of machine immortality, what would we be making immortal? Any self-learning technological agent would rapidly evolve preferences that suit its machine nature, not ours. Would an initially-human-like capacity for computation be worth sacrificing human potential for in order to create something eventually as similar to us as crabgrass (Moore 1947)? Returning to Hamilton’s Law, the answer might be “yes” if we could assume that our technology was likely to survive our species or even our civilisation, but to date digital technology formats tend to survive less than \(\frac{1}{16}\) as long as individual humans.

Bryson et al. (2002) argue that the right way to think about intelligent services (there in the context of the Internet, but here I will generalise) is as extensions of our own motivational systems. We are currently the principal agents when it comes to our own technology, and I believe it is our ethical obligation to design both our AI and our legal and moral systems to maintain that situation. Legally and ethically, AI works best as a sort of behavioural prosthetic to our own needs and desires. If we wish to extend the lifespan of our civilisation, I recommend focussing on ways to do this while maintaining a flourishing human society at the motive core.

As mentioned above, one of the best arguments I know against this human-based perspective is that maltreating something that reminds us of a human might lead us to treat other humans or animals worse as well (Parthemore and Whitby 2014). The United Kingdom’s EPSRC Principles of Robotics specifically address this problem in its fourth principle, and in two ways (Boden et al. 2011; Bryson 2017). First, robots should not have deceptive appearance—they should not fool people into thinking they are similar to empathy-deserving moral patients. Second, their AI workings should be ‘transparent’ (Wortham and Theodorou 2017). This implies that clear, generally-comprehensible descriptions of an artefact’s goals and intelligence should be available to any owner, operator, or other concerned party. This Principle was adopted despite considerable concerns about the requirement for both therapeutic and simple commercial/entertainment robots to masquerade as moral patients and companions (cf. Miller et al. 2015). Because of this consideration, the fourth Principle deliberately makes transparency available for informed long-term decisions, but not constantly apparent. The goal is that most healthy adult citizens should be able to make correctly-informed decisions about emotional and financial investment. As with fictional characters and plush toys (Ullán et al. 2014), we should be able to both experience beneficial emotional engagement, and to maintain explicit knowledge of an artefact’s lack of moral subjectivity.

One thread of theory for the construction of human-level AI² holds that it may be impossible to create the sort of intelligence we want or need unless we completely follow the existing biologically-inspired templates which therefore must include social striving, pain, etc. So far there is no evidence for this position, in fact we are persistently creating super-human AI capacities without these attributes (Bryson 2015; Bryson and Winfield 2017). But if it is ever demonstrated, even then we would not be in the position where our hand was forced—that we must permit patiency and agency. Rather, we will then, and only then, have enough information to stop, take council, and produce a literature and eventually legislation, regulation, and social norms on what is the appropriate amount of moral subjectivity to permit given the benefits it would provide.