Just research into killer robots

Devices that decide whether and when to explode, in the broadest possible sense, have been around since the nineteenth century in the form of naval mines, while homing missiles and proximity tracking explosives have existed since WWII. Before that, military use of trained animals—who have their own will and attack on their own volition in some cases—is almost as old as warfare itself. For a discussion of dogs in particular, see https://​thestrategybridg​e.​org/​the-bridge/​2017/​12/​9/​autonomous-weapons-mans-best-friend.

For a thorough description, see https://​www.​defenseindustryd​aily.​com/​phalanx-ciws-the-last-defense-on-ship-and-ashore-02620/​ (accessed 21-1-2017).

See http://​www.​stopkillerrobots​.​org/​ as well as a large scale petition signed by many information technology experts demanding a ban on LAWS research (https://​futureoflife.​org/​open-letter-autonomous-weapons/​).

The Campaign to Stop Killer Robots only calls for the ban on research of ‘fully’ autonomous weapons systems, which excludes the Phalanx and other systems in widespread use.

For this reason, this paper will not be concerned with scenarios that involve robot uprisings as described in movies like The Terminator or books like Robopocalypse. Such scenarios are implausible and, even if we thought such possibilities were realistic, it is unlikely that a ban on researching LAWS would be relevant to preventing the ‘x-risk’ of human extinction or enslavement by robot overlord. First, AI research will continue outside the military context and, second, no one is arguing for bans on technology—military or otherwise—that can be controlled through a computer network and be used to kill humans. So, the ban will neither prevent the rise of a true AI nor will it prevent that AI from having access to weapons.

The Campaign to Stop Killer Robots: https://​www.​stopkillerrobots​.​org/​the-problem/​.

Consider the worry that LAWS will be hacked, as in the 2017 Open Letter (https://​futureoflife.​org/​autonomous-weapons-open-letter-2017/​). It is surely true that LAWS will be vulnerable to hacking. But human soldiers are vulnerable to similar issues. Undercover militants joining the police are a significant problem in counter-insurgency warfare, as are betrayals due to ideological sympathy, blackmail, or bribery. Setting intrinsic objections to the side temporarily, the question is not, “Can LAWS be hacked?” but “Will replacing or supplementing this particular job, post, or position with LAWS increase the risk of harmful behavior, given equivalent ideal and non-ideal assumptions?”

Again, if we reached a judgment that a particular advanced military was indifferent to the principle of just war, then that would give us good reasons not to continue LAWS research for that particular military.

In setting up the issue this way, I am deeply indebted to Simpson and Mullers (2016). They argue that we should understand just war principles in terms of fair distributions of risk. My paper is different in at least three ways. It sets out the dynamics of the modern military system in order to more fully describe the appropriate baseline, it uses that baseline to undermine intrinsic objections, and it is about the morality of research rather than the morality of deployment.

Ibid. For the canonical statement of just war principles, see Walzer’s (2002). For international humanitarian law, see the Geneva Conventions, especially Article 51 of the First Protocol Relating to the Victims of International Armed Conflicts, which concerns discrimination and proportionality.

The heroic actions of Hugh Thompson and his helicopter crew at the My Lai Massacre during the Vietnam War are instructive (Angers 2014). Upon observing a massacre of Vietnamese civilians by American soldiers, as ordered by Lieutenant Calley, they saved many innocent people by physically blocking soldiers from approaching parts of the village and then reporting the massacre to superiors. Yet, despite the fact that Lieutenant Calley was acting in direct contravention to his orders, he had no difficulty getting the vast majority of the soldiers under his command to obey. What’s more, it is worth noting that Thompson and his crew were not under Calley’s command.

My account of the ‘modern military system’ is based on the analysis of Chap. 3 of Biddle (2006). I’ve also benefitted from Pollack (2002), especially Chap. 1. For a good illustration of these concepts in military practice: see Department of the Army (1998).

Swinton (1986) is a commonly used illustration of these principles in military academies.

According to the Iraqi Body Count, only 13% of civilian casualties in Iraq could be attributed to coalition direct fire; the majority of casualties were the result of indirect combat between insurgents and coalition forces or committed by insurgent forces alone: https://​www.​theguardian.​com/​news/​datablog/​2012/​jan/​03/​iraq-body-count-report-data.

A veteran and student in my “Ethics of War and Peace” course suggested this case based on his experience in Fallujah.

There is considerable controversy as to why force protection is morally important. Walzer (2006) argues that force protection is only relevant due to military necessity: dead soldiers cannot achieve the military objective of winning a just war. Others (Kasher and Yadlin 2005), however, have argued that soldiers need not take on risks to themselves to spare civilian lives. I remain agnostic; my paper only requires that military behavior is driven by force protection and that this is at least sometimes justified.

Singer (2009) makes the point that LAWS can be more ‘conservative’ than human warfighters. My view moves beyond his in several ways. First, it bases the claim on an understanding of the modern military system. Second, it offers the normative basis for the civilian-risk-acceptant behavior of human warfighters. Third, it describes the knock-on dynamics of this advantage for LAWS. Fourth, it uses these insights to engage with intrinsic objections as well. Fifth, Singer does not describe the issue in terms of the fair distribution of risk and so does not offer adequate normative guidance on the question.

This article aptly summarizes the available evidence that drones are more discriminate than other tactics: http://​www.​slate.​com/​articles/​news_​and_​politics/​foreigners/​2015/​04/​u_​s_​drone_​strikes_​civilian_​casualties_​would_​be_​much_​higher_​without_​them.​html. A few other studies purport to show that drone attacks are not more accurate, but they often rely on problematic comparisons. For example, some studies show that fighter-bomber attacks on ISIS military positions kill fewer civilians per sortie than drone assaults in Pakistan or Yemen. But this is not an apples to apples comparison since ISIS is engaged in conventional, full spectrum operations with operational lines and distinct warfighters. Attacks in Yemen and Pakistan are aimed at insurgents who deliberately hide amongst civilians. Conventional tactics by the Pakistani military in these areas generate civilian casualties that are orders of magnitude higher than drone attacks.

As reported by Edward Porter Alexander (1907).

For example, neither Larson (1996) nor Gartner and Segura (1998) assert a straightforward relationship between casualties and support for a war. First, war support always erodes over time but it can be revived through victories even in the face of high casualties; the way the war is fought and perceived plays a key role. Second, casualties cannot explain the level of support for war even in those cases where casualties can explain a reduction in war support.

On the (possibly problematic) difference between responsibility as attributability and responsibility as accountability, see Watson (1996) and Smith (2012).

Asaro, Johnson, and Axinn do not seem to apply these arguments to self-driving cars, which also involve AI systems making life or death decisions.

Street lights and self-driving cars are not, however, designed in order to take life even if they make choices that take it. Perhaps these human dignity concerns only apply to killings that are done by autonomous systems that are intentionally created to take life. Notice, however, that we are no longer talking about particular actions or decisions made by some autonomous system but are focusing on the telos or purpose of the system. Yet, what constitutes the purpose of a particular technology is notoriously difficult to pin down. We can, of course, describe the streetlight’s purpose as ‘directing traffic’ and a LAWS’ purpose as ‘killing’ but we could also describe the streetlight’s purpose as “the provision of transportation-related goods at an acceptable level of traffic-related deaths” and LAWS as “the provision of physical security goods through deterrence at an acceptable level of casualties.” This view would also have the perverse implication that commercial autonomous systems that were misused in order to kill would be subject to lesser moral constraints than weapon systems that could be deliberately designed to minimize civilian casualties. At any rate, much would need to be done to make this objection work. The relevant level of description would need to be set, an account of how to precisely determine the function of a weapon system would need to be created, and one would need to show why the function of a system is morally relevant. All of these steps are fraught with difficulty, and this strategy seems to be ultimately contrary to contemporary developments in warfare. In a world where insurgents increasingly use civilian technologies and the line between warfare and law enforcement is increasingly blurred, it seems like we need to develop concepts that do not assume a sharp rupture between military systems and everything else.