Jindan-Karena Mann

Autonomous Weapons Systems and the Liability Gap, Part Two: Civil Liability and State Responsibility

I. Introduction

In a prior post, I discussed how the use of autonomous weapons systems during warfare may result in a liability gap, especially because of the absence of a meaningful human decision-maker when artificial intelligence is used. While that post covered criminal liability, particularly international criminal liability in the context of the ICC Rome Statute, this Part Two will analyze whether similar gaps exist under civil liability or State responsibility regimes.

There are two aspects of autonomous weapons systems that primarily contribute to the existence of a liability gap. First, robotic weapons process high levels of information at lightning speed and are able to operate at a much faster rate than humans, meaning that there is no effective human control when the controller is unable to verify the weapon’s decision by the time the weapon would act. Second, autonomous weapons systems’ self-learning feature mean? that they may develop new skills or behavioral tendencies from the environment in which they operate, which would have been impossible for the producer or manufacturer, and even in some cases the person deploying the weapon, to foresee and mitigate.1

Since holding robots liable for their unconscious yet wrongful behavior is not an option under existing law, accountability must devolve on to another legal entity or persons in order to avoid a situation of impunity. As the prior post highlighted, the lack of intentional or willful behavior by the primary wrongdoer—a robot—means that criminal liability is difficult if not impossible to establish. However, other areas of law, namely tort law, strict liability mechanisms, or the law of State responsibility, do not require the same standard of mens rea and thus might open up avenues for assigning accountability. This post seeks to address this question by looking into corporate liability in order to explore how the programmer or manufacturer could be liable for mistakes or defects that lead to the commission of international humanitarian law violations, as well as by considering State responsibility for owning and deploying such weapons, both before the International Court of Justice and domestic courts.

II. Corporate Liability

A. Negligence

 Pursuing tort liability for harms perpetrated by autonomous weapons systems presents a number of advantages. Under tort law, liability does not necessarily require a finding of intent on the part of any of the actors involved. Furthermore, unlike under criminal liability, a company or non-human legal entity can be held liable in virtually every jurisdiction, either directly or through vicarious liability mechanisms for acts of its employees. 

One of the most fundamental claims in tort law is a claim of negligence. Typically, the elements of negligence include: 1) some sort of tortious act, which is characterized as a breach of the duty to use reasonable care,2 or an unjustified violation of a duty imposed by law or by what is generally regarded as proper social conduct or the violation of someone else’s right or entitlement;3 2) a causal link between the tortious act and the harm;4 and 3) actual damages or loss on the part of the plaintiff.5

Generally, two mechanisms exist to ensure that tort liability is duly justified and that persons are not held liable for non-culpable conduct. First, the reasonable care standard means that if a risk is not foreseeable by a reasonable person under those same circumstances, then a person should not be held accountable for failing to prevent or avoid that (reasonably foreseeable) harm. Second, a causal link is required, which in most cases means that but for the wrongful conduct, the harm would not have occurred.

The safeguards in place to ensure that non-culpable behavior falls outside the scope of negligence may also present barriers to liability when autonomous weapons systems are involved. Because of the weapon’s self-learning abilities, the programmer and manufacturer would be entirely unable to foresee all the possibilities of what the weapon might learn that would ultimately cause it to commit a wrongful act, and their inability to circumvent the harm would mean that holding them accountable would be tantamount to imposing strict liability.

Several scholars have proposed that the foreseeability barrier may be circumvented by holding the person who introduced the unpredictability accountable for failing to design a “reasonably prudent algorithm.”6 However, some have noted that because “the causal influence resulting in the unpredictability of a system stems from the datasets used for learning, not a legally responsible agent,” it nonetheless cannot be established that the defendant’s act was the actual cause of the harm, even if he breached his duty to design an appropriately prudent artificial intelligence system.7

Joint tortfeasor liability may offer a way around this causation conundrum when the human defendant cannot be determined to be the actual cause of the harm because the weapon’s own learning and decision-making functions also contribute to causing the harm. Logically, when there are two or more possible contributors to a harm, it may be impossible to prove that but for the existence of one of them, the harm would not have occurred. Therefore, courts have developed various doctrines of joint liability in which they abandon the but-for causation requirement and focus instead on whether, as a whole or alternately, the joint defendants’ conduct caused the harm. Under these rules, defendants may be jointly liable, meaning that each contributor is liable for the full amount of the harm regardless of the degree of their contribution, or they may be held proportionally liable in accordance with their respective contribution. Multiple defendants may also be held jointly and severally liable, meaning that each defendant is fully responsible, and the burden shifts to them to sort out their respective proportions of liability based. Ultimately though, whether these joint modes of liability can be applied when one of the contributors is never able to be held liable, such as in the case of a robot, remains untested.

B. Strict Liability

Because strict liability does not require a finding of fault such as negligence or wrongful intent, it may present an avenue to establish liability on the part of the programmer, manufacturer or even deployer of the weapon. However, it is limited to a narrow set of circumstances under which its imposition can be justified. Generally, courts have found that it is reasonable to impose strict liability on abnormally dangerous or ultra-hazardous activity, defined as activity that presents a significant risk of physical harm even when all participants are reasonably careful, and that is not of common usage, such as the use of explosives or dangerous chemicals, or even the ownership of wild animals.8

The argument that this class of strict liability should apply to the use of autonomous weapons systems would need to be sustained on the hypothesis that the use of these weapons is analogous to ultra-hazardous activity. While their use does present a risk of harm even when the closest human actors exercise reasonable care, their autonomous functions arguably might make them safer than traditional weapons in most cases and therefore they are not analogous to the types of conduct typically considered dangerous enough to warrant strict liability. Furthermore, some scholars concede that it is unclear where to draw the line between the types of weapons that should incur strict liability and those that should not, depending on the degree of autonomy of the weapon and its ability to learn potentially risky behavior that justifies the heightened standard.9

C. Product Liability

In addition to strict liability for abnormally dangerous conduct, product liability may also result in the imposition of strict liability. There are generally three types of product liability: for defects in marketing or presentation of a product; for design flaws, which are inherent in the product; and for manufacturing defects specific to a particular item.10

For product liability rules to apply, first it must be determined that autonomous weapons systems are even products within the meaning of applicable statutes and laws. In the context of US law, because the algorithms within the autonomous weapons systems are not marketed for products for sale, they fall outside the scope of the Uniform Commercial Code, as well as most state laws that regulate product liability.11 Meanwhile, in the European Union context, the European Commission has recognized the lack of clarity on the relationship between artificial intelligence and product liability, and expressed intentions to update its directive on product liability in 2019.12

While the strict liability aspect of product liability might at first glance appear to present a clear path for establishing accountability when things go wrong in the use of autonomous weapons, the self-learning characteristics of these weapons means that the programmer’s design no longer resembles the program once it has been enhanced with additional learning, thereby complicating product liability. Although strict liability eliminates the need for concern about culpability, it should only be imposed when the defendant has some sort of causal connection to the harm, which it may not if the program transformed significantly between the time of design or manufacture and the incident that ultimately resulted in a violation.13 Generally, designers, manufacturers, and users cannot predict behavior that is dependent on novel data input, and should not liable for conduct and consequences that are not foreseeable.14 As Woodrow Barfield points out, “strict liability cases do not impose liability on the manufacturer and other parts of the distribution chain for changes made to the product after delivery to the consumer unless those changes were foreseeable. Because autonomous robots, may ‘change’ from their original programming, their actions may not be foreseeable. Foreseeability makes strict liability poorly suited for injury caused by autonomous robots.”15

In addition to the potential lack of a causal link when an autonomous weapon transforms significantly between the time it enters the market and the moment it engages in harmful conduct, product liability regimes also have some provisions that make it relatively easy to avoid liability. For example, although a failure to warn consumers of latent risks present in the product may result in liability, in some jurisdictions the producer or manufacturer need only include a warning of the potential risks, such as possible errors in the learning algorithm, in order to avoid responsibility.16

III. State Responsibility

In addition to pursuing liability of corporations for their role in designing and producing autonomous weapons systems, another option would be to hold the State responsible because of its role in purchasing and using the weapon.

The law of State responsibility is mainly derived from the Draft articles on Responsibility of States for Internationally Wrongful Acts, which although not a binding instrument largely reflects customary international law. Under Article 1 of the Draft articles, “[e]very internationally wrongful act of a State entails the international responsibility of that State.” Unlike criminal or most civil law regimes, liability is strict in the sense that any breach of international law by a State entails its international responsibility, even without a showing of a certain mental state.

Both treaty law and customary international law hold that States are responsible for war crimes committed by members of their armed forces.17 Because there is no requirement for a finding of intent or knowledge on behalf of the State, it is not a conceptual leap to conclude that a State should also be liable for war crimes committed by the weapons it has deployed when those weapons, outfitted with sophisticated artificial intelligence software, render a decision that results in war crimes. If the current law finds it justifiable to assign State responsibility for war crimes committed by its armed forces even when the State has no information about the facts that led to the harm, then the problems of intent or foreseeability that inhibit a finding of liability under criminal or tort law regimes are not present here.

Fulfilling the elements for State responsibility is not only easier than under criminal or civil law; some have argued that holding States accountable for violations committed by autonomous weapons systems is more appropriate, because governments are perhaps the most culpable actors involved as the primary purchasers of weapons, and they are also in the best position to ensure that weapons comply with international law.18 Furthermore, the Draft articles’ accountability regime includes the possibility of a remedy, as “[t]he responsible State is under an obligation to make full reparation for the injury caused by the internationally wrongful act,”19 including  to “make restitution, that is, to re-establish the situation which existed before the wrongful act was committed.”20

Despite these positive points, there are limited fora in which to bring a claim against a State and thus the likelihood of actually securing State accountability is unlikely.21 Although the International Court of Justice has broad subject-matter jurisdiction, its personal jurisdiction is limited and it lacks enforcement mechanisms.22 Only States can submit contentious cases to the ICJ, and both parties must consent to the court’s authority either through treaty, formal acceptance of the court’s jurisdiction, or a special agreement to submit the case to a court. In the event of harms arising out of the use of autonomous weapons systems, these would likely arise in the context of an armed conflict or similar situation in which no treaty would exist. Furthermore, major military powers including the US, Russia, and China have not accepted the court’s jurisdiction and special agreements to submit to the court are rare.23

Another option would be to bring a civil claim against a State in a domestic court. For example, a plaintiff might summon a State before the national court of the another State, such as the place of the harm. On the other hand, a plaintiff may opt to bring a case before a national court of the State that allegedly committed the wrong. However, the availability of these option depends on the scope of the State’s position on sovereign immunity and the act of State doctrine.24

With regard to bringing suit against a State in the court of another State, as a general matter the defendant State would enjoy jurisdictional immunity, especially for acta jure imperii, or acts of a public and governmental nature.25 While number of exceptions exist, these generally relate to acta jure gestionis, or private, merchant-like and commercial acts of the State.26 There is one exception that in most instances does not expressly distinguish between acta jure imperii and acta jure gestionis: the territorial torts exception, which allows for claims to be brought against a foreign State arising out of death, personal injury or damage to property occurring in the territory of the forum State.27 However, the International Court of Justice held that under customary international law, the territorial torts exception does not apply to acts committed on the territory of the forum State by the armed forces of a foreign State in the course of conducting an armed conflict, even if those acts constitute war crimes or amount to jus cogens violations.28

For cases brought against a State in its own forum, “the action will most likely face an attitude of judicial abdication,” such as by invoking the act of State doctrine or political act doctrine.29 For example, when asked to adjudicate the behavior of the national government in the international sphere, US courts routinely cite the notion of the separation of powers for why the judiciary cannot consider cases involving the government’s foreign actions, since international relations are committed by the Constitution to the executive branch.30 Similarly, several courts have declined to exercise jurisdictions over the acts of its own State in cases relating to foreign affairs, arguing that in that context of foreign relations the State is a subject of the international law being applied, and that in the international legal arena the State must operate as one unitary entity over which a part of itself, the court, cannot pass judgement.31

Although the right to an effective remedy is enshrined in a number of human rights treaties, at least the European Court of Human Rights has deferred to the domestic courts’ right to determine the extent of its own power to review of acts of foreign policy such as acts of war, and has upheld the State’s decision to refuse to waive jurisdictional immunity in these cases.32

IV. Conclusion

As this and the prior post on autonomous weapons systems discuss, the introduction of the use autonomous weapons systems in warfare will present a number of legal challenges, and certain harms are at risk of going unpunished and unresolved without adjustments to current law. Because the most significant barriers arise out of the absence of intent, knowledge or foreseeability on the part of the closest human actor, a strict liability regime, as seen in the law governing ultra-hazardous conduct, product liability and the law on State responsibility, provides one of the few viable options for liability. Critics have expressed concern that strict liability may hinder research and development into potentially safer weapons systems, which have less of a likelihood of committing errors or engaging in malicious conduct when compared to humans. However, short of banning autonomous weapons systems or granting robots legal personality, certain concessions will likely need to be made in order to ensure that the law keeps up with newly-developing technologies that contain artificial intelligence software.

 

  1. Peter M. Asaro, “The Liability Problem for Autonomous Artificial Agents”, AAAI Publications, 2016 AAAI Spring Symposium Series, p. 191.
  2. See e.g. Restatement (Second) of Tort, §§ 281-283; French Civil Code, Art. 1382.
  3. See e.g. Dutch Civil Code, Art. 6:162(2).
  4. Dutch Civil Code, Art. 6:162(1); Restatement (Second) of Tort, § 430.
  5. Dutch Civil Code, Art. 6:162(1); Restatement (Second) of Tort, § 281(a).
  6. Woodrow Barfield, “Liability for Autonomous and Artificially Intelligent Robots”, 9(1) Paladyn, Journal of Behavioral Robotics 193, 201 (2018); Peter M. Asaro, “The Liability Problem for Autonomous Artificial Agents”, AAAI Publications, 2016 AAAI Spring Symposium Series, p. 192.
  7. Peter M. Asaro, “The Liability Problem for Autonomous Artificial Agents”, AAAI Publications, 2016 AAAI Spring Symposium Series, p. 192; see also Swati Malik, “Autonomous Weapon Systems: The Possibility and Probability of Accountability”, 35 Wisconsin International Law Journal 609, 627-28 (2018).
  8. See e.g. Restatement (Third) of Torts: Liability for Physical and Emotional Harm, §20; Dutch Civil Code, Arts. 6:173-77.
  9. Peter M. Asaro, “The Liability Problem for Autonomous Artificial Agents”, AAAI Publications, 2016 AAAI Spring Symposium Series, p. 193.
  10. See e.g. Dutch Civil Code, Art. 6:186(1); Restatement (Third) of Torts: Product Liability, § 2.
  11. Woodrow Barfield, “Liability for Autonomous and Artificially Intelligent Robots”, 9(1) Paladyn, Journal of Behavioral Robotics 193, 197 (2018).
  12. See Report From The Commission to the European Parliament, the Council and the European Economic and Social Committee on the Application of the Council Directive on the approximation of the laws, regulations, and administrative provisions of the Member States concerning liability for defective products (85/374/EEC), COM/2018/246 final.
  13. Council Directive 85/374/EEC of 25 July 1985 on the approximation of the laws, regulations and administrative provisions of the Member States concerning liability for defective products, Art. 7 (stating that there is no liability if the producer proves “that, having regard to the circumstances, it is probable that the defect which caused the damage did not exist at the time when the product was put into circulation by him or that this defect came into being afterwards.”).
  14. See Swati Malik, “Autonomous Weapon Systems: The Possibility and Probability of Accountability”, 35 Wisconsin International Law Journal 609, 629 (2018); Peter M. Asaro, “The Liability Problem for Autonomous Artificial Agents”, AAAI Publications, 2016 AAAI Spring Symposium Series, p. 192.
  15. Woodrow Barfield, “Liability for Autonomous and Artifically Intelligent Robots”, 9(1) Paladyn, Journal of Behavioral Robotics 193, 197-98 (2018).
  16. Daniel N. Hammond, “Autonomous Weapons and the Problem of State Accountability”, 15(2) Chicago Journal of International Law 652, 666 (2016).
  17. Rebecca Crootof, “War Torts: Accountability for Autonomous Weapons”, 164(6) University of Pennsylvania Law Review 1347, 1357 (2016).
  18. Daniel N. Hammond, “Autonomous Weapons and the Problem of State Accountability”, 15(2) Chicago Journal of International Law 652, 655 (2016).
  19. International Law Commission, Draft Articles on Responsibility of States for Internationally Wrongful Acts, November 2001, Art. 31(1).
  20. International Law Commission, Draft Articles on Responsibility of States for Internationally Wrongful Acts, November 2001, Art. 35.
  21. See Swati Malik, “Autonomous Weapon Systems: The Possibility and Probability of Accountability”, 35 Wisconsin International Law Journal 609, 630 (2018).
  22. Daniel N. Hammond, “Autonomous Weapons and the Problem of State Accountability”, 15(2) Chicago Journal of International Law 652, 655 (2016). The UNSC has never acted to enforce an ICJ judgement, despite its authority to do so. Daniel N. Hammond, “Autonomous Weapons and the Problem of State Accountability”, 15(2) Chicago Journal of International Law 652, 669 (2016).
  23. Daniel N. Hammond, “Autonomous Weapons and the Problem of State Accountability”, 15(2) Chicago Journal of International Law 652, 679 (2016).
  24. See e.g. Daniel N. Hammond, “Autonomous Weapons and the Problem of State Accountability”, 15(2) Chicago Journal of International Law 652, 682 (2016); see also Koohi v. US, 976 F.2d 1328 (9th Cir. 1992).
  25. See e.g. UN Convention on Jurisdictional Immunities of States and Their Respective Properties, Art. 5; United States Foreign Sovereign Immunities Act, § 1604; Jurisdictional Immunities of the State (Germany v. Italy), Judgment, ICJ, 3rd February 2012, para. 56 (finding that under customary international law, States enjoy jurisdictional immunity from suits in foreign States); see also European Convention on State Immunities, preamble.
  26. See e.g. UN Convention on Jurisdictional Immunities of States and Their Respective Properties, Art. 10; United States Foreign Sovereign Immunities Act, § 1605(a)(2); European Convention on State Immunities, Art. 7.
  27. Jurisdictional Immunities of the State (Germany v. Italy), Judgment, ICJ, 3rd February 2012, para. 64; see also UN Convention on Jurisdictional Immunities of States and Their Respective Properties, Art. 12; United States Foreign Sovereign Immunities Act, § 1605(a)(5); European Convention on State Immunities, Art. 11.
  28. Jurisdictional Immunities of the State (Germany v. Italy), Judgment, ICJ, 3rd February 2012, paras. 64-65; see also Lorna McGregor, “State Immunity and Human Rights Is There a Future after Germany v. Italy?”, 11 Journal of International Criminal Justice 125 (2013). The Court also observed that under both the UN Convention on Jurisdictional Immunities of States and Their Respective Properties and the European Convention on State Immunities, the territorial tort exception do not apply to situations of armed conflict. Jurisdictional Immunities of the State (Germany v. Italy), Judgment, ICJ, 3rd February 2012, paras. 68-69, 81-97.
  29. Francesco De Santis di Nicola, “Civil actions for damages caused by war crimes vs. State immunity from jurisdiction and the political act doctrine: ECtHR, ICJ and Italian Courts”, 2 International Comparative Jurisprudence 107, 108, 114-15 (2016); see also Daniele Amoroso, “Judicial Abdication in Foreign Affairs and the Effectiveness of International Law”, 14(1) Chinese Journal of International Law 99 (2015).
  30. Daniele Amoroso, “Judicial Abdication in Foreign Affairs and the Effectiveness of International Law”, 14(1) Chinese Journal of International Law 99, 105 (2015).
  31. Daniele Amoroso, “Judicial Abdication in Foreign Affairs and the Effectiveness of International Law”, 14(1) Chinese Journal of International Law 99, 110 (2015) (noting that this line of reasoning has especially been employed by civil law jurisdictions and referencing decisions from both French and Italian courts).
  32. See e.g. Case of Markovic and Others v. Italy, Judgment, ECtHR, 14th December 2006, para. 114.

Related articles

Tag cloud