Lethal autonomous weapon

From HandWiki
Short description: Autonomous military technology system
Serbian Land Rover Defender towing trailer with "Miloš" tracked combat robot

Lethal autonomous weapons (LAWs) are a type of autonomous military system that can independently search for and engage targets based on programmed constraints and descriptions. LAWs are also known as lethal autonomous weapon systems (LAWS), autonomous weapon systems (AWS), robotic weapons or killer robots. LAWs may operate in the air, on land, on water, underwater, or in space. The autonomy of current systems (As of 2018) was restricted in the sense that a human gives the final command to attack—though there are exceptions with certain "defensive" systems.

Being autonomous as a weapon

Being "autonomous" has different meanings in different fields of study. In engineering it may refer to the machine's ability to operate without human involvement. In philosophy it may refer to an individual being morally independent. In political science it may refer to an area's capability of self-governance. In terms of military weapon development, the identification of a weapon as autonomous is not as clear as in other areas.[1] The specific standard entailed in the concept of being autonomous can vary hugely between different scholars, nations and organizations.

Various people have many definitions of what constitutes a lethal autonomous weapon. The official United States Department of Defense Policy on Autonomy in Weapon Systems, defines an Autonomous Weapons Systems as, "A weapon system that, once activated, can select and engage targets without further intervention by a human operator."[2] Heather Roff, a writer for Case Western Reserve University School of Law, describes autonomous weapon systems as "armed weapons systems, capable of learning and adapting their 'functioning in response to changing circumstances in the environment in which [they are] deployed,' as well as capable of making firing decisions on their own."[3] This definition of autonomous weapon systems is a fairly high threshold compared to the definitions of scholars such as Peter Asaro and Mark Gubrud's definitions seen below.

Scholars such as Peter Asaro and Mark Gubrud are trying to set the threshold lower and judge more weapon systems as autonomous. They believe that any weapon system that is capable of releasing a lethal force without the operation, decision, or confirmation of a human supervisor can be deemed autonomous. According to Gubrud, a weapon system operating partially or wholly without human intervention is considered autonomous. He argues that a weapon system does not need to be able to make decisions completely by itself in order to be called autonomous. Instead, it should be treated as autonomous as long as it actively involves in one or multiple parts of the "preparation process", from finding the target to finally firing.[4][5]

Other organizations, however, are setting the standard of autonomous weapon system in a higher position. The British Ministry of Defence defines autonomous weapon systems as "systems that are capable of understanding higher level intent and direction. From this understanding and its perception of its environment, such a system is able to take appropriate action to bring about a desired state. It is capable of deciding a course of action, from a number of alternatives, without depending on human oversight and control - such human engagement with the system may still be present, though. While the overall activity of an autonomous unmanned aircraft will be predictable, individual actions may not be."[6]

As a result, the composition of a treaty between states requires a commonly accepted labeling of what exactly constitutes an autonomous weapon.[7]

Automatic defensive systems

The oldest automatically triggered lethal weapon is the land mine, used since at least the 1600s, and naval mines, used since at least the 1700s. Anti-personnel mines are banned in many countries by the 1997 Ottawa Treaty, not including the United States, Russia, and much of Asia and the Middle East.

Some current examples of LAWs are automated "hardkill" active protection systems, such as a radar-guided CIWS systems used to defend ships that have been in use since the 1970s (e.g., the US Phalanx CIWS). Such systems can autonomously identify and attack oncoming missiles, rockets, artillery fire, aircraft and surface vessels according to criteria set by the human operator. Similar systems exist for tanks, such as the Russian Arena, the Israeli Trophy, and the German AMAP-ADS. Several types of stationary sentry guns, which can fire at humans and vehicles, are used in South Korea and Israel. Many missile defence systems, such as Iron Dome, also have autonomous targeting capabilities.

The main reason for not having a "human in the loop" in these systems is the need for rapid response. They have generally been used to protect personnel and installations against incoming projectiles.

Autonomous offensive systems

Systems with a higher degree of autonomy would include drones or unmanned combat aerial vehicles, e.g.: "The unarmed BAE Systems Taranis jet-propelled combat drone prototype may lead to a Future Combat Air System that can autonomously search, identify, and locate enemies but can only engage with a target when authorized by mission command. It can also defend itself against enemy aircraft" (Heyns 2013, §45). The Northrop Grumman X-47B drone can take off and land on aircraft carriers (demonstrated in 2014); it is set to be developed into an Unmanned Carrier-Launched Airborne Surveillance and Strike (UCLASS) system.

According to The Economist, as technology advances, future applications of unmanned undersea vehicles might include mine clearance, mine-laying, anti-submarine sensor networking in contested waters, patrolling with active sonar, resupplying manned submarines, and becoming low-cost missile platforms.[8] In 2018, the U.S. Nuclear Posture Review alleged that Russia was developing a "new intercontinental, nuclear-armed, nuclear-powered, undersea autonomous torpedo" named "Status 6".[9]

The Russian Federation is actively developing artificially intelligent missiles,[10] drones,[11] unmanned vehicles, military robots and medic robots.[12][13][14][15]

Israeli Minister Ayoob Kara stated in 2017 that Israel is developing military robots, including ones as small as flies.[16]

In October 2018, Zeng Yi, a senior executive at the Chinese defense firm Norinco, gave a speech in which he said that "In future battlegrounds, there will be no people fighting", and that the use of lethal autonomous weapons in warfare is "inevitable".[17] In 2019, US Defense Secretary Mark Esper lashed out at China for selling drones capable of taking life with no human oversight.[18]

The British Army deployed new unmanned vehicles and military robots in 2019.[19]

The US Navy is developing "ghost" fleets of unmanned ships.[20]

In 2020 a Kargu 2 drone hunted down and attacked a human target in Libya, according to a report from the UN Security Council’s Panel of Experts on Libya, published in March 2021. This may have been the first time an autonomous killer robot armed with lethal weaponry attacked human beings.[21][22]

Ethical and legal issues

Standard used in US policy

Current US policy states: "Autonomous … weapons systems shall be designed to allow commanders and operators to exercise appropriate levels of human judgment over the use of force."[23] However, the policy requires that autonomous weapon systems that kill people or use kinetic force, selecting and engaging targets without further human intervention, be certified as compliant with "appropriate levels" and other standards, not that such weapon systems cannot meet these standards and are therefore forbidden.[24] "Semi-autonomous" hunter-killers that autonomously identify and attack targets do not even require certification.[24] Deputy Defense Secretary Robert Work said in 2016 that the Defense Department would "not delegate lethal authority to a machine to make a decision", but might need to reconsider this since "authoritarian regimes" may do so.[25] In October 2016 President Barack Obama stated that early in his career he was wary of a future in which a US president making use of drone warfare could "carry on perpetual wars all over the world, and a lot of them covert, without any accountability or democratic debate".[26][27] In the US, security-related AI has fallen under the purview of the National Security Commission on Artificial Intelligence since 2018.[28][29] On October 31, 2019, the United States Department of Defense's Defense Innovation Board published the draft of a report outlining five principles for weaponized AI and making 12 recommendations for the ethical use of artificial intelligence by the Department of Defense that would ensure a human operator would always be able to look into the 'black box' and understand the kill-chain process. A major concern is how the report will be implemented.[30]

Possible violations of ethics and international acts

Stuart Russell, professor of computer science from University of California, Berkeley stated the concern he has with LAWS is that his view is that it is unethical and inhumane. The main issue with this system is it is hard to distinguish between combatants and non-combatants.[31]

There is concern by some economists[32] and legal scholars about whether LAWs would violate International Humanitarian Law, especially the principle of distinction, which requires the ability to discriminate combatants from non-combatants, and the principle of proportionality, which requires that damage to civilians is proportional to the military aim.[33] This concern is often invoked as a reason to ban "killer robots" altogether - but it is doubtful that this concern can be an argument against LAWs that do not violate International Humanitarian Law.[34][35][36]

A 2021 report by the American Congressional Research Service states that "there are no domestic or international legal prohibitions on the development of use of LAWs," although it acknowledges ongoing talks at the UN Convention on Certain Conventional Weapons (CCW).[37]

LAWs are said by some to blur the boundaries of who is responsible for a particular killing.[38][39] Philosopher Robert Sparrow argues that autonomous weapons are causally but not morally responsible, similar to child soldiers. In each case, he argues there is a risk of atrocities occurring without an appropriate subject to hold responsible, which violates jus in bello.[40] Thomas Simpson and Vincent Müller argue that they may make it easier to record who gave which command.[41] Likewise, Steven Umbrello, Émile P. Torres and Angelo F. De Bellis argue that if the technical capacities of LAWs are at least as accurate as human soldiers, then given the psychological shortcomings of human soldiers in war warrants that only these types of ethical LAWs should be used. Likewise, they propose using the value sensitive design approach as a potential framework for designing these laws to align with human values and International Humanitarian Law.[35] Further, potential IHL violations by LAWs are – by definition – only applicable in conflict settings that involve the need to distinguish between combatants and civilians. As such, any conflict scenario devoid of civilians’ presence – i.e. in space or the deep seas – would not run into the obstacles posed by IHL.[42]

Political economists Christopher Coyne (professor) and Yahya Alshamy state concerns about LAWS incentivizing militarism and undermining peaceful solutions. They emphasize that humans are uniquely capable of engaging in the dynamic search for peace due to their morality and mortality. They also express concern over a potential arms race and the establishment of a precedent for acquiring LAWS. Whereas many scholars see proliferation as conducive to mutual deterrence by Mutual assured destruction, they state that for proliferation to be effective in deterring aggressive behaviors, refraining from using weapons must have some positive probability of breaking down. If there were no chance the weapons would be used, they would not serve to deter anyone, and there would be no point having them in the first place.[43]

Campaigns to ban LAWs

The possibility of LAWs has generated significant debate, especially about the risk of "killer robots" roaming the earth - in the near or far future. The group Campaign to Stop Killer Robots formed in 2013. In July 2015, over 1,000 experts in artificial intelligence signed a letter warning of the threat of an artificial intelligence arms race and calling for a ban on autonomous weapons. The letter was presented in Buenos Aires at the 24th International Joint Conference on Artificial Intelligence (IJCAI-15) and was co-signed by Stephen Hawking, Elon Musk, Steve Wozniak, Noam Chomsky, Skype co-founder Jaan Tallinn and Google DeepMind co-founder Demis Hassabis, among others.[44][45]

According to PAX For Peace (one of the founding organisations of the Campaign to Stop Killer Robots), fully automated weapons (FAWs) will lower the threshold of going to war as soldiers are removed from the battlefield and the public is distanced from experiencing war, giving politicians and other decision-makers more space in deciding when and how to go to war.[46] They warn that once deployed, FAWs will make democratic control of war more difficult - something that author of Kill Decision - a novel on the topic - and IT specialist Daniel Suarez also warned about: according to him it might recentralize power into very few hands by requiring very few people to go to war.[46]

There are websites protesting the development of LAWs by presenting undesirable ramifications if research into the appliance of artificial intelligence to designation of weapons continues. On these websites, news about ethical and legal issues are constantly updated for visitors to recap with recent news about international meetings and research articles concerning LAWs.[47]

The Holy See has called for the international community to ban the use of LAWs on several occasions. In November 2018, Archbishop Ivan Jurkovic, the permanent observer of the Holy See to the United Nations, stated that “In order to prevent an arms race and the increase of inequalities and instability, it is an imperative duty to act promptly: now is the time to prevent LAWs from becoming the reality of tomorrow’s warfare.” The Church worries that these weapons systems have the capability to irreversibly alter the nature of warfare, create detachment from human agency and put in question the humanity of societies.[48]

(As of March 2019), the majority of governments represented at a UN meeting to discuss the matter favoured a ban on LAWs.[49] A minority of governments, including those of Australia, Israel, Russia, the UK, and the US, opposed a ban.[49] The United States has stated that autonomous weapons have helped prevent the killing of civilians.[50]

No ban, but regulation

A third approach focuses on regulating the use of autonomous weapon systems in lieu of a ban.[51] Military AI arms control will likely require the institutionalization of new international norms embodied in effective technical specifications combined with active monitoring and informal ('Track II') diplomacy by communities of experts, together with a legal and political verification process.[52][53][54][55] In 2021, the United States Department of Defense requested a dialogue with the Chinese People's Liberation Army on AI-enabled autonomous weapons but was refused.[56]

See also

References

  1. Crootof, Rebecca (2015). "The Killer Robots Are Here: Legal and Policy Implications". 36 Cardozo L. Rev: 1837. https://heinonline.org/HOL/Page?collection=journals&handle=hein.journals/cdozo36&id=1943. 
  2. Allen, Gregory. "DOD Is Updating Its Decade-Old Autonomous Weapons Policy, but Confusion Remains Widespread". Center for Strategic and International Studies. https://www.csis.org/analysis/dod-updating-its-decade-old-autonomous-weapons-policy-confusion-remains-widespread. 
  3. Roff, Heather (2015). "Lethal Autonomous Weapons and Jus Ad Bellum Proportionality". https://scholarlycommons.law.case.edu/cgi/viewcontent.cgi?referer=&httpsredir=1&article=1006&context=jil. 
  4. Asaro, Peter (2012). "On Banning Autonomous Weapon Systems: Human Rights, Automation, and the Dehumanization of Lethal Decision-Making". Red Cross 687: 94}. https://www.cambridge.org/core/journals/international-review-of-the-red-cross/article/on-banning-autonomous-weapon-systems-human-rights-automation-and-the-dehumanization-of-lethal-decisionmaking/992565190BF2912AFC5AC0657AFECF07. 
  5. "Autonomy without Mystery: Where do you draw the line?" (in en-US). 1.0 Human. 2014-05-09. http://gubrud.net/?p=272. 
  6. "Unmanned aircraft systems (JDP 0-30.2)" (in en). https://www.gov.uk/government/publications/unmanned-aircraft-systems-jdp-0-302. 
  7. Krishnan, Armin (2016) (in en). Taylor & Francis Group. doi:10.4324/9781315591070. ISBN 9781317109129. https://www.taylorfrancis.com/books/9781317109129. Retrieved 2018-06-08. 
  8. "Getting to grips with military robotics". The Economist. 25 January 2018. https://www.economist.com/news/special-report/21735478-autonomous-robots-and-swarms-will-change-nature-warfare-getting-grips. 
  9. "US says Russia 'developing' undersea nuclear-armed torpedo". CNN. 3 February 2018. https://www.cnn.com/2018/02/02/politics/pentagon-nuclear-posture-review-russian-drone/index.html. 
  10. "Russia is building a missile that can makes its own decisions". 20 July 2017. http://www.newsweek.com/russia-military-challenge-us-china-missile-own-decisions-639926. 
  11. "Russia's digital doomsday weapons: Robots prepare for war - Russia Beyond". 2017-05-31. https://www.rbth.com/defence/2017/05/31/russias-digital-weapons-robots-and-artificial-intelligence-prepare-for-wa_773677. 
  12. "'Comrade in Arms': Russia is developing a freethinking war machine". 2017-08-09. https://www.rbth.com/defence/2017/08/09/comrade-in-arms-russia-is-developing-a-freethinking-war-machine_819686. 
  13. "Rise of the Machines: A look at Russia's latest combat robots". 2017-06-06. https://www.rbth.com/defence/2017/06/06/rise-of-the-machines-a-look-at-russias-latest-combat-robots_777480. 
  14. "Is Terminator back? Russians make major advances in artificial intelligence". 10 February 2016. https://www.rbth.com/science_and_tech/2016/02/10/is-terminator-back-russians-make-major-advances-in-artificial-intelligence_566553. 
  15. "Virtual trainer for robots and drones developed in Russia". 15 May 2017. https://www.rbth.com/news/2017/05/15/virtual-trainer-for-robots-and-drones-developed-in-russia_763016. 
  16. "Kara: I wasn't revealing state secrets about the robots". The Jerusalem Post. http://www.jpost.com/Israel-News/Politics-And-Diplomacy/Kara-I-wasnt-revealing-state-secrets-about-the-robots-482616. 
  17. Allen, Gregory. "Understanding China's AI Strategy". Center for a New American Security. https://www.cnas.org/publications/reports/understanding-chinas-ai-strategy. 
  18. "Is China exporting killer robots to Mideast?". Asia Times. 28 November 2019. https://www.asiatimes.com/2019/11/article/is-china-exporting-killer-robots-to-mideast/. 
  19. "British Army to operationally deploy new robots in 2019 | March 2019 Global Defense Security army news industry | Defense Security global news industry army 2019 | Archive News year". https://www.armyrecognition.com/march_2019_global_defense_security_army_news_industry/british_army_to_operationally_deploy_new_robots_in_2019.html. 
  20. "US Navy plans to build an unmanned Ghost Fleet". http://navyrecognition.com/index.php/news/defence-news/2019/march/6898-us-navy-plans-to-build-an-unmanned-ghost-fleet.html. 
  21. Hambling, David. "Drones may have attacked humans fully autonomously for the first time" (in en-US). https://www.newscientist.com/article/2278852-drones-may-have-attacked-humans-fully-autonomously-for-the-first-time/. 
  22. "Killer drone 'hunted down a human target' without being told to" (in en-US). 2021-05-29. https://www.foxnews.com/world/killer-drone-hunted-down-a-human-target-without-being-told-to. 
  23. US Department of Defense (2012). "Directive 3000.09, Autonomy in weapon systems". p. 2. http://www.dtic.mil/whs/directives/corres/pdf/300009p.pdf. 
  24. 24.0 24.1 Gubrud, Mark (April 2015). "Semi-autonomous and on their own: Killer robots in Plato's Cave". Bulletin of the Atomic Scientists. https://thebulletin.org/semi-autonomous-and-their-own-killer-robots-plato%E2%80%99s-cave8199. 
  25. Lamothe, Dan (30 March 2016). "Pentagon examining the 'killer robot' threat". Boston Globe. https://www.bostonglobe.com/news/nation/2016/03/30/the-killer-robot-threat-pentagon-examining-how-enemies-could-empower-machines/sFri6ZDifwIcQR2UgyXlQI/story.html. 
  26. Chait, Jonathan (2016-10-03). "Barack Obama on 5 Days That Shaped His Presidency". Daily Intelligencer. https://nymag.com/daily/intelligencer/2016/10/barack-obama-on-5-days-that-shaped-his-presidency.html. 
  27. Devereaux, Ryan; Emmons, Alex (2016-10-03). "Obama Worries Future Presidents Will Wage Perpetual, Covert Drone War". The Intercept. https://theintercept.com/2016/10/03/obama-worries-future-presidents-will-wage-perpetual-covert-drone-war/. 
  28. Stefanik, Elise M. (2018-05-22). "H.R.5356 - 115th Congress (2017–2018): National Security Commission Artificial Intelligence Act of 2018". https://www.congress.gov/bill/115th-congress/house-bill/5356. 
  29. Baum, Seth (2018-09-30). "Countering Superintelligence Misinformation". Information 9 (10): 244. doi:10.3390/info9100244. ISSN 2078-2489. 
  30. United States. Defense Innovation Board.. AI principles : recommendations on the ethical use of artificial intelligence by the Department of Defense. OCLC 1126650738. 
  31. Russell, Stuart (27 May 2015). "Take a stand on AI weapons". International Weekly Journal of Science 521. http://www.nature.com/news/robotics-ethics-of-artificial-intelligence-1.17611. 
  32. Coyne, Christopher; Alshamy, Yahya A. (2021-04-03). "Perverse Consequences of Lethal Autonomous Weapons Systems". Peace Review 33 (2): 190–198. doi:10.1080/10402659.2021.1998747. ISSN 1040-2659. https://doi.org/10.1080/10402659.2021.1998747. 
  33. Sharkey, Noel E. (June 2012). "The evitability of autonomous robot warfare*" (in en). International Review of the Red Cross 94 (886): 787–799. doi:10.1017/S1816383112000732. ISSN 1816-3831. https://www.cambridge.org/core/journals/international-review-of-the-red-cross/article/evitability-of-autonomous-robot-warfare/35D0C3294D834F23BF1C0B33FC51A166. 
  34. Müller, Vincent C. (2016). Autonomous killer robots are probably good news. Ashgate. pp. 67–81. http://philpapers.org/rec/MLLAKR. 
  35. 35.0 35.1 Umbrello, Steven; Torres, Phil; De Bellis, Angelo F. (2020-03-01). "The future of war: could lethal autonomous weapons make conflict more ethical?" (in en). AI & Society 35 (1): 273–282. doi:10.1007/s00146-019-00879-x. ISSN 1435-5655. https://doi.org/10.1007/s00146-019-00879-x. 
  36. Umbrello, Steven; Wood, Nathan Gabriel (2021-04-20). "Autonomous Weapons Systems and the Contextual Nature of Hors de Combat Status". Information 12 (5): 216. doi:10.3390/info12050216. 
  37. Kelley M. Sayler (June 8, 2021). Defense Primer: Emerging Technologies (Report). Congressional Research Service. https://fas.org/sgp/crs/natsec/IF11105.pdf. Retrieved July 22, 2021. 
  38. Nyagudi, Nyagudi Musandu (2016-12-09). "Doctor of Philosophy Thesis in Military Informatics (OpenPhD #openphd ) : Lethal Autonomy of Weapons is Designed and/or Recessive". https://works.bepress.com/nym/9/download/. 
  39. Coyne, Christopher; Alshamy, Yahya A. (2021-04-03). "Perverse Consequences of Lethal Autonomous Weapons Systems". Peace Review 33 (2): 190–198. doi:10.1080/10402659.2021.1998747. ISSN 1040-2659. https://doi.org/10.1080/10402659.2021.1998747. 
  40. SPARROW, ROBERT (2007). "Killer Robots". Journal of Applied Philosophy 24 (1): 62–77. doi:10.1111/j.1468-5930.2007.00346.x. ISSN 0264-3758. https://www.jstor.org/stable/24355087. 
  41. Simpson, Thomas W; Müller, Vincent C. (2016). "Just war and robots' killings". Philosophical Quarterly 66 (263): 302–22. doi:10.1093/pq/pqv075. http://philpapers.org/rec/SIMJWA. 
  42. Boulanin et al., "Limits on Autonomy in Weapon Systems", SIPRI & ICRC (2020): 37.
  43. Coyne, Christopher; Alshamy, Yahya A. (2021-04-03). "Perverse Consequences of Lethal Autonomous Weapons Systems". Peace Review 33 (2): 190–198. doi:10.1080/10402659.2021.1998747. ISSN 1040-2659. https://doi.org/10.1080/10402659.2021.1998747. 
  44. Zakrzewski, Cat (2015-07-27). "Musk, Hawking Warn of Artificial Intelligence Weapons". https://blogs.wsj.com/digits/2015/07/27/musk-hawking-warn-of-artificial-intelligence-weapons/. 
  45. Gibbs, Samuel (27 July 2015). "Musk, Wozniak and Hawking urge ban on warfare AI and autonomous weapons". The Guardian. https://www.theguardian.com/technology/2015/jul/27/musk-wozniak-hawking-ban-ai-autonomous-weapons. 
  46. 46.0 46.1 "Deadly Decisions - 8 objections to killer robots". p. 10. https://www.paxvoorvrede.nl/media/files/deadlydecisionsweb.pdf. 
  47. "Front page" (in en-US). Ban Lethal Autonomous Weapons. 2017-11-10. https://autonomousweapons.org/. 
  48. "Holy See renews appeal to ban killer robots". Catholic News Agency. November 28, 2018. https://www.catholicnewsagency.com/news/holy-see-renews-appeal-to-ban-killer-robots-74479. 
  49. 49.0 49.1 Gayle, Damien (29 March 2019). "UK, US and Russia among those opposing killer robot ban". http://www.theguardian.com/science/2019/mar/29/uk-us-russia-opposing-killer-robot-ban-un-ai. 
  50. Werkhäuser, Nina (27 August 2018). "Should 'killer robots' be banned?". Deutsche Welle (DW). https://www.dw.com/en/should-killer-robots-be-banned/a-45237864. 
  51. Bento, Lucas (2017). No Mere Deodands: Human Responsibilities in the Use of Violent Intelligent Systems Under Public International Law. https://dash.harvard.edu/handle/1/33813394. Retrieved 2019-09-14. 
  52. Geist, Edward Moore (2016-08-15). "It's already too late to stop the AI arms race—We must manage it instead". Bulletin of the Atomic Scientists 72 (5): 318–321. doi:10.1080/00963402.2016.1216672. ISSN 0096-3402. Bibcode2016BuAtS..72e.318G. 
  53. Maas, Matthijs M. (2019-02-06). "How viable is international arms control for military artificial intelligence? Three lessons from nuclear weapons". Contemporary Security Policy 40 (3): 285–311. doi:10.1080/13523260.2019.1576464. ISSN 1352-3260. 
  54. Ekelhof, Merel (2019). "Moving Beyond Semantics on Autonomous Weapons: Meaningful Human Control in Operation". Global Policy 10 (3): 343–348. doi:10.1111/1758-5899.12665. ISSN 1758-5899. 
  55. Umbrello, Steven (2021-04-05). "Coupling levels of abstraction in understanding meaningful human control of autonomous weapons: a two-tiered approach" (in en). Ethics and Information Technology 23 (3): 455–464. doi:10.1007/s10676-021-09588-w. ISSN 1572-8439. 
  56. Allen, Gregory C. (May 20, 2022). "One Key Challenge for Diplomacy on AI: China's Military Does Not Want to Talk" (in en). https://www.csis.org/analysis/one-key-challenge-diplomacy-ai-chinas-military-does-not-want-talk. 

Further reading