Drone Robots Al Mac
Content
1
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
Drone Robots
Drones out of human loop controversy.
Notes by Alister William Macintyre Some people are in denial that drones are being developed, which decide what to do, refuel themselves, repair themselves, communicate thru networks, without human direct controls. Think like NASA robots on Mars, autonomous from human control, feeding back info, occasionally accepting software updates. For those people who think autonomous drones are some kind of science fiction fantasy which won’t exist until distant future, I cite examples of how such robots do exist today. Using robots in warfare is controversial, given apparent lack of accountability, when under human control. There is a policy debate whether we need upgrades to laws and treaties. Last updated 2013 Mar 09
Version 5.19
Table of Contents Drone Robots Introduction (2 Dec 03) ............................................................................... 2 Autonomous Drones Debate (2 Dec 02) ............................................................................. 3 Autonomous Drones Exist (3 Mar 08) ! ..................................................................... 5 Relevant Terms Defined (3 Mar 09) ............................................................................... 6 Burglar Alarm involving drones + (3 Jan 07) ............................................................. 7 CIA+ definitions (3 Feb 08)........................................................................................ 8 Drone+ definitions (2 Dec 01) .................................................................................... 9 Future+ (2 Dec 02).................................................................................................... 10 Geneva+ (2 Dec 01) .................................................................................................. 10 GPS+ (2 Dec 02)....................................................................................................... 11 Latency+ definitions (3 Feb 28) ! ............................................................................. 12 SAWS+ fire and forget? (2 Nov 29) !....................................................................... 13 Sky Net+ and Parallels (3 Jan 10)............................................................................. 14 Semi-Autonomous Peru Drone Archeologist (3 Jan 10) .............................................. 16 USN Robot Guns (2 Nov 29)........................................................................................ 16 Israeli Iron Dome (2 Nov 29)........................................................................................ 16 Sky Net X47B lands on USN Aircraft Carrier (3 Mar 04) ! ......................................... 17 US Space Plane Drone X-37B (2 Dec 11) .................................................................... 17 US Military Drone future plans (2 Nov 28).................................................................. 17 Other sources (2 Nov 29) .............................................................................................. 18 HRW = Human Rights Watch (2 Nov 28) ! ..................................................................... 19 Lawfare on HRW (3 Feb 28) .................................................................................... 19 Allenby and Mattick (2 Dec 12) ................................................................................... 20 Anderson, Ken (2 Dec 25) ............................................................................................ 20 ASIL Autonomous Weapons Backgrounder (3 Jan 23) ............................................... 20 Bolton, Matthew IRAC Overview (2 Dec 12) .............................................................. 21
1 Drones (semi) autonomous out of human loop / Studies / Drones / folder
2
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
Center for New American Security (2 Dec 12) ............................................................ 23 Accountability Myth (2 Dec 12) ............................................................................... 24 CETMONS (2 Dec 12) ................................................................................................. 25 DoD (2 Dec 12)............................................................................................................. 26 Lawfare (2 Dec 03) ....................................................................................................... 26 Lawfare Matt & Ken (2 Dec 10)............................................................................... 26 Lawfare Concepts (2 Dec 12) ................................................................................... 28 Lawfare Reading List (2 Dec 12).............................................................................. 29 Lawfare Wittes and Malinowski (2 Dec 11)............................................................. 29 Harvard National Security Journal (3 Feb 18) .............................................................. 31 Red Cross, ICRC (2 Dec 12)......................................................................................... 31 Robinson, Paul U of Ottawa (2 Nov 29)....................................................................... 32 Accountability Complexity (2 Nov 29) .................................................................... 32 Singer, Peter on Robotics Revolution (2 Dec 12)......................................................... 33 Slate (2 Dec 12) ............................................................................................................ 34 SSRN = Social Science Research Network (2 Dec 03) ................................................ 34 USAF General Counsel Blog (2 Nov 28) !................................................................... 36 Drones and Accountability (2 Dec 12) ..................................................................... 36 Killer Robots Part 1 (2 Dec 12) ................................................................................ 36 Killer Robots Part 2 (2 Nov 29) ................................................................................ 37 Killer Robots Part 3 (2 Nov 29) ................................................................................ 37 Killer Robots Part 4 (2 Nov 29) ................................................................................ 38 Zenko, Micah (2 Dec 12) .............................................................................................. 38 Similar Treaties (2 Dec 12)............................................................................................... 40 Cluster Monition Treaty (2 Dec 12).......................................................................... 40 IHL in armed conflicts treaty (2 Dec 12).................................................................. 40 Land Mine Treaty (2 Dec 12) ................................................................................... 40 Civilized Human Principles (2 Dec 01) ............................................................................ 40 Laws of Robotics (2 Dec 03) .................................................................................... 40 Disaster Recovery (2 Dec 01) ....................................................................................... 41 Human Control Principle (2 Dec 01) ............................................................................ 41 Human Responsibility Principle (2 Dec 01) ................................................................. 42 Humanity Principle (2 Dec 01) ..................................................................................... 42 Killers Trade Ban (2 Dec 01)........................................................................................ 43 Killing with Non-Lethal Weapons (2 Dec 01).............................................................. 43
Drone Robots Introduction (2 Dec 03)
We are now moving towards killer robots, which will do their thing, without a human being in the decision loop. This raises many serious implications, which is the focus of these research notes. The first I have known of drones used in warfare was during WW II, when they were designed for weapons target practice, and German delivery of bombs to British cities, without pilots flying the delivery vehicles. Other people claim drones in warfare were not developed until many decades later. It is a matter of definition, whether various
2 Drones (semi) autonomous out of human loop / Studies / Drones / folder
3
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
rockets, aircraft which shoot other weapons, which do not have a pilot on board, should come under the same labeling, as the latest in a line of technological improvements. It is like saying the Wright Brothers did not have a airplane, because it was a technological toy, compared to the aircraft today, or that the first computers, which used punched cards and paper tape, are not really computers because they cannot compare to the digital computers of today. In Drone Issues, see chapter = Military Debates, for some issues associated with the development of killer robots, which can do their thing, without a human being in the loop. My drone documents have a version # in their naming, which I increment as I replace with re-writes. In parentheses after each heading is date in format (Year Month Day) when I last updated the contents of that chapter. So people with an earlier edition can see what has been updated since they last got a copy. Some chapter headings end in an exclamation point. These indicate either identification of a controversy, and-or links to what the facts may be. Other symbols may be added later. This helps correlation of conflicting info sources, in my further research. As I make significant additions and improvements to these research notes, I will upload revised editions various places on the Internet, for convenient access by other people who share some of my interests.1
Autonomous Drones Debate (2 Dec 02)
There is a debate raging about several related topics. 1. Persons with different professional and personal experience backgrounds can see different facets of areas of concern, while others are out of their imagination. 2. What is the status of autonomous and semi-autonomous drones, or killer robots without a human in the loop? There are conflicting stories about military plans, and I have seen video of autonomous drones, so I know they exist, even though many news stories and bloggers are in denial. I think some react to concepts saying this is science fiction, thus it is ludicrous to be even be debating this. Some of this comes from some people seeing early steps, military plans, realize possibilities, and speculate in a way, that others see as science fiction. 3. I have worked in midrange computing my entire career, and followed artificial intelligence (AI) intermittently. It seems to me that the state of art of software is heavily contaminated by mandates from non-technical managers, and leaders who believe computers are vastly superior in capabilities to reality. There is an astronomical risk here, not of Sky Net, but of mindless robots killing for no explicable reason. There have been incidents in the history of transportation and industrial automation, of robots killing humans, which sometimes took a while to
1
Scribd http://www.scribd.com/doc/115182066
3 Drones (semi) autonomous out of human loop / Studies / Drones / folder
4
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
figure out and solve.2 Fortunately in those past incidents, it was easy to divorce the machinery from the victims, while the diagnostics were conducted to figure out how to fix the problem. With killer robot drones, that might not be so easy. a. There are precedents of man made creations getting loose and doing serious damage, allegedly not intended by their creators. b. Consider what would happen if a civil engineer was modifying how a city’s sewage system worked, and made mistakes leading to every toilet in the city backing up, with fountain of recent human waste filling bathroom, spilling over into adjacent rooms of homes and offices. The responsible engineer would spend the rest of his life in prison, while his employer would go bankrupt, paying off the law suits. c. That is exactly what has happened with our Economic System. A group of financial engineers, working on Wall Street, and other places, made some modifications in how mortgages are tracked and invested in, encouraging a multi-year bubble in home prices, and mortgage securities, which ultimately led to a world wide Great Recession, an epidemic of foreclosures, massive unemployment, growth in homeless people, lost pensions, and the damage is continuing, such as in Europe. Their damage to the economic well being of the world, was just like my hypothetical engineer story. Yet instead of being held accountable, they earned multi million $ bonuses. They have created a mess so bad, that it is impossible for government regulators to put humpty dumpty back together again, to make their victims whole. 4. There are legal and ethical implications. Many people have written about them. In these notes, I identify some of those people, linking to their posts, and comment on some of what they have to say. There are huge questions about accountability. Who is to be held responsible when innocent people are killed or maimed, by one of these autonomous killer robot drones? a. Killer Bees are a plague on Central America, and spreading, with Global Warming. They are a man made disaster, with no one accountable. b. Mad Cow disease is now in many farm animals, wild animals, pets. It is a man made disaster, with no one accountable. c. Autonomous drones sounds like another man made disaster, with no one accountable. I wanted to split these notes into sections: Definitions of terminology, and popular expressions where there is some dispute what things really should be called. Artificial Intelligence capabilities for autonomous and semi-autonomous robots. State of Art, which has become unclassified, or we can find in media reports.
2
A common problem has been a lack of proper shielding for computer signals within devices, hearing signals from unrelated devices, which also lacked proper shielding. Thus: an industrial robot suddenly moves a heavy arm and kills people nearby; a high speed bullet train’s doors open, sucking many passengers out, when it passes apartment buildings with many children with hand held toys; an amusement park ride takes passengers to an unexpected location, at excessive speed, then crash; airport radar gets false signals, leading to a near miss of airliners.
4 Drones (semi) autonomous out of human loop / Studies / Drones / folder
5
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
Laws and Treaties which exist so far, which are relevant. Legal and Ethical implications, not yet satisfactorily addressed.
But the reality is that most all the reports out there, all the blogs, all the news stories, intermingle allegations and denials regarding a whole spectrum of such concepts, so it is not really practical to organize my notes that way. See my Drone Reports for: a directory of official documents, which I have located on drones; what I called my copy if I downloaded it; pointers to where I put notes if I digested the download; and links to other people discussing it.
Autonomous Drones Exist (3 Mar 08) !
There are people who believe that some day, the world will have what these notes are about, but that day is in the distant future. 3 Thus I have multiple examples in these notes showing that theoretical some day is really today, they really do exist. They are not yet Sky Net killing machines, but important technological advances are being made. They are also not exclusively in the realm of the military. 4 Here we see an impressive video of a robot plane navigating indoor obstacles at approx 22 mph, using on-board sensors, without GPS or a human pilot involved.5 It is yet another example of future spy technology able to get into some place that could be cut off from human direct controls. The next development needed for this little guy is stealth of vision and sound. It is unclear to me how this plane might cope with moving air of outdoors, or moving air of internal heating cooling conduits, or other moving objects if it got into an elevator shaft for example. I can imagine this technology being made ever smaller, into a drone smuggled into · an Iranian nuclear power plant, on behalf of USA or Israel; · Gitmo on behalf of some group planning a prison break; · The possibilities are endless. Military Robotics are on the rise,6 due to: political funding cut-backs; less staffing needed for complex missions; less money for recruiting, training, housing, feeding; insurance; they do not have an instinct for self-preservation, so they won’t lash out in fear, anger, recklessness;
3
http://singularityhub.com/2012/12/19/killer-robots-are-coming-ai-experts-to-determine-their-threattoward-humanity/ 4 http://www.technewsdaily.com/8423-autonomous-drones-not-just-military-tech-anymore-video.html 5 http://www.lawfareblog.com/2012/08/drones-keep-getting-better-and-better/ http://web.mit.edu/newsoffice/2012/autonomous-robotic-plane-flies-indoors-0810.html 6 http://icrac.net/2011/12/allvoices-the-ethical-and-moral-concerns-of-autonomous-robotic-systems/ http://www.allvoices.com/contributed-news/5223454-the-ethical-and-moral-concerns-of-autonomousrobotic-systems
5 Drones (semi) autonomous out of human loop / Studies / Drones / folder
6
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
belief they can fulfill the mission at lower cost (both financial, and harm to humans in our military);
Relevant Terms Defined (3 Mar 09)
This topic will have concepts in need of some new definitions. I should put them first in Drone Terms then here, with possible cross-pointer. From my Drone Terms:7 3d printed autonomous drones.8 911 for a Medical Drone: Here is an article about the development of a UAV which can be summoned by a smart phone, land in a battle zone, pick up an injured soldier and fly them back to base.9 10 AHAR = Autonomous High Altitude Refueling Basically, drones need the same kinds of services as manned flight.
o See Autonomous Drone. AI = Artificial Intelligence o According to an article in IRAC = International Committee for Robot Arms Control, QUOTE:11
“Research on artificial intelligence over the past 50 years has arguably been a contemporary Tower of Babel. While AI continues to be a rich field of study and innovation, much of its edifice is built upon hype, speculation, and promises that cannot be fulfilled. The U.S. military and other government agencies have been the leaders in bankrolling new computer innovations and the AI tower of babble, and they have wasted countless billions of dollars in the process. Buying into hype and promises that cannot be fulfilled is wasteful. Failure to adequately assess the dangers posed by new weapons systems, however, places us all at risk.
7
o UNQUOTE o See: Artificial Intelligence; Autonomous Drone; Needle in Haystack. American Society for the Prevention of Cruelty to Robots12
Drone Terms is my document with approx 1,000 concepts, terms, and acronyms defined, which are related to Drones, National Security, alleged misconduct by various government police agencies. 8 http://www.wired.com/design/2012/11/3d-printed-autonomous-airplane/ 9 http://www.wired.co.uk/magazine/archive/2012/12/start/autonomous-flight 10 http://aviationintel.com/2012/10/10/unmanned-game-changer-autonomous-aerial-refueling-is-a-reality/ 11 http://icrac.net/2013/03/moral-machines-guru-takes-moral-high-ground-on-autonomousweapons/ The IRAC article is a summary of a longer article here: http://scienceprogress.org/2013/01/terminating-the-terminator-what-to-do-about-autonomousweapons/ 12 Smithsonian Smart News (2013 Feb 13) - "Mistreated Robots Now Have a Advocacy Group": http://blogs.smithsonianmag.com/smartnews/2013/02/mistreated-robot-nowhave-a-advocacy-group/
Been around since 1999: http://www.aspcr.com/index.html -Gregory Foster || gfoster at entersection.org On [drone-list] https://mailman.stanford.edu/mailman/listinfo/drone-list
6 Drones (semi) autonomous out of human loop / Studies / Drones / folder
7
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
Artificial Intelligence (AI): Are Robots and Computers programmed, or can they learn by experience, being taught to do new things, without human input?13 AUSV = Autonomous Unmanned Surface Vehicle. o See: Autonomous Drone.
Autonomous Drone = A drone which does not require human involvement to pilot it, decide where it is to go, decide who it is to shoot at, refuel it, maintain it, update its programming.14 Some stories have alleged that some drones, in testing, go part way towards these goals, not yet achieve all of them in one craft. Other people think this definition is not clear enough. 15 o See: AHAR; AUSV; AWS; SAWS. AWS = Autonomous Weapon Systems. o QUOTING International Committee for Robot Arms Control (IRAC):16 o Department of Defense Directive 3000.0917 “for the development and use of autonomous and semi-autonomous functions in weapon systems” defining an AWS as “A weapon system that, once activated, can sel ect and engage targets without further intervention by a human operator.” The ability of the system to select and engage targets autonomously makes it an AWS even if it is human-supervised with a possible human override; thus “human on the loop” is assigned the same status as “human out of the loop,” a conservative (good) policy. o UNQUOTE o See: Autonomous Drone.
Burglar Alarm involving drones + (3 Jan 07)
Consider this scenario. A home is being burglarized. The burglar alarm causes a drone to take off and photograph the crooks, what they are doing, perhaps follow them until the police arrive.18 The drone needs to be able to remain aloft, out of range of the crooks trying to cause it to crash. This is an example of an autonomous drone, without a human in the loop, telling it what to do, and it is a scenario being made possible by the Japanese firm Secom.19
13 14
Cars which drive themselves, are an example of autonomous robots.20
http://www.redorbit.com/news/technology/1112787439/robot-icub-learns-a-new-language-022013/ Autonomous Drones http://blogs.cfr.org/zenko/2012/09/04/the-coming-future-of-autonomous-drones/ http://www.acq.osd.mil/dsb/reports/AutonomyReport.pdf 2012 Nov 28 I started a Drone Robots doc, on Autonomous Killer Robot Drones, what I have on it, in time may diverge from references in Drone Terms. 15 http://www.lawfareblog.com/2012/12/john-dehn-on-autonomous-robots-and-targeting/ 16 http://icrac.net/2012/11/dod-directive-on-autonomy-in-weapon-systems/ 17 I have downloaded this, naming it DOD Autonomous Weapon Policies 2012 Nov. 18 http://afgeneralcounsel.dodlive.mil/2013/01/04/we-reap-what-we-drone/ 19 http://phys.org/news/2012-12-japan-firm-private-drone.html 20 http://www.lawfareblog.com/2013/01/readings-bryant-walker-smith-automated-vehicles-are-probablylegal-in-the-united-states/
7 Drones (semi) autonomous out of human loop / Studies / Drones / folder
8
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
CIA+ definitions (3 Feb 08)
CIA = US Central Intelligence Agency. o While in general, we have Branches of Government, broken into Departments, broken down into Agencies and Offices, CIA is an exception. It is not in any Dept of US Government. It is an agency independent of all the departments. It reports to the DNI. o The CIA was originally a foreign intelligence service, but in recent years it has also become a paramilitary force. It is believed to have approx 30-40 drones, and is seeking to increase the size of its drone fleet. 21
o Allegedly all US drone attacks are carried out by either CIA or JSOC, under guidance by the President of the USA.22
o Stephen W. Preston, General Counsel of the Central Intelligence Agency, in an April 2012 speech at Harvard Law School on “CIA and the Rule of Law,” listed many applicable laws and procedures. See footnotes.23 o In Drone Terms, see: DIA; DNI; DoD; JSOC; NSA. Citizens United = A US Supreme Court Case which elevated corporations to have many of the same constitutional rights as human citizens.24 o See: Sky Net. CSER = A think tank–Centre for the Study of Existential Risk proposed by Cambridge University.25 DoD = US Dept of Defense26 HQ at the Pentagon. o DoD is one of the largest US Departments. It is composed of the various different types of military: Air Force; Army; Army Corps of Engineers; Navy; Marine Corps; Special Forces; National Guard and Reserve; Space Command; Missile Defense; Cyber Warfare; Weapons Research; Drones; Special units associated with WMD; Procurement of Weapons; Civilian Contractors, some Intelligence agencies; Regional Unified Commands, and more stuff. A small amount of this has been transferred to DHS.
21
http://www.smh.com.au/technology/technology-news/cia-seeks-expansion-of-drone-fleet-2012101927w0k.html 22 http://livingunderdrones.org/ INTERNATIONAL HUMAN RIGHTS AND CONFLICT RESOLUTION CLINIC (STANFORD LAW SCHOOL) AND GLOBAL JUSTICE CLINIC (NYU SCHOOL OF LAW), LIVING UNDER DRONES: DEATH, INJURY, AND TRAUMA TO CIVILIANS FROM US DRONE PRACTICES IN PAKISTAN (September, 2012) 23 CIA and the Rule of Law Synopsis https://www.fas.org/blog/secrecy/2012/09/targeted_killing.html I downloaded, calling my copy: CRS Kill US Citizen Legalities 2012 May. Full speech
http://www.cfr.org/rule-of-law/cia-general-counsel-stephen-prestons-remarks-rule-law-april 2012/p27912
24 25
https://en.wikipedia.org/wiki/Citizens_United_v._Federal_Election_Commission http://afgeneralcounsel.dodlive.mil/2013/02/07/stopping-killer-robots-and-other-threats-to-hjumansurvival/ http://www.foreignpolicy.com/articles/2013/02/04/doomsday_preppers_cambridge_existential_risks 26 http://www.defense.gov/ https://en.wikipedia.org/wiki/United_States_Department_of_Defense https://en.wikipedia.org/wiki/List_of_Department_of_Defense_agencies
8 Drones (semi) autonomous out of human loop / Studies / Drones / folder
9
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
o Here is 2009 Dod Drone Roadmap. (4 Meg)27 Raffaela Wakeman writes July 10, 2012 in LAWFARE Blog,28 QUOTE Conor Friedersdorf at The Atlantic29 looks closely at the 2009 Unmanned Systems Integrated Roadmap, the Pentagon’s plan for the use of drones from 2009 through 2034, and wonders whether we may get to a point when drones are completely autonomous. For the record, Missy Cummings of MIT says not in her lifetime.30 o UNQUOTE o In Drone Terms, see: CBP Suggestions; Departments of US Government; DHS; DIA; DNI; DSB; DSCA; DSS; GC Armas; ISE; JAG; NSA; PM-ISE. DoJ = US Dept of (civilian) Justice.31 Its mandate includes: ATF; Anti Trust; Criminal prosecutions, and also Civil violations; DEA; Interpol in USA; FBI; OIG; Prisons and Paroles; USMS o Some stuff which used to be in DoJ has been transferred to DHS. o In Drone Terms, see: AECA; ATF Scandals; Attorney General; BI; DEA; Departments of US Government; DoJ (more info than is here in Drone Robots); EOUSA; FBI; ISE; JAG; LIONS; OIG; OLC; PM-ISE; USAO; USMS.
Drone+ definitions (2 Dec 01)
There is a dispute about the proper use and application of some terminology. There are people who say Predators are not drones, but fail to define what they consider to be a drone. There are people who think drones are only those un-piloted vehicles which fly in the sky, don’t consider drones to be un -piloted vehicles which travel in the sea or on land. There are people who think model aircraft and other toys are not drones. I’d like to say that kites and balloons are not drones, but the problem is, for the people on the ground, you cannot tell if that thing in the sky has spy capability, or other modern technology which would qualify it as a drone. I consider a drone, any vehicle which is: Unmanned on board, but may be remotely piloted; Has the ability to spy on people, may be weaponized. This includes in the skies, in buildings, on land, on water, under water.
27 28
http://www.acq.osd.mil/psa/docs/UMSIntegratedRoadmap2009.pdf http://www.lawfareblog.com/2012/07/todays-headlines-and-commentary-222/ 29 http://www.theatlantic.com/politics/archive/2012/07/the-pentagons-vision-pervasive-drones-on-land-airand-sea/259541/ 30 http://www.lawfareblog.com/2012/03/lawfare-podcast-episode-5-missy-cummings-on-drones-dronesdrones/ However, I have seen news media reports, in 2012, of DoD testing drones which decide where to go, and what to do, based on their internal programming, instead of a human operator. Those reports shared in my Drone Notes. 31 http://www.justice.gov/ https://en.wikipedia.org/wiki/United_States_Department_of_Justice
9 Drones (semi) autonomous out of human loop / Studies / Drones / folder
10
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
Other people have other definitions. I think we may need alternative terms for various combinations of size, functions. The military has an army of acronyms. FOIA = Freedom of Information Act32 o A Citizen's Guide on Using the Freedom of Information Act and the Privacy Act of 1974 to Request Government Records, House Committee on Government Reform, Sept 20, 2005, House Report 109-226 (PDF)33. Fire and Forget o See: SAWS.
Future+ (2 Dec 02)
Future Shock = a psychological condition described in a 1970 book 34 by the futurist Alvin Toffler. People experiencing or witnessing too much change, in too short a period, can have difficulty grasping enough of it to deal effectively with it. They can become overwhelmed and disoriented. Toffler popularized the concept “information overload.” A documentary film based on the Future Shock book was released in 1972 with Orson Welles as on-screen narrator. o While this terminology arrived in the mid 20th century, the concept has existed all through history. o We see it in nations at risk of losing wars, because they failed to grasp technological change or new tactics fast enough. In WW II, the German Blitzkrieg used tactics and weapons which were not new, just used in a different mixture, with great surprise and shock for other nations whose leadership was not expecting this. o The 9/11 commission found over a dozen prior instances of terrorists flying hijacked planes into national monuments, or attempting to do so, many of which had previously been reported to national security leaders, but on the day of the attack, 99% of US leaders described the event as totally unexpected. o The 2011 Japanese tsunami was no surprise to tsunami experts, who had predicted exactly what risk was being taken at Fukushima and other nuclear power plants, whose management and government regulators had chosen to ignore the risks, then in the aftermath stated that the event was beyond their imagination. One possible explanation was future shock preventing wise planning of complex industry. o I am seeing many people suffering from future shock today, unable to have a rational conversation about national security legislation.
Geneva+ (2 Dec 01)
32 33
Geneva Convention(s),35 usually refers to international law treaties regarding humanitarian treatment of: prisoners, of war and armed conflicts (military and
https://www.fas.org/sgp/foia/foia2007.pdf https://www.fas.org/sgp/foia/citizen.html 34 https://en.wikipedia.org/wiki/Future_Shock 35 https://en.wikipedia.org/wiki/Geneva_Conventions
10 Drones (semi) autonomous out of human loop / Studies / Drones / folder
11
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
civilian), the wounded, sick, and ship-wrecked, civilians in a military conflict; prohibitions against the use of torture; etc. o The Geneva Conventions generally focus on treatment of people who are no longer combatants, or never were legal combatants, in a military conflict.36 o The Hague Conventions focus on the conduct of the war, prohibitions against certain types of weapons. o As with any international law, there’s the problem of nations which have not signed the agreements, and nations which choose not to comply with them. o Several nations, other than the USA, are saying that what the USA is doing with drones, is in violation of the Geneva Conventions, International Law, and should be labeled as War Crimes. o In Drone Terms, see: Hague Conventions; International Law; LOAC; Sovereignty; Targeted Killing; Unlawful Combatant; War Crime.
Gitmo = Short for Guantanamo Bay Cuba, a place the USA has been housing
detainees captured in the war on terror.
GPS+ (2 Dec 02)
GPS = Global Positioning System. This is a USA system. o GPS receivers receive almanac data from satellites, and also calculate position by calculating distance from visible satellites and using triangulation method to calculate its position. o GPS receivers also work on NMEA Standards. Most of the computer programs and devices, which provide position and other data given by the GPS receiver, includes info like position (latitude and longitude), altitude, speed, time etc.37 o Military Global Positioning System (GPS) signals have long been encrypted to prevent counterfeiting and unauthorized use. o Civil GPS signals were designed as an open standard, freelyaccessible to all. These virtues have made civil GPS enormously popular, but the transparency and predictability of its signals give rise to a dangerous weakness: they can be easily counterfeited, or spoofed. Like Monopoly money, civil GPS signals have a detailed structure but no built-in protection against counterfeiting.38
36
If the circling drones are used to terrify the citizens below into providing intelligence, that could be a violation of the Geneva Convention. Torture can lead to people giving out info, they think the torturers want to hear, such as the notion of WMD in Iraq. http://www.guardian.co.uk/commentisfree/2012/oct/23/truth-uk-drones-policy 37 http://www.scribd.com/doc/69421118/Drone-Mid-Year-Presentation-1 38 Congress House Transcript Drones in USA Skies 2012 July Panel I testimony by Mr. Todd E. Humphreys, Ph.D Abstract Summary = http://www.hsdl.org/?abstract&did=719102 Full 865k Transcript = https://www.hsdl.org/?view&did=719102 See Feb 2012 in Drone Dates for description of their spoofing demonstration.
11 Drones (semi) autonomous out of human loop / Studies / Drones / folder
12
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
o Hacking a UAV by GPS spoofing is but one expression of a larger problem: insecure civil GPS technology has over the last two decades been absorbed deeply into our national critical infrastructure. Besides UAVs, civil GPS spoofing also presents a danger to manned aircraft, maritime craft, communications systems, banking and finance institutions, and national power grid. o Satellite navigation systems other than GPS include: the Russian GLONASS system (fully operational), the European Galileo system (undergoing in-orbit validation of early spacecraft; may be operational by 2019), and the Chinese Compass system (global system in preliminary test phase). o Small, low power, inexpensive GPS + GLONASS receivers are now available off-the-shelf. These appear to be an excellent option for immediately improving navigation security in existing systems. o In Drone Terms, see: Bird Dog; NMEA; SCADA; UAV. The Hague Convention(s) focus on the conduct of the war, prohibitions against certain types of weapons, or used in certain ways. For example: o Article 25 of the 1907 Hague IV Convention prohibits aerial bombardment “by whatever means” of undefended towns, villages, or dwellings. Some drone strikes are in violation of this. o In Drone Terms, see: Datta Khel Nomada Bus Station, Geneva Convention; Sovereignty. HRW = Human Rights Watch, an NGO. o See section on their Nov 2012 report on autonomous killing machines. 39 IRAC = International Committee for Robot Arms Control o See: Autonomous Drone; AWS.
Latency+ definitions (3 Feb 28) !
Latency = movement shown on a drone pilot’s video screen has over the years been seconds behind what the drone sees — a delay caused by the time it takes to bounce a signal off a satellite in space. This problem, called “latency,” has long bedeviled drone pilots, making it difficult to hit a moving target. Sometimes the target hears the drone and can get to safety faster than the drone operator can do accurate shooting.40 This may explain some of the high proportion of collateral damage, and high cost of drone shooting, since wide dispersal bombs are needed to get at the escaping enemy, while manned combat could use a few bullets.
39
http://icrac.net/2012/11/dod-directive-on-autonomy-in-weapon-systems/ http://icrac.net/2011/12/new-scientist-campaign-asks-for-international-treaty-to-limit-war-robots/ http://www.popsci.com/military-aviation-amp-space/article/2009-09/no-nuke-or-space-military-botspleads-arms-control-committee http://blog.tmcnet.com/robotics/2009/09/new-group-aims-to-curb-military-use-of-robots.html 40 https://www.nytimes.com/2012/07/08/magazine/the-dronezone.html?pagewanted=2&_r=1&ref=magazine
12 Drones (semi) autonomous out of human loop / Studies / Drones / folder
13
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
Needle in Haystack41 from UPI42 is Anomaly Detection in universal data bases on IBM midrange platforms, using artificial intelligence. o The software teaches itself what is normal or anticipated via statistical analysis, then when something unusual occurs, it does an alert. Management can set thresholds for alerts. o In my opinion, this concept is needed on other kinds of computer platforms for all kinds of breaches and threats, including SCADA critical infrastructure and national security hacker activities. o In Drone Terms, see: AI; DBS; FIG-LEAF; OS; SCADA; UPI. NGO = Non-Government Organization, typically a not-for-profit humanitarian charity organization. Non-Lethal weapons is a frequent controversy, coming up again and again, with different people trotting out arguments we have heard many times before, 43 many of them answered before,44 although the weapons involved evolve over time. 45 There is no such thing as a non-lethal weapon. Every such weapon can be abused, to the point that the target people either get killed, or as seriously maimed, as if a lethal weapon was used. Petitions: several groups have petitions appealing for an end to some developments.46 Robot Governor – Still being designed, to give drones ethics desired by DoD and the rules of war. It needs to be able to do the many things required of humans, on the battlefield, to distinguish between real enemies and innocent bystanders.47 o In Drone Terms, see: DoD; Geneva Convention; LOAC. Robot Rights: when are they intelligent enough that we should grant them civil rights? How about half man half machine?48
SAWS+ fire and forget? (2 Nov 29) !
SAWS = Semi-autonomous weapon system
41 42
http://www.unbeatenpath.com/software/needle/in-a-haystack.pdf http://unbeatenpath.com/compass 43 http://www.lawfareblog.com/2012/12/the-icrc-and-slate-an-exchange-on-weapons/ 44 http://intercrossblog.icrc.org/blog/how-international-law-adapts-new-weapons-and-technologies-warfare
45
http://www.slate.com/articles/technology/future_tense/2012/11/nonlethal_weapons_and_the_law_of_war.h tml
46
http://secure.hrw.org/c.nlIWIgN2JwE/b.5571001/k.79D9/US_Stop_KIller_Robots/siteapps/advocacy/Actio nItem.aspx?msource=SKR20130220 http://www.lawfareblog.com/2013/02/human-rights-watch-campaign-on-killer-robots-and-tommalinowskis-response/ 47 http://www.allvoices.com/contributed-news/5223454-the-ethical-and-moral-concerns-of-autonomousrobotic-systems 48 http://www.lawfareblog.com/2013/02/issues-to-put-on-your-intellectual-radar-screens-cyborg-rights/ http://www.cyborgfoundation.com/
13 Drones (semi) autonomous out of human loop / Studies / Drones / folder
14
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
o The International Committee for Robot Arms Control (IRAC)49 says the DoD defines SAWS, as QUOTE: o “A weapon system that, once activated, is intended to only engage individual targets or specific target groups that have been selected by a human operator.” The category includes systems that automatically acquire, track, identify and prioritize potential targets or cue humans to their presence, “provided that human control is retained over the decision to select individual targets and specific target groups for engagement. ” It also includes weapons with terminal homing guidance, and “fire and forget” weapons where the target has been human -selected. o UNQUOTE o See: Autonomous Drone; AWS; DoD. “fire and forget” in the SAWS definition raises an issue of audit ability in my mind. In the capitalist world, that I am familiar with, there is change management control, and audit trails. We make changes to how software functions, then some time later a question is raised regarding the authority to do so, and when exactly it got changed. We can look in the software source code and find a change log: A brief summary of what the change was intended to accomplish. Identification of the names of company personnel who requested this change. Sometimes the name of a manager who approved their request. Identification of any people, outside of our company, who initiated this change request, e.g. auditors, law-suit, customers, vendors, new government regulations. Pointers at where in the source code updates were made to implement this change, using unique naming, where we can search for the precise code lines involved. Identification of names of personnel whose testing determined the software changes satisfactorily achieved what they had wanted. Likewise, we have annual auditors who show up, and ask questions about transactions which have occurred during the year. We can access history of activity to answer their questions. “fire and forget” sounds to me like any such audit trail could be lacking, if the military organization responsible for some drone, is later questioned regarding how come this or that target got fired upon..
RFID = Radio Frequency Identification.50
Sky Net+ and Parallels (3 Jan 10)
There is lots of dispute about the proper use and application of some terminology.
49 50
http://icrac.net/2012/11/dod-directive-on-autonomy-in-weapon-systems/ See discussion of implications within “Center for New American Security” such as a school in Texas mandating that students wear RFID chips, to help faculty keep track of the kids, but which also help kidnappers identify, and locate, valuable targets.
14 Drones (semi) autonomous out of human loop / Studies / Drones / folder
15
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
There is a military drone which calls itself Sky Net. There is a transportation delivery company which calls itself Sky Net. 51
Sky Net was popularized by the Terminator Science Fiction Movie series, and it is now
a TV serial. A lot of people are opposed to our government’s drone practices, either overall, or just some of them, and within those political ideologies, which includes antiwar, but is not that exclusively, when someone mentions autonomous robots, they often knee jerk react with Sky Net. Sky Net, as portrayed in the Terminator stories, is not even close, in technological development, in my opinion. However, it is a useful reference point, due to the huge volume of people in the public who equate military movement towards autonomous weapons, to be on the path to the coming of Sky Net. Sky Net, and the rise of the Machines, as depicted in the Terminator movies, will not be here for decades, but Sky Net Parallels are already here in human organizations, thanks to Citizens United,52 and the rise of the Lobbyists.
Sky Net Parallels are described well by Mark Gubrud53 in International Committee
for Robot Arms Control (IRAC)54 Analysis55 Nov 19, 2012:56 QUOTE Do I mean to invoke here the specter of Skynet, the artificial intelligence that declared war on humanity in The Terminator, reputedly because it feared we’d turn it off? Or Colossus, the military supercomputer that took control of the world in a brutal coup in order to fulfill its mission of ensuring peace? The scorn that “serious” people direct against these tropes from science fiction betrays their own nervousness. Artists have mined our apprehensions about the world we’re creating, and projected them before us in gaudy masks and cartoonish story lines that beg to be decoded. The military system is already a kind of machine, pursuing its own agenda, just as states, corporations and institutions of all kinds are. These machines are made of people, and their minds are the minds of people – increasingly augmented by information technology, from clay tablets to search engines. We like to think that this augmentation increases our effective intelligence, but as soon as words are written down, thinking rigidifies. Yet one essential fact remains: each human mind, dazzled and lost in the maze of knowledge, of law and the machinery of institutions, remains tethered intimately and existentially to a human heart. It is that tether which the military complex, enabled by technology, now threatens to break. UNQUOTE
51 52
http://www.skynet.net/ https://en.wikipedia.org/wiki/Citizens_United_v._Federal_Election_Commission 53 http://icrac.net/author/gubrud/ 54 http://icrac.net/ 55 http://icrac.net/category/analysis/ 56 http://icrac.net/2012/11/the-principle-of-humanity-in-conflict/
15 Drones (semi) autonomous out of human loop / Studies / Drones / folder
16
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
It is not just the corporations, with secret agendas, contrary to that of the common man. The Atlantic writes, Dec 14, 2012 QUOTE57 In recent decades, America's elite -- its elected officials, bureaucrats, and CEOs, for starters -- have succeeded spectacularly at insulating themselves from responsibility for their failures. As the global economy melted down, Wall Street got bailouts. CEOs who preside over shrinking companies still depart with "golden parachute" severance packages. The foreign-policy establishment remains virtually unchanged despite the catastrophic errors so many of its members made during the Iraq War. UNQUOTE
Semi-Autonomous Peru Drone Archeologist (3 Jan 10)
A drone, which can fit into a back pack, flew high in the Andes in Peru, for 3-D mapping of archaeological sites.58
USN Robot Guns (2 Nov 29)
USN = United States Navy The US Navy has an Aegis Combat System,59 which is capable of autonomously tracking enemy aircraft and guiding weapons onto them.60
Israeli Iron Dome (2 Nov 29)
I suspect, although I do not know for sure, that the Iron Dome system is like USN Aegis. Rockets come flying into Israel, in large numbers. Iron Dome radars detect them, calculate where they will land, if not intercepted, then shoot down the ones headed for population centers, with anti-rocket missiles which cost 150 times as much as the incoming missiles.61 Given the speed of decision making needed, I don’t see that having a human in the loop is essential for the perfect defense system. “Yes, shoot that down.” “Yes, shoot that down.” “Yes, shoot that down.” “Yes, shoot that down.” “Yes, shoot that down.” To infinity. Israel also has autonomous drones.62
57
http://www.theatlantic.com/politics/archive/2012/12/when-the-us-shoots-a-human-should-be-pulling-thetrigger/266254/ 58 https://www.scientificamerican.com/podcast/episode.cfm?id=drones-accelerate-archaeological-si-12-1231 59 http://www.navy.mil/navydata/fact_display.asp?cid=2100&tid=200&ct=2
60
http://www.slate.com/articles/technology/future_tense/2012/11/lethal_autonomous_robots_drones_tms_an d_other_military_technologies_raise.single.html 61 For more info on Iron Dome, see my Israel Gaza 2010 Nov Conflict notes. I have uploaded them various places, such as my Google Drive Disaster Avoidance Collection. https://docs.google.com/folder/d/0B9euafJH4bZMTA0YTM0YzktNTI0YS00NjVhLTg5NTItY2RiZjhiM2MzODkw/edit 62 http://defense-update.com/20121231_more-hermes-900-drones-for-idf.html
16 Drones (semi) autonomous out of human loop / Studies / Drones / folder
17
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
Sky Net X47B lands on USN Aircraft Carrier (3 Mar 04) !
USN = United States Navy. X-47B = a USN drone which is to operate without human control.63 Here are pictures of the X-47-B drone, allegedly flown without a human controlling it, trying to land on an aircraft carrier, and not quite making it. 64 I had heard that other drones in similar circumstances had already learned how to do this,65 so either these are old pictures, or the software which worked on other drones is not being shared, by the military, across drone manufacturers. The X47-B drone is stealth.66 The plan is for it to fly missions, by artificial intelligence (AI) and GPS, from air craft carriers.67 Contrary to some articles’ claims, we may never know which drone was the first to be piloted exclusively without a human in the loop. I suspect some time in WW II when Germany shooting at Britain with V1 and V2, until shortly after D-Day. I am surprised British news media forgot this. Then there is the indoor drone without human or GPS (which can be hacked – see Drone Terms.).68 UAV = Unmanned Air Vehicle, meaning no man on board, but men are needed to remotely operate it. I desire a shorter word than Autonomous, and Semi-Autonomous for Unmanned Unmanned Air Vehicle, meaning no men involved in remote operations, which has a clear meaning without the legal double speak of military for AWS and SAWS. For the time being, I will use Sky Net, because that’s the popular mis -conception in the anti-war movement.
US Space Plane Drone X-37B (2 Dec 11)
The X-37B bears a superficial resemblance to the space shuttle. What I have read about it so far is unclear whether it is an ordinary drone which can go into space, or if artificial intelligence is involved.69 There is a lot of sloppiness in descriptions. Not having a human pilot on board, is not the same as not having a human pilot period.
US Military Drone future plans (2 Nov 28)
DOD = Department of Defense Here from my Drone Reports:
63
http://icrac.net/2012/03/us-navy-developing-pilotless-drones/ http://www.bbc.co.uk/news/magazine-21576376 64 http://aviationintel.com/2012/11/28/x-47b-lands-on-aircraft-carrier-for-the-first-time-um-sort-of/ http://aviationintel.com/2012/12/11/x-47b-goes-to-sea/ 65 http://aviationintel.com/2012/12/20/need-an-interim-uav-solution-for-american-aircraft-carriers-take-agood-hard-look-at-the-mothballed-s-3-viking/ 66 http://www.dailymail.co.uk/sciencetech/article-2240394/X-47B-stealth-drone-hoped-carrier-borneunmanned-aircraft.html 67 http://gcn.com/articles/2012/12/11/navy-x47b-drone-first-military-robot.aspx 68 See chapter = Autonomous Drones Exist. 69 X-37-B http://aviationintel.com/2012/12/14/three-rocketsthree-historic-events-one-week/ http://aviationintel.com/2012/06/18/x-37b-returns-home-what-was-it-doing-up-there-for-over-a-year/ https://plus.google.com/u/0/107040353898400532534/posts/eTLzfhTDaNo
17 Drones (semi) autonomous out of human loop / Studies / Drones / folder
18
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
Congress House Transcript Drone Future Warfare 2010 March70 134 pages DOD Autonomous Drones 2012 July71 125 pages DOD Autonomous Weapon Policies 2012 Nov72 15 pages DOD Drone UAS Integrated Roadmap 200973 210 pages
Other sources (2 Nov 29) Here from my Drone Reports:
Evolution of Drone Warfare74 (7 pages) Brad Allenby and Carolyn Mattick explain why new technologies mean we need to rewrite the “rules of war.”75
70
2012 Oct 10 downloaded public domain. It is House Hearing Transcript 111-118 or PDF 64-921 Abstract Summary = http://www.hsdl.org/?abstract&did=4913 Full 4 Meg Transcript = https://www.hsdl.org/?view&did=4913 2012 Nov 28 I started a Drone Robots doc, on Autonomous Killer Robot Drones, which references this document, so what I have on it, in time may diverge from references in Drone Reports. 71 2012 Sep 05 I obtained this 125 page document by DSB = Defense Science Board http://www.acq.osd.mil/dsb/reports/AutonomyReport.pdf thanks to it being mentioned here: http://www.theatlantic.com/international/archive/2012/09/how-terrified-should-we-be-of-the-pentagonsplan-to-automize-drones/261944/ Also see Autonomous Drones in Drone Terms document. 2012 Sep 05 uploaded to my Google Drive Drone Info Document collection. 2012 Sep 05 uploaded to my Linked In / Box Net / Drone Info / Drone Sources sub-folder 2012 Oct 12+ Review: I have not yet read this. I tentatively plan to study DoD from smallest to largest. 2012 Nov 28 I started a Drone Robots doc, on Autonomous Killer Robot Drones, which references this document, so what I have on it, in time may diverge from references in Drone Reports. 72 2012 Nov 24 downloaded http://cryptome.org/dodi/dodd-3000-09.pdf 2012 Nov 28 I started a Drone Robots doc, on Autonomous Killer Robot Drones, which references this document, so what I have on it, in time may diverge from references in Drone Reports. There are many articles talking about this document, and related issues, such as: http://icrac.net/2012/11/dod-directive-on-autonomy-in-weapon-systems/ 73 DoD Drone roadmap as of 2009, 2012 July 15, or earlier, I had obtained this download. 2012 July 15 uploaded to my Google Drive Drone Info Document collection. 2012 Sep 03 uploaded to my Linked In / Box Net / Drone Info / Drone Sources sub-folder. 2012 Oct 08+ Review: I had barely begun studying this monster report, when I got distracted by many other smaller reports. I plan to delay return until I am caught up on the smaller ones, if that is possible. 2012 Nov 28 I started a Drone Robots doc, on Autonomous Killer Robot Drones, which references this document, so what I have on it, in time may diverge from references in Drone Reports. 74 2012 Oct 18 discovered at http://www.general-files.com/download/gs4d92955fh35i0/Evolution%20of%20Drone%20Warfare.html claimed to have come to them from Scribd 2012 Nov 28 I started a Drone Robots doc, on Autonomous Killer Robot Drones, which references this document, so what I have on it, in time may diverge from references in Drone Reports.
75
http://www.slate.com/articles/technology/future_tense/2012/11/drones_cyberconflict_and_other_military_t echnologies_require_we_rewrite.html
18 Drones (semi) autonomous out of human loop / Studies / Drones / folder
19
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
HRW = Human Rights Watch (2 Nov 28) !
Their report launched a debate, an important conversation, which previously had been much more low key, with less dimensions. Here from my Drone Reports: HRW76 Killer Robots Losing Humanity 2012 Nov77 55 pages The HRW report is accompanied by some articles.78
Lawfare on HRW (3 Feb 28)
Many great sources have more content on some topics than I find time to read and digest all of what they have. Feb 28, 2013, Lawfare’s Benjamin Wittes has another discussion of the HRW position.79 Dec 11, 2012, Lawfare shared a recommended reading list regarding recent discussion of Autonomous Robots.80 This included, QUOTE Human Rights Watch and Harvard Law School Human Rights Clinic, “Losing Humanity: The Case Against Killer Robots,” November 19, 2012. According to Lawfare, this report makes HRW’s case to ”[p]rohibit the development, production, and use of fully autonomous weapons through an international legally binding instrument. Adopt national laws and policies to prohibit the development, production, and use of fully autonomous weapons. Commence reviews of technologies and components that could lead to fully autonomous weapons. These reviews should take place at the very beginning of the development process and continue throughout the development and testing phases.” UNQUOTE
76 77
HRW = Human Rights Watch 2012 Nov 23 downloaded http://www.hrw.org/sites/default/files/reports/arms1112ForUpload_0_0.pdf from http://www.hrw.org/reports/2012/11/19/losing-humanity-0 and see press release http://www.hrw.org/news/2012/11/19/ban-killer-robots-it-s-too-late which I found out about thanks to http://afgeneralcounsel.dodlive.mil/2012/11/23/killer-robots-part-3/ and http://afgeneralcounsel.dodlive.mil/2012/11/26/killer-robots-part-4/ and http://www.cnas.org/blogs/abumuqawama/2012/11/rage-against-machines.html and http://www.lawfareblog.com/2012/11/human-rights-watch-report-on-killer-robots-and-our-critique/ many of which have some discussion of the merits of the report. 2012 Nov 28 I started a Drone Robots doc, on Autonomous Killer Robot Drones, which references this document, so what I have on it, in time may diverge from references in Drone Reports. 78 Ban ‘Killer Robots’ Before It’s Too Late | Human Rights Watch » http://www.hrw.org/news/2012/11/19/ban-killer-robots-it-s-too-late 79 http://www.lawfareblog.com/2013/02/greg-mcneal-on-human-rights-watch-and-killer-robots/ 80 http://www.lawfareblog.com/2012/12/readings-autonomous-weapon-systems-and-their-regulation/
19 Drones (semi) autonomous out of human loop / Studies / Drones / folder
20
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
Allenby and Mattick (2 Dec 12)
Brad Allenby and Carolyn Mattick jointly wrote an article, in Slate’s Future Tense explaining why they think we need new rules of war.81 Then other writers commented on their article, extending a debate. One of the responses, to the Allenby and Mattick article, came from ICRC,82 which Lawfare then commented further on.83 See separate Red Cross chapter section.
Brad Allenby is the Lincoln professor of engineering and ethics; a professor of civil,
environmental, and sustainable engineering; and the founding chairman of the Center for Earth Systems Engineering and Management at Arizona State University. 84 Brad Allenby wrote an article, in Slate’s Future Tense, explaining why he thinks Human Rights Watch does not understand military realities on the battlefield. 85
Carolyn Mattick is a Ph.D. candidate in environmental engineering and a graduate
fellow in emerging technologies at the Lincoln Center for Applied Ethics86 at Arizona State University.87
Anderson, Ken (2 Dec 25)
The USAF Ethics Blog informs us,88 Dec-16, 2012 QUOTE: In two posts, one on Opinio Juris89 and the other on LawFare,90 Kenneth Anderson has summarized leading articles, reports and blog posts on the issue of proposed limits on autonomous weapon systems. UNQUOTE I visited his brief biographies. I am familiar with all that he mentioned, and more. But the material, which he cites, is among the most important recent ingredients.
ASIL Autonomous Weapons Backgrounder (3 Jan 23)
ASIL = The American Society of International Law.
81
http://www.slate.com/articles/technology/future_tense/2012/11/drones_cyberconflict_and_other_military_t echnologies_require_we_rewrite.html 82 ICRC = International Committee of the Red Cross http://intercrossblog.icrc.org/blog/how-international-law-adapts-new-weapons-and-technologies-warfare 83 http://www.lawfareblog.com/2012/12/the-icrc-and-slate-an-exchange-on-weapons/ 84 http://www.slate.com/authors.brad_allenby.html
85
http://www.slate.com/blogs/future_tense/2012/11/20/human_rights_watch_s_case_against_killer_robots_re port_misunderstands_the.html 86 http://lincolncenter-dev.asu.edu/ 87 http://www.slate.com/authors.carolyn_mattick.html 88 http://afgeneralcounsel.dodlive.mil/2012/12/16/crs-on-detention-of-u-s-persons-as-enemy-belligerents/ 89 http://opiniojuris.org/2012/12/11/autonomous-weapon-systems-and-regulation-a-brief-bibliography/ 90 http://www.lawfareblog.com/2012/12/readings-autonomous-weapon-systems-and-their-regulation/
20 Drones (semi) autonomous out of human loop / Studies / Drones / folder
21
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
ASIL Insights = short, descriptive pieces on topical issues meant as non-technical “backgrounders” for journalists, the general public, and anyone looking for a quick path into an international law topic; they represent solely the author’s views, but are written to give an understanding of the background legal issues.) According to Lawfare Jan 19, 2013 QUOTE91 “The Law That Applies to Autonomous Weapon Systems” is written by Jeffrey S. Thurnher, a JAG officer on faculty at the Naval War College; it is short, crisp, and a useful guide to understanding the legal issues raised by the possibility of increasingly automated weapon systems that might one day be fully autonomous. (Also recommended is Major Thurnher’s more detailed Octobe r 2012 article in Joint Force Quarterly (National Defense University, Washington DC, Vol. 67, No. 4, Oct. 2012), “No One at the Controls: Legal Implications of Fully Autono mous Targeting.”) UNQUOTE According to Lawfare, this ASIL Insight piece is organized around two legal issues: 1. Is the weapon itself lawful? 2. Are there warfare conditions under which it would not be legal to use this weapon? According to Lawfare, this ASIL Insight piece also lists, and discusses, fundamental pieces of weapons law: 1. Under Geneva Conventions, weapons need to be able to discriminate between different targets. 2. Weapons should not cause unnecessary suffering or superfluous injury. It is Ok to kill people. It is not Ok to torture them.
Bolton, Matthew IRAC Overview (2 Dec 12)
Note: most of the hyper links got broken, as soon as I cut pasted them to these notes. Thus the importance of copying the relevant urls within footnotes. Dr. Matthew Bolton is assistant professor of political science at Pace University in New York City.92 He has worked as an academic, aid worker and a freelance journalist in 15 countries. He talks about similar treaties to what is being proposed. I started stubs near the bottom, for each of those treaties, so the references are conveniently placed, in case we later wish to expand upon them. M Bolton,93 of Political Mine Fields,94 in International Committee for Robot Arms Control (IRAC)95 Analysis96 Oct 3, 2012:97 QUOTE
91
http://www.lawfareblog.com/2013/01/readings-jeffrey-s-thurnher-on-law-of-armed-conflict-applied-toautonomous-weapon-systems/ 92 http://www.pace.edu/dyson/academic-departments-and-programs/political-science/faculty/matthewbolton 93 http://icrac.net/author/gubrud/ and
21 Drones (semi) autonomous out of human loop / Studies / Drones / folder
22
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
In the last few years there have been increasing calls for a global regulatory framework to govern robotic weapons.98 Oct-2, influential counter-terrorism correspondent Peter Bergen called for a treaty to manage and mitigate drone proliferation99. Earlier this year, along with Thomas Nash and Richard Moyes of the advocacy group Article 36, I called for a complete ban on autonomous armed robots,100 saying “Decisions to kill and injure should not be made by machines and, even if at times it will be imperfect, the distinction between military and civilian is a determination for human beings to make.” More conservative commentators have also recognized the need for some normative framework for robotic weapons.101 In 2011, an interdisciplinary group of legal, security and defense intellectuals from Consortium on Emerging Technologies, Military Operations, and National Security (CETMONS), argued in a paper for the Columbia Science and Technology Review that: UNQUOTE Then he goes on to quote some of it. I want to go to the original and study for myself. I have a backlog of that type of wishing. See separate CETMONS section. M Bolton 102 continues in IRAC103 Oct 3, 2012:104 QUOTE The 2009 statement by International Committee for Robot Arms Control (ICRAC), which called for bans on autonomous weapons, the arming of robotic weapons with nuclear arms and the deployment of armed robots in space 105 is probably one of the most developed proposals I have seen. As a member of ICRAC,106 I have endorsed this statement. However, I have been considering ways to flesh out the ICRAC proposal with further provisions building on the precedents of the landmine107 and cluster munition treaties,108 as well as the draft Arms Trade Treaty109 (currently still being negotiated) and the broader corpus of international humanitarian law governing conduct in war.110
http://politicalminefields.com/aboutme/ 94 http://politicalminefields.com/aboutme/ 95 http://icrac.net/ 96 http://icrac.net/category/analysis/ 97 http://icrac.net/2012/10/debating-proposals-on-a-possible-international-convention-on-robotic-weapons/ 98 http://wiredforwar.pwsinger.com/ 99 http://politicalminefields.com/2012/10/01/bergen-time-for-drone-treaty/ 100 http://www.article36.org/statements/ban-autonomous-armed-robots/ 101 https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2046375 I downloaded this “Law and Ethics for Robot Soldiers … see the chapter SSRN = Social Science Research Network. 102 http://icrac.net/author/gubrud/ 103 http://icrac.net/ 104 http://icrac.net/2012/10/debating-proposals-on-a-possible-international-convention-on-robotic-weapons/ 105 http://icrac.net/statements/ 106 http://icrac.net/who/ 107 http://www.icbl.org/index.php/icbl/Treaty 108 http://www.stopclustermunitions.org/the-solution/the-treaty/ 109 http://politicalminefields.com/2012/07/26/new-draft-of-the-arms-trade-treaty/ 110 http://www.icrc.org/eng/war-and-law/index.jsp Red Cross
22 Drones (semi) autonomous out of human loop / Studies / Drones / folder
23
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
Yesterday, I published an article (for Global Policy’s online Comment & Opinion section)111 and a post on my blog112 outlining possible provisions for an International Convention on Robotic Weapons. I am a political scientist, not a lawyer or roboticist, so my purpose in doing this is to provoke discussion and dialogue, not to pin down precise language. Moreover, while I have used the ICRAC statement as a foundation, this is my own personal adaptation and expansion of it and so should not be interpreted as an institutional position of ICRAC. UNQUOTE
Center for New American Security (2 Dec 12)
Many other sources are citing this source, which after I had read some of their articles, I conclude they are living in an alternate reality quite divorced from my own, like they are on a different planet, or have not been paying attention to the history of capitalism, government policies. Nov 21 2012, they have a “rage against the machines” write -up about the HRW report.113 QUOTE They find it a logically ludicrous concept. They say that no weapon is yet fully capable of taking humans out of the loop . UNQUOTE On the contrary, elsewhere in my research notes I share114 indications that pieces of such weapons are: In current usage;115 being tested; and I suspect that more advanced versions are military secrets. In my Drone Notes there is a chapter listing Technological Advances. In there is a citation to a drone, which looks like an insect, and is smaller than a person’s thumb. You have to be on G+ to be able to access the links.116 QUOTE This insect spy drone was developed for use in urban areas, and is already in production. Funded by the US Government, it can be remotely controlled, and is equipped with a camera and a microphone. It can land on you, and it may have the potential to take a DNA sample or leave RFID tracking nanotechnology on your skin, or be used in assassinations. UNQUOTE
111
http://www.globalpolicyjournal.com/blog/02/10/2012/time-ban-killer-robots-considering-internationalconvention-robotic-weapons 112 http://politicalminefields.com/2012/10/02/proposal-for-an-international-convention-on-robotic-weapons/ 113 http://www.cnas.org/blogs/abumuqawama/2012/11/rage-against-machines.html 114 As I come across more, I may add here. 115 See USN Robot Guns chapter. 116 https://plus.google.com/u/0/101378900168144468134/posts/W7v5pTCG4vd https://plus.google.com/u/0/103028459671171670815/posts/G2XhvRbtd4Y
23 Drones (semi) autonomous out of human loop / Studies / Drones / folder
24
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
I suspect this insect drone is semi-autonomous, in that humans would tell it who to target, it could go out of range of controller, then return to report, and get new instructions. When I was looking into the RFID controversy,117 I learned that the distance a radio signal can travel between some device and its network, is related to how large the radio antenna is, and how large the radio device. It is a limitation imposed by the laws of physics. This is why normal bar code scanners need to be within 6 inches of the bar code they are reading (there's also an RFID in there which is getting updated). We have a state of art today where new technology comes along, which is not governed by any legislation, spends a decade or more abusing people, then there is a battle thru the courts and legislatures to reign in that abuse, and an eternity putting a stop to it, after the leaders have ruled. Look at the police arresting people for using cameras in public, for decades after such behavior was legalized. Wouldn’t it be better to have a framework of civility defined before the new technology gets implemented? The Nov 21 2012 Center for New American Security “rage against the machines” write up about the HRW report, has many other reasons why they think this is ludicrous, such as military commanders would not want autonomous robots not subject to chain of command. I guess the author has missed a half century of history where millions of blue collar jobs have been automated, and also many white collar jobs eliminated by electronics, so he does not comprehend how the very top command wants the job done, by the cheapest possible means, so if eliminating humans from the loop is cheaper, then so be it. Dec 12 2012, they have an article on the relative silence in US media about what US drones are doing around the world.118
Accountability Myth (2 Dec 12)
The Nov 21 2012 Center for New American Security “rage against the machines” write up about the HRW report, says:119 QUOTE
117
RFID tags are physically almost microscopic, not visible to the naked eye. They are placed on and in all sorts of merchandise in their supply chain, for a variety of perfectly valid business tracking purposes. Some people are concerned about the ability to spy upon end consumers, through RFID tags still active after they take merchandise home. RFID tags are also injected into various farm animals and pets, also for legitimate purposes. There is a fear that they might also be injected into humans for illegitimate purposes. There are authorities who are promoting their use in ways which would place people at greater risk. I believe this is because they have not thought through security implications. Other observers speculate more sinister motives for the proposals. A Texas school mandates students have RFID tag on them at all times, http://www.bbc.co.uk/news/technology-20461752 which at least one student is protesting on alleged religious grounds. https://www.rutherford.org/files_images/general/11-21-2012_TRO-Petition_Hernandez.pdf School tried to suspend student for refusing to wear student ID/tracking device, but this effort was reversed by a court. http://www.techdirt.com/articles/20121125/15041521137/court-temporarilyblocks-school-district-suspending-student-refusing-to-wear-student-idtracking-device.shtml 118 http://www.cnas.org/blogs/abumuqawama/2012/12/silence-and-drones.html 119 http://www.cnas.org/blogs/abumuqawama/2012/11/rage-against-machines.html
24 Drones (semi) autonomous out of human loop / Studies / Drones / folder
25
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
The distinction HRW draws between “H uman-on-the-Loop Weapons” and “Human-out-of-the-loop” weapons is totally arbitrary, particularly since one key criteria HRW draws is that “human-on-the-loop” weapons have human oversight which can override and veto their actions, and “human -out-of-the-loop” weapons do not. UNQUOTE I am troubled by this belief that “human-on-the-loop” weapons have human oversight which can override and veto their actions. The belief does not understand latency, and is ignoring the reality of court cases around the world, particularly in Pakistan and Britain, and FOIA in the USA, demanding to know the basis for which this or that allegedly innocent family was killed, and the US military and CIA and DoJ basically giving the finger to those requests.120 There is also the lack of accountability with respect to the basis, for which military decisions are made, to target this or that alleged enemy. Remember WMD in Iraq? We still do not know, for sure, where that info came from. Theories include: Innocent suspects being tortured, who told their torturers what they thought the torturers wanted to hear, to stop the torture. Freedom Fighters in Iraq, who knew they could not over-throw Saddam dictator by themselves, but if USA would believe this lie, then USA would over-throw Saddam for them. Were there really WMD in Iraq, which are now some place else, or was this an intelligence and policy failure? Suspicions abound that similar intelligence sources and methods are being used as input to choice of who should be killed by drones, where they should be targeted. The cost, of acting on false intelligence, can be enormous.
CETMONS (2 Dec 12)
In 2011, an interdisciplinary group of legal, security and defense intellectuals from Consortium on Emerging Technologies, Military Operations, and National Security (CETMONS), argued in a paper for the Columbia Science and Technology Review… See IRAC Matthew Bolton. I downloaded the CETMONS report, naming it: STLR121 Killer Robot Governance 2011122 44 pages o Cite as http://www.stlr.org/cite.cgi?volume=12&article=7 o This work is made available under the o Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 License123
120 121
See Relevant Terms Defined for many concepts and acronyms etc. found in these notes. STLR = Science and Technology Law Review at Columbia. www.stlr.org 122 2012 Dec 1 downloaded http://www.stlr.org/html/volume12/marchant.pdf finding out about it from http://icrac.net/2012/10/debating-proposals-on-a-possible-international-convention-on-robotic-weapons/ 2012 Nov 28 I started a Drone Robots doc, on Autonomous Killer Robot Drones, which references this document, so what I have on it, in time may diverge from references in Drone Reports. 123 https://creativecommons.org/licenses/by-nc-nd/3.0/
25 Drones (semi) autonomous out of human loop / Studies / Drones / folder
26
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
DoD (2 Dec 12)
DoD = US Department of Defense. Dec 11, 2012, Lawfare shared a recommended reading list regarding recent discussion of Autonomous Robots.124 This included, QUOTE Ashton B. Carter, Deputy Secretary of Defense, “Autonomy in Weapons Systems,” Department of Defense Directive, Number 3000.09, November 21, 2012.125 According to Lawfare, the Directive establishes “DoD policy and assigns responsibilities for the development and use of autonomous and semi-autonomous functions in weapon systems, including manned and unmanned platform s.” It further “establishes guidelines designed to minimize the probability and consequences of failures in autonomous and semi-autonomous weapon systems that could lead to unintended engagements.” A key concept in the document – which is designed to be applicable across many different kinds of weapons systems over time and during their development – is that the “appropriate” level and kind of human involvement in the system will depend upon many factors and circumstances. UNQUOTE
Lawfare (2 Dec 03)
I am following several blogs, which quote extensively from each other. I would rather have a chapter heading on each source, than duplicate the material. This approach encourages study of original sources, instead of just what someone else chooses to quote. Sept 12, 2012, in Defense Science Board on Autonomous Systems,126 they briefly quote from the 125 page report.
Lawfare Matt & Ken (2 Dec 10)
Nov 26 2012, in their Critique of Human Rights Watch Report on Killer Robots,127 Written by Matt and Ken, QUOTE
124 125
http://www.lawfareblog.com/2012/12/readings-autonomous-weapon-systems-and-their-regulation/ 2012 Sep 05 I obtained this 125 page document http://www.acq.osd.mil/dsb/reports/AutonomyReport.pdf thanks to it being mentioned here: http://www.theatlantic.com/international/archive/2012/09/how-terrified-should-we-be-of-the-pentagonsplan-to-automize-drones/261944/ Also see Autonomous Drones in Drone Terms document. 2012 Sep 05 uploaded to my Google Drive Drone Info Document collection. 2012 Sep 05 uploaded to my Linked In / Box Net / Drone Info / Drone Sources sub-folder 2012 Oct 12+ Review: I have not yet read this. I tentatively plan to study my many DoD downloads from smallest to largest. 2012 Nov 28 I started a Drone Robots doc, on Autonomous Killer Robot Drones, which references this document, so what I have on it, in time may diverge from references in Drone Reports. 126 http://www.lawfareblog.com/2012/09/defense-science-board-on-autonomous-systems/ See in US Military drone future plans and Drone Reports, that I have downloaded this report. 127 http://www.lawfareblog.com/2012/11/human-rights-watch-report-on-killer-robots-and-our-critique/ See in Human Rights Watch Chapter and Drone Reports, that I have downloaded HRW report.
26 Drones (semi) autonomous out of human loop / Studies / Drones / folder
27
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
Last week, Human Rights Watch (along with the Harvard Law School International Human Rights Clinic) published a report titled “Losing Humanity: The Case Against Killer Robots.” It argues for a preemptive prohibition by multilateral treaty on the development and use of “fully autonomous weapons that could select and engage targets without human intervention” — a technological feat likely to be possible a few decades from now (and, in fact, already possible in some very limited forms). Certainly the direction of weapons development is toward greater automation, and this raises a legitimate concern about whether highly automated or autonomous weapons systems would comply with the laws of war. While HRW’s new report is an important contribution to this debate – containing useful background discussion of emergent precursor systems currently in use or under development – we think it rests on some questionable premises both factual and moral, and reaches some overly sweeping and unwise conclusions. UNQUOTE The critique has a US military centric viewpoint, which fails to appreciate the arms race around the world.128 There are always nations that the US is in a conflict with, where the US cannot be in a leadership position, so long as we are on the brink of war with each other. I believe there needs to be treaty negotiation thru the UN. I agree that treaties can pre-judge science, as we have seen with peaceful uses of outer space, opposition to stem cells, cloning, research banned in one nation, becomes a multi-million dollar industry in another. But treaties, like laws, can and should be rethought as technology advances. An important challenge is the issue of holding individuals criminally liable for war crimes when lethal systems operate autonomously. I agree that International Law Courts have failed to live up to their expectations, but this is because most nations of the world give the finger to their rulings, when they consider their national interests more important. Nov 27 2012, in Pentagon Directive on “Autonomy in Weapon Systems” »129 QUOTE Speaking of autonomous weapons systems, which Matt and Ken130 and Human Rights Watch131 all were, the Department of Defense has issued a directive entitled “Autonomy in Weapon Systems.”132 It declares: Then they quote some of it. UNQUOTE
128
See my Drone Nations which documents that drones are active in over 100 nations in our world, with many serious adversaries competing in a drone technology arms race. 129 http://www.lawfareblog.com/2012/11/pentagon-directive-on-autonomy-in-weapon-systems/ 130 See Nov 26 2012 from Lawfare. 131 See in Human Rights Watch Chapter and Drone Reports, that I have downloaded HRW report. 132 See in US Military drone future plans and Drone Reports, that I have downloaded this report. See DoD chapter section.
27 Drones (semi) autonomous out of human loop / Studies / Drones / folder
28
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
Dec 01, 2012 in Does Human Rights Watch Prefer Disproportionate and Indiscriminate Humans to Discriminating and Proportionate Robots?133 QUOTE To reach this conclusion, the report’s authors m ake a lot of assumptions about the technology that may well prove wrong QUOTE Al Mac Observation: It seems to me that a lot of authors are making assumptions, without any evidence to substantiate their positions. I have seen human nature in the work place since the 1960’s. Human nature is not going to change, unless there are government regulations, and audits, with good oversight. I agree that over time, technology development may get some things functioning better than human decision-making. But rather than: HRW position that because, right now, humans can do it better, but as I have noted elsewhere, often do not,134 that we should ban the inferior machines. Opposition position that has confidence in technology development, so there should be no such ban. I think there could be standards: Do not deploy said technology until it achieves specified standards.
Lawfare Concepts (2 Dec 12)
Dec 11, 2012, Lawfare came out with a recommended reading list regarding recent discussion of Autonomous Robots, which included:135 QUOTE William Marra and Sonia McNeil, “Understanding “The Loop’: Regulating the Next Generation of War Machines,” 36 Harvard Journal of Law and Public Policy 3 (2013), Lawfare Research Paper Series 1-2012. According to Lawfare, this paper (which inaugurated the Lawfare Research Paper Series in May 2012) provides a detailed explanation of the distinct technical meanings of “automation,” “autonomy,” “The Loop,” an d other key concepts essential to intelligently regulating gradually automating weapons systems, using the evolution of today’s rapidly automating drones as an example. The paper brings together international law, US government regulation and weapons review processes, engineering concepts, and military strategy in order to yield a precise, common vocabulary for debating the proper forms and modes of regulation of incrementally automating weapons systems. In the language of engineers, say Marra and McNeil, “tomorrow’s drones are expected to leap from ‘automation’ to true ‘autonomy’.” As a consequence, regulations for today’s drones must be “crafted with an eye towards tomorrow’s technologies.
133
http://www.lawfareblog.com/2012/12/does-human-rights-watch-prefer-disproportionate-andindiscriminate-humans-to-discriminating-and-proportionate-robots/ 134 See for example, the Feb 2010 attack in Drone Dates, where US military attacked children by mistake, then waited 12 hours before reporting it. 135 http://www.lawfareblog.com/2012/12/readings-autonomous-weapon-systems-and-their-regulation/
28 Drones (semi) autonomous out of human loop / Studies / Drones / folder
29
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
Yet today’s debates about humans and ‘the loop’ rely on langua ge too imprecise to successfully analyze the relevant differences between drones and predecessor technologies.” They conclude that needless confusion and possibly policy error pervade discussions about “when an advanced technological system is autonomous and what the implications of autonomy might be … language useful to the policymaking process has already been developed in the same places as drones themselves — research and engineering laboratories around the country and abroad.” UNQUOTE
Lawfare Reading List (2 Dec 12)
Dec 11, 2012, Lawfare came out with a recommended reading list regarding recent discussion of Autonomous Robots.136 QUOTE
Ashton B. Carter, Deputy Secretary of Defense, “Autonomy in Weapons Systems,” Department of Defense Directive, Number 3000.09, November 21, 2012. See DoD chapter section for more info. Human Rights Watch and Harvard Law School Human Rights Clinic, “Losing Humanity: The Case Against Killer Robots,” November 19, 2012. See HRW section for more info. Michael N. Schmitt, International Law Department, US Naval War College, “Autonomous Weapon Systems and International Humanitarian Law: A Reply to the Critics,” SSRN Working Paper, Draft of December 4, 2012. See SSRN chapter section for more info. William Marra and Sonia McNeil, “Understanding “The Loop’: Regulating the Next Generation of War Machines,” 36 Harvard Journal of Law and Public Policy 3 (2013), Lawfare Research Paper Series 1-2012. See Lawfare Concepts chapter for more info. Kenneth Anderson and Matthew Waxman, “Law and Ethics for Robot Soldiers,” Policy Review, December-January 2012-13 (final published version at Policy Review here, and working draft with footnotes at SSRN here). See SSRN chapter section for more info.
UNQUOTE They plan to make additions to this over time.
Lawfare Wittes and Malinowski (2 Dec 11)
Benjamin Wittes is a Senior Fellow in Governance Studies at the Brookings Institution, where he co-directs the Harvard Law School-Brookings Project on Law and Security. He
136
http://www.lawfareblog.com/2012/12/readings-autonomous-weapon-systems-and-their-regulation/
29 Drones (semi) autonomous out of human loop / Studies / Drones / folder
30
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
is the author of several books and a member of the Hoover Institution's Task Force on National Security and Law.137 Benjamin Wittes has posted several times on Lawfare. Tom Malinowski is with Human Rights Watch. They each write articles, which comment on each other’s prior writings. Here, Lawfare says they are quoting Mr Malinowski’s post in its entirety, after some preliminary remarks:138 QUOTE Some of the arguments below (and by others in favor of a treaty ban on autonomous lethal systems) assume the likelihood of an effective international prohibition. We don’t share that rosy assumption, especially with regard to compliance by states we’re most worried about.139 Specifically regarding Tom’s point below that an international ban on autonomous lethal systems will help ensure that they’re never used by brutal dictators like Syria’s Assad on their own people, as we write this we’re also reading U.S. government warnings to Assad that he’ll probably face direct U.S. action if he uses his vast caches of chemical weapons against the rebels. We don’t think it’s the Chemical Weapons Convention that’s stopping him (the CWC hasn’t stopped him from gaining this capability in the first place, and we’re dubious that it’s serving as a direct barrier to him using chemical weapons; international law here probably does serve as an important background marker for the U.S. and others legitimately to act against him, though). Nor do we think it’s the moral code of his commanders that will determine whether he uses these weapons. And nor do we think he feels he needs sophisticated robotic systems to wipe out civilians and their property. This is a case where only a strong combination of power and principles will stop Assad, and that’s what we are trying to promote in our proposals. UNQUOTE Here, Mr. Wittes shares what Mr. Malinowski recently wrote, regarding lethal autonomous systems, and replies to that.140 The good news for them, is that their areas of disagreement are narrowing. For example, Mr. Malinowski does not appear opposed to the principle of weapon systems which shoot down incoming attacks. Mr. Wittes talks about the potential for future weapon systems which might solve some of today’s challenges. QUOTE Imagine a robot designed for targeted strikes against individual targets. The target, a senior terrorist figure, has already been selected, but he’s living in a compound with large numbers of civilians—wives and kids, say—with whom he is interacting on an ongoing basis. Precisely to discourage drone strikes, he keeps
137 138
http://www.lawfareblog.com/author/benjamin/ http://www.lawfareblog.com/2012/12/tom-malinowski-responds-on-autonomous-lethal-systems/ 139 I share Lawfare’s observation about the ineffectiveness of International bans. 140 http://www.lawfareblog.com/2012/12/tom-malinowski-responds-on-lethal-autonomous-systems-part-ii/
30 Drones (semi) autonomous out of human loop / Studies / Drones / folder
31
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
himself close to these and other civilians. Now imagine that our new robot has a few advances—none of them unthinkable, in my view—over current technology. First, it is very small and quiet and thus hovers very low, allowing it to strike incredibly quickly. Second, it has very long loiter time, so it can watch the target for long periods. Third, its on-board weapon has an extremely small blast radius; maybe it crashes into the target’s head and explodes, but it’s specifically designed not to destroy the building (and the other people in the building) in doing so. And fourth—and critically—to make sure that the strike does not kill the civilians in the compound, the robot decides autonomously when to attack, exploiting splitsecond calculations about the target’s distance from the civilians in relation to the weapon’s blast radius. If you think this robot sounds fantastical, consider this one—which already exists.141 UNQUOTE
Harvard National Security Journal (3 Feb 18)
“Out of the Loop” Autonomous Weapon Systems and the Law of Armed Conflict, published Feb 5, 2013, may be downloaded from Social Science Research Network (SSRN).142 According to Lawfare, this article, by two members of the international law department of the US Naval War College, discusses various issues related to lethal autonomous weapon systems. It argues vigorously against recent calls by Human Rights Watch (HRW) and others for a sweeping, total international ban on autonomous weapon systems.
Red Cross, ICRC (2 Dec 12)
ICRC = International Committee of the Red Cross. One of the responses, to the Allenby and Mattick article,143 came from ICRC,144 which Lawfare then commented further on.145 The Red Cross says that two of the questions raised by Allenby and Mattick reflect a serious misunderstanding of the nature of weapons, and the laws of war. The two questions the Red Cross is referring to are: QUOTE One question was whether international humanitarian law is properly suited to emerging new means and methods of warfare. Another was why “lethal” weapons are preferable over those intended to be “non-lethal". UNQUOTE The Red Cross goes on to explain QUOTE Often when new technologies and weapons emerge there are concerns expressed that they are not covered by existing law . UNQUOTE They explain why this is a bogus understanding. New weapons are not
141
http://www.google.com/hostednews/afp/article/ALeqM5jTPhInUoZJNu1kT604y1CV0xM0w?docId=CNG.b0b893fb04aee2445cb9d54b02462bd7.2f1 142 https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2212188 http://www.lawfareblog.com/2012/12/readings-autonomous-weapon-systems-and-their-regulation/ http://www.lawfareblog.com/2013/02/readings-autonomous-weapon-systems-links-post-updated/ 143 See separate chapter section on what they had to say. 144 ICRC = International Red Cross http://intercrossblog.icrc.org/blog/how-international-law-adapts-new-weapons-and-technologies-warfare 145 http://www.lawfareblog.com/2012/12/the-icrc-and-slate-an-exchange-on-weapons/
31 Drones (semi) autonomous out of human loop / Studies / Drones / folder
32
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
allowed under international law, until they are specifically authorized, through amendments to the various treaties. The lack of regulations, for a weapon system, means it cannot be used.
Al Mac observation: If the ICRC explanation is valid, then this might explain why
various nations, and the UN, have declared that US military use of drones constitute war crimes … they are not being used in accordance with international human rights laws. The ICRC points out that sometimes, thanks to new technology, the old laws and treaties may seem unclear. Thus there is an international discussion seeking to improve that clarity. They also explain why it is meaningless to characterize any weapon as lethal or non-lethal. A handgun can be fired into the air. Hundreds of people have been killed using weapons labeled as non-lethal.
Robinson, Paul U of Ottawa (2 Nov 29) Paul Robinson is a professor in the Graduate School of Public and International
Affairs at the University of Ottawa. He is the author and editor of numerous books on military history and military ethics, including Military Honour and the Conduct of War: From Ancient Greece to Iraq.146 He wrote an article Who Will Be Accountable for Military Technology? It was published Nov 15, 2012 in Slate’s Future Tense. 147 This article was referenced in the USAF General Counsel Blog. See chapter section on Killer Robots Part I. He points to several people who have called for an international convention regulating the use of lethal robot drones, with many relevant links in his article. There’s the issue that machines, such as robot drones, cannot be held legally accountable for their actions. QUOTE As technology advances, we face a very real danger that it will become increasingly difficult to hold those who wage war on our behalf accountable for what they do. Artificial intelligence is not the only technology to pose such accountability problems. Others, such as bio-enhancement, do, too, although of a different sort. UNQUOTE
Accountability Complexity (2 Nov 29)
In my view, the accountability difficulty is not just because of the advance of technology, where future shock can impact many people, it is also the complexity of human organizations, their secrecy, and lack of seeing the big picture.
146 147
http://www.slate.com/authors.paul_robinson.html
http://www.slate.com/articles/technology/future_tense/2012/11/lethal_autonomous_robots_drones_tms_an d_other_military_technologies_raise.single.html
32 Drones (semi) autonomous out of human loop / Studies / Drones / folder
33
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
Companies exist in a government regulated industry. They buy software to satisfy their record keeping meets government and other mandates. Then it is discovered that the software was not doing the job properly. Who is accountable for this? The personnel who made the software acquisition decisions, and implementation, are no longer with the company, either the end user, or the software company. Several years ago, a Stanford Professor got his hands on Voting Machines made by multiple manufacturers. He assigned one class the task of finding ways to spoof them, then a second class to deduce how the first class had spoofed voting info in specific machines. Collectively they found scores of ways to spoof all the machines. This info was then published. The reaction was for many state legislatures to prohibit such behavior in the future, since it supposedly violated trade secrets of the Voting Machine companies. So now we have a reality in which we know these machines can be hacked and spoofed, but it is illegal to do anything about it. Many states have no paper trail, so no way to audit whether the votes were sabotaged. Who is accountable for this? State Legislators who bowed more to Industry Lobbyists than to Academic Ethical desires. With the voting compromised, can voters even get this fixed? Drones are manufactured, using chips from other vendors, which are supposed to perform well defined functions. The drones are then sold to government agencies, which use info from independent intelligence agencies, and personnel in other places in government hierarchy to decide what the drones are to do. It is then found that they are not doing what was desired. Is the problem misinformation, malfunction, misunderstanding?
Singer, Peter on Robotics Revolution (2 Dec 12) Peter W. Singer is the director of the 21st Century Defense Initiative and a senior
fellow in Foreign Policy at Brookings. Singer’s research focuses on three core issues: the future of war, current U.S. defense needs and future priorities, and the future of the U.S. defense system. Singer lectures frequently to U.S. military audiences and is the author of several books and articles, including Wired for War: The Robotics Revolution and Conflict in the 21st Century.148 Peter W. Singer wrote about the Robotics Revolution, Dec 11, 2012, in Open Canada, a hub for International Affairs. 149 He writes about the history of drones, and what we have been seeing in the news media, QUOTE While unmanned systems have a long history, dating back to Da Vinci’s designs for a robotic knight and including things like German remote-controlled torpedo boats in WW I, it wasn’t until a decade ago that they truly took off in war. Advances in technology made unmanned systems more usable, especially through the incorporation of GPS technology which allowed such systems to locate themselves in the world. At the same time, the new conflicts that followed 9/11 drove demand. UNQUOTE
148 149
More info about Peter Singer here: http://opencanada.org/author/petersinger/ http://opencanada.org/features/the-think-tank/essays/the-robotics-revolution/ See Center for New American Security commenting on this article, Dec 12, 2012. I found out about both, thanks to USAF General Counsel. http://afgeneralcounsel.dodlive.mil/2012/12/12/rpas-drones-and-accountability-for-military-action/
33 Drones (semi) autonomous out of human loop / Studies / Drones / folder
34
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
We are at the beginnings of what a new technology will be able to accomplish. It can revolutionize our society like the horseless carriage, and computer. It also has a dark side. This is a huge article, exploring many implications. Peter Singer writes, QUOTE One of the laws in action when it comes to technology is Moore’s Law, that the computing power that can fit on a microchip doubles just under every two years or so. It has become an encapsulation of broader exponential trends in technology that have occurred through history, with technology constantly doubling upon itself in everything from power to storage to broader innovation patterns. If Moore’s Law holds true over the next 25 years, the way it has held true over the last 40 years, then our chips, our computers, and, yes, our robots will be as much as a billion times more powerful than today. UNQUOTE However, he may not recognize that there is a limitation, in physics, to how small things can be made. We now have quantum computers, where their smallest bits are electrons neutrons protons. Science does not yet know how to get smaller than that.
Slate (2 Dec 12)
According to USAF General Counsel Blog,150 there have been several articles recently on Slate, that ICRC disagrees with.151 QUOTE One Slate post argued that international law would need to be revised to reflect the new weapons and war we are fighting.152 The second argued for a change in the treatment of nonlethal weapons. 153 In a post on the ICRC blog, ICRC legal advisor Neil Davison offers a detailed response that is well worth reading. Not surprisingly, he disagrees with both Slate articles.154 UNQUOTE As usual, I want to study original sources, crediting how I found them.
SSRN = Social Science Research Network (2 Dec 03)
April 28 2012, they posted Law and Ethics for Robot Soldiers.155 I am adding it to my Drone Reports, to study as my time permits. I named my copy: SSRN156 Robot Soldier Law and Ethics 2012 April 157 20 pages
150 151
http://afgeneralcounsel.dodlive.mil/2012/12/05/the-icrc-responds-to-slate/ See chapter section on the Red Cross. 152 Page not found, and http://afgeneralcounsel.dodlive.mil/2012/11/13/war-with-nonstate-actors/ See Allenby and Mattick chapter section. 153 http://afgeneralcounsel.dodlive.mil/2012/11/14/the-impact-of-nonlethal-weapons/ http://www.slate.com/articles/technology/future_tense/2012/11/nonlethal_weapons_and_the_law_of_war.h tml 154 http://intercrossblog.icrc.org/blog/how-international-law-adapts-new-weapons-and-technologies-warfare 155 https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2046375 156 SSRN = Social Science Research Network
34 Drones (semi) autonomous out of human loop / Studies / Drones / folder
35
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
Matt and Ken have also written about this topic, other places. 158 (See footnotes.) I have downloaded some other SSRN documents on drones, but not on autonomous ones. Dec 11, 2012, Lawfare shared a recommended reading list regarding recent discussion of Autonomous Robots.159 This included, QUOTE Michael N. Schmitt, International Law Department, US Naval War College, “Autonomous Weapon Systems and International Humanitarian Law: A Reply to the Critics,” SSRN Working Paper, Draft of December 4, 2012. According to Lawfare, Professor Schmitt, a leading laws of war scholar at the Naval War College, has posted this draft paper as a brief (24 page) response to the HRW report, “Losing Humanity.” The draft article analyzes weapons automation and autonomy through the lens of international humanitarian law, as well as the specific US practices to comply with international law obligations to review weapons systems, against HRW’s analysis. The article finds that the “Human Rights Watch position” in favor of a preemptive ban on development, production, and use of autonomous weapon systems is “unlikely to find traction.” It concludes that “autonomous weapon systems are not unlawful per se. Their autonomy has no direct bearing on the probability they would cause unnecessary suffering or superfluous injury, does not preclude them from being directed at combatants and military objectives, and need not result in their having effects that an attacker cannot control. Individual systems could be developed that would violate these norms, but autonomous weapon systems are not prohibited on this basis as a category.” UNQUOTE Dec 11, 2012, Lawfare recommended reading list regarding recent discussion of Autonomous Robots, also included.160 QUOTE Kenneth Anderson and Matthew Waxman, “Law and Ethics for Robot Soldiers,” Policy Review, December-January 2012-13 (final published version at Policy Review here, and working draft with footnotes at SSRN here). According to Lawfare, this brief (6,000 word) policy essay points out that the while autonomous weapons systems will enter the battlefield, they will do so only
157
2012 Nov 28 downloaded from https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2046375 found out about it here: http://www.lawfareblog.com/2012/11/human-rights-watch-report-on-killer-robots-andour-critique/ 2012 Nov 28 I started a Drone Robots doc, on Autonomous Killer Robot Drones, which references this document, so what I have on it, in time may diverge from references in Drone Reports. This paper has been cited and discussed many places, including: http://www.lawfareblog.com/2012/12/readings-matt-and-ken-on-law-and-ethics-for-robot-soldiers/ 158 http://www.hoover.org/publications/policy-review/article/135336 159 http://www.lawfareblog.com/2012/12/readings-autonomous-weapon-systems-and-their-regulation/ 160 http://www.lawfareblog.com/2012/12/readings-autonomous-weapon-systems-and-their-regulation/
35 Drones (semi) autonomous out of human loop / Studies / Drones / folder
36
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
incrementally, through small technological changes that are sometimes unrelated to weapons as such. The essay rejects either a wait-and-see, “don’t try to regulate what doesn’t exist yet” attitude toward regulation of gradually automating weapons systems, or a preemptive ban treaty of the kind urged by HRW. It argues instead that the US should work to create transparent international expectations about the legal requirements of autonomous weapons through the promulgation of its internal norms, standards, and best practices. These stategenerated norms are not generally matters of international law, but what the US government would regard as a plausible way forward on the basis of existing law of war norms in an area of novel technological and other issues. A wait-and-see attitude risks allowing technology to become locked into development paths that might lead to weapons that would be less desirable from a legal and ethical standpoint. The HRW-type ban treaty would unacceptably give up potentially large gains in humanitarian protection that might emerge from new technologies of automation over coming decades. UNQUOTE
USAF General Counsel Blog (2 Nov 28) !
USAF = United States Air Force I am following several blogs, which quote extensively from each other. I would rather have a chapter heading on each source, than duplicate the material. This approach encourages study of original sources, instead of just what someone else chooses to quote.
Drones and Accountability (2 Dec 12)
Dec 12, 2012 he links to what other people have been debating, about alleged Drone implications vs. US military accountability.161 Without interjecting his own comments, he suggests both the Peter Singer article 162 and Dan Trombly’s response163 are well worth a read.
Killer Robots Part 1 (2 Dec 12)
In Killer Robots Part 1, Nov 20 2012, he writes:164 QUOTE The Air Force’s Predators and Reapers are not drones. Indeed, they are not even “unmanned vehicles.” Instead they are remotely piloted vehicles with as much (if not more) human control as an F-16 or F-15. Human beings are still very much in the “kill chain.” The technology to allow completely autonomous weapon systems, however, does exist, and there are serious moral and legal issues that
161 162
http://afgeneralcounsel.dodlive.mil/2012/12/12/rpas-drones-and-accountability-for-military-action/ http://opencanada.org/features/the-think-tank/essays/the-robotics-revolution/ 163 http://www.cnas.org/blogs/abumuqawama/2012/12/silence-and-drones.html See Center for New America Security. 164 http://afgeneralcounsel.dodlive.mil/2012/11/20/killer-robots/
36 Drones (semi) autonomous out of human loop / Studies / Drones / folder
37
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
need to be considered before we think about adopting these autonomous weapon systems. Paul Robinson of University of Ottawa describes some of these issues: He goes on to quote some of this, then: Read it all here. You might also find the perspective of the former Chief Scientist of the Air Force to be of interest as well. He thinks there is little to be gained– and much to be lost–by removing humans from the kill chain. UNQUOTE I explore what Robinson has to say, in a separate chapter section = Robinson, Paul U of Ottawa.
Killer Robots Part 2 (2 Nov 29)
In Killer Robots Part 2, Nov 20 2012, he writes:165 QUOTE We earlier blogged about concerns about autonomous weapon systems. Bonnie Doherty of Human Rights Watch and Harvard Law School has an essay at Foreign Policy that argues for international agreements barring weapon systems that remove humans from the decision-making process: He quotes some of what they had to say, then: Read it all here. Then he comments on the overall topic. In large measure this is a technology issue better answered by experts in artificial intelligence rather than lawyers. He quotes the former chief scientist of the Air Force, concluding with: Read it all here. For a contrary view, see this Air Power article by Caitlin Lee. The Defense Science Board recently released a report that suggests more interest in autonomy than Dr. Dahm. UNQUOTE
Killer Robots Part 3 (2 Nov 29)
In Killer Robots Part 3, Nov 23 2012, he writes:166 QUOTE Perhaps as a result of the recent Human Rights Watch report on autonomous weapons167, and that organizations’ very effective op -ed strategy on the topic, there has been quite a bit written about whether we should have an international ban on autonomous weapon systems. Dan Trombly at CNAS has an interesting response, that emphasizes that even with so-called autonomous weapon systems, there will always be a human in the loop: He quotes some of it, then: Read it all here. Trombly’s analysis of the topic is quite comprehensive and is worth reading for anyone interested in the Human Rights Watch concerns . UNQUOTE
165 166
http://afgeneralcounsel.dodlive.mil/2012/11/20/more-on-killer-robots/ http://afgeneralcounsel.dodlive.mil/2012/11/23/killer-robots-part-3/ 167 See in Human Rights Watch Chapter and Drone Reports, that I have downloaded HRW report.
37 Drones (semi) autonomous out of human loop / Studies / Drones / folder
38
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
I wrote a comment, with a link to a video of a drone which did not have a human in the loop, but the posted comment dropped the url.
Killer Robots Part 4 (2 Nov 29)
In Killer Robots Part 4, Nov 26 2012, he writes:168 QUOTE In response to the Human Rights Watch Report,169 Kenneth Anderson and Mathew Waxman have published their analysis in a new Policy Review essay, Law and Ethics for Robot Soldiers. On LawFare, they give a brief summary of their argument: Then he quotes some of it, which I will link separately, in this general section. Then Read it all here. Second, the Department of Defense has issued a new policy directive on autonomous weapon systems.170 UNQUOTE
Zenko, Micah (2 Dec 12)
Micah Zenko wrote about Lethal Drones Dec 10, 2012, in Open Canada, a hub for International Affairs.171 He starts with excerpts and examples from the history of heavier than air flight, then how they got used in warfare, but says uses and abuses of drones are different than manned flight history. QUOTE First, unlike manned aircraft or raids, drones can fly directly over [allegedly] hostile territory without placing pilots or ground troops at risk of injury, capture, or death. Second, drones allow for sustained persistence over potential targets. For example, the U.S. arsenal of armed drones – primarily the Predator and Reaper – can remain aloft, fully loaded with munitions, for more than 14 hours, compared to four hours or less for F-16 fighter jets and A-10 ground-attack aircraft. Third, drones provide a near-instantaneous responsiveness which dramatically shrinks what U.S. military targeting experts call the “find-fixfinish” loop. A drone-fired missile travels faster than the speed of sound, striking a target within seconds – often before it is heard by people on the ground. Contrast this with the August 1998 cruise-missile salvo that
168 169
http://afgeneralcounsel.dodlive.mil/2012/11/26/killer-robots-part-4/ See in Human Rights Watch Chapter and Drone Reports, that I have downloaded HRW report. 170 See in US Military drone future plans and Drone Reports, that I have downloaded this report. 171 http://opencanada.org/features/the-think-tank/essays/lethal-drones/
38 Drones (semi) autonomous out of human loop / Studies / Drones / folder
39
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
targeted Osama bin Laden, an operation which required predicting where he would be in four to six hours, to analyze the intelligence, obtain presidential authorization, program the missiles, and fly them to the target. Intercontinental ballistic missiles loaded with conventional munitions can reach distant targets much faster than cruise missiles, but they carry the dire risk of misattribution as a nuclear first-strike. Drone-fired missiles can also be – and have been – diverted172 at the last moment if noncombatants enter the likely blast radius.173 At the same time, however, drones suffer two significant limitations. First, the precision and discrimination of drones are only as good as the supporting intelligence, which is derived from multiple sources. In states which lack a vast network of enabling intelligence, there will be significantly less situational awareness and precise targeting information for drones. Second, U.S. armed drones benefit from host-state support, which the US secures with payments in foreign aid and security assistance. Without such tacit or explicit support, drone strikes would be much less effective. For example, the CIA and U.S. military co-operate with their Pakistani counterparts to collect intelligence to identify and track suspected militants. The Pakistani army clears the airspace for U.S. drones, and when they inadvertently crash, Pakistani troops have repeatedly fought against the Taliban to recover the wreckage. Similarly, in Yemen, President Abed Rabbo Mansour Hadi has praised U.S. drones:174 “Every operation, before taking place, they take permission from the president.” As a result, the United States has taken advantage of permissive environments, unthreatened by antiaircraft guns or surface-to-air missiles, where non-battlefield targeted killings have occurred. However, according to former Air Force lieutenant-general David Deptula,175 armed drones that are deployed against a capable air-defense system would be vulnerable to attack: “Some of the [drones] which we have today, you put in a high-threat environment, and they’ll start falling from the sky like rain.” UNQUOTE
172 173
http://articles.latimes.com/2011/feb/22/world/la-fg-drone-strikes-20110222 And also have often not been diverted, thanks to poor intelligence. 174 http://www.washingtonpost.com/world/national-security/yemeni-president-acknowledges-approving-usdrone-strikes/2012/09/29/09bec2ae-0a56-11e2-afff-d6c7f20a83bf_story.html 175 http://www.foreignpolicy.com/articles/2012/02/27/10_things_you_didnt_know_about_drones?page=full
39 Drones (semi) autonomous out of human loop / Studies / Drones / folder
40
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
Similar Treaties (2 Dec 12)
We can pattern the proposed HRW treaties on pre-existing treaties. M Bolton,176 of Political Mine Fields,177 in International Committee for Robot Arms Control (IRAC)178 Analysis179 Oct 3, 2012:180 QUOTE I have been considering ways to flesh out the ICRAC proposal with further provisions building on the precedents of the landmine181 and cluster munition treaties,182 as well as the draft Arms Trade Treaty183 (currently still being negotiated) and the broader corpus of international humanitarian law governing conduct in war.184 UNQUOTE
Cluster Monition Treaty (2 Dec 12)
Here is a reference to it.185
IHL in armed conflicts treaty (2 Dec 12)
IHL = International Humanitarian Law, discussed extensively in my Drone Terms.
Land Mine Treaty (2 Dec 12)
Here is a reference to it.186
Civilized Human Principles (2 Dec 01)
Various people have suggested clauses, or chapters, which belong in a future international law convention.
Laws of Robotics (2 Dec 03)
The US Military and CIA laws of robotics are probably something like this: 1. If you see a useful target, please shoot at it. 2. If you get a software update, first work the decryption verification check that it has the USAF or CIA owner-code, to make sure it is not a hack job. (This is a concept IBM invented 50 years ago, but the other computer companies have yet to incorporate in their systems,)
176
http://icrac.net/author/gubrud/ and http://politicalminefields.com/aboutme/ See IRAC Matthew Bolton section. 177 http://politicalminefields.com/aboutme/ 178 http://icrac.net/ 179 http://icrac.net/category/analysis/ 180 http://icrac.net/2012/10/debating-proposals-on-a-possible-international-convention-on-robotic-weapons/ 181 http://www.icbl.org/index.php/icbl/Treaty 182 http://www.stopclustermunitions.org/the-solution/the-treaty/ 183 http://politicalminefields.com/2012/07/26/new-draft-of-the-arms-trade-treaty/ 184 http://www.icrc.org/eng/war-and-law/index.jsp Red Cross 185 http://www.stopclustermunitions.org/the-solution/the-treaty/ 186 http://www.icbl.org/index.php/icbl/Treaty
40 Drones (semi) autonomous out of human loop / Studies / Drones / folder
41
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
3. When you run low on gas or ammo, get a fuel up, or return to the nearest friendly base.
Disaster Recovery (2 Dec 01)
There are innocent communities which have been devastated by drone assaults. 187 They need foreign aid similar to that which goes to victims of an earthquake, severe weather disaster. Funding should be demanded, of the nations and people to be held accountable, for those violations of human decency and dignity. Matthew Bolton calls for,188 QUOTE 1. International cooperation in the reconstruction of communities damaged by robotic weapons; 2. Recognition of the rights of survivors of robotic weapon violence and international cooperation in the provision of victim assistance; 3. National and international mechanisms for recording the casualties of robotic weapon violence; 4. National reporting on treaty implementation; 5. International cooperation on treaty implementation, including technical assistance. UNQUOTE
Human Control Principle (2 Dec 01) Principle of Human Control = the notion that machines should not be deciding
that this or that human is to be killed.189 This is a proposed new principle, not previously stated within any body of law or philosophy. “We believe… it is unacceptable for machines to control, determine, or decide upon the application of force or violence in conflict or war. In all cases where such a decision must be made, at least one human being must be held personally responsible and legally accountable for the decision….” The principle is not just about killing, but also torture, interrogation, mind reading. It is a human right not to be killed on the decision of machines, nor to be subjected to violent force or pain on the decision of machines, nor to be threatened with violent force, pain or death as a form of coercion on the decision of machines, nor to be ruled in the conduct of life through the agency of overseeing machines which may decide the use of force as coercion.
187 188
In Drone Terms, for example, see: Datta Khel Nomada Bus Station. http://icrac.net/2012/10/debating-proposals-on-a-possible-international-convention-on-robotic-weapons/ 189 http://icrac.net/2012/11/the-principle-of-humanity-in-conflict/ http://icrac.net/2012/10/debating-proposals-on-a-possible-international-convention-on-robotic-weapons/
41 Drones (semi) autonomous out of human loop / Studies / Drones / folder
42
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
I would amend this. If we ever give artificial intelligence beings equal status under the law, if there is some logical way to punish them when they misbehave, then perhaps they would have the status of humans. Currently, in the USA, the US Supreme Court has declared Corporations to be persons, with rights like humans. This has all sorts of implications, starting with the difficulty of proving accountability, and the impact of big money on negative advertising in politics. Where the initial draft proposes that “machines should not do this or that to humans”, the word “machines” might be adjusted to include institutions rules and policies, without proper human oversight. There might also be a chapter defining what proper human oversight should look like. Also see my Mitigation Suggestions in Drone Terms, where: 1. I have evaluated many human systems which are obviously broken. 2. I try to figure out how come they are broken, and why premeditation efforts fail. 3. I have proposed new activities to provide alternative solutions. Similar analysis is needed in this discussion, to avoid repeating the same kinds of mistakes. I think we need more professions like medicine, engineering, and accounting, where people are certified as competent, and must sign their name to key decisions and documents. This is especially relevant with drone killing. There should be an audit trail. This person was ordered killed, on the basis of what alleged facts with an audit trail of where those facts came from. It may need to be classified, but that record should not get destroyed because of old age, as many corporate and government records are shredded when thought to no longer be needed. It should be declassified at some point in the distant future.
Human Responsibility Principle (2 Dec 01) Principle of Human Responsibility = We must hold ourselves responsible for
the decision to use violent force, and cannot delegate that responsibility to machines. At least one person (a human person), and preferably exactly one (e.g. a commanding officer), must be accountable for each decision to use violent force against a particular person or object at a particular time and place. Where data has been recorded pertaining to the circumstances of such decisions, it must be retained and made available for judicial review .190
Humanity Principle (2 Dec 01) Principle of Humanity = a concept found in early Hague and Geneva Conventions
first drafted a century ago. In the conduct of war we must not be needlessly cruel, inflict unnecessary suffering, or make use of weapons that are inherently inhumane.191
190 191
http://icrac.net/2012/11/the-principle-of-humanity-in-conflict/ http://icrac.net/2012/11/the-principle-of-humanity-in-conflict/
42 Drones (semi) autonomous out of human loop / Studies / Drones / folder
43
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
Matthew Bolton would ban,192 QUOTE arming robotic, particularly ’unmanned’, systems with nuclear, biological or chemical weapons, blinding lasers, landmines, cluster munitions or other weapons prohibited by international humanitarian norms. UNQUOTE But these weapons exist, and in war, leaders consider most options on the table. Is it better to send them blindly at an enemy, or use systems able to target them better at military targets, as opposed to a mass of combatants and non-combatants in a similar area?
Killers Trade Ban (2 Dec 01) See my Drone Nations notes. In there, I share info about treaties, which I found out
from GAO, which ban the sale of delivery systems for WMD etc. to certain nations. There are several US government agencies whose job it is to stop this from happening. But, as the GAO investigation makes clear, they are not only not doing their job, they are helping grease the wheels of trade to make sure the weapons go to inappropriate national customers. It is not enough to have a non-proliferation treaty. It has to be honored by nations which have the power to enforce it.193 Matthew Bolton calls for,194 QUOTE 1. Prohibition of trade or transfer of robotic weapons, parts, components, related technologies and munitions, to countries and armed groups that commit serious violations of international human rights and humanitarian law; 2. Transparent reporting on the trade and transfer of robotic weapons. UNQUOTE
Killing with Non-Lethal Weapons (2 Dec 01)
I have written citations in my earlier notes, such as Drone Terms, about non-lethal weapons like Tasers, which have killed hundreds of people, and how tear gas canisters aimed, at people’s heads, do serious damage, and other instances where police supposedly armed with non-lethal weapons, have permanently harmed innocent people. If a weapon has been banned for use by military on the battlefield, shouldn’t it also be banned for use by police against the civilians of a nation, let alone via police drones? Matthew Bolton would require regulation of,195 QUOTE use of so-called ‘less-lethal’, ‘non-lethal’ or ‘pain’ weapons by robotic systems.
192 193
http://icrac.net/2012/10/debating-proposals-on-a-possible-international-convention-on-robotic-weapons/ Over the weekend 2012 Dec 1-2, I e-mailed Matthew Bolton pointing this out, supplying links to the GAO reports, where I uploaded my Drone Nations notes, and mentioning a few other related topics. 194 http://icrac.net/2012/10/debating-proposals-on-a-possible-international-convention-on-robotic-weapons/ 195 http://icrac.net/2012/10/debating-proposals-on-a-possible-international-convention-on-robotic-weapons/
43 Drones (semi) autonomous out of human loop / Studies / Drones / folder
44
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
UNQUOTE
44 Drones (semi) autonomous out of human loop / Studies / Drones / folder
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
Drone Robots
Drones out of human loop controversy.
Notes by Alister William Macintyre Some people are in denial that drones are being developed, which decide what to do, refuel themselves, repair themselves, communicate thru networks, without human direct controls. Think like NASA robots on Mars, autonomous from human control, feeding back info, occasionally accepting software updates. For those people who think autonomous drones are some kind of science fiction fantasy which won’t exist until distant future, I cite examples of how such robots do exist today. Using robots in warfare is controversial, given apparent lack of accountability, when under human control. There is a policy debate whether we need upgrades to laws and treaties. Last updated 2013 Mar 09
Version 5.19
Table of Contents Drone Robots Introduction (2 Dec 03) ............................................................................... 2 Autonomous Drones Debate (2 Dec 02) ............................................................................. 3 Autonomous Drones Exist (3 Mar 08) ! ..................................................................... 5 Relevant Terms Defined (3 Mar 09) ............................................................................... 6 Burglar Alarm involving drones + (3 Jan 07) ............................................................. 7 CIA+ definitions (3 Feb 08)........................................................................................ 8 Drone+ definitions (2 Dec 01) .................................................................................... 9 Future+ (2 Dec 02).................................................................................................... 10 Geneva+ (2 Dec 01) .................................................................................................. 10 GPS+ (2 Dec 02)....................................................................................................... 11 Latency+ definitions (3 Feb 28) ! ............................................................................. 12 SAWS+ fire and forget? (2 Nov 29) !....................................................................... 13 Sky Net+ and Parallels (3 Jan 10)............................................................................. 14 Semi-Autonomous Peru Drone Archeologist (3 Jan 10) .............................................. 16 USN Robot Guns (2 Nov 29)........................................................................................ 16 Israeli Iron Dome (2 Nov 29)........................................................................................ 16 Sky Net X47B lands on USN Aircraft Carrier (3 Mar 04) ! ......................................... 17 US Space Plane Drone X-37B (2 Dec 11) .................................................................... 17 US Military Drone future plans (2 Nov 28).................................................................. 17 Other sources (2 Nov 29) .............................................................................................. 18 HRW = Human Rights Watch (2 Nov 28) ! ..................................................................... 19 Lawfare on HRW (3 Feb 28) .................................................................................... 19 Allenby and Mattick (2 Dec 12) ................................................................................... 20 Anderson, Ken (2 Dec 25) ............................................................................................ 20 ASIL Autonomous Weapons Backgrounder (3 Jan 23) ............................................... 20 Bolton, Matthew IRAC Overview (2 Dec 12) .............................................................. 21
1 Drones (semi) autonomous out of human loop / Studies / Drones / folder
2
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
Center for New American Security (2 Dec 12) ............................................................ 23 Accountability Myth (2 Dec 12) ............................................................................... 24 CETMONS (2 Dec 12) ................................................................................................. 25 DoD (2 Dec 12)............................................................................................................. 26 Lawfare (2 Dec 03) ....................................................................................................... 26 Lawfare Matt & Ken (2 Dec 10)............................................................................... 26 Lawfare Concepts (2 Dec 12) ................................................................................... 28 Lawfare Reading List (2 Dec 12).............................................................................. 29 Lawfare Wittes and Malinowski (2 Dec 11)............................................................. 29 Harvard National Security Journal (3 Feb 18) .............................................................. 31 Red Cross, ICRC (2 Dec 12)......................................................................................... 31 Robinson, Paul U of Ottawa (2 Nov 29)....................................................................... 32 Accountability Complexity (2 Nov 29) .................................................................... 32 Singer, Peter on Robotics Revolution (2 Dec 12)......................................................... 33 Slate (2 Dec 12) ............................................................................................................ 34 SSRN = Social Science Research Network (2 Dec 03) ................................................ 34 USAF General Counsel Blog (2 Nov 28) !................................................................... 36 Drones and Accountability (2 Dec 12) ..................................................................... 36 Killer Robots Part 1 (2 Dec 12) ................................................................................ 36 Killer Robots Part 2 (2 Nov 29) ................................................................................ 37 Killer Robots Part 3 (2 Nov 29) ................................................................................ 37 Killer Robots Part 4 (2 Nov 29) ................................................................................ 38 Zenko, Micah (2 Dec 12) .............................................................................................. 38 Similar Treaties (2 Dec 12)............................................................................................... 40 Cluster Monition Treaty (2 Dec 12).......................................................................... 40 IHL in armed conflicts treaty (2 Dec 12).................................................................. 40 Land Mine Treaty (2 Dec 12) ................................................................................... 40 Civilized Human Principles (2 Dec 01) ............................................................................ 40 Laws of Robotics (2 Dec 03) .................................................................................... 40 Disaster Recovery (2 Dec 01) ....................................................................................... 41 Human Control Principle (2 Dec 01) ............................................................................ 41 Human Responsibility Principle (2 Dec 01) ................................................................. 42 Humanity Principle (2 Dec 01) ..................................................................................... 42 Killers Trade Ban (2 Dec 01)........................................................................................ 43 Killing with Non-Lethal Weapons (2 Dec 01).............................................................. 43
Drone Robots Introduction (2 Dec 03)
We are now moving towards killer robots, which will do their thing, without a human being in the decision loop. This raises many serious implications, which is the focus of these research notes. The first I have known of drones used in warfare was during WW II, when they were designed for weapons target practice, and German delivery of bombs to British cities, without pilots flying the delivery vehicles. Other people claim drones in warfare were not developed until many decades later. It is a matter of definition, whether various
2 Drones (semi) autonomous out of human loop / Studies / Drones / folder
3
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
rockets, aircraft which shoot other weapons, which do not have a pilot on board, should come under the same labeling, as the latest in a line of technological improvements. It is like saying the Wright Brothers did not have a airplane, because it was a technological toy, compared to the aircraft today, or that the first computers, which used punched cards and paper tape, are not really computers because they cannot compare to the digital computers of today. In Drone Issues, see chapter = Military Debates, for some issues associated with the development of killer robots, which can do their thing, without a human being in the loop. My drone documents have a version # in their naming, which I increment as I replace with re-writes. In parentheses after each heading is date in format (Year Month Day) when I last updated the contents of that chapter. So people with an earlier edition can see what has been updated since they last got a copy. Some chapter headings end in an exclamation point. These indicate either identification of a controversy, and-or links to what the facts may be. Other symbols may be added later. This helps correlation of conflicting info sources, in my further research. As I make significant additions and improvements to these research notes, I will upload revised editions various places on the Internet, for convenient access by other people who share some of my interests.1
Autonomous Drones Debate (2 Dec 02)
There is a debate raging about several related topics. 1. Persons with different professional and personal experience backgrounds can see different facets of areas of concern, while others are out of their imagination. 2. What is the status of autonomous and semi-autonomous drones, or killer robots without a human in the loop? There are conflicting stories about military plans, and I have seen video of autonomous drones, so I know they exist, even though many news stories and bloggers are in denial. I think some react to concepts saying this is science fiction, thus it is ludicrous to be even be debating this. Some of this comes from some people seeing early steps, military plans, realize possibilities, and speculate in a way, that others see as science fiction. 3. I have worked in midrange computing my entire career, and followed artificial intelligence (AI) intermittently. It seems to me that the state of art of software is heavily contaminated by mandates from non-technical managers, and leaders who believe computers are vastly superior in capabilities to reality. There is an astronomical risk here, not of Sky Net, but of mindless robots killing for no explicable reason. There have been incidents in the history of transportation and industrial automation, of robots killing humans, which sometimes took a while to
1
Scribd http://www.scribd.com/doc/115182066
3 Drones (semi) autonomous out of human loop / Studies / Drones / folder
4
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
figure out and solve.2 Fortunately in those past incidents, it was easy to divorce the machinery from the victims, while the diagnostics were conducted to figure out how to fix the problem. With killer robot drones, that might not be so easy. a. There are precedents of man made creations getting loose and doing serious damage, allegedly not intended by their creators. b. Consider what would happen if a civil engineer was modifying how a city’s sewage system worked, and made mistakes leading to every toilet in the city backing up, with fountain of recent human waste filling bathroom, spilling over into adjacent rooms of homes and offices. The responsible engineer would spend the rest of his life in prison, while his employer would go bankrupt, paying off the law suits. c. That is exactly what has happened with our Economic System. A group of financial engineers, working on Wall Street, and other places, made some modifications in how mortgages are tracked and invested in, encouraging a multi-year bubble in home prices, and mortgage securities, which ultimately led to a world wide Great Recession, an epidemic of foreclosures, massive unemployment, growth in homeless people, lost pensions, and the damage is continuing, such as in Europe. Their damage to the economic well being of the world, was just like my hypothetical engineer story. Yet instead of being held accountable, they earned multi million $ bonuses. They have created a mess so bad, that it is impossible for government regulators to put humpty dumpty back together again, to make their victims whole. 4. There are legal and ethical implications. Many people have written about them. In these notes, I identify some of those people, linking to their posts, and comment on some of what they have to say. There are huge questions about accountability. Who is to be held responsible when innocent people are killed or maimed, by one of these autonomous killer robot drones? a. Killer Bees are a plague on Central America, and spreading, with Global Warming. They are a man made disaster, with no one accountable. b. Mad Cow disease is now in many farm animals, wild animals, pets. It is a man made disaster, with no one accountable. c. Autonomous drones sounds like another man made disaster, with no one accountable. I wanted to split these notes into sections: Definitions of terminology, and popular expressions where there is some dispute what things really should be called. Artificial Intelligence capabilities for autonomous and semi-autonomous robots. State of Art, which has become unclassified, or we can find in media reports.
2
A common problem has been a lack of proper shielding for computer signals within devices, hearing signals from unrelated devices, which also lacked proper shielding. Thus: an industrial robot suddenly moves a heavy arm and kills people nearby; a high speed bullet train’s doors open, sucking many passengers out, when it passes apartment buildings with many children with hand held toys; an amusement park ride takes passengers to an unexpected location, at excessive speed, then crash; airport radar gets false signals, leading to a near miss of airliners.
4 Drones (semi) autonomous out of human loop / Studies / Drones / folder
5
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
Laws and Treaties which exist so far, which are relevant. Legal and Ethical implications, not yet satisfactorily addressed.
But the reality is that most all the reports out there, all the blogs, all the news stories, intermingle allegations and denials regarding a whole spectrum of such concepts, so it is not really practical to organize my notes that way. See my Drone Reports for: a directory of official documents, which I have located on drones; what I called my copy if I downloaded it; pointers to where I put notes if I digested the download; and links to other people discussing it.
Autonomous Drones Exist (3 Mar 08) !
There are people who believe that some day, the world will have what these notes are about, but that day is in the distant future. 3 Thus I have multiple examples in these notes showing that theoretical some day is really today, they really do exist. They are not yet Sky Net killing machines, but important technological advances are being made. They are also not exclusively in the realm of the military. 4 Here we see an impressive video of a robot plane navigating indoor obstacles at approx 22 mph, using on-board sensors, without GPS or a human pilot involved.5 It is yet another example of future spy technology able to get into some place that could be cut off from human direct controls. The next development needed for this little guy is stealth of vision and sound. It is unclear to me how this plane might cope with moving air of outdoors, or moving air of internal heating cooling conduits, or other moving objects if it got into an elevator shaft for example. I can imagine this technology being made ever smaller, into a drone smuggled into · an Iranian nuclear power plant, on behalf of USA or Israel; · Gitmo on behalf of some group planning a prison break; · The possibilities are endless. Military Robotics are on the rise,6 due to: political funding cut-backs; less staffing needed for complex missions; less money for recruiting, training, housing, feeding; insurance; they do not have an instinct for self-preservation, so they won’t lash out in fear, anger, recklessness;
3
http://singularityhub.com/2012/12/19/killer-robots-are-coming-ai-experts-to-determine-their-threattoward-humanity/ 4 http://www.technewsdaily.com/8423-autonomous-drones-not-just-military-tech-anymore-video.html 5 http://www.lawfareblog.com/2012/08/drones-keep-getting-better-and-better/ http://web.mit.edu/newsoffice/2012/autonomous-robotic-plane-flies-indoors-0810.html 6 http://icrac.net/2011/12/allvoices-the-ethical-and-moral-concerns-of-autonomous-robotic-systems/ http://www.allvoices.com/contributed-news/5223454-the-ethical-and-moral-concerns-of-autonomousrobotic-systems
5 Drones (semi) autonomous out of human loop / Studies / Drones / folder
6
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
belief they can fulfill the mission at lower cost (both financial, and harm to humans in our military);
Relevant Terms Defined (3 Mar 09)
This topic will have concepts in need of some new definitions. I should put them first in Drone Terms then here, with possible cross-pointer. From my Drone Terms:7 3d printed autonomous drones.8 911 for a Medical Drone: Here is an article about the development of a UAV which can be summoned by a smart phone, land in a battle zone, pick up an injured soldier and fly them back to base.9 10 AHAR = Autonomous High Altitude Refueling Basically, drones need the same kinds of services as manned flight.
o See Autonomous Drone. AI = Artificial Intelligence o According to an article in IRAC = International Committee for Robot Arms Control, QUOTE:11
“Research on artificial intelligence over the past 50 years has arguably been a contemporary Tower of Babel. While AI continues to be a rich field of study and innovation, much of its edifice is built upon hype, speculation, and promises that cannot be fulfilled. The U.S. military and other government agencies have been the leaders in bankrolling new computer innovations and the AI tower of babble, and they have wasted countless billions of dollars in the process. Buying into hype and promises that cannot be fulfilled is wasteful. Failure to adequately assess the dangers posed by new weapons systems, however, places us all at risk.
7
o UNQUOTE o See: Artificial Intelligence; Autonomous Drone; Needle in Haystack. American Society for the Prevention of Cruelty to Robots12
Drone Terms is my document with approx 1,000 concepts, terms, and acronyms defined, which are related to Drones, National Security, alleged misconduct by various government police agencies. 8 http://www.wired.com/design/2012/11/3d-printed-autonomous-airplane/ 9 http://www.wired.co.uk/magazine/archive/2012/12/start/autonomous-flight 10 http://aviationintel.com/2012/10/10/unmanned-game-changer-autonomous-aerial-refueling-is-a-reality/ 11 http://icrac.net/2013/03/moral-machines-guru-takes-moral-high-ground-on-autonomousweapons/ The IRAC article is a summary of a longer article here: http://scienceprogress.org/2013/01/terminating-the-terminator-what-to-do-about-autonomousweapons/ 12 Smithsonian Smart News (2013 Feb 13) - "Mistreated Robots Now Have a Advocacy Group": http://blogs.smithsonianmag.com/smartnews/2013/02/mistreated-robot-nowhave-a-advocacy-group/
Been around since 1999: http://www.aspcr.com/index.html -Gregory Foster || gfoster at entersection.org On [drone-list] https://mailman.stanford.edu/mailman/listinfo/drone-list
6 Drones (semi) autonomous out of human loop / Studies / Drones / folder
7
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
Artificial Intelligence (AI): Are Robots and Computers programmed, or can they learn by experience, being taught to do new things, without human input?13 AUSV = Autonomous Unmanned Surface Vehicle. o See: Autonomous Drone.
Autonomous Drone = A drone which does not require human involvement to pilot it, decide where it is to go, decide who it is to shoot at, refuel it, maintain it, update its programming.14 Some stories have alleged that some drones, in testing, go part way towards these goals, not yet achieve all of them in one craft. Other people think this definition is not clear enough. 15 o See: AHAR; AUSV; AWS; SAWS. AWS = Autonomous Weapon Systems. o QUOTING International Committee for Robot Arms Control (IRAC):16 o Department of Defense Directive 3000.0917 “for the development and use of autonomous and semi-autonomous functions in weapon systems” defining an AWS as “A weapon system that, once activated, can sel ect and engage targets without further intervention by a human operator.” The ability of the system to select and engage targets autonomously makes it an AWS even if it is human-supervised with a possible human override; thus “human on the loop” is assigned the same status as “human out of the loop,” a conservative (good) policy. o UNQUOTE o See: Autonomous Drone.
Burglar Alarm involving drones + (3 Jan 07)
Consider this scenario. A home is being burglarized. The burglar alarm causes a drone to take off and photograph the crooks, what they are doing, perhaps follow them until the police arrive.18 The drone needs to be able to remain aloft, out of range of the crooks trying to cause it to crash. This is an example of an autonomous drone, without a human in the loop, telling it what to do, and it is a scenario being made possible by the Japanese firm Secom.19
13 14
Cars which drive themselves, are an example of autonomous robots.20
http://www.redorbit.com/news/technology/1112787439/robot-icub-learns-a-new-language-022013/ Autonomous Drones http://blogs.cfr.org/zenko/2012/09/04/the-coming-future-of-autonomous-drones/ http://www.acq.osd.mil/dsb/reports/AutonomyReport.pdf 2012 Nov 28 I started a Drone Robots doc, on Autonomous Killer Robot Drones, what I have on it, in time may diverge from references in Drone Terms. 15 http://www.lawfareblog.com/2012/12/john-dehn-on-autonomous-robots-and-targeting/ 16 http://icrac.net/2012/11/dod-directive-on-autonomy-in-weapon-systems/ 17 I have downloaded this, naming it DOD Autonomous Weapon Policies 2012 Nov. 18 http://afgeneralcounsel.dodlive.mil/2013/01/04/we-reap-what-we-drone/ 19 http://phys.org/news/2012-12-japan-firm-private-drone.html 20 http://www.lawfareblog.com/2013/01/readings-bryant-walker-smith-automated-vehicles-are-probablylegal-in-the-united-states/
7 Drones (semi) autonomous out of human loop / Studies / Drones / folder
8
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
CIA+ definitions (3 Feb 08)
CIA = US Central Intelligence Agency. o While in general, we have Branches of Government, broken into Departments, broken down into Agencies and Offices, CIA is an exception. It is not in any Dept of US Government. It is an agency independent of all the departments. It reports to the DNI. o The CIA was originally a foreign intelligence service, but in recent years it has also become a paramilitary force. It is believed to have approx 30-40 drones, and is seeking to increase the size of its drone fleet. 21
o Allegedly all US drone attacks are carried out by either CIA or JSOC, under guidance by the President of the USA.22
o Stephen W. Preston, General Counsel of the Central Intelligence Agency, in an April 2012 speech at Harvard Law School on “CIA and the Rule of Law,” listed many applicable laws and procedures. See footnotes.23 o In Drone Terms, see: DIA; DNI; DoD; JSOC; NSA. Citizens United = A US Supreme Court Case which elevated corporations to have many of the same constitutional rights as human citizens.24 o See: Sky Net. CSER = A think tank–Centre for the Study of Existential Risk proposed by Cambridge University.25 DoD = US Dept of Defense26 HQ at the Pentagon. o DoD is one of the largest US Departments. It is composed of the various different types of military: Air Force; Army; Army Corps of Engineers; Navy; Marine Corps; Special Forces; National Guard and Reserve; Space Command; Missile Defense; Cyber Warfare; Weapons Research; Drones; Special units associated with WMD; Procurement of Weapons; Civilian Contractors, some Intelligence agencies; Regional Unified Commands, and more stuff. A small amount of this has been transferred to DHS.
21
http://www.smh.com.au/technology/technology-news/cia-seeks-expansion-of-drone-fleet-2012101927w0k.html 22 http://livingunderdrones.org/ INTERNATIONAL HUMAN RIGHTS AND CONFLICT RESOLUTION CLINIC (STANFORD LAW SCHOOL) AND GLOBAL JUSTICE CLINIC (NYU SCHOOL OF LAW), LIVING UNDER DRONES: DEATH, INJURY, AND TRAUMA TO CIVILIANS FROM US DRONE PRACTICES IN PAKISTAN (September, 2012) 23 CIA and the Rule of Law Synopsis https://www.fas.org/blog/secrecy/2012/09/targeted_killing.html I downloaded, calling my copy: CRS Kill US Citizen Legalities 2012 May. Full speech
http://www.cfr.org/rule-of-law/cia-general-counsel-stephen-prestons-remarks-rule-law-april 2012/p27912
24 25
https://en.wikipedia.org/wiki/Citizens_United_v._Federal_Election_Commission http://afgeneralcounsel.dodlive.mil/2013/02/07/stopping-killer-robots-and-other-threats-to-hjumansurvival/ http://www.foreignpolicy.com/articles/2013/02/04/doomsday_preppers_cambridge_existential_risks 26 http://www.defense.gov/ https://en.wikipedia.org/wiki/United_States_Department_of_Defense https://en.wikipedia.org/wiki/List_of_Department_of_Defense_agencies
8 Drones (semi) autonomous out of human loop / Studies / Drones / folder
9
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
o Here is 2009 Dod Drone Roadmap. (4 Meg)27 Raffaela Wakeman writes July 10, 2012 in LAWFARE Blog,28 QUOTE Conor Friedersdorf at The Atlantic29 looks closely at the 2009 Unmanned Systems Integrated Roadmap, the Pentagon’s plan for the use of drones from 2009 through 2034, and wonders whether we may get to a point when drones are completely autonomous. For the record, Missy Cummings of MIT says not in her lifetime.30 o UNQUOTE o In Drone Terms, see: CBP Suggestions; Departments of US Government; DHS; DIA; DNI; DSB; DSCA; DSS; GC Armas; ISE; JAG; NSA; PM-ISE. DoJ = US Dept of (civilian) Justice.31 Its mandate includes: ATF; Anti Trust; Criminal prosecutions, and also Civil violations; DEA; Interpol in USA; FBI; OIG; Prisons and Paroles; USMS o Some stuff which used to be in DoJ has been transferred to DHS. o In Drone Terms, see: AECA; ATF Scandals; Attorney General; BI; DEA; Departments of US Government; DoJ (more info than is here in Drone Robots); EOUSA; FBI; ISE; JAG; LIONS; OIG; OLC; PM-ISE; USAO; USMS.
Drone+ definitions (2 Dec 01)
There is a dispute about the proper use and application of some terminology. There are people who say Predators are not drones, but fail to define what they consider to be a drone. There are people who think drones are only those un-piloted vehicles which fly in the sky, don’t consider drones to be un -piloted vehicles which travel in the sea or on land. There are people who think model aircraft and other toys are not drones. I’d like to say that kites and balloons are not drones, but the problem is, for the people on the ground, you cannot tell if that thing in the sky has spy capability, or other modern technology which would qualify it as a drone. I consider a drone, any vehicle which is: Unmanned on board, but may be remotely piloted; Has the ability to spy on people, may be weaponized. This includes in the skies, in buildings, on land, on water, under water.
27 28
http://www.acq.osd.mil/psa/docs/UMSIntegratedRoadmap2009.pdf http://www.lawfareblog.com/2012/07/todays-headlines-and-commentary-222/ 29 http://www.theatlantic.com/politics/archive/2012/07/the-pentagons-vision-pervasive-drones-on-land-airand-sea/259541/ 30 http://www.lawfareblog.com/2012/03/lawfare-podcast-episode-5-missy-cummings-on-drones-dronesdrones/ However, I have seen news media reports, in 2012, of DoD testing drones which decide where to go, and what to do, based on their internal programming, instead of a human operator. Those reports shared in my Drone Notes. 31 http://www.justice.gov/ https://en.wikipedia.org/wiki/United_States_Department_of_Justice
9 Drones (semi) autonomous out of human loop / Studies / Drones / folder
10
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
Other people have other definitions. I think we may need alternative terms for various combinations of size, functions. The military has an army of acronyms. FOIA = Freedom of Information Act32 o A Citizen's Guide on Using the Freedom of Information Act and the Privacy Act of 1974 to Request Government Records, House Committee on Government Reform, Sept 20, 2005, House Report 109-226 (PDF)33. Fire and Forget o See: SAWS.
Future+ (2 Dec 02)
Future Shock = a psychological condition described in a 1970 book 34 by the futurist Alvin Toffler. People experiencing or witnessing too much change, in too short a period, can have difficulty grasping enough of it to deal effectively with it. They can become overwhelmed and disoriented. Toffler popularized the concept “information overload.” A documentary film based on the Future Shock book was released in 1972 with Orson Welles as on-screen narrator. o While this terminology arrived in the mid 20th century, the concept has existed all through history. o We see it in nations at risk of losing wars, because they failed to grasp technological change or new tactics fast enough. In WW II, the German Blitzkrieg used tactics and weapons which were not new, just used in a different mixture, with great surprise and shock for other nations whose leadership was not expecting this. o The 9/11 commission found over a dozen prior instances of terrorists flying hijacked planes into national monuments, or attempting to do so, many of which had previously been reported to national security leaders, but on the day of the attack, 99% of US leaders described the event as totally unexpected. o The 2011 Japanese tsunami was no surprise to tsunami experts, who had predicted exactly what risk was being taken at Fukushima and other nuclear power plants, whose management and government regulators had chosen to ignore the risks, then in the aftermath stated that the event was beyond their imagination. One possible explanation was future shock preventing wise planning of complex industry. o I am seeing many people suffering from future shock today, unable to have a rational conversation about national security legislation.
Geneva+ (2 Dec 01)
32 33
Geneva Convention(s),35 usually refers to international law treaties regarding humanitarian treatment of: prisoners, of war and armed conflicts (military and
https://www.fas.org/sgp/foia/foia2007.pdf https://www.fas.org/sgp/foia/citizen.html 34 https://en.wikipedia.org/wiki/Future_Shock 35 https://en.wikipedia.org/wiki/Geneva_Conventions
10 Drones (semi) autonomous out of human loop / Studies / Drones / folder
11
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
civilian), the wounded, sick, and ship-wrecked, civilians in a military conflict; prohibitions against the use of torture; etc. o The Geneva Conventions generally focus on treatment of people who are no longer combatants, or never were legal combatants, in a military conflict.36 o The Hague Conventions focus on the conduct of the war, prohibitions against certain types of weapons. o As with any international law, there’s the problem of nations which have not signed the agreements, and nations which choose not to comply with them. o Several nations, other than the USA, are saying that what the USA is doing with drones, is in violation of the Geneva Conventions, International Law, and should be labeled as War Crimes. o In Drone Terms, see: Hague Conventions; International Law; LOAC; Sovereignty; Targeted Killing; Unlawful Combatant; War Crime.
Gitmo = Short for Guantanamo Bay Cuba, a place the USA has been housing
detainees captured in the war on terror.
GPS+ (2 Dec 02)
GPS = Global Positioning System. This is a USA system. o GPS receivers receive almanac data from satellites, and also calculate position by calculating distance from visible satellites and using triangulation method to calculate its position. o GPS receivers also work on NMEA Standards. Most of the computer programs and devices, which provide position and other data given by the GPS receiver, includes info like position (latitude and longitude), altitude, speed, time etc.37 o Military Global Positioning System (GPS) signals have long been encrypted to prevent counterfeiting and unauthorized use. o Civil GPS signals were designed as an open standard, freelyaccessible to all. These virtues have made civil GPS enormously popular, but the transparency and predictability of its signals give rise to a dangerous weakness: they can be easily counterfeited, or spoofed. Like Monopoly money, civil GPS signals have a detailed structure but no built-in protection against counterfeiting.38
36
If the circling drones are used to terrify the citizens below into providing intelligence, that could be a violation of the Geneva Convention. Torture can lead to people giving out info, they think the torturers want to hear, such as the notion of WMD in Iraq. http://www.guardian.co.uk/commentisfree/2012/oct/23/truth-uk-drones-policy 37 http://www.scribd.com/doc/69421118/Drone-Mid-Year-Presentation-1 38 Congress House Transcript Drones in USA Skies 2012 July Panel I testimony by Mr. Todd E. Humphreys, Ph.D Abstract Summary = http://www.hsdl.org/?abstract&did=719102 Full 865k Transcript = https://www.hsdl.org/?view&did=719102 See Feb 2012 in Drone Dates for description of their spoofing demonstration.
11 Drones (semi) autonomous out of human loop / Studies / Drones / folder
12
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
o Hacking a UAV by GPS spoofing is but one expression of a larger problem: insecure civil GPS technology has over the last two decades been absorbed deeply into our national critical infrastructure. Besides UAVs, civil GPS spoofing also presents a danger to manned aircraft, maritime craft, communications systems, banking and finance institutions, and national power grid. o Satellite navigation systems other than GPS include: the Russian GLONASS system (fully operational), the European Galileo system (undergoing in-orbit validation of early spacecraft; may be operational by 2019), and the Chinese Compass system (global system in preliminary test phase). o Small, low power, inexpensive GPS + GLONASS receivers are now available off-the-shelf. These appear to be an excellent option for immediately improving navigation security in existing systems. o In Drone Terms, see: Bird Dog; NMEA; SCADA; UAV. The Hague Convention(s) focus on the conduct of the war, prohibitions against certain types of weapons, or used in certain ways. For example: o Article 25 of the 1907 Hague IV Convention prohibits aerial bombardment “by whatever means” of undefended towns, villages, or dwellings. Some drone strikes are in violation of this. o In Drone Terms, see: Datta Khel Nomada Bus Station, Geneva Convention; Sovereignty. HRW = Human Rights Watch, an NGO. o See section on their Nov 2012 report on autonomous killing machines. 39 IRAC = International Committee for Robot Arms Control o See: Autonomous Drone; AWS.
Latency+ definitions (3 Feb 28) !
Latency = movement shown on a drone pilot’s video screen has over the years been seconds behind what the drone sees — a delay caused by the time it takes to bounce a signal off a satellite in space. This problem, called “latency,” has long bedeviled drone pilots, making it difficult to hit a moving target. Sometimes the target hears the drone and can get to safety faster than the drone operator can do accurate shooting.40 This may explain some of the high proportion of collateral damage, and high cost of drone shooting, since wide dispersal bombs are needed to get at the escaping enemy, while manned combat could use a few bullets.
39
http://icrac.net/2012/11/dod-directive-on-autonomy-in-weapon-systems/ http://icrac.net/2011/12/new-scientist-campaign-asks-for-international-treaty-to-limit-war-robots/ http://www.popsci.com/military-aviation-amp-space/article/2009-09/no-nuke-or-space-military-botspleads-arms-control-committee http://blog.tmcnet.com/robotics/2009/09/new-group-aims-to-curb-military-use-of-robots.html 40 https://www.nytimes.com/2012/07/08/magazine/the-dronezone.html?pagewanted=2&_r=1&ref=magazine
12 Drones (semi) autonomous out of human loop / Studies / Drones / folder
13
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
Needle in Haystack41 from UPI42 is Anomaly Detection in universal data bases on IBM midrange platforms, using artificial intelligence. o The software teaches itself what is normal or anticipated via statistical analysis, then when something unusual occurs, it does an alert. Management can set thresholds for alerts. o In my opinion, this concept is needed on other kinds of computer platforms for all kinds of breaches and threats, including SCADA critical infrastructure and national security hacker activities. o In Drone Terms, see: AI; DBS; FIG-LEAF; OS; SCADA; UPI. NGO = Non-Government Organization, typically a not-for-profit humanitarian charity organization. Non-Lethal weapons is a frequent controversy, coming up again and again, with different people trotting out arguments we have heard many times before, 43 many of them answered before,44 although the weapons involved evolve over time. 45 There is no such thing as a non-lethal weapon. Every such weapon can be abused, to the point that the target people either get killed, or as seriously maimed, as if a lethal weapon was used. Petitions: several groups have petitions appealing for an end to some developments.46 Robot Governor – Still being designed, to give drones ethics desired by DoD and the rules of war. It needs to be able to do the many things required of humans, on the battlefield, to distinguish between real enemies and innocent bystanders.47 o In Drone Terms, see: DoD; Geneva Convention; LOAC. Robot Rights: when are they intelligent enough that we should grant them civil rights? How about half man half machine?48
SAWS+ fire and forget? (2 Nov 29) !
SAWS = Semi-autonomous weapon system
41 42
http://www.unbeatenpath.com/software/needle/in-a-haystack.pdf http://unbeatenpath.com/compass 43 http://www.lawfareblog.com/2012/12/the-icrc-and-slate-an-exchange-on-weapons/ 44 http://intercrossblog.icrc.org/blog/how-international-law-adapts-new-weapons-and-technologies-warfare
45
http://www.slate.com/articles/technology/future_tense/2012/11/nonlethal_weapons_and_the_law_of_war.h tml
46
http://secure.hrw.org/c.nlIWIgN2JwE/b.5571001/k.79D9/US_Stop_KIller_Robots/siteapps/advocacy/Actio nItem.aspx?msource=SKR20130220 http://www.lawfareblog.com/2013/02/human-rights-watch-campaign-on-killer-robots-and-tommalinowskis-response/ 47 http://www.allvoices.com/contributed-news/5223454-the-ethical-and-moral-concerns-of-autonomousrobotic-systems 48 http://www.lawfareblog.com/2013/02/issues-to-put-on-your-intellectual-radar-screens-cyborg-rights/ http://www.cyborgfoundation.com/
13 Drones (semi) autonomous out of human loop / Studies / Drones / folder
14
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
o The International Committee for Robot Arms Control (IRAC)49 says the DoD defines SAWS, as QUOTE: o “A weapon system that, once activated, is intended to only engage individual targets or specific target groups that have been selected by a human operator.” The category includes systems that automatically acquire, track, identify and prioritize potential targets or cue humans to their presence, “provided that human control is retained over the decision to select individual targets and specific target groups for engagement. ” It also includes weapons with terminal homing guidance, and “fire and forget” weapons where the target has been human -selected. o UNQUOTE o See: Autonomous Drone; AWS; DoD. “fire and forget” in the SAWS definition raises an issue of audit ability in my mind. In the capitalist world, that I am familiar with, there is change management control, and audit trails. We make changes to how software functions, then some time later a question is raised regarding the authority to do so, and when exactly it got changed. We can look in the software source code and find a change log: A brief summary of what the change was intended to accomplish. Identification of the names of company personnel who requested this change. Sometimes the name of a manager who approved their request. Identification of any people, outside of our company, who initiated this change request, e.g. auditors, law-suit, customers, vendors, new government regulations. Pointers at where in the source code updates were made to implement this change, using unique naming, where we can search for the precise code lines involved. Identification of names of personnel whose testing determined the software changes satisfactorily achieved what they had wanted. Likewise, we have annual auditors who show up, and ask questions about transactions which have occurred during the year. We can access history of activity to answer their questions. “fire and forget” sounds to me like any such audit trail could be lacking, if the military organization responsible for some drone, is later questioned regarding how come this or that target got fired upon..
RFID = Radio Frequency Identification.50
Sky Net+ and Parallels (3 Jan 10)
There is lots of dispute about the proper use and application of some terminology.
49 50
http://icrac.net/2012/11/dod-directive-on-autonomy-in-weapon-systems/ See discussion of implications within “Center for New American Security” such as a school in Texas mandating that students wear RFID chips, to help faculty keep track of the kids, but which also help kidnappers identify, and locate, valuable targets.
14 Drones (semi) autonomous out of human loop / Studies / Drones / folder
15
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
There is a military drone which calls itself Sky Net. There is a transportation delivery company which calls itself Sky Net. 51
Sky Net was popularized by the Terminator Science Fiction Movie series, and it is now
a TV serial. A lot of people are opposed to our government’s drone practices, either overall, or just some of them, and within those political ideologies, which includes antiwar, but is not that exclusively, when someone mentions autonomous robots, they often knee jerk react with Sky Net. Sky Net, as portrayed in the Terminator stories, is not even close, in technological development, in my opinion. However, it is a useful reference point, due to the huge volume of people in the public who equate military movement towards autonomous weapons, to be on the path to the coming of Sky Net. Sky Net, and the rise of the Machines, as depicted in the Terminator movies, will not be here for decades, but Sky Net Parallels are already here in human organizations, thanks to Citizens United,52 and the rise of the Lobbyists.
Sky Net Parallels are described well by Mark Gubrud53 in International Committee
for Robot Arms Control (IRAC)54 Analysis55 Nov 19, 2012:56 QUOTE Do I mean to invoke here the specter of Skynet, the artificial intelligence that declared war on humanity in The Terminator, reputedly because it feared we’d turn it off? Or Colossus, the military supercomputer that took control of the world in a brutal coup in order to fulfill its mission of ensuring peace? The scorn that “serious” people direct against these tropes from science fiction betrays their own nervousness. Artists have mined our apprehensions about the world we’re creating, and projected them before us in gaudy masks and cartoonish story lines that beg to be decoded. The military system is already a kind of machine, pursuing its own agenda, just as states, corporations and institutions of all kinds are. These machines are made of people, and their minds are the minds of people – increasingly augmented by information technology, from clay tablets to search engines. We like to think that this augmentation increases our effective intelligence, but as soon as words are written down, thinking rigidifies. Yet one essential fact remains: each human mind, dazzled and lost in the maze of knowledge, of law and the machinery of institutions, remains tethered intimately and existentially to a human heart. It is that tether which the military complex, enabled by technology, now threatens to break. UNQUOTE
51 52
http://www.skynet.net/ https://en.wikipedia.org/wiki/Citizens_United_v._Federal_Election_Commission 53 http://icrac.net/author/gubrud/ 54 http://icrac.net/ 55 http://icrac.net/category/analysis/ 56 http://icrac.net/2012/11/the-principle-of-humanity-in-conflict/
15 Drones (semi) autonomous out of human loop / Studies / Drones / folder
16
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
It is not just the corporations, with secret agendas, contrary to that of the common man. The Atlantic writes, Dec 14, 2012 QUOTE57 In recent decades, America's elite -- its elected officials, bureaucrats, and CEOs, for starters -- have succeeded spectacularly at insulating themselves from responsibility for their failures. As the global economy melted down, Wall Street got bailouts. CEOs who preside over shrinking companies still depart with "golden parachute" severance packages. The foreign-policy establishment remains virtually unchanged despite the catastrophic errors so many of its members made during the Iraq War. UNQUOTE
Semi-Autonomous Peru Drone Archeologist (3 Jan 10)
A drone, which can fit into a back pack, flew high in the Andes in Peru, for 3-D mapping of archaeological sites.58
USN Robot Guns (2 Nov 29)
USN = United States Navy The US Navy has an Aegis Combat System,59 which is capable of autonomously tracking enemy aircraft and guiding weapons onto them.60
Israeli Iron Dome (2 Nov 29)
I suspect, although I do not know for sure, that the Iron Dome system is like USN Aegis. Rockets come flying into Israel, in large numbers. Iron Dome radars detect them, calculate where they will land, if not intercepted, then shoot down the ones headed for population centers, with anti-rocket missiles which cost 150 times as much as the incoming missiles.61 Given the speed of decision making needed, I don’t see that having a human in the loop is essential for the perfect defense system. “Yes, shoot that down.” “Yes, shoot that down.” “Yes, shoot that down.” “Yes, shoot that down.” “Yes, shoot that down.” To infinity. Israel also has autonomous drones.62
57
http://www.theatlantic.com/politics/archive/2012/12/when-the-us-shoots-a-human-should-be-pulling-thetrigger/266254/ 58 https://www.scientificamerican.com/podcast/episode.cfm?id=drones-accelerate-archaeological-si-12-1231 59 http://www.navy.mil/navydata/fact_display.asp?cid=2100&tid=200&ct=2
60
http://www.slate.com/articles/technology/future_tense/2012/11/lethal_autonomous_robots_drones_tms_an d_other_military_technologies_raise.single.html 61 For more info on Iron Dome, see my Israel Gaza 2010 Nov Conflict notes. I have uploaded them various places, such as my Google Drive Disaster Avoidance Collection. https://docs.google.com/folder/d/0B9euafJH4bZMTA0YTM0YzktNTI0YS00NjVhLTg5NTItY2RiZjhiM2MzODkw/edit 62 http://defense-update.com/20121231_more-hermes-900-drones-for-idf.html
16 Drones (semi) autonomous out of human loop / Studies / Drones / folder
17
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
Sky Net X47B lands on USN Aircraft Carrier (3 Mar 04) !
USN = United States Navy. X-47B = a USN drone which is to operate without human control.63 Here are pictures of the X-47-B drone, allegedly flown without a human controlling it, trying to land on an aircraft carrier, and not quite making it. 64 I had heard that other drones in similar circumstances had already learned how to do this,65 so either these are old pictures, or the software which worked on other drones is not being shared, by the military, across drone manufacturers. The X47-B drone is stealth.66 The plan is for it to fly missions, by artificial intelligence (AI) and GPS, from air craft carriers.67 Contrary to some articles’ claims, we may never know which drone was the first to be piloted exclusively without a human in the loop. I suspect some time in WW II when Germany shooting at Britain with V1 and V2, until shortly after D-Day. I am surprised British news media forgot this. Then there is the indoor drone without human or GPS (which can be hacked – see Drone Terms.).68 UAV = Unmanned Air Vehicle, meaning no man on board, but men are needed to remotely operate it. I desire a shorter word than Autonomous, and Semi-Autonomous for Unmanned Unmanned Air Vehicle, meaning no men involved in remote operations, which has a clear meaning without the legal double speak of military for AWS and SAWS. For the time being, I will use Sky Net, because that’s the popular mis -conception in the anti-war movement.
US Space Plane Drone X-37B (2 Dec 11)
The X-37B bears a superficial resemblance to the space shuttle. What I have read about it so far is unclear whether it is an ordinary drone which can go into space, or if artificial intelligence is involved.69 There is a lot of sloppiness in descriptions. Not having a human pilot on board, is not the same as not having a human pilot period.
US Military Drone future plans (2 Nov 28)
DOD = Department of Defense Here from my Drone Reports:
63
http://icrac.net/2012/03/us-navy-developing-pilotless-drones/ http://www.bbc.co.uk/news/magazine-21576376 64 http://aviationintel.com/2012/11/28/x-47b-lands-on-aircraft-carrier-for-the-first-time-um-sort-of/ http://aviationintel.com/2012/12/11/x-47b-goes-to-sea/ 65 http://aviationintel.com/2012/12/20/need-an-interim-uav-solution-for-american-aircraft-carriers-take-agood-hard-look-at-the-mothballed-s-3-viking/ 66 http://www.dailymail.co.uk/sciencetech/article-2240394/X-47B-stealth-drone-hoped-carrier-borneunmanned-aircraft.html 67 http://gcn.com/articles/2012/12/11/navy-x47b-drone-first-military-robot.aspx 68 See chapter = Autonomous Drones Exist. 69 X-37-B http://aviationintel.com/2012/12/14/three-rocketsthree-historic-events-one-week/ http://aviationintel.com/2012/06/18/x-37b-returns-home-what-was-it-doing-up-there-for-over-a-year/ https://plus.google.com/u/0/107040353898400532534/posts/eTLzfhTDaNo
17 Drones (semi) autonomous out of human loop / Studies / Drones / folder
18
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
Congress House Transcript Drone Future Warfare 2010 March70 134 pages DOD Autonomous Drones 2012 July71 125 pages DOD Autonomous Weapon Policies 2012 Nov72 15 pages DOD Drone UAS Integrated Roadmap 200973 210 pages
Other sources (2 Nov 29) Here from my Drone Reports:
Evolution of Drone Warfare74 (7 pages) Brad Allenby and Carolyn Mattick explain why new technologies mean we need to rewrite the “rules of war.”75
70
2012 Oct 10 downloaded public domain. It is House Hearing Transcript 111-118 or PDF 64-921 Abstract Summary = http://www.hsdl.org/?abstract&did=4913 Full 4 Meg Transcript = https://www.hsdl.org/?view&did=4913 2012 Nov 28 I started a Drone Robots doc, on Autonomous Killer Robot Drones, which references this document, so what I have on it, in time may diverge from references in Drone Reports. 71 2012 Sep 05 I obtained this 125 page document by DSB = Defense Science Board http://www.acq.osd.mil/dsb/reports/AutonomyReport.pdf thanks to it being mentioned here: http://www.theatlantic.com/international/archive/2012/09/how-terrified-should-we-be-of-the-pentagonsplan-to-automize-drones/261944/ Also see Autonomous Drones in Drone Terms document. 2012 Sep 05 uploaded to my Google Drive Drone Info Document collection. 2012 Sep 05 uploaded to my Linked In / Box Net / Drone Info / Drone Sources sub-folder 2012 Oct 12+ Review: I have not yet read this. I tentatively plan to study DoD from smallest to largest. 2012 Nov 28 I started a Drone Robots doc, on Autonomous Killer Robot Drones, which references this document, so what I have on it, in time may diverge from references in Drone Reports. 72 2012 Nov 24 downloaded http://cryptome.org/dodi/dodd-3000-09.pdf 2012 Nov 28 I started a Drone Robots doc, on Autonomous Killer Robot Drones, which references this document, so what I have on it, in time may diverge from references in Drone Reports. There are many articles talking about this document, and related issues, such as: http://icrac.net/2012/11/dod-directive-on-autonomy-in-weapon-systems/ 73 DoD Drone roadmap as of 2009, 2012 July 15, or earlier, I had obtained this download. 2012 July 15 uploaded to my Google Drive Drone Info Document collection. 2012 Sep 03 uploaded to my Linked In / Box Net / Drone Info / Drone Sources sub-folder. 2012 Oct 08+ Review: I had barely begun studying this monster report, when I got distracted by many other smaller reports. I plan to delay return until I am caught up on the smaller ones, if that is possible. 2012 Nov 28 I started a Drone Robots doc, on Autonomous Killer Robot Drones, which references this document, so what I have on it, in time may diverge from references in Drone Reports. 74 2012 Oct 18 discovered at http://www.general-files.com/download/gs4d92955fh35i0/Evolution%20of%20Drone%20Warfare.html claimed to have come to them from Scribd 2012 Nov 28 I started a Drone Robots doc, on Autonomous Killer Robot Drones, which references this document, so what I have on it, in time may diverge from references in Drone Reports.
75
http://www.slate.com/articles/technology/future_tense/2012/11/drones_cyberconflict_and_other_military_t echnologies_require_we_rewrite.html
18 Drones (semi) autonomous out of human loop / Studies / Drones / folder
19
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
HRW = Human Rights Watch (2 Nov 28) !
Their report launched a debate, an important conversation, which previously had been much more low key, with less dimensions. Here from my Drone Reports: HRW76 Killer Robots Losing Humanity 2012 Nov77 55 pages The HRW report is accompanied by some articles.78
Lawfare on HRW (3 Feb 28)
Many great sources have more content on some topics than I find time to read and digest all of what they have. Feb 28, 2013, Lawfare’s Benjamin Wittes has another discussion of the HRW position.79 Dec 11, 2012, Lawfare shared a recommended reading list regarding recent discussion of Autonomous Robots.80 This included, QUOTE Human Rights Watch and Harvard Law School Human Rights Clinic, “Losing Humanity: The Case Against Killer Robots,” November 19, 2012. According to Lawfare, this report makes HRW’s case to ”[p]rohibit the development, production, and use of fully autonomous weapons through an international legally binding instrument. Adopt national laws and policies to prohibit the development, production, and use of fully autonomous weapons. Commence reviews of technologies and components that could lead to fully autonomous weapons. These reviews should take place at the very beginning of the development process and continue throughout the development and testing phases.” UNQUOTE
76 77
HRW = Human Rights Watch 2012 Nov 23 downloaded http://www.hrw.org/sites/default/files/reports/arms1112ForUpload_0_0.pdf from http://www.hrw.org/reports/2012/11/19/losing-humanity-0 and see press release http://www.hrw.org/news/2012/11/19/ban-killer-robots-it-s-too-late which I found out about thanks to http://afgeneralcounsel.dodlive.mil/2012/11/23/killer-robots-part-3/ and http://afgeneralcounsel.dodlive.mil/2012/11/26/killer-robots-part-4/ and http://www.cnas.org/blogs/abumuqawama/2012/11/rage-against-machines.html and http://www.lawfareblog.com/2012/11/human-rights-watch-report-on-killer-robots-and-our-critique/ many of which have some discussion of the merits of the report. 2012 Nov 28 I started a Drone Robots doc, on Autonomous Killer Robot Drones, which references this document, so what I have on it, in time may diverge from references in Drone Reports. 78 Ban ‘Killer Robots’ Before It’s Too Late | Human Rights Watch » http://www.hrw.org/news/2012/11/19/ban-killer-robots-it-s-too-late 79 http://www.lawfareblog.com/2013/02/greg-mcneal-on-human-rights-watch-and-killer-robots/ 80 http://www.lawfareblog.com/2012/12/readings-autonomous-weapon-systems-and-their-regulation/
19 Drones (semi) autonomous out of human loop / Studies / Drones / folder
20
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
Allenby and Mattick (2 Dec 12)
Brad Allenby and Carolyn Mattick jointly wrote an article, in Slate’s Future Tense explaining why they think we need new rules of war.81 Then other writers commented on their article, extending a debate. One of the responses, to the Allenby and Mattick article, came from ICRC,82 which Lawfare then commented further on.83 See separate Red Cross chapter section.
Brad Allenby is the Lincoln professor of engineering and ethics; a professor of civil,
environmental, and sustainable engineering; and the founding chairman of the Center for Earth Systems Engineering and Management at Arizona State University. 84 Brad Allenby wrote an article, in Slate’s Future Tense, explaining why he thinks Human Rights Watch does not understand military realities on the battlefield. 85
Carolyn Mattick is a Ph.D. candidate in environmental engineering and a graduate
fellow in emerging technologies at the Lincoln Center for Applied Ethics86 at Arizona State University.87
Anderson, Ken (2 Dec 25)
The USAF Ethics Blog informs us,88 Dec-16, 2012 QUOTE: In two posts, one on Opinio Juris89 and the other on LawFare,90 Kenneth Anderson has summarized leading articles, reports and blog posts on the issue of proposed limits on autonomous weapon systems. UNQUOTE I visited his brief biographies. I am familiar with all that he mentioned, and more. But the material, which he cites, is among the most important recent ingredients.
ASIL Autonomous Weapons Backgrounder (3 Jan 23)
ASIL = The American Society of International Law.
81
http://www.slate.com/articles/technology/future_tense/2012/11/drones_cyberconflict_and_other_military_t echnologies_require_we_rewrite.html 82 ICRC = International Committee of the Red Cross http://intercrossblog.icrc.org/blog/how-international-law-adapts-new-weapons-and-technologies-warfare 83 http://www.lawfareblog.com/2012/12/the-icrc-and-slate-an-exchange-on-weapons/ 84 http://www.slate.com/authors.brad_allenby.html
85
http://www.slate.com/blogs/future_tense/2012/11/20/human_rights_watch_s_case_against_killer_robots_re port_misunderstands_the.html 86 http://lincolncenter-dev.asu.edu/ 87 http://www.slate.com/authors.carolyn_mattick.html 88 http://afgeneralcounsel.dodlive.mil/2012/12/16/crs-on-detention-of-u-s-persons-as-enemy-belligerents/ 89 http://opiniojuris.org/2012/12/11/autonomous-weapon-systems-and-regulation-a-brief-bibliography/ 90 http://www.lawfareblog.com/2012/12/readings-autonomous-weapon-systems-and-their-regulation/
20 Drones (semi) autonomous out of human loop / Studies / Drones / folder
21
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
ASIL Insights = short, descriptive pieces on topical issues meant as non-technical “backgrounders” for journalists, the general public, and anyone looking for a quick path into an international law topic; they represent solely the author’s views, but are written to give an understanding of the background legal issues.) According to Lawfare Jan 19, 2013 QUOTE91 “The Law That Applies to Autonomous Weapon Systems” is written by Jeffrey S. Thurnher, a JAG officer on faculty at the Naval War College; it is short, crisp, and a useful guide to understanding the legal issues raised by the possibility of increasingly automated weapon systems that might one day be fully autonomous. (Also recommended is Major Thurnher’s more detailed Octobe r 2012 article in Joint Force Quarterly (National Defense University, Washington DC, Vol. 67, No. 4, Oct. 2012), “No One at the Controls: Legal Implications of Fully Autono mous Targeting.”) UNQUOTE According to Lawfare, this ASIL Insight piece is organized around two legal issues: 1. Is the weapon itself lawful? 2. Are there warfare conditions under which it would not be legal to use this weapon? According to Lawfare, this ASIL Insight piece also lists, and discusses, fundamental pieces of weapons law: 1. Under Geneva Conventions, weapons need to be able to discriminate between different targets. 2. Weapons should not cause unnecessary suffering or superfluous injury. It is Ok to kill people. It is not Ok to torture them.
Bolton, Matthew IRAC Overview (2 Dec 12)
Note: most of the hyper links got broken, as soon as I cut pasted them to these notes. Thus the importance of copying the relevant urls within footnotes. Dr. Matthew Bolton is assistant professor of political science at Pace University in New York City.92 He has worked as an academic, aid worker and a freelance journalist in 15 countries. He talks about similar treaties to what is being proposed. I started stubs near the bottom, for each of those treaties, so the references are conveniently placed, in case we later wish to expand upon them. M Bolton,93 of Political Mine Fields,94 in International Committee for Robot Arms Control (IRAC)95 Analysis96 Oct 3, 2012:97 QUOTE
91
http://www.lawfareblog.com/2013/01/readings-jeffrey-s-thurnher-on-law-of-armed-conflict-applied-toautonomous-weapon-systems/ 92 http://www.pace.edu/dyson/academic-departments-and-programs/political-science/faculty/matthewbolton 93 http://icrac.net/author/gubrud/ and
21 Drones (semi) autonomous out of human loop / Studies / Drones / folder
22
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
In the last few years there have been increasing calls for a global regulatory framework to govern robotic weapons.98 Oct-2, influential counter-terrorism correspondent Peter Bergen called for a treaty to manage and mitigate drone proliferation99. Earlier this year, along with Thomas Nash and Richard Moyes of the advocacy group Article 36, I called for a complete ban on autonomous armed robots,100 saying “Decisions to kill and injure should not be made by machines and, even if at times it will be imperfect, the distinction between military and civilian is a determination for human beings to make.” More conservative commentators have also recognized the need for some normative framework for robotic weapons.101 In 2011, an interdisciplinary group of legal, security and defense intellectuals from Consortium on Emerging Technologies, Military Operations, and National Security (CETMONS), argued in a paper for the Columbia Science and Technology Review that: UNQUOTE Then he goes on to quote some of it. I want to go to the original and study for myself. I have a backlog of that type of wishing. See separate CETMONS section. M Bolton 102 continues in IRAC103 Oct 3, 2012:104 QUOTE The 2009 statement by International Committee for Robot Arms Control (ICRAC), which called for bans on autonomous weapons, the arming of robotic weapons with nuclear arms and the deployment of armed robots in space 105 is probably one of the most developed proposals I have seen. As a member of ICRAC,106 I have endorsed this statement. However, I have been considering ways to flesh out the ICRAC proposal with further provisions building on the precedents of the landmine107 and cluster munition treaties,108 as well as the draft Arms Trade Treaty109 (currently still being negotiated) and the broader corpus of international humanitarian law governing conduct in war.110
http://politicalminefields.com/aboutme/ 94 http://politicalminefields.com/aboutme/ 95 http://icrac.net/ 96 http://icrac.net/category/analysis/ 97 http://icrac.net/2012/10/debating-proposals-on-a-possible-international-convention-on-robotic-weapons/ 98 http://wiredforwar.pwsinger.com/ 99 http://politicalminefields.com/2012/10/01/bergen-time-for-drone-treaty/ 100 http://www.article36.org/statements/ban-autonomous-armed-robots/ 101 https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2046375 I downloaded this “Law and Ethics for Robot Soldiers … see the chapter SSRN = Social Science Research Network. 102 http://icrac.net/author/gubrud/ 103 http://icrac.net/ 104 http://icrac.net/2012/10/debating-proposals-on-a-possible-international-convention-on-robotic-weapons/ 105 http://icrac.net/statements/ 106 http://icrac.net/who/ 107 http://www.icbl.org/index.php/icbl/Treaty 108 http://www.stopclustermunitions.org/the-solution/the-treaty/ 109 http://politicalminefields.com/2012/07/26/new-draft-of-the-arms-trade-treaty/ 110 http://www.icrc.org/eng/war-and-law/index.jsp Red Cross
22 Drones (semi) autonomous out of human loop / Studies / Drones / folder
23
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
Yesterday, I published an article (for Global Policy’s online Comment & Opinion section)111 and a post on my blog112 outlining possible provisions for an International Convention on Robotic Weapons. I am a political scientist, not a lawyer or roboticist, so my purpose in doing this is to provoke discussion and dialogue, not to pin down precise language. Moreover, while I have used the ICRAC statement as a foundation, this is my own personal adaptation and expansion of it and so should not be interpreted as an institutional position of ICRAC. UNQUOTE
Center for New American Security (2 Dec 12)
Many other sources are citing this source, which after I had read some of their articles, I conclude they are living in an alternate reality quite divorced from my own, like they are on a different planet, or have not been paying attention to the history of capitalism, government policies. Nov 21 2012, they have a “rage against the machines” write -up about the HRW report.113 QUOTE They find it a logically ludicrous concept. They say that no weapon is yet fully capable of taking humans out of the loop . UNQUOTE On the contrary, elsewhere in my research notes I share114 indications that pieces of such weapons are: In current usage;115 being tested; and I suspect that more advanced versions are military secrets. In my Drone Notes there is a chapter listing Technological Advances. In there is a citation to a drone, which looks like an insect, and is smaller than a person’s thumb. You have to be on G+ to be able to access the links.116 QUOTE This insect spy drone was developed for use in urban areas, and is already in production. Funded by the US Government, it can be remotely controlled, and is equipped with a camera and a microphone. It can land on you, and it may have the potential to take a DNA sample or leave RFID tracking nanotechnology on your skin, or be used in assassinations. UNQUOTE
111
http://www.globalpolicyjournal.com/blog/02/10/2012/time-ban-killer-robots-considering-internationalconvention-robotic-weapons 112 http://politicalminefields.com/2012/10/02/proposal-for-an-international-convention-on-robotic-weapons/ 113 http://www.cnas.org/blogs/abumuqawama/2012/11/rage-against-machines.html 114 As I come across more, I may add here. 115 See USN Robot Guns chapter. 116 https://plus.google.com/u/0/101378900168144468134/posts/W7v5pTCG4vd https://plus.google.com/u/0/103028459671171670815/posts/G2XhvRbtd4Y
23 Drones (semi) autonomous out of human loop / Studies / Drones / folder
24
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
I suspect this insect drone is semi-autonomous, in that humans would tell it who to target, it could go out of range of controller, then return to report, and get new instructions. When I was looking into the RFID controversy,117 I learned that the distance a radio signal can travel between some device and its network, is related to how large the radio antenna is, and how large the radio device. It is a limitation imposed by the laws of physics. This is why normal bar code scanners need to be within 6 inches of the bar code they are reading (there's also an RFID in there which is getting updated). We have a state of art today where new technology comes along, which is not governed by any legislation, spends a decade or more abusing people, then there is a battle thru the courts and legislatures to reign in that abuse, and an eternity putting a stop to it, after the leaders have ruled. Look at the police arresting people for using cameras in public, for decades after such behavior was legalized. Wouldn’t it be better to have a framework of civility defined before the new technology gets implemented? The Nov 21 2012 Center for New American Security “rage against the machines” write up about the HRW report, has many other reasons why they think this is ludicrous, such as military commanders would not want autonomous robots not subject to chain of command. I guess the author has missed a half century of history where millions of blue collar jobs have been automated, and also many white collar jobs eliminated by electronics, so he does not comprehend how the very top command wants the job done, by the cheapest possible means, so if eliminating humans from the loop is cheaper, then so be it. Dec 12 2012, they have an article on the relative silence in US media about what US drones are doing around the world.118
Accountability Myth (2 Dec 12)
The Nov 21 2012 Center for New American Security “rage against the machines” write up about the HRW report, says:119 QUOTE
117
RFID tags are physically almost microscopic, not visible to the naked eye. They are placed on and in all sorts of merchandise in their supply chain, for a variety of perfectly valid business tracking purposes. Some people are concerned about the ability to spy upon end consumers, through RFID tags still active after they take merchandise home. RFID tags are also injected into various farm animals and pets, also for legitimate purposes. There is a fear that they might also be injected into humans for illegitimate purposes. There are authorities who are promoting their use in ways which would place people at greater risk. I believe this is because they have not thought through security implications. Other observers speculate more sinister motives for the proposals. A Texas school mandates students have RFID tag on them at all times, http://www.bbc.co.uk/news/technology-20461752 which at least one student is protesting on alleged religious grounds. https://www.rutherford.org/files_images/general/11-21-2012_TRO-Petition_Hernandez.pdf School tried to suspend student for refusing to wear student ID/tracking device, but this effort was reversed by a court. http://www.techdirt.com/articles/20121125/15041521137/court-temporarilyblocks-school-district-suspending-student-refusing-to-wear-student-idtracking-device.shtml 118 http://www.cnas.org/blogs/abumuqawama/2012/12/silence-and-drones.html 119 http://www.cnas.org/blogs/abumuqawama/2012/11/rage-against-machines.html
24 Drones (semi) autonomous out of human loop / Studies / Drones / folder
25
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
The distinction HRW draws between “H uman-on-the-Loop Weapons” and “Human-out-of-the-loop” weapons is totally arbitrary, particularly since one key criteria HRW draws is that “human-on-the-loop” weapons have human oversight which can override and veto their actions, and “human -out-of-the-loop” weapons do not. UNQUOTE I am troubled by this belief that “human-on-the-loop” weapons have human oversight which can override and veto their actions. The belief does not understand latency, and is ignoring the reality of court cases around the world, particularly in Pakistan and Britain, and FOIA in the USA, demanding to know the basis for which this or that allegedly innocent family was killed, and the US military and CIA and DoJ basically giving the finger to those requests.120 There is also the lack of accountability with respect to the basis, for which military decisions are made, to target this or that alleged enemy. Remember WMD in Iraq? We still do not know, for sure, where that info came from. Theories include: Innocent suspects being tortured, who told their torturers what they thought the torturers wanted to hear, to stop the torture. Freedom Fighters in Iraq, who knew they could not over-throw Saddam dictator by themselves, but if USA would believe this lie, then USA would over-throw Saddam for them. Were there really WMD in Iraq, which are now some place else, or was this an intelligence and policy failure? Suspicions abound that similar intelligence sources and methods are being used as input to choice of who should be killed by drones, where they should be targeted. The cost, of acting on false intelligence, can be enormous.
CETMONS (2 Dec 12)
In 2011, an interdisciplinary group of legal, security and defense intellectuals from Consortium on Emerging Technologies, Military Operations, and National Security (CETMONS), argued in a paper for the Columbia Science and Technology Review… See IRAC Matthew Bolton. I downloaded the CETMONS report, naming it: STLR121 Killer Robot Governance 2011122 44 pages o Cite as http://www.stlr.org/cite.cgi?volume=12&article=7 o This work is made available under the o Creative Commons Attribution-Noncommercial-No Derivative Works 3.0 License123
120 121
See Relevant Terms Defined for many concepts and acronyms etc. found in these notes. STLR = Science and Technology Law Review at Columbia. www.stlr.org 122 2012 Dec 1 downloaded http://www.stlr.org/html/volume12/marchant.pdf finding out about it from http://icrac.net/2012/10/debating-proposals-on-a-possible-international-convention-on-robotic-weapons/ 2012 Nov 28 I started a Drone Robots doc, on Autonomous Killer Robot Drones, which references this document, so what I have on it, in time may diverge from references in Drone Reports. 123 https://creativecommons.org/licenses/by-nc-nd/3.0/
25 Drones (semi) autonomous out of human loop / Studies / Drones / folder
26
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
DoD (2 Dec 12)
DoD = US Department of Defense. Dec 11, 2012, Lawfare shared a recommended reading list regarding recent discussion of Autonomous Robots.124 This included, QUOTE Ashton B. Carter, Deputy Secretary of Defense, “Autonomy in Weapons Systems,” Department of Defense Directive, Number 3000.09, November 21, 2012.125 According to Lawfare, the Directive establishes “DoD policy and assigns responsibilities for the development and use of autonomous and semi-autonomous functions in weapon systems, including manned and unmanned platform s.” It further “establishes guidelines designed to minimize the probability and consequences of failures in autonomous and semi-autonomous weapon systems that could lead to unintended engagements.” A key concept in the document – which is designed to be applicable across many different kinds of weapons systems over time and during their development – is that the “appropriate” level and kind of human involvement in the system will depend upon many factors and circumstances. UNQUOTE
Lawfare (2 Dec 03)
I am following several blogs, which quote extensively from each other. I would rather have a chapter heading on each source, than duplicate the material. This approach encourages study of original sources, instead of just what someone else chooses to quote. Sept 12, 2012, in Defense Science Board on Autonomous Systems,126 they briefly quote from the 125 page report.
Lawfare Matt & Ken (2 Dec 10)
Nov 26 2012, in their Critique of Human Rights Watch Report on Killer Robots,127 Written by Matt and Ken, QUOTE
124 125
http://www.lawfareblog.com/2012/12/readings-autonomous-weapon-systems-and-their-regulation/ 2012 Sep 05 I obtained this 125 page document http://www.acq.osd.mil/dsb/reports/AutonomyReport.pdf thanks to it being mentioned here: http://www.theatlantic.com/international/archive/2012/09/how-terrified-should-we-be-of-the-pentagonsplan-to-automize-drones/261944/ Also see Autonomous Drones in Drone Terms document. 2012 Sep 05 uploaded to my Google Drive Drone Info Document collection. 2012 Sep 05 uploaded to my Linked In / Box Net / Drone Info / Drone Sources sub-folder 2012 Oct 12+ Review: I have not yet read this. I tentatively plan to study my many DoD downloads from smallest to largest. 2012 Nov 28 I started a Drone Robots doc, on Autonomous Killer Robot Drones, which references this document, so what I have on it, in time may diverge from references in Drone Reports. 126 http://www.lawfareblog.com/2012/09/defense-science-board-on-autonomous-systems/ See in US Military drone future plans and Drone Reports, that I have downloaded this report. 127 http://www.lawfareblog.com/2012/11/human-rights-watch-report-on-killer-robots-and-our-critique/ See in Human Rights Watch Chapter and Drone Reports, that I have downloaded HRW report.
26 Drones (semi) autonomous out of human loop / Studies / Drones / folder
27
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
Last week, Human Rights Watch (along with the Harvard Law School International Human Rights Clinic) published a report titled “Losing Humanity: The Case Against Killer Robots.” It argues for a preemptive prohibition by multilateral treaty on the development and use of “fully autonomous weapons that could select and engage targets without human intervention” — a technological feat likely to be possible a few decades from now (and, in fact, already possible in some very limited forms). Certainly the direction of weapons development is toward greater automation, and this raises a legitimate concern about whether highly automated or autonomous weapons systems would comply with the laws of war. While HRW’s new report is an important contribution to this debate – containing useful background discussion of emergent precursor systems currently in use or under development – we think it rests on some questionable premises both factual and moral, and reaches some overly sweeping and unwise conclusions. UNQUOTE The critique has a US military centric viewpoint, which fails to appreciate the arms race around the world.128 There are always nations that the US is in a conflict with, where the US cannot be in a leadership position, so long as we are on the brink of war with each other. I believe there needs to be treaty negotiation thru the UN. I agree that treaties can pre-judge science, as we have seen with peaceful uses of outer space, opposition to stem cells, cloning, research banned in one nation, becomes a multi-million dollar industry in another. But treaties, like laws, can and should be rethought as technology advances. An important challenge is the issue of holding individuals criminally liable for war crimes when lethal systems operate autonomously. I agree that International Law Courts have failed to live up to their expectations, but this is because most nations of the world give the finger to their rulings, when they consider their national interests more important. Nov 27 2012, in Pentagon Directive on “Autonomy in Weapon Systems” »129 QUOTE Speaking of autonomous weapons systems, which Matt and Ken130 and Human Rights Watch131 all were, the Department of Defense has issued a directive entitled “Autonomy in Weapon Systems.”132 It declares: Then they quote some of it. UNQUOTE
128
See my Drone Nations which documents that drones are active in over 100 nations in our world, with many serious adversaries competing in a drone technology arms race. 129 http://www.lawfareblog.com/2012/11/pentagon-directive-on-autonomy-in-weapon-systems/ 130 See Nov 26 2012 from Lawfare. 131 See in Human Rights Watch Chapter and Drone Reports, that I have downloaded HRW report. 132 See in US Military drone future plans and Drone Reports, that I have downloaded this report. See DoD chapter section.
27 Drones (semi) autonomous out of human loop / Studies / Drones / folder
28
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
Dec 01, 2012 in Does Human Rights Watch Prefer Disproportionate and Indiscriminate Humans to Discriminating and Proportionate Robots?133 QUOTE To reach this conclusion, the report’s authors m ake a lot of assumptions about the technology that may well prove wrong QUOTE Al Mac Observation: It seems to me that a lot of authors are making assumptions, without any evidence to substantiate their positions. I have seen human nature in the work place since the 1960’s. Human nature is not going to change, unless there are government regulations, and audits, with good oversight. I agree that over time, technology development may get some things functioning better than human decision-making. But rather than: HRW position that because, right now, humans can do it better, but as I have noted elsewhere, often do not,134 that we should ban the inferior machines. Opposition position that has confidence in technology development, so there should be no such ban. I think there could be standards: Do not deploy said technology until it achieves specified standards.
Lawfare Concepts (2 Dec 12)
Dec 11, 2012, Lawfare came out with a recommended reading list regarding recent discussion of Autonomous Robots, which included:135 QUOTE William Marra and Sonia McNeil, “Understanding “The Loop’: Regulating the Next Generation of War Machines,” 36 Harvard Journal of Law and Public Policy 3 (2013), Lawfare Research Paper Series 1-2012. According to Lawfare, this paper (which inaugurated the Lawfare Research Paper Series in May 2012) provides a detailed explanation of the distinct technical meanings of “automation,” “autonomy,” “The Loop,” an d other key concepts essential to intelligently regulating gradually automating weapons systems, using the evolution of today’s rapidly automating drones as an example. The paper brings together international law, US government regulation and weapons review processes, engineering concepts, and military strategy in order to yield a precise, common vocabulary for debating the proper forms and modes of regulation of incrementally automating weapons systems. In the language of engineers, say Marra and McNeil, “tomorrow’s drones are expected to leap from ‘automation’ to true ‘autonomy’.” As a consequence, regulations for today’s drones must be “crafted with an eye towards tomorrow’s technologies.
133
http://www.lawfareblog.com/2012/12/does-human-rights-watch-prefer-disproportionate-andindiscriminate-humans-to-discriminating-and-proportionate-robots/ 134 See for example, the Feb 2010 attack in Drone Dates, where US military attacked children by mistake, then waited 12 hours before reporting it. 135 http://www.lawfareblog.com/2012/12/readings-autonomous-weapon-systems-and-their-regulation/
28 Drones (semi) autonomous out of human loop / Studies / Drones / folder
29
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
Yet today’s debates about humans and ‘the loop’ rely on langua ge too imprecise to successfully analyze the relevant differences between drones and predecessor technologies.” They conclude that needless confusion and possibly policy error pervade discussions about “when an advanced technological system is autonomous and what the implications of autonomy might be … language useful to the policymaking process has already been developed in the same places as drones themselves — research and engineering laboratories around the country and abroad.” UNQUOTE
Lawfare Reading List (2 Dec 12)
Dec 11, 2012, Lawfare came out with a recommended reading list regarding recent discussion of Autonomous Robots.136 QUOTE
Ashton B. Carter, Deputy Secretary of Defense, “Autonomy in Weapons Systems,” Department of Defense Directive, Number 3000.09, November 21, 2012. See DoD chapter section for more info. Human Rights Watch and Harvard Law School Human Rights Clinic, “Losing Humanity: The Case Against Killer Robots,” November 19, 2012. See HRW section for more info. Michael N. Schmitt, International Law Department, US Naval War College, “Autonomous Weapon Systems and International Humanitarian Law: A Reply to the Critics,” SSRN Working Paper, Draft of December 4, 2012. See SSRN chapter section for more info. William Marra and Sonia McNeil, “Understanding “The Loop’: Regulating the Next Generation of War Machines,” 36 Harvard Journal of Law and Public Policy 3 (2013), Lawfare Research Paper Series 1-2012. See Lawfare Concepts chapter for more info. Kenneth Anderson and Matthew Waxman, “Law and Ethics for Robot Soldiers,” Policy Review, December-January 2012-13 (final published version at Policy Review here, and working draft with footnotes at SSRN here). See SSRN chapter section for more info.
UNQUOTE They plan to make additions to this over time.
Lawfare Wittes and Malinowski (2 Dec 11)
Benjamin Wittes is a Senior Fellow in Governance Studies at the Brookings Institution, where he co-directs the Harvard Law School-Brookings Project on Law and Security. He
136
http://www.lawfareblog.com/2012/12/readings-autonomous-weapon-systems-and-their-regulation/
29 Drones (semi) autonomous out of human loop / Studies / Drones / folder
30
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
is the author of several books and a member of the Hoover Institution's Task Force on National Security and Law.137 Benjamin Wittes has posted several times on Lawfare. Tom Malinowski is with Human Rights Watch. They each write articles, which comment on each other’s prior writings. Here, Lawfare says they are quoting Mr Malinowski’s post in its entirety, after some preliminary remarks:138 QUOTE Some of the arguments below (and by others in favor of a treaty ban on autonomous lethal systems) assume the likelihood of an effective international prohibition. We don’t share that rosy assumption, especially with regard to compliance by states we’re most worried about.139 Specifically regarding Tom’s point below that an international ban on autonomous lethal systems will help ensure that they’re never used by brutal dictators like Syria’s Assad on their own people, as we write this we’re also reading U.S. government warnings to Assad that he’ll probably face direct U.S. action if he uses his vast caches of chemical weapons against the rebels. We don’t think it’s the Chemical Weapons Convention that’s stopping him (the CWC hasn’t stopped him from gaining this capability in the first place, and we’re dubious that it’s serving as a direct barrier to him using chemical weapons; international law here probably does serve as an important background marker for the U.S. and others legitimately to act against him, though). Nor do we think it’s the moral code of his commanders that will determine whether he uses these weapons. And nor do we think he feels he needs sophisticated robotic systems to wipe out civilians and their property. This is a case where only a strong combination of power and principles will stop Assad, and that’s what we are trying to promote in our proposals. UNQUOTE Here, Mr. Wittes shares what Mr. Malinowski recently wrote, regarding lethal autonomous systems, and replies to that.140 The good news for them, is that their areas of disagreement are narrowing. For example, Mr. Malinowski does not appear opposed to the principle of weapon systems which shoot down incoming attacks. Mr. Wittes talks about the potential for future weapon systems which might solve some of today’s challenges. QUOTE Imagine a robot designed for targeted strikes against individual targets. The target, a senior terrorist figure, has already been selected, but he’s living in a compound with large numbers of civilians—wives and kids, say—with whom he is interacting on an ongoing basis. Precisely to discourage drone strikes, he keeps
137 138
http://www.lawfareblog.com/author/benjamin/ http://www.lawfareblog.com/2012/12/tom-malinowski-responds-on-autonomous-lethal-systems/ 139 I share Lawfare’s observation about the ineffectiveness of International bans. 140 http://www.lawfareblog.com/2012/12/tom-malinowski-responds-on-lethal-autonomous-systems-part-ii/
30 Drones (semi) autonomous out of human loop / Studies / Drones / folder
31
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
himself close to these and other civilians. Now imagine that our new robot has a few advances—none of them unthinkable, in my view—over current technology. First, it is very small and quiet and thus hovers very low, allowing it to strike incredibly quickly. Second, it has very long loiter time, so it can watch the target for long periods. Third, its on-board weapon has an extremely small blast radius; maybe it crashes into the target’s head and explodes, but it’s specifically designed not to destroy the building (and the other people in the building) in doing so. And fourth—and critically—to make sure that the strike does not kill the civilians in the compound, the robot decides autonomously when to attack, exploiting splitsecond calculations about the target’s distance from the civilians in relation to the weapon’s blast radius. If you think this robot sounds fantastical, consider this one—which already exists.141 UNQUOTE
Harvard National Security Journal (3 Feb 18)
“Out of the Loop” Autonomous Weapon Systems and the Law of Armed Conflict, published Feb 5, 2013, may be downloaded from Social Science Research Network (SSRN).142 According to Lawfare, this article, by two members of the international law department of the US Naval War College, discusses various issues related to lethal autonomous weapon systems. It argues vigorously against recent calls by Human Rights Watch (HRW) and others for a sweeping, total international ban on autonomous weapon systems.
Red Cross, ICRC (2 Dec 12)
ICRC = International Committee of the Red Cross. One of the responses, to the Allenby and Mattick article,143 came from ICRC,144 which Lawfare then commented further on.145 The Red Cross says that two of the questions raised by Allenby and Mattick reflect a serious misunderstanding of the nature of weapons, and the laws of war. The two questions the Red Cross is referring to are: QUOTE One question was whether international humanitarian law is properly suited to emerging new means and methods of warfare. Another was why “lethal” weapons are preferable over those intended to be “non-lethal". UNQUOTE The Red Cross goes on to explain QUOTE Often when new technologies and weapons emerge there are concerns expressed that they are not covered by existing law . UNQUOTE They explain why this is a bogus understanding. New weapons are not
141
http://www.google.com/hostednews/afp/article/ALeqM5jTPhInUoZJNu1kT604y1CV0xM0w?docId=CNG.b0b893fb04aee2445cb9d54b02462bd7.2f1 142 https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2212188 http://www.lawfareblog.com/2012/12/readings-autonomous-weapon-systems-and-their-regulation/ http://www.lawfareblog.com/2013/02/readings-autonomous-weapon-systems-links-post-updated/ 143 See separate chapter section on what they had to say. 144 ICRC = International Red Cross http://intercrossblog.icrc.org/blog/how-international-law-adapts-new-weapons-and-technologies-warfare 145 http://www.lawfareblog.com/2012/12/the-icrc-and-slate-an-exchange-on-weapons/
31 Drones (semi) autonomous out of human loop / Studies / Drones / folder
32
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
allowed under international law, until they are specifically authorized, through amendments to the various treaties. The lack of regulations, for a weapon system, means it cannot be used.
Al Mac observation: If the ICRC explanation is valid, then this might explain why
various nations, and the UN, have declared that US military use of drones constitute war crimes … they are not being used in accordance with international human rights laws. The ICRC points out that sometimes, thanks to new technology, the old laws and treaties may seem unclear. Thus there is an international discussion seeking to improve that clarity. They also explain why it is meaningless to characterize any weapon as lethal or non-lethal. A handgun can be fired into the air. Hundreds of people have been killed using weapons labeled as non-lethal.
Robinson, Paul U of Ottawa (2 Nov 29) Paul Robinson is a professor in the Graduate School of Public and International
Affairs at the University of Ottawa. He is the author and editor of numerous books on military history and military ethics, including Military Honour and the Conduct of War: From Ancient Greece to Iraq.146 He wrote an article Who Will Be Accountable for Military Technology? It was published Nov 15, 2012 in Slate’s Future Tense. 147 This article was referenced in the USAF General Counsel Blog. See chapter section on Killer Robots Part I. He points to several people who have called for an international convention regulating the use of lethal robot drones, with many relevant links in his article. There’s the issue that machines, such as robot drones, cannot be held legally accountable for their actions. QUOTE As technology advances, we face a very real danger that it will become increasingly difficult to hold those who wage war on our behalf accountable for what they do. Artificial intelligence is not the only technology to pose such accountability problems. Others, such as bio-enhancement, do, too, although of a different sort. UNQUOTE
Accountability Complexity (2 Nov 29)
In my view, the accountability difficulty is not just because of the advance of technology, where future shock can impact many people, it is also the complexity of human organizations, their secrecy, and lack of seeing the big picture.
146 147
http://www.slate.com/authors.paul_robinson.html
http://www.slate.com/articles/technology/future_tense/2012/11/lethal_autonomous_robots_drones_tms_an d_other_military_technologies_raise.single.html
32 Drones (semi) autonomous out of human loop / Studies / Drones / folder
33
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
Companies exist in a government regulated industry. They buy software to satisfy their record keeping meets government and other mandates. Then it is discovered that the software was not doing the job properly. Who is accountable for this? The personnel who made the software acquisition decisions, and implementation, are no longer with the company, either the end user, or the software company. Several years ago, a Stanford Professor got his hands on Voting Machines made by multiple manufacturers. He assigned one class the task of finding ways to spoof them, then a second class to deduce how the first class had spoofed voting info in specific machines. Collectively they found scores of ways to spoof all the machines. This info was then published. The reaction was for many state legislatures to prohibit such behavior in the future, since it supposedly violated trade secrets of the Voting Machine companies. So now we have a reality in which we know these machines can be hacked and spoofed, but it is illegal to do anything about it. Many states have no paper trail, so no way to audit whether the votes were sabotaged. Who is accountable for this? State Legislators who bowed more to Industry Lobbyists than to Academic Ethical desires. With the voting compromised, can voters even get this fixed? Drones are manufactured, using chips from other vendors, which are supposed to perform well defined functions. The drones are then sold to government agencies, which use info from independent intelligence agencies, and personnel in other places in government hierarchy to decide what the drones are to do. It is then found that they are not doing what was desired. Is the problem misinformation, malfunction, misunderstanding?
Singer, Peter on Robotics Revolution (2 Dec 12) Peter W. Singer is the director of the 21st Century Defense Initiative and a senior
fellow in Foreign Policy at Brookings. Singer’s research focuses on three core issues: the future of war, current U.S. defense needs and future priorities, and the future of the U.S. defense system. Singer lectures frequently to U.S. military audiences and is the author of several books and articles, including Wired for War: The Robotics Revolution and Conflict in the 21st Century.148 Peter W. Singer wrote about the Robotics Revolution, Dec 11, 2012, in Open Canada, a hub for International Affairs. 149 He writes about the history of drones, and what we have been seeing in the news media, QUOTE While unmanned systems have a long history, dating back to Da Vinci’s designs for a robotic knight and including things like German remote-controlled torpedo boats in WW I, it wasn’t until a decade ago that they truly took off in war. Advances in technology made unmanned systems more usable, especially through the incorporation of GPS technology which allowed such systems to locate themselves in the world. At the same time, the new conflicts that followed 9/11 drove demand. UNQUOTE
148 149
More info about Peter Singer here: http://opencanada.org/author/petersinger/ http://opencanada.org/features/the-think-tank/essays/the-robotics-revolution/ See Center for New American Security commenting on this article, Dec 12, 2012. I found out about both, thanks to USAF General Counsel. http://afgeneralcounsel.dodlive.mil/2012/12/12/rpas-drones-and-accountability-for-military-action/
33 Drones (semi) autonomous out of human loop / Studies / Drones / folder
34
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
We are at the beginnings of what a new technology will be able to accomplish. It can revolutionize our society like the horseless carriage, and computer. It also has a dark side. This is a huge article, exploring many implications. Peter Singer writes, QUOTE One of the laws in action when it comes to technology is Moore’s Law, that the computing power that can fit on a microchip doubles just under every two years or so. It has become an encapsulation of broader exponential trends in technology that have occurred through history, with technology constantly doubling upon itself in everything from power to storage to broader innovation patterns. If Moore’s Law holds true over the next 25 years, the way it has held true over the last 40 years, then our chips, our computers, and, yes, our robots will be as much as a billion times more powerful than today. UNQUOTE However, he may not recognize that there is a limitation, in physics, to how small things can be made. We now have quantum computers, where their smallest bits are electrons neutrons protons. Science does not yet know how to get smaller than that.
Slate (2 Dec 12)
According to USAF General Counsel Blog,150 there have been several articles recently on Slate, that ICRC disagrees with.151 QUOTE One Slate post argued that international law would need to be revised to reflect the new weapons and war we are fighting.152 The second argued for a change in the treatment of nonlethal weapons. 153 In a post on the ICRC blog, ICRC legal advisor Neil Davison offers a detailed response that is well worth reading. Not surprisingly, he disagrees with both Slate articles.154 UNQUOTE As usual, I want to study original sources, crediting how I found them.
SSRN = Social Science Research Network (2 Dec 03)
April 28 2012, they posted Law and Ethics for Robot Soldiers.155 I am adding it to my Drone Reports, to study as my time permits. I named my copy: SSRN156 Robot Soldier Law and Ethics 2012 April 157 20 pages
150 151
http://afgeneralcounsel.dodlive.mil/2012/12/05/the-icrc-responds-to-slate/ See chapter section on the Red Cross. 152 Page not found, and http://afgeneralcounsel.dodlive.mil/2012/11/13/war-with-nonstate-actors/ See Allenby and Mattick chapter section. 153 http://afgeneralcounsel.dodlive.mil/2012/11/14/the-impact-of-nonlethal-weapons/ http://www.slate.com/articles/technology/future_tense/2012/11/nonlethal_weapons_and_the_law_of_war.h tml 154 http://intercrossblog.icrc.org/blog/how-international-law-adapts-new-weapons-and-technologies-warfare 155 https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2046375 156 SSRN = Social Science Research Network
34 Drones (semi) autonomous out of human loop / Studies / Drones / folder
35
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
Matt and Ken have also written about this topic, other places. 158 (See footnotes.) I have downloaded some other SSRN documents on drones, but not on autonomous ones. Dec 11, 2012, Lawfare shared a recommended reading list regarding recent discussion of Autonomous Robots.159 This included, QUOTE Michael N. Schmitt, International Law Department, US Naval War College, “Autonomous Weapon Systems and International Humanitarian Law: A Reply to the Critics,” SSRN Working Paper, Draft of December 4, 2012. According to Lawfare, Professor Schmitt, a leading laws of war scholar at the Naval War College, has posted this draft paper as a brief (24 page) response to the HRW report, “Losing Humanity.” The draft article analyzes weapons automation and autonomy through the lens of international humanitarian law, as well as the specific US practices to comply with international law obligations to review weapons systems, against HRW’s analysis. The article finds that the “Human Rights Watch position” in favor of a preemptive ban on development, production, and use of autonomous weapon systems is “unlikely to find traction.” It concludes that “autonomous weapon systems are not unlawful per se. Their autonomy has no direct bearing on the probability they would cause unnecessary suffering or superfluous injury, does not preclude them from being directed at combatants and military objectives, and need not result in their having effects that an attacker cannot control. Individual systems could be developed that would violate these norms, but autonomous weapon systems are not prohibited on this basis as a category.” UNQUOTE Dec 11, 2012, Lawfare recommended reading list regarding recent discussion of Autonomous Robots, also included.160 QUOTE Kenneth Anderson and Matthew Waxman, “Law and Ethics for Robot Soldiers,” Policy Review, December-January 2012-13 (final published version at Policy Review here, and working draft with footnotes at SSRN here). According to Lawfare, this brief (6,000 word) policy essay points out that the while autonomous weapons systems will enter the battlefield, they will do so only
157
2012 Nov 28 downloaded from https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2046375 found out about it here: http://www.lawfareblog.com/2012/11/human-rights-watch-report-on-killer-robots-andour-critique/ 2012 Nov 28 I started a Drone Robots doc, on Autonomous Killer Robot Drones, which references this document, so what I have on it, in time may diverge from references in Drone Reports. This paper has been cited and discussed many places, including: http://www.lawfareblog.com/2012/12/readings-matt-and-ken-on-law-and-ethics-for-robot-soldiers/ 158 http://www.hoover.org/publications/policy-review/article/135336 159 http://www.lawfareblog.com/2012/12/readings-autonomous-weapon-systems-and-their-regulation/ 160 http://www.lawfareblog.com/2012/12/readings-autonomous-weapon-systems-and-their-regulation/
35 Drones (semi) autonomous out of human loop / Studies / Drones / folder
36
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
incrementally, through small technological changes that are sometimes unrelated to weapons as such. The essay rejects either a wait-and-see, “don’t try to regulate what doesn’t exist yet” attitude toward regulation of gradually automating weapons systems, or a preemptive ban treaty of the kind urged by HRW. It argues instead that the US should work to create transparent international expectations about the legal requirements of autonomous weapons through the promulgation of its internal norms, standards, and best practices. These stategenerated norms are not generally matters of international law, but what the US government would regard as a plausible way forward on the basis of existing law of war norms in an area of novel technological and other issues. A wait-and-see attitude risks allowing technology to become locked into development paths that might lead to weapons that would be less desirable from a legal and ethical standpoint. The HRW-type ban treaty would unacceptably give up potentially large gains in humanitarian protection that might emerge from new technologies of automation over coming decades. UNQUOTE
USAF General Counsel Blog (2 Nov 28) !
USAF = United States Air Force I am following several blogs, which quote extensively from each other. I would rather have a chapter heading on each source, than duplicate the material. This approach encourages study of original sources, instead of just what someone else chooses to quote.
Drones and Accountability (2 Dec 12)
Dec 12, 2012 he links to what other people have been debating, about alleged Drone implications vs. US military accountability.161 Without interjecting his own comments, he suggests both the Peter Singer article 162 and Dan Trombly’s response163 are well worth a read.
Killer Robots Part 1 (2 Dec 12)
In Killer Robots Part 1, Nov 20 2012, he writes:164 QUOTE The Air Force’s Predators and Reapers are not drones. Indeed, they are not even “unmanned vehicles.” Instead they are remotely piloted vehicles with as much (if not more) human control as an F-16 or F-15. Human beings are still very much in the “kill chain.” The technology to allow completely autonomous weapon systems, however, does exist, and there are serious moral and legal issues that
161 162
http://afgeneralcounsel.dodlive.mil/2012/12/12/rpas-drones-and-accountability-for-military-action/ http://opencanada.org/features/the-think-tank/essays/the-robotics-revolution/ 163 http://www.cnas.org/blogs/abumuqawama/2012/12/silence-and-drones.html See Center for New America Security. 164 http://afgeneralcounsel.dodlive.mil/2012/11/20/killer-robots/
36 Drones (semi) autonomous out of human loop / Studies / Drones / folder
37
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
need to be considered before we think about adopting these autonomous weapon systems. Paul Robinson of University of Ottawa describes some of these issues: He goes on to quote some of this, then: Read it all here. You might also find the perspective of the former Chief Scientist of the Air Force to be of interest as well. He thinks there is little to be gained– and much to be lost–by removing humans from the kill chain. UNQUOTE I explore what Robinson has to say, in a separate chapter section = Robinson, Paul U of Ottawa.
Killer Robots Part 2 (2 Nov 29)
In Killer Robots Part 2, Nov 20 2012, he writes:165 QUOTE We earlier blogged about concerns about autonomous weapon systems. Bonnie Doherty of Human Rights Watch and Harvard Law School has an essay at Foreign Policy that argues for international agreements barring weapon systems that remove humans from the decision-making process: He quotes some of what they had to say, then: Read it all here. Then he comments on the overall topic. In large measure this is a technology issue better answered by experts in artificial intelligence rather than lawyers. He quotes the former chief scientist of the Air Force, concluding with: Read it all here. For a contrary view, see this Air Power article by Caitlin Lee. The Defense Science Board recently released a report that suggests more interest in autonomy than Dr. Dahm. UNQUOTE
Killer Robots Part 3 (2 Nov 29)
In Killer Robots Part 3, Nov 23 2012, he writes:166 QUOTE Perhaps as a result of the recent Human Rights Watch report on autonomous weapons167, and that organizations’ very effective op -ed strategy on the topic, there has been quite a bit written about whether we should have an international ban on autonomous weapon systems. Dan Trombly at CNAS has an interesting response, that emphasizes that even with so-called autonomous weapon systems, there will always be a human in the loop: He quotes some of it, then: Read it all here. Trombly’s analysis of the topic is quite comprehensive and is worth reading for anyone interested in the Human Rights Watch concerns . UNQUOTE
165 166
http://afgeneralcounsel.dodlive.mil/2012/11/20/more-on-killer-robots/ http://afgeneralcounsel.dodlive.mil/2012/11/23/killer-robots-part-3/ 167 See in Human Rights Watch Chapter and Drone Reports, that I have downloaded HRW report.
37 Drones (semi) autonomous out of human loop / Studies / Drones / folder
38
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
I wrote a comment, with a link to a video of a drone which did not have a human in the loop, but the posted comment dropped the url.
Killer Robots Part 4 (2 Nov 29)
In Killer Robots Part 4, Nov 26 2012, he writes:168 QUOTE In response to the Human Rights Watch Report,169 Kenneth Anderson and Mathew Waxman have published their analysis in a new Policy Review essay, Law and Ethics for Robot Soldiers. On LawFare, they give a brief summary of their argument: Then he quotes some of it, which I will link separately, in this general section. Then Read it all here. Second, the Department of Defense has issued a new policy directive on autonomous weapon systems.170 UNQUOTE
Zenko, Micah (2 Dec 12)
Micah Zenko wrote about Lethal Drones Dec 10, 2012, in Open Canada, a hub for International Affairs.171 He starts with excerpts and examples from the history of heavier than air flight, then how they got used in warfare, but says uses and abuses of drones are different than manned flight history. QUOTE First, unlike manned aircraft or raids, drones can fly directly over [allegedly] hostile territory without placing pilots or ground troops at risk of injury, capture, or death. Second, drones allow for sustained persistence over potential targets. For example, the U.S. arsenal of armed drones – primarily the Predator and Reaper – can remain aloft, fully loaded with munitions, for more than 14 hours, compared to four hours or less for F-16 fighter jets and A-10 ground-attack aircraft. Third, drones provide a near-instantaneous responsiveness which dramatically shrinks what U.S. military targeting experts call the “find-fixfinish” loop. A drone-fired missile travels faster than the speed of sound, striking a target within seconds – often before it is heard by people on the ground. Contrast this with the August 1998 cruise-missile salvo that
168 169
http://afgeneralcounsel.dodlive.mil/2012/11/26/killer-robots-part-4/ See in Human Rights Watch Chapter and Drone Reports, that I have downloaded HRW report. 170 See in US Military drone future plans and Drone Reports, that I have downloaded this report. 171 http://opencanada.org/features/the-think-tank/essays/lethal-drones/
38 Drones (semi) autonomous out of human loop / Studies / Drones / folder
39
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
targeted Osama bin Laden, an operation which required predicting where he would be in four to six hours, to analyze the intelligence, obtain presidential authorization, program the missiles, and fly them to the target. Intercontinental ballistic missiles loaded with conventional munitions can reach distant targets much faster than cruise missiles, but they carry the dire risk of misattribution as a nuclear first-strike. Drone-fired missiles can also be – and have been – diverted172 at the last moment if noncombatants enter the likely blast radius.173 At the same time, however, drones suffer two significant limitations. First, the precision and discrimination of drones are only as good as the supporting intelligence, which is derived from multiple sources. In states which lack a vast network of enabling intelligence, there will be significantly less situational awareness and precise targeting information for drones. Second, U.S. armed drones benefit from host-state support, which the US secures with payments in foreign aid and security assistance. Without such tacit or explicit support, drone strikes would be much less effective. For example, the CIA and U.S. military co-operate with their Pakistani counterparts to collect intelligence to identify and track suspected militants. The Pakistani army clears the airspace for U.S. drones, and when they inadvertently crash, Pakistani troops have repeatedly fought against the Taliban to recover the wreckage. Similarly, in Yemen, President Abed Rabbo Mansour Hadi has praised U.S. drones:174 “Every operation, before taking place, they take permission from the president.” As a result, the United States has taken advantage of permissive environments, unthreatened by antiaircraft guns or surface-to-air missiles, where non-battlefield targeted killings have occurred. However, according to former Air Force lieutenant-general David Deptula,175 armed drones that are deployed against a capable air-defense system would be vulnerable to attack: “Some of the [drones] which we have today, you put in a high-threat environment, and they’ll start falling from the sky like rain.” UNQUOTE
172 173
http://articles.latimes.com/2011/feb/22/world/la-fg-drone-strikes-20110222 And also have often not been diverted, thanks to poor intelligence. 174 http://www.washingtonpost.com/world/national-security/yemeni-president-acknowledges-approving-usdrone-strikes/2012/09/29/09bec2ae-0a56-11e2-afff-d6c7f20a83bf_story.html 175 http://www.foreignpolicy.com/articles/2012/02/27/10_things_you_didnt_know_about_drones?page=full
39 Drones (semi) autonomous out of human loop / Studies / Drones / folder
40
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
Similar Treaties (2 Dec 12)
We can pattern the proposed HRW treaties on pre-existing treaties. M Bolton,176 of Political Mine Fields,177 in International Committee for Robot Arms Control (IRAC)178 Analysis179 Oct 3, 2012:180 QUOTE I have been considering ways to flesh out the ICRAC proposal with further provisions building on the precedents of the landmine181 and cluster munition treaties,182 as well as the draft Arms Trade Treaty183 (currently still being negotiated) and the broader corpus of international humanitarian law governing conduct in war.184 UNQUOTE
Cluster Monition Treaty (2 Dec 12)
Here is a reference to it.185
IHL in armed conflicts treaty (2 Dec 12)
IHL = International Humanitarian Law, discussed extensively in my Drone Terms.
Land Mine Treaty (2 Dec 12)
Here is a reference to it.186
Civilized Human Principles (2 Dec 01)
Various people have suggested clauses, or chapters, which belong in a future international law convention.
Laws of Robotics (2 Dec 03)
The US Military and CIA laws of robotics are probably something like this: 1. If you see a useful target, please shoot at it. 2. If you get a software update, first work the decryption verification check that it has the USAF or CIA owner-code, to make sure it is not a hack job. (This is a concept IBM invented 50 years ago, but the other computer companies have yet to incorporate in their systems,)
176
http://icrac.net/author/gubrud/ and http://politicalminefields.com/aboutme/ See IRAC Matthew Bolton section. 177 http://politicalminefields.com/aboutme/ 178 http://icrac.net/ 179 http://icrac.net/category/analysis/ 180 http://icrac.net/2012/10/debating-proposals-on-a-possible-international-convention-on-robotic-weapons/ 181 http://www.icbl.org/index.php/icbl/Treaty 182 http://www.stopclustermunitions.org/the-solution/the-treaty/ 183 http://politicalminefields.com/2012/07/26/new-draft-of-the-arms-trade-treaty/ 184 http://www.icrc.org/eng/war-and-law/index.jsp Red Cross 185 http://www.stopclustermunitions.org/the-solution/the-treaty/ 186 http://www.icbl.org/index.php/icbl/Treaty
40 Drones (semi) autonomous out of human loop / Studies / Drones / folder
41
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
3. When you run low on gas or ammo, get a fuel up, or return to the nearest friendly base.
Disaster Recovery (2 Dec 01)
There are innocent communities which have been devastated by drone assaults. 187 They need foreign aid similar to that which goes to victims of an earthquake, severe weather disaster. Funding should be demanded, of the nations and people to be held accountable, for those violations of human decency and dignity. Matthew Bolton calls for,188 QUOTE 1. International cooperation in the reconstruction of communities damaged by robotic weapons; 2. Recognition of the rights of survivors of robotic weapon violence and international cooperation in the provision of victim assistance; 3. National and international mechanisms for recording the casualties of robotic weapon violence; 4. National reporting on treaty implementation; 5. International cooperation on treaty implementation, including technical assistance. UNQUOTE
Human Control Principle (2 Dec 01) Principle of Human Control = the notion that machines should not be deciding
that this or that human is to be killed.189 This is a proposed new principle, not previously stated within any body of law or philosophy. “We believe… it is unacceptable for machines to control, determine, or decide upon the application of force or violence in conflict or war. In all cases where such a decision must be made, at least one human being must be held personally responsible and legally accountable for the decision….” The principle is not just about killing, but also torture, interrogation, mind reading. It is a human right not to be killed on the decision of machines, nor to be subjected to violent force or pain on the decision of machines, nor to be threatened with violent force, pain or death as a form of coercion on the decision of machines, nor to be ruled in the conduct of life through the agency of overseeing machines which may decide the use of force as coercion.
187 188
In Drone Terms, for example, see: Datta Khel Nomada Bus Station. http://icrac.net/2012/10/debating-proposals-on-a-possible-international-convention-on-robotic-weapons/ 189 http://icrac.net/2012/11/the-principle-of-humanity-in-conflict/ http://icrac.net/2012/10/debating-proposals-on-a-possible-international-convention-on-robotic-weapons/
41 Drones (semi) autonomous out of human loop / Studies / Drones / folder
42
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
I would amend this. If we ever give artificial intelligence beings equal status under the law, if there is some logical way to punish them when they misbehave, then perhaps they would have the status of humans. Currently, in the USA, the US Supreme Court has declared Corporations to be persons, with rights like humans. This has all sorts of implications, starting with the difficulty of proving accountability, and the impact of big money on negative advertising in politics. Where the initial draft proposes that “machines should not do this or that to humans”, the word “machines” might be adjusted to include institutions rules and policies, without proper human oversight. There might also be a chapter defining what proper human oversight should look like. Also see my Mitigation Suggestions in Drone Terms, where: 1. I have evaluated many human systems which are obviously broken. 2. I try to figure out how come they are broken, and why premeditation efforts fail. 3. I have proposed new activities to provide alternative solutions. Similar analysis is needed in this discussion, to avoid repeating the same kinds of mistakes. I think we need more professions like medicine, engineering, and accounting, where people are certified as competent, and must sign their name to key decisions and documents. This is especially relevant with drone killing. There should be an audit trail. This person was ordered killed, on the basis of what alleged facts with an audit trail of where those facts came from. It may need to be classified, but that record should not get destroyed because of old age, as many corporate and government records are shredded when thought to no longer be needed. It should be declassified at some point in the distant future.
Human Responsibility Principle (2 Dec 01) Principle of Human Responsibility = We must hold ourselves responsible for
the decision to use violent force, and cannot delegate that responsibility to machines. At least one person (a human person), and preferably exactly one (e.g. a commanding officer), must be accountable for each decision to use violent force against a particular person or object at a particular time and place. Where data has been recorded pertaining to the circumstances of such decisions, it must be retained and made available for judicial review .190
Humanity Principle (2 Dec 01) Principle of Humanity = a concept found in early Hague and Geneva Conventions
first drafted a century ago. In the conduct of war we must not be needlessly cruel, inflict unnecessary suffering, or make use of weapons that are inherently inhumane.191
190 191
http://icrac.net/2012/11/the-principle-of-humanity-in-conflict/ http://icrac.net/2012/11/the-principle-of-humanity-in-conflict/
42 Drones (semi) autonomous out of human loop / Studies / Drones / folder
43
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
Matthew Bolton would ban,192 QUOTE arming robotic, particularly ’unmanned’, systems with nuclear, biological or chemical weapons, blinding lasers, landmines, cluster munitions or other weapons prohibited by international humanitarian norms. UNQUOTE But these weapons exist, and in war, leaders consider most options on the table. Is it better to send them blindly at an enemy, or use systems able to target them better at military targets, as opposed to a mass of combatants and non-combatants in a similar area?
Killers Trade Ban (2 Dec 01) See my Drone Nations notes. In there, I share info about treaties, which I found out
from GAO, which ban the sale of delivery systems for WMD etc. to certain nations. There are several US government agencies whose job it is to stop this from happening. But, as the GAO investigation makes clear, they are not only not doing their job, they are helping grease the wheels of trade to make sure the weapons go to inappropriate national customers. It is not enough to have a non-proliferation treaty. It has to be honored by nations which have the power to enforce it.193 Matthew Bolton calls for,194 QUOTE 1. Prohibition of trade or transfer of robotic weapons, parts, components, related technologies and munitions, to countries and armed groups that commit serious violations of international human rights and humanitarian law; 2. Transparent reporting on the trade and transfer of robotic weapons. UNQUOTE
Killing with Non-Lethal Weapons (2 Dec 01)
I have written citations in my earlier notes, such as Drone Terms, about non-lethal weapons like Tasers, which have killed hundreds of people, and how tear gas canisters aimed, at people’s heads, do serious damage, and other instances where police supposedly armed with non-lethal weapons, have permanently harmed innocent people. If a weapon has been banned for use by military on the battlefield, shouldn’t it also be banned for use by police against the civilians of a nation, let alone via police drones? Matthew Bolton would require regulation of,195 QUOTE use of so-called ‘less-lethal’, ‘non-lethal’ or ‘pain’ weapons by robotic systems.
192 193
http://icrac.net/2012/10/debating-proposals-on-a-possible-international-convention-on-robotic-weapons/ Over the weekend 2012 Dec 1-2, I e-mailed Matthew Bolton pointing this out, supplying links to the GAO reports, where I uploaded my Drone Nations notes, and mentioning a few other related topics. 194 http://icrac.net/2012/10/debating-proposals-on-a-possible-international-convention-on-robotic-weapons/ 195 http://icrac.net/2012/10/debating-proposals-on-a-possible-international-convention-on-robotic-weapons/
43 Drones (semi) autonomous out of human loop / Studies / Drones / folder
44
Drones (semi) autonomous out of human loop
3/15/2013 10:47:11 PM
UNQUOTE
44 Drones (semi) autonomous out of human loop / Studies / Drones / folder
