Photo: U.S. War Department
IN THIS REPORT
- An Overview of Catastrophic AI Risks
- Anthropic Says Trump Administration Ignores Safety
- Dangers of Fully Autonomous Weapons
- Challenges in Countering the Threat
BY KEVIN POLICARPO
A.I. is growing more powerful and more dangerous and poses growing national security risks that are largely ignored.
The Center for AI Safety is an American nonprofit organization based in San Francisco that promotes the safe development and deployment of artificial intelligence.
In its report An Overview of Catastrophic AI Risks, it cites the following:
- Malicious use: “People could intentionally harness powerful AIs to cause widespread harm. AI could be used to engineer new pandemics or for propaganda, censorship, and surveillance, or released to autonomously pursue harmful goals. To reduce these risks, we suggest improving biosecurity, restricting access to dangerous AI models, and holding AI developers liable for harms.”
- AI race: “Competition could push nations and corporations to rush AI development, relinquishing control to these systems. Conflicts could spiral out of control with autonomous weapons and AI-enabled cyberwarfare. Corporations will face incentives to automate human labor, potentially leading to mass unemployment and dependence on AI systems. As AI systems proliferate, evolutionary dynamics suggest they will become harder to control. We recommend safety regulations, international coordination, and public control of general-purpose AIs.”
- Organizational risks: “There are risks that organizations developing advanced AI cause catastrophic accidents, particularly if they prioritize profits over safety. AIs could be accidentally leaked to the public or stolen by malicious actors, and organizations could fail to properly invest in safety research. We suggest fostering a safety-oriented organizational culture and implementing rigorous audits, multi-layered risk defenses, and state-of-the-art information security.”
- Rogue AIs: “We risk losing control over AIs as they become more capable. AIs could optimize flawed objectives, drift from their original goals, become power-seeking, resist shutdown, and engage in deception. We suggest that AIs should not be deployed in high-risk settings, such as by autonomously pursuing open-ended goals or overseeing critical infrastructure, unless proven safe. We also recommend advancing AI safety research in areas such as adversarial robustness, model honesty, transparency, and removing undesired capabilities.”[1]
One such method of misuse is the development and use of biological weapons: “Biological agents, including viruses and bacteria, have caused some of the most devastating catastrophes in history. Despite our advancements in medicine, engineered pandemics could be designed to be even more lethal or easily transmissible than natural pandemics.”[2]
Weaponized biological agents have caused some of the most devastating disasters in history, with some states keeping a stockpile of such pathogens and non-state actors posing threats with biological weapons. A.I. could allow for the creation of new biological weapons that could bypass traditional safety screenings.
Anthropic Says Trump Administration Ignores Safety
The Trump Administration has been pushing for the use of A.I. for national security, regardless of the risks involved without the necessary safeguards that brought it into conflict with Anthropic, which is known for its stance on developing A.I. in a safe and responsible way. The Trump administration’s stance caused Anthropic to protest AI use by the Department of War.
Anthropic CEO, Dario Amodei explained his company’s differences with the Department of War:
“Anthropic understands that the Department of War, not private companies, makes military decisions. We have never raised objections to particular military operations nor attempted to limit use of our technology in an ad hoc manner. However, in a narrow set of cases, we believe AI can undermine, rather than defend, democratic values.”
He explained that there were two cases that were never included in their contracts with the Department of War:
- Mass domestic surveillance. “We support the use of AI for lawful foreign intelligence and counterintelligence missions. But using these systems for mass domestic surveillance is incompatible with democratic values. AI-driven mass surveillance presents serious, novel risks to our fundamental liberties. To the extent that such surveillance is currently legal, this is only because the law has not yet caught up with the rapidly growing capabilities of AI…”[3]
- Fully autonomous weapons. “Partially autonomous weapons, like those used today in Ukraine, are vital to the defense of democracy. Even fully autonomous weapons (those that take humans out of the loop entirely and automate selecting and engaging targets) may prove critical for our national defense. But today, frontier AI systems are simply not reliable enough to power fully autonomous weapons. We will not knowingly provide a product that puts America’s warfighters and civilians at risk. We have offered to work directly with the Department of War on R&D to improve the reliability of these systems, but they have not accepted this offer. In addition, without proper oversight, fully autonomous weapons cannot be relied upon to exercise the critical judgment that our highly trained, professional troops exhibit every day…”[4]
Amodei further explained the stance the Department of War has in terms of working with A.I. companies and why the DOW was willing to remove their contract with Anthropic:
“The Department of War has stated they will only contract with AI companies who accede to ‘any lawful use and remove safeguards in the cases mentioned above. They have threatened to remove us from their systems if we maintain these safeguards; they have also threatened to designate us a “supply chain risk”—a label reserved for US adversaries, never before applied to an American company—and to invoke the Defense Production Act to force the safeguards’ removal.”[5]
Dangers of Fully Autonomous Weapons
In recent years, the explosive growth of drone warfare changed the calculus of war in light of conflicts such as the Russo-Ukrainian War. One particular focus in drone warfare is the creation of fully autonomous weapons, controlled only by A.I. to improve their performance. However, these weapons come with their own risks which could make them dangerous for the user and civilians caught up in war.
According to Crawford, a provider of claims management, the threat from autonomous drones is getting worse and counter measures are not yet in place. In a report entitled “AI drones – the unknown risk that’s already here,” the report says: “The proliferation and integration of drone technology is unprecedented.”
The areas where these threats are likely to materialize fall into four main categories:
- Physical attacks (weaponisation)
Drones can be weaponised to carry explosives, deliver dangerous substances, or launch direct attacks on infrastructure and populated areas… - Surveillance and espionage:
Autonomous drones can be used for surveillance purposes to gather intelligence on individuals, companies and government facilities… - Cyberattacks:
Drones can serve as entry points for cyberattacks by exploiting vulnerabilities in their control systems. Hackers could seize control of drones to manipulate them for surveillance, delivery of malicious payloads, or disruptions to operations… As commercial drones are increasingly adopted, the associated risks of network compromise increases. - Disruption of operations/events:
Drones can be used to interfere with operations in critical infrastructure sectors, such as energy plants and transportation hubs. They also serve as a propaganda weapon, easily able to disrupt events. A well-timed drone disruption can halt supply chains, delay air traffic, or cause severe damage to operations, leading to significant financial losses. An example of this was seen in 2018, when hundreds of flights were cancelled at Gatwick Airport owing to the mere presence of an unauthorised drone in a controlled airspace. - Implications for insurance claims:
The use of drones by threat actors introduces new and complex scenarios for insurance claims. Insurers will need to adapt to these emerging risks by considering various factors that have previously not been part of their risk models.
Challenges in Countering the Threat
The rapid pace of technological development “means that countermeasures are lagging far behind, leaving security forces and businesses scrambling to develop effective defences against a constantly evolving threat. Combined with the ease of access to drone technology for non-state actors, these factors make drones a particularly potent and challenging threat to counter. Drone countering technology falls into two categories: hard and soft techniques. A hard countermeasure involves a physical action or direct intervention against the drone to disable or destroy it. Such measures are not currently applicable to a commercial environment…”[6]
The Bulletin of the Atomic Scientists, an academic journal, summarized the threat:
“The US and Chinese militaries are starting to test swarming drones – distributed collaborative systems made up of many small, cheap, unmanned aircraft. This new subset of independently operating or “autonomous” weapons is giving rise to new strategic, ethical, and legal questions. Swarming drones could offer real advantages, including reducing the loss of both human life and expensive equipment in battle. But they also come with potential dangers. There is already great international concern about deploying weapons without “meaningful human control.” Proliferation is another danger, and a problem that could be particularly acute in the case of swarming drones. The risks posed by swarming drones should be considered sooner rather than later, before their destructive potential reaches maturity.”[7]
FOOTNOTES
[2] https://safe.ai/ai-risk
[3] https://www.anthropic.com/news/statement-department-of-war
[4] https://www.anthropic.com/news/statement-department-of-war
[5] Ibid.
[6] https://www.crawco.com/blog/ai-drones-the-unknown-risk-thats-already-here
[7] https://www.tandfonline.com/doi/pdf/10.1080/00963402.2017.1290879