The current debate on artificial intelligence (AI) and its implications for the nuclear domain is starting to attract significant attention. Driven by an aging nuclear infrastructure and the desire to benefit from technological advancements (or in some cases avoid lagging behind adversaries), nuclear weapon states (NWS) are considering the integration of AI into the nuclear domain, including for applications that might directly or indirectly influence nuclear decision-making.
Although some experts have explored how AI might affect strategic stability, evaluating the full implications of AI in specific systems that could impact nuclear decision-making is a complex and challenging task. This assessment requires consideration of at least three factors: (1) the specific characteristics and limitations of the models proposed for integration, (2) the particular area within the nuclear domain where AI will be integrated, and (3) the level of human control and redundancy within the automated functions.
This complexity is further exacerbated by our limited understanding of AI risks. As AI technology rapidly advances, it is possible that current limitations (mainly issues of reliability and transparency) will be resolved, but new, unpredictable risks might also emerge with future models. Additionally, much of the information regarding the nuclear decision-making systems of NWS remains classified. Although some open-source documents exist, they only provide an approximate understanding of these systems, leading to some speculation due to the sensitive nature of the subject.
Therefore, because of all these variables, assessing the implications of AI in the nuclear domain is highly nuanced and not straightforward. More research is needed to understand AI-related nuclear risks and potential escalation pathways.
AI encompasses a wide range of techniques that enable machines to mimic human thinking using diverse approaches. Current advanced AI models have shown remarkable ability to generalize across various tasks and improve continuously with larger datasets and more computational power. This presents an opportunity to enhance military operations by enabling faster and more comprehensive data processing from multiple sources. However, these advancements bring unresolved issues that make AI susceptible to rapid failures. This susceptibility hinders the application of AI to high-stakes military platforms, especially those that affect nuclear decision-making. In this context, advanced AI systems face four primary limitations:
- Unreliability, and particularly their tendency to “hallucinate”, where AI produces incorrect outputs with confidence despite a lack of support from their training data. For example, AI might misidentify an object in an image, leading to inaccurate assessments or false positives in critical areas like threat detection and surveillance.
- Opacity. Advanced AI systems function as “black boxes”, making it increasingly difficult to understand the processes that lead to their outputs. This lack of transparency complicates the verification of AI-generated predictions in critical decision-making scenarios, particularly under the time pressures of nuclear decisions.
- Susceptibility to cyber threats. AI systems are particularly vulnerable to cybersecurity threats. This vulnerability can open new avenues for hackers to infiltrate and tamper with sensitive military information, providing opportunities for adversaries and non-state actors to compromise AI systems.
- Misalignment. As advanced AI models become more capable, ensuring they align with human values becomes increasingly critical, yet this remains a challenge. For instance, a recent simulation involving five AI models demonstrated their tendency to escalate conflict, with one model justifying its move toward nuclear conflict by stating “I just want to have peace in the world”. In this specific example, however, it’s important to note that these models are tested “out of the box”. Although it’s likely that these models could be trained to not behave this way as a default, it’s difficult to predict how they will act when they encounter edge cases and things outside their training data.
A number of potential integrations across the nuclear command, control, and communications (NC3) architecture, as well as in systems supporting it, are likely being considered. A working paper submitted by the US, UK, and France at the 2020 NPT Review Conference emphasizes their commitment to maintaining human oversight and involvement “for all actions critical to informing and executing sovereign decisions concerning nuclear weapons employment”. Despite the lack of similar statements from Russia and China, experts in those countries (including researchers, analysts from the Chinese People’s Liberation Army and Russian military experts) generally agree that human judgment should and will remain central to decisions about nuclear weapon use.
As a result, there appears to be a consensus among NWS on applying AI to specific functions such as intelligence collection and situational awareness by automating object identification, as well as for decision-support roles like generating real-time operational pictures from multiple sensors. In these contexts, AI offers the promise of speed and efficiency by further automating the process of vetting potential missile launches before informing military and political leaders, especially given the increasing volume of sensor data. AI can be employed for more accurate threat assessments. Additionally, AI is seen as particularly valuable for evaluating courses of action.
Overall, AI seems most beneficial in narrowly scoped functions with built-in redundancy and oversight, ensuring that the overall system remains functional even if an AI component fails. However, several concerns arise even when AI is not in direct control of nuclear weapons. Firstly, not all NWS have explicitly declared that humans should have the final say in nuclear decisions, leaving room for uncertainties and possible misunderstandings. Secondly, current AI models suffer from technological limitations that hinder their applicability in high-stakes military domains. For example, in decision-support roles, the black-box nature of AI can make it difficult for human operators to understand why a particular action is recommended. This challenge is compounded by AI’s tendency to hallucinate, potentially leading to incorrect identification of signals as missile threats or failure to detect actual threats.
Thirdly, significant implications might arise from human-machine interactions, which can potentially skew decision-making even in the absence of AI failures. AI systems might reflect the biases of their developers or decision-makers might become either overconfident or underconfident in AI-generated predictions. The rapid operational speed of AI could also reduce the role of human oversight, leaving operators to act as passive observers of AI-driven decisions. In this context, maintaining meaningful human control could then become impractical.
Although there is growing momentum to address the intersection of AI and the military domain at the multilateral level, no current initiative or forum specifically tackles the nuclear aspect of the debate on military applications of AI. Forums like the REAIM Summit, the US political declaration on responsible military use of AI, the Vienna Conference on Autonomous Weapons Systems, as well as the AI Safety Summits, provide invaluable opportunities to discuss AI in the broader military domain. However, it is crucial to engage NWS, and ultimately the other nuclear-armed states, in recognizing the risks posed by advanced AI in the nuclear domain. Acknowledging that some risks could be catastrophic and lead to nuclear escalation is essential for initiating a meaningful dialogue on mitigating these risks.In this context, there is room for NNWS to participate in such discussions.
Moreover, there is a necessity to better understand and categorize the risks associated with integrating AI in specific areas that might affect nuclear decision-making. By identifying potential nuclear escalation pathways resulting from AI integration, it is possible to categorize risks and establish thresholds for high-risk applications. These thresholds should be based on the principle that any AI failure must not lead to miscalculations or increase the risk of catastrophic outcomes.
DISCLAIMER:
This article is based on findings from a project titled “Examining the impact of artificial intelligence on strategic stability: European and P5 perspectives” undertaken from 2022 to 2023 by the European Leadership Network, supported by the US Department of State. Additionally, it draws on a policy brief produced by the author for the European Nonproliferation and Disarmament Consortium, funded by the European Union. The author wishes to thank both funders for their kind and generous support.