A transformative moment in the life sciences
The advent of generative artificial intelligence (AI) marks a transformative moment in the life sciences. It enables new approaches to data analysis, hypothesis generation, experimental design, and management of scientific knowledge. However, for all its potential benefits, AI also introduces a range of ethical, social and technical challenges.
Key security concerns of adding advanced pattern recognition to genomic data are that it could significantly facilitate (a) the enhancement of pathogens to make them more dangerous; (b) the modification of low-risk pathogens to become high impact; (c) the engineering of entirely new pathogens; or (d) the recreation of extinct, high-impact pathogens such as the variola virus that causes smallpox. Compounding the challenge is that these possibilities are arising at a time when new delivery mechanisms for transporting pathogens into human bodies are being developed.
There are also concerns that AI will (a) enable easier access to knowledge, materials and tools with dual-use (i.e. both peaceful and weapon-related) applications, including dangerous pathogens and toxic molecules; (b) facilitate and speed up dual-use biomedical and life sciences research; and (c) increase the repurposing potential of biological data for nefarious uses. In addition, the intersection of AI and biology (hereafter ‘AI–bio intersection’) has intensified security concerns around ultra-targeted biological warfare.
The Biological and Toxin Weapons Convention and AI
The 1972 Biological and Toxin Weapons Convention (BWC) is the principal legal instrument
banning biological warfare and the deliberate use of bacteria, viruses and toxins to inflict harm. States parties to the convention agree that the BWC unequivocally covers all microbial or other biological agents or toxins, naturally or artificially created or altered, as well as their components, whatever their origin or method of production.
While there is potential for AI to be used to counter the acquisition, development and use of biological weapons, the policy focus and political responses have been largely on the risks of AI convergence with biology. Risk conceptions and political responses to biology and AI can be characterized to date as having involved two main stages: risk awareness-raising and hyperbole. A third notable stage has recently started to emerge: the reality check.
Risk conceptions and political responses
Risk awareness-raising
The risk awareness-raising stage, when some of the security concerns arising from the AI–bio
intersection were first formulated and introduced into the policy world, began with a slow
trickle of interest. That slow trickle of interest became a much more steady stream
after a proof-of-concept experiment in 2021 demonstrated in no uncertain terms that AI could relatively easily be used to design toxic compounds such as the exceptionally toxic chemical warfare nerve agent VX. The thought experiment was a powerful example of the dual-use or repurposing potential of converging technologies, and its publication in an article in the journal Nature Machine Intelligence drew significant media and policy attention.
Hyperbole
A second thought experiment conducted at the Massachusetts Institute of Technology (MIT)
was published as a news story in the journal Science in June 2023. This experiment suggested that undergraduate students had been able to use chatbots like ChatGPT to gain the know-how to devise a biological weapon and ushered in a second wave of media and political interest in the AI–bio intersection.
OpenAI, which developed ChatGPT, stress-tested the chatbot for security concerns and released a ‘system card’ three months prior to the Science news publication. During the stress test, the company found that ‘a key risk driver is GPT-4’s ability to generate publicly accessible but difficult-to-find information, shortening the time users spend on research and compiling this information in a way that is understandable to a non-expert user’.
A slew of reports followed the OpenAI system card and the Science news publication of the MIT experiment. The main focus of these reports was on threats from LLMs and biodesign tools.
Awareness of potential security implications of the AI–bio intersection reached the very
highest political levels. In advance of the AI Safety Summit hosted by the United Kingdom in
November 2023, which brought together global political and tech leaders, British Prime
Minister Rishi Sunak warned that AI could make it easier to build biological weapons. The concern was also captured in the Bletchley Declaration released during the summit, which
emphasized potential catastrophic harm from AI and biotechnology. Around the same time, in the United States, the administration of President Joe Biden issued a landmark Executive Order on the Safe, Secure, and Trustworthy Development and Use of Artificial Intelligence that included a plan to probe how emerging AI systems might aid malicious actors in plots to develop bioweapons.
Few of the reports and political statements and initiatives highlighted the many uncertainties in how AI and machine learning affect the security dimension of the life sciences. This is important because the security impacts of AI are significantly contested.
Reality check
There is a substantial knowledge gap in the expert community on how the AI–bio intersection
will affect biosecurity. While AI can be used to predict and design new toxic compounds or
proteins that have harmful effects and can also be used to predict and design enhancements of
pathogens that make them even more harmful or to identify and manipulate key genetic com-
ponents affecting pathogenesis, currently there is no demonstrated data showing this is actually the case.
LLMs like ChatGPT may make it easier for non-experts to access dual-use knowledge and thereby lower barriers to intentional misuse. Yet, at the present time, the anticipated risk is hypothetical. More recent studies on the biothreat from AI are starting to recognize this.14
For efficient AI training, high-quality data is essential. Data sets must be sufficiently large and representative to reduce bias and optimize AI performance. In biology and the life sciences, data availability can be restricted for all kinds of reasons, including licensing policies, ethical and security considerations, proprietary rights and so on. Therefore, the availability of high-quality biological data sets is not a given. Furthermore, there can be issues with the completeness of available data sets, including in areas such as recorded parameters, context-related information, uncertainty quantification, reliability and evaluation of negative outcomes. Therefore, the completeness of biological data sets is also not a given.
In addition to the variable quality and completeness of the data and data sets used to train AI, scientists still need to evaluate computational results and validate them experimentally. For example, the VX-like molecules from the thought experiment described above only ever existed on screen; they would still need to be verified experimentally.
The extent to which the threat of biological weapons will change as a result of AI is, in fact, not at all clear. A much better understanding of current and potential future uses, and the limitations, based on empirically informed data of AI in biology and the life sciences, is sorely needed. While it is right to pay close attention to the security implications of the AI–bio intersection, a more sophisticated, more informed, more evidence-based dialogue must be encouraged to develop more realistic assessments of new biothreats.
Acknowledgements
This is an abridged version of Non-Proliferation and Disarmament Brief No.35 (May 2024) produced by the European Non-proliferation and disarmament consortium and funded by the European Union.