The maths bust-up fuelling fears that AI is undermining science.

A bank of white freezers hums away with a constant, low thrum at a former watch factory in central London. Inside their gleaming, sterile depths, dozens of small boxes hold meticulously barcoded glass tubes, collectively safeguarding an astonishing one hundred billion genes. Dr. Oliver Vince, co-founder of Basecamp Research, points out that just two of these unassuming boxes may contain ten times more information than all the text ever published on the internet. This staggering volume of genetic data represents an unprecedented resource, a vast library of life’s blueprints waiting to be deciphered.

His firm, Basecamp Research, has embarked on an ambitious quest: to collect as much DNA as humanly possible, from diverse species and environments, in order to create an artificial intelligence capable of truly "speaking the language of biology." This endeavour mirrors the foundational work undertaken by US and Chinese tech giants who have systematically hoovered up virtually all available written material online to construct Large Language Models (LLMs), the intricate neural networks underpinning today’s most advanced AI systems like ChatGPT. Just as LLMs learned to understand and generate human language by processing billions of words, Basecamp’s AI aims to comprehend the complex syntax and semantics of genetic code, protein structures, and biological pathways. The potential applications are revolutionary, ranging from accelerating drug discovery and understanding disease mechanisms to pioneering new fields in synthetic biology.

Vince, alongside countless colleagues in both commercial enterprises and academic institutions, firmly believes that AI is poised to supercharge scientific research, ushering in an era of unprecedented discovery. Yet, this burgeoning optimism is increasingly shadowed by controversy, as the very tools promising to unlock new frontiers also raise profound questions about integrity, ethics, and the fundamental nature of scientific progress.

Last month, a significant tremor ran through the scientific community when OpenAI, the creator of the ubiquitous ChatGPT, made a bold claim: it had solved one of mathematics’ great unsolved problems, specifically relating to the infamous Navier-Stokes equations. These equations are fundamental to understanding fluid dynamics, describing everything from ocean currents and weather patterns to the flow of blood in our arteries and the aerodynamics of aircraft. Their complexity has baffled mathematicians for centuries, and a successful, rigorous solution is famously one of the seven Millennium Prize Problems, carrying a $1 million award from the Clay Mathematics Institute. OpenAI announced that its sophisticated AI agents had cracked this formidable challenge in an astonishing 88 hours. The Clay Mathematics Institute, while cautious, acknowledged that the problem had "apparently been settled," adding that its customary rigorous review process would naturally take considerable time.

However, just hours before OpenAI’s public announcement, a dissonant note was struck. Tristan Buckmaster, a distinguished mathematician at New York University, went public with grave concerns that cast a shadow over the celebrated achievement. Buckmaster had been independently working towards solving the same Navier-Stokes problem, forming a collaborative team with Levent Alpöge, a mathematician employed by the rival AI company Anthropic. The two researchers had been diligently developing their solution, keeping their drafts and code within OpenAI’s own coding tool, Codex. Buckmaster disclosed that his co-researcher had been "tipped off" – though he did not specify the precise mechanism – that news of their significant progress had somehow "reached" OpenAI. This revelation immediately sparked an unsettling question: could his research, meticulously stored and developed within OpenAI’s proprietary Codex platform, have inadvertently fed into OpenAI’s own internal work on the problem? While Buckmaster was careful to stress he was "not accusing anyone of anything" directly, the implications were stark. The incident ignited a firestorm of debate regarding data privacy, intellectual property, and the ethical boundaries of AI development, particularly when the very tools researchers use might also be learning from their proprietary work. It highlights a critical vulnerability in the evolving landscape of AI-driven research: the opaque nature of how these powerful models are trained and the potential for their creators to gain an unfair advantage from user-generated data.

Beyond the specific "maths bust-up," a broader threat looms over the bedrock of scientific integrity: the institution of scientific journals. These journals, traditionally the guardians of research quality and veracity through rigorous peer review, are increasingly seen as struggling to cope with a veritable tsunami of AI-generated distortions. "Generative AI has basically disrupted the whole scientific process," asserts Dalmeet Singh Chawla, a seasoned science journalist who has dedicated a decade to reporting on research misconduct. He tells the BBC that the integrity staff employed by publishers – the dedicated individuals responsible for screening submissions for accuracy, reliability, and originality – will need to "drastically increase [in number and effort] if they’re anywhere near the fight." His sobering assessment: "Right now, it seems like they’re not."

The types of distortions are manifold: AI can generate highly plausible but entirely fabricated data, concoct non-existent references, plagiarize existing works with sophisticated rephrasing, or even produce entire papers that, on a superficial reading, appear coherent and scientifically sound, yet lack any real substance or empirical basis. Such output can easily bypass human reviewers, especially those overwhelmed by volume or lacking specialized expertise in detecting AI patterns. The long-term consequences of compromised scientific literature are dire, potentially eroding public trust in science, propagating flawed research, and ultimately slowing down genuine progress by forcing researchers to sift through an ever-growing pile of noise. The scientific community grapples with how to adapt, discussing everything from developing advanced AI detection tools to fundamentally re-evaluating the peer review process itself.

On the surface, AI appears to be the quintessential tool to propel scientific research forward. Scientists routinely grapple with incomprehensibly vast amounts of information. Modern instruments like cutting-edge telescopes and particle accelerators, for instance, collect billions of data points every second – volumes far too immense for the human brain to process or even fully comprehend. Here, AI excels, possessing an unparalleled capacity to sift through this colossal data, identify subtle patterns, correlate disparate pieces of information, and discern the crucial scientific highlights from the surrounding humdrum of noise.

The undeniable poster child of scientific AI, illustrating its transformative potential, is the UK-based firm Google DeepMind. In 2020, DeepMind achieved a monumental breakthrough by solving one of biology’s most enduring and complex problems: precisely how proteins, the microscopic workhorses of all living things, fold into their intricate, three-dimensional shapes. It may sound like an esoteric detail, but the phenomenon of protein folding is anything but dull. Proteins are the tiny, molecular machines that orchestrate virtually every process of life, from building cells and facilitating chemical reactions to fighting infections and forming the tissues and organs that comprise our bodies. Crucially, the precise shapes into which they fold dictate their specific functions. A misfolded protein can lead to severe diseases like Alzheimer’s or Parkinson’s.

Before the advent of AI, scientists had to conduct arduous, time-consuming, and often serendipitous experiments in the lab to determine a single protein’s structure – a process that could take years for just one molecule. Then, Google DeepMind developed an AI model called AlphaFold. This revolutionary system, trained on existing protein structures, could accurately predict the precise folded shapes of proteins purely from their amino acid sequences. Its impact was immediate and profound: AlphaFold accurately predicted the structures of 200 million protein folds in just one year. The equivalent experimental work, conducted using traditional laboratory methods, would have demanded hundreds of millions of years of painstaking effort. In an extraordinary act of scientific altruism, DeepMind made the AlphaFold database and its underlying code freely available to researchers worldwide, catalyzing an explosion of new discoveries in biology and medicine. The head of DeepMind, Demis Hassabis, and the project’s lead scientist, John Jumper, were both subsequently awarded the Nobel Prize for Chemistry in 2024 for their groundbreaking contributions to protein structure prediction, a testament to the profound impact of their work.

It was, as Dr. Pushmeet Kohli, who leads DeepMind’s science work, told me, a moment of universal celebration and "big hugs all round." Kohli characterized AlphaFold as nothing less than a "watershed moment," signaling a paradigm shift in how biological research would be conducted. It demonstrated AI’s capacity not merely to assist, but to fundamentally accelerate and redefine the very limits of scientific discovery, offering solutions to problems once deemed intractable.

Yet, as the "maths bust-up" and the "tsunami of AI-generated distortions" starkly illustrate, this immense promise is inextricably linked to profound risks. The very power that allows AI to accelerate discovery at an unprecedented pace also harbours the potential to corrupt the scientific process itself. The core tension lies in navigating this dual nature: harnessing AI’s unparalleled capabilities while simultaneously establishing robust ethical frameworks, ensuring transparency in its development and deployment, and fostering accountability among its creators and users. Without these safeguards, the fears that AI is undermining the very foundations of science – eroding trust, compromising integrity, and clouding the path to genuine knowledge – will only intensify, threatening to overshadow its transformative potential. The future of scientific progress hinges on our ability to manage this delicate balance.