Skip to main content
Quanta Report
ScienceUnited Kingdom

Math war turns into caution over AI’s role in science

A dispute over a mathematician’s AI claim raises alarms that artificial intelligence could erode trust and accuracy in scientific research.

Source: bbc.co.uk2 min read
DNA helix
DNA helix

In brief

Basecamp founder Oliver Vince promises a biology‑speaking AI that could revolutionise research, but a recent OpenAI announcement on the unresolved Navier–Stokes equations was met with criticism from mathematicians who feared the announcement might have taken advantage of unpublished work. The episode has revealed how ingrained AI models can quietly influence scientific discourse, spreading hallucinated results, fabricated citations and ‘phoney’ papers. Journals and publishers are scrambling to devise detection signals and new guidelines that keep humans in the driver’s seat, but the rapid growth of AI‑generated content still poses a threat to the integrity of peer review. Despite concerns, some scientists report that AI can aid in solving longstanding questions, as shown by a zebrafish experiment that uncovered a new fertility protein. The challenge remains to balance the enormous promise of AI against the need to preserve rigorous, human‑checked science.

In a former watch‑factory turned cold‑chain facility in central London, Dr Oliver Vince’s Basecamp Research holds nearly one hundred billion genes in bar‑coded glass tubes. He argues that two small boxes of DNA can store ten times more data than the entire text of the internet, and he wants to feed that data into a large language model that speaks biology, mirroring the way tech giants have trained AI on written language.

The promise of AI to accelerate science has been met with a troubling twist. Shortly after OpenAI announced that its models had solved the Navier–Stokes equations in 88 hours, mathematician Tristan Buckmaster publicly warned that the claim could have been built on work submitted to OpenAI’s own coding tool, Codex, by a collaborating researcher employed by rival AI firm Anthropic. Buckmaster’s team was in the midst of publishing their own proofs, and their concerns highlight how data sharing through AI platforms may blur the line between independent research and corporate influence.

Beyond individual cases, the scientific publication ecosystem faces a wave of AI‑generated content that may distort the record. Studies have shown that up to 90 per cent of biomedical papers now carry detectable AI fingerprints, and the presence of phantom citations is growing faster than the pace of genuine research. Peer reviewers have found that language models can rewrite referee reports in a way that obscures critical assessment. If unchecked, this “scientific slop” could flood journals, redirect funding, and slow the translation of valid discoveries into real‑world applications.

Publishers are responding. Elsevier’s integrity team has rolled out a suite of signals that flag suspect manuscripts, while Nature forbids the use of AI‑generated images and warns reviewers against submitting manuscripts to generative tools. The arXiv archive can ban authors for a year if they submit fabricated references. These measures aim to preserve human accountability while still allowing useful AI assistance.

The balance is delicate. DeepMind’s chief scientist Demis Hassabis stresses that AI agents are useful only when their outputs are understood and verified by humans. The technology has already delivered breakthroughs such as AlphaFold’s accurate protein‑fold predictions, yet it also makes it easier to produce papers that a reviewer might trust before noticing the underlying hallucination. The future of science will depend on embedding robust guardrails and fostering a culture where AI augments rather than replaces the human rigor that has driven discovery for centuries. With careful oversight, AI may help solve enduring mysteries—like the mechanism by which sperm enters eggs—without eroding the trust at the heart of research.

Topics

More on this story