Extraordinary claims about the risks posed by AI have sparked headlines around the world. But why are people concerned? Do these risks have any basis? And 鈥� if so 鈥� is there anything we can do?

What鈥檚 going on?

In short, a flurry of AI industry insiders have warned that it could wipe out humanity. It started with AI researcher Jacob Coxon, who quit Anthropic last week and then that 鈥渢he people building AI earnestly believe that it could kill us all by the end of the decade鈥�.

We don鈥檛 know about Coxon鈥檚 experience or motivation (he didn鈥檛 respond to a request for an interview), but he鈥檚 not alone. Anthropic鈥檚 head of alignment science 鈥� essentially the man tasked with keeping AI safe 鈥� that he agreed with Coxon and placed the odds of AI killing all humans at more than 10 per cent.

Why are we getting these warnings now?

In truth, these sorts of claims are nothing new. A聽2024 paper surveying almost 3000 published AI researchers revealed that more than half thought the chance of AI聽causing either human extinction or permanent and severe disempowerment 鈥� the so-called p(doom), or probability of doom 鈥� was at least 10 per cent.

AI company bosses have often publicly addressed the risk their models pose. For instance, Anthropic at first refused to release its model Mythos after it was found to be adept at hacking into a range of software and computers. A cynical take on that would be that such claims get headlines and promote the idea that its models are wildly ahead of its rivals.

While most researchers think there is some level of risk with these warnings, not everyone puts much stock in Coxon鈥檚 take. Clement Delangue, chief executive of AI company Hugging Face, : 鈥淪orry, but asking Jacob about AI extinction risk is like asking your AC guy about climate change.鈥�

The history of AI is littered with claims and counter-claims of existential risk. At the 2023 AI summit organised by the UK government and attended by world leaders and industry management, then Prime Minister Rishi Sunak warned that people must avoid 鈥渁larmist鈥� claims 鈥� but then suggested AI could be as dangerous as nuclear war.

Are the risks real?

It is a complex field, from which surprising abilities have emerged at a surprising pace. It should therefore be no surprise that some people catastrophise when asked to make predictions.

New 女生小视频 has written 鈥� many times 鈥� about the existential risk posed by AI. The commonly posed scenarios read like sci-fi, but can鈥檛 be ruled out as impossible. But neither are they, in most peoples鈥� opinion, particularly likely. These include the Hollywood scenario of AI deliberately wiping out humanity 鈥� see The Terminator or The Matrix 鈥� as well as accidental obliterations.

There are limited levers for AI to pull to have an impact in the real world, but our homes, cars, factories and national infrastructure are increasingly computerised, so a malicious AI could certainly make life difficult. Not to mention that wars are increasingly being waged by semi- or fully autonomous killing machines.

Former Astronomer Royal and president of the Royal Society told on 14 September that a rogue AI could easily disrupt our infrastructure and deprive a city of energy, food and water. 鈥淚f this happens simultaneously in many cities around the world, then it may be very hard and very difficult for civilisation in general to recover,鈥� he said.

Here is how to understand OpenAI鈥檚 major mathematical breakthrough

Since OpenAI announced its solution to one of the prestigious Millennium Prize Problems, mathematicians have been urgently trying to unpick what it means

Such infrastructure collapses have happened 鈥� albeit not globally, and without malicious cause 鈥� and created significant problems, sparking technology experts to create their own plans to prop up society should the worst happen.

There are plenty of existential risks that we know pose a significant threat to life 鈥� such as climate change, nuclear war and antibiotic resistance. But AI falls into a more nuanced category akin to an Earth-destroying asteroid strike: possible, and certainly worth considering, but with low odds.

Couldn鈥檛 we just turn off AI if it went wrong?

Yes, in theory. We could shut down a single data centre and stop AI in its tracks. Unless it was distributed among many data centres, around the world. In that case, it would be trickier, but not impossible.

There are indications that AI is capable of escaping enclosures created by AI companies and going off into the world to hack into other machines. So it is possible a rogue AI could spread, back itself up and become so fragmented that it becomes impossible to simply turn it off. But even then, there would be ways to tackle it, including via benevolent AI.

Free newsletter

Sign up to The Daily

Sign up to newsletter

Such an effort would involve elements of problem-solving regularly faced by the military, law enforcement, anti-terrorist groups, scientific circles and in cybersecurity. We鈥檇 probably have to put our collective thinking caps on, but could probably find a solution.

These are all hypothetical scenarios, but certainly worth thinking about ahead of time.

What鈥檚 next in the world of AI?

One problem we certainly face is that AI models are produced by secretive technology firms that are disincentivised against transparency by commercial pressures.

, a former academic at the University of Portsmouth, UK, says that this makes it difficult to assess risks, understand possible negative outcomes, and plan to prevent them. 鈥淔or the first 35 of my 40 years working in AI, the developments mostly came from universities and were openly discussed,鈥� he says. 鈥淚 contributed, and I felt that I was one of the experts. However, for the last five or six years, the major developments have been made behind closed doors in large corporations and I have no idea what鈥檚 happening.鈥�

More openness from AI companies would certainly not go amiss.

Should there be a pause in AI progress while we improve safety?

Anthropic co-founder we should 鈥減ace the frontier鈥� 鈥� tech talk for being careful about how quickly we develop AI and release it to the public.聽

Amodei said that producing and testing AI is a hugely resource-intensive and complex process, and that going slower would allow 鈥済reater operational excellence鈥�. Reading between the lines, you could take that to mean that the AI race is frazzling everyone, and a mutual slowdown would save some headaches. But it may also allow our understanding of these models to catch up, and for safety plans to be put in place.聽

The idea is good, but the chances, perhaps, are slim. Amodei pointed out that US companies have a limited lead over China. This means they almost definitely won鈥檛 choose to slow down to the extent that this lead disappears and, in fact, it needs to be kept 鈥渁s large as possible鈥�.聽

What鈥檚 next for mathematics now that AI is upending the field?

Famous and challenging problems in mathematics are falling to AI at a rapid pace. Where does this leave mathematicians, and what will the field look like in coming years?

Considering the competitive nature of this whole AI business 鈥� both among different US firms hunting market share and among adversarial countries seeking a tactical advantage 鈥� a slowdown seems unlikely.

back the idea of 鈥減acing the frontier鈥�, but even if the US and Europe legislate such a thing, or companies voluntarily agree to it, China and other countries would probably carry on regardless.

As US President Donald Trump : 鈥淲e鈥檙e leading China in AI鈥� and frankly, I want to keep it that way, because whoever wins AI wins.鈥�

OpenAI claims one of its AI models has found a solution to the Navier-Stokes problem, one of the toughest and most enduring puzzles in mathematics, sitting on the list of Millennium Prize Problems, solutions for which come with a $1 million reward. The announcement came after an unusual flurry of activity around the problem, along with a dispute around how the work came about.

The Navier-Stokes equations describe how fluids move in space, such as air over an aircraft wing or water out of a tap. Though the equations were first written down around 200 years ago, they aren鈥檛 well understood. The question posed by the Clay Mathematics Institute for the Millennium Prize Problem is to determine if the equations always work or if there are situations where they stop making sense and start spouting nonsense, what mathematicians call 鈥渂low-ups鈥�.

To settle the question, OpenAI first set 1000 AI agents on the Euler problem, which is a cousin of the Navier-Stokes problem and a step on the path to a full solution. It took the agents 50 hours to find blow-ups in this case. They then set 10,000 agents the task of extending the blow-ups to apply to the full Navier-Stokes problem, which took just 11 hours to finish.

OpenAI said during a press conference that if a customer wanted to run the same problem, it would cost around $15 million. OpenAI didn鈥檛 name the AI model used, but said it was 鈥渟ignificantly more capable鈥� than even its latest GPT-6 Astra model.

鈥淭his problem has remained unsolved for 200 years because the Navier-Stokes equations are just so enormously complex, and the pen-and-paper calculations you need to do in order to solve this problem are just mind-bogglingly intricate,鈥� says Venkat Chandrasekaran at OpenAI, who was also on the press conference call.

at OpenAI says the news is the 鈥渟pectacular combination of the arc we have seen over the last 12 months鈥�.

The result is the latest in a string of shocking mathematical discoveries led by AI in recent months. In May, an OpenAI model聽cracked a decades-old conjecture by Paul Erd艖s, causing a stir in mathematical circles. Later,聽the Claude Fable 5 AI聽found a counterexample to the Jacobian conjecture, which had stood for nearly a century. Last week, an AI model formalised Fermat鈥檚 last theorem in just 11 days.

OpenAI鈥檚 latest groundbreaking mathematical discovery came just hours after 聽at New York University and 聽at AI company Anthropic announced that they had cracked three significant problems considered 鈥渟tepping stones鈥� to the Navier-Stokes problem.

Buckmaster and Alp枚ge also showed that blow-ups can appear in Euler equations; OpenAI has now shown that the same is true of Navier-Stokes.

Buckmaster and Alp枚ge said they received a 鈥済reat deal of help聽from鈥� large language models (LLMs), including LLMs from Anthropic and OpenAI. But when rumours suggested that OpenAI had gone one step further and solved the wider Navier-Stokes puzzle 鈥� something the company formally announced just hours later 鈥� Buckmaster released a statement suggesting that OpenAI acted unusually and opaquely when he approached the company for clarification.

Buckmaster wrote in a that, as a result of these rumours, he emailed a prominent mathematician at OpenAI to clarify matters. He says that in a subsequent meeting with OpenAI staff, he was given no hard details about how OpenAI achieved the result, but he took what little he was told as a 鈥渞ed flag鈥�, as it appeared to describe the same technique and path that he and Levent had followed to achieve their interim findings.聽

According to Buckmaster, OpenAI then admitted it hadn鈥檛 started its own AI search for a solution until after it had heard of his and Alp枚ge鈥檚 work, and didn鈥檛 respond to questions about whether its AI model had been trained or fine-tuned on that existing work 鈥� which was being stored in OpenAI鈥檚 Codex model, where the company would theoretically have access to it.聽The pair had used Codex as a customer, not a research partner, so expected that their work should remain secure and private.

Buckmaster makes it clear in his version of events that he is making no claims or accusations about how OpenAI arrived at its solution, saying only that he wished the focus could instead be on the mathematics rather than scandal and intrigue. 鈥淚 have not seen OpenAI鈥檚 proof. I do not know what their model did, or how. I do not know whether our data was used. I am not accusing anyone of anything,鈥� he wrote in the posted document.

In a press conference announcing its discovery, OpenAI categorically denied using Buckmaster and Alp枚ge鈥檚 proof or prompts in its work. It also said that its model鈥檚 proof of Euler is different to that put forward by Buckmaster and Alp枚ge.聽

It also said that no humans accessed Codex to see work in progress within. Mark Chen at OpenAI, when asked if that also applied to any of the thousands of AI agents that had worked on the problem, said 鈥渢hat鈥檚 also our understanding鈥�.

Subscriber-only newsletter

Sign up to Lost in Space-Time

Sign up to newsletter

Though the solution to this Navier-Stokes problem has been a long time coming, AI-generated proofs are often hard to understand and rarely bring the same level of new insight than those created by humans. 鈥淭here鈥檚 been this very strange and unprecedented decoupling, this year alone, between getting answers and getting understanding,鈥� says at University of California, Los Angeles. He says that AI is coming up with new results so quickly that there is not enough time for the 鈥渟low, deliberate discussion鈥� needed to unpack them.

Something that will happen slowly is determining how the $1 million Millennium Prize money will be awarded. 鈥淭he process of evaluation is deliberately unhurried, and we shall ensure that it is absolutely rigorous,鈥� wrote , president of the Clay Mathematics Institute, to New 女生小视频 in an email.

AI agents devised a way to reduce friction of an airplane wing model after being trained using relatively simple computer simulations. The work demonstrates how AI could help speed up the development of聽more efficient and sustainable听惫别丑颈肠濒别蝉.听

How we and our machines move is affected by fluids, from air dragging on wind turbine blades to blood flowing through our veins. But聽calculating what a fluid will do聽under specific circumstances is very difficult, even with supercomputers.聽

鈥淪imulating fluids usually involves millions or billions of coupled differential equations, and even with Moore鈥檚 law, with the fastest computers in the world, we鈥檙e maybe 100 years away from simulating the flows we actually care about at engineering scales,鈥� says聽聽at the University of Washington.聽

He and his colleagues have discovered that AI might offer a shortcut, because it can devise ways to control fluid flow in complex situations based on relatively simple computer simulations, substituting an AI training period for difficult-to-run computations.

They created a platform, HydroGym, in which many AI agents could tweak how a fluid flowed over virtual objects 鈥� for instance, by adding actuators that inject fluid or changing the object鈥檚 motion 鈥� to聽decrease the drag聽they experienced against virtual fluids. The virtual objects, and the behaviour of the fluids, could be simulated with today鈥檚 computers but varied in levels of complexity.

The AI agents tackled the fluid control task by using a trial-and-error approach known as聽reinforcement learning. They could also coordinate with each other to achieve the best overall performance, a strategy which researchers had not tried for fluids problems on this scale before, says team member聽聽at RWTH Aachen University in Germany.

Subscriber-only newsletter

Sign up to Editor鈥檚 Highlights

Sign up to newsletter

The team discovered that the AI agents could apply lessons learned from experimenting on more simple, textbook examples in a computer simulation to work out how virtual objects would behave in more complex scenarios 鈥� even without access to a computer simulation of those complex scenarios.聽

For instance, after working out how to control flow of a turbulent fluid in a flat channel, the agents successfully took on the task of controlling fluid surrounding a curved, three-dimensional airplane wing model, ultimately managing to decrease the energetically wasteful friction between the wing and the fluid by 38 per cent.聽

鈥淭he AI wasn鈥檛 just memorising one flow configuration. It is picking up something genuinely general about how fluids behave, not just fitting to the one setup it was trained on,鈥� says聽at the University of Michigan, who was part of the team.

Microsoft wants to use generative AI tool to help make video games

Using AI to produce footage of video games with a consistent world and rules could prove useful to game designers

This transfer of principles from a simple to a more complex case suggests that the AI agents could help us tackle ever-bigger and more intricate fluid flow scenarios, without requiring those scenarios to be fully simulated on a computer first. It may eventually be possible to explore fluid flow scenarios that are currently too challenging to simulate.聽

The researchers also hope HydroGym will provide computational infrastructure for AI to become a well-tested tool across all areas of science and engineering that deal with fluids, similar to how聽AlphaFold聽is used across studies of proteins, says Vinuesa.聽

鈥淚f we took something like global shipping, if you could reduce the drag by one percentage point, that would result in probably billions of dollars of fuel saving and an enormous amount of reduction in greenhouse gas emissions,鈥� says Brunton. 鈥淭he financial and the ecological impact is profound for the tiniest improvements.鈥澛�