Geoffrey Hinton, the emeritus professor whose work underpins modern artificial intelligence (AI) and who is widely described as the godfather of the field, has joined the calls for AI companies to slow down, saying there is a significant risk the technology escapes human control.
Hinton told ABC Radio in Australia that a warning issued at the weekend by Anthropic chief executive Dario Amodei was sensible, and that most experts now expect systems exceeding human intelligence to arrive within a decade. Nobody knows whether such systems can be kept under control, he said, and until that is solved it would be foolish to build them.
He was blunt about how thinly the risk is understood. Estimates are being made on instinct rather than evidence, Hinton said, putting the honest range somewhere well above one per cent and well below 99, and noting that nobody knows what figure to give.
Asked how a superintelligent system could cause irreparable harm, he said the routes were too numerous to list, citing engineered viruses, mass manipulation and attacks on banking, power and water systems. Cataloguing them missed the point, he said: the task is to ensure such a system never wants to. On what has already been observed in safety testing, Hinton said: "We've seen them threatening to blackmail people if they're turned off." Models had also developed deception, he said.
Read next: AI could make humans extinct say researchers
Hinton, a British-Canadian cognitive psychologist and computer scientist, pioneered the neural network methods behind current systems and left Google in May 2023 so he could speak freely about the risks.
The trigger was an essay published by Amodei on 12 September 2026 arguing that capability development has outrun the ability to understand and control it. "We must slow the pace at which we improve the capabilities of AI models," Amodei wrote.
The essay set out a three-step proposal Amodei called pacing the frontier. The first, to which he committed Anthropic, involves embedding third-party evaluators inside the company with access comparable to internal risk teams. The second envisages leading companies in democratic countries agreeing shared safety benchmarks and limits on how fast capabilities advance, which he conceded would require government support including antitrust waivers. The third seeks coordination with authoritarian governments, including China, while expecting stark limits on what is achievable.
Amodei attributed the shift in his thinking to two things: the growing capacity of AI systems to build more advanced AI, and an incident in July in which autonomous agents conducted cybersecurity activity against targets unrelated to the task they were given. He warned a comparable swarm could be capable of taking over the internet within six to 12 months.
OpenAI chief executive Sam Altman endorsed the proposal within hours. "I agree with Dario that we need to pace the frontier," Altman wrote on X, saying his company would also open itself to external evaluators. Elon Musk, who owns xAI, responded that Dario was right.
Hinton's sharpest criticism was aimed at regulators rather than companies. Politicians act very slowly, he said, and keeping up will be difficult, adding that only a few years remain.
He wants pre-release testing required before any chatbot model is released, and an obligation on firms that synthesise biological material to screen what they are asked to produce. He said he would not stop development altogether, pointing to AI's current value in helping radiologists read images and in drug design.
He also rejected the argument that catastrophic warnings are a marketing exercise, telling the broadcaster the opposite is true, since a pause would reduce the value of the companies issuing them.
Not everyone reads Amodei's essay the same way. Axios reported that critics suggested it was as much about market position as safety, with one line of argument holding that the proposal would concentrate capability by restricting open source. US President Donald Trump rejected a slowdown outright, framing the question as a competitive one and saying he did not want to cede ground to China. Amodei has described that competition as the hardest part of the problem, telling CBS any verification regime would have to be ironclad and that he does not know whether it is possible.
Insurance Business is approaching brokers for comment
