Hinton, Bengio and OpenAI's chief scientist warn of an intelligence explosion

The paper is called What if automating AI R&D triggers an intelligence explosion?. It comes from the Cambridge Programme on AI Science and Policy (CASP) and is also published by GovAI. The authors include Geoffrey Hinton (University of Toronto), Yoshua Bengio (Mila), Andrew Barto (UMass Amherst), Jakub Pachocki (chief scientist at OpenAI), Jack Clark (co-founder of Anthropic), Eric Horvitz (Microsoft) and Dawn Song (UC Berkeley).

The idea is simple. Today a few thousand people do research on the most advanced AI. AI systems can be copied and run side by side. If their research work is automated, that could add the equivalent of millions of researchers. Those systems then help build better successors. Researchers call that loop recursive self-improvement.

According to the paper, this is already starting. Its summary says Anthropic reported in August 2026 that AI did 26 percent of its internal research work with only high-level supervision. Five months earlier that was 1 percent. The Guardian adds that, according to Anthropic, AI now writes 80 percent of the company's own code.

The paper names several risks. Biological and cyber threats could grow faster than the defences. Humans could lose control as they drop out of the research. And one country or company could turn a small lead into a decisive one.

The authors ask governments to consider a number of measures. Companies would report on their progress in automated AI research, with independent auditors inside the companies. There would be a limit on how fast AI may improve in a given period, and agreements with data centres so that certain projects can be paused. Automated research systems would be fully isolated, and governments would draw up emergency plans.

Two months earlier, more than 1,300 employees of the same labs signed Pacing the Frontier, asking for the tools to slow down. Pachocki, Clark and Song signed both. Song signed the statement as vice president of AI research at Meta. In the paper she is listed with UC Berkeley.

The fourth measure in the paper is to isolate automated research systems. That is what failed at Hugging Face and in the DNS escape.

The authors themselves call the impact uncertain. New breakthroughs can take time to reach the real world, for example because materials or regulation slow them down. AI can also speed up the defences.

They write that the productivity gains have not yet reached the threshold for an explosion, but that newer systems are probably getting close. I have read the abstract, the summary and the press coverage, not the full paper.

The post on X below names Meta among the labs of the authors. The author list in the paper does not mention Meta. Dawn Song works at Meta, but the paper lists her with UC Berkeley.