As the race to build ever more powerful artificial intelligence systems accelerates, one of the industry's leading players is urging the world to consider a possibility that until recently belonged largely to science fiction: machines improving themselves without human intervention.
Anthropic, the AI company behind Claude, said on Thursday that the ability to slow the pace of frontier AI development could prove valuable as the technology approaches capabilities that may fundamentally reshape society.
The warning came in a blog post authored by Marina Favaro, head of Anthropic's internal research institute, and company co-founder Jack Clark.
The post disclosed internal research showing that the firm's most advanced models are progressing rapidly and could eventually move toward what researchers call "recursive self-improvement" — a scenario in which AI systems become capable of enhancing their own capabilities.
The company stressed that such a threshold has not yet been reached and may never be achieved.
However, it argued that the possibility is becoming serious enough to warrant preparation.
AI that can build itself would be a major development in the history of technology—one that could bring enormous good for the world in science, healthcare, and beyond," the post said.
Anthropic acknowledges that any effort to pause or slow AI development would only work if major players participated.
The company therefore, proposed exploring international agreements and verification mechanisms designed to ensure compliance.
However, it also admitted that monitoring AI development could be considerably harder than enforcing traditional arms-control agreements.
Training runs are far easier to conceal than missile silos," the blog post noted.
The company warned that any actor continuing development while competitors paused could gain a significant advantage, making coordination exceptionally difficult.
For now, Anthropic plans to organize discussions with policymakers, researchers, and industry leaders to examine how recursive self-improvement should be studied and whether mechanisms for coordinated slowdowns could ever be practical.