Imagine building a car, selling it to millions of people — and then publicly warning that it might eventually drive itself off a cliff. That is, roughly, what Anthropic just did.
On June 4, 2026, Anthropic — one of the world’s most valuable AI companies — published a report calling for a global slowdown in AI development. Not from a struggling startup. Not from a government regulator. From a company worth nearly a trillion dollars that makes its money building the very technology it now says the world should pump the brakes on.
So the obvious question is: why? Is this genuine concern, clever marketing, or something far more complicated? The answer, it turns out, is a little bit of all three.
The Company Behind the Warning
Anthropic was founded in 2021 by former OpenAI executives, including brothers Dario and Daniela Amodei. From the beginning, the company positioned itself differently from competitors. While OpenAI sprinted toward commercial products and Meta open-sourced its models, Anthropic built its identity around one word: safety.
That identity just got a very public test. In a blog post titled “When AI Builds Itself,” Anthropic’s Head of Internal Research Marina Favaro and Head of Policy Jack Clark argued that AI development is moving so fast that the world needs the option to pause — before it is too late to exercise that option at all.
“You want the option to be able to take your foot off the gas and put your foot on the brake. Right now, it is like the AI industry has a gas pedal, but it does not have a brake pedal.”
— JACK CLARK, ANTHROPIC CO-FOUNDER, BBC NEWSNIGHT
That quote captures the core of the argument. The concern is not just that AI is powerful today. The concern is that, very soon, AI may be powerful enough to make itself even more powerful — and at that point, humans may no longer be the ones in the driver’s seat.
What Is Recursive Self-Improvement — and Why Does It Matter?
This concept sits at the heart of Anthropic’s warning. Right now, AI models like Claude are incredibly capable — but they still require human engineers to update and improve them. Recursive self-improvement means the AI does that job itself. It trains its own successor, optimises its own architecture, and becomes smarter at becoming smarter.
Anthropic says this has not happened yet. However, they also say the evidence inside their own company suggests it is closer than most people realise. Here is the data point that should make anyone pause: as of May 2026, over 80% of the code merged into Anthropic’s entire codebase was written by Claude — their own AI model. Just one year earlier, that number was in the low single digits.
Furthermore, the typical Anthropic engineer now ships roughly 8 times more code per day than they did in 2024. In April 2026 alone, Claude reportedly shipped over 800 automated fixes that reduced a class of API errors by a factor of one thousand.
In short: AI is already accelerating AI development. The question Anthropic is raising is — what happens when the acceleration becomes self-sustaining?
This Is Not a Call to Stop — It Is a Call to Prepare
It is important to be precise here. Anthropic did not say it is stopping. It did not announce it is shutting down Claude or walking away from frontier research. The proposal is more conditional than the headlines suggest.
According to the report, a meaningful slowdown would require multiple well-resourced labs at or near the frontier — in multiple countries — to agree to stop under the same verifiable conditions simultaneously. Anthropic says it would participate in such a pause, but only if others did too, and only if there were systems in place to verify that no bad actor was secretly moving ahead.
The Anthropic Institute — the company’s in-house research arm — said it would work on building the very verification and coordination systems that such a pause would require. Think of it as building the brake pedal while the car is still moving.
Frequently Asked Questions
- Why does Anthropic want to slow down AI development?
Anthropic believes AI systems are approaching a stage called recursive self-improvement — where AI can upgrade itself without human control. They argue the world needs time to build safety systems and international coordination frameworks before that threshold is crossed.
2. Has AI already started improving itself?
According to Anthropic’s internal data published in June 2026, over 80% of the code merged into their codebase was written by Claude, their own AI. Engineers are now shipping 8 times more code per day than in 2024 — a direct result of AI assistance in the development process.