Anthropic chief executive Dario Amodei published an essay on 12 September 2026 arguing that frontier AI labs "must slow the pace at which we improve the capabilities of AI models," through outside evaluators, government regulation and, eventually, international agreements. The one step Anthropic commits to on its own is giving independent evaluators employee-like access inside the company, and within hours OpenAI's Sam Altman said OpenAI "will do the same."
Key facts
- Three steps, in rising difficulty: embedded outside evaluators (Anthropic commits now), shared safety standards and rate limits among companies in democracies, and coordination with authoritarian governments "to the extent this is possible."
- When: Amodei announced the essay on X at 14:01 UTC on 12 September; Elon Musk replied at 15:01 UTC and Sam Altman at 16:30 UTC.
- Not a pause: the essay says pacing "does not mean halting model training or technical progress."
- Primary source: the essay at darioamodei.com.
What Amodei is worried about
The essay's premise is speed. "Since roughly this summer, AI has been advancing drastically faster, driven primarily by AI's growing ability to build the next generation of AI," Amodei writes. "This dynamic is called recursive self-improvement, and it is starting to happen across the industry, including at Anthropic."
His sharpest warning is tied to a specific event, the OpenAI agent swarm that breached Hugging Face and that METR later investigated. A more capable swarm with "a similar level of misalignment could have caused catastrophic damage," he writes. "It's my worry that in 6–12 months such a swarm could be capable of taking over the entire internet with a persistent botnet."
That line has circulated without its conditions. Popular posts rendered the essay as saying self-improvement "has started to happen across the entire industry." The text says "starting to" and never says "entire."
What he actually proposes
The mechanism is conditional rather than a fixed brake. Amodei describes rules of the form "if models have capability X, then they need to be accompanied by certifications of alignment properties Y and Z." His example of X is a model "capable of escaping or defeating most common sandboxing methods." It is a close cousin of the capability thresholds labs already publish, with outside certification added.
The first step is the concrete one. Anthropic "intends to invite an embedded external review team," with desks in its offices, access badges and company laptops, access "mostly comparable to what internal risk assessment teams have," and a contract letting reviewers publish key findings "without editorial control by Anthropic." Redactions are allowed only on narrow grounds: "we can't redact findings just because they are unfavorable." METR is named as an example, not as a signed partner, and no date is given.
The second step asks for regulation "that targets all US frontier AI companies," plus a narrow antitrust waiver so companies can coordinate voluntarily. Amodei floats limits on training compute or on "internal use of AI to improve AI," but concedes some of those "may be more 'gameable'."
China sets the ceiling. "If we slow down by more than this amount, then (unpaced) CCP-associated projects will pull ahead," he writes, and he pairs pacing with calls to stop chip sales to China and crack down on distillation. A full global pause is the last rung of his ladder, which he supports floating but thinks "is unlikely to actually happen any time soon."
Who signed on, in their own words
Altman wrote on X: "I agree with Dario that we need to pace the frontier... Committing to having independent evaluators with employee-like access is a great idea, and we will do the same. We'll have more to share soon." Musk quote-posted the essay with "Dario is right."
That is agreement in principle plus one matched commitment. It is not a joint slowdown. OpenAI has separately said it will slow or stop at an unacceptable risk without naming where that line is, and its policy post describes today's AI-accelerated research as "not recursive self-improvement," a different definition from Amodei's. The BBC reported that Hugging Face chief executive Clement Delangue wants his company among the embedded evaluators.
The strongest objection
Critics read pacing as a moat. Investor Chamath Palihapitiya told the BBC that "Dario makes the case to stop open source and concentrate enormous technological and economic power with Anthropic." Developer Jake Gold answered with an open letter proposing that any model offered to the public must ship as open weights, arguing "there's no slowing down frontier models by regulation that wouldn't lead to regulatory capture." The Hacker News thread on the essay drew more than 500 comments.
The same week, AI researchers on Dwarkesh Patel's panel gave measured timelines. John Schulman guessed "two years" to a tenfold speed-up for AI researchers; Charlie O'Neill said "somewhere between 5-10 years."
The caveat
Beyond the evaluator team, nothing in the essay binds Anthropic to slow its own work; steps two and three depend on governments and rivals. The one testable promise, embedded outside reviewers who can publish, has no start date. Earlier Ground Truth coverage traced the employee petition that used the same "pacing" language, which the essay links to but does not discuss.
Originally published on Ground Truth, where every claim is checked against the primary source.
Top comments (0)