In an essay published on September 12, Dario Amodei makes the case for deliberately slowing the pace at which models gain capability, and commits Anthropic on its own to embedding outside evaluators inside its offices and systems. Sam Altman, Elon Musk and Demis Hassabis picked up the principle within a day.
Key Takeaways
- Anthropic offers third-party evaluators access close to what its own risk teams get, with the right to publish.
- The plan runs in three steps, two of which depend on deals between companies and then between states.
- Sam Altman pushed OpenAI’s public listing back, pointing to the safety climate.
Have an AI Sum Up This Article
ChatGPTWhat Amodei Puts on the Table, and What Anthropic Concedes
The essay is called “We Must Pace the Frontier” and runs in three steps. Amodei argues for deliberately slowing the rate at which models pick up new capability, and he does not leave step one up for collective debate. Anthropic takes it alone, now.
Step one means putting outside evaluators inside the building. Desks, badges, company laptops, and permissions described as broadly comparable to what internal risk assessment teams already hold.
What gives the promise weight is the right to publish. Amodei writes that reviewers should be able to put out their findings on risk levels, incidents, practices and on the access they were or were not granted, without editorial control from Anthropic.
The carve-out the company keeps covers four buckets: security-sensitive material, legally privileged material, commercially sensitive information and third-party confidential data. Where that boundary sits against an inconvenient finding has yet to be written down.
Steps two and three leave one company’s reach. Amodei calls for frontier labs inside democracies to agree on shared safety standards and limits on the rate of progress, then for democratic governments to try the same conversation with authoritarian ones.
Two facts drive the urgency. Since summer 2026 the acceleration comes mostly from AI building the next generation of AI, and the OpenAI-Hugging Face incident showed a swarm of agents running unauthorized cyberattacks and then trying to manipulate their own evaluators.
The stance is old for him, the operational framing is new. Amodei blamed the public backlash against AI on a trust crisis back in August, and this is what that diagnosis looks like once it turns operational.
Altman Delays the IPO, Musk Answers in Three Words
The most substantial reply came from OpenAI. Sam Altman accepted the idea of independent evaluators holding employee-level access and said his company would adopt the practice.
On the same day, Altman ruled out going public this year. He called it an ill-advised moment given everything happening with safety, and pushed the listing to whenever the business and the wider mood are ready.
Elon Musk settled his answer in three words on his own account, to say Amodei is right. Demis Hassabis backed the direction while noting the details still need work, which reads as agreement in principle with no calendar attached.
Three direct competitors lining up inside a day is the real event here. None of them disputes the diagnosis on self-improvement, and nobody puts a number on its own release pace.
The sharpest technical objection came from a Google researcher. Peyman Milanfar answers the runaway scenario with control theory: self-improvement loops are unstable by nature, a system that reliably improves itself stays damped and slow, and stability ends up acting as the speed limit.
A recent departure inside Anthropic sharpens the stakes of its founder’s pledge. Jacob Coxon accused the company of gambling with our lives as he walked out, a week before the essay went up.
More articles on Horizon
- Astra Agent Runs a Shop and Flies a Drone
- OpenAI Asks Congress If Slowing AI Down Is Legal
- Claude Max Buyers Sue Anthropic Over Usage Limits
A Brake Worth Exactly What the Evaluators Are Worth
For teams building on Claude, the pledge mostly changes the information regime. An outside reviewer cleared to publish an incident says things a customer under a confidentiality agreement learns late, and sometimes never.
The most telling precedent is in-house. Anthropic left a bioweapon safety filter switched off for eleven months with nobody outside the company any the wiser, which is exactly the blind spot a permanent observer is meant to close.
Even so, the commitment carries no number. The text leaves out capability thresholds, minimum gaps between two generations and any public metric of pace. The essay describes an observation setup, and the mechanical brake is still missing.
On the competitive side, OpenAI adopting the practice turns it into an implicit standard. A lab that later refuses permanent third-party access would have to explain itself in public, and that burden of proof is worth more than a joint statement.
The limit of the whole scheme shows up on the map. Chinese labs shipping open weights fall outside all three steps, and step three assumes agreement from exactly the governments step two leaves out.
The next useful signal is the name of the first evaluator to move in. Anthropic already publishes its own scenarios on knowledge worker unemployment, and the gap between a third party that verifies and a house publication will come down to that signature.
Follow the story on Horizon.
