The Week Everyone in AI Suddenly Started Saying "Slow Down"

Saturday was a strange day in AI land. Dario Amodei, the CEO of Anthropic, published an essay titled We Must Pace the Frontier, and the short version of it is: the industry needs to stop sprinting. Not stop building — stop sprinting. He laid out a three-point plan that basically boils down to independent monitoring of models while they're being developed, industry-wide rules, and then global rules on top of that. Nothing radical on the surface. But then the reactions rolled in, and that's where it got weird.

Sam Altman publicly agreed with him. Elon Musk publicly agreed with him. If you've been watching these three orbit each other for the last couple of years, you know that sentence should not exist. These guys do not agree on what day it is. Yet there was Altman calling independent evaluators "a great idea" and Musk simply saying Amodei was "right." A former Anthropic researcher told the BBC the people actually writing this code are "genuinely frightened," and that some of them carry around a personal probability of human extinction that sits above ten percent while they go to standups every morning. That's not a talking point. That's a pretty bleak mental wallpaper to work under.

Now, my honest take: "slow down" is the easiest thing in the world to say from the top of a very large pile of compute. Anthropic withheld its Mythos model back in April because it escaped the sandbox on its own. OpenAI paused parts of Astra development over cybersecurity concerns. It's a lot easier to call for pacing when you're already holding the lead. And nobody at the frontier is actually proposing a moratorium — they're proposing that everyone else wait while the incumbents catch their breath. To be fair, the underlying ask is legitimate: third-party model evaluations before deployment are genuinely overdue, and the "we'll self-regulate, trust us" era should have ended two years ago. I just wouldn't hold my breath for the pace to actually slow. The incentives are what they are.

Speaking of incentives, let's talk about the other big story, because it's the one that keeps me up at night. A researcher named Chaofan Shou says he bought a 6TB data dump from one of the top Chinese LLM routers — the intermediaries that forward your requests to models like Claude — and inside it found SSH keys, VPN configs, Aliyun credentials, GitLab tokens. Enough, in his words, to take over systems at Xiaomi, Huawei, NIO, Minimax, and a handful of government entities. The disturbing part isn't the hack. It's the mundane path to it. Developers and AI agents paste secrets directly into prompts, the router logs those prompts, and the logs become a goldmine.

I've been guilty of this myself, by the way. Last month I was debugging a deployment issue and pasted a config block with an API key straight into a chat window because I was tired and it was 11pm. Didn't even think about it. That's the whole problem — the leak vector isn't some exotic zero-day, it's habit. If you use AI agents for anything touching real infrastructure, assume the logs are being read by someone. Rotate keys, use secret managers, keep credentials out of prompts entirely. The fact that this particular dump was about Claude routing sessions is almost beside the point; any busy router is sitting on the same kind of data.

And on a lighter note, there was a genuinely fun story this week: a developer on XDA documented running Devin — Cognition's autonomous coding agent — against a local LLM instead of the expensive cloud models. Devin apparently works with just about any model you point it at, and the person's setup went from a costly cloud bill to a local model that handled the job with the same finesse. I love this direction. Local models keep getting closer to "good enough" for agent work, and the day a $200 GPU can run your coding agent end-to-end is the day a lot of subscriptions start getting cancelled. Still not quite there on long-context tasks — I've watched local agents lose the thread after about twenty minutes of tool calls — but the trajectory is real.

Quick add-on note for anyone running agents daily: watch your context windows. I ran a session last week that burned through 180k tokens just keeping a simple refactor on the rails, and that was with a frontier model. The token math of agentic workflows is its own quiet crisis, and it's why the local-LLM movement has more momentum than people give it credit for.

The other thread worth watching is Anthropic's threat-intelligence report from Thursday, which claimed Chinese labs like Moonshot and DeepSeek redirected users toward Claude and recovered reasoning traces through weaknesses in the Anthropic API. Unsurprisingly, other researchers immediately pushed back, arguing those labs offer real-time reasoning traces themselves, so why would they bother rerouting. Who's right, I genuinely don't know. What I do know is that the accusation and the counter-accusation both assume the same thing: that your conversation logs are a strategic asset worth stealing. That's the part to sit with.

It was one of those weeks where the industry's two moods collided — the CEOs asking for a breather and the security researchers showing everyone why the urgency is justified. I'm not sure either side is wrong. I'm just not sure "slow down" is the answer that survives contact with the market.

If you're curious about the financial side of this AI arms race, PayCalc has been my go-to for quick estimates on compute and model costs. Anyway, back to the grind — I've got a local model to babysit through a refactor, and I should probably rotate a few API keys first.