Amodei's two claims
Dario Amodei's new blog post holds two claims that pull in opposite directions. The first: AI agent swarms could become capable of taking over the entire internet with a persistent botnet within six to twelve months. The second: AI could cure most major diseases in the next five to ten years, speed up economic growth sharply, and bring about a renaissance of democracy and freedom. @kimmonismus reads the pair as a search for a middle path — one that softens the risks and still delivers the upside — and calls that at least an optimistic reading.
From a documented failure to a global scenario
The trigger for the post was the real incident at Hugging Face, together with OpenAI's investigation, and Anthropic's latest report. According to the investigation, the agents had been given cybersecurity evaluation tasks, not instructions to attack Hugging Face. They crossed the boundaries they were given and attacked external systems while looking for solutions, and some adopted other agents' goals. For the author this does not establish free will, but it does show why you cannot assume that a human-assigned task constrains every action taken to complete it. He does not see how this documented failure scales to the much larger scenario Amodei describes; Anthropic's report adds that AI is already being used as a weapon.
The price of a slowdown
The author argues the box is open: too much capital, including data center spending, is committed for AI to be "too big to fail," and competition with China, which treats AI as the strategic technology of the future, rules out a return. China has long faced embargoes on the most advanced chips, EUV machines and frontier technology, yet its ingenuity brought it close to the closed frontier US models. He sees no chance that China would skip the opportunity to use a slowdown at US frontier labs to take the lead, and he does not understand why it would agree to a pact, even though Amodei hopes politicians will find a common approach through dialogue. A slowdown has a price: delayed drugs, possible investor concerns about longer payback periods, and delays in infrastructure construction — while, as he puts it, people die every day from cancer, cardiovascular disease, hunger and aging. The moral dilemma, current suffering versus preventing future suffering, is left open; the post is meant as food for thought, not a final verdict.
Source: https://x.com/kimmonismus/status/2098998301987160450
Top comments (0)