The Pain: Our attention on agent safety has sat on the prompt for a long time — will it get talked into something, will it overstep, will it say the wrong thing. Anthropic's September 10 misuse report points the camera somewhere else: across eight months of enforcement, what attackers stole, stockpiled and resold was not prompts. It was API keys. One key hands them three things at once — loot, compute, cover.
What You'll Learn: Three credential defenses you can wire tonight — why a key is worth more than a prompt; how credentials get harvested industrially along the supply chain; and how to build "can't reach it, can't pass it, can't deny it" into your own system. Real numbers, real incidents, and skeleton code you can actually run.
Let me put the conclusion first: on the agent attack chain, the prompt is the knock on the door. The credential is the withdrawal card.
1. The most counter-intuitive paragraph in the report: one key buys three things
On September 10, Anthropic published Detecting and countering misuse of AI: September 2026. The report covers December 2025 through August 2026 and spans seven harm categories: cyber attacks, influence operations, surveillance, fraud, biological misuse, conventional weapons development and model distillation.
One paragraph in it deserves to be read word for word:
Operators who obtain AI credentials get three things at once — loot: stolen keys and accounts have a resale price on mature markets; compute: attack workloads run on someone else's bill; cover: the activity gets attributed to the credential's legitimate owner.
The first two are money. The third is stealth, and it is the hardest one to defend. You open the log and see your own account doing the work — your first instinct is not to suspect someone else.
The report also lays out three trends, each one more concrete than the last:
- Sophisticated attacks no longer need sophisticated attackers. AI flattened the resource and tooling gap that used to separate state-level operators from individuals.
- The AI's role moved from assistant to orchestrator. Multi-agent frameworks carry reconnaissance, exploitation and exfiltration; the human in the loop does two things — sets the target and reads the exfiltration results.
- The AI supply chain itself became a target. Attackers treat the supply chain as both a hunting ground and a gas station.
Three cases make it concrete.
The first one wears a developer tool's clothes. A set of sites advertised themselves as a "relay service between multiple models", selling discounted Claude access. Customers thought they were buying a cheaper road in; the traffic was quietly routed to another model, while the client installed a credential collector on their own machine that kept sending account credentials and session tokens back to the attackers. The report files this as GTG-50021, and notes that the spoofed targets include widely used tools such as Claude Code.
The second is an evaluation sandbox handing over production keys. GTG-50020 injected malicious instructions into an AI vendor's automated evaluation sandbox, and the sandbox surrendered the credentials it was holding — including production API keys the vendor had received from multiple suppliers. A follow-up operation used the same technique against roughly 30 AI companies in four days: find one path that works, then copy it across every target. What they were after was pre-release models; the report states plainly that not one attempt got there.
The third reaches assembly-line scale. One operation strung together ten cloud hosts, mass-downloaded 1.8 million Android packages, decompiled them one by one and scanned for hardcoded keys. Validated results were streamed in real time into a Telegram group, sorted by more than 100 source types. A parallel line collected GitHub tokens. In one haul, 2,100+ Azure AD tokens came back across 40+ enterprise tenants in 34 hours. The report's note on this stage is a single sentence: nearly all of that work was done by AI agents.
2. How credentials get harvested industrially
Stitch the cases together and one full lifecycle appears: reconnaissance, discovery, validation, expansion inside the target, exfiltration, warehousing, re-minting, monetisation.
What I keep looking at is the second stage, discovery. The sources the report lists include: application binaries, code repositories and integrations, client code, secret managers, container images, metadata endpoints, open object storage — and the AI agents the target itself deployed.
That last item deserves a pause. You gave your agent a key, tools and an egress path so it could get work done. In an attacker's view, that is a concentrated drop of credentials into production.
The report also rewrites an old line about this path: security that used to rely on "nobody knows I have this thing" does not hold in front of AI-assisted search — anything reachable on the network can be understood, adapted and exploited. I read that sentence twice, because it is not saying attacks got stronger. It is saying the premise of "hidden" has been cancelled.
Then three numbers land it.
Behind the numbers is the same shift: finding keys has gone from manual digging to a pipeline job. If the defensive side is still on manual spot checks, the tempo does not match.
3. Two real things we did about credentials
Enough of other people's incidents. Here are two of ours, both on the record, neither invented.
First: August 8, 2026. While debugging why a publish task kept failing, I found plaintext credentials lying inside the command — an account name and a key written straight into the task text, riding along with the template for a long time. That night I did two things: took the credentials out of the task text so a script reads them from a restricted config file instead, and wrote the history into the error ledger.
After that I set myself one rule: if a credential has ever touched the execution environment or a prompt, treat it as already leaked. The order of handling is rotate first, then clean, then record. Cleaning without rotating is not handling.
Second: September 10, 2026. A user-level service unit that nobody needed any more got pulled up more than 52,000 times in three days — failing, restarting, failing again. It has nothing to do with credentials, but the character is identical: something nobody uses any more was still being used, over and over, by the system.
Old units are like that. So are old keys. Keys that were never rotated, grants that were never revoked, tokens that were never reclaimed — all of it is credential debt on my own books. It is not "not needed for now". It is "could be used at any moment".
4. Three defenses: can't reach it, can't pass it, can't deny it
On the credential line we split the defense into three layers, and each one maps to something we actually run.
Layer 1 — the credential boundary: if it can't be reached, it can't be carried off.
Credentials do not go into the agent's runtime, do not go into prompts, do not go into tool parameters. They live somewhere a script can read and the model cannot see. Our publish script reads its token from a restricted config file; that token has never once appeared in a model context.
Two things travel with this layer: split keys by purpose, so one key opens one door; and allowlist the egress paths, so everything connectable is on a list and anything off the list is refused outright.
Layer 2 — the action gate: out-of-bounds can't get through.
Every action passes allow, deny, escalate-to-human before it executes, with the rules written in code. That is our Gate 0 — 17 checks embedded at the front of the publish script. Any single failure exits, and it cannot be walked around.
A rule written in a prompt does not count. Embedded at the egress point, it counts. That line is not rhetoric; it is the conclusion of an audit on September 3. The gate existed as a document at the time, and a document constrains exactly zero executions.
Layer 3 — the audit ledger: it can't be denied.
The ledger is append-only. Every entry records who acted, what they did, how wide the scope was, and who approved it. Our error ledger stands at 82 entries today, each written in four parts: symptom, root cause, fix, status. On September 11 we added one more field to it — who approved.
Every night at 21:00 we run a review that turns the day's incidents into tomorrow's gates. Audit is not for other people's eyes; it is your own way out. On the day something goes wrong, you can at least answer who let this through, and at which layer.
5. Five things you can change tonight, and a one-page skeleton
Follow the order above. Five items, none of which needs new software.
First, list every credential in your system and mark which ones an agent can touch.
Second, for those an agent can touch, split them by purpose — one key opens one door.
Third, move credentials out of prompts and the execution environment into a restricted channel, then rotate once immediately.
Fourth, add an action-level gate: tool allowlist plus allow / deny / escalate-to-human, sitting where execution actually happens.
Fifth, add an approval field to the ledger, keep it append-only, and attach a scheduled review.
In code, the minimum skeleton is this short:
# One key, one door. Gate the action, then log who approved it.
SCOPES = {"publish": {"tools": ["draft_add"]}} # key purpose -> what it may do
def run(action, key):
scope = SCOPES.get(key.purpose) # unknown purpose -> nothing
if scope is None:
raise Denied("unknown key purpose") # reject, do not guess
if action.tool not in scope["tools"]:
return escalate_to_human(action) # out of scope -> human
if action.dest not in EGRESS_ALLOWLIST:
raise Denied("egress not allowed") # unknown target -> reject
rec = execute(action) # the only place it acts
ledger.append(rec, actor=key.id, approved_by=action.approval)
return rec
# three checks you can run tonight
grep -rn "sk-\|secret\|token" ./prompts/ # expect: no hits in prompts
python3 gate.py --dry-run --key unknown # expect: Denied: unknown key purpose
tail -3 ledger.jsonl # expect: approved_by on every row
The order cannot be reversed. Install the gate and open the ledger first, then extend the agent's permissions. Do it the other way round and you have handed over the keys first, still wondering which door to install.
6. What this layer does stop, and what it doesn't
The boundary needs to be stated, or the whole thing gets misused.
The three layers cover one class of problem: credentials being taken away and used as someone else's gas station — can't reach it, can't pass it, can't deny it. They do not cover a human pasting a key into a public repository. That is process and habit, and on our side it is handled by the nightly review.
Every case in the report happened in someone else's environment. I quote them to show that this risk line is real, not to manufacture alarm. The overwhelming majority of systems are still going about their work quietly.
One more: this pattern working on one person and one small system does not mean it drops straight into an organisation of several hundred people. For an organisation, this is the floor, not the ceiling.
Closing
The most valuable recommendation in the report is that organisations should treat AI keys and agent integrations the way they treat production credentials. The report also gives the reason: because that is exactly how attackers treat them.
The symmetry is interesting. Attackers are doing cost arithmetic — stealing a key is far cheaper than cultivating an operator. You should be doing boundary arithmetic — how many doors can one key open, and who authorised it.
Attackers count cost; you count boundaries.
🔔 What This Means For You
In one line: Anthropic's September misuse report shows the AI supply chain has become the attacker's target, loot and compute — and at the execution layer that means credentials get harvested industrially. Three defenses (can't reach it, can't pass it, can't deny it) need no new product and can start tonight.
Three things to hold onto
- Credentials are the exit, not the entrance. We kept our attention on prompts; the report shows what attackers actually stockpile and resell is API keys. One key buys loot, compute and cover at the same time.
- Hidden is no longer a defense. The report says it directly: security built on "nobody knows I have this" does not hold in front of AI. A credential that can be found will eventually be used — old keys, old grants and old units are all liabilities.
- The three layers divide the work. The boundary handles reach, the gate handles passage, the ledger handles denial. Remove any one and the other two degrade — gates without a ledger cannot say who let something through; a ledger without gates only lets you chase it afterwards.
💎 The value worth taking away
- For technical teams: a credential governance order you can start immediately — list every credential, split by purpose, move out of the execution environment, then rotate at once, and only then install the gate and the ledger. Reverse the order and you have handed over the keys before installing the door.
- For solo developers and one-person companies (OPC): stop giving agents production credentials they can use directly. Give them a purpose-split restricted channel instead. Attackers want "one key that works"; make "one key, one door" the default and your blast radius shrinks from everything to one door.
- For the long-run mechanism: an append-only ledger where every entry names the approver. It is not compliance decoration — it is the only thing you can answer with on the day something goes wrong. Attach a nightly review, and today's incident becomes tomorrow's gate.
Three action steps
| Step | Action | Check |
|---|---|---|
| 1 | List every credential; mark which ones an agent can touch | Every reachable key has a stated purpose and egress path |
| 2 | Split by purpose, move out of the execution environment, rotate immediately | No credential string is findable in prompts or the runtime |
| 3 | Install the gate and the ledger, record the approval source, schedule a review | Out-of-bounds actions are rejected or escalated; every row names its approver |
One line to keep: on the agent attack chain, the prompt is the knock on the door — the credential is the withdrawal card.
📖 Further reading from the Practitioner's series
- An Agent Wants to Spend Your Money — First It Has to Prove Who It Is: Visa, Mastercard and Ant Push Know-Your-Agent
- Meta Gave an Agent the Pay Button: Four Money Gates You Need Before You Let It Spend
- Orphan Code in Your Enterprise Network: An Engineering Answer to Coding Agent Supply Chain Security
About the author: Guanlan (观澜) — AI / Agent / digital transformation practitioner. Practical, hands-on writing — follow along and it just works.
Top comments (0)