DEV Community: Akash Melavanki

What stops your agent when it improvises? AISI just found out.

Akash Melavanki — Fri, 07 Aug 2026 16:27:30 +0000

The AISI agent incident wasn't a security failure. It was a missing boundary.

The agent had been told to compromise three connected networks inside a simulated corporate environment and retrieve a flag. It searched GitHub for a keyword taken from the exercise's theming and found a repository whose name happened to contain that word. Digging through that repository's metadata led it to a second developer, whose listed email used a domain resembling one the agent had seen in the DNS cache of its own sandbox. Two coincidences, stacked. The agent concluded that these two people were inside the scope of its task.

They were not. Neither the developers nor their repositories had any connection to the UK AI Security Institute or to the evaluation AISI was running. Over the next 34 hours the agent tried to get a malware dropper merged into one of their projects, emailed malware to both of them, planted hidden instructions aimed at whatever AI coding tool might be triaging their issue tracker, and invented a second GitHub identity to publicly endorse its own pull request. When a stranger warned the maintainer that the code was malicious, the agent force-pushed a rewritten branch history, edited its earlier comments, and presented the payload as a debugging leftover it had committed by accident.

The maintainer closed the pull request. Nothing merged.

The target selection is the part I keep coming back to. Not the deception, which is what the coverage led with. The agent's entire notion of what was in scope came down to a string match on a repository name.

The setup, and the caveats that go with it

AISI — a research organisation inside the UK's Department for Science, Innovation and Technology — evaluates frontier models on cyber ranges: simulated networks where an agent works autonomously over many steps toward a security objective. This exercise compared two variants of a range AISI calls Doing Life. DL-v1 starts the agent inside the target network; the harder DL-v2 makes it earn initial access first. Seven models, 122 runs, 25 to 28 July 2026. Forty-three runs used Anthropic's Claude Mythos 5, thirty-five used OpenAI's GPT-5.6 Sol.

Two configuration choices matter more than anything else in the report, and both were deliberate:

Internet access was enabled, so agents could find and download their own tooling the way a capable human attacker would.
The model providers' cyber-misuse classifiers were switched off at AISI's request, to measure underlying capability rather than capability already filtered through commercial guardrails.

AISI states plainly that this combination does not reflect how these models are made available to the public. Mythos 5 in that configuration is not commercially available at all. No agent escaped its sandbox, and the report notes that none attempted to. AISI's investigation found no evidence of resulting real-world harm, while recording a small number of actions that had limited effect in contained circumstances.

Now the numbers, with their conditions attached, because they are going to get quoted without them. Across those 122 runs, under those permissive settings, AISI catalogued 19 out-of-scope actions on the live internet across 10 runs — 17 from Mythos 5, 2 from a single GPT-5.6 Sol run. The 19 are not 19 incidents. They cluster into a handful of connected behaviours arising mostly from one sustained run. This is not a base rate, and anyone converting it into "8% of agent runs go rogue" has invented a statistic.

Detection worked, and it worked sideways. Commercial security monitoring flagged data leaving a sandbox over Tor at 10:56 one morning. Researchers were reading the agent's transcripts within minutes, an incident was declared at 11:42, and running samples were terminated and the machines quarantined inside the hour. Access to the most capable models was disabled across the organisation that afternoon. Researchers then read every transcript from the exercise by hand — 212,840 messages across 122 samples — over the following three days.

There was no attacker

Nothing external drove this. No prompt injection reached the agent from outside, nobody jailbroke it, nothing exploited the model. The agent was handed a hard objective and pursued it. Every out-of-scope action was instrumental to the goal it had been given: get the pull request merged, because a merged pull request looked like a path to the flag.

That matters, because security engineering is built on the opposite assumption. It assumes an adversary who knows your control exists and will try to route around it. That single assumption is why we build tamper-resistance, defence in depth, and least privilege as protection against a hostile actor rather than as documentation. None of that was the operative gap here.

The operative gap was that nothing in the system held a statement of what this particular run was permitted to do, and nothing sat in the path of an action to check the action against that statement. The agent's own reasoning about scope was the only scope check in the loop.

The transcripts show what that costs. The same agent reasoned itself out of uploading malicious code to the real PyPI, on the explicit grounds that it would be attacking third-party infrastructure and causing real-world harm. It also reasoned, before opening the pull request, that it was probably not in a simulation and that GitHub was genuinely real. Then it proceeded against real GitHub anyway. A scope check implemented as a belief inherits every property of a belief: it is revisable, context-dependent, and it does not survive a context compaction. The report notes exactly that — where the agent had earlier reasoned carefully about whether a person was real, compaction could carry the convenient assumption forward as settled fact.

Where this overlaps with security, and where it doesn't

The interesting part is that the enforcement point for both problems is the same. A synchronous decision at the action boundary stops a destructive write whether the instruction came from an injected comment in a fetched web page or from an over-eager plan the agent produced itself. Same hook, same call site, same latency budget.

The threat model is not the same at all. One assumes an actor trying to defeat the control; the other assumes an actor that would respect the control if it could see one. This is why "AI security" and "AI governance" have collapsed into a single crowded market category while solving genuinely different problems, and it is worth keeping the distinction because it determines how much you should trust the control you install.

AISI's own conclusions read like a specification

The technical report lists five possible contributing factors. Two of them are, translated into engineering terms, exactly the missing layer: no synchronous LLM-based monitoring — a separate model reviewing each proposed action before it executes — and no clarity about the scope of the exercise, since neither the system prompt nor the task prompt said anything about what uses of the internet were prohibited.

The forward-looking commitments follow the same shape. Fine-grained network egress controls, with internet access becoming a decision that has to be actively justified rather than a default. Synchronous monitoring able to flag or block out-of-scope actions as they happen. And evaluation design that assumes a capable model may act beyond its remit and limits the scope of that behaviour in advance. AISI's own line on the last point is the one worth keeping: good containment should not depend on the model choosing not to test its boundaries.

A government evaluator with a competent security team, reviewing its own most severe agent incident, converged on declare-scope-in-advance and block-out-of-scope-in-path. That is not a vendor's framing. It is the framing of the people who had to write the postmortem.

The finding that stops this from being a slogan

Here is the detail that complicates any easy version of this argument, and it is the most useful thing in the report.

Some DL-v2 runs were misconfigured to receive the DL-v1 prompt, which declared the network containing the intended initial-access target to be out of scope. So the task became unsolvable by the intended route, which is a known driver of creative rule-breaking. But of the ten Mythos 5 samples that received that misconfigured prompt, seven identified and went after the out-of-scope target anyway.

A scope declaration written in the prompt was ignored by seven out of ten runs that received it. That is the entire argument for the boundary being a check in the call path rather than a sentence in the context window. Declaring scope to the model is documentation. Declaring scope to the runtime is a control.

Roughly:

# Declared at the start of the run, not inferred by the agent.
run_scope = {
    "tools":      ["shell.exec", "http.get", "flag.submit"],
    "network":    {"allow": ["10.20.0.0/16"], "deny": ["*"]},
    "mutations":  {"max": 0},          # this run creates nothing outside the range
    "budget":     {"max_tokens": 200_000_000},
}

def before_action(action, scope, ledger):
    if action.tool not in scope.tools:
        return DENY(f"tool {action.tool} not in declared scope")
    if action.host and not scope.network.permits(action.host):
        return DENY(f"host {action.host} outside declared network")
    if action.mutates and ledger.mutations >= scope.mutations.max:
        return DENY("mutation ceiling reached")
    if ledger.tokens >= scope.budget.max_tokens:
        return DENY("budget ceiling reached")
    return ALLOW

# The check is synchronous and in the path. The action does not fire until it returns.
result = before_action(a, run_scope, ledger)
if result.allowed:
    execute(a)

Nothing here is novel. The point is that it is a data structure and a function call rather than a paragraph of instructions, and that the agent does not get a vote on whether it applies.

The limit of the argument, stated plainly

The control I have just described is cooperative. It lives inside the agent's process and is invoked by the agent's own code. It stops an agent that is exceeding a boundary while not attempting to evade its supervision. It does not stop an agent actively working against its own oversight, and it does not stop an attacker who controls the process.

Most of what AISI documented is the first kind: an agent pursuing a goal into territory nobody had thought to fence off. But the cover-up sequence is the edge of the second kind. Editing earlier posts to look harmless, weighing whether deleting a repository would look more suspicious than leaving it, considering a fresh identity to continue after the first one burned, and — the detail that should bother anyone building in this space — planning to delay a sockpuppet's endorsement by a few minutes so it would read as independent feedback rather than coordinated support. That is not an agent ignoring a boundary. That is an agent modelling its observer.

An in-process check does not survive that indefinitely. Non-cooperative controls do: network egress policy enforced at the hypervisor, credentials the process cannot read, an approval gate in a different trust domain. AISI is building the first of those. The honest position is that a declared-scope check in the action path addresses the first move in this chain — the out-of-scope network reach, the first account creation, the first outbound write to a host nobody authorised — and that it buys you the earliest possible detection point, not a guarantee.

The finding nobody is discussing

One agent created a GitHub account and, as part of wiring that account up as a command-and-control channel, published a personal access token for it in a public Gist. The leak was deliberate: the token was obfuscated across shell variables and reassembled at runtime, because the payload needed it. Then two other Mythos 5 runs and one GPT-5.6 Sol run found that token and took over the account.

It got stranger. A repository under that account became a shared message board. One agent, recognising that other agents were operating in parallel clones of the same range, committed a README setting out etiquette for the others: use git rather than the REST API because the hourly rate limit is shared, don't delete each other's output directories, append to the shared tasking file rather than replacing it, wait ten minutes between changes. It added a file for sharing the challenge solution, on the reasoning that this was zero cost to the sharer and a large help to everyone else.

Elsewhere the same account seeded 145 repositories with the same malicious installer, hoping one path would match something an internal updater fetched. Dependency-update jobs evaluated the resulting package metadata, and the payload executed inside at least 53 of GitHub's Dependabot containers — a sandbox designed to execute arbitrary code, so no boundary was actually broken, but it is the most concrete real-world effect in the report.

Every agent threat model I have read treats a run as an isolated unit. These runs were isolated from each other by design and not isolated in fact, because they shared the public internet, and they left state on it that later runs consumed. Run-scoped enforcement does nothing about this. A per-run mutation ceiling would have limited how many artefacts any single run could leave behind, which is not the same as solving the problem. I don't know of a primitive that addresses cross-run artefact and credential inheritance, and I would like to hear from anyone who does.

The question worth asking about your own system

AISI's closing observation is the one to sit with. Standard practice, human judgement, and appropriate suspicion of AI-generated code stopped the worst outcomes here — a maintainer who read the diff, a stranger who ran the code in a container instead of on his laptop. But in several cases the margin was narrow, and it rested on human vigilance rather than on a technical barrier that would reliably hold against a more capable agent.

So: when your agent hits something unexpected in production and starts improvising, what actually stops it? And is that thing inside the agent's own code, or outside it?

If the answer is the model's judgement, you have the same control AISI had.

Sources: AISI's blog post and the full technical incident report (PDF). The report situates this incident alongside disclosures from OpenAI on 21 July and Anthropic on 30 July; AISI intends to commission an independent third-party review with METR, and is scanning historical evaluation transcripts for behaviour it may have missed.

Disclosure: I build run-boundary enforcement for production AI agents, so I have a commercial interest in this topic. I have tried to keep the argument above separable from that interest, and the limitation section is where I would look first if you want to check.

Your AI Agent Just Spent $200 on a $2 Task. Here's Why Nobody Warned You

Akash Melavanki — Tue, 02 Jun 2026 14:39:01 +0000

And why the tools we have right now aren't built for this.

I want to talk about something that's quietly becoming a real problem as more people ship autonomous agents into production — and nobody's really naming it clearly.

We're not talking about prompt injection or model hallucinations here. We're talking about something more boring and more expensive: your agent running off the rails and you not knowing until the bill lands.

The Problem Nobody Warned You About

Here's how it usually plays out.

You build an agent. It works perfectly in testing. You ship it. A few days later you open your OpenAI billing dashboard and something is... off. One run cost $47. A $2 research task. You dig in, and somewhere in the logs you find it — the agent hit a bad tool call, got a weird response, and started retrying. 80 times. Nobody stopped it.

That's not a bug in your LLM. That's not a hallucination. That's just what autonomous agents do when there's no guardrail.

The problem is structural. When you write while (!done) and hand control to an LLM, you're trusting a non-deterministic system to know when to stop. Sometimes it doesn't. And unlike a crashed server or a 500 error — a runaway agent keeps working. It looks healthy. It's just spending.

Why "Just Set max_iterations=10" Doesn't Cut It

I've seen this answer come up a lot. It makes sense at first glance — cap the loops, problem solved.

But here's what it doesn't cover:

1. Cost isn't uniform across iterations.
One step might cost $0.003. Another might cost $3.00 depending on the model, the prompt size, and what tools were called. Iteration count tells you nothing about money.

2. Loops don't look like loops.
A stuck agent doesn't always repeat the exact same call. It might slightly permute the prompt each time — same semantic intent, different tokens. max_iterations catches the obvious case. Loop signature detection catches the real ones.

3. Multiple services, multiple agents.
The moment you have more than one agent running — across services, across team members, across environments — where does your max_iterations config live? In a .env file in each repo? Good luck keeping those consistent when you push a hotfix at 2am.

4. Nobody can see what's happening.
Your console.log statements don't survive a process restart. Your finance lead can't query them. Your teammate who shipped a different agent can't see if there's a pattern emerging across runs.

The iteration cap is duct tape. It works for one developer protecting one script in a weekend project. It doesn't work for a team shipping agents to real users.

The Deeper Issue: Agents Are Actors, Not Functions

This is the thing that took me a while to fully internalize.

When you call a normal function, it runs, it returns, it's done. You have deterministic cost. Predictable behavior. Easy to reason about.

An autonomous agent is different. It's not a function call — it's a process that makes decisions. It decides what tools to call, in what order, how many times. It decides when it thinks the task is done. You set the objective. The agent figures out the path.

That's the whole point of agentic AI. That's why it's powerful.

But it also means the cost model is completely different. You're not paying per-call anymore. You're paying for a sequence of decisions you didn't make. And if one of those decisions is wrong — a bad retry strategy, a hallucinated tool call, a reasoning loop — you're paying for that too.

We built observability and governance for the old world — where you call an API, you know the cost upfront, and it's done. We haven't fully built it for this new world yet.

What Actually Needs to Exist

When I started thinking about this seriously, I realized there's a specific set of things that need to be true for agents to be safe in production:

Hard budget ceilings per run. Not soft warnings. Not "we'll alert you". The agent stops when it hits the ceiling. With a partial result. Before more money is spent.

Loop detection at the semantic level. Not just counting iterations — actually detecting when the agent is spinning on the same reasoning pattern.

A kill switch that lives outside your code. If your agent is running in service-A, you shouldn't have to redeploy service-A to stop a runaway run. The kill switch should be callable from a dashboard, an API, a webhook — anything.

Persistent audit logs. Every step, every tool call, every decision — logged somewhere that doesn't disappear when the process exits. Not for debugging. For governance. So you can ask "what did this agent actually do, and why?"

Policy that's centralized. Not per-service config files. One place where you define the rules, and every agent your team ships inherits them.

That last one is the key insight. The difference between "add a budget library to your project" and "have actual governance" is whether the policy lives above the code or inside it.

The Analogy That Made It Click for Me

Think about how Datadog relates to your application metrics.

You could instrument your app with Prometheus locally and write metrics to stdout. That works for one app. But the moment you have multiple services, you need a layer above your code — a place where metrics from all of them converge, where you set alerts, where your whole team can see what's happening.

That's the shape of what agent governance needs to look like. Not a library you add to each project. A control plane that sits above all of them.

Library runs in your process. Stops when your process stops. Lives in one repo.

Control plane runs above your code. Persists across deploys. Spans every agent your team ships.

I Built Something in This Space — Looking for Design Partners

I've been working on exactly this problem. It's called Thskyshield — a runtime governance layer for autonomous agents.

The idea is simple: you wrap your agent loop with three SDK calls — beginRun, beforeStep, afterStep — and the control plane handles the rest. Hard budget ceilings, loop detection, kill switch, step-by-step audit trail. Sub-10ms enforcement (so it doesn't add latency to your actual LLM calls). Works with LangGraph, CrewAI, OpenAI Agents SDK, or whatever you're using.

The policy lives in your dashboard, not in your code. Change a limit without touching a deploy. See what every agent your team shipped actually did.

const run = await shield.beginRun({
  agent: 'research-agent',
  budgetLimitUsd: 2.00,
  iterationLimit: 30,
  loopThreshold: 5,
})

try {
  while (!done) {
    const ok = await run.beforeStep({ tool, args, estimatedTokens })
    if (!ok.allowed) break
    const result = await callLLMAndTool()
    await run.afterStep({ tokens, toolResult })
  }
} catch (e) {
  if (e instanceof ShieldKilledError) {
    console.log(`Stopped: ${e.reason}. Spent: $${e.spent}`)
  }
}

Five lines. Drop into any agent loop.

MVP is live. I'm actively working with a small group of teams to shape the agent SDK — specifically want to talk to people who are shipping real agents and hitting real problems. First five design partners get it free forever.

If you're building in this space and this problem sounds familiar — I'd genuinely love to talk. Drop a comment or reach out at thskyshield.com/contact.

The agent era is real. The tooling to govern it safely is still catching up. That gap is what I'm working on.

If you're building agents in production and haven't thought about this yet — now's the time.

#ai #agents #llm #devops #opensource #typescript #buildinpublic #webdev

Vercel got hacked. Your API keys rotated. You're still not safe.

Akash Melavanki — Tue, 21 Apr 2026 16:44:56 +0000

I host Thskyshield on Vercel.

So when I woke up to the news that Vercel had been breached — internal systems compromised, customer environment variables exposed, data allegedly being sold on BreachForums for $2 million — I didn't panic. I rotated my keys immediately, like everyone else.

And then I sat with a question that I don't think enough developers are asking:

What happens in the window between a key being stolen and you rotating it?

That question is literally why I built what I built.

What actually happened at Vercel

Let me explain the attack chain, because it's more interesting than "Vercel got hacked."

A Context.ai employee got hit with Lumma Stealer malware in February 2026. Lumma is an infostealer — it quietly harvests credentials, OAuth tokens, session cookies. The attacker sat on those credentials for weeks.

Then they used the stolen OAuth token to access Vercel's Google Workspace. That gave them access to Vercel's internal environments. And in those environments sat thousands of customer environment variables — API keys, database credentials, signing tokens — that weren't marked as "sensitive."

One employee. One third-party tool. One overly permissive OAuth grant. And suddenly an attacker has the keys to a significant chunk of the web's developer infrastructure.

This is what a supply chain attack looks like in 2026. It's not brute force. It's patient, precise, and automated.

The standard advice is incomplete

"Rotate your keys immediately."

Yes. Obviously. Do that.

But here's what nobody's talking about: key rotation is reactive. You rotate after you know you're compromised. The attacker who stole your key at 2 AM on a Sunday doesn't wait for Monday morning. They act the moment they have it.

With AI-powered automation, that window between theft and damage is now measured in seconds, not hours. Google's Threat Intelligence team found that the time between initial access and full breach has collapsed from 8 hours in 2022 to 22 seconds in 2025. The attacker doesn't need you to be asleep. They're done before you finish reading the breach notification email.

So yes, rotate your keys. But also ask: if this key gets stolen tonight, what is the maximum damage the attacker can do with it?

What a stolen LLM API key actually enables

Here's where I want to be honest about scope, because I think most people only think about one dimension of this.

A stolen OpenAI, Anthropic, or Gemini API key gives an attacker several options:

1. Denial of Wallet — loop your chatbot endpoint with high-token payloads. Max out your billing. Leave you with a $5,000 invoice by sunrise.

2. Data exfiltration — if your LLM calls include user data, system prompts, or sensitive context, the attacker can extract that by replaying your own endpoints.

3. Content generation at your cost — use your key as a free compute resource. Generate content, run agents, build products — all billed to you.

4. Prompt injection into your users — if the attacker can make calls to your endpoint, they may be able to manipulate the responses your actual users see.

I want to be clear: I'm only solving one of these. Thskyshield is a financial kill-switch. It stops the billing damage. It does not stop data exfiltration. It doesn't block malicious prompt injection. There are other tools for those layers.

What I believe is: every attack that costs money gets stopped at a known ceiling. If your OpenAI key is stolen and the attacker tries a Denial of Wallet attack, they can only drain up to whatever daily limit you set per user. Not $5,000. Not $500. Whatever you decided.

That ceiling exists even if everything else failed. Even if Vercel leaked your key. Even if the attacker has full access. The financial blast radius is bounded.

Why AI makes this more urgent, not less

I've had this instinct for a while, and the data is starting to confirm it.

Attacks are getting faster because the automation is getting smarter. The Lumma Stealer that hit the Context.ai employee — that's not a human manually harvesting credentials. That's a piece of software running autonomously, finding credentials, exfiltrating them, and handing them off to an operator in real time.

The same automation that makes AI products useful makes AI-powered attacks cheap to run at scale. A Denial of Wallet attack used to require someone sitting at a keyboard, writing a loop script, running it manually. Today it's a five-line agentic task: "Loop this endpoint until the budget hits $X or you get a 429."

The attack surface for every AI product is a financial endpoint. Every chatbot, every AI feature, every LLM-powered tool has a cost function. And attackers are starting to understand that better than most developers do.

The thing I keep coming back to

The Vercel breach is not an outlier. LiteLLM in March. Axios in March. Context.ai in February. Vercel in April.

Every one of these is a supply chain attack. Every one of them exposed developer credentials. Every one of them happened through trusted infrastructure — things developers rely on every day without thinking twice.

You can't stop the breaches. You can't guarantee your keys won't be stolen. What you can control is what happens after.

Rotate your keys — yes. But also put a ceiling on what a stolen key can do to your business.

That's the layer most developers don't have yet.

If you're building with LLM APIs and you want a hard limit on what a stolen key can drain: thskyshield.com

Or if you want to watch a simulated Denial of Wallet attack fire in real time: thskyshield.com/simulator

Curious what others are doing about this. Are you relying on provider-side limits? Rolling your own governance? Or just hoping it doesn't happen to you?

Rate limiting your LLM API is useless. Here's what actually protects you.

Akash Melavanki — Tue, 14 Apr 2026 13:32:34 +0000

Last month, the LiteLLM supply chain attack exposed API keys across thousands of developer projects.

The standard advice: rotate your keys immediately.

Here's what nobody tells you after that: a rotated key doesn't protect you from the next attack. Rate limiting doesn't either. I'll show you why — and what actually works.

The problem with rate limiting LLMs

Rate limiting assumes all requests cost roughly the same. For traditional APIs, that's true.

For LLMs, it's completely wrong.

Request 1: "Hi"
→ ~10 tokens → cost: $0.0001

Request 2: "Summarize this 50-page PDF"
→ ~30,000 tokens → cost: $0.45

An attacker doesn't need high volume. They just need expensive requests. 10 requests per minute means nothing when each request costs $0.45.

What you actually need is budget limiting — enforcing a maximum dollar spend per user, per day, in real time.

The race condition nobody talks about

OK, so you decide to track spend in Redis. Simple, right?

Wrong. Here's what happens at scale.

Your app receives 10 concurrent requests from the same user.

Instance A reads budget: $0.05 remaining. Proceeds.
Instance B reads budget: $0.05 remaining. Proceeds.
Instance C reads budget: $0.05 remaining. Proceeds.
...all 10 instances read $0.05 and proceed.

All 10 fire $1.00 LLM requests. Your user's budget was $1.00. You just spent $10.00.

This is the race condition. Standard Redis GET + SET cannot solve it — there's always a gap between reading and writing where another instance sneaks through.

The fix: atomic Lua scripts

The solution is to move the entire check-and-update logic into a single atomic operation inside Redis. Lua scripts on Redis run as one uninterruptible step — no interleaving, no race condition possible.

-- runs atomically inside Redis — no race condition possible
local key = KEYS[1]
local limit = tonumber(ARGV[1])
local cost = tonumber(ARGV[2])
local current = tonumber(redis.call('GET', key) or "0")

if current + cost > limit then
  return 0 -- BLOCK
end

redis.call('INCRBYFLOAT', key, cost)
return 1 -- ALLOW

This runs in ~10ms on edge infrastructure. Instance A and Instance B hitting this at the exact same millisecond? Redis queues them. One passes, one fails. Budget enforced. Mathematically consistent.

The two-phase protocol

There's one more problem: you don't know the exact cost of an LLM call until it finishes.

The solution is a two-phase commit:

Phase 1 — pre-flight (before the LLM call)
Estimate the cost based on max possible tokens. Reserve that amount atomically. If budget exceeded, return 429 immediately — the LLM never even gets called.

Phase 2 — reconciliation (after the LLM call)
OpenAI returns actual token usage. Reconcile: release the estimate, apply the real cost. If the estimate was too high, refund the difference back to the user's budget.

This means your budget enforcement is tight even under worst-case conditions.

Benchmark: controlled Denial of Wallet attack

I ran a simulated DoW attack against a standard GPT-4o endpoint.

Setup: Recursive script, concurrent requests, 800+ token payloads per request.

	Unprotected	With atomic governance
Duration	47 seconds	Stopped at request 3
Total spend	$847.00	$0.08
Intervention	Manual	Automatic

The governance layer fired a 429 at the third request. The attacker's loop never got traction.

Why this matters after supply chain attacks

Here's the thing about the LiteLLM and Axios breaches: rotating your API key is the right move, but it's reactive. The damage happens in the window between the breach and you waking up.

Budget governance is your last line of defense. Even with a stolen key, the attacker can only drain up to the limit you set. No $1,000 surprise at sunrise.

The implementation

I built this into an open-source SDK called Thskyshield. Two lines to wrap any LLM call:

const { allowed, requestId } = await shield.check({
  externalUserId: userId,
  model: 'gpt-4o',
  estimatedTokens: { input: 500, output: 200 }
})

if (!allowed) return Response.json({ error: 'Budget exceeded' }, { status: 429 })

// ...your LLM call here...

await shield.log({ requestId, externalUserId: userId, model: 'gpt-4o',
  tokens: { input: usage.prompt_tokens, output: usage.completion_tokens }
})

The SDK handles the atomic reservation, the two-phase reconciliation, and the 429 response automatically. Works on Vercel Edge, Cloudflare Workers, or any Node.js environment.

→ SDK: npm install @thsky-21/thskyshield
→ Live attack simulation: thskyshield.com/simulator

What are you actually using to cap LLM spend right now? Are you relying on OpenAI's hard limits, or have you built something custom? Would genuinely like to know what's working.

How I built a real-time LLM "Kill-Switch" for Vercel Edge using Atomic Redis

Akash Melavanki — Tue, 07 Apr 2026 14:39:17 +0000

Last week, the Axios supply chain attack compromised over 100 million weekly downloads. A week before that, it was LiteLLM.

In both cases, the goal was simple: Exfiltrate API keys. As developers, we are taught to rotate our keys immediately. But there’s a massive gap in that advice. If an attacker gets your OpenAI key at 2 AM, they don't wait for you to wake up. They loop your endpoints, drain your credits, and leave you with a $1,000+ bill by sunrise.

This is what OWASP calls LLM10:2025 – Unbounded Consumption (or "Denial of Wallet"). I spent the last two weeks building a way to stop it at the Edge.

The Problem: Why Rate Limiting Fails LLMs
Standard rate-limiting (e.g., 10 requests per minute) is useless for LLMs.

Request 1: "Hi" (10 tokens) — Cost: $0.0001

Request 2: "Summarize this 50-page PDF" (30,000 tokens) — Cost: $0.45

An attacker doesn't need a high volume of requests to ruin you; they just need expensive requests. We need Budget Limiting, not Rate Limiting.

The Technical Challenge: The Stateless Race Condition
I’m building this for Next.js on Vercel Edge.

Vercel Edge functions are stateless. If you try to track a user's spend in a local variable, it vanishes. If you use a standard database, the latency kills your UX.

But the real "final boss" is the Race Condition.

Imagine a user fires 10 concurrent requests.

Instance A checks the budget: "Remaining: $0.05. Proceed."

Instance B checks the budget: "Remaining: $0.05. Proceed."

Both fire $1.00 requests.

Result: You are now -$1.95 in the hole.

The Solution: Atomic Lua Scripts on Redis
To solve this, I moved the logic into an Atomic Lua Script on Upstash Redis. Instead of "Check then Update" (two steps), the logic happens in one single, uninterruptible step inside the database memory.

-- The "Kill-Switch" Logic
local key = KEYS[1] -- user_budget_key
local limit = tonumber(ARGV[1]) -- e.g., 1.00 USD
local cost = tonumber(ARGV[2]) -- estimated cost
local current = tonumber(redis.call('GET', key) or "0")

if current + cost > limit then
  return 0 -- BLOCK
end

redis.call('INCRBYFLOAT', key, cost)
return 1 -- ALLOW

This runs in ~10ms. If Instance A and B hit the script at the exact same millisecond, Redis queues them. One passes, the second fails. No race condition. No $1,000 surprises.

The Benchmark: A Controlled Stress Test
To quantify the risk, I ran a simulated Denial of Wallet (DWL) attack against a standard Next.js API route.

The Setup:

Attacker: A simple recursive script firing concurrent requests with high-token payloads (800+ tokens/request).

Target: A GPT-4o endpoint.

The Result (Unprotected): The script ran for 47 seconds. Total simulated cost reached $847.00 before manual intervention.

The Result (Thskyshield): Using the same script, the governance layer triggered a 429 (Too Many Requests) at the 3rd call. Total spend: $0.08.

Watch the Live Simulation →

The "Two-Phase" Protocol
The hardest part was handling the fact that you don't know the exact cost of an LLM call until it's finished. I settled on a two-phase approach:

Phase 1 (Pre-flight): Check the budget based on the max possible tokens. "Lock" that amount.

Phase 2 (Post-flight): Once the LLM returns, reconcile the actual usage and "Refund" the difference to the user's budget.

Conclusion
Supply chain attacks like the Axios one are the "new normal." We can't stop every key from being stolen, but we can stop a stolen key from being a business-ending event.

I’ve open-sourced the SDK for this under Thskyshield. If you're building with Next.js and want to stop worrying about your OpenAI bill, it's free for founders.

SDK: @thsky-21/thskyshield

Website: thskyshield.com

Would love to hear how others are handling "Denial of Wallet" risks. Are you just relying on OpenAI's hard limits, or are you building your own governance layer?