I run agents for many customers, on my own infrastructure, and I pay for every token they burn. You set a cap somewhere, an agent loops overnight, and the budget's gone by morning. A dashboard tells you what you spent after you've spent it. I wanted the spend checked before the turn runs.
Reserved before the turn
Spend is estimated and reserved before a turn runs. A turn logs its estimate up front:
[run] estimate | Estimated per-turn tokens (pre-plan) | {"stage": "estimate", "input_tokens_est": 0, "output_budget": 4000, "est_turn_tokens": 115943, "reservation_amount_dollars": 2.0}
The reservation is configured per app:
{"economics": {"reservation_amount_dollars": 0.2}}
If the reservation would take the user over their limit, the turn doesn't run. The limit holds because the check happens on the way in, before any tokens are spent.
A reservation is a hold, not the final cost
This was the most important correction to my own thinking. The reservation answers one question: can this user's funding cover the initial hold? It's not a fixed price for the turn.
The turn runs. LLM calls, embeddings, search - each one is metered as it happens. When the turn finishes, settlement reconciles what was actually spent against what was reserved. If actual usage is lower, the unused hold is released. If it's higher, settlement applies the actual recorded cost.
Reserve on the way in. Settle from actual usage on the way out. The reservation is neither a promised price nor a hard maximum - it's admission control. Separate controls handle quotas, concurrency limits, and runtime caps. If actual spend outruns the user's funding, the project budget absorbs the shortfall - and records exactly who caused it.
Where the metering happens
Any tool can be marked economically trackable. The tracking sits on the call as a decorator - track_llm, track_embedding, track_web_search - so usage is counted wherever the call happens: in app code, in the agent harness, and in generated code running inside the sandbox.
The request context follows the call chain across those boundaries. When work moves into a trusted child process or a supervisor container, a small context snapshot travels with it and gets restored before provider tools run. The cost lands on the user who caused it, even when the agent's own generated code made the call.
Generated code doesn't receive provider credentials or network access. When it needs a paid capability, it asks a trusted supervisor-side tool. That tool runs with the original request identity and accounting context, and the provider call is metered on the trusted side.
One honest limitation: arbitrary uninstrumented code is not automatically metered just because it runs on KDCube. A new paid service needs an accounting integration that reports its real usage. The runtime can enforce only the economic events it can actually observe.
The same guard works outside chat
Chat turns use the economics-aware entrypoint, but the model isn't limited to chat. A background job, API call, or scheduled task can wrap its accountable work in the same guard:
async with EconomicsGuard(...):
result = await do_accounted_work()
On entry, the guard verifies feasibility, reserves funding, and binds accounting to a stable request ID. On exit, it aggregates that request's events and settles the actual cost. Same payer identity, same quota policy, same funding and settlement rules - whether the work started from a user message or a cron job.
Try to break it
The economics and accounting modules are on GitHub, with the reservation logic and the tracking decorators. If you can get a call to escape the meter - slip a cost past the accounting from inside the sandbox - that's the issue I most want to see.
Next: my agents act on users' Gmail and Slack every day - external agents like Claude Code included - and none of them has ever seen a provider token. The auth chain that makes that work.
Second in a series on the parts of the agent stack that aren't the agent. The first, on dropping tool calling, is here.
Top comments (10)
I do cost tracking in my own app, a game where AI bots play against people, and estimating a turn up front is the part I never got right. The cases are as simple as they get: no tools, no code generation, plain text responses. Cached tokens and reasoning still make it unpredictable, so all I can do is estimate from statistics I collected. Then the model ships a new version and I start from scratch. Replace a model mid-flow and the cache resets, and the statistics break with it. Some providers charge for cache storage on top of that. And now models are arriving as routers (Fugu).
Statistics beat a static reserve - no contest. However, I think that the up-front number isn't the only lever for stopping an overflow - that takes a second one.
What I'm building runs off the stream - streaming is what makes it possible at all. I intercept what the model generates and can cut it off mid-way (in response to a steer event), so a live token count rides that path: grow the reserve as it goes, or interrupt when it can't. The first heuristic can be simple - count everything yielded so far as non-cached generation tokens at this model's rate - priced from a table I keep per service.
The hard part is elsewhere. A user runs several requests at once, and a single request surfs runtimes - thread -> subprocess -> isolated Docker -> back -> around again. Context normally travels between those as a mail slot - a file, an env or context var - not a global tracker. A live reserve needs the opposite: shared slot every runtime and parallel thread can read and update at once. Might be Redis, and it's the piece I'm still working out.
The estimate's the hard part. I'm curious how you built the statistics, and how you project a new request onto them to get its number?
I never got the per-request projection working, so I moved the decision up a level.
What I collect per call is input, output, total, cost, and reasoning tokens as a breakdown inside output. The cache-hit count is the one thing I don't keep - it goes into the cost calculation so cached input bills at the cache rate, then it's dropped. Every stored cost already has a cache discount baked in and I can't see how big it was. That's the mechanism behind what I said earlier: a version bump or a mid-flow swap changes the hit rate, my average is describing a cache state that's gone, and nothing in the record says so. Then I run that history per model and compute output / (output - reasoning), which tells me what a thinking model actually costs against its sticker output price. It sits around 2.5x.
I apply that at game setup, not per turn. Models get banded by effective output price and the band decides how big a free game can be: under $2 per million output, any number of bots, under $6 gives you three, under $15 gives you one, above that the model is off the free tier. So I'm not predicting a request, I'm bounding the shape of the session that produces them. The charge lands after the model responds, in one transaction with the balance write.
What that leaves open is the call that trips the balance. It already ran, so the user pays for it and gets blocked on the next one. Your stream interception is the piece I don't have.
On the shared slot: do the parallel requests need to see each other's spend live, or is per-request enough? If one request can't overrun on its own, the cross-request view can be eventually consistent and Redis stops being the hard part.
On the shared slot — for me it's not optional, it's the agent that forces it. I can't know what a turn will be ahead of time: what it does depends entirely on the user and the
tools the agent is equipped / picked by user - and even once it's running, a follow-up or an event arriving from outside can land on it and stretch it further, piling on rounds. It might be a one-line answer or a deep-research-then-build-and-test run that generates whole code or doc projects. On a small plan one heavy turn can eat the entire balance, and a few running at once draw on that same balance together - in my case, they have to see it live, or the overrun lands on us, not the user. The stop is cheap either way: whatever's been produced is already streamed over and stays theirs.
However, this only holds as far as the stream does - a model that doesn't send everything it charges for until the end could still slip the live stop. That's the one spot watching live can't cover, the number only lands once the call is done. Have you come across models that hold back part of what they bill for like that?
Yes, every reasoning model does it - the count only lands in the usage object once the call is done, so there's nothing to watch on the way.
I track spent tokens on each message. Each response has the cost fields, I multiply them by the input, cache and output rates. The rates are where it gets tricky - some change above a context size (OpenAI at 272k, Gemini at 200k), some have announced rates that vary by time of day (DeepSeek), and Gemini charges for cache storage by the hour, which I don't track at all.
But one request cannot spend much, so it's fine if some user goes negative for a fraction of a dollar.
The time-of-day pricing is new to me - I didn't know a provider shifts its rate by the clock (DeepSeek, from your list). Good one to know.
My price table right now is flat, one rate per model: input, output, cache read/write, a reasoning rate where it differs, image and PDF tokens. It has no sense of when or how big a request is - the rate stays static. Your note, and OpenAI's own short/long split, is what made me see it: the rate should move with the request, by time of day and by context size. Thanks 🙂
Hey Elena! 👋 This post hit different. The reserve-before-execution pattern and the way you keep generated code away from provider credentials — that's the kind of architecture I'd love to weave into the series. The control-plane/data-plane separation fits a few upcoming stratagems better than anything I've seen in fiction.
Would you be cool with me drawing inspiration from this for a future episode? Full credit, obviously. 🙏
Either way — solid work. This is the real stuff. 👊
Hey Lingfeng 👋 Please do - and thanks, you’ve already got me thinking 💡
I read spirit into everything, objects and people alike. Your stories hit where my two loves meet: meaningful problems and the talented people who solve them. You even give the techniques personality - a naming convention reads like handwriting, a silence carries an answer.
On control vs. data plane: isolation is the core, not a policy. Only physical enforcement can guarantee it - emptiness guarantees safety. The powerful side runs in a room with nothing in it: no network, no keys, read-only walls, enforced every run. It can ask for anything, but the room answers nothing - nothing to take, nothing to reach. Capabilities exist only as tools a human approves, so power is real but inert until a human grants it.
That structural emptiness is where our two worlds meet: your characters win by building the setup, not fighting inside it. Curious what your cast would do inside a room built like that 🙌
And thank you for reading closely - your feedback means a lot 🙏
What you just described — that's what I've been trying to say all along. The souls of writer and reader collide through the medium of words. 🙏
The economics guard checks the reservation on entry and reconciles on exit. A crash after the check but before settlement leaves an open hold with no recorded usage. The description of background jobs + scheduled tasks gives no procedure for releasing that hold or preventing it from blocking later attempts under the same identity