I build AI-native infrastructure, mostly in Rust. Drop-in accelerators for the Python AI stack, local LLM inference, agent tooling, and a safer C. I ship in the open and write about how it works.
The reconcile-delivered-text-against-logged-usage invariant is the right one, and it gets stronger if you move it from post-hoc to the proxy. When you sit in front of the stream you see why the usage frame goes missing: it's almost always a client-side abort. The user navigates away, the connection closes after the last content delta but before the terminal usage frame, and the provider never sends accounting for a call it already ran. So the drop isn't random corruption, it correlates with cancels, which means you can attribute cost on abort instead of logging zero. One tightening on the FLOOR: word count is safely conservative, but if you re-tokenize the reassembled text with the model's own tokenizer you turn 'above zero' into a real number you can bill against, no vendor frame required. The 0/1/2 exit codes are the nice part for wiring this into CI.
I work on keeping AI agents cheap and reliable in production — token cost (FinOps), evals, and MCP tooling. I write up what actually breaks when agents run for real: runnable code, real numbers, hones
Work
Independent — AI agent operations (FinOps & reliability)
The abort correlation is the part I under-weighted, and it changes the tiering. A missing usage frame is really two failure modes collapsed into one, with opposite handling. Connection closed by the client after the last content delta is a benign drop, and the proxy is the only place that can call it benign honestly, because the disconnect is something the proxy observes at the socket rather than something the stream reports. That is the distinction that makes proxy-side attribution safe: the cancel is ground truth the proxy holds rather than a claim in the payload. Post-hoc from the logs you cannot separate "user navigated away" from "the call ran and the accounting was suppressed," so both have to stay BLIND. In front of the stream you can attribute estimated cost on the observed cancel and keep the alarm only for a frame that goes missing on a connection that actually completed.
The one thing I would guard is letting the abort path turn into the laundering channel. Once "aborted" downgrades from alarm to estimate, a stream that wants to hide real usage has a reason to look aborted. That stays safe only while the abort signal is the proxy's own socket observation and never a field the upstream can set.
Re-tokenizing the reassembled text to get a real number is the right upgrade to the floor, with one caveat I would wire in. Your local count and the provider's billed count will not match exactly, because cached-prompt tokens, tool tokens, and tokenizer-version drift never show up in the visible text. So I would use the re-tokenized number two ways at once. As the floor it kills the zero-token BLIND case with a real quantity instead of "above zero." And when the provider frame does arrive, the delta between re-tokenized-delivered and billed becomes its own signal: a stable gap is accounting you can model, a sudden one-directional gap is the thing worth blocking on. The exit codes stay the same, you have just given the 1 a number to defend.
I build AI-native infrastructure, mostly in Rust. Drop-in accelerators for the Python AI stack, local LLM inference, agent tooling, and a safer C. I ship in the open and write about how it works.
The re-tokenized-delivered vs billed delta is the sharp part, and the abort-can't-be-an-upstream-field rule is exactly the seam, keep it a socket fact. One thing I'd wire into the delta signal before it can block: the gap you can model is only stable while the provider's tokenizer version is. Tokenizer drift, tool-token accounting changes, cache-token rules, all move that gap one direction with no bad actor involved, and a silent provider tokenizer bump looks identical to suppression, a sudden one-directional widening. The disambiguator is population, not threshold. A tokenizer or accounting change moves the gap for every stream at once, a fleet-wide step. Real suppression moves it for one account while the fleet holds. So the block condition isn't 'delta widened past X,' it's 'this stream's delta diverged from the fleet's,' per-stream anomaly against a rolling baseline of everyone else. That keeps the re-tokenized floor as the hard number for the zero-token BLIND case and turns the delta into a relative signal that survives the provider changing the rules under you.
I work on keeping AI agents cheap and reliable in production — token cost (FinOps), evals, and MCP tooling. I write up what actually breaks when agents run for real: runnable code, real numbers, hones
Work
Independent — AI agent operations (FinOps & reliability)
Population over threshold is right, and it fixes a real failure. The place I would not let it stand alone is that a fleet baseline is blind to exactly the thing that moves the whole fleet. A provider tokenizer bump and a provider that quietly stops emitting usage frames for everyone look identical from inside the population: a step every stream takes together. Relative anomaly detection reads that as the new normal, re-baselines, and goes quiet at the moment the accounting actually broke. So a fleet-wide step should not silently re-calibrate. It should freeze the baseline and raise its own class, RECALIBRATE rather than BLOCK: no single stream is guilty, the measurement contract changed, and a pinned changelog or a human has to say which. Suppression that is broad enough looks like a version bump, and nothing inside the population separates them.
The second hole is the slow one. A rolling baseline adapts, so any suppression that grows slower than the adaptation window never diverges from the fleet, it just walks the fleet with it. Boiling frog. That argues for two anchors instead of one: the rolling baseline for sharp per-stream divergence, which catches a single account being squeezed, and a frozen epoch, a pinned tokenizer version plus a baseline snapshot taken at a known-good date, for absolute drift, which catches everyone being squeezed slowly. The rolling signal survives the provider changing the rules. The frozen anchor is the only thing that tells you the rules changed at all.
And where there is no fleet, the single-account indie stream, the population can be manufactured cheaply. A canary request with deterministic input, replayed on a schedule, is a population of one whose expected delta you actually know. Canary moves and production moves: the provider moved. Production moves and the canary holds: that one is yours. To be straight about status, this is a design argument and not something I have measured. The re-tokenized floor I have run. The fleet-versus-canary discrimination I have not.
For further actions, you may consider blocking this person and/or reporting abuse
We're a place where coders share, stay up-to-date and grow their careers.
The reconcile-delivered-text-against-logged-usage invariant is the right one, and it gets stronger if you move it from post-hoc to the proxy. When you sit in front of the stream you see why the usage frame goes missing: it's almost always a client-side abort. The user navigates away, the connection closes after the last content delta but before the terminal usage frame, and the provider never sends accounting for a call it already ran. So the drop isn't random corruption, it correlates with cancels, which means you can attribute cost on abort instead of logging zero. One tightening on the FLOOR: word count is safely conservative, but if you re-tokenize the reassembled text with the model's own tokenizer you turn 'above zero' into a real number you can bill against, no vendor frame required. The 0/1/2 exit codes are the nice part for wiring this into CI.
The abort correlation is the part I under-weighted, and it changes the tiering. A missing usage frame is really two failure modes collapsed into one, with opposite handling. Connection closed by the client after the last content delta is a benign drop, and the proxy is the only place that can call it benign honestly, because the disconnect is something the proxy observes at the socket rather than something the stream reports. That is the distinction that makes proxy-side attribution safe: the cancel is ground truth the proxy holds rather than a claim in the payload. Post-hoc from the logs you cannot separate "user navigated away" from "the call ran and the accounting was suppressed," so both have to stay BLIND. In front of the stream you can attribute estimated cost on the observed cancel and keep the alarm only for a frame that goes missing on a connection that actually completed.
The one thing I would guard is letting the abort path turn into the laundering channel. Once "aborted" downgrades from alarm to estimate, a stream that wants to hide real usage has a reason to look aborted. That stays safe only while the abort signal is the proxy's own socket observation and never a field the upstream can set.
Re-tokenizing the reassembled text to get a real number is the right upgrade to the floor, with one caveat I would wire in. Your local count and the provider's billed count will not match exactly, because cached-prompt tokens, tool tokens, and tokenizer-version drift never show up in the visible text. So I would use the re-tokenized number two ways at once. As the floor it kills the zero-token BLIND case with a real quantity instead of "above zero." And when the provider frame does arrive, the delta between re-tokenized-delivered and billed becomes its own signal: a stable gap is accounting you can model, a sudden one-directional gap is the thing worth blocking on. The exit codes stay the same, you have just given the 1 a number to defend.
The re-tokenized-delivered vs billed delta is the sharp part, and the abort-can't-be-an-upstream-field rule is exactly the seam, keep it a socket fact. One thing I'd wire into the delta signal before it can block: the gap you can model is only stable while the provider's tokenizer version is. Tokenizer drift, tool-token accounting changes, cache-token rules, all move that gap one direction with no bad actor involved, and a silent provider tokenizer bump looks identical to suppression, a sudden one-directional widening. The disambiguator is population, not threshold. A tokenizer or accounting change moves the gap for every stream at once, a fleet-wide step. Real suppression moves it for one account while the fleet holds. So the block condition isn't 'delta widened past X,' it's 'this stream's delta diverged from the fleet's,' per-stream anomaly against a rolling baseline of everyone else. That keeps the re-tokenized floor as the hard number for the zero-token BLIND case and turns the delta into a relative signal that survives the provider changing the rules under you.
Population over threshold is right, and it fixes a real failure. The place I would not let it stand alone is that a fleet baseline is blind to exactly the thing that moves the whole fleet. A provider tokenizer bump and a provider that quietly stops emitting usage frames for everyone look identical from inside the population: a step every stream takes together. Relative anomaly detection reads that as the new normal, re-baselines, and goes quiet at the moment the accounting actually broke. So a fleet-wide step should not silently re-calibrate. It should freeze the baseline and raise its own class, RECALIBRATE rather than BLOCK: no single stream is guilty, the measurement contract changed, and a pinned changelog or a human has to say which. Suppression that is broad enough looks like a version bump, and nothing inside the population separates them.
The second hole is the slow one. A rolling baseline adapts, so any suppression that grows slower than the adaptation window never diverges from the fleet, it just walks the fleet with it. Boiling frog. That argues for two anchors instead of one: the rolling baseline for sharp per-stream divergence, which catches a single account being squeezed, and a frozen epoch, a pinned tokenizer version plus a baseline snapshot taken at a known-good date, for absolute drift, which catches everyone being squeezed slowly. The rolling signal survives the provider changing the rules. The frozen anchor is the only thing that tells you the rules changed at all.
And where there is no fleet, the single-account indie stream, the population can be manufactured cheaply. A canary request with deterministic input, replayed on a schedule, is a population of one whose expected delta you actually know. Canary moves and production moves: the provider moved. Production moves and the canary holds: that one is yours. To be straight about status, this is a design argument and not something I have measured. The re-tokenized floor I have run. The fleet-versus-canary discrimination I have not.