DEV Community

Ali Suleyman TOPUZ
Ali Suleyman TOPUZ

Posted on Originally published at topuzas.medium.com on

The Real Cost Curve of Running Agents in Production, From Four Different Companies

I almost skipped writing this one up as its own piece, because on the surface it looked like four unrelated news items that happened to land in the same week. A big rideshare company posted about keeping AI spend flat. A streaming company open sourced a plugin. A search company published a compression cookbook. A Chinese lab dropped pricing on a new model tier. My first instinct was to treat them as four separate case studies and move on, the way I’d bookmark four blog posts and forget three of them by Friday.

Then I went back and actually read all four closely instead of skimming the headlines, and noticed they’re not four separate stories. They’re the same story, told from four different points in a pipeline. One company routes requests between models based on what the task actually needs. One company enforces that routing with hard rules instead of hoping the agent behaves. One company strips the tokens out of what the model receives before routing even enters the picture. And one company makes the case that the cheap tier in that routing decision is a lot more capable than “cheap” usually implies. Put together, they’re not four case studies, they’re four layers of the same optimization stack, and a small team can adopt them roughly in that order.

I run a small operation, not a company with Uber’s request volume, so I want to be upfront that I can’t replicate their numbers at their scale. What I can do is pull the real, checkable numbers out of each story, be honest about where the sourcing gets thin, and then build the two pieces of infrastructure a small .NET team would actually need to try any of this themselves: a cost-tracking middleware that sits around your HTTP client and tells you the truth about every call, and a router function that picks a model per task instead of sending everything through the same expensive one out of habit.

Here’s the four-way summary before I get into each one individually.

CASE STUDY SUMMARY, SAME WEEK, FOUR COMPANIES
Company Technique Reported savings
--------------------------------------------------------------------------------
Uber Live model routing (Pareto-optimal Total AI spend held
                     picks) + defaulting subagents to roughly flat while
                     cheaper models + code-mode batching weekly agent requests
                     of multiple tool calls into scripts grew 9.4x (Feb-Aug
                                                                2026); cost per
                                                                session down 52%
                                                                from its June peak
Spotify (Portal) Hook-enforced two-model routing: ~90% mean token
                     cheap model (Gemini 2.5 Flash in reduction on bulk-
                     their examples) handles bulk file read tasks, measured
                     reads and boilerplate, Claude is across 4 scenarios
                     reserved for reasoning and editing on a Java monorepo
Google (Headroom) Context-compression layer that 21-57% reduction in
                     strips redundant tokens out of tool prompt tokens across
                     outputs, logs, and search results four workload types,
                     before they reach the model, kept reversible via a
                     reversible via a retrieval tool retrieve tool
DeepSeek A cheaper model tier benchmarked Matches or slightly
(V4.1-Flash) against a frontier model, specifically beats a frontier
                     on tool-using tasks rather than model on a tool-use
                     raw reasoning benchmark at roughly
                                                                  1/30th the per-
                                                                  token input price
Enter fullscreen mode Exit fullscreen mode

Uber: routing that moves, not routing that’s configured once

The number that made me stop and actually read Uber’s post twice was the ratio between two growth curves. Weekly active employees using their internal agent tooling grew 7x between February and August 2026. Weekly agent requests grew 9.4x over the same stretch, a faster curve than the headcount curve, meaning each person wasn’t just adopting the tools, they were leaning on them harder over time. And total spend, across that entire period, “relatively stabilized.” Not shrank, but held roughly flat while the thing generating the bill grew almost tenfold.

That’s the headline, but the mechanism underneath it is more interesting than the headline, and it’s not one trick, it’s three stacked together. The first is what they describe as benchmark-driven routing rather than a fixed default model. They run their agent harness against whatever’s currently Pareto-optimal across frontier and open-weight models on their own real workloads, and they move the default when a better tradeoff shows up, rather than picking a model once at launch and leaving it there for a year while pricing and capability both shift underneath that choice.

The second, and what they call the most impactful lever, is subagent delegation. Their setup splits a session into a primary model handling task decomposition and subagents handling the well-defined, narrower pieces of work that decomposition produces. The subagents default to weaker, more cost-effective models, and as the proportion of sessions that spin up subagents has grown, that default has done more of the total cost-control work than anything else in the stack. This is the same shape as the subagent-isolation pattern I tested for a different piece a few weeks back, but pointed at cost instead of context size, and it only pays off because the subagent’s model tier is deliberately cheaper, not because isolation is free on its own.

The third is what they call code-mode batching: instead of a model making five sequential tool calls, each one a full round trip through the model, it writes one script that makes all five calls and returns a single combined result. For a wide SELECT * query against a 50-row table, this cut token consumption by something close to 100 percent compared to the standard MCP tool-use pattern, and even trivial queries saw more than 50 percent savings. The mechanism is simple: five separate tool-use turns each resend the growing transcript, while one script executing five calls internally only pays the transcript cost once.

Put those three together and you get the unit numbers: cost per 1,000 model requests down 34 percent from its peak, cost per session down 52 percent from its June peak, both improving even as raw volume kept climbing. None of the three levers is exotic. Routing to whatever benchmarks best this month, defaulting subagents cheap, and batching tool calls into scripts are all things a small team can do without Uber’s internal tooling. The router function later in this piece is a stripped-down version of lever one and two combined.

Spotify: making the routing rule impossible to skip

Portal, Spotify’s open-sourced plugin, tackles the same idea from a different angle: instead of routing being a suggestion the agent can ignore when it’s in a hurry, it’s a rule the tooling enforces mechanically. The plugin ships as part of a broader system called AiKA Modes, ephemeral, declarative sub-agents that run more like a Lambda function than a persistent process, and two of those modes are the ones doing the cost work: bulk-reader, which handles large file analysis and returns a structured summary instead of the raw file, and code-writer, which generates predictable boilerplate, tests, configs, type stubs, matching whatever pattern already exists in the repo.

What makes this different from just writing a system prompt that says “use the cheap model for boilerplate” is the enforcement layer. Portal uses a PreToolUse hook that intercepts file reads before they happen and blocks anything over a configurable line-count threshold (350 lines by default), redirecting the read through bulk-reader instead of letting the primary, more expensive model ingest the whole file directly. That's a hard rule sitting below the model's own judgment, not a suggestion sitting above it. Below that hook sit bash wrapper scripts for the actual delegation calls, and below those, markdown documentation that guides the agent's behavior when the hooks do fire. The ordering matters: the hook is what actually saves money, the scripts and docs are what make the hook usable, and if you strip the hook out and keep only the guidance, my read of their own writeup is that the savings mostly evaporate, because nothing stops the agent from reading the big file directly when reading it directly feels like the path of least resistance mid-task.

The reported number is a mean 90 percent token reduction on bulk-read tasks, measured across four scenarios against a real Java monorepo, comparing a Claude session reading files directly against the same session delegating those reads to the cheap-model summarizer. That’s a real, specific measurement against a real codebase, not a synthetic benchmark, which is part of why I trust it more than a vaguer “up to 90%” claim would deserve.

What I want to give equal weight to is what they say it can’t do, because this is the part most vendor writeups skip and Spotify’s didn’t. Editing is off the table for the cheap tier, because a worker model’s summary doesn’t reliably preserve line numbers, and an edit against the wrong line is worse than no edit. Reasoning and debugging stay with the expensive model, because the cheap tier does shallow pattern matching that misses the subtle failure mode, the one that’s actually why you’re debugging in the first place. And large generations are a bad fit purely on latency grounds, each delegation costs 10 to 30 seconds and is capped at 30 seconds total, so a task small enough to finish in five seconds on the primary model becomes slower, not just cheaper, if you route it through the cheap tier anyway. Routing isn’t a free win you apply everywhere; it’s a win specifically shaped like “large, low-judgment, tolerant of a short delay,” and outside that shape it costs you speed for no savings.

Google’s Headroom: the layer underneath routing

Where Uber and Spotify both route the request to a different model, Headroom, the open-source context-compression layer associated with Google’s ecosystem, changes what’s inside the request in the first place, regardless of which model eventually reads it. It sits between your tools and your model, intercepts whatever those tools return, tool outputs, database results, file reads, RAG retrievals, and runs each one through a compressor matched to its content type: statistical summarization for JSON arrays that preserves anomalies and errors rather than repeating every normal row, log compression that keeps failures and drops the thousand lines of passing test noise, and similar type-specific handling for source code, diffs, and search results.

The mechanism it uses to stay safe, in their own framing, is “Compress-Cache-Retrieve.” Nothing is actually discarded, the original stays available in a cache, and the model gets a headroom_retrieve tool it can call if the compressed version turns out to be missing something it needed. That's the part of the design I find genuinely clever: most compression techniques are a one-way door, once you've thrown away detail you can't get it back mid-task, but this keeps a way back in if the model hits a wall.

The reported numbers, across four real workload types, come out to a 21 percent reduction on code search (17,199 tokens down to 13,597), 57 percent on SRE incident debugging (55,957 down to 24,340), 42 percent on codebase exploration (58,801 down to 33,895), and 30 percent on GitHub issue triage (46,067 down to 32,429). The spread across those four numbers is itself informative, the documentation notes savings depend on how repetitive the tool output is, and an SRE incident with pages of repeated log noise compresses far better than a search result set that’s already fairly dense.

Here’s the caveat I have to be honest about, because the source material I was handed for this piece claimed the accuracy story was settled: “no measured loss in task accuracy.” When I went and read Headroom’s own documentation directly, I couldn’t find that claim substantiated anywhere. What’s actually there is a description of the retrieval mechanism and the framing that it delivers “the same answers, a fraction of the tokens,” but no benchmark table, no before-and-after task success rate, nothing empirical backing that framing up. That’s not the same as evidence that accuracy does drop, I found nothing pointing that direction either, but it’s also not the same as a verified claim that it doesn’t. If you adopt this, the retrieval-fallback mechanism is the right safety net in principle, and I’d still want to run your own task-success comparison before and after turning compression on, rather than taking “same answers” as a settled fact just because it appears on the product page.

DeepSeek V4.1-Flash: what “a fraction of the cost” actually buys you

The fourth story is different in kind from the other three: it’s not a technique, it’s an argument that the cheap tier you’d route to is underrated. DeepSeek V4.1-Flash prices at $0.15 per million input tokens on a cache miss, $0.003 per million on a cache hit, and $0.60 per million output tokens off-peak (peak hours, 01:00 to 04:00 and 06:00 to 10:00 UTC on weekdays, run double those rates). Set next to Claude Opus 5’s $5 input / $25 output per million, that’s roughly a 33x gap on input and a 42x gap on output, before you even get to the cache-hit rate, which is another 50x cheaper again on top of that.

The benchmark comparison is where I want to slow down, because the honest version of this story is more interesting than the flattened “cheaper and just as good” version that usually gets repeated. On tool-using tasks specifically, HLE-with-tools, V4.1-Flash scores 63.9, which actually edges out Claude Opus 5’s 63.6. On Terminal-Bench 2.1 it hits 90.6, on DeepSWE 74.2, both strong agentic-coding numbers for a model priced two orders of magnitude below the frontier tier. But strip the tools away and test raw reasoning, plain Humanity’s Last Exam with no tool access, and the gap reappears: V4.1-Flash scores 36.8 against Opus 5’s 56.3 and GPT-5.6 Sol’s 44.5, about 65 percent of Opus 5’s score on that axis.

That split is the actual finding, and it’s a more useful one than a single blended number would be. This model isn’t “almost as good as the frontier tier” in general, it’s specifically as good, or slightly better, at the exact category of task that dominates real agent workloads, calling tools, reading their output, and acting on it, while trailing meaningfully on cold reasoning with no tools in hand. If your workload is mostly agentic (read a file, run a test, check the result, fix the thing) rather than mostly abstract reasoning, that’s precisely the axis where the cheap tier holds up, and it’s worth testing against your own tasks rather than assuming either the marketing framing or my summary of it transfers cleanly to what you’re building.

Building the instrumentation: middleware around the HTTP client

All four of those stories share a precondition I want to make concrete instead of leaving abstract: none of them would have a number to report without call-level instrumentation sitting underneath the technique. You can’t know your routing saved money if you don’t know what the unrouted baseline cost. So before the router function, here’s the piece that actually has to exist first, a DelegatingHandler that wraps whatever HttpClient your app uses to call an LLM API, and logs every call's model, token counts, cache status, and running per-session cost without your application code having to think about any of it.

// CostTrackingHandler.cs
// .NET 8, no external packages required.
using System.Text.Json;
using System.Text;
using System.Linq;
using System.Collections.Concurrent;
namespace AgentCostAudit;
public sealed record ModelPricing(
    decimal InputPerMillion,
    decimal OutputPerMillion,
    decimal CacheWritePerMillion,
    decimal CacheReadPerMillion);
public static class PricingTable
{
    // Published per-million-token USD rates at time of writing. Check your
    // provider's current pricing page before trusting these past this week.
    public static readonly Dictionary<string, ModelPricing> Models = new()
    {
        ["claude-opus-5"] = new(5.00m, 25.00m, 6.25m, 0.50m),
        ["claude-sonnet-5"] = new(3.00m, 15.00m, 3.75m, 0.30m),
        ["deepseek-v4.1-flash"] = new(0.15m, 0.60m, 0.15m, 0.003m),
    };
}
public sealed record CallLogEntry(
    string SessionId,
    string Model,
    string TaskType,
    int InputTokens,
    int OutputTokens,
    int CacheWriteTokens,
    int CacheReadTokens,
    bool CacheHit,
    decimal CallCostUsd,
    decimal SessionRunningTotalUsd,
    DateTimeOffset Timestamp);
public sealed class CostTrackingHandler : DelegatingHandler
{
    private readonly string _logPath;
    private readonly ConcurrentDictionary<string, decimal> _sessionTotals = new();
    private static readonly object FileLock = new();
    public CostTrackingHandler(HttpMessageHandler innerHandler, string logPath)
        : base(innerHandler)
    {
        _logPath = logPath;
    }
    protected override async Task<HttpResponseMessage> SendAsync(
        HttpRequestMessage request, CancellationToken cancellationToken)
    {
        // Session and task type ride in as headers set by the caller, so this
        // handler stays stateless per call rather than needing an ambient context.
        string sessionId = request.Headers.TryGetValues("X-Session-Id", out var sid)
            ? sid.First() : "unknown";
        string taskType = request.Headers.TryGetValues("X-Task-Type", out var tt)
            ? tt.First() : "unclassified";
        string? requestBody = request.Content is null
            ? null
            : await request.Content.ReadAsStringAsync(cancellationToken);
        string model = ExtractModel(requestBody) ?? "unknown";
        var response = await base.SendAsync(request, cancellationToken);
        // Buffer the body so we can inspect it and still hand a readable
        // response back to whatever called this handler.
        string responseBody = await response.Content.ReadAsStringAsync(cancellationToken);
        response.Content = new StringContent(responseBody, Encoding.UTF8, "application/json");
        var (inTok, outTok, cacheWrite, cacheRead) = ExtractUsage(responseBody);
        var pricing = PricingTable.Models.GetValueOrDefault(model)
            ?? PricingTable.Models["claude-sonnet-5"];
        decimal cost =
            inTok / 1_000_000m * pricing.InputPerMillion +
            cacheWrite / 1_000_000m * pricing.CacheWritePerMillion +
            cacheRead / 1_000_000m * pricing.CacheReadPerMillion +
            outTok / 1_000_000m * pricing.OutputPerMillion;
        decimal runningTotal = _sessionTotals.AddOrUpdate(sessionId, cost, (_, existing) => existing + cost);
        var entry = new CallLogEntry(sessionId, model, taskType, inTok, outTok,
            cacheWrite, cacheRead, cacheRead > 0, cost, runningTotal, DateTimeOffset.UtcNow);
        AppendLogLine(entry);
        Console.WriteLine(
            $"[{taskType}] model={model} in={inTok} out={outTok} cacheRead={cacheRead} " +
            $"cacheWrite={cacheWrite} cost=${cost:0.0000} sessionTotal=${runningTotal:0.00}");
        return response;
    }
    private void AppendLogLine(CallLogEntry entry)
    {
        string line = JsonSerializer.Serialize(entry);
        lock (FileLock)
        {
            File.AppendAllText(_logPath, line + Environment.NewLine);
        }
    }
    private static string? ExtractModel(string? requestBody)
    {
        if (requestBody is null) return null;
        using var doc = JsonDocument.Parse(requestBody);
        return doc.RootElement.TryGetProperty("model", out var m) ? m.GetString() : null;
    }
    private static (int input, int output, int cacheWrite, int cacheRead) ExtractUsage(string responseBody)
    {
        using var doc = JsonDocument.Parse(responseBody);
        if (!doc.RootElement.TryGetProperty("usage", out var usage))
            return (0, 0, 0, 0);
        int input = usage.TryGetProperty("input_tokens", out var i) ? i.GetInt32() : 0;
        int output = usage.TryGetProperty("output_tokens", out var o) ? o.GetInt32() : 0;
        int cacheWrite = usage.TryGetProperty("cache_creation_input_tokens", out var cw) ? cw.GetInt32() : 0;
        int cacheRead = usage.TryGetProperty("cache_read_input_tokens", out var cr) ? cr.GetInt32() : 0;
        return (input, output, cacheWrite, cacheRead);
    }
}
Enter fullscreen mode Exit fullscreen mode

Wiring it into a real client is three lines, and the important part is that nothing downstream of HttpClient has to know the tracker exists:

// Wiring it up needs one extra using for JsonContent.Create:
// using System.Net.Http.Json;
var handler = new CostTrackingHandler(new HttpClientHandler(), "agent-cost-log.jsonl");
var http = new HttpClient(handler) { BaseAddress = new Uri("https://api.anthropic.com/") };
http.DefaultRequestHeaders.Add("x-api-key", Environment.GetEnvironmentVariable("ANTHROPIC_API_KEY"));
http.DefaultRequestHeaders.Add("anthropic-version", "2023-06-01");
var request = new HttpRequestMessage(HttpMethod.Post, "v1/messages")
{
    Content = JsonContent.Create(new
    {
        model = "claude-sonnet-5",
        max_tokens = 500,
        messages = new[] { new { role = "user", content = "Summarize this incident report." } }
    })
};
request.Headers.Add("X-Session-Id", "session-2026-09-13-01");
request.Headers.Add("X-Task-Type", "summarize");
var response = await http.SendAsync(request);
Enter fullscreen mode Exit fullscreen mode

And querying it at the end of a session is a plain LINQ pass over the JSON-lines file, no database needed:

public static class SessionSummary
{
    public static void PrintSummary(string logPath)
    {
        var entries = File.ReadLines(logPath)
            .Select(line => JsonSerializer.Deserialize<CallLogEntry>(line)!)
            .ToList();
        foreach (var session in entries.GroupBy(e => e.SessionId))
        {
            decimal total = session.Sum(e => e.CallCostUsd);
            int calls = session.Count();
            double cacheHitRate = session.Count(e => e.CacheHit) / (double)calls;
            Console.WriteLine($"Session {session.Key}: {calls} calls, ${total:0.00} total, {cacheHitRate:P0} cache hits");
            foreach (var byTask in session.GroupBy(e => e.TaskType))
                Console.WriteLine($" {byTask.Key}: ${byTask.Sum(e => e.CallCostUsd):0.0000} across {byTask.Count()} calls");
        }
    }
}
Enter fullscreen mode Exit fullscreen mode

I got this wrong on my first pass, worth admitting: I initially had the handler subtract cache_creation_input_tokens and cache_read_input_tokens from input_tokens before pricing the remainder, assuming input_tokens counted the whole prompt including cached portions. It doesn't. Anthropic's input_tokens field already excludes anything that went through the cache, it's only the genuinely fresh, uncached tokens. Subtracting the cache fields a second time silently zeroed out real cost. The version above prices the three buckets independently, which is the correct read of the usage object.

A router function, and the baseline it has to beat

With the tracker in place, the second piece is the one Uber’s post and Spotify’s plugin both lean on: something that picks a model per task instead of sending everything through whichever model you reach for by habit. I want to be explicit that what follows is a starting-point heuristic, not a finished classifier, in the same spirit as the “explicitly labeled as a starting point” caveat in the brief for this piece. It looks at input length and a short keyword list, nothing more sophisticated, and it’s meant to be replaced with something trained on your own task history once you have a few weeks of the log above to train it on.

// TaskRouter.cs
using System.Linq;
namespace AgentCostAudit;
public enum TaskComplexity { Trivial, Simple, Complex }
public sealed record RouteDecision(string Model, string Endpoint, TaskComplexity Complexity, string Reason);
public static class TaskRouter
{
    // Crude on purpose. Replace with a classifier trained on your own logged
    // task types before trusting this on anything that matters.
    private static readonly string[] ComplexSignals =
        { "refactor", "design", "architecture", "debug", "why does", "root cause", "trade-off" };
    private static readonly string[] TrivialSignals =
        { "summarize", "list", "extract", "format", "convert", "rename" };
    public static RouteDecision Route(string taskDescription, int estimatedInputTokens)
    {
        string lower = taskDescription.ToLowerInvariant();
        if (estimatedInputTokens > 20_000 || ComplexSignals.Any(lower.Contains))
        {
            return new RouteDecision(
                "claude-sonnet-5",
                "https://api.anthropic.com/v1/messages",
                TaskComplexity.Complex,
                "long context or a keyword suggesting multi-step judgment");
        }
        if (estimatedInputTokens < 3_000 && TrivialSignals.Any(lower.Contains))
        {
            // Local, self-hosted alternative for the trivial tier: no API key,
            // no per-token bill, just `ollama pull llama3.1:8b` first.
            return new RouteDecision(
                "llama3.1:8b",
                "http://localhost:11434/api/chat",
                TaskComplexity.Trivial,
                "short input matching a trivial-task keyword, routed to local Ollama");
        }
        return new RouteDecision(
            "deepseek-v4.1-flash",
            "https://api.deepseek.com/v1/chat/completions",
            TaskComplexity.Simple,
            "no strong signal either way, defaulting to the cheap hosted tier");
    }
}
Enter fullscreen mode Exit fullscreen mode

And the piece that actually tells you whether routing was worth building: a comparison against a naive baseline that sends every task through a single model, run over the same task list.

// RoutingComparison.cs
using System.Linq;
namespace AgentCostAudit;
public static class RoutingComparison
{
    public static void Compare(IEnumerable<(string Task, int EstInputTokens, int ActualOutputTokens)> tasks)
    {
        decimal naiveTotal = 0m, routedTotal = 0m;
        var naivePricing = PricingTable.Models["claude-sonnet-5"];
        foreach (var (task, estIn, actualOut) in tasks)
        {
            decimal naiveCost =
                estIn / 1_000_000m * naivePricing.InputPerMillion +
                actualOut / 1_000_000m * naivePricing.OutputPerMillion;
            naiveTotal += naiveCost;
            var decision = TaskRouter.Route(task, estIn);
            bool selfHosted = decision.Model == "llama3.1:8b";
            var routedPricing = PricingTable.Models.GetValueOrDefault(decision.Model, naivePricing);
            decimal routedCost = selfHosted
                ? 0m
                : estIn / 1_000_000m * routedPricing.InputPerMillion +
                  actualOut / 1_000_000m * routedPricing.OutputPerMillion;
            routedTotal += routedCost;
            Console.WriteLine(
                $"[{decision.Complexity}] -> {decision.Model} (${routedCost:0.0000}) " +
                $"vs sonnet baseline (${naiveCost:0.0000}), reason: {decision.Reason}");
        }
        Console.WriteLine($"\nNaive single-model total: ${naiveTotal:0.00}");
        Console.WriteLine($"Routed total: ${routedTotal:0.00}");
        if (naiveTotal > 0)
            Console.WriteLine($"Savings vs baseline: {(1 - routedTotal / naiveTotal):P0}");
    }
}
Enter fullscreen mode Exit fullscreen mode

The router in this article is deliberately not smart. What it gives you isn’t a good routing policy, it’s a place to plug your own task history into a comparison against a fixed baseline, so the savings number you get out the other end is measured against your real workload instead of borrowed from someone else’s blog post.

The adoption order, for a small team with limited time

If I were advising a small .NET team on which of these four techniques to try first, effort against expected savings, here’s the order I’d actually work through, and it doesn’t match the order I introduced them in above.

Start with instrumentation, before any of the four techniques, because none of the other numbers mean anything without it. The handler above is an afternoon of work, and every subsequent decision on this list depends on being able to measure it.

Second, try benchmarking a cheaper model against your own real tasks, the DeepSeek story. This is the lowest-effort of the four actual techniques, it’s a config change and a comparison run, not new infrastructure, and the router’s naive-versus-routed comparison above is enough to tell you in an afternoon whether a cheaper tier holds up on your workload the way it held up on HLE-with-tools.

Third, cheap-model delegation for I/O-heavy work, Spotify’s pattern. This needs slightly more than a config change, a hook or middleware layer that actually enforces the rule, because the Portal writeup’s own experience suggests guidance alone doesn’t hold under time pressure. But it’s bounded work, and the 90 percent number on bulk reads is specific enough to chase.

Fourth, live routing across your whole request stream, Uber’s pattern. This is the most valuable technique at their scale and the most infrastructure-heavy at any scale, it needs the benchmark-driven comparison running continuously rather than once, plus the subagent delegation logic to actually route work rather than just measure it. Worth building once the first three have already paid for themselves and you have real data on which of your task types are cheap-model-safe.

Fifth, and last, context compression, Headroom’s pattern. Not because it’s less valuable, the 21 to 57 percent range is real, but because it’s the one place in this list where I found the accuracy claim genuinely unverified rather than just under-explained, and a compression layer sitting in front of every model call is exactly the kind of change where I’d want my own before-and-after task-success numbers in hand before I trusted it broadly, not just the token count going down.

However you order it, the check at the end of each step is the same one, and it’s the actual point of all four company stories put together: compare against your own naive baseline, on your own tasks, with your own instrumentation, rather than assuming a savings number that held for someone else’s traffic pattern holds for yours. Uber’s 52 percent, Spotify’s 90 percent, Headroom’s 21 to 57 percent, and DeepSeek’s roughly-a-third-the-cost are all real, specific, individually checkable numbers. None of them is a promise about what happens when you run the same technique against a workload that isn’t theirs.

Tags: ai-agents, llm-cost-optimization, dotnet, model-routing, prompt-caching, agentic-ai, finops

Top comments (0)