DEV Community: Lara Mateo

Nobody audits their OpenAI invoice

Lara Mateo — Fri, 14 Aug 2026 01:56:01 +0000

Every team running LLMs in production has two numbers for last month's spend: the one in their tracking tool, and the one on the invoice the provider sent. They almost never match, and the reasons are documented: providers report cached tokens differently, the community pricing data is an estimate by its own admission, the tools themselves treat deltas under roughly 10% as normal, and the calls you never instrumented don't show up in your numbers at all.

I've spent the past few weeks reading up on this, and the surprise was where the evidence lives. It's mostly in the docs of the tools doing the estimating, written against their own interest.

Why the numbers diverge

OpenAI and Anthropic report cached tokens differently

OpenAI folds cache reads into the reported input token count. Anthropic reports cache fields separately from regular input. Both choices are documented and both are reasonable, but any tool that computes cost through a single path across both providers will be wrong on one of them. This isn't theoretical: Langfuse issue #12306, opened February 2026, Anthropic cache tokens double-counted, roughly 2x inflated cost. LiteLLM has its own family of cache accounting issues.

The pricing registries call themselves estimates

Pydantic maintains genai-prices, a community pricing registry of the kind most estimation tools depend on. Its README has a warning section literally titled "these prices will not be 100% accurate". Providers don't publish exact pricing for the direct OpenAI, Anthropic and Gemini APIs in any reliably machine-readable format, which is the whole reason the registry has to exist. (Azure, AWS and GCP do publish machine-readable SKU prices for their hosted variants, so this is about the direct APIs.)

A 10% gap is treated as normal

The closest thing to a published drift number I found is a troubleshooting band. LiteLLM's cost-discrepancy doc says deltas under roughly 10% between tracked spend and the bill are commonly explained by rounding and boundary effects, and bigger ones usually mean something got miscounted, dropped or categorized differently. "Usually". That's a heuristic in a troubleshooting doc, not a measurement, and it's still the best public number there is.

Untracked calls still get billed

Everything above assumes the call went through your tracking in the first place. The one-off script, the teammate's experiment, the key that leaked. Your tracking shows you the calls it saw. The invoice shows everything.

What can you use to reconcile LLM costs?

I'm building a product in this space, so discount everything that follows, and check it against the vendors' own pages.

Option	What it gives you	The limit
Trackers (Langfuse, Helicone, LiteLLM)	Traces, evals, per-request cost estimates	Cost is estimated from tokens, not checked against the bill
Gateways (OpenRouter, Vercel AI Gateway)	Authoritative cost for routed traffic	It's the biller's own receipt, and only for traffic through it
Spend and cloud cost platforms	The provider's number, organized and forecasted	One source, nothing to check it against
Enterprise audit services	Recovered overcharges, at enterprise scale	A project, not a product
DIY with the provider cost APIs	A real reconciliation, once	Breaks down at per-key, per-feature granularity, every month
Kenda (mine)	Both numbers and the delta, labeled by evidence	Early, I'm building it

If you're on Langfuse, Helicone or LiteLLM: they're great at what they're actually for, and to their credit, their own docs are where I learned most of what's in this post. Checking the invoice was never their job.

If your traffic goes through a gateway, the gateway's figure is authoritative for that traffic, because the gateway is the one charging you. That's also the limit: a receipt is not an audit, and calls that don't go through it don't exist to it.

The spend management and cloud cost platforms mostly read the provider's billing APIs, so they give you the provider's number back, organized. Useful, but you can't check a number against itself. The few products that hold both a client-side measurement and the billing data don't, as far as I can find, show you the difference between the two as a first-class view. If you know one that does, tell me in the comments, I'd like to be wrong about this.

And the spreadsheet genuinely works at small scale. OpenAI exposes an organization costs endpoint, Anthropic a cost report. Pull both, export your tracked spend, join by hand. If the numbers match within noise, you're done. The pain shows up when you want the answer per key, per feature, per customer, across billing-cycle boundaries instead of calendar months, every month.

What is Kenda?

Kenda is an AI spend reconciliation tool: it compares the LLM costs your own code logged against the dollars the provider actually billed, and shows the difference, line by line. On one side, the events your code produced, priced at provider rates. On the other, the spend the provider itself asserts. Both numbers side by side, with the delta, per billing period. That's what I'm building at kenda.app.

The design decision I care most about: every figure says how far the evidence goes. Reconciled means the provider asserted dollars at that scope. Priced means the provider asserted the tokens and Kenda applied the rates. Allocated means only Kenda's own events exist there. Whatever can't be attributed shows up as its own visible line instead of getting smeared across the rest. If a cost tool never shows you a residual, it's hiding one.

It's boring infrastructure. Reconciliation always is. I spent six years reconciling money nobody could account for, in healthcare billing and fintech, and this is the same discipline pointed at AI spend.

FAQ

How far off is tracked LLM spend from the real invoice?
Nobody has published a measured distribution. The closest public reference is LiteLLM's troubleshooting doc, which treats deltas under roughly 10% as commonly explained by rounding and boundary effects. If you've measured your own gap, that number is more valuable than anything in this post: leave it in the comments.

How do I check my OpenAI or Anthropic bill programmatically?
OpenAI exposes an organization costs endpoint that breaks spend down by project, API key and line item. Anthropic exposes a cost report at workspace level. Pull those, export what your tracking recorded for the same billing period, and compare. The billing period matters: provider cycles don't always align with calendar months.

Why is my OpenAI bill higher than my tracking shows?
The usual suspects, in order: calls that never went through your tracking (scripts, experiments, leaked keys), cached tokens counted differently than your tool assumes, and stale pricing data. Whatever is left after those is the residual, and the residual is the part worth investigating, not smearing across the rest.

If you've ever compared your tracked LLM spend against the actual invoice, I want two data points: how far off it was, and whether you could explain the difference. Rough numbers are fine. "We never checked" is a genuinely useful answer, and I expect it to be the most common one.

SSE vs WebSocket vs WebTransport: How to Choose in 2026

Lara Mateo — Thu, 25 Jun 2026 06:40:18 +0000

SSE, WebSocket, or WebTransport? A practical 2026 guide to picking a real-time web protocol, with a Next.js example streaming LLM tokens over Server-Sent Events.

A few weeks ago I shipped Pagewise, a tool that takes any public URL and streams back an AI summary token by token. The streaming is the whole experience. Watching the summary type itself out feels alive in a way that a spinner followed by a wall of text never does.

To build that, I had to answer a question that sounds simple and isn't: how do you get the server to push data to the browser in real time?

There are three real answers in 2026: Server-Sent Events (SSE), WebSocket, and WebTransport. They are not interchangeable, and picking the wrong one means either fighting your infrastructure or building machinery you never needed. Here is what each one is for, where it fits, and why I landed on SSE.

(In my first post on this topic I called WebTransport a "fingers crossed" bet on the future. That changed in March. More on that below.)

The short answer: Use SSE when only the server needs to push data, such as streaming AI responses, live dashboards, and notifications. Use WebSocket when the client and server both send messages constantly, such as chat, multiplayer, and collaborative editing. Use WebTransport, supported in every major browser since March 2026, when you need unreliable low-latency datagrams or QUIC's resilience on shaky networks, and always pair it with a WebSocket fallback.

Start with one question: who's doing the talking?

Most real-time decisions come down to direction.

If only the server pushes and the client just listens, you want the simplest thing that streams. That is SSE.
If both sides send messages constantly, you want a two-way channel. That is WebSocket.
If you need two-way plus unreliable, low-latency data (or you are on flaky mobile networks), that is where WebTransport comes in.

Worth saying plainly: a smaller share of features truly need bidirectional communication than most of us assume. "Real-time" gets used as a synonym for "WebSocket", but a lot of real-time problems (notifications, dashboards, progress bars, and yes, LLM output) only flow one way. Naming the direction first saves you from over-building.

What is SSE (Server-Sent Events)?

Server-Sent Events (SSE) is a one-way channel that lets a server push a stream of text updates to the browser over a single, long-lived HTTP connection. It is the boring-in-a-good-way option, running over plain HTTP. The client opens a normal GET request, the server responds with Content-Type: text/event-stream and keeps the connection open, pushing chunks as they are ready. No upgrade, no special port. Any proxy, load balancer, or CDN that already understands HTTP understands SSE.

On the browser, the entire client side is the EventSource API:

const events = new EventSource("/api/summarize");

events.addEventListener("token", (e) => {
  appendToSummary(JSON.parse(e.data).text);
});

The wire format is just UTF-8 text. Each message is a small block of lines ending in a blank line:

event: token
data: {"text": "Hello"}
id: 42

Four fields carry the weight: data (the payload), event (the event name), id (the last-seen event ID), and retry (the reconnect delay in milliseconds).

What I like about it:

Reconnection is free. If the connection drops, the browser reconnects on its own (after about three seconds by default) and sends a Last-Event-ID header with the last ID it saw, so the server can resume exactly where it left off. With raw WebSocket, all of that is your job.
It is about as simple as real-time gets. No handshake, no protocol of your own, no sticky sessions.
It works with the boring infrastructure you already have.

What to watch out for:

Text only. SSE carries UTF-8. You can send binary by base64-encoding it, but that adds roughly 33% to the payload, so it is the wrong tool for binary throughput.
EventSource is GET-only and cannot set custom headers, so no Authorization: Bearer. You work around it with cookies, or you skip EventSource and read the stream yourself with fetch() and a ReadableStream reader (which is what most AI SDKs do, trading the free reconnection for the ability to send headers).
On HTTP/1.1, browsers cap you at six connections per domain, and every open stream eats one slot, which gets rough across multiple tabs. HTTP/2 multiplexes everything over one connection and the limit disappears. On a modern host you are almost certainly fine.
The classic deployment bug: a proxy buffering your stream so tokens arrive in one dump instead of one at a time. If you see that, disable buffering (for Nginx, the X-Accel-Buffering: no header).

What is WebSocket, and how is it different from SSE?

WebSocket is a protocol that holds a single TCP connection open as a full-duplex channel, so the client and server can both send messages at any time. When the client also needs to push constantly, this is the answer. It starts as an HTTP request with an Upgrade header, the server replies 101 Switching Protocols, and from that point the connection is full-duplex: both sides send whenever they want, with no request and response pairing. It speaks ws:// and wss:// (the TLS version, the one you should use), and it carries binary natively, no base64 tax.

The cost is everything SSE gives you for free. WebSocket has no built-in reconnection and no resume. You implement reconnection (usually exponential backoff), you track sequence numbers if you need to replay missed messages, and you run your own heartbeats (ping and pong frames) to notice dead connections. All of that is doable, it is just work you are signing up for.

Reach for WebSocket when the interaction is two-way and high-frequency: chat, multiplayer games, collaborative editing (think live cursors in a shared doc), and trading terminals. If you find yourself adding WebSocket and then only ever sending from the server, that is a sign SSE was the better fit.

What is WebTransport, and is it ready in 2026?

WebTransport is a browser API for low-latency, two-way communication over HTTP/3 and QUIC, supporting both reliable streams and unreliable datagrams. It is the newest of the three, and the one whose status just changed. Because QUIC sits on UDP, it gives you things the TCP-based protocols cannot:

Both reliable, ordered streams (like WebSocket) and unreliable datagrams (like UDP), so you choose per message. A player position update that is already stale by the time it arrives should be dropped, not retransmitted.
No head-of-line blocking. On TCP, one lost packet stalls everything queued behind it. QUIC multiplexes independent streams, so a loss on one does not freeze the others.
Connection migration. Switch from Wi-Fi to cellular mid-session and the connection survives, because QUIC identifies it by a connection ID rather than the IP and port pairing that WebSocket depends on.

The 2026 update: WebTransport reached Baseline browser support when Safari 26.4 shipped it on 24 March 2026. Chrome, Edge, Firefox, and Opera had it for a while; Safari was the holdout that kept it out of cross-browser production, and that is now resolved. Apple positioned it plainly as a modern alternative to WebSocket for low-latency use cases like multiplayer and live collaboration.

It is not a free lunch yet, though:

The specs are still drafts at both the W3C and the IETF, so breaking changes can land, and server libraries sometimes lag the browser.
It rides UDP on port 443, which plenty of corporate and public networks block. If the handshake fails there, you need a WebSocket fallback. In practice you ship WebTransport with a safety net, not on its own.
The server tooling is younger than the WebSocket ecosystem.

Track it, prototype with it if you need datagrams or connection migration, and keep WebSocket as the dependable default for now.

SSE vs WebSocket vs WebTransport: a comparison table

	SSE	WebSocket	WebTransport
Direction	Server to client	Two-way	Two-way
Transport	Plain HTTP	TCP (after upgrade)	HTTP/3 and QUIC (UDP)
Data	Text (base64 for binary)	Text and binary	Binary streams and datagrams
Reconnect	Automatic, with resume	Build it yourself	Build it yourself
Infrastructure	Simplest	Needs upgrade-aware proxies	Needs UDP/QUIC and a fallback
Browser support	Universal since 2011	Universal	Baseline since March 2026
Best for	LLM streaming, dashboards, notifications	Chat, multiplayer, collaboration	Low-latency media, gaming, telemetry

Rules of thumb I use:

Only the server pushes, the client listens: SSE.
Both sides push, a lot: WebSocket.
You need unreliable datagrams, stream multiplexing without head-of-line blocking, or survival across a network change: WebTransport, with a WebSocket fallback.

Why I chose SSE for Pagewise (streaming LLM tokens)

Run Pagewise through that filter and the answer is immediate. The server streams a summary; the client just renders it. The data is text (markdown tokens). It needs to survive a flaky connection without losing its place. And it runs on serverless.

That is the SSE column, top to bottom.

There is a second reason that sealed it: the entire LLM ecosystem already streams this way. OpenAI and Anthropic both deliver tokens as text/event-stream (OpenAI sends data: lines ending in a data: [DONE] marker; Anthropic sends named events like content_block_delta). The Vercel AI SDK standardized on SSE for its streaming protocol as of version 5. Choosing SSE meant swimming with the current instead of against it.

How to implement SSE in a Next.js route handler

Here is the shape of it in Pagewise, a Next.js route handler that streams Claude's output as typed events (trimmed here to the streaming essentials):

// app/api/summarize/route.ts
export async function POST(req: Request) {
  const encoder = new TextEncoder();

  const stream = new ReadableStream({
    start(controller) {
      const send = (event: string, data: unknown) =>
        controller.enqueue(
          encoder.encode(`event: ${event}\ndata: ${JSON.stringify(data)}\n\n`)
        );

      // Kick the work off in the background so chunks flush as they arrive,
      // instead of buffering the whole response until the handler returns.
      (async () => {
        try {
          send("metadata", { title, faviconUrl });

          for await (const token of streamFromClaude(url)) {
            send("token", { text: token });
          }

          send("done", { sessionId, wordCount });
        } catch {
          send("error", { code: "LLM_FAILURE", message: "Generation failed" });
        } finally {
          controller.close();
        }
      })();
    },
  });

  return new Response(stream, {
    headers: {
      "Content-Type": "text/event-stream",
      "Cache-Control": "no-cache",
      "Connection": "keep-alive",
    },
  });
}

Two things in there are the difference between "it streams" and "it dumps everything at the end":

It is a route handler, not a server action. Server actions are built for mutations and cannot stream an HTTP response incrementally, so streaming has to live in a route.
The producing loop runs in the background inside start(). If you await the whole generation before returning the Response, the framework buffers it, and your user gets the spinner-then-wall-of-text experience you were trying to avoid.

The typed events (metadata, token, done, and error) map cleanly onto the client, where each one gets its own handler. And because this is serverless, the one number you cannot ignore is the function's maximum duration. A long generation can outlast it, so set maxDuration deliberately, and if your responses can run very long, reach for resumable streams rather than hoping the function stays alive.

Frequently asked questions

Is SSE better than WebSocket?

Neither is better in the abstract; they solve different problems. SSE is the simpler choice when only the server pushes data: notifications, dashboards, and LLM token streaming. WebSocket is the right tool when the client and server both send messages continuously, like chat or multiplayer.

Does SSE work with HTTP/2?

Yes, and it is better there. On HTTP/1.1 a browser allows only six connections per domain, and each open SSE stream uses one. HTTP/2 multiplexes many streams over a single connection, so that limit effectively disappears. Most modern hosts serve HTTP/2 by default.

Can SSE send binary data?

Not directly. An SSE stream is UTF-8 text. You can base64-encode binary to send it, but that inflates the payload by about a third, so for real binary throughput WebSocket or WebTransport is the better fit.

What does the Vercel AI SDK use for streaming?

Server-Sent Events. As of AI SDK 5, Vercel standardized its server-to-client streaming on SSE, which is also how the OpenAI and Anthropic APIs deliver tokens. If you are streaming LLM output, you are almost certainly using SSE under the hood.

Is WebTransport production-ready in 2026?

It is now usable across every major browser: Safari shipped it in version 26.4 on 24 March 2026, joining Chrome, Edge, Firefox, and Opera. The specifications are still drafts, and some networks block its UDP traffic, so ship it with a WebSocket fallback.

Why does my SSE stream arrive all at once instead of streaming?

Usually a proxy or framework is buffering the response. Disable buffering at the proxy (for Nginx, set X-Accel-Buffering: no), and make sure your server flushes each event instead of awaiting the whole response before it returns.

The short version

Real-time on the web is not one decision, it is three, and the first question does most of the work: who needs to push data, and in which direction? Answer that clearly and the protocol usually picks itself.

For anything that looks like streaming an AI response, a live feed, a dashboard, or a progress indicator, SSE is very likely the right call, and it is a lot less work than its reputation suggests. WebSocket earns its keep the moment the client becomes a real sender. And WebTransport, as of this year, is finally a production option rather than a someday one.

I'm curious: has anyone here moved off WebSocket to SSE for an AI feature, or are you already running WebTransport in production? Tell me in the comments.