DEV Community

BuildOpsy
BuildOpsy

Posted on Originally published at buildopsy.com

The three meters behind a $65 million observability bill

$65 million is a strange number to hear on an earnings call. It gets stranger when you work out what it buys at list price.

On 4 May 2023, Datadog reported first-quarter billings of $511 million, up 15% year over year, and flagged one exception: a large upfront bill for a single customer in Q1 2022 that did not recur at the same level or timing. A JPMorgan analyst, Mark Murphy, did the subtraction on the call and arrived at roughly $65 million. Datadog's CFO, David Obstler, confirmed the shape without the name: a crypto company, an early optimizer, now billed in smaller chunks.

A week later, Gergely Orosz reported that current and former Coinbase engineers had confirmed to him that the $65 million was Coinbase's Datadog bill for 2021, settled in the first quarter of 2022. Coinbase has never confirmed it publicly. Datadog never named the customer. Everything below rests on that sourcing.

The number is worth taking seriously rather than gawking at, because it is not really a story about one company overspending. It is a story about three meters that compound independently, and what happens when nobody owns any of them.

Datadog observability bill waterfall: host fleet, log ingest, and indexed events combine into the monthly meter, with Coinbase's $65M 2021 bill as the scale check

Three meters, three units, three growth curves

Observability vendors do not sell one meter. At Datadog, the ones that stack are these, at the annual-billing list rates published today:

Meter List price What grows it
Infrastructure, Enterprise $23 per host per month Fleet size
Log ingest $0.10 per GB ingested Verbosity × volume
Standard indexing, 15-day retention $1.70 per million events Event volume × retention
APM $31 per host per month, with infrastructure attached Services × hosts

On-demand rates are higher across the board: $27 per host, $2.55 per million indexed events, $36 per APM host.

The compounding is the point. Hosts grow with the fleet. Ingested gigabytes grow with log verbosity, and retention pressure multiplies whatever you ship. Indexed events grow with cardinality, and custom metrics beyond the per-host allotment — 100 on Pro, 200 on Enterprise — bill by usage, so every new tag value on a custom metric mints billable series. A team that adds hosts, turns up log verbosity, and tags everything has tripled its bill three independent ways before writing a single alert.

What $65 million means in each unit

At $23 per host per month, $65 million is about 235,000 host-years of Enterprise infrastructure monitoring. That is arithmetic on list price, not a claim about what Coinbase bought, but it is the right order of magnitude for a company running a large fleet at the peak of a growth phase.

The small-scale version is more useful. Twenty hosts at $23 comes to $460 a month. A thousand gigabytes of ingest at $0.10 adds $100. That is $560 a month, or $6,720 a year — the default output of a cost calculator I built for exactly this, and a modeled figure rather than anyone's real invoice.

Divide one into the other: $65 million is roughly 9,700 years of that model.

The gap between $6,720 and $65,000,000 is not a pricing scandal. It is what four orders of magnitude of fleet growth, log verbosity, and cardinality look like when no individual decision was ever expensive enough to argue about.

How 2021 produced that number

Coinbase went public in 2021 and had a year most companies never see. Infrastructure cost was not a constraint anyone enforced, because revenue covered everything. AWS, Snowflake, and Datadog all grew. A bill that size is what three meters running at startup speed produce when nobody is assigned to watch them.

Then the market turned. Revenue fell hard through 2022, and the bills that growth had hidden became the agenda. Datadog's CEO described the pattern on the same call: a customer whose business got cut to a third or a quarter gets its contract restructured, because the vendor would rather be part of the solution than part of the problem. The customer stayed, on chunked billing matched to a smaller business.

One thing to be careful about: an anonymous commenter claimed the $65 million covered a multi-year prepayment. Orosz could not verify that and neither can anyone else, so it stays out. What is documented is the single-year bill, the settlement quarter, and the renegotiation.

The build-vs-buy line, stated properly

Coinbase's response was to price the alternative: a dedicated observability team with a mandate to move onto Grafana, Prometheus, and ClickHouse, built for control and ownership first, with cost as the second motive once the market turned. They double-wrote the new stack for months before cutting anything over.

The arithmetic that mattered is a single inequality:

infrastructure cost + platform team cost < vendor bill
Enter fullscreen mode Exit fullscreen mode

Orosz's version: ten senior and staff engineers in the Bay Area cost under $5 million a year, with infrastructure on top in the low double-digit millions. Against $65 million, building is not a close call. Against a well-managed $2 million bill, it is a way to hire a team whose job is maintaining dashboards.

His rule of thumb for where the question becomes real is $2–5 million a year in vendor spend — roughly where a team of four or five engineers plus a manager, at $150–400K fully loaded each, starts to pencil against the invoice.

The breakeven moves with the invoice. Which is why the invoice has to be measured before the decision is made, not after someone sees a headline and asks for a migration plan.

They didn't leave

The part the invoice-watchers usually miss: with the replacement stack validated and the plug ready to pull, Coinbase got terms it could accept and stayed. The customer is still on Datadog.

That is not a contradiction. Deep integrations — custom tracing, per-service instrumentation, dashboards wired into every team's workflow — make migration expensive in a way the license math never captures. That lock-in cut both ways: it gave Coinbase leverage for a steep discount, and it gave Datadog a retained logo.

The lesson is uncomfortable for both sides of the table. Price your exit before you threaten it, or the threat is theater.

Four things to do on Monday

Budget tag cardinality like headcount. Every new tag value on a custom metric mints series, and series beyond the per-host allotment bill by usage. Audit which teams emit the most series per host before approving more hosts.

Tier retention by access pattern. Logs you investigate regularly earn standard indexing with alerting. Everything else belongs in a cheaper tier or an archive with rehydration on demand. Retention is the quietest multiplier on the invoice.

Decide sampling as a cost input. Full-fidelity tracing on every request is a second invoice hiding inside the first. Pick the sampling rate when you design the tracing, not when you are debugging.

Alert on ingest slope, not the total. Gigabytes per day times price per gigabyte, tracked against the monthly commit, turns the bill into a metric the on-call actually owns. A meter nobody watches is a meter without a limit.

And the cheapest one of all: assign the bill to the teams that generate it. Chargeback turns "observability is expensive" into named owners with named levers. Coinbase assigned a whole team to the problem. Most companies only need to assign the invoice.

What this doesn't establish

The identification rests on Orosz's reporting, not on a Coinbase or Datadog statement. Neither company has confirmed the customer. The $65 million covers one year of spend, not a multi-year contract that anyone has verified.

Coinbase's current spend is undisclosed, so there is no way to check how much the renegotiation actually saved, or whether the in-house stack would have been cheaper over five years.

The list prices are Datadog's published rates as of October 2026 and will not match any negotiated contract, including the one in this story. And no vendor published utilisation figures, so the three-meter model here describes the structure of the bill, not its actual composition.

Three clocks ran in this story: the meters compounded through 2021, the invoice came due in Q1 2022, and the renegotiation reset the economics after that. Treating any one of them as "the Datadog incident" misses how metered billing actually fails — slowly, across independent units, and only discovered when growth stops covering it.

Sources

Top comments (0)