Self-hosted LLM observability: six stacks, weighed by what they cost you to run

TL;DR: If your LLM traffic sends personal data, or your obs invoice scales with token volume, self-hosting the trace pipeline is worth costing out. Storage is the cheap part (roughly 60 GB and about 18 dollars a month for a million spans a day at 30-day retention). The expensive part is that you now run it: retention, cardinality, ingest lag, and the pager. Here are six open-source stacks you can run yourself, what each is actually good at, and what each costs you to operate.

Why self-host at all

Two reasons show up in real incidents. The first is data residency: if a trace payload contains a prompt with a customer's PII, sending it to someone else's cloud is a compliance conversation you do not want to have after the fact. Self-hosting keeps the payload inside your VPC. The second is cost shape. Managed LLM observability tends to price on events or token volume, so the invoice grows exactly when your product succeeds. I inherited a managed bill that had quietly tripled over two quarters because traffic tripled. We moved the high-volume traces in-house. The invoice went down. The number of dashboards I now own went up.

That is the trade. You are swapping a predictable invoice for operational surface area. Before you make it, cost out the operational side honestly, because "open-source" is a license, not a free lunch. The question is never "is it free," it is "what does it take to keep it green at 3x the traffic."

What "self-hostable" has to mean here

I only included stacks you can actually run on your own infrastructure without a sales call, that speak a standard trace format (most of these are built on OpenTelemetry), and that a small team can operate. Every option below is open-source and self-hostable today. I have put them in no particular ranking order, because the right pick depends on whether you want pure tracing, a full platform, or just instrumentation you point at a backend you already run.

The six stacks

Langfuse. The one most teams reach for first. It is open-source and self-hostable, and it combines tracing with evals and prompt management, so it is more than a trace viewer. Strong at the prompt-iteration loop: versioned prompts, scores, and a clean trace UI. What it costs to run: a Postgres plus a ClickHouse-backed deployment for the self-hosted stack, which is a real database to operate at volume. What it can miss at scale: high-cardinality trace attributes will punish your storage if you do not sample. github.com/langfuse/langfuse

Arize Phoenix. Open-source, self-hostable, and built on OpenTelemetry. Like Langfuse it is more than tracing: it pairs trace collection with an evaluation layer, and it is notebook-friendly, which makes it strong for offline analysis and debugging a RAG pipeline span by span. What it costs to run: light to stand up for a single team, heavier once you want durable multi-tenant storage rather than an analysis session. What it can miss at scale: it started life analysis-first, so treat the production, always-on deployment as the part you validate under load. github.com/Arize-ai/phoenix

Helicone. Proxy-first. You route model calls through it and get logging, cost, latency, and caching almost immediately, which is the fastest path to a spend dashboard. Open-source and self-hostable. What it costs to run: low to start, but a proxy in the request path is now a thing on your critical path, so its availability is your availability. What it can miss at scale: proxy-based capture is excellent for cost and latency and less focused on deep multi-span agent traces than the OTel-native tools. github.com/Helicone/helicone

Future AGI. Open-source and self-hostable, and OpenTelemetry-native for tracing through its traceAI framework. It sits at the platform end of this list rather than the pure-tracing end: the same stack also carries evaluation, simulation, and a model gateway, so the draw is running one self-hosted system instead of several. Honest placement: it is younger at pure tracing than Langfuse or Phoenix, so if all you want is the most battle-tested trace viewer, it is not your first pick. What it costs to run: a full platform, so you operate more surface than a single-purpose tracer. What it can miss at scale: validate the tracing path at your volume before you retire the incumbent. github.com/future-agi/future-agi

SigNoz. The general-purpose APM option. It is open-source, self-hostable, and OpenTelemetry-native, and it treats LLM spans as spans inside your broader application traces. Strong if you already want one observability backend for services and models rather than an LLM-only tool. What it costs to run: a ClickHouse-backed APM, which your infra team may already know how to operate, which is a point in its favor. What it can miss at scale: it is not LLM-specialized, so prompt-level ergonomics (diffing prompt versions, judge scores) are not its focus. github.com/SigNoz/signoz

Traceloop OpenLLMetry. Not a backend at all, and that is the point. It is an open-source set of OpenTelemetry instrumentations for LLM apps that you point at whatever OTel-compatible backend you already run. Strong if you have an observability stack and just want your model calls to show up in it without vendor lock-in on the storage side. What it costs to run: almost nothing on its own, because it is instrumentation, but you still need a backend, so its true cost is whatever you send the data to. What it can miss at scale: it gives you spans, not opinions, so the eval and prompt-management layers are on you. github.com/traceloop/openllmetry

Cost out the storage before the on-call

The invoice you escaped was the easy number. Here is the one people skip. This is a rough hot-storage estimate, not a full TCO, but it anchors the conversation.

def trace_storage_gb(spans_per_day, bytes_per_span, retention_days):
    return spans_per_day * bytes_per_span * retention_days / 1e9

def monthly_cost(spans_per_day, bytes_per_span=2_000, retention_days=30,
                 usd_per_gb_month=0.10, replication=3):
    gb = trace_storage_gb(spans_per_day, bytes_per_span, retention_days)
    return gb, gb * usd_per_gb_month * replication

for spd in (1_000_000, 20_000_000):
    gb, cost = monthly_cost(spd)
    print(f"{spd:>12,} spans/day -> {gb:8.1f} GB hot, ~${cost:7.2f}/mo storage (x3 repl)")

   1,000,000 spans/day ->     60.0 GB hot, ~$  18.00/mo storage (x3 repl)
  20,000,000 spans/day ->   1200.0 GB hot, ~$ 360.00/mo storage (x3 repl)

Storage is cheap. Eighteen dollars a month for a million spans a day is not what makes self-hosting expensive. What makes it expensive is the ClickHouse or Postgres you are now running, the retention job you have to get right, the cardinality that blows up when someone puts a UUID in a span attribute, and the fact that when ingest lags, that is your page. Plug your real numbers in. If the storage line is trivial and the operator line is not, you have learned the actual shape of the decision.

What I'd page on

If you self-host any of these, do not page on "the dashboard is down." Page on the things that mean you are silently losing data:

1. Ingest lag over threshold (your traces are minutes behind reality, so every incident you debug is stale).
2. Dropped or refused spans greater than zero (the exporter queue is full, and you are blind to exactly the traffic spike you most need to see).
3. Storage growth rate outrunning your retention budget, or trace attribute cardinality spiking (a UUID landed in a label, and your index is about to hurt).
4. Sampling rate drift (someone changed the sampler, and your tail-latency traces quietly stopped being collected).