Seven Open-Source LLM Ops Platforms, One Table: Pick by the Row You Can't Ship Without

Nobody wins this table. Seven self-hostable LLM ops platforms, eleven rows, and every column has at least two cells it would rather you didn't read.

An LLM ops platform is the layer between your application and the model provider, or beside it, that records every call, versions the prompts, runs evaluations, and tracks spend. You install one when a production app has more prompts, models and bills than you can follow by reading logs.

A table with no overall winner is still useful, because each row has a clear one. If you need a spend cap or a cached response, three of the seven can do it and four can't, by architecture. If you need a human labeling queue, the list flips: three have one, four don't. Pick the row you can't ship without, read that row across, and the shortlist is usually two or three names long before you look at anything else.

Disclosure: we build AcruxCore, one of the seven. Columns are alphabetical, every row AcruxCore fails is in the table, and each claim below links to the other project's own documentation so you can check it without taking our word.

The table

We self-hosted all seven and built the same prompt on each: a support triage agent that changes its instructions for VIP customers and lists their open tickets. Same customer message, same downstream model. The rows are the questions that came up while doing that, plus every capability at least three competitors have and AcruxCore doesn't.

Same table as text

#		AcruxCore	Helicone	Langfuse	Laminar	MLflow	Opik	Phoenix
1	License	Apache 2.0	Apache 2.0	MIT, some parts paid-only	Apache 2.0	Apache 2.0	Apache 2.0	Elastic 2.0
2	Self-host	✅ 1 command	✅ 1 command	✅ 1 command	✅ 1 command	✅ 1 command	✅ 1 command	✅ 1 command
3	Gateway in the request path	✅	✅	❌ ingest-only	❌ ingest-only	✅ + guardrails	❌ ingest-only	❌ ingest-only
4	Versioned, executed tool catalog	✅	❌	⚠️ schema only	⚠️ schema only	⚠️ MCP servers	❌	❌
5	{% if %} / {% for %} in prompts	✅	❌ substitution	❌ substitution	❌ no registry	✅ full Jinja2	⚠️ SDK only	❌ substitution
6	Audit log without paying	✅	❌	❌ Enterprise	❌	❌	❌	❌
7	Built-in guardrails (PII / safety)	❌	✅	⚠️ SDK hook	✅ PII only	✅	✅	❌ 3rd-party
8	Alerts to Slack or webhooks	❌ email only	✅	✅	✅	✅ spend only	✅	❌ paid AX only
9	Human labeling queue	❌	❌	✅	✅	❌ paid host only	✅	⚠️ no queue
10	Organization → project hierarchy	❌ single team	⚠️ org only	✅	✅ workspace	❌	❌	❌
11	GitHub stars	new project	6.1k	34.3k	3.2k	27.8k	21.8k	11.3k

Checked August to September 2026. Star counts as of 7 September 2026.

The rest of this post walks the rows in the order they tend to decide a choice: where the platform sits, what your prompt can do, who runs your tools, what "open source" means for each, what each one leaves for you to build, and what sitting in the request path costs in milliseconds.

Row 3 decides more than any other: record or decision point

Two different shapes hide under one category name.

Beside your request path. Your code calls the provider directly. An SDK wraps that call and ships a trace afterwards. Langfuse, Laminar, Opik and Phoenix work this way.

In your request path. Your code calls the platform, and the platform calls the provider. Helicone, MLflow's AI Gateway and AcruxCore work this way.

A trace ingested afterwards is a record. A request passing through a gateway is a decision point. Only a decision point can swap in a cheaper model, return a cached response instead of paying for a new one, refuse a key that has passed its monthly budget, or hand your team a virtual key that never exposes the real provider key.

You can't add those four later from beside the path. By the time an ingest-only tool sees the call, the money is spent and the response exists. That's architecture, not roadmap, and it's why one cell in row 3 predicts most of what a platform can and can't do about cost.

The trade runs the other way too. A tool beside your path works with any provider, any SDK and any framework, with nothing new in front of production traffic that can fail. Laminar commits fully to that shape and auto-instruments fifteen-plus agent frameworks from one line of setup. If you already enforce budgets in your own code, or you run a separate proxy, the gateway row may not matter to you at all.

Row 5: two platforms let a prompt branch, five make you do it in code

Prompt templating looked like a formality when we built the grid. It wasn't.

Our test prompt needs a branch and a loop. VIP customers get one instruction, everyone else gets another, and the ticket list has unknown length. On MLflow and AcruxCore that logic lives in the stored template, and the server renders it:

You are a support triage agent for {{ company }}.
{% if is_vip %}
This customer is VIP. Prioritize them and skip standard hold times.
{% else %}
Standard support flow applies.
{% endif %}
{% for ticket in tickets %}
- #{{ ticket.id }}: {{ ticket.title }}
{% endfor %}

The other five do flat {{variable}} substitution, so the branch can't live in the stored prompt. It moves into your code. We wrote that flattening five times, once per platform. This is the shape it takes:

# Same prompt, on a platform that only substitutes variables.
# The stored template can't branch or loop, so this runs before every call.
def build_system_prompt(variables):
    lines = [f"You are a support triage agent for {variables['company']}."]
    if variables["is_vip"]:
        lines.append("This customer is VIP. Prioritize them and skip standard hold times.")
    else:
        lines.append("Standard support flow applies.")
    for ticket in variables["tickets"]:
        lines.append(f"- #{ticket['id']}: {ticket['title']}")
    return "\n".join(lines)

Both produce identical text. What changes is where the prompt's behaviour is versioned. In the first, changing how VIPs are treated is a new prompt version, reviewable and revertable by whoever owns the prompt. In the second it's a code deploy, and the platform's version history shows a string that never changed.

The partial cells need a word each. Langfuse documents storing a Jinja2 template and rendering it yourself with an external library, which keeps the version history but takes the playground out of the loop. Opik has a Jinja2 prompt type in its SDK, and rendering it in the playground is an open feature request. Laminar has no prompt registry at all: its playground is one mutable row of messages, which is deliberate, because the product is about agent runs rather than the calls inside them.

MLflow's prompt registry is the one competitor that matches AcruxCore here, with full Jinja2 plus a version diff and @production style aliases.

Row 4: six store the tool's schema, one runs it

Every tool you give a model has two halves. The definition is the JSON schema the model reads to decide whether to call it. The implementation is the code that runs when it does. Each platform has to decide who owns which.

Six of the seven own only the first half, or neither. Langfuse and Laminar keep a schema in a field beside a prompt, and nothing executes it. MLflow catalogs whole MCP servers, where MCP is the Model Context Protocol, a standard way to expose tools to a model. The unit there is a server, not a tool, so there's no per-tool version history. Helicone, Opik and Phoenix have nothing in this row, which fits their shape: your tool call arrives as one more span in the trace.

AcruxCore is the only one where a tool is a versioned record the gateway itself calls and measures. That gives the tool the same version history and rollback your prompt gets. It also has a cost worth naming: the tool's endpoint and its transform then live on the platform instead of in your repo. For the other six, the implementation stays in your code, which is where plenty of teams want it.

Rows 1 and 6: three different things called open source

Five of the seven are plain Apache 2.0: AcruxCore, Helicone, Laminar, MLflow and Opik. Install it, change it, run it commercially.

Langfuse is MIT with an exception. Its LICENSE covers the repo except an ee/ folder under a separate commercial license, unlocked with an enterprise license key. The audit log lives in that folder. That's why the free self-hosted install doesn't have one, and why row 6 has exactly one ✅ across seven platforms.

Phoenix is Elastic License 2.0. That's source-available: you can read it and self-host it, but it isn't OSI-approved, isn't permissive, and you can't offer it as a managed service. If your legal review has a "must be OSI-approved" line, Phoenix fails it and the other six pass.

Two more of the seven are open-source front ends to paid products, and that's the easiest thing in this table to get wrong. MLflow's human labeling sessions need Databricks' hosted MLflow. Phoenix's alerting lives in the paid Arize AX product. A feature in the docs isn't always a feature in your install. Check which host the page you're reading describes.

Rows 7 to 10: what each platform leaves for you to build

These four rows exist because a comparison drawn only from our own feature list would show AcruxCore winning nearly everything, and that list is our feature set. So the grid also carries every capability at least three of the six competitors have and we don't. AcruxCore loses all four.

Guardrails. Opik ships topic and PII guardrails configurable per project. Helicone runs Prompt Guard and Llama Guard behind a request header. MLflow attaches PII and safety guardrails per gateway endpoint. Laminar has a project-level PII redaction toggle. Langfuse offers a masking hook in the SDK, with server-side masking behind Enterprise. Phoenix traces a third-party setup rather than shipping its own. AcruxCore inspects nothing in a call's content.

Alerts. Helicone wires a threshold to a Slack channel or an email address. Langfuse's Monitors alert on cost, quality or latency to Slack, webhooks or GitHub Actions. Opik has Slack, PagerDuty and webhook destinations. Laminar's Signals can page Slack. MLflow's budget webhooks cover spend only. AcruxCore's only channel is email.

Human labeling queue. Langfuse, Opik and Laminar each have a queue with reviewer assignment and a score schema. Phoenix has annotations but no queue in the open-source build. MLflow's sessions need the paid host. AcruxCore and Helicone have nothing here.

Hierarchy. Langfuse has a real organization-above-project structure with a role at each level. Laminar has workspace above project with three roles. Helicone has an organization tier with no projects beneath it. AcruxCore, MLflow, Opik and Phoenix are each one flat team, and in self-hosted MLflow, Opik and Phoenix there is no login screen at all.

Put together, alphabetically:

Platform	What it leaves for you to build
AcruxCore	Guardrails, non-email alerts, a labeling queue, and an organization layer above the team
Helicone	Prompt logic, a tool catalog, a labeling queue, and projects under the organization
Langfuse	Prompt logic, a request-path gateway, and an audit log outside the ee/ license
Laminar	Any prompt registry, a request-path gateway, and guardrails beyond PII
MLflow	Alerts beyond spend, labeling outside Databricks, and any hierarchy or auth in the OSS build
Opik	A request-path gateway, prompt logic in the UI, and a tool catalog
Phoenix	A gateway, prompt logic, guardrails, a queue for its annotations, alerting outside Arize AX, and any team concept

What sitting in the request path costs

We timed each platform against its own direct-to-provider baseline, in its own session, on different days. So read down a row, not across. These are seven separate measurements, not one benchmark.

Platform	Where it sits	Overhead vs. a direct provider call
AcruxCore	in the path	+4 to +63 ms across six sessions
Helicone	in the path	−15 to +3 ms, forwarding only, nothing logged
Langfuse	beside the path	−22 ms, confidence interval crosses zero
Laminar	beside the path	−8 to +18 ms
MLflow	in the path	+135 to +225 ms, interval never crosses zero
Opik	beside the path	+2 to +22 ms
Phoenix	beside the path	−14 to +21 ms

Two things fall out.

Platforms beside the path measure at roughly zero, as they should. Their confidence intervals cross zero in most sessions because the trace ships on a background task after the response already went back to the caller. A negative number there is sampling noise, not evidence that wrapping a client makes the provider faster.

Among the three in the path, the cost differs by more than an order of magnitude. MLflow's gateway was the only leg whose interval never crossed zero: it added a tenth of a second or more in every run. Helicone's figure isn't like-for-like, because that leg forwarded the call without logging it. Sitting in the request path has a price, and the size of the price comes from the implementation, not the architecture. Measure it on your own network before trusting any published number, including ours.

Who owns what you're installing

Worth ten minutes before you commit a year of tooling to one of these.

• Helicone joined Mintlify on 3 March 2026 and is in maintenance mode: security fixes, new models and bug fixes, with no stated commitment to open source or self-hosting going forward.
• Langfuse was acquired by ClickHouse on 16 January 2026, with an explicit commitment in the announcement that it stays fully open source.
• MLflow is Databricks', Phoenix is Arize's and Opik is Comet's. Three open-source projects attached to commercial products, which is exactly what puts some features on a paid host.
• Laminar is a Y Combinator S24 company and the smallest of the six with a public star count.
• AcruxCore is the newest and has no community to show yet. That's a real column in the table.

How to pick

There's no overall winner because these rows aren't the same size for any two teams. Choosing from scratch:

• You need routing, caching, spend caps or virtual keys. You need a gateway, so the shortlist is Helicone, MLflow or AcruxCore before you look at anything else. Weigh Helicone's maintenance mode and MLflow's measured overhead against that.
• Your prompt's behaviour has to be versioned outside your code. MLflow or AcruxCore, the only two that render conditionals and loops server-side.
• Legal needs an OSI-approved license. Drop Phoenix. If you also need a free audit log, drop Langfuse.
• You're instrumenting agents built on existing frameworks. Laminar's one-line integrations are the shortest path, and having no prompt registry may not bother you.
• You need guardrails or Slack alerts in the box. Opik, Helicone or Langfuse cover both. AcruxCore covers neither today.
• You need human labeling to build eval datasets. Langfuse, Opik or Laminar.

Each platform has its own write-up, built by running that same prompt on both sides: Langfuse, Phoenix, Opik, Helicone, MLflow and Laminar. The matrix with a source link on every cell is at acruxcore.com/compare.

LangSmith isn't in the table because it's closed source. It does offer self-hosting, so that's the reason, not "hosted only".

The row that's hardest to call is the third one. Is a gateway in your request path worth one more thing that can fail in front of production traffic, or are you better off beside the path and giving up routing and spend caps entirely? Both answers hold up, and the latency table doesn't settle it. If you've run any of these seven in production, which row turned out to matter more than you expected when you chose?