DEV Community

Shubham Palriwala
Shubham Palriwala

Posted on Originally published at agnost.ai

Best free and open-source AI agent monitoring tools (2026)

The best open-source tools to monitor AI agents are Langfuse, Opik, and Laminar; Arize Phoenix is free to self-host but source-available, not open source. Langfuse (MIT, except its ee folders) and Opik (Apache-2.0) are the most complete, with tracing, LLM-as-a-judge evals, datasets, and prompts. Laminar (Apache-2.0) suits engineering teams that want tracing plus evals in CI. OpenLLMetry (Apache-2.0) is instrumentation, not a backend. If you'd rather not run servers, the largest free hosted tiers are Langfuse Hobby (50,000 units), Traceloop Free (50,000 spans), Datadog Free (40,000 LLM spans), and Opik Free (25,000 spans).

"Free" means two different things, and buyers often mix them up. Open source means you can read, run, and modify the code under a licence such as MIT or Apache-2.0, usually by self-hosting. A free tier means a vendor hosts it for you at no charge, up to a limit. This guide keeps them apart.

This page is updated for 2026. Since 2025, Helicone has gone into maintenance mode after Mintlify acquired it, Traceloop has announced it is joining ServiceNow (OpenLLMetry stays open source), and Langfuse retired its trace-level evaluators.

Disclosure: We make Agnost AI, which is not open source. It has a free tier, and we list it with the other free tiers below. Facts about every other tool come from that vendor's own docs, repositories, and pricing pages, checked on November 18, 2026. This is a documentation comparison, not a hands-on benchmark.

Open source, self-hostable

Tool Licence What you can self-host What it runs on Main limits
Langfuse MIT, except ee folders Full platform; a few add-ons need a licence key Web and worker containers, Postgres, ClickHouse, Redis or Valkey, S3 Several stateful services to operate
Opik Apache-2.0 Full platform: server, web app, core features Docker Compose locally, Helm on Kubernetes for production Local install isn't production-ready
Laminar Apache-2.0 Full stack via Docker Compose docker compose up, or a fuller compose file Vendor recommends managed or the full compose file for production
Arize Phoenix Elastic License 2.0 (source-available) Full platform, no feature gates Docker, Kubernetes, Helm, one-click cloud templates Not OSI open source
OpenLLMetry Apache-2.0 Nothing: it's instrumentation Your app, exporting to any OTel backend Needs a backend from this list or elsewhere
Helicone Apache-2.0 Gateway and dashboard Docker Compose; Helm via enterprise In maintenance mode since March 2026
W&B Weave Apache-2.0 SDK only Not the trace server — Backend isn't in the open-source repo

Free tiers of hosted tools

Tool Free allowance per month Retention Seats Open source?
Langfuse Hobby 50,000 units (traces, observations, scores) 30 days 2 Yes
Traceloop Free Forever 50,000 spans 24 hours Up to 5 in the table; FAQ says no seat limit OpenLLMetry yes; platform no
Datadog Free 40,000 LLM spans 15 days Not stated No
Opik Free 25,000 spans 60 days Up to 10 Yes
Arize AX Free 25,000 spans and 1 GB 15 days Unlimited No (Phoenix is ELv2)
Braintrust Starter 1 GB and 10,000 scores 14 days Unlimited No
Laminar Free 1 GB 7 days 1 Yes
W&B Weave Free 1 GB of ingestion Not stated Up to 5 model seats SDK only
Helicone Hobby 10,000 requests 7 days 1 Yes (maintenance mode)
LangSmith Developer 5,000 base traces 14 days 1 No
Agnost AI Free 1,000 events 7 days Not stated No

Units differ. A Langfuse unit counts the trace and every observation in it, so one agent run with five spans uses about six units. Datadog counts only calls to a model provider. Agnost counts each conversation turn and each tool call as an event. See how we priced the same workload across tools.

Langfuse

Licence and self-hosting. MIT, except for the ee folders. Langfuse says you can self-host for free with Docker Compose (single VM, no high availability), Kubernetes with Helm (its preferred production setup), or Terraform templates for AWS, Azure, and GCP. Self-hosted instances send basic usage statistics by default; set TELEMETRY_ENABLED=false to turn that off.

What you get. Tracing over OpenTelemetry or its Python and JS/TS SDKs, cost and token tracking, prompt management, a playground, LLM-as-a-judge and code evaluators, datasets with experiments, and dashboards with alerts.

Limits. Production self-hosting means running Postgres, ClickHouse, Redis or Valkey, and blob storage. Organization Creators, the Instance Management API, and UI customization need an enterprise licence key. LLM judges now run on observations or experiments, not whole traces or sessions.

Free hosted option. Hobby: 50,000 units a month, 2 users, 30 days of data.

Opik

Licence and self-hosting. Apache-2.0, and Comet says the full platform, including server, web app, and core capabilities, can be self-hosted without a commercial licence. Use ./opik.sh with Docker Compose to try it locally, and the Helm chart on Kubernetes for production.

What you get. Tracing, cost tracking, LLM-as-a-judge and heuristic metrics, prompt management, an optimizer, and online rules that score spans, traces, or whole threads. Thread rules wait 15 minutes after the last message by default, and alerts go to Slack, PagerDuty, or a webhook. It accepts OpenTelemetry.

Limits. Alerts fire when an average score crosses a threshold, not on individual failures. The local install is for testing, not production.

Free hosted option. 25,000 spans a month with 60-day retention, up to 10 users.

Laminar

Licence and self-hosting. Apache-2.0. Clone the repo and run docker compose up -d for a lightweight full stack, or the docker-compose-full.yml file for production. Laminar's pricing page lists on-premise under Enterprise, for teams that want vendor support.

What you get. OpenTelemetry-native tracing in Rust, auto-tracing for the Vercel AI SDK, OpenAI, Anthropic, and LangChain, an evals SDK and CLI that run in CI, SQL access to traces, dashboards, and Signals, which check each finished trace against a plain-English instruction.

Limits. Signals aren't on the free hosted plan. The free hosted plan has one seat and 7-day retention.

Free hosted option. 1 GB of data a month.

Arize Phoenix (source-available)

Licence and self-hosting. Elastic License 2.0. Arize says self-hosting "is free and fully permitted" and that there are "no feature gates." Deploy with Docker, Kubernetes, Helm, or one-click templates for AWS, Railway, Render, Google Cloud Run, and Azure.

What you get. Tracing built on OpenTelemetry and OpenInference, with auto-instrumentation for LangChain, LlamaIndex, DSPy, Mastra, the Vercel AI SDK, and major providers; LLM, code, and human evals; a prompt playground with span replay; and datasets with experiments.

Limits. ELv2 is a source-available licence, not an OSI-approved open-source one. Self-hosting is allowed, but have your legal team read the licence terms. If you require OSI licences, choose Langfuse, Opik, or Laminar.

Free hosted option. Arize AX Free: 25,000 spans and 1 GB a month, 15-day retention.

OpenLLMetry

Licence. Apache-2.0, maintained by Traceloop.

What it is. OpenTelemetry extensions for LLM apps. Call Traceloop.init() and spans flow to any of 24 tested backends, including the OpenTelemetry Collector, Datadog, Grafana, Honeycomb, Laminar, New Relic, SigNoz, and Sentry.

Why it matters. If you instrument with OpenLLMetry or plain OpenTelemetry, you can change backends later without touching agent code. Pair it with one of the self-hosted tools above.

Status. Traceloop announced in March 2026 that it is joining ServiceNow and said "OpenLLMetry will remain open-source." Traceloop's hosted free plan keeps 50,000 spans a month for 24 hours.

Helicone

Licence and self-hosting. Apache-2.0, self-hosted with Docker Compose.

Status. Mintlify acquired Helicone on March 3, 2026 and said it "will continue operating in maintenance mode," with security updates, bug fixes, and new models, and that it will help customers migrate. Helicone's site now says it is part of Experiential Labs, with details to come. Don't start a new deployment on it.

W&B Weave

Licence. The Python SDK is Apache-2.0. The repository says the trace server "is developed in wandb/core," so you can't self-host the backend from open-source code. Private hosting is an Enterprise option.

Free hosted option. 1 GB of Weave ingestion a month.

Agnost AI (free tier, not open source)

Agnost AI is not open source. We list it because its free tier is a real option for one job the tools above don't focus on.

What it does. It monitors chat agents, voice agents (through transcripts), copilots, and MCP servers in production. It groups traffic by user and session and judges whole conversations, detecting failures, frustration, intents, tool-call errors, loops, and silent wrong answers. Plain-language rules are judged pass or fail on each assistant message once a conversation has been idle 15 minutes, with no sampling. Alerts go to Slack or email, and Auto Improve opens pull requests with fixes tested against past runs.

Free tier. 1,000 events a month with 7-day retention. An event is one conversation turn or one tool call, so this covers early production or a pilot, not a busy agent.

Limits. No open-source code, no free self-hosting (self-hosted VPC is Enterprise), no cost tracking, no offline datasets or experiments, and no CI gating. Many teams run it next to Langfuse or Opik.

How to choose

If you need… Pick
OSI open source, full feature set, permissive licence Opik (Apache-2.0) or Langfuse (MIT, except ee)
Open-source tracing plus evals in CI Laminar
Free self-hosting and ELv2 is acceptable Arize Phoenix
Instrumentation that works with any backend OpenLLMetry or plain OpenTelemetry
No servers, the largest free hosted allowance Langfuse Hobby or Traceloop Free
Free hosted with long retention Opik Free (60 days) or Langfuse Hobby (30 days)
Whole-conversation failure detection, free to try Agnost AI Free, next to an open-source tracer

Before you self-host, cost it honestly. "Free" software still needs servers, upgrades, backups, and someone on call. Our self-hosted agent observability checklist lists what to check, and the pricing guide has a worksheet for comparing self-hosted with managed.

Related reading

Frequently asked questions

What are the best free or open-source tools to monitor AI agents?

For truly open-source tools you can self-host, Langfuse (MIT, except its ee folders) and Opik (Apache-2.0, full platform) are the most complete, and Laminar (Apache-2.0) is a strong engineering-focused option. Arize Phoenix is free to self-host with no feature gates, but its Elastic License 2.0 is source-available rather than OSI open source. OpenLLMetry (Apache-2.0) instruments your agent with OpenTelemetry but is not a backend. For a free hosted tier with no servers to run, Langfuse Hobby, Opik Free, Datadog Free and Traceloop Free have the largest allowances.

Is Langfuse really open source?

Mostly. Langfuse's repository is MIT-licensed except for its ee folders, and Langfuse says you can self-host it for free. A few add-ons, such as the Instance Management API, Organization Creators and UI customization, need an enterprise licence key.

Is Arize Phoenix open source?

Phoenix is distributed under the Elastic License 2.0. Arize says self-hosting is free and fully permitted with no feature gates, but ELv2 is a source-available licence, not an OSI-approved open-source one.

Is Agnost AI open source?

No. Agnost AI is a commercial product. It has a free tier of 1,000 events a month with 7-day retention, and self-hosted VPC deployment is available on the Enterprise plan. If you need open-source code you can run and modify, use Langfuse, Opik or Laminar.

What is the best free tier for monitoring AI agents without self-hosting?

By volume: Langfuse Hobby gives 50,000 units a month with 30-day data access, Traceloop Free 50,000 spans with 24-hour retention, Datadog Free 40,000 LLM spans with 15-day retention, and Opik Free and Arize AX Free 25,000 spans each. Braintrust Starter and Laminar Free give 1 GB of data. Allowances use different units, so compare them against your own traces.

Can I self-host AI agent monitoring for free?

Yes. Langfuse, Opik, Laminar and Arize Phoenix can be self-hosted with no licence fee. You still pay for the servers and the time to run them. Langfuse, for example, needs Postgres, ClickHouse, Redis or Valkey, and S3-compatible storage in production.

Is Helicone still a good open-source option?

Not for new projects. Helicone is Apache-2.0, but Mintlify acquired it on March 3, 2026 and said it will continue in maintenance mode, with security updates, bug fixes and new models, and help customers migrate elsewhere. Helicone's site now says it is part of Experiential Labs, with details to come.

Is W&B Weave open source?

Its Python SDK is Apache-2.0, but the repository says the trace server it talks to is developed in W&B's core platform, not in that repo. You can't self-host the Weave backend from open-source code; private hosting is an Enterprise option. Weave's free hosted plan includes 1 GB of ingestion a month.


Originally published on agnost.ai.

Top comments (0)