The strongest enterprise AI agent monitoring platforms in 2026 are LangSmith, Arize AX, Langfuse Enterprise, Braintrust, Datadog Agent Observability, Splunk Agent Observability (formerly Galileo), Fiddler, and Agnost AI. LangSmith, Arize AX, Langfuse, and Braintrust offer the most complete documented enterprise controls for tracing and evals: SAML SSO, custom roles, audit logs, self-hosted or hybrid deployment, and a SOC 2 Type II report. Datadog and Splunk fit companies that want agent data next to their existing APM. Fiddler fits regulated teams that want model and agent governance in a VPC or on-premise. Agnost AI is the pick for conversation-level quality on customer-facing agents, running next to whichever of the others you use for tracing.
Large companies don't buy agent monitoring on features alone. Security review, procurement, and data residency decide most deals before anyone opens a trace. This guide lists what enterprises need, compares eight platforms on those requirements using each vendor's own security, pricing, and docs pages, and ends with a build-vs-buy section and a two-week pilot plan.
Disclosure: We make Agnost AI. Facts about every other tool come from that vendor's own security, trust, pricing, and docs pages, checked on November 18, 2026. "Not stated" means we could not find it on those pages, not that the vendor lacks it. This is a documentation comparison, not a hands-on benchmark or a compliance opinion. Ask each vendor for its current reports and contract terms.
What enterprises need from agent monitoring
Use this as the checklist for your security review and RFP:
- [ ] SSO/SAML and SCIM. Sign-in through Okta or Entra ID, enforced for everyone, with automatic deprovisioning when people leave.
- [ ] RBAC. Roles per team or project, so a support lead can read conversations for one agent without seeing every agent's traces. Custom roles if your org chart doesn't fit the defaults.
- [ ] Audit logs. A record of who signed in, who read or exported which data, and who changed settings, that you can export to your SIEM.
- [ ] Data residency, self-hosting, or VPC. Where traces live, and whether the vendor can run in your cloud account or region. Check the full data path: model-based judges and integrations may send data out even from a self-hosted install.
- [ ] PII redaction. Masking before data leaves your app, or server-side redaction, plus access rules on traces that contain personal data.
- [ ] Retention. Defaults, paid extensions, deletion on request, and retention per project.
- [ ] SLAs and support. Uptime and response-time commitments, a named support contact, and a severity-1 escalation path.
- [ ] Compliance attestations. A current SOC 2 Type II report, a signed HIPAA BAA if you handle health data, and ISO 27001 if your procurement asks for it. A badge on a website is not a report.
- [ ] Multi-agent and multi-team scale. Many agents, many teams, one place to compare quality across them, with workspaces or projects to keep data apart.
- [ ] Release confidence and model-change regressions. A way to test a prompt or model change against real past conversations before rollout, and to see within a day if quality dropped after it.
- [ ] Build vs buy. An honest estimate of what it costs to run the same controls yourself.
Comparison: access and data controls
| Platform | SSO | RBAC | Audit logs | Self-host / VPC | Data residency | PII redaction | Retention |
|---|---|---|---|---|---|---|---|
| LangSmith | SAML or OIDC, SCIM (Enterprise) | Custom roles and ABAC (Enterprise) | Enterprise, incl. self-hosted | Self-hosted on Kubernetes or hybrid (data plane in your VPC), Enterprise | US or EU cloud; APAC listed on enterprise page | ABAC deny policies; masking before data reaches LangSmith | 14 days base, 180 days extended; custom on Enterprise |
| Arize AX | SAML 2.0 | Account, org, and space roles, custom roles | Sign-ins, operations, exports | Self-hosted in your VPC; air-gapped available (Enterprise) | US, EU, or Canada (Enterprise) | Not stated | Custom (Enterprise) |
| Langfuse | Free when self-hosted; Cloud via $300/mo Teams add-on or Enterprise | Org-level on all tiers; project-level on Teams add-on or Enterprise | Enterprise (cloud and self-hosted) | Free MIT self-host; Enterprise self-host; air-gapped on Government tier | US, EU, Japan; HIPAA US region on Enterprise | Client-side masking; server-side masking on Enterprise self-host | 3 years data access on Pro and Enterprise; retention management |
| Datadog Agent Observability | SAML | Default and custom roles | Audit Trail | No self-hosted option described | US, EU (Germany), UK, Japan, Australia sites; not on US gov sites | Sensitive Data Scanner included | 15-day traces; 30, 60, or 90 days as paid add-ons |
| Braintrust | SAML or OIDC (Enterprise) | Basic roles on Pro; custom roles on Enterprise | Enterprise, incl. data reads | On-prem, or hybrid data plane in your cloud (Enterprise) | Your cloud region via hybrid deployment | SDK masking and ingestion redaction | 14 days Starter, 30 days Pro, custom Enterprise |
| Galileo (Splunk Agent Observability) | OIDC or SAML on-prem; SAML via Splunk Observability Cloud | Org roles and project access; groups on paid on-prem | Not stated | On-premises / self-hosted edition | Not stated | PII detection evaluator (detects, not redacts) | Longer retention at extra cost |
| Fiddler | SSO (from Developer tier) | Yes (from Developer tier) | Not stated | SaaS, VPC, or on-premise (Enterprise) | Not stated | Guardrails detect PII and PHI | Not stated |
| Agnost AI | Not stated | Not stated | Enterprise | Self-hosted VPC (Enterprise) | Via self-hosted VPC | Not stated | Custom (Enterprise) |
Comparison: assurance and commercial terms
| Platform | SOC 2 / HIPAA / ISO 27001 (vendor-stated) | SLA and support | Enterprise pricing model | Best for |
|---|---|---|---|---|
| LangSmith | SOC 2 Type 2, ISO 27001, HIPAA compliant; BAA not stated | Custom SLAs (uptime, latency, response); direct escalation; deployed engineers | Custom, invoiced annually upfront | LangChain and LangGraph teams |
| Arize AX | SOC 2 Type II, HIPAA, PCI DSS 4.0, CSA STAR L1; ISO 27001 on self-hosted page | Custom SLAs; dedicated support | Custom | Managed or air-gapped tracing and evals |
| Langfuse | SOC 2 Type II and ISO 27001 reports; HIPAA BAA available | Uptime and support SLA; 1-hour sev-1 on committed contracts; named support engineer | Cloud Enterprise $2,499/mo; self-hosted Enterprise custom | Open source with enterprise add-ons |
| Datadog Agent Observability | SOC 2 Type 2, HIPAA, ISO 27001, FedRAMP High (company-wide); Agent Observability HIPAA-eligible under BAA | Support plans sold separately; SLA not stated for this product | Per LLM span; Pro from $160/mo annual | Teams already on Datadog APM |
| Braintrust | SOC 2 Type II; BAA on Enterprise; ISO 27001 not stated | Uptime SLA; premium support; shared Slack | Custom | Release workflows built on experiments and scorers |
| Galileo (Splunk Agent Observability) | Not stated | Standard support included; premium available | From $100/mo per 1.2M spans, billed annually | Splunk shops; runtime guardrails |
| Fiddler | SOC 2 Type II, HIPAA; ISO 27001 not stated | Dedicated channels; named CSM; SLA not stated | Developer $0.002/trace; Enterprise contact sales | Regulated model and agent governance |
| Agnost AI | Ask (security review on request) | Custom SLAs and SLOs | Free, Starter $49/mo, Pro $499/mo, Enterprise custom | Conversation-level quality on customer-facing agents |
LangSmith
What it is. LangChain's tracing, evaluation, and monitoring platform. It traces LangChain and LangGraph natively and accepts OpenTelemetry from other stacks.
Enterprise controls. SAML or OIDC SSO with SCIM, custom roles (Enterprise only), and attribute-based access rules that can block specific users from traces containing PII. Audit logs are Enterprise-only and work on self-hosted installs too. You can self-host on Kubernetes or run hybrid, with LangChain's control plane and your data plane in your VPC. LangChain says LangSmith is SOC 2 Type 2 and ISO 27001 certified and HIPAA compliant, with reports in its Trust Center. Its pages don't mention a BAA, so ask.
Best for. Large teams building on LangChain and LangGraph who want one vendor for tracing, evals, and deployment.
Limits. Base traces are kept 14 days, and 180-day extended traces cost extra. Custom roles, audit logs, and self-hosting all require Enterprise, which is invoiced annually upfront.
For more, see LangSmith alternatives.
Arize AX
What it is. Arize's managed platform for tracing, evals, and monitoring. Its open-source sibling, Phoenix, is free to self-host.
Enterprise controls. SAML SSO, roles at account, organization, and space level with custom roles, and audit logs that cover sign-ins, operations, and data exports. Enterprise runs as SaaS in the US, EU, or Canada, or self-hosted in your VPC, with air-gapped deployment for regulated and classified environments. Arize lists SOC 2 Type II, HIPAA, PCI DSS 4.0, and CSA STAR Level 1, and its self-hosted page adds ISO 27001.
Best for. Enterprises that want managed tracing and evals with a choice of US, EU, or Canadian data region, or a fully self-hosted install where data never leaves their perimeter.
Limits. Its pricing page doesn't make clear which security features need Enterprise, and none of the pages we checked describe PII redaction. Ask both in your review.
For more, see Arize alternatives.
Langfuse
What it is. An open-source LLM engineering platform for tracing, prompts, evals, and datasets, built on OpenTelemetry.
Enterprise controls. Self-hosted Langfuse is free under MIT and already includes enterprise SSO and org-level roles. Its Enterprise self-hosted plan adds project-level roles, audit logs, SCIM, server-side data masking, retention management, and a support SLA. On cloud, Enterprise is $2,499 a month and includes SSO, audit logs, SCIM, AWS PrivateLink, and a HIPAA data region in the US. Langfuse provides SOC 2 Type II and ISO 27001 reports and a HIPAA BAA. Committed contracts get a 1-hour severity-1 response around the clock.
Best for. Enterprises that want open source and the option to self-host, with a published price for the managed enterprise tier.
Limits. On cloud, SSO and project-level roles below Enterprise need the $300-a-month Teams add-on, and the HIPAA data region is Enterprise-only. On self-hosted, audit logs, SCIM, and the SOC 2 and ISO reports need the Enterprise plan.
Datadog Agent Observability
What it is. Datadog's LLM Observability, now sold as Agent Observability. It traces agents, runs evaluations, and links agent traces to Datadog APM, infrastructure, and Real User Monitoring.
Enterprise controls. Datadog-wide SAML SSO, default and custom roles, and Audit Trail. Datadog lists SOC 2 Type 2, HIPAA, ISO 27001, and FedRAMP High among many attestations, and Agent Observability is on its list of HIPAA-eligible services for organizations with a Datadog BAA. Sensitive Data Scanner is included at no extra cost. Data lives on one of Datadog's regional sites (US, EU, UK, Japan, or Australia).
Best for. Companies whose services already run on Datadog and want one vendor, one contract, and one trace ID from front end to model call.
Limits. No self-hosted option is described, and Agent Observability is not available on Datadog's US government sites. Traces are kept 15 days unless you pay for 30, 60, or 90 days. Support plans are sold separately, and we found no SLA stated for this product.
Braintrust
What it is. An evaluation and observability platform built around experiments, datasets, and scorers.
Enterprise controls. SAML SSO, custom roles, and audit logs are Enterprise-only. Audit logs record admin actions and, by default, every data read. Enterprise can run on-prem, or hybrid with the data plane in your own cloud account so traces, prompts, and datasets stay in your region. Braintrust is SOC 2 Type II, signs BAAs on Enterprise, and offers an uptime SLA and a shared Slack channel.
Best for. Teams that gate releases on experiments and scorers and want enterprise controls around that workflow.
Limits. SSO, custom roles, and audit logs all require Enterprise. Its pages don't mention ISO 27001.
Galileo (now Splunk Agent Observability)
What it is. Cisco has completed its acquisition of Galileo, and the product is now Splunk Agent Observability. It offers out-of-the-box and custom evaluators and runtime guardrails.
Enterprise controls. SSO through OIDC or SAML on the on-premises edition, and through Splunk Observability Cloud's SAML on SaaS. SSO is not available in the free or hosted version. Organization roles and project access levels are documented, with groups on paid on-premises plans. Pricing starts at $100 a month per 1.2 million spans, billed annually.
Best for. Splunk customers, and teams that want guardrails that block bad outputs in real time.
Limits. We found no audit log, data residency, retention default, or compliance attestation documented for Agent Observability itself. Its PII evaluator detects PII rather than redacting it. Ask Splunk for the product's SOC 2 scope and BAA terms.
Fiddler
What it is. An AI observability and guardrails platform with roots in model monitoring and explainability.
Enterprise controls. SSO and role-based access from the Developer tier ($0.002 per trace). Enterprise adds SaaS, VPC, or on-premise deployment, dedicated support channels, and a named customer success manager. Fiddler states SOC 2 Type II (report under NDA) and HIPAA compliance, and encrypts data at rest with AES-256.
Best for. Banks, insurers, and healthcare companies that already govern traditional models with Fiddler and want agents under the same controls.
Limits. We found no audit logs, data residency, retention defaults, SLA, or ISO 27001 on Fiddler's pricing and security pages.
Agnost AI
What it is. Agnost AI monitors AI agents in production: chat, voice (through transcripts), copilots, and MCP servers. It is OpenTelemetry-native and works with the OpenAI Agents SDK, Vercel AI SDK, LangChain and LangGraph, Mastra, Pydantic AI, Google ADK, and Agno. It groups activity by user and session and judges whole conversations, detecting failures, user frustration, intents, tool-call errors, and silent wrong answers.
How it handles quality at scale. Online evals are plain-language rules, judged by an LLM for each assistant message as pass or fail, with alerts to Slack or email. Because rules are written in plain language, a support or compliance lead can add one without code, and the same rule can run across every agent. Auto Improve turns a detected failure into a pull request with a fix tested against past runs, for an engineer to review. That is how Agnost answers the release-confidence question: changes are checked against real past conversations before anyone merges.
Enterprise plan. Custom event volume and retention, self-hosted VPC deployment, audit logs, custom SLAs and SLOs, custom improvement workflows, and opt-in model fine-tuning on your own trace data. Agnost does not train models on customer data unless the customer opts in. For SOC 2, HIPAA, and ISO 27001 status, request Agnost's security documentation as part of your review. Best for. Enterprises running customer-facing agents whose failures only show up across a whole conversation, such as a wrong answer three turns in or a user who gives up, and who want those failures turned into reviewed fixes.
Limits. Agnost is not a full observability platform. It has no infrastructure APM and no cost tracking, and SSO and RBAC are not listed as features, so confirm them in your review. It is not open source. Run it next to your existing APM or tracing tool, not instead of it.
Pricing. Free, Starter at $49 a month, Pro at $499 a month, and Enterprise at a custom price. Agnost is a Y Combinator company.
Build vs buy
Building agent monitoring in-house looks cheap because the first step is easy: export OpenTelemetry spans to a warehouse you already run. The cost is everything after that.
What you have to build to match the checklist above:
- Storage and queries for long, nested traces at your volume, with retention and deletion per project.
- SSO, roles, and audit logs on the monitoring tool itself, because conversation logs are some of the most sensitive data you hold.
- PII redaction before export. Our trace redaction guide covers the boundary.
- Judges: prompts, models, and a way to check that the judges agree with people, then rerun them when you change the model.
- Grouping by user and session, dashboards per agent and team, alert routing, and on-call for the monitoring system.
- A regression set from past failures and a way to replay it before each release.
Build when you have one or two agents, an existing data platform team, strict rules that rule out any vendor, and mostly need traces for debugging.
Buy, or self-host a product, when many teams ship agents, security wants SSO and audit logs before launch, or nobody has time to own the judges. Langfuse and Arize Phoenix lower the cost of self-hosting. Commercial vendors add SLAs and someone to call.
The middle path most enterprises take: keep OpenTelemetry as the contract, send the same spans to your own storage and to one or two vendors, and swap vendors without re-instrumenting. Our OpenTelemetry instrumentation guide shows the setup, and the self-hosted agent observability checklist covers what changes when you run the stack yourself.
How to run a 2-week pilot
Week 1: security and setup.
- Send your security questionnaire on day one and ask for the SOC 2 Type II report, BAA terms if you need them, the data-flow diagram, and the sub-processor list. Don't wait until the end of the pilot to find a blocker.
- Pick one customer-facing agent with real traffic and known failures. Agree on a written list of 20 to 50 past conversations: some known failures, some healthy controls.
- Instrument once with OpenTelemetry and send the same traffic to two or three candidates. Turn on redaction before any production content leaves your network.
- Set up SSO, roles, and audit log export in each tool, or confirm in writing which plan you would need.
Week 2: quality and operations.
- Write the same five rules in each tool, such as "never promise a refund" or "hand off to a human after two failed attempts."
- Score each tool on the known-failure list: what it caught, what it missed, and how many false alarms it raised.
- Make a planned change, such as a new prompt or a model switch, and check whether each tool shows the effect within a day.
- Price your real volume. Spans, traces, units, events, and GB don't convert one-to-one. The pricing guide has a worksheet.
- Write a decision record: what each tool caught, which controls it met, what it costs, and who will own it.
The agent monitoring pilot checklist has a scorecard for this, and the self-hosted agent observability checklist covers the extra checks for a VPC or on-prem install.
How to choose
| If your enterprise needs… | Shortlist |
|---|---|
| Tracing and evals for LangChain and LangGraph, with SSO, SCIM, and audit logs | LangSmith |
| A fully self-hosted or air-gapped install | Arize AX, Langfuse (Government tier), Fiddler (on-premise) |
| Open source with a published enterprise price | Langfuse |
| Agent data next to existing APM | Datadog, or Splunk Agent Observability (Galileo) |
| Release gating on experiments and scorers | Braintrust |
| Governance for regulated models and agents in one place | Fiddler, Arize AX |
| Conversation-level quality on customer-facing agents, with fixes as reviewed PRs | Agnost AI, next to your tracing or APM |
Most enterprises end up with two layers: one tracing or APM platform that engineering already trusts, and one tool that tells product, support, and compliance whether conversations went well. Conversation-level vs trace-level online evals explains why each catches failures the other misses.
Related reading
- Best AI agent monitoring tools (2026): pick by the job
- Self-hosted agent observability: what to evaluate before deploying
- An AI agent monitoring pilot checklist you can actually run
- LangSmith alternatives for agent evals and monitoring
- Arize alternatives: options to Arize AX and Phoenix
- Turn production agent failures into regression tests
- How to catch wrong answers that don't show up as errors in traces
Frequently asked questions
What are the best enterprise AI agent monitoring tools?
For tracing and evals with enterprise controls: LangSmith (LangChain and LangGraph teams), Arize AX (managed or self-hosted, including air-gapped), Langfuse (open source, free to self-host, Enterprise $2,499 a month on cloud) and Braintrust (experiments and scorers, with a hybrid data plane in your cloud). For teams already on an APM: Datadog Agent Observability, or Splunk Agent Observability, which is what Galileo became after Cisco bought it. For regulated model and agent governance: Fiddler, which offers SaaS, VPC or on-premise deployment. For conversation-level quality on customer-facing agents, next to your existing tracing: Agnost AI, which judges whole conversations against plain-language rules and opens pull requests with fixes for human review.
Which AI agent monitoring tools are SOC 2 or HIPAA compliant?
According to each vendor's own pages: LangSmith says it is SOC 2 Type 2 and ISO 27001 certified and HIPAA compliant. Arize lists SOC 2 Type II, HIPAA, PCI DSS 4.0 and CSA STAR Level 1, and its self-hosted page adds ISO 27001. Langfuse offers SOC 2 Type II and ISO 27001 reports and a HIPAA BAA, with a HIPAA data region on Enterprise. Datadog lists SOC 2 Type 2, HIPAA, ISO 27001 and FedRAMP High, and Agent Observability is a HIPAA-eligible service under a Datadog BAA. Braintrust is SOC 2 Type II and signs BAAs on Enterprise. Fiddler states SOC 2 Type II and HIPAA compliance. Splunk's Agent Observability pages don't state certifications. Agnost AI shares its security documentation on request. Always ask for the current report rather than relying on a web page.
Which LLM observability tools can run on-premises or self-hosted for enterprise data?
Langfuse is free to self-host under MIT, with an Enterprise self-hosted plan for audit logs, SCIM and project-level RBAC. Arize AX runs in your VPC, with air-gapped deployment available. LangSmith offers self-hosted Kubernetes or a hybrid setup with the data plane in your VPC, both on Enterprise. Braintrust offers on-prem or a hybrid data plane in your cloud on Enterprise. Fiddler Enterprise offers VPC or on-premise. Splunk Agent Observability (Galileo) has an on-premises edition. Agnost AI offers a self-hosted VPC deployment on Enterprise. Datadog's docs describe no self-hosted option for Agent Observability.
What are enterprise alternatives to Langfuse and LangSmith?
Arize AX, Braintrust, Datadog Agent Observability, Splunk Agent Observability (formerly Galileo), Fiddler and Agnost AI. Arize AX and Braintrust are the closest swaps for tracing and evals with SSO, audit logs and self-hosting. Datadog and Splunk fit teams that want agent data next to their APM. Fiddler fits model and agent governance in regulated settings. Agnost AI is not a tracing replacement: it adds conversation-level quality checks and fix pull requests on top of whatever tracing you keep.
Which agent monitoring vendors offer enterprise SLAs and dedicated support?
LangSmith offers custom SLAs for uptime, latency and support response, plus direct escalation to engineers. Arize AX Enterprise lists custom SLAs and dedicated support. Langfuse Enterprise lists uptime and support SLAs, a committed 1-hour severity-1 response around the clock on committed contracts, and a named lead support engineer. Braintrust Enterprise has an uptime SLA, premium support and a shared Slack channel. Fiddler Enterprise has dedicated channels and a named customer success manager. Agnost AI Enterprise includes custom SLAs and SLOs. Datadog sells separate support plans, and the pages we checked don't state an SLA for Agent Observability.
Should we build our own AI agent monitoring in-house or buy a tool?
Build if you only need traces in a warehouse you already run and have engineers to own the judges, dashboards, alerting and on-call. Buy, or self-host an existing product, when you need SSO, RBAC, audit logs and retention controls on day one, when many teams ship agents, or when you need someone to judge conversation quality every day. A common middle path is to send OpenTelemetry to your own storage and to a vendor at the same time, so you keep the raw data and avoid lock-in.
How do I know if switching LLM models broke my agent in production?
Before the switch, replay a set of real past conversations, including known failures, against the new model and compare results rule by rule. After the switch, compare failure rates, frustration and tool-call errors per agent for the same rules before and after the change date, and alert on any jump. Tools that judge whole conversations catch regressions that single-span checks miss, such as an agent that now asks for the same detail twice.
How do enterprises get confidence before rolling out a change to a customer-facing AI agent?
Turn past production failures into a regression set, run the changed prompt or model against it before release, roll out to a slice of traffic, and compare conversation-level pass rates for the same rules against the old version before going to 100%. Braintrust and LangSmith build this around experiments and datasets. Agnost AI's Auto Improve opens a pull request with a fix tested against past runs, for an engineer to review before merge.
Originally published on agnost.ai.
Top comments (0)