DEV Community

Cover image for Waxell vs. LangSmith: When You Need a Control Plane, Not a Dashboard
Waxell profile image Logan
Logan for Waxell

Posted on • Edited on • Originally published at waxell.ai

Waxell vs. LangSmith: When You Need a Control Plane, Not a Dashboard

#ai #langchain #agents #architecture

Two teams, same problem: they need visibility into their production AI agents. Team A is all-in on LangChain, building a customer support bot with limited external tool access. Team B has a multi-framework agent fleet that touches customer PII, calls financial APIs, and operates under compliance requirements. Both teams evaluate LangSmith.

Team A deploys it and never looks back. The native LangChain integration requires almost no setup, the trace explorer is excellent for debugging, and the dashboards show everything they need.

Team B runs into a wall. LangSmith shows them everything their agents are doing — and gives them no way to stop it. When an agent starts calling a PII-adjacent endpoint it shouldn't have access to, they see it in the trace. They can't block it. When they need to demonstrate to their compliance team that certain data handling policies are enforced at runtime, not just monitored, there's nothing to show. LangSmith is doing exactly what it's designed to do. It just isn't designed for what Team B needs.

The distinction here — between observability and runtime governance — is what this comparison is actually about.

LangSmith is an observability, evaluation, and deployment platform for LLM applications and agents — LangChain's own product suite now spans tracing, evals, a managed agent-deployment runtime ("LangSmith Deployment," formerly LangGraph Platform), a no-code agent builder ("Fleet"), and an autonomous failure-diagnosis layer ("Engine"). It gives you traces, debugging, quality dashboards, and infrastructure to ship agents. Waxell is a runtime governance platform that combines observability with policy enforcement, guardrails, and compliance controls that operate before agent actions execute. LangSmith tells you what your agent did (and increasingly, helps you build and run it). Waxell governs what your agent is allowed to do. If you need one, you probably know it. If you need both — that's a different conversation.

What does LangSmith actually do well?

LangSmith's core value is its deep, native integration with LangChain, and it earns that reputation. For teams building on LangChain, the developer experience is the best in the observability space. LangSmith is also no longer just an observability tool — LangChain has expanded it into a broader agent platform, and that expansion matters for anyone comparing the two products today.

Tracing and debugging. One decorator — @traceable — and your Python functions (LangChain-based or not) automatically stream to LangSmith's dashboard. The trace explorer renders the full execution tree: each LLM call, its inputs and outputs, latency, token count, and cost. For debugging a misbehaving chain, the visibility is exceptional. You can see exactly where the model went wrong, replay specific steps, and compare outputs across runs.

Evaluation framework. LangSmith includes built-in tooling to score outputs, run test datasets against your production prompts, and track quality regressions over time, including CI/CD integration (pytest, Vitest, GitHub workflows). If you're iterating on prompts and want to measure whether your changes actually improved things, the eval workflow is well-integrated with the tracing data.

Cost and latency dashboards. Per-run cost tracking, latency distributions, and alert configuration (webhooks, PagerDuty) are first-class features. For teams optimizing their agent applications, the operational dashboards are solid.

Deployment, Fleet, and Engine. Beyond observability, LangChain now ships LangSmith Deployment (managed infrastructure for running agents in production, with durable execution, streaming, and native MCP/A2A/Agent Protocol support), LangSmith Fleet (a no-code agent builder), and LangSmith Engine (a scheduled process that autonomously clusters trace failures, diagnoses root causes, and proposes fixes). None of these are governance products — they don't enforce policy before an action executes — but they mean LangSmith's footprint in a production agent stack is larger than "tracing and evals" alone.

For a pure LangChain shop, LangSmith is hard to beat on the observability dimension. The integration overhead is nearly zero, the UX is polished, and the debugging workflow is purpose-built for the way LangChain structures execution.

The limits appear when you step outside pure observability into runtime control.

Where does LangSmith fall short?

No policy enforcement. This is the fundamental gap. Across Observability, Evaluation, Deployment, Fleet, and Engine, LangSmith has no runtime policy enforcement, no tool access controls, no output filtering, and no rate limiting. It can show you that an agent called an endpoint it shouldn't have — after the call completed. It cannot stop that call from happening. LangSmith Deployment's "Agent Authorization" (beta) and Enterprise-tier SSO/RBAC control who can access the platform and its APIs; neither is a mechanism for blocking what an agent itself is allowed to do mid-execution. For teams that need "prevent this, not just detect this," LangSmith is the wrong tool.

LangChain gets the deepest integration, but LangSmith is not LangChain-only. LangChain has repositioned LangSmith as framework-agnostic: its own FAQs state plainly that you don't need LangChain or LangGraph to use it, and it now ships native integrations for OpenAI, Anthropic, CrewAI, LlamaIndex, Vercel AI SDK, Pydantic AI, and OpenTelemetry. The gap that remains is depth, not access — LangChain-based agents get the most turnkey tracing experience, while other frameworks are fully supported but require more manual instrumentation to reach the same coverage. Multi-framework teams no longer face a hard wall, but they should expect an uneven setup experience.

MCP support exists, but it's connectivity, not governance. LangSmith is no longer MCP-blind: LangSmith Deployment can expose an agent as an MCP server, and LangSmith Fleet can add tools to an agent via a remote MCP server. What LangSmith does not have is an MCP-layer policy or enforcement engine — there's no mechanism to scope, rate-limit, or block individual MCP tool calls before they execute. Teams building MCP-native agent stacks get protocol compatibility from LangSmith, but not governance at that layer.

Compliance limitations. LangSmith provides logs and, on Enterprise, access controls (custom SSO, RBAC, self-hosted/hybrid deployment). What it still does not provide is an enforcement record — documented evidence that a specific policy was evaluated and applied before a specific action executed. "Here are our traces, and here's who could access this workspace" is different from "here is proof that Policy X was enforced before every relevant action." Enterprise teams operating under SOC 2, HIPAA, or financial compliance requirements find this gap significant.

Feature comparison

Capability Waxell LangSmith
Observability
Trace collection ✅ Yes (3-line SDK) ✅ Yes (native LangChain + @traceable for any Python)
Cost & latency dashboards ✅ Yes ✅ Yes
Debugging UI ✅ Yes ✅ Yes (best-in-class for LC)
Evaluation framework ⚠️ Manual ✅ Yes (built-in, CI/CD-integrated)
Automated failure triage ⚠️ Manual ✅ Yes (LangSmith Engine)
Governance & Runtime Control
Runtime policy enforcement ✅ Yes (core capability) ❌ No
Tool access control ✅ Yes ❌ No
Output filtering / guardrails ✅ Yes ❌ No
Rate limiting (per session) ✅ Yes ❌ No
Compliance audit trail (enforcement record) ✅ Yes ⚠️ Logs + access controls, no enforcement record
Human-in-the-loop approval gates ✅ Yes ⚠️ Deployment supports HIL interrupts; not policy-gated
Framework & Stack
LangChain ✅ Yes ✅ Yes (native, deepest integration)
CrewAI, LlamaIndex, custom Python ✅ Yes (core design) ✅ Yes (supported, less turnkey)
MCP-native support ✅ Yes (governed) ⚠️ Protocol support only (no MCP-layer policy enforcement)
Deployment
Cloud SaaS ✅ Yes ✅ Yes
Self-hosted / on-premises ✅ Yes ⚠️ Enterprise only (Cloud, Hybrid, or Self-Hosted)

What governance actually looks like in production

The governance gap between the two platforms isn't abstract. Here's what it looks like in practice.

Scenario: An agent with access to a customer database starts querying fields it shouldn't.

  • LangSmith: You see the query in the trace — after it executed.
  • Waxell: The tool call is checked against a scope policy before execution. If the field is out of scope, the call is blocked and logged. The database is never queried.

Scenario: A cost spike in an agent session — one session is making hundreds of tool calls.

  • LangSmith: You see the spend spike in the dashboard. The session runs until it hits an external rate limit or you manually intervene.
  • Waxell: A cost policy fires when the session exceeds its threshold. The session is halted. Other sessions continue unaffected.

Scenario: A compliance audit requires documentation that PII handling policies were enforced at runtime.

  • LangSmith: You can show traces and, on Enterprise, who had access. You cannot show evidence that a data-handling policy was evaluated and enforced before each relevant action — because policy enforcement isn't part of the platform.
  • Waxell: The execution trace includes policy evaluation records alongside every tool call. The audit trail demonstrates enforcement, not just logging.

These aren't edge cases. They're the standard operational questions that production agent deployments at scale encounter.

What migration looks like

If you're moving from LangSmith to Waxell — or running them in parallel:

From LangSmith:

# Before: LangSmith
from langsmith import traceable

@traceable
def my_agent():
    return chain.invoke(input)
Enter fullscreen mode Exit fullscreen mode 
# After: Waxell
from waxell import WaxellSDK

waxell = WaxellSDK(api_key="...")
agent_trace = waxell.trace("my_agent")

def my_agent():
    with agent_trace:
        return chain.invoke(input)
Enter fullscreen mode Exit fullscreen mode

The instrumentation change is minimal. The difference is that Waxell evaluates governance policies at each step of the execution — not just records it.

Running both: Many teams run LangSmith for its debugging, evaluation, and deployment tooling while adding Waxell as the governance layer. The two tools don't conflict — LangSmith observes (and increasingly ships and runs), Waxell governs. If your team relies on LangSmith's trace explorer or LangSmith Deployment for day-to-day work, you don't have to give it up.

When to choose LangSmith

  • Your stack is LangChain end-to-end and you have no plans to change that.
  • Debugging and trace exploration are your primary use case — LangSmith's UI is purpose-built for this.
  • You want a managed runtime to ship and host agents (LangSmith Deployment) alongside your observability, without a separate governance requirement.
  • You don't have compliance requirements that need runtime enforcement documentation.
  • Evaluation tooling (scoring, regression tracking, automated failure triage) is important to your workflow.

When to choose Waxell

  • Your agent fleet spans multiple frameworks (LangChain plus anything else) and you want the same governance coverage on all of them without extra instrumentation work.
  • You need runtime governance: tool access control, output filtering, cost enforcement, rate limiting.
  • You operate in a regulated environment where compliance documentation requires enforcement records, not just logs and access controls.
  • Your agents use MCP for tool definitions and you need policy enforcement at the MCP layer, not just MCP connectivity.
  • You need the ability to stop specific running sessions without credential revocation or service restarts.
  • You want a single platform for observability and governance rather than two separate systems.

How Waxell handles this: Waxell's governance plane operates above agent code — evaluated before each tool call and output, independent of the agent's own logic. Governance policies enforce tool access scope, cost limits, output filters, and approval gates at runtime. Execution tracing captures the full session record including policy evaluations and enforcement actions, producing an audit trail that documents both what the agent did and what the governance layer enforced. Framework-agnostic: the same three-line integration works across LangChain, CrewAI, and custom Python agents.

LangSmith is good at what it does, and it's doing more of it than it used to — tracing, evals, and now deployment infrastructure for shipping agents. If you're a LangChain team that needs debugging visibility, evaluation tooling, and a managed way to run agents, it's a strong choice and there's no reason to move off it.

The limit isn't a product shortcoming — it's a design decision. LangSmith is built to help you build, ship, and understand agent behavior. Waxell is built to control it. When your agents are in production at scale, touching sensitive data, or operating under compliance requirements, "understand" and "control" aren't the same requirement.

If you're evaluating both: build the prototype with whatever gives you the fastest feedback loop. When you're ready to deploy to production with governance, get started with Waxell.

Frequently Asked Questions

Is Waxell a LangSmith alternative?
It depends on what you mean by "alternative." For observability and tracing, Waxell is a full replacement: it provides traces, spans, cost tracking, and session monitoring across any framework, not just LangChain. For evaluation, deployment infrastructure, and automated failure triage — scoring outputs, running test datasets, shipping agents to a managed runtime, auto-diagnosing trace failures — LangSmith has more purpose-built tooling across a wider product surface. The more important comparison is governance: Waxell adds runtime policy enforcement, tool access control, output filtering, and compliance audit trails that LangSmith doesn't offer anywhere in its product line. Teams that need governance need Waxell; teams that only need observability, evals, or agent hosting can use either.

Can I use Waxell and LangSmith together?
Yes, and many teams do. LangSmith provides debugging, evaluation, and (via LangSmith Deployment) agent-hosting infrastructure; Waxell provides the governance layer that operates at runtime. The two tools don't conflict — LangSmith records and runs, Waxell enforces policies before tool calls execute. If your team depends on LangSmith's trace explorer or deployment tooling, you don't have to give it up to add Waxell governance.

Does Waxell work with LangChain?
Yes. Waxell integrates with LangChain, CrewAI, LlamaIndex, and custom Python agents through the same SDK. For LangChain specifically, the instrumentation replaces LangSmith's @traceable decorator with a Waxell trace context manager — a minimal change that adds governance enforcement to the existing observability. LangChain support is part of Waxell's core framework compatibility, not a special integration.

Does LangSmith support MCP?
As of LangSmith Deployment and Fleet, yes — LangSmith can expose an agent as an MCP server and connect agents to tools via a remote MCP server. That's MCP protocol connectivity, not MCP governance: LangSmith has no mechanism to scope, rate-limit, approve, or block individual MCP tool calls before they execute. Waxell's MCP Gateway is a governed surface for MCP traffic — every tool call is evaluated against policy before it reaches the model or the downstream server.

What governance features does LangSmith lack?
LangSmith has no runtime policy enforcement anywhere in its product line: it records what agents do and, via Deployment, controls who can access the platform, but it cannot prevent actions, block tool calls, enforce tool access scope, filter outputs, or rate-limit sessions mid-execution. It also has limited compliance audit trail capability — it produces logs and access records but not enforcement records that demonstrate policies were evaluated and applied before sensitive actions executed. For teams with compliance requirements that need evidence of runtime controls, not just post-hoc logs and access lists, LangSmith's audit coverage is insufficient.

What does migration from LangSmith to Waxell involve?
For LangChain applications, the instrumentation change is minimal: replace LangSmith's @traceable decorators with Waxell trace context managers. The SDK initialization adds three lines at setup. Framework detection is automatic for LangChain chains. The main effort is configuration: defining governance policies (tool access scope, cost limits, output filters) that match your production requirements. LangSmith trace history stays in LangSmith; Waxell begins a fresh record. Teams can run both in parallel during transition to validate coverage parity.

Sources

Top comments (0)