Adding AgentInspect to LangGraph Without Changing Production

What I learned from adding an environment-gated local trace to a NestJS, LangGraph, and Gemini service that already used New Relic and Braintrust

My TypeScript service did not have an observability vacuum. New Relic already covered application performance, and selected AI paths produced evaluation-oriented telemetry in Braintrust. Those tools answered important production questions: Is latency increasing? Which requests are failing? How is model behavior changing across a dataset?

The gap appeared during local development.

When one Gemini-backed sub-agent returned an unexpected result, I wanted to answer a narrower question:

What happened, in order, inside this one agent invocation?

The usual logs showed that the request started, the model responded, and the handler completed. They did not clearly show the nested LangChain operations, where structured-output parsing sat in the execution path, or whether an apparently successful run had left an incomplete callback lifecycle behind.

I did not want to replace our production stack or introduce another hosted collector. I wanted a local execution tree that could be enabled on demand, checked deterministically, and converted into a reviewable artifact when necessary.

That is the role for which I tested AgentInspect, specifically agent-inspect and @agent-inspect/langchain at version 6.17.4.

The constraint: tracing must disappear when disabled

The service is a NestJS API with BullMQ workers. User-facing flows fan out to several LangGraph-style ReAct sub-agents, including a router, a copywriter, a pre-trip assistant, and an email parser. The copywriter accepts structured travel and marketing context, calls Gemini, and returns validated JSON through LangChain's structured-output path.

This is production-shaped code, so the integration had one non-negotiable condition: no AgentInspect callback, trace file, or network behavior when the feature flag was off.

I centralized that rule in a small callback factory. The service uses CommonJS at this boundary, so the optional adapter can be loaded only when local tracing is enabled.

import type { BaseCallbackHandler } from "@langchain/core/callbacks/base";

import type { CaptureMode } from "@agent-inspect/langchain";

const ENABLED_VALUES = new Set(["1", "true", "yes", "on", "enabled"]);

function isAgentInspectEnabled(): boolean {
  return ENABLED_VALUES.has(
    (process.env.AGENT_INSPECT ?? "").trim().toLowerCase(),
  );
}

function getCaptureMode(value: string | undefined): CaptureMode {
  if (value === "none" || value === "preview") return value;

  return "metadata-only";
}

export function buildAgentInspectCallbacks(
  runName: string,
): BaseCallbackHandler[] {
  if (!isAgentInspectEnabled()) return [];

  try {
    const { AgentInspectCallback } = require("@agent-inspect/langchain");

    return [
      new AgentInspectCallback({
        runName,
        traceDir:
          process.env.AGENT_INSPECT_TRACE_DIR ??
          "./.agent-inspect/langchain",
        capture: getCaptureMode(process.env.AGENT_INSPECT_CAPTURE),
        persist: true,
        stream: true,
      }),
    ];
  } catch (error) {
    const code = (error as NodeJS.ErrnoException).code;

    if (code === "MODULE_NOT_FOUND") return [];

    throw error;
  }
}

The capture-mode parser matters. AgentInspectCallback accepts only "none", "metadata-only", or "preview"; passing an arbitrary environment string weakens the TypeScript contract.

At each existing LangChain entry point, I appended the optional callback without disturbing the handlers already present:

await copywriterAgent.invoke(input, {
  callbacks: [
    ...existingCallbacks,
    ...buildAgentInspectCallbacks("copywriter"),
  ],
});

When AGENT_INSPECT is unset, the helper returns an empty array. No AgentInspect callback is instantiated and no AgentInspect trace I/O occurs. Existing Braintrust callbacks remain in place wherever they were already configured.

I also added .agent-inspect/ to .gitignore. Local traces are development artifacts, not source files.

Why structured output made the tree useful

The copywriter is not a single model call. LangChain's withStructuredOutput path can produce multiple pieces of framework scaffolding: a RunnableSequence containing the LLM call and a parser chain that may appear as another root-level operation.

That shape is important. A clean-looking single-root diagram would be misleading if the callback relationships do not establish one. AgentInspect's current LangGraph fidelity contract explicitly allows multiple root-level framework scaffolds for structured-output chains and prefers visible unresolved relationships over invented hierarchy.

I enabled tracing for one local copywriter run:

export AGENT_INSPECT=1

export AGENT_INSPECT_TRACE_DIR=./.agent-inspect/langchain

export AGENT_INSPECT_CAPTURE=metadata-only

npm run inspect:sub-agent -- copywriter

npx agent-inspect list --dir .agent-inspect/langchain

npx agent-inspect view <run-id> \
  --dir .agent-inspect/langchain \
  --verbose

The invocation took approximately 25 seconds, almost all of it inside the live model call. The local tree reduced the run to the structure I needed:

AgentInspect Run: copywriter

Status: success

Duration: ~25s

Execution Tree:

✔ chain:RunnableSequence (~25s)

  ✔ llm:gemini-3.1-pro-preview (~25s)

✔ chain:parser (~1ms)

[Screenshot 1: Redacted terminal output from agent-inspect view, showing the sequence, Gemini call, and parser root.]

Four details were immediately useful:

The standalone run had a terminal success state.

The adapter preserved the model identifier supplied through LangChain metadata.

The LLM appeared below the sequence that invoked it.

The parser remained visible rather than being forced into an unsupported parent relationship.

With capture: "metadata-only", the trace did not contain the full prompt or generated response. It retained lifecycle and bounded metadata needed for diagnosis. That default reduced the amount of sensitive material I had to manage, although metadata can still be sensitive and still requires review.

This was more actionable than the flat log sequence I had before:

[INFO] copywriter.invoke.start

[INFO] llm.request model=gemini-3.1-pro-preview

[INFO] llm.response tokens=1847

[INFO] copywriter.invoke.end status=ok

Those lines confirm that work occurred. They do not show nesting, sibling framework operations, hidden retries, or whether every callback lifecycle closed correctly. The execution tree gave me that orientation without searching across several log fields.

Turning the same trace into a deterministic check

Inspection solved the immediate debugging problem, but a trace becomes more valuable when the same evidence can protect a pull request.

For the copywriter path, the first useful rule was deliberately modest: the run must be complete.

npx agent-inspect check <run-id> \
  --dir .agent-inspect/langchain \
  --require-completed

For workflows that record explicit observed outcomes, I can extend the command without adding an LLM judge:

npx agent-inspect check <run-id> \
  --dir .agent-inspect/langchain \
  --require-completed \
  --fail-on-observation failed

These checks are trajectory assertions, not semantic quality scores. They can prove that a run completed, that expected tools appeared, that forbidden tools did not appear, or that a recorded outcome failed. They cannot decide whether the copy is persuasive or whether a trip recommendation is good. Those questions remain in the evaluation layer.

The version progression mattered

This integration also exposed why agent-observability checks need production-shaped traces rather than only tidy demos.

My first pass on 6.12.1 produced a failing check even though the local invocation had completed:

Run contains incomplete running events

structure.orphan: parentId not present

status: fail

The failure correlated with the multi-root structured-output shape. I reported the behavior and continued retesting rather than disabling the check. The same moderate path passed on 6.14.1; later releases added broader relationship-conformance coverage and closed the remaining deep-swarm findings.

The progression I observed was:

Version tested	Result on the copywriter workflow
6.12.1	False failure involving incomplete/orphan structure diagnostics
6.14.1	Moderate structured-output check passed
6.16.0	Check and share workflow passed in the retest
6.17.4	Current version used for the final integration and article commands

There is an important distinction between the release I personally retested and the release that documented a fix. AgentInspect's changelog places the token-configuration safety correction in 6.14.2; I next validated the complete share flow on 6.16.0. The project's public LangGraph hardening case study describes the wider four-round progression from capture blockers to zero open moderate and deep-swarm findings by 6.16.0.

That feedback loop was one of the more convincing parts of the evaluation. The integration did not merely work on a canonical example. A real callback shape revealed check and safety precision problems, the maintainer converted those shapes into regression coverage, and later versions behaved differently on the same workflow.

[Screenshot 2: Redacted before-and-after check output showing the early false failure and later pass.]

Creating a reviewable artifact

The third step was sharing evidence without treating “local” as automatically safe.

npx agent-inspect verify-safe <run-id> \
  --dir .agent-inspect/langchain

npx agent-inspect bundle <run-id> \
  --dir .agent-inspect/langchain \
  --profile share \
  --out ./evidence/copywriter

npx agent-inspect bundle verify ./evidence/copywriter

An earlier build classified ls_max_tokens as a credential-like key even though it represented model configuration. AgentInspect documents that token-configuration fields stopped failing by key alone in 6.14.2. When I retested at 6.16.0, this trace reached a SAFE assessment without --allow-unsafe, and the bundle could be generated normally.

SAFE is not a compliance certification or permission to publish a trace blindly. verify-safe is a best-effort local assessment. I still inspect the redacted JSONL and generated artifact before attaching it to a pull request or sending it outside the team. The project's safe-sharing guidance makes the same boundary explicit.

The useful property is continuity: view, check, verify-safe, and bundle operate on the same captured evidence. I do not have to reconstruct a debugging story manually after the fact.

Regression safety when tracing is off

Before merging the helper, I ran the existing unit suite with AgentInspect disabled. All 187 tests passed.

unset AGENT_INSPECT

npm test

# Test Suites: all passed

# Tests:       187 passed, 187 total

[Screenshot 3: Test output showing 187/187 passing with AGENT_INSPECT unset.]

That result did not prove that tracing was free. It proved the narrower condition I cared about: the disabled path preserved existing application behavior. The helper performed only an environment check and returned no callback; it did not create trace files or modify the production callback list.

For integration tests that intentionally enable capture, a persisted trace can also be read in-process:

import { openTraceFile } from "agent-inspect/readers";

const trace = await openTraceFile("./fixtures/copywriter.safe.jsonl");

openTraceFile became part of the documented persisted-trace API in the 6.15.0 line. I prefer the CLI for interactive investigation, but the programmatic reader is useful when a custom test or CI helper needs structured access without shelling out.

What AgentInspect replaced—and what it did not

Need

Tool I continue to use
Node service latency, errors, and fleet behavior	New Relic
Dataset experiments and semantic evaluation	Braintrust and existing evaluation tooling
One local invocation as an ordered execution tree	AgentInspect
Deterministic trajectory checks in a PR workflow	AgentInspect
Redacted offline evidence for technical review	AgentInspect bundle workflow

That division kept the integration small. AgentInspect did not need production credentials, a hosted project, or a collector. New Relic and Braintrust did not need to be removed or reconfigured.

The library's own comparison guidance describes this as the laptop-to-PR evidence loop. That framing matches my experience better than calling it a general observability replacement.

Limitations I would plan around

The integration was useful, but it was not frictionless.

Framework scaffolding remains visible. Names such as RunnableSequence, parser chains, and DynamicStructuredTool can be less meaningful than domain names. Where business-level assertions matter, I prefer explicit metadata and observed outcomes over checking framework class names.

Callback fidelity depends on callback input. If a LangGraph node does not propagate runnable configuration or the framework never emits a terminal callback, the adapter cannot manufacture certainty. The current diagnostics expose unresolved relationships instead of inventing them.

Metadata-only is safer, not harmless. Model names, token counts, tool identifiers, correlation IDs, and paths can still reveal information. Redaction and manual review remain necessary.

Local JSONL is not a team dashboard. For retention, fleet search, alerts, and shared URLs, I still use hosted production tooling.

Trajectory checks do not score output quality. A structurally correct run can still return poor copy. Deterministic checks and semantic evals solve different problems.

These limitations are acceptable because the tool stays within a narrow job. Problems arise when a local trace debugger is evaluated as if it were production APM or a full evaluation platform.

A practical adoption path

For another LangChain or LangGraph TypeScript service, I would repeat the integration in this order:

1. Install agent-inspect and @agent-inspect/langchain alongside the existing LangChain dependencies.
2. Add one environment-gated callback factory.
3. Start with capture: "metadata-only" and one sub-agent, not the entire system.
4. Confirm that view represents the real callback structure faithfully.
5. Add one deterministic rule such as --require-completed before introducing a large contract.
6. Retain a sanitized regression fixture for the callback shape that matters.
7. Run verify-safe, create a share-profile bundle, and inspect it manually before external sharing.
8. Keep production APM and evaluation tools responsible for the jobs they already perform well.

If you want to test the adapter without a provider key, the repository includes a LangChain callback example and fixture-backed LangGraph coverage. Those examples are useful for learning the surface; a production-shaped trace is still the real test of whether the tree and checks fit your application.

Final assessment

On this NestJS, LangGraph, and Gemini service, the AgentInspect integration stayed appropriately small: one helper, one callback spread per sub-agent, and an environment flag that kept the default path unchanged. The local tree made a structured-output invocation easier to understand than flat logs, and the same persisted trace could later support a deterministic completion check and a redacted evidence bundle.

The most valuable result was not another dashboard. It was a shorter evidence loop:

run locally, inspect the actual path, protect the path with a deterministic check, and share only the reviewed artifact.

I still use New Relic for production operations and Braintrust for evaluation-oriented work. AgentInspect earns its place before those layers: while I am reproducing one failure, reviewing one pull request, or turning one real callback edge case into a regression fixture.