Your AI Agent Needs Tracing, Not Just Logs
You've probably already called an LLM from a Node.js backend. That part's easy — every provider ships a solid SDK. The part that actually trips people up is what happens after: turning that one API call into an agent that reasons, uses tools, loops a few times, and still behaves once real users are hitting it.
Here's a small, honest pattern for that — plus the one thing most tutorials skip: making the loop debuggable.
Why Node.js is doing this job
Node has quietly become the default home for the application layer around AI. It's become the preferred middle layer for deploying modern AI agents, wrapping heavier model inference behind fast Node APIs. Python still owns training and the heavy orchestration frameworks — Node owns the gateway, the auth, the streaming UI, and the business logic wrapped around all of it.
On the SDK side, things consolidated fast: OpenAI's Node SDK holds roughly a third of weekly npm downloads across the major JS AI SDKs, and Anthropic's TypeScript SDK has grown nearly tenfold in a year. And despite all the framework noise, most production teams just use the Claude or OpenAI SDK directly — reaching for LangChain.js or Mastra only once multi-agent coordination actually earns its keep.
The loop: reason, act, repeat
Almost every "agent" in 2026 runs on the same loop: reason about the task, act through a tool call, look at what came back, reason again — repeat until done. That's it. The engineering is in the guardrails around it, not the loop itself.
// agent.js
import Anthropic from "@anthropic-ai/sdk";
const anthropic = new Anthropic(); // reads ANTHROPIC_API_KEY from env
const tools = [
{
name: "get_order_status",
description: "Look up the status of a customer order by order ID.",
input_schema: {
type: "object",
properties: { orderId: { type: "string" } },
required: ["orderId"],
},
},
];
async function getOrderStatus({ orderId }) {
// stand-in for a real DB/service call
return { orderId, status: "shipped", eta: "2 days" };
}
const MAX_ITERATIONS = 10; // cap simple agents; complex ones can go higher
export async function runAgent(userMessage) {
const messages = [{ role: "user", content: userMessage }];
for (let i = 0; i < MAX_ITERATIONS; i++) {
// Check Anthropic's docs for the current model ID before shipping —
// model strings are versioned and change over time.
const response = await anthropic.messages.create({
model: "claude-sonnet-4-5",
max_tokens: 1024,
tools,
messages,
});
const toolUse = response.content.find((b) => b.type === "tool_use");
if (!toolUse) {
return response.content.find((b) => b.type === "text")?.text;
}
const result = await getOrderStatus(toolUse.input);
messages.push({ role: "assistant", content: response.content });
messages.push({
role: "user",
content: [
{ type: "tool_result", tool_use_id: toolUse.id, content: JSON.stringify(result) },
],
});
}
throw new Error("Agent exceeded max iterations without resolving");
}
Two details separate this from a toy demo:
- The iteration cap. Skip it, and a confused agent can loop forever, quietly burning tokens. Capping iterations — 10 for simple agents, 25 for complex ones — is one of the patterns that carries the most weight in production agent code.
- Tight tool schemas. The model is only as reliable as the contract you give it. Vague schemas are where most "why did it do that" bugs come from.
Tip: Never log raw tool inputs/outputs or API keys without checking what's in them first — they can carry customer PII.
Now make it debuggable
Here's the part everyone skips, then regrets: an agent isn't one request, it's a sequence of decisions. When it breaks three steps in, a single log line at the end won't tell you why. Treat each loop iteration as its own span:
import { trace } from "@opentelemetry/api";
const tracer = trace.getTracer("ai-agent");
export async function runAgent(userMessage) {
return tracer.startActiveSpan("agent.run", async (rootSpan) => {
const messages = [{ role: "user", content: userMessage }];
try {
for (let i = 0; i < MAX_ITERATIONS; i++) {
const stepResult = await tracer.startActiveSpan("agent.step", async (span) => {
const response = await anthropic.messages.create({
model: "claude-sonnet-4-5",
max_tokens: 1024,
tools,
messages,
});
span.setAttribute("agent.iteration", i);
span.setAttribute(
"agent.tool_used",
response.content.some((b) => b.type === "tool_use")
);
// Never attach API keys or raw user PII to span attributes —
// trace data usually lands in a third-party APM backend.
span.end();
return response;
});
const toolUse = stepResult.content.find((b) => b.type === "tool_use");
if (!toolUse) {
rootSpan.setAttribute("agent.resolved", true);
return stepResult.content.find((b) => b.type === "text")?.text;
// rootSpan.end() fires once, in the finally block below.
}
const result = await getOrderStatus(toolUse.input);
messages.push({ role: "assistant", content: stepResult.content });
messages.push({
role: "user",
content: [
{ type: "tool_result", tool_use_id: toolUse.id, content: JSON.stringify(result) },
],
});
}
rootSpan.setAttribute("agent.resolved", false);
rootSpan.recordException(new Error("Max iterations exceeded"));
throw new Error("Agent exceeded max iterations without resolving");
} finally {
rootSpan.end();
}
});
}
What this buys you:
- Per-step latency and cost — which iteration is slow, which one calls a tool vs. resolves outright.
- A trace of the reasoning path — not just input/output, but how it got there. Great for debugging and for spotting drift over time.
- Exceptions tied to the exact step that failed, instead of one vague "agent broke" error at the top.
Already running AppSignal, Datadog, or Honeycomb? These spans export straight into your existing dashboards through standard OpenTelemetry — no bespoke agent-monitoring tooling needed.
Note: Tracing isn't free — each span adds a sliver of overhead, and it adds up at scale. Sample a percentage of requests in production instead of tracing every single call at full fidelity.
Do you even need a framework?
Not on day one. A reasonable default:
- Provider SDK directly (Anthropic/OpenAI) — full control, easiest to instrument. Good for one agent with a handful of tools.
- Vercel AI SDK — when you want a streaming chat UI in Next.js/React.
- Mastra or LangGraph.js — once you've got multiple cooperating agents, need durable/resumable workflows, or built-in memory — and the coordination complexity actually justifies it.
The takeaway
Calling an LLM from Node isn't the hard part anymore — every provider's SDK handles that fine. The real work is building the loop with guardrails (iteration caps, tight schemas) and making it observable step-by-step, the same way you'd instrument any other multi-step system. Do that, and "why did the agent do that" stops being a mystery and starts being a five-minute trace lookup.
What's the weirdest thing your own agent has done silently, before you added tracing?
Top comments (0)