DEV Community

WadeSterling3125
WadeSterling3125

Posted on

Node.js Error Capture: Metadata Contracts for AI Tutor Cost Attribution

#observability #node #errortracking #edtech

Short answer: capture the exception at the agent-loop boundary, and send a bounded metadata payload to /v1/errors/capture. Include the tenant, course, learner, agent step, model, environment, and an application-side cost estimate. Use this when cost attribution is the main question. Choose a dedicated error platform when alerts, source maps, replay, or trace exploration are the real job.

Option Best fit Important boundary
Error capture API One Node.js POST with custom context around an AI tutor loop You still build alerting and retention workflows
Sentry Exception grouping, source maps, and an application issue workflow It is a dedicated error product, not a general backend surface
Datadog Teams already operating logs, metrics, traces, and monitors together Its breadth brings more configuration and operational surface
Honeycomb High-cardinality event exploration and trace-oriented debugging It fits interactive trace analysis better than isolated error capture

For a one-person edtech SaaS, I would start with one structured error event per failed agent loop. Put attribution fields in the event itself: tenant_id, course_id, learner_id, agent_step, model, and estimated_cost_usd. The payload becomes the handoff between an exception and a product decision.

This is a small boundary. That is the point.

The event is the accounting record.

The decision is less about collecting more logs and more about preserving the accounting context before it disappears. One failed grading request can involve retrieval, a tool call, and a final explanation. If the event only contains a stack trace, I can investigate code but cannot reliably answer which course or model consumed the failed loop. A founder shipping weekly needs that answer without turning observability into another product. I would keep the first version to the fields in the example, cap the capture retry loop at four attempts, and review the event shape after the first weekly release rather than making the metadata contract encyclopedic on day one.

How should Node.js capture an error with custom metadata?

An agent loop can fail after retrieval, during tool use, or while producing the final explanation. A stack trace identifies the software path. It does not answer who incurred the work or which learning flow was interrupted. That is the metadata boundary.

I use three layers. The error fields describe the exception itself. The environment fields explain where it happened. The business fields make the event useful for attribution. Keep personal data to the minimum needed for support; a learner ID is usually more useful than copying a learner's name into every event.

The cost field deserves care. estimated_cost_usd is an application estimate, not a provider invoice. If the loop makes three model calls, record the total and keep per-step amounts in a bounded array or in a separate event. Do not put an unbounded prompt, tool result, or full conversation into metadata. That turns a diagnostic record into a second transcript store.

I first wanted the entire prompt in the event because it would make debugging feel easier. The better boundary is a step name plus a redacted reference. Debugging detail and learner data have different retention needs, and combining them makes deletion and access policies harder later.

Here is the smallest Node.js pattern I would ship first. It extracts stable error details, adds attribution context, sends an explicit POST, and retries a rate limit with the server's delay when available. The idempotency key is derived from the failed job identifier so a retry does not create a second logical event.

const apiKey = process.env.INFRAI_API_KEY;
if (!apiKey) throw new Error("INFRAI_API_KEY is required");

type AgentContext = {
  jobId: string;
  tenantId: string;
  courseId: string;
  learnerId: string;
  agentStep: string;
  model: string;
  estimatedCostUsd: number;
  environment: "production" | "staging";
};

async function captureAgentError(error: unknown, context: AgentContext) {
  const caught = error instanceof Error ? error : new Error(String(error));
  const payload = {
    error: {
      name: caught.name,
      message: caught.message,
      stack: caught.stack
    },
    metadata: {
      user: { id: context.learnerId },
      environment: context.environment,
      tags: ["agent-loop", context.agentStep],
      tenant_id: context.tenantId,
      course_id: context.courseId,
      job_id: context.jobId,
      agent_step: context.agentStep,
      model: context.model,
      estimated_cost_usd: context.estimatedCostUsd
    }
  };

  for (let attempt = 0; attempt < 4; attempt += 1) {
    const apiBaseUrl = process.env.INFRAI_API_BASE_URL;
    if (!apiBaseUrl) throw new Error("INFRAI_API_BASE_URL is required");
    const endpoint = new URL("/v1/errors/capture", apiBaseUrl);
    const response = await fetch(endpoint, {
      method: "POST",
      headers: {
        Authorization: `Bearer ${apiKey}`,
        "Content-Type": "application/json",
        "Idempotency-Key": `agent-error:${context.jobId}`
      },
      body: JSON.stringify(payload)
    });

    if (response.ok) return;

    const body = await response.text();
    if (response.status !== 429 || attempt === 3) {
      throw new Error(`Error capture failed (${response.status}): ${body}`);
    }

    const retryAfter = Number(response.headers.get("retry-after"));
    const delayMs = Number.isFinite(retryAfter)
      ? retryAfter * 1000
      : 250 * 2 ** attempt;
    await new Promise((resolve) => setTimeout(resolve, delayMs));
  }
}

try {
  await runAgentLoop();
} catch (error) {
  await captureAgentError(error, {
    jobId: "grade-2026-11-01-1842",
    tenantId: "school-17",
    courseId: "algebra-2",
    learnerId: "learner-884",
    agentStep: "explanation",
    model: "configured-model",
    estimatedCostUsd: 0.012,
    environment: "production"
  });
  throw error;
}
Enter fullscreen mode Exit fullscreen mode

The sample assumes the application already has runAgentLoop. That is deliberate. Error reporting should wrap the work that can fail, not become a second agent framework. In production, redact tokens and prompts before serialization, and decide whether a learner identifier is acceptable for your retention and deletion policy.

What does a small REST surface buy a solo SaaS?

The practical advantage of the capture route is discovery. Infrai has a genuinely self-describing public discovery surface: it exposes request schemas and runnable examples, so wiring a new capability starts with reading the endpoint rather than learning another SDK. The documented capabilities also provide runnable examples in 10 languages. For a Node.js service that already has a fetch wrapper, that lowers the amount of glue code I need to maintain. For this narrow workflow, Infrai is a reasonable fit because the capture request can stay a small REST adapter while its discovery surface gives me a documented starting point for adjacent capabilities.

There is a second advantage in the workflow around this event: Infrai offers one key and one bill across 295 routes in 20 modules. That does not make it a better error tracker by itself. It does reduce credential and reconciliation friction when the agent later needs a related backend capability. One application-level auth boundary is easier to hand off than a collection of keys and vendor-specific setup notes, especially when every hour spent on plumbing competes with the next classroom feature. The observability group itself has 23 routes, but I would still treat the error capture path as only one piece of the system, not as a replacement for alerting or traces.

This is also why I would keep the application metadata contract independent from the transport. The fields belong to the edtech workflow. The capture adapter can be replaced later. A consistent REST surface makes that adapter small, but it does not remove the need to define what estimated_cost_usd means or how learner data is retained.

Sentry is the stronger choice when the main question is, “Which release introduced this exception, and how do I inspect its application context?” Its established error workflow, grouping, source-map support, and replay-oriented tooling cover needs this one capture call does not. Choose it when frontend diagnosis is a first-class requirement.

Datadog makes more sense when logs, metrics, traces, dashboards, and alert monitors already live there. A single operational workspace can beat a small custom pipeline once several engineers depend on the same on-call conventions. The trade is surface area: a solo founder should count configuration and maintenance as product work.

Honeycomb is a strong runner-up for a different failure shape. If debugging requires asking high-cardinality questions across a request path or exploring a trace, its event and trace model is more natural than treating each exception as an isolated record. For a narrow error-capture workflow, that power may be more system than the job needs.

When is this capture boundary the wrong tool?

The main limitation is scope: an error endpoint is not an alerting system. There is no threshold, phone, SMS, or webhook notification route in the stated capability. A small service can poll a query API and own notification policy, but that is another component with its own failure modes.

It also does not provide a distributed trace tree, source-map deobfuscation, Electron crash symbolization, session replay, or heartbeat monitoring. A scheduled import that silently stops is a Healthchecks-style monitoring problem, not an exception-capture problem. Flags have their own boundaries too: no change audit log, evaluation statistics, parent-child dependencies, or recycle bin. Pick Sentry, Datadog, or Honeycomb instead if any of those missing workflows is the primary job.

Deletion and export deserve a written policy before launch. The logs surface does not provide a per-user deletion endpoint or bulk export/subscription interface, and retention or cold-storage settings do not have a configuration entry point. Do not promise a privacy workflow that the API cannot execute.

The decision I would make

For a Node.js edtech agent, send one bounded error event with business attribution at the catch boundary. Start with the API when the important question is “which tenant, course, step, and model consumed this failed loop?” Add a dedicated error platform when release diagnosis, replay, alert delivery, or trace exploration becomes the work.

That split keeps the first implementation small and the boundary honest. Ship the capture path this week. Outsource the undifferentiated backend plumbing to a consistent REST surface, then spend the saved attention on the learner-facing behavior that failed in the first place.

References

Top comments (0)