DEV Community

EngelbertPierce7942
EngelbertPierce7942

Posted on

Integrate Error Tracking in Node.js with a Backend API — Quiet Course Imports

#node #observability #edtech #errortracking

TL;DR: Treat a scheduled import that produces zero usable rows as a business error, not merely a missing log line. Send a small, structured error event from the Node.js backend, attach the import identity and result count, and alert only after a freshness window plus a useful-result check. The decision is signal quality versus noise. A failed request is easy to count; a successful request that imports nothing is the expensive one to miss.

Choice Best fit Main risk
Request-level error events The importer already fails loudly Quiet zero-result runs remain invisible
Run-summary events The job has a stable completion boundary Requires a clear definition of usable output
Run summary plus delayed alert Imports can be late or legitimately empty More state and test cases to maintain

My default is the third option for course imports. Emit one completion event per scheduled run, classify a zero-result run with the input window and expected cadence, then page only when the condition survives the grace period. This keeps the backend API simple while putting the alert decision where the business context exists. A solo SaaS operator has limited revenue-per-hour to spend on false alarms. I would outsource transport and storage to a standards-compatible service or self-hosted component, but keep the classification rule in application code.

What does an error mean for a scheduled import?

A thrown exception is one kind of error. It is not the whole contract. An edtech import can return HTTP 200, parse every row, and still produce no courses because the source window was wrong, a filter excluded every record, or the upstream export was empty. Those runs are operationally different from a network timeout, yet both deserve a review path.

The quiet case is the dangerous case.

That distinction matters.

Define the run boundary first. A useful completion record needs a stable runId, source name, scheduled start, completion time, input window, rows read, rows accepted, and a status. Keep learner data out of the event. Course identifiers or counts are usually enough for diagnosis, and reducing payload data reduces the consequences of an accidental disclosure.

The OpenTelemetry logs model is a useful mental model here: a log record carries a body plus attributes and timestamps, while context can connect records to the operation that produced them. The exact transport can change. The event vocabulary should not.

A practical rule is: acceptedRows === 0 is suspicious only when the input window was expected to contain data. That qualifier matters. A weekend schedule, a newly created tenant, and a genuine no-op export should not produce the same alert as a missing weekday feed.

How do I integrate Node.js error tracking using a backend API?

The application should send a structured event after it knows the import outcome. The following TypeScript example uses a generic JSON endpoint; the receiving system can be an internal collector or another HTTP service. The route is an interface owned by this application, not a vendor-specific API.

type ImportOutcome = {
  runId: string;
  source: string;
  scheduledFor: string;
  windowStart: string;
  windowEnd: string;
  rowsRead: number;
  rowsAccepted: number;
  status: "completed" | "failed";
  errorCode?: string;
};

export async function reportImportOutcome(
  outcome: ImportOutcome,
  endpoint: string,
  apiToken: string,
): Promise<void> {
  const signal = {
    type: "scheduled_course_import",
    occurredAt: new Date().toISOString(),
    severity: outcome.status === "failed" || outcome.rowsAccepted === 0 ? "error" : "info",
    attributes: {
      runId: outcome.runId,
      source: outcome.source,
      scheduledFor: outcome.scheduledFor,
      windowStart: outcome.windowStart,
      windowEnd: outcome.windowEnd,
      rowsRead: outcome.rowsRead,
      rowsAccepted: outcome.rowsAccepted,
      status: outcome.status,
      errorCode: outcome.errorCode,
    },
  };

  const response = await fetch(endpoint, {
    method: "POST",
    headers: {
      "content-type": "application/json",
      authorization: `Bearer ${apiToken}`,
      "idempotency-key": outcome.runId,
    },
    body: JSON.stringify(signal),
  });

  if (!response.ok) {
    throw new Error(`Error signal rejected with status ${response.status}`);
  }
}
Enter fullscreen mode Exit fullscreen mode

There are two deliberate choices in this small function. It reports one summary instead of one event per imported row, so a large catalog does not multiply noise. It also checks the response, because a client-side fetch call completing is not proof that the backend accepted the signal. If the response is 429, retry with bounded exponential backoff and preserve the same idempotency key; do not create a new event for every attempt. The import itself should not be marked failed solely because reporting failed; record that secondary failure locally and retry it. Otherwise an observability outage can rewrite the business result.

Do not put access tokens, email addresses, row contents, or full upstream responses into the event. Error tracking is part of the data boundary. Redaction belongs before serialization, not in a dashboard query after sensitive fields have already been stored.

How should the alert avoid noisy empty runs?

The alert policy needs three inputs: schedule, freshness, and usefulness. Suppose a weekday import is expected by 06:00, has a 30-minute grace period, and normally accepts at least one course. A run that completes at 06:12 with zero accepted rows is a warning candidate. A run that has not produced a completion event by 06:30 is a different condition: freshness failure. A run that fails parsing is a processing failure. Keeping these states separate makes the notification actionable.

For an illustrative batch, consider 1,240 rows read and 1,180 accepted. That is a healthy completion even if 60 rows were rejected. Now consider 1,240 rows read and zero accepted. That should carry a reason code and a linkable runId, not just the text Import failed. The numbers are evidence for triage; they are not an alert threshold by themselves.

Use deduplication on the stable identity of the condition: source, schedule date, and failure class. Resolve the alert when a later run produces usable output or an operator explicitly records a legitimate empty window. A retry loop should have a bounded delay and a maximum attempt count. Infinite retries create a second quiet failure.

This is where I would spend design time before adding another dashboard. A dashboard can show that zero happened. The alert needs to explain why zero is abnormal and what changed since the last healthy run.

The limitation is real: this design cannot infer whether an empty upstream export is legitimate without schedule or tenant context. If that context is unavailable, use request-level errors and a separate freshness check; pretending that every empty result is a failure will train people to ignore the alert.

Where does the runner-up approach fit?

Request-level tracking is still the right choice when the import has no trustworthy completion boundary, or when each upstream request represents an independent unit of work. It gives fast visibility into timeouts and status failures. It is also easier to add to an existing service. Its boundary is semantic: successful requests can hide an empty result.

A periodic count check can work when the source system exposes a reliable count and the course catalog has a known baseline. Its weakness is delayed diagnosis. A count mismatch says that something changed, but not whether parsing, filtering, authorization, or scheduling caused it.

The run-summary pattern has more application work, but it has the best signal for this edtech job because the importer knows what “usable” means. For a one-person team, that trade is attractive when it prevents a morning support ticket from becoming a forensic exercise. It is a poor fit for a fire-and-forget script with no durable run identity; adding a database and retry queue just to report a one-off task can cost more operational attention than the signal is worth. Ship the event contract weekly, test the zero-result and late-run cases, and keep the transport replaceable.

Sources

Top comments (0)