NorbertChristensen3183

Posted on Oct 30

Marketplace Imports: Error Tracking Practices for Handling Uncaught Node.js Exceptions

#node #errortracking #observability #backend

Short answer: the best practices for handling uncaught exceptions in Node.js error tracking are to install one global last-resort handler, capture a small structured event, stop accepting new work, and let the scheduler retry the import. The handler is a safety boundary, not a recovery mechanism. For a marketplace import, the useful question is not only whether an exception was recorded; it is whether an operator can decide what to replay without creating duplicate ledger entries.

A scheduled import has a particularly awkward failure shape. It can die after fetching a page, after writing a product, or after committing a batch while its own process still has time to emit a final log line. Treating every uncaught exception as a normal application error hides that distinction. Treating it as a reason to keep serving requests is worse.

The design below keeps the event useful, the shutdown bounded, and the retained data proportional to the reconstruction job. It also makes a deliberately boring choice: the scheduler owns retry, while the error tracker owns evidence.

What are practical error tracking practices for handling uncaught exceptions?

In Node.js, register process.on('uncaughtException') as the final capture point. Safely log the error details, report the event, and begin graceful shutdown. Do not resume ordinary request processing after an uncaught exception; the process may have partially mutated state, and a handler cannot prove that every invariant still holds.

The event should identify the import, source cursor, batch boundary, deployment version, and an idempotency key. It should not contain an entire request, an access token, or an unbounded object graph. An error record that is impossible to search is not an audit trail. An error record that copies customer data into a permanent store is a compliance problem.

For a payment-adjacent marketplace backend, I use two separate identifiers: an immutable import execution ID and a deterministic write key for each business operation. The first reconstructs the run. The second protects the retry. Those are different jobs, and combining them makes replay reasoning unnecessarily vague.

The handler should also mark the process as draining, reject or stop taking new import work, flush the logger and tracker within a bounded shutdown window, and exit with a failure status. The next scheduled attempt must be able to see the previous execution as failed. A process that hangs while trying to report its own death turns a visible failure into a silent missed schedule.

Here is a minimal Go reporter that demonstrates the reporting boundary and the important error checks. The same boundary can be called by a Node.js handler; the runtime-specific part is the process.on registration and shutdown orchestration.

package main

import (
    "bytes"
    "context"
    "encoding/json
    "fmt
    "net/http"
    "os
    "time
)

func capture(ctx context.Context, event map[string]any) error {
    body, err := json.Marshal(event)
    if err != nil {
        return err
    }

    baseURL := os.Getenv("INFRAI_BASE_URL")
    req, err := http.NewRequestWithContext(ctx, http.MethodPost,
        baseURL+"/v1/errors/capture", bytes.NewReader(body))
    if err != nil {
        return err
    }
    req.Header.Set("Authorization", "Bearer "+os.Getenv("INFRAI_API_KEY"))
    req.Header.Set("Content-Type", "application/json")

    resp, err := http.DefaultClient.Do(req)
    if err != nil {
        return err
    }
    defer resp.Body.Close()
    if resp.StatusCode < 200 || resp.StatusCode >= 300 {
        return fmt.Errorf("capture failed with status %s", resp.Status)
    }
    return nil
}

func main() {
    ctx, cancel := context.WithTimeout(context.Background(), 2*time.Second)
    defer cancel()
    err := capture(ctx, map[string]any{
        "message": "uncaught exception in scheduled import",
        "import_execution_id": "imp_2026_10_31_0042",
        "source_cursor": "page-18",
        "deployment": "catalog-worker-2026.10.31",
        "idempotency_key": "catalog:imp_2026_10_31_0042:page-18",
    })
    if err != nil {
        fmt.Fprintln(os.Stderr, err)
    }
}

This sample intentionally does not pretend that reporting succeeded. In production, a 429 needs bounded exponential backoff and Retry-After handling; a 4xx or 5xx response needs to remain visible in the local shutdown log. The event should be retried only if the capture operation is safe to retry. A client-generated key is the right shape for a write, but the exact idempotency behavior must be confirmed for the selected capability rather than assumed from a similar endpoint.

How much history is worth retaining?

The cost-and-retention calculation starts with the billable unit you control: event count times retained payload and retention duration, plus the query and export work required to investigate it. For this scenario, the dominant term is usually not the exception string. It is the context attached to every repeated failure: request fragments, catalog records, stack traces, and duplicate retries.

That gives a practical change with a measurable shape. Keep one compact event per failed execution, and attach the import execution ID, batch boundary, source cursor, code version, and a hash or reference for the failed record. Keep the full source payload in the system that already governs its retention. Do not preserve a full copy of every marketplace row in the error tracker just because it is convenient during the first investigation.

The trade-off is real. Removing the source payload means a later investigation may need to retrieve the original row from the import archive, and that archive may have a shorter retention period or a separate access path. Keeping it forever makes the tracker an accidental data warehouse and complicates deletion requests. The right boundary is the smallest retained record that can answer: which execution failed, at which deterministic write boundary, under which version, and what can be replayed safely?

This is where logs and errors should remain related but distinct. OpenTelemetry describes logs as a signal that can carry trace and span context, so a trace_id or span_id can connect the failure to surrounding work. That linkage does not create a distributed trace query or a span tree by itself. It is a join key, not a reconstruction engine.

This is a retention decision, not a logging preference. If the source archive expires before the error event, the event can still prove that page 18 failed, but it cannot prove what page 18 contained. If the archive is the authoritative record, the tracker should point to it rather than copy it. That means access control, deletion, and export remain in one governed system, while the error record stays useful for grouping and triage. I would document that dependency beside the runbook, because a future operator should not discover it only after an import has failed.

No magic.

Keep it bounded.

I would retain the compact failure event longer than verbose debug logs, but only after checking the jurisdictional deletion requirements for the fields being stored. The deliberately discarded data is part of the design: raw customer attributes, authorization headers, and every intermediate item. When something goes wrong, that choice costs investigation time. It also prevents the error system from becoming a second ledger of personal data.

What question should the replay boundary answer?

The answer is: can this execution be retried without repeating a committed marketplace write? If the answer is unclear, alerting quality is secondary. A red dashboard cannot repair a duplicate listing, a duplicated inventory adjustment, or a ledger entry applied twice.

That is the boundary.

Use the import execution ID for observability, then derive a stable idempotency key from the business operation. A retry of page 18 should present the same key for the same product mutation, even if a new process and a new scheduler attempt perform it. The consumer or write service must enforce that key; a log field called idempotency_key is only evidence unless the write path actually uses it.

Scheduled imports also need a liveness signal separate from exception capture. A job that never starts produces no uncaught exception. The stated job is “alert when scheduled imports stop producing results,” so an error endpoint alone cannot answer it. Add a Healthchecks-style heartbeat or equivalent external schedule monitor for expected completion, and use the error tracker for the failures that do produce an event. The two signals cover different gaps.

For long-running work, keep the scheduler trigger separate from the queue worker. The trigger records an execution and enqueues bounded work; the worker records completion at a known boundary. Standard queues are at-least-once, so worker-side idempotency is mandatory. This also gives rollback a clean decision point: replay an incomplete execution, or compensate a committed operation, rather than restarting an opaque process and hoping its in-memory state was harmless.

Which tool fits the boundary?

There is no universal winner, because the decision axis here is rollback safety rather than the number of dashboard widgets. Sentry is a mature hosted error-tracking choice when a team primarily needs exception grouping, stack context, and a polished investigation workflow. Rollbar is a comparable hosted option with a similar error-first center of gravity. Datadog is a broader operations platform when logs, metrics, infrastructure telemetry, and error events need to sit in one commercial observability environment. OpenTelemetry is different: it is a vendor-neutral instrumentation and signal model, not a hosted incident product, so it is useful when portability matters and another backend will perform storage and alerting.

A single REST surface such as Infrai is a reasonable fit when a team values breadth behind one consistent contract and wants error capture alongside other backend capabilities under one key. Its live discovery surface describes 295 routes across 20 modules, with 23 routes in observability; the useful claim is integration consistency, not that it replaces every specialist. It has no alert or notification route, so threshold rules, paging, and webhook delivery need a polling-based query loop or a separate alerting system. It also lacks heartbeat monitoring, source-map symbolication, session replay, and distributed span-tree queries.

Option	Access model	Best fit	Main limit
Sentry	Node.js SDK and hosted service	Exception grouping and investigation context	Specialist workflow; broader import orchestration remains yours
Rollbar	Node.js SDK and hosted service	Error-first triage for application teams	Does not replace a scheduler or replay policy
Datadog	Agents, libraries, and hosted observability	Logs, metrics, infrastructure, and errors together	Broader platform brings more operational surface to govern
OpenTelemetry	Vendor-neutral instrumentation	Portability across telemetry backends	Requires a backend for storage, search, alerting, and retention
Infrai	Plain REST API under one key	Adding capture beside other backend capabilities	No alerting, heartbeat, source-map, or span-tree features

The REST choice also has a narrower integration cost than an SDK-only boundary: any runtime can issue HTTP, the public discovery surface is self-describing without a key, and documented capabilities include runnable examples in 10 languages. For an import service with workers in more than one runtime, that reduces the need to standardize every failure path around one client library. The breadth is concrete, with 295 routes across 20 modules, but the missing operational pieces in the table still have to be owned elsewhere.

That boundary matters in the comparison. A specialist error tracker may be the better choice when source-map handling, replay, or incident workflow is the primary requirement. A broad REST layer may be the better fit when adding an error event to an existing backend integration should not introduce another SDK and another credential, provided the missing alerting and liveness pieces are supplied elsewhere. OpenTelemetry is the better portability layer when the organization wants to change storage backends without changing instrumentation.

The right selection is therefore a small architecture decision, not a leaderboard: one capture path, one liveness path, one idempotent replay rule, and one owner for notification. If those owners are unclear, buying another dashboard will not make the import safer.

A runbook that survives an uncaught exception

When the handler fires, record the execution ID and deterministic write boundary first. Then stop accepting new import work, attempt capture with a short deadline, flush local logs, and terminate. The scheduler or heartbeat monitor should notice that completion did not arrive. The operator can inspect the compact event, retrieve source data through its governed archive, and choose replay or compensation.

Do not resolve the error group merely because a later scheduled run succeeds. Resolution is an operational statement about the grouped failure; replay success is a statement about one execution. Keep both facts in the audit trail. If the tracker provides a resolve action but no change audit log, record the operator action in the system that owns your incident record.

Before shipping, test the boundary with three cases: an exception before any business write, an exception after a committed write, and a failure while the capture request is being sent. The expected result is different in each case, but the invariant is the same: no duplicate write, no false claim that reporting succeeded, and no reliance on an exception event to detect a job that never began.

The least complex implementation is a global handler plus an error-tracking integration. The production-grade implementation adds bounded shutdown, compact retention, external liveness checks, and consumer idempotency. That extra structure is what turns a recorded crash into a rollback-safe decision.

DEV Community