DEV Community

UrielDonovan6839
UrielDonovan6839

Posted on

Marketplace Exception Evidence: Preserving Context for Uncaught Node.js Failures

#node #observability #errortracking #marketplace

Short answer: treat an uncaught exception as the end of a process, not as a recoverable request error. Capture the exception, flush a small evidence record, stop accepting work, and let a supervisor start a clean process. For a marketplace, the useful record is not a stack trace alone. It is the smallest set of facts that lets me reconstruct which order, seller action, and billing decision were in flight without copying customer data into logs.

Here is the decision matrix I use when reviewing this path:

Choice Preserves Main risk
Keep serving after uncaughtException Availability for a little longer Corrupted process state and misleading success responses
Exit immediately with only the stack A clean restart boundary Too little evidence to explain the customer incident
Record bounded context, flush, then exit Reproducible evidence and a clean boundary A small amount of carefully designed shutdown code

My recommendation is the third row. The goal is not to make the crash disappear. The goal is to leave behind trustworthy evidence and avoid claiming that work succeeded when the process can no longer be trusted. That is a revenue-per-hour decision: one extra hour of debugging costs more than a short, deliberate evidence path, while a custom incident system is undifferentiated work I would rather outsource to standard telemetry components.

The trade-off is real. This pattern is a poor fit for a tiny command-line job with no durable work and no customer-facing state; a short process can write its failure to stderr and let its scheduler retry. It is also a poor fit when the only reporting channel routinely takes longer than the shutdown budget. For a long-lived marketplace API, though, the limitation is preferable to silently continuing with unknown state.

What evidence does a marketplace incident actually need?

Start with an incident envelope. Give it a stable event ID, a timestamp, a process ID, a deployment revision, and a severity that means “the process is leaving.” Add request and trace identifiers when they exist. Then add business identifiers as opaque values: order ID, seller ID, and the rule or workflow name. Do not add the full request body, authorization header, payment token, or customer address.

The distinction matters. A stack says where the program stopped. The envelope says which piece of work was at risk. Those fields also let a later query join the exception to request logs, job state, and audit records without turning the exception handler into a second database.

I initially thought a single serialized error object would be enough. It was not. Error objects frequently carry non-enumerable fields, and a circular value in attached context can make the reporting path throw its own exception. Normalize known fields into a plain object, cap strings, and treat serialization as fallible. The handler must have fewer dependencies than the code it is observing.

A useful event can be this small:

type IncidentEnvelope = {
  eventId: string;
  occurredAt: string;
  processId: number;
  revision: string;
  error: { name: string; message: string; stack?: string };
  request?: { requestId?: string; traceId?: string };
  marketplace?: { orderId?: string; sellerId?: string; workflow?: string };
};

function normalizeError(value: unknown): IncidentEnvelope["error"] {
  if (value instanceof Error) {
    return {
      name: value.name,
      message: value.message.slice(0, 1000),
      stack: value.stack?.slice(0, 12000),
    };
  }

  return {
    name: "NonErrorThrow",
    message: String(value).slice(0, 1000),
  };
}
Enter fullscreen mode Exit fullscreen mode

The limits are policy, not decoration. They keep one malformed value from consuming the shutdown budget and make cost attribution legible: event volume, retained bytes, and high-cardinality fields are choices the team can measure.

Should the process keep handling requests after an uncaught exception?

No. Node.js documents uncaughtException as a last resort, and its guidance is to perform synchronous cleanup before shutting down rather than resume normal operation. That boundary is especially important in a marketplace checkout or seller payout path. An exception can leave an in-memory lock, partially updated cache, open transaction assumption, or retry decision in an unknown state.

The shutdown path should be boring:

  1. Build one bounded envelope.
  2. Write it to the configured error channel.
  3. Wait for that channel's flush or timeout.
  4. Exit with a non-zero status.

A supervisor handles replacement. The application does not need to know whether that is a service manager, a container runtime, or a process manager. It only needs a clear failure status. Also install unhandledRejection handling. An unhandled promise rejection is a different signal, but it needs the same policy when the application cannot prove the process is safe to continue.

let shuttingDown = false;

async function reportAndExit(reason: unknown): Promise<void> {
  if (shuttingDown) return;
  shuttingDown = true;

  const envelope: IncidentEnvelope = {
    eventId: crypto.randomUUID(),
    occurredAt: new Date().toISOString(),
    processId: process.pid,
    revision: process.env.APP_REVISION ?? "unknown",
    error: normalizeError(reason),
    request: getCurrentRequestContext(),
    marketplace: getCurrentMarketplaceContext(),
  };

  try {
    await Promise.race([
      writeIncident(envelope),
      new Promise((_, reject) =>
        setTimeout(() => reject(new Error("incident flush timeout")), 1500),
      ),
    ]);
  } finally {
    process.exitCode = 1;
    process.exit();
  }
}

process.on("uncaughtException", reportAndExit);
process.on("unhandledRejection", reportAndExit);
Enter fullscreen mode Exit fullscreen mode

The helper names above are intentionally interfaces, not a product recipe. writeIncident can target a structured log collector or another error-tracking boundary. The important contract is bounded input, one attempt, and no request handling after the fatal signal. In production code, I would also make the shutdown timeout configurable and test it with a fake writer.

How do I prove the record is enough to reconstruct the incident?

Test the evidence, not only the handler. Create a fixture where a seller listing update throws after the request context has been attached. Assert that the event contains the request ID, trace ID, deployment revision, and opaque marketplace identifiers. Assert that it excludes headers and payment fields. Then test the unpleasant cases: a thrown string, an error with a circular property, a missing trace context, a writer that never resolves, and two fatal signals arriving together.

A second test belongs outside the unit suite. Run a disposable process, trigger an uncaught exception, and verify three outcomes: the event reaches the collector, the process exits non-zero, and a fresh process can start. This catches wiring mistakes that a mocked writeIncident cannot.

Keep the collector schema stable enough for joins. OpenTelemetry describes logs as records that can carry attributes and trace context; that gives the incident envelope a standards-shaped place to live without making the marketplace fields part of the standard. Put order.id and seller.id in a controlled attribute namespace, define retention by evidence value, and sample ordinary failures separately from fatal process exits. Never sample away the only record of a process termination.

When is the runner-up approach reasonable?

There is one legitimate runner-up: a narrowly scoped worker that can be discarded after each job, with an external queue and an idempotent job key. In that shape, the worker may report the failed job and exit while the queue preserves the unit of work. It is still not a reason to continue serving from a process after an uncaught exception.

For a long-lived HTTP process, “catch everything at the top” is not an alternative. It is a way to turn an unknown state into a plausible response. That creates harder marketplace questions later: did the seller update commit, did the buyer see a confirmation, and should a retry create a duplicate? The answer belongs in durable workflow state and idempotency checks, not in a heroic exception handler.

The decision rule is simple: preserve enough bounded context to explain the customer impact, then make the process disposable. That gives a solo team a weekly-shippable operating habit, keeps telemetry spend attributable to real evidence, and leaves product work ahead of bespoke crash plumbing.

Sources

Top comments (0)