Short answer: For Node.js error tracking during a gaming cohort experiment, send a small custom metadata API payload with the release, experiment, variant, tenant cohort, stage, and one stable event ID; keep the stack trace separate and make the event safe to retry.
A rollback-safe Node.js error event should carry the experiment assignment, tenant cohort, and release identifier as structured metadata. The deciding constraint is reversibility: when a gaming experiment behaves badly for one tenant cohort, the operator needs enough identity to disable that assignment without guessing from a stringified message.
The tempting implementation is a catch block that sends { message, stack, metadata } to an error API. It works until metadata becomes a bag of arbitrary values, retries duplicate events, or a player identifier leaks into a field meant for aggregation. Before copying any payload, measure the fields required for a cohort decision and the payload size accepted by the collector.
How should Node.js tracking report errors with custom metadata?
The useful question is not “did this request throw?” It is “which reversible decision does this failure change?” For a tenant-based gaming experiment, one event should answer:
- Which release produced the event?
- Which experiment and variant were assigned?
- Which tenant cohort saw it?
- Was the failure in assignment, a game action, or result reporting?
- Can the same event be recognized if the sender retries?
Those answers belong in typed attributes, not exception text. OpenTelemetry describes logs as records with a body and attributes. A stable attribute name gives a query something to match; a message is for a human reading the failure.
For example, experiment.variant = "physics-b" is a decision dimension. error.message = "unexpected response" is diagnostic content. Combining them into one sentence makes filtering unreliable. A player ID or raw request body may help a narrow investigation, but it is a poor default attribute because it increases cardinality without identifying a rollback policy.
A compact TypeScript contract for custom metadata
The contract below gives the transport adapter one shape to serialize and gives rollback code a finite set of fields to inspect. The endpoint is an application-owned interface, not a product-specific setup.
type ExperimentFailureStage =
| "assignment"
| "game-action"
| "result-reporting";
type ErrorMetadata = {
service: string;
release: string;
environment: "production" | "staging";
tenantCohort: string;
experiment: string;
variant: string;
stage: ExperimentFailureStage;
requestId: string;
};
type ErrorEvent = {
eventId: string;
timestamp: string;
error: { name: string; message: string; stack?: string };
metadata: ErrorMetadata;
};
function makeErrorEvent(
error: unknown,
metadata: ErrorMetadata,
eventId: string,
): ErrorEvent {
const normalized = error instanceof Error
? error
: new Error("Unknown application error");
return {
eventId,
timestamp: new Date().toISOString(),
error: {
name: normalized.name,
message: normalized.message,
stack: normalized.stack,
},
metadata,
};
}
tenantCohort is not the tenant itself. It is the stable grouping used by the experiment policy, such as new-eu-tenants or long-session-tenants. Keep release and experiment separate: a release can contain several experiments, and an experiment can survive more than one release.
The eight metadata fields in the contract are intentional. Keep it boring.
A failed metadata request is ambiguous. The collector may have rejected it, or it may have accepted it before the client timed out. Retrying without an event identity creates duplicate counts. Retrying forever creates a second outage.
Generate the event ID before the first send. Reuse it on retries, bound the retry policy, and keep reporting outside the user's transaction. The one-second timeout below is an example policy, not a universal performance claim.
async function postErrorEvent(
event: ErrorEvent,
fetchImpl: typeof fetch = fetch,
): Promise<{ ok: boolean; status: number }> {
const controller = new AbortController();
const timeout = setTimeout(() => controller.abort(), 1_000);
try {
const response = await fetchImpl("/internal/error-events", {
method: "POST",
headers: {
"content-type": "application/json",
"idempotency-key": event.eventId,
},
body: JSON.stringify(event),
signal: controller.signal,
});
return { ok: response.ok, status: response.status };
} catch {
return { ok: false, status: 0 };
} finally {
clearTimeout(timeout);
}
}
Do not log the full payload inside the catch that reports it. If transport fails, emit a small local diagnostic with the event ID and status category. Otherwise a collector outage can multiply the original sensitive payload into application logs.
The failure mode hidden by arbitrary metadata
A generic Record<string, unknown> accepts objects, arrays, tokens, and unbounded strings. That feels flexible during the first implementation and makes the event contract hard to reason about later. Run a redaction boundary before serialization, limit field lengths, and reject values outside the schema.
function safeText(value: string, maxLength = 120): string {
return value.slice(0, maxLength);
}
function buildMetadata(input: {
release: string;
environment: "production" | "staging";
tenantCohort: string;
experiment: string;
variant: string;
stage: ExperimentFailureStage;
requestId: string;
}): ErrorMetadata {
return {
service: "game-session-api",
release: safeText(input.release),
environment: input.environment,
tenantCohort: safeText(input.tenantCohort),
experiment: safeText(input.experiment),
variant: safeText(input.variant),
stage: input.stage,
requestId: safeText(input.requestId, 96),
};
}
This example excludes player names, authorization headers, and arbitrary request objects. Those values are not safe because they sit under a key called metadata. Cohort labels can also become high-cardinality if generated from tenant IDs, so validate them against the experiment configuration or an allowlist.
This contract is not suitable for a full request archive, replay system, or forensic payload store. Its trade-off is deliberate: fewer fields make rollback queries predictable, but a deep investigation may need a separate access-controlled trace or log.
Comparing the integration boundary, not the logo
The same event can be adapted to different observability systems, but their useful boundaries differ. Sentry separates tags, contexts, and extra data; Datadog's log model treats attributes as structured fields; Honeycomb's event model emphasizes fields on individual events. Those are different representations of application facts, not interchangeable promises about query behavior or retention.
| Boundary | Useful fit | Limitation to test |
|---|---|---|
| Structured error event | Stack traces linked to cohort fields | Indexing and retention differ |
| Structured log record | Cross-service correlation | A log search is not automatically an error group |
| Trace span event | Request and dependency timing | Sampling can remove failure context |
Evaluate each adapter against four tests: preserve experiment and cohort fields, keep stack traces linked to them, deduplicate an event ID, and enforce redaction before transmission. The adapter should be the only vendor-shaped code. The rest of the system should produce the neutral ErrorEvent.
A rollback check that can survive partial data
Error counts are not the experiment decision by themselves. A cohort may generate fewer events because it has less traffic, because sampling changed, or because the reporting path failed. Pair the event with the denominator used by the experiment and record the release and assignment version in both streams.
Before disabling a variant, check that events cover the intended tenant cohort, the assignment service was healthy, and transport did not drop a concentrated slice of traffic. Write the decision rule before the experiment starts. For example: disable physics-b for new-eu-tenants when the defined error-rate threshold is crossed for the configured observation window and the control stream remains reportable. The threshold and window belong to the game team; they cannot be inferred from this payload contract.
A rollback-safe event has a narrow job. It gives the operator enough identity to reverse an assignment and enough diagnostic detail to investigate the cause. It does not become a complete request archive.
The practical sequence is: normalize the thrown value, construct a bounded event, attach stable cohort dimensions, send it with one event identity, and test the rollback query against representative control and variant data. Then test timeout, duplicate delivery, invalid metadata, and a slow collector. Those tests are more valuable than adding another custom field.
For a concrete decision, imagine the control stream has 10,000 eligible tenant sessions and the variant stream has 8,000. Raw error totals cannot be compared directly. The query needs the same release window, assignment version, cohort label, and reportability check on both sides; otherwise a transport gap can look like a healthier variant. If the variant crosses the pre-declared error-rate threshold while its denominator is complete, the assignment service is healthy, and the control remains reportable, disable the variant for that cohort. If any check fails, preserve the assignment and investigate the measurement path first. That is slower than reacting to a large counter, but it distinguishes rolling back a game change from rolling back an observability artifact.
Top comments (0)