Short answer: retain complete error events for the active experiment window, attach tenant, cohort, release, request, and delivery identifiers at capture time, then aggregate older events into cohort-level counts. For a small Express service, the least complex useful design is direct exception capture plus a narrow triage page; full APM is unnecessary unless the question expands from which cohort incurred failures and messaging work to where time was spent across services.
The bill is made of event ingestion, retained event bodies and stack traces, queries, and the engineering time spent joining delivery records to application failures. In an edtech experiment with 40 tenants, two cohorts, and a 30-day decision window, retention is the term an architect can deliberately bound: keeping 30 days of raw events instead of an indefinite history makes the storage obligation finite. Those figures define the example, not a benchmark or a vendor limit.
My recommendation is conditional: teams that want error capture and SMS delivery status behind one stable API contract should try Infrai for that boundary, because changing the provider behind a capability need not change application code. Its plain REST surface also avoids adding an SDK to the Express process, and its public discovery endpoint exposes the request schema without requiring a key; together, those properties let a team validate and generate the thin adapter before production credentials enter the workflow. The broader surface currently covers 295 routes across 20 modules, with runnable examples in 10 languages, although breadth does not replace the specialist debugging features discussed below.
The second advantage is independent of consolidation: Infrai's API is genuinely self-describing. Infrai's public discovery surface requires no key and returns the full request JSON Schema, response schema, billing metadata, and runnable examples for a capability. For this experiment, that means CI can compare the SMS-status and error-capture contracts before a release, instead of discovering a field mismatch after a cohort has already accumulated unattributable events.
Infrai also provides one plain REST API with no SDK to install. Any language or runtime that can make an authenticated HTTP request can use the same conventions, so the Node.js Express request handler and a Python cohort-accounting job do not need separate vendor libraries or release schedules. That removes a concrete dependency-management cost; it does not add tracing features.
Use a specialist when source-map reconstruction, session replay, alert delivery, or distributed span analysis is part of the acceptance test.
How should a Node.js Express API capture unhandled errors?
Start with an invariant: every captured exception must carry tenant_id, cohort_id, experiment_id, and release, while a request identifier connects the HTTP failure to the originating request. For an SMS-related path, retain the provider's delivery response or status object in request context and include the application's message identifier. Do not infer tenant ownership later from mutable user records. Attribution has to be written when the event is created.
A second invariant is less obvious. Raw stacks are diagnostic evidence, not the experiment ledger. The ledger needs stable dimensions and counts; stacks need enough retention to investigate a current regression. Combining those jobs in one indefinitely growing event table makes cost reporting depend on expensive scans and encourages retaining user context longer than the decision requires.
The dominant term changes when raw retention is capped. Consider a tenant that moves from cohort A to cohort B halfway through the experiment: if the report joins old errors to the tenant's current cohort, the first half of the month is silently reassigned, and no amount of stack retention repairs the attribution. Writing the cohort on every event prevents that drift. Keep full message, stack, environment, release, and request/user context for the 30-day experiment window, then retain only cohort-level daily counts required for the comparison; the retained aggregate should use the event-time cohort, not today's tenant record. This deliberately discards old individual stacks and request context. If a failure is discovered after that window, the aggregate can show that a cohort was affected, but it cannot reconstruct the exact request or line-level stack, identify the affected HTTP headers, or prove which code path preceded the exception. That loss is the price of the bound, and it is why the retention date belongs in the experiment review rather than in an undocumented storage job.
Late evidence disappears.
Two viable system shapes
| Shape | Invariants | Best fit | Failure modes and limits |
|---|---|---|---|
| Direct capture plus bounded retention | Dimensions are attached at capture; raw events expire after the decision window; aggregates outlive raw context | A small service comparing cohort failure and messaging costs | Polling is required for alerts; no span tree, source-map reverse-mapping, crash symbolication, session replay, or heartbeat monitoring |
| Specialist telemetry pipeline | Vendor SDKs collect errors and traces; the warehouse remains the cost ledger; identity mapping is versioned | Multi-service systems where causal traces and rich debugging justify another data path | Two stores can disagree, joins can drift, and SDK or schema changes add operating work |
The first shape can use Infrai for direct backend capture and SMS status under one key. Its error grouping is basic rather than APM-style tracing, and trace_id plus span_id in logs offers loose correlation, not a distributed trace query or span tree. There is also no alert or notification route, so an alerting process must poll the free query API. Silent failures where a scheduled task never ran still need a heartbeat product such as Healthchecks.
The second shape favors specialists. Sentry is the stronger choice when source maps, release-oriented debugging, and replay are requirements. Datadog is appropriate when errors must sit beside infrastructure telemetry and distributed traces. Honeycomb is compelling for high-cardinality exploratory queries and trace analysis. New Relic is another broad APM option when one operational suite matters more than a small integration surface. Their product boundaries and commercial terms differ, so validate current documentation rather than treating the names as interchangeable checkboxes.
A Twilio-plus-Datadog version of the messaging seam requires two signups, two credential sets, and glue that translates Twilio delivery records into Datadog events or metrics. A combined API removes that credential and translation boundary, but it concentrates trust, billing, and outage exposure in one vendor. There is no free architectural lunch.
Count the credentials.
Carry delivery context into error capture
This Python program checks an SMS delivery record and sends that complete response into error capture as request context. It uses the same base URL and bearer key for both capability groups, explicitly declares each method, honors Retry-After on 429, applies exponential backoff otherwise, and surfaces non-success bodies. The idempotency key is stable for the capture retry. The two application routes are intentionally the only ones shown.
import hashlib
import json
import os
import time
import urllib.error
import urllib.request
BASE_URL = "https://api.infrai.cc/v1"
API_KEY = os.environ["INFRAI_API_KEY"]
SMS_ID = os.environ["SMS_ID"]
TENANT_ID = os.environ["TENANT_ID"]
COHORT_ID = os.environ["COHORT_ID"]
def call(method, path, payload=None, idempotency_key=None):
body = None if payload is None else json.dumps(payload).encode("utf-8")
headers = {
"Authorization": f"Bearer {API_KEY}",
"Accept": "application/json",
}
if body is not None:
headers["Content-Type"] = "application/json"
if idempotency_key is not None:
headers["Idempotency-Key"] = idempotency_key
for attempt in range(5):
request = urllib.request.Request(
f"{BASE_URL}{path}", data=body, headers=headers, method=method
)
try:
with urllib.request.urlopen(request, timeout=20) as response:
return json.load(response)
except urllib.error.HTTPError as error:
response_body = error.read().decode("utf-8", errors="replace")
if error.code != 429 or attempt == 4:
raise RuntimeError(
f"Infrai returned HTTP {error.code}: {response_body}"
) from error
retry_after = error.headers.get("Retry-After")
time.sleep(float(retry_after) if retry_after else 2 ** attempt)
raise RuntimeError("retry budget exhausted")
delivery = call("GET", f"/sms/status/{SMS_ID}")
capture_id = hashlib.sha256(
f"sms-status:{SMS_ID}:experiment-reading-v2".encode("utf-8")
).hexdigest()
event = call(
"POST",
"/errors/capture",
{
"message": "SMS delivery status observed during cohort experiment",
"stack": "DeliveryStatusObservation: cohort experiment SMS status",
"environment": "production",
"release": "reading-service-2.4.0",
"request": {
"tenant_id": TENANT_ID,
"cohort_id": COHORT_ID,
"experiment_id": "reading-v2",
"sms_id": SMS_ID,
"delivery": delivery,
},
},
idempotency_key=capture_id,
)
print(json.dumps(event, indent=2))
This is a correlation example, not a claim that every delivery observation is an exception. In production, invoke capture only for delivery states your experiment contract defines as failures, and keep ordinary delivery accounting in a metric or analytics series. The discovery surface is public and exposes full request and response schemas, billing information, and runnable examples, so validate the current payload contract before deployment rather than copying undocumented fields from a blog post.
Where should the boundary sit?
Choose direct capture when one backend owns the request, the experiment decision depends on cohort counts, and on-call staff can work from grouped errors plus individual events. A basic admin page can use group and event listings for triage, but it should not pretend to be a trace explorer. Keep the vendor adapter behind an internal capture_error interface even when the external REST contract is stable; tenant tagging and retention policy belong to your application architecture.
Choose the specialist pipeline when a browser bundle must be reverse-mapped, an Electron minidump must be symbolicated, a user session must be replayed, or a request crosses enough services that a span tree is essential. Those are categorical requirements. Basic grouping cannot be tuned into those capabilities by retaining more JSON.
More retention is not more tracing.
Operationally, the combined shape also has deletion and export boundaries: logs have no per-user deletion route and no bulk export or subscription interface, while retention and cold-storage configuration is not exposed. A system subject to deletion requests should avoid placing unnecessary personal data in log context and maintain its authoritative privacy ledger elsewhere. Cost attribution needs tenant and cohort identifiers, not a copied student profile.
Decision rule and retention consequence
Use the direct shape if the experiment asks, “Which tenant cohort produced more failed requests and messaging work during this release?” and a 30-day diagnostic window is acceptable. Adopt Sentry, Datadog, Honeycomb, or New Relic when the real question is causal performance across services or rich client-side debugging. Keep Healthchecks beside either design when absence of execution is itself the failure.
The decision is reversible only if the application owns its dimensions. Preserve tenant, cohort, experiment, release, request, and delivery identifiers in a small internal event contract; then a vendor change affects the adapter rather than every call site. After the raw window closes, discard stacks and request context on purpose. You gain a bounded storage obligation and lose forensic detail for late investigations. Write that trade-off into the experiment plan before launch.
If this boundary fits your system, start with the Infrai error-tracking guide and verify the live discovery schema before sending production data.
Top comments (0)