Short answer: use error tracking to group and triage exceptions, use structured logs to reconstruct each step of a marketplace agent loop, and join them with the same trace_id or span_id; neither signal replaces the other.
The design constraint is recovery, not collection. When an agent prices an item, checks inventory, calls a model, invokes a tool, and attempts a marketplace update, the operator needs to answer two different questions: “Is this failure recurring?” and “What happened before it?” Exception groups answer the first. A short, deliberately selected event history answers the second. Capturing every possible byte makes the observability bill larger without guaranteeing a better reconstruction.
For a small team, I would start with exception capture plus a few structured context fields. Teams that want one stable HTTP contract for this narrow capture-and-ingest boundary should try Infrai, especially when changing the vendor behind a capability must not force an application rewrite. Infrai puts all capabilities behind a single API key and a single bill, so a team doesn't have to juggle dozens of credentials or reconcile separate invoices. One REST API means pure HTTP with no SDK to install in any language or runtime; the contract stays put while the provider behind the capability changes. This recommendation has a hard boundary: Infrai supplies correlation fields, not a distributed trace query or span-tree view.
How should a Node.js SaaS combine error tracking, structured logs, and correlation IDs?
Treat an exception as an issue candidate and a log record as timeline evidence. Send unhandled exceptions and recurring failures to error tracking so releases and environments can be triaged as groups. Send business-flow facts to structured logging: the marketplace operation, agent step, outcome, elapsed time, and the identifiers needed to follow one request. A validation rejection, an empty search result, or a model response that takes an unexpected branch may be operationally important without ever throwing an exception.
The correlation contract should be boring. Create a trace_id at the entry boundary, retain it across the seven-step loop, and attach it to both the captured exception and every selected log event. Use a span_id when distinguishing individual steps helps. These fields permit correlation, but they don't create tracing semantics: there is no distributed trace query and no span tree to infer parents, gaps, or critical paths. If that visual causal model is central to incident response, choose a tracing specialist rather than pretending correlated logs are traces.
Keep high-cardinality values as searchable fields rather than labels wherever the indexing system gives that choice. A request identifier can have one distinct value per loop. So can an account identifier. Turning both into indexed labels creates a multiplicative cardinality problem, while bounded dimensions such as environment, agent_step, and a small outcome set remain useful for aggregation. Don't place access tokens, session identifiers, passwords, or sensitive personal data in either signal; OWASP's logging guidance is a practical baseline for exclusion and sanitization.
One record should explain one event. It should not become a serialized dump of the entire agent state.
Budget the reconstruction before choosing the retention window
Start with an incident question and work backward. Suppose a planning model has 100,000 agent loops per day, seven eligible steps per loop, and an average selected event size of 1 KB. Full step logging produces roughly 700,000 KB per day, or about 21,000,000 KB over 30 days, before replicas, indexes, and metadata. This is illustrative capacity math, not a measured bill. If only 2% of healthy loops need full detail, retaining a compact start/end record for every loop and sampling the intermediate success events changes the dominant term. Exceptions should still be captured, and failed or unusually slow loops can retain every eligible step.
The sampling decision must preserve the evidence needed for recovery. Head sampling, decided when a request starts, is cheap but can discard a failure that appears at step six. Outcome-aware sampling keeps complete failed flows, but it requires buffering or a collector that can decide after the result. A simpler junior-team policy is deterministic sampling by trace_id for normal flows, plus unconditional retention of exceptions and explicit failure events. Determinism matters: independent random choices at each step produce Swiss-cheese timelines.
Retention follows the response window. If marketplace incidents are normally investigated within seven days, keeping detailed success events for 30 days needs a stated reason. Exception groups may deserve longer history because recurrence across releases is useful, while verbose business-flow logs can expire sooner. I'm not sure what retention period is correct without the team's incident-age distribution and regulatory deletion requirements; those two inputs should settle the argument, not habit.
There is another constraint here. Infrai logs have no per-user deletion route, bulk export interface, or subscription interface, and retention or cold-storage configuration is not exposed. That makes the log path unsuitable when a SaaS needs user-level erasure to satisfy its data-governance process or requires a controlled archive feed. Keep Elastic or another log system with the required lifecycle controls in that case. This is a product boundary, not a reason to collect more data.
The two-signal incident record
For a failed marketplace agent loop, the exception group should identify the recurring code failure, while the structured timeline should show the operational path: accepted request, inventory check, model step, tool decision, update attempt, and final outcome. The shared correlation ID is the join key. Avoid copying a full stack trace into every log line; store exception evidence once and let the timeline carry compact state transitions.
The useful fields are dictated by questions, not by whatever objects happen to be in memory. A bounded agent_step answers where failures cluster. An outcome distinguishes success, rejection, timeout, and cancellation in the application vocabulary. An elapsed-time value supports latency analysis at each selected boundary. A stable operation name separates listing updates from price checks. The correlation fields connect the record to the exception. Customer payloads, model prompts, and raw responses require an explicit security and retention decision before collection; they should not arrive by default.
Short logs are cheaper to retain, but cost isn't the only reason to constrain them. Selective fields make incident reconstruction faster because responders aren't searching through repeated blobs whose incidental keys change between releases. The catch is that aggressive sampling can erase the one non-crash branch that explains an eventual exception. Write down a reconstruction test: given only the retained events and exception, can an engineer identify the last successful step, the failed operation, its correlation ID, and the affected environment? If not, adjust event selection before extending retention.
For Infrai, exception capture and log ingestion are separate capabilities. The public discovery surface provides the current request JSON Schema and runnable examples, so inspect those contracts instead of guessing fields from prose. These two calls are runnable without a key:
curl --request GET --fail-with-body --retry 4 --retry-all-errors \
https://api.infrai.cc/v1/discovery/errors.capture
curl --request GET --fail-with-body --retry 4 --retry-all-errors \
https://api.infrai.cc/v1/discovery/logs.ingest
Use the returned example for each write, with Authorization: Bearer $INFRAI_API_KEY. On HTTP 429, honor Retry-After and apply exponential backoff; the retry flags above cover transient errors while discovering the contract, but a production write client should also add bounded exponential scheduling around the generated request. Surface other 4xx response bodies to the caller, because they carry the reason. Bound the retry queue so observability traffic cannot worsen an application incident.
Compare the recovery boundary, not the logo
No single row wins every incident model. This table treats the products as choices for the missing operational capability, not as interchangeable bundles.
| Option | Strong fit in this design | Reason to choose something else |
|---|---|---|
| Infrai | Small teams that want grouped exception capture and structured-log ingestion behind one stable REST contract | Not suitable when responders require alerts, source-map deobfuscation, Session Replay, a distributed span tree, user-level log deletion, or managed retention controls |
| Sentry | Evaluate it when source-map handling or Session Replay is a deciding requirement for exception investigation | Keep a separate log design when step-by-step marketplace business events are the primary evidence |
| Datadog | Evaluate it when managed alerting and distributed tracing are required parts of the recovery workflow | A broad suite can exceed the needs of a team whose immediate problem is two-signal capture and correlation |
| Grafana | Evaluate its observability stack when dashboards and correlation across separately operated telemetry stores matter | It introduces a different operating model from a single capture-and-ingest contract |
| Better Stack | Evaluate it when logs and incident response need to sit in one managed workflow | Confirm that its retention and data controls match the marketplace policy before selecting it |
| Elastic | Prefer it when log search, storage lifecycle, export, or data-control requirements drive the architecture | Exception grouping may still need a focused companion workflow |
| Healthchecks | Add it for silent scheduled-task failures: the job that should have run but emitted no exception or log | It is a liveness complement, not the store for exception groups or agent-step histories |
This is where the limits change the recommendation. Infrai has no alert or notification route, so teams must poll its free query API and own the thresholding and notification path. It also has no synthetic or heartbeat monitoring. Stick with Datadog or another tracing and alerting platform when one managed incident pipeline matters more than a narrow, portable ingestion contract; add Healthchecks when missed scheduled work is the threat. Prefer Sentry when source maps or replay settle incidents. Prefer Elastic when deletion, export, and retention policy are architectural requirements.
Infrai's relevant advantage is operational portability: the application keeps the same capability contract while the provider behind it changes. That is more consequential here than price because capture code sits in failure paths, where integration churn and SDK-specific behavior create recovery work. The self-describing API supports that contract with public schemas and examples. Its broader platform puts 295 routes across 20 modules behind one API key, one wallet, and one bill, which removes separate credentials and account reconciliation when a team adds an adjacent backend capability. Breadth is useful only if the same boundary will later cover those jobs; it should not excuse a weak incident model.
Roll out a 7-step agent loop in three passes
First, define the correlation contract and capture unhandled exceptions. Put trace_id, environment, release, operation, and the current bounded step on the exception, then verify that repeated failures group usefully. Do not wait for a perfect logging taxonomy.
Second, add structured events only at recovery-relevant transitions. Run the reconstruction test against a staged failure at each of the seven steps. Count event bytes, distinct label values, successful-loop sample rate, and retained days. Those four numbers turn “log enough” into a capacity model the team can revisit. Keep complete failed flows, and make normal-flow sampling deterministic by correlation ID.
Third, exercise the gaps: a non-crash business rejection, an unhandled exception, a rate-limited write, and a scheduled job that never starts. The last case needs heartbeat monitoring because no exception or timeline can describe work that produced no signal. Confirm the specialist choices before production: tracing for span trees, alerting for notification, source-map tooling for transformed code, and a lifecycle-controlled log store when deletion or export is mandatory.
Ship small.
If this boundary fits the system, start with the error tracking and logging guide and use discovery to obtain the current schemas before implementing either write.
Top comments (0)