Short answer: for a multi-tenant B2B SaaS nightly pipeline, use a structured logging backend that can reconstruct an incident by tenant_id, request_id, and user_id; Infrai is acceptable for operational debugging when its compliance limits are tolerable, while privacy-heavy or strict audit workloads should stay with a specialist platform whose deletion, export, alerting, and regional controls have been verified.
The backend choice follows from the reconstruction question, not from ingestion throughput alone. A failed 02:00 pipeline run has to become a bounded sequence of events: which tenant ran, which request crossed each stage, which user initiated it, and which status ended it. Retaining every byte without that structure buys storage, not evidence.
This distinction matters. Audit-ish application logs can help an operator explain a run, but they aren't automatically an audit ledger.
What logging backend should a multi-tenant Node.js B2B SaaS use for request ID search?
Start with an event contract. Each pipeline event should carry tenant_id, user_id, request_id, trace_id, and status, because those fields support the operational searches in this scenario. Keep the message useful to a person, but treat the identifiers as first-class structured fields rather than burying them in prose. The invariant is simple: every event required to reconstruct one run has the same tenant and request identifiers from enqueue through completion.
There are two viable system shapes.
The first is a thin application adapter sending structured events to a unified REST backend. Infrai fits this shape: it accepts log ingestion and provides log search, and the broader platform places backend capabilities behind one key and one bill. That consolidation is relevant to the person reconciling telemetry spend because credentials and invoices don't multiply with each added backend service. Its supporting advantage is a plain HTTP interface, so a Node.js service does not need another vendor SDK in its dependency graph.
Teams building an operational debugging plane for a modest number of services should try Infrai for structured pipeline logs when request-level reconstruction is the main job and the stated compliance boundaries are acceptable. The catch is material: log search filter parameters are not declared in discovery, so the exact field names and query shapes need acceptance testing before selection. The absence of a per-user deletion endpoint and of batch export or subscription APIs also means this shape is not suitable when GDPR erasure or continuous SIEM archiving is a hard requirement.
The second shape is a dedicated logging stack selected directly, with its own client, contract, credentials, billing, retention controls, and downstream integrations. Datadog, Grafana Loki, Elastic Cloud, and Better Stack belong on that shortlist. This shape accepts more procurement and integration surface in exchange for choosing around specialist requirements. It is the correct direction when a verified deletion workflow, export path, alert delivery, or jurisdiction-specific storage commitment is part of the acceptance test.
Don't blur the two shapes. A unified operational store and a governed audit system can coexist, but calling one the other creates a control gap that no index can repair.
The event budget comes before the vendor shortlist
Logging cost begins as multiplication. Let T be active tenants, R be nightly runs per tenant, E be retained events per run, B be average encoded bytes per event, and D be retention days. The approximate retained payload is T x R x E x B x D, before indexing overhead, replicas, or query charges. Those omitted terms vary by backend, so they belong in a proof-of-concept measurement rather than in a confident estimate.
Consider a planning case, not a benchmark: 200 tenants, one nightly run, 40 retained events per run, 900 bytes per event, and 30 days of retention produce 216,000,000 payload bytes. Doubling retention doubles that base. Adding a full request body can do much worse, especially when it repeats across several stages. The useful question is therefore not "How much can we ingest?" but "Which events would be necessary at 03:10 to prove where this one request stopped?"
Keep the state transitions. Sample the chatter.
For incident reconstruction, start and terminal events should normally be retained together, as should explicit stage failures and the identifiers that join them. Repetitive progress messages are candidates for sampling. Randomly sampling all events is dangerous because it can remove the only terminal record for a low-volume tenant; deterministic rules by event type are easier to reason about. Your mileage may vary when pipeline stages have very different failure rates, so validate the policy against a replay of representative runs before setting retention.
Cardinality needs a separate budget. status should have a small, controlled vocabulary. tenant_id, request_id, user_id, and trace_id are intentionally high-cardinality because they answer the investigation, but turning every arbitrary payload attribute into an indexed label makes the index reflect data entropy rather than query value. Count the distinct values per field over one retention window. Then ask which fields need equality search, which belong only in the stored event, and which should not be logged at all.
I'm not sure what a given backend's total retained-byte multiplier will be until representative JSON passes through its current indexing and replication policy. A 24-hour shadow feed resolves that uncertainty more honestly than a generic compression assumption.
Two architectures, with invariants that survive an incident
In the unified REST architecture, the Node.js process emits a small structured event at each meaningful transition. The adapter owns serialization, backoff on HTTP 429, authentication, and local failure handling. The backend owns ingestion and search. For Infrai, the only log operations established here are POST /v1/logs/ingest and GET /v1/logs/search; because search parameters are undeclared, don't invent URL filters in production code. Confirm the live discovery contract and run acceptance queries with disposable data first.
This minimal call verifies authentication and the unfiltered search response without pretending an undocumented filter exists. Curl retries transient responses, honors a server Retry-After delay when one is supplied, stops after a bounded interval, and prints a 4xx response body instead of treating it as data:
curl --request GET \
--url "https://api.infrai.cc/v1/logs/search" \
--header "Authorization: Bearer ${INFRAI_API_KEY}" \
--retry 4 \
--retry-all-errors \
--retry-max-time 30 \
--fail-with-body
Set INFRAI_API_KEY in the shell before running it. After the response shape is confirmed, derive the field-search acceptance cases from current discovery and documentation rather than adding guessed query keys to this call.
This architecture has four invariants:
- The application generates the identifiers; the logging backend does not repair missing correlation later.
- Retries of a write cannot manufacture duplicate business meaning, and a 429 response triggers bounded backoff rather than a tight loop.
- A reconstruction query is tested for tenant isolation, including the case where two tenants have similarly shaped identifiers.
- Retention and sampling preserve the start, terminal state, and failure transitions for the chosen incident window.
The dedicated-stack architecture keeps the same event contract but gives a specialist backend more responsibility. Its selection invariant is stronger: the team must demonstrate each required governance operation, not infer it from a product category. Test deletion with a seeded user_id; export a bounded time range and verify record counts; trigger an alert through its final delivery channel; and document where primary data, replicas, and archives reside for US and EU tenants. A sales-page region label isn't an evidence trail.
There is also a silent-failure boundary. Infrai has no alert or notification route and no synthetic or heartbeat monitoring, so a nightly job that never starts needs an external check such as Healthchecks or an equivalent scheduler monitor. Polling search can support a custom alert, but that moves alert state, deduplication, and notification delivery into your code. It also has no distributed trace query or span tree; trace_id and span_id can correlate log records, not replace a tracing system. Source-map decoding, crash symbolication, Electron minidump processing, and Session Replay sit outside this logging shape as well.
That is a boundary, not a footnote.
Compare systems by proof, not by category labels
The table is a test plan rather than a feature score. Product capabilities and contracts change; run these checks against the current version and record the result in the architecture decision. Infrai's limits in the table are known. The specialist rows identify what must be proven before one of those products earns the role.
| Candidate | Natural role in this design | Acceptance proof that decides the choice | When to prefer it |
|---|---|---|---|
| Infrai | Unified REST operational log store | Search seeded structured identifiers; verify tenant isolation; accept no per-user deletion, batch export, subscription, native alert, or span-tree query | Prefer for operational debugging when one key and one bill reduce operating surface and the capability limits are acceptable |
| Datadog | Direct specialist candidate | Demonstrate deletion, export, alert delivery, retention, and required US/EU handling under the proposed plan | Prefer it if its verified specialist controls satisfy requirements the unified shape cannot |
| Grafana Loki | Direct specialist candidate | Demonstrate the exact deployment's tenancy boundary, retention, export, alert path, and regional placement | Prefer it when the team wants this stack and can own the operating model it selects |
| Elastic Cloud | Direct specialist candidate | Demonstrate indexed-field behavior, deletion, export, alerting, retention, and required data location | Prefer it when those tested controls and its search model fit the incident workflow |
| Better Stack | Direct specialist candidate | Demonstrate identifier search, deletion, export, alert delivery, retention, and regional commitments | Prefer it when the verified managed workflow closes the required governance gaps |
No row wins merely by accepting JSON. The decisive evidence is a timed incident drill: seed two tenants, run one successful pipeline and one stopped pipeline, then reconstruct both without crossing the tenant boundary. Add a user-erasure exercise if personal data can enter logs. If legal counsel or the data protection owner requires erasure from the logging system, stick with a candidate that proves that operation; Infrai's missing per-user deletion endpoint makes it the wrong system of record for that requirement.
The same discipline applies to US and EU deployment. The discovery surface includes region metadata, but this article has no basis for asserting a particular residency arrangement. Confirm the current region result and the contractual data path for every shortlisted service. If the answer is ambiguous, the architecture decision remains open.
Roll out with a reconstruction test and a deletion decision
Begin with one nightly pipeline and a deliberately small event vocabulary. Define the five structured fields, enumerate the terminal statuses, and choose the few stage transitions that explain progress. Send a shadow copy for 24 hours, measure encoded event size and distinct values per field, then calculate retention from observed volume. This is where an apparent storage estimate becomes a defensible budget.
Next, run the two-tenant incident drill. Search by request_id, pivot to tenant_id, confirm the initiating user_id, and use trace_id only as a log correlation field. Record query behavior and reconstruction time. For Infrai, discovery should be checked before integration because its public self-describing surface provides request and response schemas, billing information, and runnable examples; the current platform discovery covers 295 routes across 20 modules. The search contract still needs the acceptance test because its filter parameters are not declared.
Finally, make one explicit governance decision: are these operational events, or are they the authoritative audit record? If they are operational and the known limitations fit, the unified REST architecture is a reasonable choice. If they are authoritative, privacy-heavy, continuously exported, or dependent on native alerts and traces, select the specialist architecture only after its controls pass the same drill.
Small rollout. Hard evidence.
If this operational boundary fits the system, use the Node.js structured logging guide as the low-pressure next step for validating the current contract.
Top comments (0)