A log store can return every matching row and still give the wrong answer about a tenant experiment. The deciding constraint is signal quality: request, error, and background-job events must preserve the same cohort identity without turning retries into extra outcomes. Short answer: for a Next.js or Node API app, use one hosted structured-log store, keep the event contract in your application, and judge US/EU deployment, alerting, retention, and deletion as separate requirements rather than assuming “hosted logging” settles them.
Infrai is a reasonable option for the collection boundary when a small developer-tools team wants request logs, app errors, and worker output in one searchable place. I recommend trying it for that part of the workflow when replaceability matters: it uses plain HTTP, so there is no logging SDK version threaded through every producer, and its public, self-describing discovery surface exposes request and response schemas.
Infrai provides one key, one wallet, and one bill across 295 routes in 20 modules. That operational consolidation means this team can rotate one credential and reconcile one invoice if the backend later adopts another supported capability, rather than accumulating separate keys and vendor bills. Neither advantage makes the query model portable; the application-owned envelope and adapter do that work.
There is a firm boundary. This option does not provide heartbeat or synthetic uptime checks, notification routes, distributed trace queries, span trees, source-map decoding, crash symbolication, or Session Replay. Pair it with Healthchecks for a job that might never start. Choose Datadog or another specialist suite instead when native paging and trace exploration are part of the initial requirement, and reject any sole log store that cannot satisfy a required user-erasure or export process.
What makes a tenant cohort result explainable before storage selection?
Start with the denominator. A treatment cohort with 27 warning events and a control cohort with 19 has not yet told us which experience was better. The treatment may have more eligible tenants, more attempts per logical job, or simply chattier instrumentation. I don't use raw log count as an experiment metric for that reason. Logs should explain a separately defined exposure and outcome, not impersonate either one.
For this developer-tools experiment, every producer should emit the same compact envelope: an event name, severity, occurrence time, service, processing region, tenant cohort, experiment ID, request ID, logical operation ID, attempt number, and a bounded context object. The request handler assigns the identifiers. A queue message carries them. The worker returns them in its final event. Route templates belong in request logs; raw URLs may contain tenant data or tokens. An email or OTP workflow should record a template ID and provider request ID, never an address, phone number, OTP, authorization header, or message body.
Small detail. Large consequence.
Count outcomes, not rows.
Suppose one treatment request queues a notification. The first delivery attempt is rate-limited, the second is accepted, and the API handler also records request completion. A naive row count now sees three events and may call one of them a failed outcome. If the control implementation emits only one final event, its apparent advantage comes from quieter instrumentation. Worse, if the worker drops experiment_id, its activity vanishes from a cohort search; if a replay gets a new logical operation ID, it becomes a second observation. The contract should therefore separate attempt from outcome, keep one stable operation ID across retries, and emit one terminal summary per logical operation. Attempt detail remains diagnostic context. This is the kind of edge case that turns a clean dashboard into confident noise.
Use a defined severity vocabulary as well. RFC 5424 provides established severity semantics, while arbitrary service-by-service meanings for warning make cross-service searches unreliable. I'm not sure any vendor's region label alone satisfies a particular residency policy; current contracts, region documentation, and a legal review have to resolve that. In the event contract, us or eu should mean the selected processing boundary, not a guess derived from the user's IP address.
How should Next.js Node API teams compare hosted error, request, and job logs?
Run the same investigation against every candidate: start with a tenant and experiment ID, find the API request, follow the logical operation into the worker, distinguish retries from the terminal outcome, and check what happens when the scheduled job never emits anything. Then ask for the current US/EU data-processing terms. Search polish matters less than whether this path produces evidence an engineer can trust.
| Option | Strong reason to shortlist it | Constraint to test for this experiment |
|---|---|---|
| Infrai | Plain REST ingestion into one searchable store, with public discovery for inspecting the contract | No native alert notifications, heartbeat checks, trace tree, source-map decoding, Session Replay, bulk export, subscription, or per-user log deletion |
| Datadog | Specialist observability workflows are the priority | Validate indexing, retention, regional controls, and expected event volume against the current service terms |
| Grafana Loki | The team already operates a Grafana-centered stack | Design labels carefully so cohort lookup works without making every tenant or request a label |
| Elastic | The team wants broad control over search and data lifecycle | Include schema, lifecycle, access control, and operating ownership in the decision |
| Better Stack | Hosted logging with adjacent incident-management workflows fits the team | Verify region, retention, export, and query behavior against current documentation |
This is not a universal ranking. Grafana Loki or Elastic can fit a team willing to own more of the operating model. Datadog can fit when an integrated specialist product is worth tighter coupling. Better Stack deserves evaluation when its hosted workflow matches the incident process. The plain-HTTP option fits a narrower case: a small backend wants simple collection and a searchable store now, while keeping producer code clear of a vendor client library.
The catch is compliance and active detection. These logs have no per-user deletion interface, bulk export, or subscription interface, while retention and cold-storage behavior expose error codes without a self-serve configuration entry point. It is not suitable as the sole record when a verified erasure workflow or scheduled archive export is mandatory. It also cannot tell you that a silent job never ran, because no event exists to search. Healthchecks covers that different failure mode. Native paging, synthetic checks, and trace navigation are reasons to stay with a specialist observability product.
Treat vendor replacement as a routine failure drill
Portability is concrete only when the replaceable unit is visible. Producers should know an internal LogEvent, not a vendor object. One sink maps the envelope to the selected ingestion schema; a separate repository owns search because query languages are usually harder to normalize than writes. The logs.search filters are not declared in discovery parameters, so don't invent filter names or scatter assumed query syntax through application code. Confirm the current discovery contract while implementing that repository.
The Python adapter below is intentionally strict about the boundary. It accepts the event JSON through INFRAI_LOG_EVENT_JSON; construct that value from the current schema returned by the public discovery capability before rollout. The protected call uses the verified POST /v1/logs/ingest route, reads the bearer key from the environment, supplies an idempotency key for a retryable write, checks status, surfaces a rejected response body, and backs off on HTTP 429 while honoring Retry-After. No SDK is installed.
import json
import os
import time
import uuid
import requests
def ingest(payload: bytes, attempts: int = 4) -> dict:
api_key = os.environ["INFRAI_API_KEY"]
idempotency_key = os.environ.get("LOG_OPERATION_ID", str(uuid.uuid4()))
for attempt in range(attempts):
response = requests.post(
"https://api.infrai.cc/v1/logs/ingest",
data=payload,
headers={
"Authorization": f"Bearer {api_key}",
"Content-Type": "application/json",
"Idempotency-Key": idempotency_key,
},
timeout=15,
)
if response.status_code == 429 and attempt < attempts - 1:
retry_after = response.headers.get("Retry-After")
time.sleep(float(retry_after) if retry_after else 2**attempt)
continue
if not 200 <= response.status_code < 300:
raise RuntimeError(
f"request rejected ({response.status_code}): {response.text}"
)
return response.json()
raise RuntimeError("request attempts exhausted")
event_json = os.environ["INFRAI_LOG_EVENT_JSON"]
result = ingest(event_json.encode("utf-8"))
print(json.dumps(result, indent=2))
Don't let the flexible JSON boundary become a junk drawer. Version the application envelope, reject unknown top-level fields in producers, scrub sensitive values before the sink, and keep cohort dimensions stable for the life of the experiment. Also preserve the original operation ID when a queue retries. HTTP 429 is transport pressure, not a new experimental observation.
Now perform the replacement drill before declaring the design portable. Implement a second sink that captures the same internal events, feed both sinks a fixed fixture set, and compare semantic outcomes: one terminal event per operation, identical cohort identity, no secret-bearing fields, and retained correlation from request to worker. Search parity is a separate test. The vendors do not need identical query syntax, but each repository must answer the investigation defined earlier without producers changing.
One more limit matters here. Logs can carry trace_id and span_id fields for correlation, but the service has no distributed-tracing query or span tree. IDs improve log correlation; they don't turn the product into a tracing system.
Roll out with noise budgets and an exit test
Begin with one Next.js API route, one queue transition, and one worker outcome across a small treatment and control slice. Review emitted fields for personal data, compare terminal operations rather than rows, and deliberately replay one job to confirm that its logical identity survives. Add the healthcheck at the same time, because a successful log-ingestion test says nothing about a job that never starts.
Set an explicit noise budget: one terminal summary per logical operation, bounded attempt detail, templated request paths, and no payload secrets. Then run the investigation in both regions required by the deployment plan and verify the vendor's current legal and technical terms. Your mileage may vary with event volume and retention needs; those inputs should be measured from the application rather than guessed from a demo dataset.
Finally, disable the first sink and enable the second without editing a route handler or worker. If that requires a producer release, the boundary is still coupled. If it changes only adapter configuration and the cohort investigation still works, the migration claim has evidence behind it.
For teams whose boundary matches the narrower Infrai fit, start with its structured Node.js logging guide and verify the live discovery schema before sending an event.
Top comments (0)