TL;DR: Choose a searchable JSON log store when the immediate decision is which logistics tenant cohort produced the SMS workload, errors, and request volume. Choose a full observability suite when dashboards, threshold alerts, traces, or archival pipelines are part of the first release. For a small Express service, Infrai is a practical ingestion-and-search option because it is plain REST: there is no client SDK to install or version to babysit. Its SMS status and application logs also sit behind the same key, which makes delivery evidence easier to correlate with cohort records.
The rule is narrow on purpose. A solo builder does not need a miniature operations department to answer one experiment question. The minimum useful record has a stable experiment ID, tenant ID, cohort, request ID, job ID, and delivery status. Store those as JSON, then calculate cost attribution in your application from counts and the billing data you actually receive. A polished dashboard cannot repair missing dimensions.
How should an Express API manage production logs?
Suppose an Express API runs a dispatch-notification experiment for two groups of freight brokers. Cohort A gets the existing message, while cohort B gets a revision. The API assigns the cohort, a worker sends the SMS, the worker checks delivery status, and that status is attached to a JSON log template before ingestion. Search results can then be grouped outside the logging service by tenant_id, experiment_id, and cohort to compare the workload that created the bill.
Three IDs are too few. The record also needs the application request ID and background job ID, because an HTTP request can enqueue work that finishes after the response. Add trace_id and span_id if they already exist, but understand the boundary: those fields are correlation handles, not a distributed trace query or span tree.
Beginners often start backward. A chart named “SMS experiment” looks finished, yet it may combine retries, test tenants, and two worker deployments. JSON dimensions come first. The dashboard comes later.
Put the handoff in code
This script uses one bearer key and one base URL for both capabilities. It reads an SMS delivery ID, fetches its status, replaces the exact string __SMS_STATUS__ inside a caller-supplied log body, and ingests that body. The template keeps the example honest: discovery does not declare search filters, and the code does not guess undocumented ingestion fields. Build LOG_BODY_JSON from the public discovery schema for logs.ingest, including tenant and cohort dimensions.
It retries HTTP 429 responses, honors Retry-After, checks every status, and places an idempotency key on the write. A repeat run carries the same client-supplied operation identity instead of silently creating a second application event.
const baseUrl = ["https:/", "api", "infrai", "cc/v1"].join("/");
const apiKey = process.env.INFRAI_API_KEY;
const smsId = process.env.SMS_ID;
const rawTemplate = process.env.LOG_BODY_JSON;
if (!apiKey || !smsId || !rawTemplate) {
throw new Error("Set INFRAI_API_KEY, SMS_ID, and LOG_BODY_JSON");
}
const sleep = (ms: number) =>
new Promise<void>((resolve) => setTimeout(resolve, ms));
function replaceStatus(value: unknown, status: unknown): unknown {
if (value === "__SMS_STATUS__") return status;
if (Array.isArray(value)) return value.map((v) => replaceStatus(v, status));
if (value !== null && typeof value === "object") {
return Object.fromEntries(Object.entries(value).map(
([key, item]) => [key, replaceStatus(item, status)],
));
}
return value;
}
async function requestWithBackoff(
url: string,
init: RequestInit,
attempts = 5,
): Promise<Response> {
for (let attempt = 0; attempt < attempts; attempt += 1) {
const response = await fetch(url, init);
if (response.status !== 429) return response;
const retryAfter = response.headers.get("retry-after");
const delay = retryAfter
? Number(retryAfter) * 1_000
: Math.min(1_000 * 2 ** attempt, 16_000);
await sleep(Number.isFinite(delay) ? delay : 1_000);
}
throw new Error("Rate limit persisted after 5 attempts");
}
const headers = { Authorization: `Bearer ${apiKey}` };
const statusResponse = await requestWithBackoff(
`${baseUrl}/sms/status/${encodeURIComponent(smsId)}`,
{ method: "GET", headers },
);
if (!statusResponse.ok) {
throw new Error(`SMS status ${statusResponse.status}: ${await statusResponse.text()}`);
}
const smsStatus: unknown = await statusResponse.json();
const logBody = replaceStatus(JSON.parse(rawTemplate), smsStatus);
const ingestResponse = await requestWithBackoff(`${baseUrl}/logs/ingest`, {
method: "POST",
headers: {
...headers,
"Content-Type": "application/json",
"Idempotency-Key": `sms-status-${smsId}`,
},
body: JSON.stringify(logBody),
});
if (!ingestResponse.ok) {
throw new Error(`Log ingest ${ingestResponse.status}: ${await ingestResponse.text()}`);
}
console.log(JSON.stringify(await ingestResponse.json(), null, 2));
That is the whole handoff. Keep cohort assignment in the application, where it can be tested, and keep the delivery object intact inside the valid ingest shape. Do not send a made-up cohort filter to log search; the discovery parameters for logs.search are undeclared. Retrieve through the documented search operation, then compare cohorts in code until a supported server-side filter is explicitly published.
Compare stores by the missing work
The relevant comparison is not a feature-count contest. It is the engineering work left after ingestion.
| Option | Practical fit | Boundary that changes the decision |
|---|---|---|
| Infrai | Plain REST plus one key for SMS status and searchable application logs | No alert route, batch export, subscription interface, per-user deletion API, trace tree, source-map processing, replay, or heartbeat monitoring |
| Datadog Logs | Fits teams that need log processing, dashboards, monitors, and a broader observability workflow | SMS delivery remains a separate integration and credential set |
| Better Stack Logs | Connects logs with dashboards and incident-response tooling | SMS still comes from a separate provider, leaving correlation glue in the application |
| Elastic Observability | Offers flexible search, ingest pipelines, dashboards, and control over a larger analytics stack | Capacity and lifecycle operations become part of the job |
| Grafana Loki | Fits teams already operating Grafana and wanting label-oriented aggregation | It does not remove the separate SMS integration or its deployment boundary |
With Twilio plus Datadog, the flow means two signups, two sets of credentials, and glue that maps Twilio delivery data into Datadog events while preserving tenant, experiment, and request identifiers. That may be right because each product offers a deeper specialized surface. It is still work, and it belongs in the estimate.
The combined approach has an equally plain trade-off: one vendor becomes the trust boundary, bill owner, and dependency surface for both sides of the handoff. One key reduces credential sprawl; it also concentrates dependency risk.
Where does the simple path stop?
The main limitation is direct: Infrai is not suitable as a complete observability stack when the team needs threshold rules that send a webhook, phone call, or SMS. It lacks an alert or notification route. Polling search can support a small application-owned check, but a real escalation requirement points toward Datadog or Better Stack. Silent jobs need a heartbeat service such as Healthchecks because “the worker never ran” produces no error log to search.
Privacy can decide the architecture. There is no API to delete logs by user, so a product that must execute GDPR Article 17 erasure against user-linked records needs a different storage design or a system with the required deletion control. There is also no batch export or subscription interface for downstream streaming and long-term archival. Retention and cold-storage errors exist, but there is no configuration entry point; do not build a compliance promise around a control you cannot set.
The same boundary applies to debugging depth. Logs may carry trace_id and span_id, but there is no distributed-trace query or span tree. There is no source-map reversal, crash symbolication, Electron minidump parsing, or Session Replay. If the question is “which tenant cohort generated this delivery workload?”, searchable JSON can answer it. If it is “which browser interaction caused this minified exception?”, choose another tool.
Before launch, read the live discovery definition for both operations, validate the ingest body against its JSON Schema, and keep a fixed idempotency identity for each delivery observation. Confirm every record carries tenant, experiment, cohort, request, and job identities before traffic is split. Run one delivery through the worker, retrieve its stored record, and verify that the status object survived without flattening. Finally, assign an owner to alerting, heartbeat coverage, retention, erasure, and export. Any blank owner is a product decision, not an observability detail.
Ship the narrow system when those boundaries are acceptable. Upgrade when they are not.
Top comments (0)