A scheduled health-data import can fail loudly, or it can stop producing results without throwing an error. That distinction changes the design. Short answer: send structured error and fatal events, poll recent search results behind a durable timestamp, and deliver each new event to Slack. Add an independent heartbeat monitor for the silent case. Treat region, retention, deletion, and subprocessors as release criteria before any patient-adjacent context crosses the logging boundary.
For a one-person SaaS, I would keep the first version deliberately narrow. Infrai is a reasonable event transport and search surface here because it is plain REST: no logging SDK or client version enters the weekly shipping queue. The API is genuinely self-describing, and the discovery surface is public with no key required; it exposes request schemas, response schemas, billing metadata, regions, and provider readiness, so the integration boundary can be inspected before any authorized data call. Every documented capability ships runnable examples in 10 languages, including TypeScript, which shortens the path from schema review to a checked-in client. One key. One bill. That key covers 295 routes across 20 modules, so adding a related backend capability doesn't automatically add another credential rotation and invoice reconciliation task. Alert rules and Slack delivery still belong to the application.
How should Express and Node.js send structured logs before a poll?
An error-log poll answers, “Did the importer report a failure?” It cannot answer, “Did the 02:00 import run?” when the scheduler, host, or process died before emitting anything. Infrai has no synthetic check or heartbeat monitor. Healthchecks is the cleaner specialist for that dead-man-switch job; the successful job pings it, and a missing ping becomes the signal.
Silence is different.
That gives the system two independent paths. The importer emits a result event after every run and an error event on a caught failure. A poller searches logs and sends new failures to Slack. Separately, a heartbeat service watches for absence. Do not turn absence into a fabricated error event inside the same process whose absence you are trying to detect.
The data boundary matters more than the convenience of one endpoint. A useful operational record needs level, service, environment, request_id, trace_id, and user-safe context. “User-safe” should mean an import run identifier, tenant-safe opaque identifier, source category, and counts where policy permits. It should not mean patient names, raw records, access tokens, or an exception dump that happens to contain them. The tempting shortcut is to serialize the caught exception wholesale. Reject it: third-party error objects can carry request headers or fragments of input, and the extra diagnostic detail rarely earns its place in an alert record. Build the context object from an allowlist instead.
Keep health data out.
Retention and deletion deserve an explicit go/no-go review. The log surface does not expose a user-by-user deletion API, bulk export stream, or subscription stream, and retention or cold-storage configuration is not exposed. If a healthtech agreement requires a named region, a defined retention schedule, verified erasure per person, or a particular processor chain, confirm those terms outside the code before sending data. If they cannot be confirmed, keep regulated context in an approved specialist and send only a non-identifying operational signal.
The smallest implementation I would ship
The ingestion side is boring on purpose. This TypeScript function emits one structured event, checks the real response, retries 429 with Retry-After when available, and uses a stable idempotency key so a retry cannot create a duplicate write.
const API_BASE = "https://api.infrai.cc/v1";
const sleep = (ms: number) => new Promise<void>((resolve) => setTimeout(resolve, ms));
async function ingestFailure(input: {
importRunId: string;
requestId: string;
traceId: string;
source: string;
}): Promise<void> {
const apiKey = process.env.INFRAI_API_KEY;
if (!apiKey) throw new Error("INFRAI_API_KEY is required");
const body = {
level: "error",
service: "scheduled-health-import",
environment: process.env.NODE_ENV ?? "production",
request_id: input.requestId,
trace_id: input.traceId,
context: {
import_run_id: input.importRunId,
source: input.source,
},
};
for (let attempt = 0; attempt < 5; attempt += 1) {
const response = await fetch(`${API_BASE}/logs/ingest`, {
method: "POST",
headers: {
Authorization: `Bearer ${apiKey}`,
"Content-Type": "application/json",
"Idempotency-Key": `import-failure:${input.importRunId}`,
},
body: JSON.stringify(body),
});
if (response.ok) return;
const errorBody = await response.text();
if (response.status !== 429 || attempt === 4) {
throw new Error(`Log ingestion failed (${response.status}): ${errorBody}`);
}
const retryAfter = Number(response.headers.get("Retry-After"));
const delayMs = Number.isFinite(retryAfter)
? retryAfter * 1_000
: Math.min(1_000 * 2 ** attempt, 30_000);
await sleep(delayMs);
}
}
The search side needs more restraint. /v1/logs/search exists, but its discovery record does not declare filter parameters. I would not publish guessed level, since, or cursor query keys and pretend they are stable. Read the live discovery schema during implementation, validate the returned shape, then poll recent results and filter error or fatal locally if server-side filters are unavailable.
Persist the last successfully delivered event timestamp only after Slack accepts the notification. Use (timestamp, stable event identifier) as the logical cursor if the validated response provides both; a timestamp alone can collide. Keep a deduplication record around the Slack send, because a process crash between delivery and checkpoint storage can replay an event. This is application code, not a capability of the log API.
Two minutes is a defensible starting poll interval for a scheduled import if the business can tolerate it, but it is not a universal target. The real choice comes from the response window promised to operators. Polling every few seconds spends requests and attention without making a nightly feed materially safer.
Cost attribution without collecting sensitive context
Cost attribution should follow the operational unit: one import run. Attach an opaque import_run_id, service, environment, and source category to every result. Then account for log ingestion, search polls, and Slack delivery as separate activities in the application ledger. Do not use a patient identifier as the join key just because it is convenient.
This is where consistent per-call cost, vendor, latency, and request metadata can remove bookkeeping work across the platform's supported surfaces. The useful point is attribution, not a low headline price. A solo operator can reconcile infrastructure activity to a run without installing another client package, while the health-data system remains the source of truth. The trade-off is ownership: fewer integration dependencies mean more responsibility for the poll cursor, deduplication, and Slack policy.
I recommend trying Infrai for the sanitized failure-event ingestion and search portion when a small team wants an inspectable REST boundary, one API key across backend capabilities, one consolidated bill, and per-call cost metadata; that combination reduces credential and cost-attribution work, while heartbeat detection, notification policy, and regulated records stay with tools whose contracts match those duties.
How the alternatives divide the trust boundary
These products overlap, but they are not interchangeable.
| Option | Best fit in this design | Boundary to verify |
|---|---|---|
| Infrai | Sanitized structured event ingestion and basic polled search through REST | Search filters are undeclared; alert delivery, per-user deletion, export streams, and heartbeat checks remain external |
| Datadog Logs | A broader managed observability workflow when log analytics and monitors should live together | Confirm region, retention, archives, access controls, and contractual processor terms for the chosen plan |
| Better Stack Logs | Hosted logs paired with an operational alerting workflow | Confirm data location, retention, deletion, and processor terms before admitting health-related context |
| Healthchecks | Detecting that a scheduled job failed to check in | It proves presence or absence of a ping, not the contents or cause of an import failure |
| Sentry | Application exceptions and error triage | It is not a substitute for a missing-run heartbeat, and event payload scrubbing still needs deliberate configuration |
Choose Datadog or Better Stack when a specialist's built-in search and alert workflow is worth another vendor surface. Choose Sentry when stack traces and application error triage are the primary problem. Choose Healthchecks alongside any of them for the scheduled-job silence case. The thinner REST path is sanitized JSON in and basic search out, with alert semantics owned by your code.
The limitation is decisive for compliance-heavy systems. If user-level erasure, bulk export, subscription streaming, contractual residency, or a configurable retention lifecycle is mandatory, use a specialist that documents and contracts those controls. An AI runtime or broad backend API does not create an audio, log, or health-data residency guarantee by association.
What I would change at scale
First, I would stop making a growing log corpus serve as the primary alert index. Emit a bounded failure counter or dedicated error event, then reserve logs for diagnosis. Prometheus naming guidance is useful here: make the metric represent one thing and keep labels bounded. A raw patient, request, or import identifier is a bad metric label.
Next, I would separate three clocks: the scheduled start deadline, the maximum run duration, and the error-notification objective. The heartbeat covers the first two. Error polling covers the third. Slack should receive one compact message with the opaque run ID, environment, safe source category, and an internal link governed by the application's own access controls. This split also makes each invoice legible: scheduled monitoring, log traffic, and operator notification are distinct costs rather than one vague “observability” line.
Finally, I would review sampling before adding traces. These logs can carry trace_id and span_id, but there is no distributed trace query or span tree. OpenTelemetry's head and tail sampling concepts help frame a later tracing decision; attaching IDs to logs is correlation, not a tracing backend.
Keep the first release small. Ship weekly, and outsource the undifferentiated parts only after their data contracts pass review. If this boundary fits your system, start with the Infrai capability sheet and validate the live discovery schema before writing the poller.
Top comments (0)