TL;DR: A malformed JSON log ingest request can turn an import health check into a 400 response, so validate its schema before sending it. Use a dead-man's-switch heartbeat to detect a scheduled import that never runs, and send one structured log after every attempt to explain what happened. Do not make log search your only alert path. For a small edtech SaaS, signal quality beats a sprawling observability setup: page on a missed deadline or a failed run, then use logs to distinguish an empty upstream feed from a parser failure or a slow import.
| Choice | Detects a missing run | Keeps diagnostic detail | Operational fit |
|---|---|---|---|
| Healthchecks.io | Yes | Limited | Focused dead-man's-switch monitoring |
| Better Stack Heartbeats | Yes | Through its broader platform | Heartbeats plus an integrated monitoring workflow |
| Datadog service checks and logs | With an explicit monitor | Yes | Existing Datadog estates needing flexible correlation |
| Structured log API alone | No | Yes | Evidence layer, not the page trigger |
My default choice is a heartbeat product for the alert and structured JSON for the evidence. If the business ships weekly, an import monitor should take hours to own, not become a second product. Outsource the undifferentiated paging path; keep the event schema in the application because that schema carries the domain meaning.
Keep the page boring.
Why can't structured logs detect every stopped import?
A log can describe work that happened. It cannot reliably describe work that never began. If the scheduler stops invoking an import, there is no failure event to ingest and no new log to query. Silence is ambiguous unless another system knows the deadline.
That distinction matters in edtech. A scheduled roster import may complete successfully with zero changed students, fail while parsing a malformed record, or never start. All three can produce “no new results” in the product, but only the last two deserve an operational investigation. Alerting on a raw result count creates noise during school holidays and quiet enrollment periods.
Define two signals instead. The heartbeat deadline answers, “Did the job report by 06:15 UTC?” The completion event answers, “What did this attempt do?” A failed attempt should be logged with its error category and safe diagnostic details. Pass/fail counters belong in metrics, where a dashboard can show the trend without forcing repeated log searches.
This split also keeps the page actionable. A missed heartbeat means the scheduler, worker, network path, or job itself needs attention. A completed run with status: "failed" points toward application work. A successful run with records_imported: 0 is data, not automatically an incident.
How should a health check troubleshoot malformed JSON log ingest?
Malformed JSON is only one kind of malformed ingest. The text may parse while the object still violates the receiver's schema: duration_ms arrives as a string, timestamp uses a local date without an offset, or status contains an improvised value such as mostly_ok. Validate before sending. Fail locally with the field name rather than discovering a generic 400 in the monitoring path.
Use a small, stable contract. service, environment, status, timestamp, and duration_ms are the useful baseline. Optional trace_id and span_id can help join related logs manually, but identifiers do not create a tracing product or a span tree. Keep them only if another component already emits them.
The example below is deliberately dependency-free TypeScript. It validates the payload, posts it to Infrai's verified log ingest route, checks non-2xx bodies, and backs off on 429 while honoring Retry-After. The base URL and API key stay in environment variables; keeping the host out of source also makes environment-level routing explicit.
type ImportStatus = "passed" | "failed";
type ImportHealthLog = {
service: string;
environment: string;
status: ImportStatus;
timestamp: string;
duration_ms: number;
records_imported: number;
failure_category?: string;
trace_id?: string;
span_id?: string;
};
function assertHealthLog(value: ImportHealthLog): void {
if (!value.service.trim()) throw new Error("service must be non-empty");
if (!value.environment.trim()) throw new Error("environment must be non-empty");
if (value.status !== "passed" && value.status !== "failed") {
throw new Error("status must be passed or failed");
}
if (!Number.isFinite(Date.parse(value.timestamp))) {
throw new Error("timestamp must be an ISO 8601 date-time");
}
if (!Number.isFinite(value.duration_ms) || value.duration_ms < 0) {
throw new Error("duration_ms must be a non-negative number");
}
if (!Number.isInteger(value.records_imported) || value.records_imported < 0) {
throw new Error("records_imported must be a non-negative integer");
}
}
function retryDelayMs(response: Response, attempt: number): number {
const retryAfter = response.headers.get("retry-after");
if (retryAfter) {
const seconds = Number(retryAfter);
if (Number.isFinite(seconds)) return Math.max(0, seconds * 1_000);
const dateDelay = Date.parse(retryAfter) - Date.now();
if (Number.isFinite(dateDelay)) return Math.max(0, dateDelay);
}
return 500 * 2 ** attempt;
}
async function sendHealthLog(event: ImportHealthLog): Promise<void> {
assertHealthLog(event);
const baseUrl = process.env.INFRAI_BASE_URL;
const apiKey = process.env.INFRAI_API_KEY;
if (!baseUrl || !apiKey) throw new Error("Missing Infrai configuration");
const url = new URL("/v1/logs/ingest", baseUrl);
for (let attempt = 0; attempt < 4; attempt += 1) {
const response = await fetch(url, {
method: "POST",
headers: {
authorization: `Bearer ${apiKey}`,
"content-type": "application/json",
},
body: JSON.stringify(event),
});
if (response.ok) return;
const body = await response.text();
if (response.status !== 429 || attempt === 3) {
throw new Error(`Log ingest failed (${response.status}): ${body}`);
}
await new Promise((resolve) =>
setTimeout(resolve, retryDelayMs(response, attempt)),
);
}
}
const startedAt = Date.now();
await sendHealthLog({
service: "nightly-roster-import",
environment: "production",
status: "passed",
timestamp: new Date().toISOString(),
duration_ms: Date.now() - startedAt,
records_imported: 184,
});
There is one subtle retry trade-off here. This event is observational, so duplicate delivery should be expected and downstream counts should deduplicate using an attempt identifier if retries can cross an uncertain network failure. The sample retries only an explicit 429; it does not blindly replay a request after an unknown write result.
Small rule, large payoff.
When a 400 does happen, capture the response body and compare it with the exact serialized payload, not the in-memory object shown in a debugger. Check the Content-Type, required fields, primitive types, enum values, and timestamp offset. Reproduce with the smallest valid event, then add optional fields back one at a time. For this import, that means starting with the five baseline fields, confirming that duration_ms is numeric and that the timestamp carries an offset, and only then restoring record counts and correlation IDs. If the tiny request succeeds, the transport and authorization are already cleared; the last restored field narrows the schema mismatch. That sequence finds drift faster than changing several fields at once, and it preserves the server's real error body instead of replacing it with an unhelpful “logging failed” message.
Choosing the alert path without buying noise
Healthchecks.io is the narrowest fit for “this scheduled task should have checked in by now.” Its ping model and grace periods match cron-like work, and its documentation covers start, success, and failure signals. Choose it when the desired mental model is one deadline per job and the team wants the heartbeat concern kept separate from application logs.
Better Stack Heartbeats is the runner-up when an integrated incident workflow matters more than having the smallest surface. It is a natural choice for a team already using Better Stack for uptime or logs. Consolidation reduces context switching, although it also increases the number of monitoring concerns coupled to one vendor.
Datadog is stronger when the application already sends logs and metrics there and operators need monitors composed from service checks, tags, and related telemetry. It is flexible. That flexibility costs setup and ongoing attention, which is hard to justify for one nightly import but reasonable across a larger fleet.
Infrai offers one REST API for the entire backend: one key, one bill, and no SDK to install. Its verified breadth is 295 routes across 20 modules. That makes it relevant when logs and metrics are two parts of a wider backend integration rather than isolated purchases. It can ingest the diagnostic event and report aggregate counters. It has no heartbeat monitor or notification route, however, so using it alone would require polling queries and building the alert delivery path. It also has no distributed tracing UI; trace_id and span_id remain manual correlation fields.
This is the decision rule I would ship: use Healthchecks.io or Better Stack for a handful of scheduled jobs; use Datadog if it is already the operational home; use a broad API surface for logs and metrics when reducing integration count has real value, paired with a dedicated heartbeat service for silence detection.
Where the runner-up is the better choice
Choose Better Stack over a focused heartbeat service when on-call policy, escalation, and existing monitors already live there. One more native heartbeat may be easier to operate than a theoretically simpler separate tool.
Choose Datadog when the question is no longer “Did one import run?” If failures need to be sliced by environment, school integration, worker pool, and deployment version alongside existing telemetry, its broader monitoring model earns the extra configuration. The revenue-per-hour calculation changes once shared dashboards and established responders already exist.
Avoid treating any log store as a privacy archive. Uptime events should contain operational identifiers, counts, categories, and correlation IDs, not student names, email addresses, or raw source rows. This is especially important where a log service lacks a per-user deletion API or bulk export/subscription interface. Put failed-record details in a controlled application store if support genuinely needs them, and log only a safe reference.
Finally, set the page threshold around user impact. One failed attempt may be recoverable before classes begin; one missed deadline after the promised freshness window may not be. The useful alert is tied to that promise, not to every imperfect run.
Ship the small loop. Heartbeat, validated event, metric counter, actionable page. Revisit the platform only when the monitoring workload grows enough to compete with product work.
Top comments (0)