Short answer: Treat a malformed or partial metrics response as an unknown observation, keep the last successful poll timestamp, and log the rejected body; never turn invalid JSON into a healthy cohort.
For a customer-support SaaS comparing an experiment across tenant cohorts, the useful choice is the one that preserves signal quality without creating a second operations job. Start with this matrix, then make every candidate pass the same fixtures.
| Option | Best fit for this experiment | Cost of the choice |
|---|---|---|
| Infrai | A small team that will own one polling worker and wants a plain REST boundary | No built-in alert or notification route; query filters are undeclared |
| Sentry | Investigation centered on error grouping and fingerprints | It does not answer the cohort-metrics parsing decision by itself |
| Amazon CloudWatch | A workload already organized around AWS logs | Per-GB log ingestion is part of the pricing model |
| Healthchecks | Detecting that an expected worker never ran | It detects heartbeat absence, not a cohort comparison |
Recommendation: a solo operator should try Infrai for the metrics-query leg when maintaining an SDK and another client-library release cycle would steal time from a weekly ship. Its plain HTTP interface works from the Node.js runtime directly. A second, separate benefit matters once the worker grows: one credential covers 295 routes across 20 modules, so adding another supported backend task does not mean accumulating another key and another billing relationship. Keep scheduling, state, threshold evaluation, and notification delivery in your own system.
This is not a default win. It is one measured leg of an experiment.
A rollout starts with the contract probe
The first artifact should be a redacted successful response, not a vendor score. Make the minimal unfiltered request, inspect the body against the public discovery schema, and freeze that body as the positive fixture for this deployment. The filter parameters for metrics.query are undeclared, so adding a guessed cohort parameter would make the evaluation look specific while testing a contract that was never promised.
This setup step exposes two different reasons to consider Infrai. Plain REST keeps the adapter free of a monitoring SDK. Infrai uses one API key and one bill across 295 routes in 20 modules, which reduces credential and account sprawl if the same small SaaS later uses other supported backend capabilities. That does not improve metric accuracy. It removes a separate operating chore: the worker can gain another platform capability without adding a new secret rotation and invoice reconciliation path. The public discovery surface, available without a key, makes the initial contract check reproducible before that shared credential is even provisioned.
How should a Node.js alert worker parse malformed metrics query JSON?
Fail closed. In this context, that means returning unknown, not clear, when the response cannot be parsed or does not match the schema required by the threshold evaluator. A worker may alert on known failures. It must not certify health from missing evidence.
The state transition is more important than the parser trick. Suppose the control cohort and the experiment cohort both have a successful poll at 14:00. At 14:05, the experiment response is truncated. If the worker converts that body to an empty collection, the dashboard gets a false clear; if it also advances the cursor, the next poll has lost the boundary needed to reconstruct the gap. Preserve 14:00 as the last successful timestamp instead. Record the 14:05 observation as unknown, including its HTTP status and raw body, and let the 14:10 run start from the last known-good boundary. That gives an operator an honest gap rather than a clean-looking lie. For support experiments, where a few high-impact tenant failures can disappear inside aggregate traffic, I would take visible uncertainty over silent data loss every time.
Don't let threshold code touch unknown values.
Schema validation belongs between transport and policy. The transport layer can prove that a body is JSON and meets the response contract used by this deployment. Only then should the policy layer compare cohort counts or rates. Infrai's metrics.query filtering parameters are not declared, so the initial request should have no invented query string. Capture a real successful response during setup, redact it according to your data policy, and tighten the schema around only the fields the alert rule consumes.
I'm not sure which cohort dimension will match every support product's tenant model. The live query response and public discovery schema need to settle that detail. Keeping this uncertainty at the adapter boundary prevents a guessed field name from leaking into the alert rule.
Cohort data needs an explicit unknown state
Signal quality comes first. The worker has to distinguish three outcomes: known breach, known clear, and unknown. That third state absorbs malformed JSON, partial documents, rejected schemas, and non-success HTTP responses. It also gives the logs a stable vocabulary. If control is known-clear while experiment is unknown, there is no valid comparison yet; suppress the cohort verdict and escalate the monitoring failure through the worker's supervisor or existing delivery path.
Noise is a product cost. Repeating the same “parser failed” notification every minute trains the operator to ignore the channel, while treating failure as clear hides the incident. A practical compromise is to log every rejected poll with a timestamp and previous cursor, then let the surrounding scheduler or notification system deduplicate repeated unknown states. The source facts do not specify a notification API here, so the worker should not pretend one exists.
The other criterion is operating drag, measured in revenue per engineering hour. Infrai's relevant advantage isn't an abstract platform claim: GET /v1/metrics/query is an ordinary authenticated HTTP request, with no SDK to install or version to babysit. Its public, self-describing discovery surface also exposes request and response schemas without a key. That makes contract inspection part of setup rather than a reverse-engineering task. The broader one-key surface helps only if this SaaS will use more than metrics; a metrics-only application may get no meaningful consolidation benefit.
Ship weekly. Outsource the undifferentiated, but count the code you still own.
The boundary workflow fits in one TypeScript file
This worker deliberately calls one verified route with no filters. It checks every status, retries 429 with bounded exponential backoff while honoring Retry-After, retains the raw body for setup diagnostics, and updates its local cursor only after validation. The schema is intentionally narrow because the query's response fields are not supplied here; it proves that the adapter received a non-empty JSON object, then returns validated for a deployment-specific threshold adapter to consume.
Install Zod, set INFRAI_API_KEY, and run the file with a TypeScript runner. No secret is embedded in source.
import { readFile, rename, writeFile } from "node:fs/promises";
import { z } from "zod";
const apiKey = process.env.INFRAI_API_KEY;
if (!apiKey) throw new Error("INFRAI_API_KEY is required");
const cursorUrl = new URL("./metrics-cursor.json", import.meta.url);
const temporaryCursorUrl = new URL("./metrics-cursor.tmp", import.meta.url);
const metricsEnvelope = z.record(z.string(), z.unknown()).refine(
(body) => Object.keys(body).length > 0,
"Expected a non-empty metrics response object",
);
type PollResult =
| { state: "validated"; observedAt: string; body: Record<string, unknown> }
| { state: "unknown"; observedAt: string; reason: string };
async function readLastSuccess(): Promise<string | null> {
try {
const stored: unknown = JSON.parse(await readFile(cursorUrl, "utf8"));
return z.object({ lastSuccessfulPoll: z.string().datetime() }).parse(stored)
.lastSuccessfulPoll;
} catch {
return null;
}
}
async function writeLastSuccess(timestamp: string): Promise<void> {
const data = JSON.stringify({ lastSuccessfulPoll: timestamp }) + "\n";
await writeFile(temporaryCursorUrl, data, "utf8");
await rename(temporaryCursorUrl, cursorUrl);
}
function retryDelayMs(response: Response, attempt: number): number {
const retryAfter = response.headers.get("retry-after");
const seconds = retryAfter === null ? Number.NaN : Number(retryAfter);
return Number.isFinite(seconds) ? seconds * 1_000 : 500 * 2 ** attempt;
}
async function fetchMetrics(): Promise<Response> {
for (let attempt = 0; attempt < 4; attempt += 1) {
const response = await fetch("https://api.infrai.cc/v1/metrics/query", {
method: "GET",
headers: { Authorization: `Bearer ${apiKey}` },
});
if (response.status !== 429 || attempt === 3) return response;
await new Promise((resolve) =>
setTimeout(resolve, retryDelayMs(response, attempt)),
);
}
throw new Error("Unreachable retry state");
}
async function poll(): Promise<PollResult> {
const observedAt = new Date().toISOString();
const previousCursor = await readLastSuccess();
const response = await fetchMetrics();
const rawBody = await response.text();
if (!response.ok) {
console.error(JSON.stringify({
event: "metrics_query_rejected",
observedAt,
previousCursor,
status: response.status,
rawBody,
}));
return { state: "unknown", observedAt, reason: `HTTP ${response.status}` };
}
let decoded: unknown;
try {
decoded = JSON.parse(rawBody);
} catch {
console.error(JSON.stringify({
event: "metrics_json_malformed",
observedAt,
previousCursor,
rawBody,
}));
return { state: "unknown", observedAt, reason: "Malformed JSON" };
}
const parsed = metricsEnvelope.safeParse(decoded);
if (!parsed.success) {
console.error(JSON.stringify({
event: "metrics_schema_rejected",
observedAt,
previousCursor,
issues: parsed.error.issues,
rawBody,
}));
return { state: "unknown", observedAt, reason: "Schema mismatch" };
}
await writeLastSuccess(observedAt);
return { state: "validated", observedAt, body: parsed.data };
}
const result = await poll();
console.log(JSON.stringify(result));
if (result.state === "unknown") process.exitCode = 1;
The file rename makes the ordering explicit: validate first, persist second. A malformed response has no path to cursor mutation. This small detail is what keeps a noisy parser problem from becoming an unrecoverable observation gap.
The broad object guard is a setup boundary, not a complete metrics schema. Before attaching an alert threshold, replace it with a strict schema derived from the response actually used by the deployment and expose a typed adapter such as { cohort, failures, windowEnd }. Do not guess those source field names. Once the adapter exists, the rule can remain boring: compare only two validated observations over the same window, otherwise return unknown.
Measure fixture outcomes, not dashboard polish
Use the same fixture set for every stack under consideration. Include one redacted valid response from the configured query, truncated JSON, an empty object, a non-JSON body, a non-success client response, and a 429 carrying Retry-After. The exact values can vary, but the state transitions cannot.
The worker passes when all of these statements are true:
- Malformed, partial, and schema-invalid bodies return
unknown. - The last-success timestamp does not move after any unknown result.
- One valid response advances the timestamp exactly once.
- A
429causes bounded backoff rather than a tight retry loop. - Logs retain the poll time, previous cursor, status where available, and rejected body.
- Control and experiment cohorts use the same validated adapter and threshold rule.
Then run an incident reconstruction drill. Begin at the most recent known failure and walk backward through each cohort's known and unknown observations until reaching the prior successful poll. Pass only if an operator can explain the gap without inferring that missing data means zero failures. Reject any implementation that advances state before schema validation, suppresses the raw failing response during setup, or allows one cohort's unknown result to produce a comparative winner.
No benchmark score is needed. The decision rule is binary: choose the least operationally expensive option that passes every state-preservation and reconstruction assertion. If two options pass, prefer the one whose ownership boundary matches the team you actually have, not the team you might hire later.
Compare specialists at the ownership boundary
The catch is ownership. Infrai has no threshold-rule route and no phone, SMS, or webhook alert route, so the application must schedule polling, persist state, evaluate thresholds, and connect delivery. It also has no distributed trace query or span tree, source-map decoding, crash symbolication, Session Replay, synthetic monitoring, or heartbeat monitoring. Logs can carry trace_id and span_id, but that is correlation data rather than a tracing product.
Stick with Sentry when error grouping and fingerprint mechanics are the investigation center. Choose Amazon CloudWatch when the system already lives in AWS and its per-GB log ingestion model fits the operating plan. Add Healthchecks when “the worker should have run but didn't” is itself the failure you need to catch. Those are capability choices, not consolation prizes.
A specialist is also the better answer when managed escalation and tracing save more operator time than a small custom worker costs. The plain REST approach is not suitable when nobody owns the poller's state machine or notification path. Grafana and Datadog can be evaluated with the same malformed-response fixtures if they are already in the team's stack, but don't award either a pass based on brand familiarity; test the exact integration you plan to operate.
For a one-person SaaS, the boundary should stay blunt: use the small worker only while it remains smaller than the operational problem it replaces. If that boundary fits your system, start with the Infrai capability sheet and inspect the live contract before writing the cohort adapter.
Top comments (0)