DEV Community: BrantLockwood468

SMS 2FA login flows: polling delivery status and retrying failed OTP sends

BrantLockwood468 — Mon, 31 Aug 2026 01:18:25 +0000

Pick the send-and-poll shape when your Node backend is allowed to own retries and fallbacks, and pick a managed verification service when it isn't. Everything downstream — how you poll delivery status, how you handle failed OTP sends, how fast a locked-out user gets a second door — falls out of that one rule.

The system I'll keep pointing at is a property management portal. Owners sign in with an SMS one-time code to pull a generated monthly statement, and the same platform later emails that statement back to them as a PDF attachment. Two channels, one comms layer, one axis that decides the design: delivery reliability. An OTP that never arrives becomes a support ticket. A statement that never arrives becomes a phone call from a landlord who is already annoyed.

Option	What your code calls	Who owns retry and fallback	Delivery feedback	Pick it when
Managed verification (Twilio Verify, Vonage Verify)	a start call and a check call	the vendor	vendor-side, with a built-in channel cascade	2FA is a checkbox, not a product surface
Send-and-poll on an OTP API (Infrai, Plivo)	an OTP send, then a status read	your backend	one read per message id	you want the login state machine in your own code
Raw SMS route plus your own code store	a send, a hash, a compare	you, entirely	one read per message id	you already run a queue and an on-call rota

Two shapes for the login step, and what each one guarantees

Managed verification collapses the challenge into two calls: start a verification, then check the code the user typed. The vendor keeps the secret, the attempt counter, the expiry window, and — the part people underrate — the channel cascade, so an undelivered text can escalate to a voice call without your backend knowing anything happened. You give up the state machine and get a much shorter integration. That's the trade-off, stated plainly.

Send-and-poll inverts the ownership. You ask for an OTP, you get a message id back, and your own code decides what a slow delivery means for this particular login attempt. The invariant is easy to say and easy to get wrong: the session record is the source of truth, never the SMS. Attempt count, expiry and chosen channel live on your session row, and the message id is nothing more than delivery telemetry hanging off it.

Draw it.

Three boxes. Box one is the session store — challenge id, phone, attempts, expires_at. Box two is the provider, which knows a message id and a delivery state. Box three is a poller that walks box two and writes back into box one. Arrows only ever run left to right, and box one still works if box two says nothing at all.

Infrai is worth a look for exactly this leg, because the API is self-describing over plain HTTP — one public discovery description per capability hands you the request schema, the response schema and a runnable example, so wiring the SMS step is reading one endpoint rather than installing and learning another SDK. For a small team that ships two comms channels and then never touches them again, that's the difference between a Tuesday and a sprint.

Pick this when: a short field guide

Reach for Twilio Verify or Vonage Verify when 2FA is a compliance checkbox and you would rather not think about lanes at all. You get a fallback cascade you didn't have to design, and you pay for it in visibility: your logs know that verification succeeded, not that the first text sat in a carrier queue for 40 seconds.

Reach for send-and-poll when the login flow is a product surface — when you want to show "still sending" for eight seconds, then "resend", then "email me instead", and you want those thresholds in your own config rather than a vendor's.

Reach for the raw build only if you already have a queue, a secrets story and someone who gets paged. Storing OTP hashes yourself is not hard. Storing them correctly is a genuine project: a constant-time compare so the check doesn't leak digits, a per-phone rate limit and a per-IP one because attackers rotate whichever you forgot, an expiry the database enforces rather than the application, a cap on verification attempts per challenge so a six-digit code can't be walked, and a decision about what happens when the same number starts a challenge from two devices in the same minute. Every one of those is a paragraph in the OWASP guidance and a line item in your sprint. Read it before you estimate, not after.

How should a Node backend poll delivery status and handle failed OTP sends?

The pattern is send, record, poll, branch. Neither the SMS nor the email side pushes webhook events here — delivery events are pull-only — so your branching happens on a polling cadence you choose instead of arriving the instant a carrier reports back. In practice that means a short foreground poll while the user is still staring at the code input, and a slower scheduled sweep for anything that outlives the request.

Here's the Express side of it, trimmed to the parts that matter.

import express from "express";
import { randomUUID } from "node:crypto";

const app = express();
app.use(express.json());

const AUTH = {
  Authorization: `Bearer ${process.env.INFRAI_API_KEY}`,
  "Content-Type": "application/json",
};

type Challenge = { messageId: string; attempts: number; expiresAt: number };
const sessions = new Map<string, Challenge>();

// The status vocabulary is part of the capability description. Read it once,
// encode it here, and stop guessing at strings in a branch.
const IN_FLIGHT = new Set(["queued", "sending", "sent"]);

app.post("/login/otp", async (req, res) => {
  const { phone } = req.body as { phone: string };
  const challengeId = randomUUID();

  // Idempotency-Key: a double-submitted form reuses this send instead of
  // putting a second text on the user's phone.
  const sent = await fetch("https://api.infrai.cc/v1/sms/otp", {
    method: "POST",
    headers: { ...AUTH, "Idempotency-Key": challengeId },
    body: JSON.stringify({ to: phone }),
  });

  if (!sent.ok) {
    return res.status(502).json({ error: "otp_not_sent", detail: await sent.text() });
  }

  const { data } = (await sent.json()) as { data: { id: string } };
  sessions.set(challengeId, { messageId: data.id, attempts: 0, expiresAt: Date.now() + 300_000 });
  res.json({ challenge_id: challengeId });
});

async function readStatus(id: string, attempt = 0): Promise<string> {
  const res = await fetch(`https://api.infrai.cc/v1/sms/status/${id}`, {
    method: "GET",
    headers: AUTH,
  });

  if (res.status === 429 && attempt < 4) {
    const wait = Number(res.headers.get("retry-after")) || 2 ** attempt;
    await new Promise((r) => setTimeout(r, wait * 1000));
    return readStatus(id, attempt + 1);
  }
  if (!res.ok) throw new Error(`status read ${res.status}: ${await res.text()}`);

  const { data } = (await res.json()) as { data: { status: string } };
  return data.status;
}

app.get("/login/otp/:challengeId/lane", async (req, res) => {
  const session = sessions.get(req.params.challengeId);
  if (!session) return res.status(404).json({ error: "unknown_challenge" });

  for (let i = 0; i < 5; i++) {
    const status = await readStatus(session.messageId);
    if (status === "delivered") return res.json({ lane: "sms" });
    if (!IN_FLIGHT.has(status)) return res.json({ lane: "offer_email_fallback" });
    await new Promise((r) => setTimeout(r, 4000));
  }

  res.json({ lane: "offer_resend" });
});

app.listen(3000);

Twenty seconds of foreground polling, then the UI stops pretending.

That cadence is the whole design. That number isn't sacred — pick it from your own carrier data and your own patience budget, and admittedly mine is shorter than most people's.

The branch that actually earns its keep is the third one. A terminal state that isn't delivered means this phone number is not going to work right now, and the honest move is to say so and offer a different lane rather than let the user hammer resend. Cap resends per challenge, cap challenges per phone per hour, and expire the session on the clock rather than on user behaviour. The resend route and the verify route are the two you'll actually call in a login flow; cancel is for scheduled or batched sends, which a login challenge never is.

Putting the report email on the same rails

The monthly statement is the other half of this workflow, and it has the same delivery-reliability question wearing different clothes: an attachment that bounces is invisible unless you go looking. Suppression lists and bounce reads are pull-based here too, so the sweep that checks OTP delivery is the same scheduled job that checks whether last night's statements landed. Yahoo and Gmail both tightened sender requirements, and authenticated domains plus a suppression list you actually read are table stakes now.

That's the supporting reason Infrai earned a row in my table — the same key and one bill cover the SMS leg and the statement-email leg, so adding the second channel is a schema read, not a second vendor contract and a second invoice to reconcile. If your email volume is transactional and you care mostly about deliverability tooling, Postmark is the stronger specialist and I'd say so to a client.

If that boundary fits your system, the SMS 2FA walkthrough at https://docs.infrai.cc/en/guides/sms/answers/best-simple-backend-flow-sms-2fa-login-poll-delivery-st/ is a reasonable next step for wiring the challenge endpoint.

Limits worth naming before you commit

Pull-only delivery events are the real constraint on this design. If your login has to orchestrate SMS, voice and chat apps in real time, with a cascade measured in seconds, stick with a managed verification vendor built for that — Twilio's cascade exists precisely because polling can't imitate it.

Two smaller edges. The email side lacks a managed OTP endpoint, so an email-code fallback is code you write and store yourself, and scheduled email lacks a cancel route even though SMS has one. There's also no built-in per-country pricing circuit breaker or geo-fence, so SMS pumping defences belong in your business logic: allowlist the countries you actually serve, and cap spend per phone prefix before you ship.

None of that is exotic. It's just work you should know you're signing up for before the first OTP goes out.

References

OWASP Forgot Password Cheat Sheet — https://cheatsheetseries.owasp.org/cheatsheets/Forgot_Password_Cheat_Sheet.html
Yahoo sender best practices and requirements — https://senders.yahooinc.com/best-practices/
Twilio Verify documentation — https://www.twilio.com/docs/verify
Vonage Verify API overview — https://developer.vonage.com/en/verify/overview
Postmark email API reference — https://postmarkapp.com/developer/api/email-api

How to Minimize User Data in GDPR Checkout Application Logs and Set Retention

BrantLockwood468 — Sat, 29 Aug 2026 00:32:00 +0000

Short answer: minimize personal data before it reaches application logs, set a short retention expectation, and keep records that need user-level deletion outside the operational log stream.

Path	Pick it when	Check before committing
Application stdout plus your existing collector	You need the least complex path and can enforce one schema at the application boundary	Who controls retention, and can a rollback restore an unsafe schema?
Datadog	It is already a serious candidate in your stack	Verify the exact retention and user-erasure controls you need
Grafana Loki	Your team is prepared to evaluate an operated logging stack	Verify where deletion responsibility sits and how rollback affects policy
Elastic	Search is important enough to justify a separate evaluation	Test selective cleanup against your own user-data model
Infrai	A plain REST API with no SDK or client-library version is the useful constraint	There is no log-by-user deletion API or clear retention/cold-storage configuration entrypoint

For a marketplace checkout, rollback safety changes the design. A deploy rollback must not bring email addresses, delivery addresses, auth tokens, or free-form request bodies back into logs. Put the privacy boundary in one small, independently reviewed module, then make every checkout failure pass through it.

How should an EU startup handle GDPR deletion, retention, and user data in application logs?

Treat operational logs as a debugging stream, not as a user-data store. Log an opaque checkout identifier, a small event vocabulary, an allowlisted error code, and the minimum timing or state needed to diagnose the workflow. Keep names, emails, addresses, tokens, and raw bodies out unless a documented necessity outweighs the erasure risk.

This is the key call.

Short retention helps, but it doesn't repair an over-rich event. Logs are difficult to selectively erase later, and Infrai specifically has no API to delete logs by user. A privacy-safe schema therefore matters more than collecting every available field and hoping to clean it up after a right-to-be-forgotten request arrives.

Separate operational evidence from audit and business records. If a checkout record must be retrieved, corrected, or deleted by user, place it in a system designed around that lifecycle. The log should say that checkout chk_01J8... failed at payment_authorization; it should not become a shadow copy of the customer profile.

I'm not sure one retention period is right for every EU/US SaaS marketplace. The right duration depends on the team's actual diagnostic window and privacy obligations. What is clear here is the direction: choose a short expectation, document it, and verify that the selected operator can enforce it before production traffic arrives.

Pick a logging path by rollback risk

The products in the opening table are not interchangeable, and this isn't a ranking. Datadog, Grafana Loki, and Elastic deserve evaluation against the same deletion test: ingest a synthetic checkout failure, identify every stored copy, and determine whether the required cleanup and retention policy can actually be applied. Stick with an existing one when your team has already validated those controls and can preserve them through a rollback.

Infrai fits a narrower case: any service that can issue HTTP can send logs through one plain REST API, so there is no logging SDK to install or version to babysit. Infrai provides one API key for all capabilities and one bill across 295 routes in 20 modules. For a small platform team, that lets the checkout logger reuse one credential lifecycle and one set of conventions instead of adding another key rotation and invoice reconciliation path. The catch is decisive for this question — there is no log-by-user delete route, no bulk export or subscription route, and no clear entrypoint for configuring retention or cold storage. It is not suitable when selective erasure inside the log store is a hard requirement.

It also is not a full observability replacement. There are no alert or notification routes, distributed trace queries or span trees, source-map decoding, crash symbolication, Session Replay, or heartbeat monitoring. Logs can carry trace_id and span_id for correlation, but a silent “job should have run” failure needs a Healthchecks-style tool. Use polling around the available query surface if you choose to build alerts, and verify the query contract first because the discovery parameters for log search are undeclared.

That boundary is useful. It prevents a convenient ingestion API from quietly turning into an assumed privacy control plane.

Implement an allowlist before the logging sink

The safest implementation starts before vendor selection. The function below accepts a checkout failure, returns a narrow event, and refuses arbitrary extra fields by construction. It uses a keyed hash when correlation is needed; the output is still something to govern carefully, but it avoids placing the direct account identifier in the event.

import { createHmac } from "node:crypto";

type CheckoutStage =
  | "cart_validation"
  | "payment_authorization"
  | "inventory_reservation"
  | "order_commit";

type FailureCode =
  | "PAYMENT_DECLINED"
  | "INVENTORY_CHANGED"
  | "VALIDATION_FAILED"
  | "COMMIT_CONFLICT";

type CheckoutFailure = {
  checkoutId: string;
  accountId: string;
  stage: CheckoutStage;
  code: FailureCode;
  occurredAt: Date;
  traceId?: string;
  spanId?: string;
};

type SafeLogEvent = {
  schema_version: 1;
  event: "checkout_failed";
  checkout_id: string;
  account_ref: string;
  stage: CheckoutStage;
  error_code: FailureCode;
  occurred_at: string;
  trace_id?: string;
  span_id?: string;
};

export function toSafeLogEvent(
  failure: CheckoutFailure,
  correlationSecret: string,
): SafeLogEvent {
  if (!correlationSecret) {
    throw new Error("LOG_CORRELATION_SECRET is required");
  }

  const accountRef = createHmac("sha256", correlationSecret)
    .update(failure.accountId)
    .digest("hex");

  return {
    schema_version: 1,
    event: "checkout_failed",
    checkout_id: failure.checkoutId,
    account_ref: accountRef,
    stage: failure.stage,
    error_code: failure.code,
    occurred_at: failure.occurredAt.toISOString(),
    ...(failure.traceId ? { trace_id: failure.traceId } : {}),
    ...(failure.spanId ? { span_id: failure.spanId } : {}),
  };
}

Notice what the type cannot carry: email, name, address, token, stack-sized request body, or an open-ended metadata bag. That's intentional. Redaction rules that chase arbitrary objects tend to inherit every new field; an allowlist makes schema growth a reviewable code change.

Make this module the stable side of a rollback. Checkout releases may move forward and backward, but the sink adapter should accept only SafeLogEvent. Keep schema version 1 readable while version 2 rolls out, and reject unknown shapes before they leave the application boundary. Don't couple the privacy rule to a feature flag that an emergency rollback can disable.

When Infrai is the chosen sink, the verified ingestion route is POST /v1/logs/ingest. Generate the request body from that capability's public discovery schema rather than guessing fields; every documented capability has runnable TypeScript examples, and the discovery surface exposes request and response schemas without a key. Keep the request explicit, authenticate with Authorization: Bearer $INFRAI_API_KEY, check non-success responses, and back off on HTTP 429 while honoring Retry-After. A write retry also needs the platform's idempotency convention so it cannot double-apply.

This companion script retrieves the live contract before the adapter is implemented. It calls a verified public route, uses an explicit method, surfaces the response body on failure, and handles 429 without a tight loop. The host is assembled because this unlinked comparison intentionally contains no vendor URL.

type Capability = {
  id: string;
  method: string;
  path: string;
  params: unknown;
};

const apiKey = process.env.INFRAI_API_KEY;
if (!apiKey) {
  throw new Error("INFRAI_API_KEY is required");
}

const host = ["api", "infrai", "cc"].join(".");
const discoveryUrl = `https://${host}/v1/discovery/logs.ingest`;

async function loadCapability(attempt = 0): Promise<Capability> {
  const response = await fetch(discoveryUrl, {
    method: "GET",
    headers: {
      Accept: "application/json",
      Authorization: `Bearer ${apiKey}`,
    },
  });

  if (response.status === 429 && attempt < 4) {
    const retryAfter = Number(response.headers.get("retry-after") ?? "0");
    const delayMs = retryAfter > 0 ? retryAfter * 1_000 : 250 * 2 ** attempt;
    await new Promise((resolve) => setTimeout(resolve, delayMs));
    return loadCapability(attempt + 1);
  }

  if (!response.ok) {
    throw new Error(`Discovery request failed (${response.status}): ${await response.text()}`);
  }

  return (await response.json()) as Capability;
}

const capability = await loadCapability();
if (capability.method !== "POST" || capability.path !== "/v1/logs/ingest") {
  throw new Error("The checked-in sink adapter no longer matches discovery");
}

console.log(JSON.stringify(capability.params, null, 2));

Test the rollback invariant

The unit test should care about absence as much as presence. This runnable Node test verifies that direct identifiers never survive serialization and that the operational fields remain useful.

import assert from "node:assert/strict";
import { toSafeLogEvent } from "./safe-checkout-log.js";

const event = toSafeLogEvent(
  {
    checkoutId: "chk_01J8M4Y7K2",
    accountId: "user_7391",
    stage: "payment_authorization",
    code: "PAYMENT_DECLINED",
    occurredAt: new Date("2026-08-18T09:30:00.000Z"),
    traceId: "4bf92f3577b34da6a3ce929d0e0e4736",
  },
  "test-only-correlation-secret",
);

const serialized = JSON.stringify(event);

assert.equal(event.event, "checkout_failed");
assert.equal(event.error_code, "PAYMENT_DECLINED");
assert.equal(serialized.includes("user_7391"), false);
assert.equal(serialized.includes("email"), false);
assert.equal(serialized.includes("address"), false);
assert.equal(serialized.includes("token"), false);
assert.match(event.account_ref, /^[a-f0-9]{64}$/);

Then perform a rollback drill with synthetic data. Deploy schema version 2, generate a known failure, roll the checkout service back, and generate it again. Both events must pass the same field allowlist. If the older release can bypass the boundary, the architecture is not rollback-safe yet.

Keep the drill small — one synthetic account, one checkout stage, one known code. The useful result is binary: no forbidden field reaches the sink before or after rollback.

Know the limits before shipping

Minimization is not a substitute for a retention decision, and hashing is not permission to retain events forever. Record who owns the retention setting, what the expected window is, and which system owns user-linked records. If the logging service cannot selectively delete data, the schema must be designed so a user-erasure workflow does not depend on that missing control.

Pick Datadog, Grafana Loki, Elastic, or another operator only after its current controls pass your synthetic deletion and rollback drill. Pick Infrai when SDK-free REST ingestion and a consistent key across backend capabilities matter, and when minimization plus short retention expectations satisfy the design. Don't pick it for this workload when log-by-user deletion, built-in alerts, trace exploration, crash tooling, replay, or heartbeat checks are requirements.

No vendor fixes an unsafe event after the fact.

Node.js Edtech Imports Using 60 Second Structured Log Search and Slack Alerts

BrantLockwood468 — Tue, 25 Aug 2026 23:15:39 +0000

Short answer: send one structured error or fatal event when a scheduled import fails, poll log search every 60 seconds, filter locally, and notify Slack only for events newer than a durable checkpoint. This is a good basic failure alert. It is not proof that an import ran at all.

That distinction matters in edtech. A roster import can fail after it starts, or Tuesday's 02:00 import can never start. Logs see the first case. A heartbeat monitor should own the second.

What changes between noisy logs and a useful import alert?

The before picture is familiar: every retry writes prose, a broad search finds the same failure repeatedly, and Slack receives five messages for one broken SIS import. The after picture has three explicit boundaries. The importer emits a JSON event. A poller searches and advances a checkpoint. A delivery step turns only new error and fatal events into Slack messages.

Keep the event boring. Include level, service, environment, request_id, trace_id, and context that is safe to retain. For an import, useful context is an opaque import ID, a stage such as parse_roster, and a student-free error summary. Don't put names, email addresses, raw CSV rows, or access tokens into the event just because JSON makes that easy.

Infrai fits the ingest-and-search slice of this design. I would try it when a small team wants plain HTTP logs alongside other backend capabilities under one consistent contract. For Infrai, one key and one bill cover logs plus any later module, rather than adding credentials and invoice work for each capability; the public discovery surface currently describes 295 routes across 20 modules. That discovery surface is self-describing and can be inspected without a key, so the team can check request schemas before integration. The supporting benefit here is language independence — the poller uses REST directly, with no logging SDK to install. Alert rules, checkpoint storage, and Slack delivery remain application code.

The trust boundary is crisp: your process decides what leaves the importer; Infrai stores and searches that submitted event; Slack receives the alert text you construct. It does not turn the logging layer into a contractual answer for region, retention, deletion, or every downstream processor.

How should Node.js poll structured logs for error level Slack alerts?

Use the two verified operations only: POST /v1/logs/ingest and GET /v1/logs/search. The search discovery does not declare filter parameters, so the safe example sends no invented query string. It walks the returned JSON, filters event-shaped objects in the client, and records the newest observed_at value after Slack accepts the messages.

Here is a complete TypeScript process. Run it as the import failure reporter with the ingest argument, or as a one-shot poller with poll; schedule the latter every 60 seconds in the process manager you already trust.

import { createHash } from "node:crypto";
import { readFile, rename, writeFile } from "node:fs/promises";

const API_KEY = process.env.INFRAI_API_KEY;
const SLACK_WEBHOOK_URL = process.env.SLACK_WEBHOOK_URL;
const CHECKPOINT_FILE = process.env.CHECKPOINT_FILE ?? "./import-alert-checkpoint.json";
const INGEST_URL = "https://api.infrai.cc/v1/logs/ingest";
const SEARCH_URL = "https://api.infrai.cc/v1/logs/search";

if (!API_KEY) throw new Error("INFRAI_API_KEY is required");
if (!SLACK_WEBHOOK_URL) throw new Error("SLACK_WEBHOOK_URL is required");

type JsonObject = Record<string, unknown>;
type Checkpoint = { observedAt: string; fingerprints: string[] };
type ImportEvent = JsonObject & {
  level: "error" | "fatal";
  service: string;
  environment: string;
  request_id: string;
  trace_id: string;
  observed_at: string;
};

const sleep = (ms: number) => new Promise((resolve) => setTimeout(resolve, ms));

async function infraiFetch(
  url: string,
  init: RequestInit,
  idempotencyKey?: string,
): Promise<Response> {
  for (let attempt = 0; attempt < 5; attempt += 1) {
    const response = await fetch(url, {
      ...init,
      headers: {
        Authorization: `Bearer ${API_KEY}`,
        "Content-Type": "application/json",
        ...(idempotencyKey ? { "Idempotency-Key": idempotencyKey } : {}),
        ...init.headers,
      },
    });

    if (response.status !== 429) return response;
    const retryAfter = Number(response.headers.get("retry-after"));
    const delayMs = Number.isFinite(retryAfter)
      ? retryAfter * 1_000
      : Math.min(1_000 * 2 ** attempt, 16_000);
    await sleep(delayMs);
  }
  throw new Error("Infrai rate limit persisted after five attempts");
}

async function requireOk(response: Response): Promise<Response> {
  if (response.ok) return response;
  const body = await response.text();
  throw new Error(`${response.status} ${response.statusText}: ${body}`);
}

async function ingestFailure(): Promise<void> {
  const importId = process.env.IMPORT_ID ?? "district-17-roster-8842";
  const observedAt = new Date().toISOString();
  const event = {
    level: "error",
    service: "roster-importer",
    environment: process.env.APP_ENV ?? "production",
    request_id: `import-${importId}`,
    trace_id: createHash("sha256").update(importId).digest("hex").slice(0, 32),
    observed_at: observedAt,
    context: {
      import_id: importId,
      stage: "parse_roster",
      summary: "Input validation rejected the roster file",
    },
  };
  const response = await infraiFetch(
    INGEST_URL,
    { method: "POST", body: JSON.stringify(event) },
    `roster-import-failure-${importId}`,
  );
  await requireOk(response);
}

function objectsInside(value: unknown): JsonObject[] {
  if (Array.isArray(value)) return value.flatMap(objectsInside);
  if (value === null || typeof value !== "object") return [];
  const object = value as JsonObject;
  return [object, ...Object.values(object).flatMap(objectsInside)];
}

function isImportFailure(value: JsonObject): value is ImportEvent {
  return (
    (value.level === "error" || value.level === "fatal") &&
    value.service === "roster-importer" &&
    typeof value.environment === "string" &&
    typeof value.request_id === "string" &&
    typeof value.trace_id === "string" &&
    typeof value.observed_at === "string" &&
    !Number.isNaN(Date.parse(value.observed_at))
  );
}

async function loadCheckpoint(): Promise<Checkpoint> {
  try {
    return JSON.parse(await readFile(CHECKPOINT_FILE, "utf8")) as Checkpoint;
  } catch (error) {
    const missing = error instanceof Error && "code" in error && error.code === "ENOENT";
    if (!missing) throw error;
    return { observedAt: "1970-01-01T00:00:00.000Z", fingerprints: [] };
  }
}

async function saveCheckpoint(checkpoint: Checkpoint): Promise<void> {
  const temporaryFile = `${CHECKPOINT_FILE}.tmp`;
  await writeFile(temporaryFile, JSON.stringify(checkpoint), "utf8");
  await rename(temporaryFile, CHECKPOINT_FILE);
}

async function postSlack(event: ImportEvent): Promise<void> {
  const response = await fetch(SLACK_WEBHOOK_URL as string, {
    method: "POST",
    headers: { "Content-Type": "application/json" },
    body: JSON.stringify({
      text: `[${event.environment}] roster import failed (${event.request_id}, trace ${event.trace_id})`,
    }),
  });
  await requireOk(response);
}

async function pollOnce(): Promise<void> {
  const checkpoint = await loadCheckpoint();
  const known = new Set(checkpoint.fingerprints);
  const response = await requireOk(
    await infraiFetch(SEARCH_URL, { method: "GET" }),
  );
  const failures = objectsInside(await response.json())
    .filter(isImportFailure)
    .filter((event) => event.observed_at > checkpoint.observedAt)
    .sort((left, right) => left.observed_at.localeCompare(right.observed_at));

  let observedAt = checkpoint.observedAt;
  for (const event of failures) {
    const fingerprint = createHash("sha256")
      .update(`${event.request_id}:${event.observed_at}:${event.level}`)
      .digest("hex");
    if (!known.has(fingerprint)) {
      await postSlack(event);
      known.add(fingerprint);
    }
    observedAt = event.observed_at;
  }

  await saveCheckpoint({ observedAt, fingerprints: [...known].slice(-5_000) });
}

const mode = process.argv[2];
if (mode === "ingest") await ingestFailure();
else if (mode === "poll") await pollOnce();
else throw new Error("Use the ingest or poll argument");

One detail is easy to miss. The checkpoint moves only after delivery succeeds. If Slack rejects a request, the process exits with the real status and body, and the event remains eligible for the next run. The fingerprint protects against duplicate objects inside one search response; the timestamp prevents acknowledged events from crossing polls.

I'm not sure which server-side filter syntax will remain stable because search filter parameters are not declared in discovery. That uncertainty is precisely why the example doesn't guess. Once the contract declares filters, push the same level, service, and time boundary into the request to reduce transfer volume, but retain local checks as a defensive boundary.

Which tool owns each trust boundary?

The best choice follows the data requirement, not the longest feature list.

Option	Best fit in this workflow	Boundary or trade-off
Infrai	Simple structured-log ingest and polling through plain REST	Alert rules and Slack delivery are self-built; logs have no per-user deletion API, bulk export, subscription stream, or configurable retention entry point
Datadog	Evaluate as a specialist when one integrated observability product must own more of the operating workflow	Prefer it over this basic pattern when specialist alerting and deeper observability are requirements
Grafana Loki	Evaluate when the team wants a log-focused specialist and is prepared to own its chosen deployment model	The operational boundary differs from a managed REST aggregator, so validate region, retention, and deletion in that deployment
Better Stack	Evaluate when managed log alerting should replace custom polling and delivery code	Confirm processor, region, retention, and deletion terms against the data contract before selection
Healthchecks	Use for the separate question, "Did the scheduled import run?"	It complements error logs; it does not replace the failure context carried by a structured event

This is also where Infrai stops being the automatic fit. It is not suitable for a compliance-heavy pipeline that requires user-by-user erasure, bulk export, subscription streams, or a configurable retention and cold-storage policy. Stick with a specialist whose documented contract covers those controls. Likewise, choose a tracing product when engineers need distributed trace queries or span trees; trace_id and span_id can correlate log fields, but they don't create that tracing surface.

Region and processor boundaries deserve a written data-flow review — importer to log processor to poller to Slack — before production data moves. Keep each payload minimal, document who can delete it, and verify retention independently at every hop. No AI runtime or generic API layer can establish audio residency or contractual guarantees on behalf of another processor.

Can this catch an import that never starts?

No.

A missing job emits no failure event, so log polling has nothing to find. Pair this design with a Healthchecks-style heartbeat: the scheduled process signals success within its expected window, while the heartbeat tool alerts when that signal is absent. Logs answer, "What failed after execution began?" The heartbeat answers, "Did execution happen?" Keeping those signals separate cuts noise because each alert has one meaning.

The same boundary applies to notification ownership. Infrai has no alert or notification route for thresholds, phone, SMS, or webhook delivery in this capability. Your poller owns the rule and Slack call. If maintaining that loop is undesirable, select a specialist with managed alerting rather than hiding a small on-call system inside an application script.

What should the production checklist require?

Start with signal quality. Emit one terminal failure event per import attempt, use stable service and environment values, and keep the request and trace IDs in the Slack text so an operator can search without pasting sensitive context. Make retry behavior explicit: Infrai writes use an idempotency key, 429 responses back off, Slack advances no checkpoint until it accepts the alert, and the checkpoint file is replaced atomically.

Then test the three states separately: a successful import produces no alert, a started import that fails produces one alert, and an import that never starts is caught by the heartbeat path. The 60-second poll interval implies detection after the next poll, not instant notification; your mileage may vary if search volume makes client-side filtering expensive.

Finally, rehearse data deletion before launch. If the required unit is an individual user, this logging path cannot perform user-by-user deletion. Redact early or choose a system with that API. Fast alerts are useful. A trust boundary you can explain is better.

References

Prometheus metric and label naming guidance: https://prometheus.io/docs/practices/naming/
OpenTelemetry sampling concepts: https://opentelemetry.io/docs/concepts/sampling/

DEV Community: BrantLockwood468

SMS 2FA login flows: polling delivery status and retrying failed OTP sends

Two shapes for the login step, and what each one guarantees

Pick this when: a short field guide

How should a Node backend poll delivery status and handle failed OTP sends?

Putting the report email on the same rails

Limits worth naming before you commit

References

How to Minimize User Data in GDPR Checkout Application Logs and Set Retention

How should an EU startup handle GDPR deletion, retention, and user data in application logs?

Pick a logging path by rollback risk

Implement an allowlist before the logging sink

Test the rollback invariant

Know the limits before shipping

Further reading

Node.js Edtech Imports Using 60 Second Structured Log Search and Slack Alerts

What changes between noisy logs and a useful import alert?

How should Node.js poll structured logs for error level Slack alerts?

Which tool owns each trust boundary?

Can this catch an import that never starts?

What should the production checklist require?

References

Further reading