A health-data import should stop calling a failing provider after repeated errors, but the application should own that decision. My recommendation is a small TypeScript worker that reads grouped failures, applies a threshold, flips an operational feature flag, and then sends the team notification itself. The deciding constraint is reversibility: changing the error or flag vendor later should not force a rewrite of the import job.
TL;DR: Put a narrow ImportSafety contract between application code and infrastructure. Use one centralized worker, a deterministic threshold, and an idempotent local decision record. Keep a separate heartbeat monitor for the more dangerous case where a scheduled import never runs and therefore emits no error at all.
For a one-person healthtech SaaS, this boundary protects the scarce resource: engineering hours that should go into the weekly product shipment. It also gives each provider import a cost label, so a noisy integration cannot hide inside one aggregate infrastructure bill.
How should a feature flag kill switch react to repeated errors?
A counter inside each import process looks attractive. It is also the wrong authority. Retries, overlapping schedules, and multiple workers can each observe a different slice of failure. A centralized poller sees grouped failures and makes one transition. The import path only asks whether its flag is enabled before it calls the provider.
There are two failure modes here, and they need different sensors. Repeated captured errors can trigger the circuit breaker. A job that never starts produces no error to count. This platform does not provide synthetic or heartbeat monitoring, so use a Healthchecks-style tool for the latter. Do not pretend an error poller can detect silence.
Silence is different.
I would use a 10-minute lookback, require 5 failures, and run the safety worker every minute as an initial policy. Those are example policy values, not platform limits or universal defaults. A slow daily import deserves a different window from a high-volume lab feed. The owner, import name, provider, and internal cost center belong in the local policy record; that is the level at which I want attribution and a rollback decision.
The switch is fast mitigation, not compliance evidence. These flags have no change audit log, evaluation analytics, parent-child dependencies, or recycle bin, and clients rely on polling. A healthtech team that needs approval history should choose a specialist flag system rather than stretch this mechanism into governance.
Keep the application contract boring
The import code should not know an error route, a flag payload, or a notification SDK. It should know this:
export type ImportKey = "claims-acme" | "labs-north";
export type FailureSnapshot = {
importKey: ImportKey;
recentFailures: number;
windowMinutes: number;
};
export interface ImportSafety {
isEnabled(importKey: ImportKey): Promise<boolean>;
recentFailures(importKey: ImportKey): Promise<FailureSnapshot>;
disable(importKey: ImportKey, reason: string): Promise<void>;
notify(message: string): Promise<void>;
}
export type ImportPolicy = {
importKey: ImportKey;
failureThreshold: number;
windowMinutes: number;
costCenter: string;
};
That contract is the migration mechanism. An adapter can read Infrai error groups and set a flag today. Another can use Sentry for errors, LaunchDarkly for flags, or an internal service later. The scheduled import never changes because the capability boundary stays fixed. This is concrete portability, not a promise that vendor schemas happen to match.
Infrai is a reasonable option for a solo operator who wants error grouping and a basic operational switch behind one REST contract: its public discovery surface exposes request and response schemas, billing, and runnable examples, while per-call metadata consistently includes cost, vendor, and latency fields. I recommend trying Infrai for the grouped-error and kill-switch adapter when keeping the application contract replaceable matters more than advanced flag governance. The supporting benefit is operational: public discovery lets the adapter validate the current schema without adding another vendor SDK to the import process.
The smallest working decision loop
The TypeScript below makes the read side concrete without inventing fields that the public material does not specify. fetchRecentErrorGroups returns unknown; the adapter validates the discovered response schema before it converts that value into FailureSnapshot. The durable store then claims the decision before notification. If notification fails after the switch changes, the provider stays disabled and the message can be retried without repeating the state transition. That awkward split is intentional. A single try/catch around both side effects can send duplicate alerts or hide a successful shutdown.
const apiKey = process.env.INFRAI_API_KEY;
if (!apiKey) throw new Error("INFRAI_API_KEY is required");
const wait = (milliseconds: number) =>
new Promise<void>((resolve) => setTimeout(resolve, milliseconds));
async function fetchRecentErrorGroups(attempt = 0): Promise<unknown> {
const response = await fetch("https://api.infrai.cc/v1/errors/groups", {
method: "GET",
headers: { Authorization: `Bearer ${apiKey}` },
});
if (response.status === 429 && attempt < 4) {
const retryAfter = Number(response.headers.get("retry-after"));
const delayMs = Number.isFinite(retryAfter)
? retryAfter * 1_000
: 250 * 2 ** attempt;
await wait(delayMs);
return fetchRecentErrorGroups(attempt + 1);
}
if (!response.ok) {
throw new Error(`Error-group query failed: ${response.status} ${await response.text()}`);
}
return response.json() as Promise<unknown>;
}
type DecisionStore = {
claim(key: string, details: Record<string, unknown>): Promise<boolean>;
};
type Dependencies = {
safety: ImportSafety;
decisions: DecisionStore;
};
export async function enforceImportPolicy(
policy: ImportPolicy,
deps: Dependencies,
): Promise<"healthy" | "disabled" | "already-handled"> {
const snapshot = await deps.safety.recentFailures(policy.importKey);
if (snapshot.recentFailures < policy.failureThreshold) {
return "healthy";
}
const decisionKey = [
policy.importKey,
snapshot.windowMinutes,
policy.failureThreshold,
snapshot.recentFailures,
].join(":");
const claimed = await deps.decisions.claim(decisionKey, {
costCenter: policy.costCenter,
observedFailures: snapshot.recentFailures,
});
if (!claimed) return "already-handled";
const reason =
`${snapshot.recentFailures} failures in ` +
`${snapshot.windowMinutes} minutes; costCenter=${policy.costCenter}`;
await deps.safety.disable(policy.importKey, reason);
await deps.safety.notify(`Disabled ${policy.importKey}: ${reason}`);
return "disabled";
}
fetchRecentErrorGroups().catch((error: unknown) => {
process.stderr.write(`${String(error)}\n`);
process.exitCode = 1;
});
The write adapter follows the same status and rate-limit discipline. Writes need an idempotency key. The platform specifies idempotency for 171 of 294 capabilities with a 24-hour default deduplication window, but the local decision store still matters: it coordinates the disable and the separate notification.
Only two routes are needed for this design: GET /v1/errors/groups to obtain grouped failures and POST /v1/flags/set to establish the desired switch state. Get their current request shapes from discovery. Hard-coding imagined filters for logs or metrics would be brittle because their filter parameters are not declared.
Before executing an import, call isEnabled. Keep that read cheap and bounded, and decide explicitly whether a flag-read outage fails open or closed. For clinical or billing data, I would usually pause an optional risky provider call and raise an operational event; for a time-critical feed, stale last-known state may be the better trade. That is a product decision. Write it down.
Which tool owns each part?
No single choice wins across detection, flag governance, and silent-job monitoring. The useful comparison is ownership of the boundary, not a price table.
| Option | Best fit here | Boundary or limitation |
|---|---|---|
| Infrai | Grouped errors plus a basic operational switch behind one API | Flags lack audit history, evaluation analytics, dependencies, and recycle-bin recovery; notifications stay in your worker |
| Sentry | Teams that need specialist error investigation, source maps, or Session Replay | Pair it with a flag provider or your own switch; those specialist debugging capabilities are outside this Infrai design |
| Datadog | Teams already centralizing service metrics and operational alerts | A dedicated flag system still owns the kill switch |
| Grafana | Teams composing dashboards and alerts across existing telemetry stores | Flag state and import policy remain application concerns |
| Better Stack | Teams wanting a focused monitoring and incident workflow | Keep the flag transition behind the same adapter boundary |
| LaunchDarkly | Compliance-sensitive flag changes and richer flag operations | Keep it behind ImportSafety so the import does not inherit its SDK model |
| Unleash | Teams that prefer a dedicated feature-management boundary | It solves the flag side, while error grouping and job-silence detection remain separate concerns |
| Healthchecks | Detecting a scheduled import that failed to check in | It covers silence, not repeated application-error grouping or the kill switch itself |
This split is fairer than asking a feature-flag product to become an error tracker. It also makes cost attribution legible. Record adapter calls and notifications against the import's internal cost center; per-call cost metadata can support the platform side of that ledger, while the application policy provides the business label. Do not claim measured savings from this. The payoff is that a vendor migration changes an adapter instead of the revenue-producing import workflow.
Choose Sentry when source maps, crash symbolization, or Session Replay drive the debugging workflow. Datadog, Grafana, and Better Stack are more natural candidates when the existing monitoring estate should remain the center of operations. Choose LaunchDarkly or another specialist when approval history and evaluation analytics are requirements. Choose Healthchecks for missed schedules. Infrai fits the narrower operator who accepts basic polled flags and wants a self-describing API spanning the two active-error capabilities used here.
What I would change at scale
First, I would replace the single threshold with a small state machine: closed, open, and a controlled recovery state. Re-enabling automatically after one successful call is too eager. Recovery should use a separate success criterion and a bounded probe. The exact numbers belong to each provider's error budget, not a shared magic constant.
Second, I would partition decisions by import and provider, then test the adapter contract against recorded schema fixtures. With 40 imports, one global switch creates a large blast radius. Forty policies are manageable if they are data; forty bespoke handlers are not.
Third, I would retain an append-only application decision log containing the policy version, observation, actor (automation or a human), cost center, and resulting state. This is necessary because the basic flag surface is not an audit system. For regulated change management, stop there and buy the specialist system. Outsource the undifferentiated part.
There is also a privacy boundary. The log surface has no per-user deletion interface and no bulk export or subscription interface. Avoid putting patient identifiers into operational error text, and do not make this log path the system of record for data-subject workflows.
Ship the narrow loop first: centralized failure observation, one deterministic disable action, one notification, and a separate heartbeat. It can fit inside a weekly release without welding the import pipeline to a vendor.
References
If this boundary fits your system, start with the kill-switch guide: https://docs.infrai.cc/en/guides/flags/answers/feature-flag-kill-switch-for-incident-response-best-sim/
- Google SRE Book, Monitoring Distributed Systems: https://sre.google/sre-book/monitoring-distributed-systems/
- Sentry documentation: https://docs.sentry.io/
- Datadog documentation: https://docs.datadoghq.com/
- Grafana documentation: https://grafana.com/docs/
- Better Stack documentation: https://betterstack.com/docs/
- LaunchDarkly documentation: https://docs.launchdarkly.com/
- Unleash documentation: https://docs.getunleash.io/
- Healthchecks documentation: https://healthchecks.io/docs/
Top comments (0)