Short answer: choose a remote, cached feature flag as the Node.js kill switch for incident response; it can disable a broken logistics import without waiting for a production rollout, while an environment variable remains the hard-off fallback.
| Choice | Shutdown path | Main signal risk | Best fit |
|---|---|---|---|
| Environment variable plus restart | Edit configuration, then replace processes | Mixed versions can report different states | One process where restart is accepted recovery |
| Remote flag with local cache | Change one value; workers read cached state | A stale cache delays convergence | Multiple workers and scheduled jobs |
Recommendation: gate admission into the import pipeline with one remote boolean. Cache its last valid value in each worker, and retain a startup environment variable that can force the feature off.
Stop new imports before they create work, leave accepted jobs alone, and alert on the absence of completed results rather than on the flag value. The flag limits damage. The observability signal proves health. Those are separate jobs.
How should a feature flag kill switch support incident response?
A scheduled carrier import has two useful modes: accepting runs and draining accepted work. An environment-variable change couples the first mode to process lifecycle. During a rolling replacement, workers can disagree.
Remote evaluation removes that deployment dependency. Signal quality matters more than API elegance. Check once per scheduled run, at admission, and record the decision with the import identifier and flag revision. Do not evaluate per parcel row. That adds traffic and can split one import into a half-on, half-off result.
Consider the sequence, because this is where a superficially simple toggle becomes noisy. A scheduler creates run carrier-42-0900, the admission check records revision 18, and the queue accepts it. An operator then disables revision 19. Run carrier-42-0900 must drain under the decision already made, while the next scheduled run records disabled and never enters the queue. If workers instead check every row, revision 19 can interrupt an accepted run after some shipment records have landed. The missing-result monitor then sees an incomplete result, the error counter sees a stopped worker, and the control log says the switch worked. Three signals describe one intentional action. Admission-only evaluation avoids that ambiguity.
The switch should not page anyone merely because it is off. Operators may disable an import deliberately. Page when the expected result does not arrive inside the service's declared completion window, then attach the gate state as context. Otherwise every mitigation creates another predictable alert.
Silence is the failure mode.
So is ambiguity.
The 2 criteria that matter
First, define failure behavior. If refresh fails, a worker can use its last valid cached value for a bounded period. After expiry, choose from the operation's risk: an importer that can corrupt shipment state should stop admission; a reversible, read-only enrichment may continue. Document that choice beside the flag. Do not bury it in an SDK default.
This design has a real limitation: the remote control plane and its cache become operational dependencies. It is not suitable when the team cannot own cache expiry, audit access, and convergence drills. The trade-off is deliberate. Faster control costs more moving parts than a startup variable.
Second, measure convergence. In a drill, change the value and observe how long every active scheduler and worker takes to report the new revision. Derive the target from the tolerated damage window, not a polling default. Benchmark with production worker count, cache interval, and network path. Record the slowest worker. A median can make a bad kill switch look healthy.
I would reject a design that needs a dashboard tour, three variants, or a custom rules language. One boolean, one owner, one expiry review, one audit record. Boring wins.
A small TypeScript boundary
Keep the evaluator behind an interface. The scheduler should not know where control data lives, and it should decide before queue admission.
interface KillSwitch {
isImportEnabled(): Promise<{ enabled: boolean; revision: string }>;
}
interface ImportRun {
id: string;
carrier: string;
scheduledAt: Date;
}
interface RunQueue {
enqueue(run: ImportRun): Promise<void>;
}
async function admitImport(
run: ImportRun,
gate: KillSwitch,
queue: RunQueue,
): Promise<"queued" | "disabled"> {
const decision = await gate.isImportEnabled();
console.info("import admission", {
runId: run.id,
enabled: decision.enabled,
revision: decision.revision,
});
if (!decision.enabled) return "disabled";
await queue.enqueue(run);
return "queued";
}
A disabled run is not a failed run, so it should not inflate the import-error counter. Track it as a control decision. Emit a completion event only after the importer produces its expected durable result. An absence check can compare scheduled runs with completion events and alert when the gap exceeds the declared window.
Test enabled admission, disabled admission, cached evaluation during refresh failure, and the chosen behavior after cache expiry. Also test duplicate scheduler delivery. Use the run ID as an idempotency key so recovery does not turn one missed window into duplicate imports.
Keep decision logs lean: import ID, decision, revision, and timestamp. Do not copy shipment payloads or personal data into flag metadata. If logs contain personal data, retention and erasure handling need explicit ownership; GDPR Article 17 defines a right to erasure and its conditions.
When is the environment variable better?
Use it when the system is one process, restarts are fast and safe, and the team has no control-plane reliability budget. It is inspectable, easy to exercise locally, and has little integration surface. It also works as a hard-off startup override when the remote path is unavailable.
It loses once several schedulers or long-lived workers must agree without a rollout. Deployment state is then doing incident-control work. That coupling is the problem, even if the configuration file looks simpler.
Before release, run a drill. Start an import, disable admission, confirm no later run enters the queue, allow accepted work to drain, and verify that the missing-result alert includes the revision without classifying the intentional stop as an application error. Re-enable only after the broken path has a separate fix and the queued workload is understood.
Choose remote cached evaluation when shutdown speed and cross-worker agreement matter more than avoiding one small control dependency. Choose the environment variable when replacement already fits the tolerated damage window. For scheduled logistics imports, the remote gate wins because it separates mitigation from deployment and keeps the alert focused on missing results.
Top comments (0)