TL;DR: In a Node.js backend, feature flags need a kill switch with a disabled fallback default, a known-good path, and structured decision logs for safe rollout and outage handling. A switch that cannot be found in the logs is only half a kill switch.
For a nightly e-commerce pipeline, useful signal is not another line saying a flag was checked. It is the run ID, store ID, chosen path, control revision, reason, duration, and outcome on one searchable event. That record answers the urgent question: did the new path cause this failed import, or did the input fail before the decision?
This is a revenue-per-hour decision. A one-person SaaS does not need a grand control-plane project. It needs a small boundary that can stop a risky path without a deploy, plus enough evidence to know when stopping it is justified. Ship weekly. Outsource undifferentiated control storage if that trade fits, but own the fallback semantics in application code.
How should feature flags become a backend kill switch?
A boolean says which path to take. It does not explain why that path was selected, how fresh the decision was, or what happened next. During an outage, those missing fields turn a fast control into a guessing exercise. Control lookup can time out; a value can be absent or malformed; the new pipeline path can fail after the decision succeeds; and a process can retain an older value in memory. Each case needs an explicit behavior, not an accidental exception path. For this job, I would choose a conservative default: the new enrichment path is disabled unless the evaluator returns a valid, enabled decision. The old import path remains available. That sacrifices new functionality during control-plane trouble, but preserves the nightly pipeline's established behavior. Signal quality wins over novelty.
Keep it dull.
Emit one decision event per pipeline run, then one completion event. Do not log every item-level check. If a run handles thousands of catalog records, item-level decision logs create noise without changing the answer to the operational question. Errors still deserve their own event, linked by the same run ID.
The smallest working boundary
Keep evaluation behind an interface. The application should not know whether state came from a file, database, self-hosted service, or commercial provider. More important, callers should not invent fallback behavior at every call site.
type Decision = {
enabled: boolean;
revision: string;
reason: "configured" | "missing" | "unavailable" | "invalid";
};
interface Controls {
evaluate(key: string, context: { storeId: string }): Promise<Decision>;
}
interface PipelineLog {
write(event: Record<string, string | number | boolean>): void;
}
async function runNightlyImport(
runId: string,
storeId: string,
controls: Controls,
log: PipelineLog,
): Promise<void> {
const startedAt = Date.now();
let decision: Decision;
try {
decision = await controls.evaluate("catalog-enrichment-v2", { storeId });
} catch {
decision = { enabled: false, revision: "unknown", reason: "unavailable" };
}
const path = decision.enabled ? "enriched" : "baseline";
log.write({
event: "pipeline_path_selected",
runId,
storeId,
path,
controlRevision: decision.revision,
decisionReason: decision.reason,
});
let outcome = "succeeded";
try {
if (decision.enabled) await runEnrichedImport(runId, storeId);
else await runBaselineImport(runId, storeId);
} catch (error) {
outcome = "failed";
throw error;
} finally {
log.write({
event: "pipeline_run_completed",
runId,
storeId,
path,
outcome,
durationMs: Date.now() - startedAt,
});
}
}
declare function runEnrichedImport(runId: string, storeId: string): Promise<void>;
declare function runBaselineImport(runId: string, storeId: string): Promise<void>;
Missing and invalid values belong inside the evaluator. They should return the same disabled state with different reasons. A disabled configuration is ordinary; an unavailable evaluator is a control-system incident.
One boundary. One policy.
Do not catch an enriched-import failure and immediately rerun the baseline path unless the work is provably idempotent. A partial write followed by a retry can duplicate or overwrite catalog state. The switch controls admission to a path; it is not transaction recovery. Let the job's existing retry and idempotency policy handle execution failures.
Search for decisions, not chatter
Structured events should support a few questions: Which path did a run take? Which revision made that decision? Are failures concentrated on the enriched path? Are lookups becoming unavailable? How long did each path take?
Keep field types stable. durationMs should always be numeric, while outcome and decisionReason should come from bounded vocabularies. High-cardinality identifiers such as runId and storeId are valuable for investigation, but they should not automatically become metric labels. Logs hold detail; metrics summarize rates and latency.
Google's SRE guidance groups monitoring around latency, traffic, errors, and saturation. Here, duration maps to latency, completed runs to traffic, failed outcomes to errors, and worker or queue capacity to saturation. The control decision adds the comparison dimension: baseline versus enriched.
One event is not a dashboard. I would start with two views: failure count by path and decisionReason, plus run duration by path. Then keep direct log search by runId for diagnosis. This is enough to judge a rollout without manufacturing dozens of alerts.
Noise has a carrying cost. Every alert reviewed is time not spent shipping, so page only on conditions that require action. A failed nightly run may demand action because the next opportunity is a day away; one malformed catalog row may belong in a report. The business deadline determines the alert, not the existence of an error log.
This design has a clear limitation. It is not suitable when the baseline path cannot safely process the current input, when a job must stop immediately midway through execution, or when partially completed writes lack idempotent recovery. In those cases, pause admission and repair the transaction boundary instead of pretending that a feature flag is a rollback mechanism. The trade-off is intentional: a once-per-run decision is easier to reason about, but it reacts more slowly than checking a control inside every processing step.
Rollout and outage handling
Start disabled. Enable the new path for a bounded cohort of stores, using stable identifiers so one store does not bounce between paths on successive runs. Compare outcomes and duration against the baseline. Expand only when the signal is clean.
A kill decision should be boring: set the control off, confirm new runs select baseline, and use the revision field to distinguish decisions made before and after the change. In-flight work needs a declared policy. This implementation checks once at job admission, so a mid-run change affects the next run rather than interrupting writes halfway through. That is deliberate.
Separate two outages. If control evaluation is unavailable, the code selects the baseline and records unavailable. If the pipeline itself fails, the completion event records failed. Combining both into a generic error count hides the difference between a control dependency problem and a commerce-data problem.
Test enabled, disabled, missing, invalid, and unavailable states. Then test execution success and failure for both paths. Assert the emitted path, decisionReason, controlRevision, and outcome, not merely which function ran. Observability is part of the contract.
What I would change at scale
At larger volume, I would batch log transport, define retention by event value, and materialize common comparisons in an analytical store. ClickHouse is one documented option for analytical workloads, but the schema and questions should survive a storage change. The application boundary stays generic.
I would also separate emergency authorization from ordinary rollout editing, keep an audit trail of control changes, and rehearse the shutdown procedure. Those additions increase governance work. They become worthwhile when more people can change production behavior or many concurrent jobs make revision tracking harder.
The core remains small: fail closed for the risky capability, preserve the known-good path, log one high-quality decision, and measure the result. That gives a small team a kill switch it can trust without turning feature delivery into an infrastructure program.
Top comments (0)