Percentage Feature Flag Rollout: 100 Stable Cohorts for Import Alerts

TL;DR: Put each tenant into one of 100 deterministic cohorts, then use a backend feature flag to enable silent-import alerts for an increasing percentage of those cohorts. Hash a stable tenant ID plus the flag key and a fixed version, evaluate the flag only when an import is due, and record the decision beside the alert signal. This gives a solo operator one useful control: widen exposure without making the same tenant jump in and out of the alerting cohort.

For a B2B SaaS import pipeline, the least complex useful design is scheduler -> import result state -> eligibility check -> alert state machine -> notification. The percentage flag belongs at the eligibility check. It should decide which tenants participate; it should not decide whether a participating tenant is healthy. Keeping those two questions separate makes false positives diagnosable and rollback immediate.

How can Node.js keep percentage rollout feature flags stable?

Use an identifier whose ownership matches the behavior. An alert about a scheduled customer import should normally be assigned by tenant ID, not request ID, worker ID, or a randomly generated value. The same tenant must reach the same cohort in every process and after every restart.

The hash input also needs a namespace. If every flag hashes only the tenant ID, all flags expose exactly the same tenants first. Adding the flag key decorrelates flags. A version field makes an intentional future reshuffle explicit instead of turning a small implementation edit into a surprise cohort migration.

This example uses the built-in cryptographic hash API and maps the first unsigned 32 bits of the digest onto 100 cohorts. It validates rollout configuration, normalizes neither IDs nor keys, and makes the byte encoding explicit. Those choices are part of the contract.

import { createHash } from "node:crypto";

type Rollout = {
  key: string;
  percent: number;
  version: number;
};

function cohortFor(subjectId: string, rollout: Pick<Rollout, "key" | "version">): number {
  if (subjectId.length === 0) throw new Error("subjectId must not be empty");
  if (rollout.key.length === 0) throw new Error("rollout key must not be empty");

  const input = JSON.stringify([rollout.version, rollout.key, subjectId]);
  const digest = createHash("sha256").update(input, "utf8").digest();
  return digest.readUInt32BE(0) % 100;
}

function isEnabled(subjectId: string, rollout: Rollout): boolean {
  if (!Number.isInteger(rollout.percent) || rollout.percent < 0 || rollout.percent > 100) {
    throw new Error("percent must be an integer from 0 through 100");
  }

  return cohortFor(subjectId, rollout) < rollout.percent;
}

const silentImportAlerts: Rollout = {
  key: "silent-import-alerts",
  percent: 10,
  version: 1,
};

for (const tenantId of ["tenant-a17", "tenant-b04", "tenant-c91"]) {
  console.log({
    tenantId,
    cohort: cohortFor(tenantId, silentImportAlerts),
    enabled: isEnabled(tenantId, silentImportAlerts),
  });
}

The important property is deterministic assignment, not secrecy. Ten percent means cohorts 0 through 9, so moving from 10 to 25 percent retains the original group and adds cohorts 10 through 24. Consider a tenant in cohort 7 whose nightly import is due at 02:00 with a 30-minute grace interval: it remains eligible at both rollout levels, but the detector still waits until 02:30 and still checks the completion record before opening anything. A tenant in cohort 18 becomes eligible only at 25 percent. That distinction matters during review because a newly opened alert can be attributed either to wider exposure or to changed import state; the recorded cohort, percentage, due time, and reason settle the question without reconstructing a worker's memory. Moving back to zero disables evaluation for everyone without deleting state.

There is a sharp edge here: modulo 100 does not produce perfectly equal counts in a small tenant population. It creates 100 possible cohorts; it does not promise that exactly 10 of 100 current tenants land below 10. Treat the configured percentage as exposure over the identifier space, then inspect the actual eligible tenant count before widening. Do not silently compensate by hand-picking tenants, because that destroys the reproducible rule.

That is the trade-off.

This approach is a poor fit when release membership must be exact, such as selecting precisely 10 named design partners, or when policy requires manual approval for each tenant. Use an explicit allowlist for those cases. Hash cohorts also cannot balance tenants by import volume: one tenant may run one small import while another runs hundreds, so use a separate capacity control when workload, rather than tenant exposure, is the limiting factor.

Make the alert stateful, not twitchy

A scheduler miss and an empty import are different signals. The first says an expected execution did not complete. The second says an execution completed with zero results, which may be valid for that source. The alert evaluator needs the expected schedule, last successful completion, result count, and a source-specific grace interval. A single rows === 0 condition throws away that context and creates noise.

I optimize this decision for signal quality rather than maximum coverage. Start with tenants whose schedules and success timestamps are trustworthy. Exclude paused connections and imports still inside their grace interval before applying the flag. Otherwise the experiment measures bad input state, not alert quality.

A compact evaluator can keep rollout assignment and health state visibly separate:

type ImportStatus = {
  tenantId: string;
  connectionId: string;
  paused: boolean;
  dueAtMs: number;
  graceMs: number;
  lastCompletedAtMs: number | null;
  lastResultCount: number | null;
};

type AlertDecision =
  | { kind: "skip"; reason: "paused" | "not-due" | "outside-rollout" }
  | { kind: "healthy" }
  | { kind: "open"; reason: "missing-completion" | "zero-results" };

function evaluateImport(
  status: ImportStatus,
  nowMs: number,
  rollout: Rollout,
): AlertDecision {
  if (status.paused) return { kind: "skip", reason: "paused" };
  if (nowMs < status.dueAtMs + status.graceMs) {
    return { kind: "skip", reason: "not-due" };
  }
  if (!isEnabled(status.tenantId, rollout)) {
    return { kind: "skip", reason: "outside-rollout" };
  }
  if (status.lastCompletedAtMs === null || status.lastCompletedAtMs < status.dueAtMs) {
    return { kind: "open", reason: "missing-completion" };
  }
  if (status.lastResultCount === 0) {
    return { kind: "open", reason: "zero-results" };
  }
  return { kind: "healthy" };
}

In production, the open decision should feed an idempotent state transition keyed by connection and expected run, not send a notification directly. Repeated evaluator runs can then observe the same condition without producing repeated pages. Recovery should close the same alert when a later qualifying completion arrives.

Short code, strict boundary.

Measure noise before widening exposure

The rollout percentage is a release control, not the success metric. Count evaluated imports, skipped reasons, newly opened alerts, repeated open observations, recovered alerts, and acknowledged false positives. Segment those counts by cohort and alert reason. Never put tenant IDs into metric labels; keep identifiers in structured event records where access and retention can be controlled.

Use consistent severity semantics for those event records. RFC 5424 defines eight numerical severity values from Emergency through Debug and states that lower numerical values are more severe. That gives teams a shared vocabulary, but it does not choose a business threshold for an import. A delayed routine import might be a warning, while a prolonged loss of a contractual data feed may justify an error. Encode that policy once and test it.

Evidence	What it answers	Widen only when
Eligible versus evaluated imports	Is the rollout reaching the intended population?	Unexpected skips are explained
Alerts by reason	Which detector creates the load?	The reason mix matches the import contract
Recovery time	Does the state machine clear useful alerts?	Open alerts do not remain stale
False-positive review	Are operators learning anything actionable?	Noise stays inside the team's stated budget

No universal false-positive percentage belongs here. The acceptable level depends on who receives the notification, how quickly they can verify the upstream source, and what a missed import costs the customer. Write that budget down before the first cohort is enabled. A rollout without a stop condition is just a slow launch.

Stop when it gets noisy.

Cost matters too, especially for a small team. Evaluate from stored import state rather than adding a probe to every customer source. Aggregate low-cardinality metrics, sample verbose success events if necessary, and retain the decision fields needed to explain an alert. Cutting the evidence that distinguishes outside-rollout from not-due saves little and makes support slower.

Freeze the contract across workers

Every runtime that evaluates the flag must agree on the exact hash algorithm, UTF-8 encoding, tuple serialization, unsigned integer read, and modulo operation. Put golden vectors in the repository: fixed inputs with expected cohorts. A test should cover 0 and 100 percent, an empty subject, boundary cohorts, non-ASCII identifiers already accepted by the application, and the same tenant under two different flag keys.

Do not change normalization casually. Lowercasing Acme to acme, trimming whitespace, or replacing the JSON tuple with string concatenation changes assignments. If canonical tenant IDs are required, canonicalize them where they are created and pass the canonical form into the bucketer. The flag function should not guess.

Deployment order is equally practical. Ship dormant evaluation code first, verify decision events without notifying, enable a small percentage, and widen only after the signal review. A rollback sets the percentage to zero. Keep the alert records already created so operators can close or investigate them; disabling a flag should stop new evaluations, not erase operational history.

Native process failures sit outside ordinary application exceptions. If the evaluator runs in a desktop process, Electron's crashReporter documentation describes collecting native crash reports and notes platform-specific behavior. That is a separate telemetry path from import-health events, and it should remain separate: a process crash can explain missing evaluation, but it is not evidence that a customer's import returned zero rows.

Operational handoff

Before enabling notifications, confirm that one stable tenant identifier reaches every evaluator, the schedule and grace rules are stored rather than inferred from wall-clock guesses, and golden vectors pass in each deployed runtime. Check that alert opening and recovery are idempotent, decision records include flag key, version, cohort, percentage, reason, and evaluation time, and metrics avoid tenant-level label cardinality. Finally, name the person who can set exposure to zero and the evidence they will use.

Then rehearse the boring cases: a paused connection, a late completion inside the grace interval, a completion with zero results, a worker retry, a rollout decrease, and a recovery after an alert opens. If those outcomes are predictable, increasing exposure is an operational decision rather than a leap of faith.

The useful mechanism is small: deterministic tenant cohorts around a stateful detector. Its value comes from preserving the boundary between release eligibility and import health, because that boundary lets a small team tune signal quality without losing the ability to explain any alert.

Further reading

• RFC 5424, The Syslog Protocol: https://datatracker.ietf.org/doc/html/rfc5424
• Electron crashReporter API: https://www.electronjs.org/docs/latest/api/crash-reporter