A feature flag service becomes difficult at the moment a toggle starts making an experiment decision. The hard constraint is signal quality: a flag evaluation must tell us which tenant saw which variant, while the event stream must not turn every retry, health check, and malformed request into evidence.
TL;DR: keep the toggle path deterministic and small, attach an immutable evaluation event to the tenant and experiment cohort, and measure exposure separately from outcome. A Node.js service can expose one evaluation operation and one administrative write path; the useful observability work is the contract around those operations, not a large dashboard.
How do you build a Node.js feature flags service with a stable toggle?
For a developer-tools experiment, the basic record is not “flag X was on.” It is: tenant t-184 received variant control for experiment editor-v2 at a known time, under a known flag version. That distinction matters when a tenant makes five requests and only one request reaches the new code. Request count is traffic. Exposure is an experimental fact.
I would make the evaluation result explicit:
def evaluate(flag_key, tenant_id, rules, salt):
assignment = stable_assignment(f"{salt}:{flag_key}:{tenant_id}")
variant = rules.variant_for(assignment)
return {
"flag_key": flag_key,
"tenant_id": tenant_id,
"variant": variant,
"rule_version": rules.version,
"assignment": assignment,
}
The assignment must be stable for the same tenant and flag version. Do not use a random number per request. Random request-level toggles create noisy cohorts, make a support ticket impossible to reproduce, and can expose a tenant to both variants during one session. The implementation can live behind a Node.js HTTP boundary, but the rule is language-independent.
The event should include experiment_key, tenant_id, variant, rule_version, evaluation_id, and a timestamp. Keep outcome events separate. A successful evaluation is not a successful product action.
That is the first trade-off: a small evaluation contract is easier to audit, but it cannot answer an outcome question by itself.
Where do the three signals disagree?
I use three signals because each catches a different false conclusion.
- Exposure says that the evaluated variant was returned and accepted by the caller.
- Outcome says that the behavior under test completed, such as a document save or build submission.
- Operational health says whether latency, errors, retries, and delivery failures changed during the rollout.
A cohort with 10,000 evaluations and 40 completed actions is not automatically a bad variant. It may contain tenants that never reached the relevant workflow. Conversely, a clean outcome rate can hide a timeout spike if the clients abandon requests before reporting the result. I started out treating evaluation volume as the denominator. Later I found that it was usually the easiest number to inflate.
Log the evaluation as a structured event, with a schema version and a correlation identifier. OpenTelemetry describes logs as a signal that can be correlated with traces and metrics; that makes it a useful transport-neutral vocabulary for this boundary, not a reason to put every experiment calculation in a logging backend.
def exposure_event(result, trace_id):
return {
"event_name": "flag.exposure",
"schema_version": 1,
"trace_id": trace_id,
"attributes": {
"experiment_key": result["flag_key"],
"tenant_id": result["tenant_id"],
"variant": result["variant"],
"rule_version": result["rule_version"],
"evaluation_id": result["evaluation_id"],
},
}
Do not put email addresses, phone numbers, raw authorization headers, or full request bodies into this event. In notification systems, observability data can outlive the request that created it. Tenant identifiers still deserve access control and retention rules.
How should the service boundary stay small?
The read path should answer one question: given a tenant and flag key, which variant applies? The write path should publish a new immutable rule version. A cache may make reads fast, but a cache miss must have a defined behavior. For an experiment, returning a default variant is often safer than blocking the application request, provided the default is recorded as fallback and excluded from the primary analysis.
Use a monotonic rule version and reject stale writes. Validate that a percentage allocation sums to 100, that a variant exists, and that a tenant cannot match two mutually exclusive rules. These checks belong before publication. A malformed rule should fail an administrative request, not become a quiet production default.
Keep the control plane and evaluation plane conceptually separate. The control plane changes rules; the evaluation plane reads a published snapshot. This lets an audit trail answer who changed a rule without making each application request depend on the control database. It also gives a rollout its first useful rollback primitive: restore the previous version, then verify new evaluations against that version.
The practical failure modes are ordinary: a client retries after a timeout, a worker evaluates again, or a deployment changes the default while the cache still serves the old snapshot. Add evaluation_id, rule_version, and source fields so a duplicate can be recognized instead of counted as another exposure. Idempotent event ingestion helps, but it does not repair a bad denominator.
Determinism wins.
What changes when you compare existing flag systems?
The comparison should start with the boundary above, not with a feature checklist. LaunchDarkly documents prerequisite targeting and variation evaluation; Unleash exposes strategies and constraints; Flagsmith documents identity and segment-based evaluation. Those models differ in where targeting rules execute and which context the client must provide.
That difference is operational. A local evaluation path can reduce dependency on a network round trip, while a remote path centralizes rule freshness and policy enforcement. A system that evaluates on the server gives the backend more control over sensitive tenant attributes. None of these choices proves that a cohort is clean. You still need a stable assignment key, an exposure event, a joinable outcome, and a plan for missing telemetry.
The trade-off is unavoidable: centralization improves policy consistency, while local evaluation can keep an application moving during a control-plane outage. Choose the failure mode that your tenant contract can explain, and document the fallback variant instead of pretending the two paths are equivalent.
For a neutral review, I would test three cases before selecting an implementation: the same tenant evaluated 100 times, a rule update racing with a request, and a retry that replays the same evaluation. Record the expected variant, rule version, event identity, and fallback behavior. The boring cases are where the experiment denominator gets damaged.
A compact rollout plan
Start with one flag, two variants, and one tenant cohort. Store the published rule version beside every exposure. During a dry run, calculate exposure-to-outcome joins without changing user behavior; this catches missing tenant IDs and outcome events before they affect a decision.
Then release to a small, explicitly named cohort. Watch operational health independently from the experiment result. If the service reports a spike in latency or evaluation failures, pause the rollout even when the outcome metric looks favorable. Finally, remove the flag after the decision. A permanent toggle is an extra branch, an extra state, and another place for an old rule version to be misread.
The rollout record should make the decision reversible: name the cohort, retain the published rule version, record the fallback path, and link the exposure event to the outcome event without copying sensitive request data into either one. When a result looks strange, this gives an engineer a bounded investigation: verify the assignment, verify the rule snapshot, verify that the caller accepted the variant, then inspect the business outcome. Without those joins, a dashboard can show a clean percentage while hiding that one tenant was counted in both variants after a retry.
The useful design is modest: deterministic assignment, immutable versions, explicit exposure, separate outcomes, and observable fallbacks. That combination gives a tenant cohort a defensible denominator. It also keeps the Node.js implementation replaceable, which is exactly what a feature-flag boundary should be.
Sources
- OpenTelemetry, “Logs signal concepts”: https://opentelemetry.io/docs/concepts/signals/logs/
- LaunchDarkly, “Targeting and variations”: https://launchdarkly.com/docs/home/flags/targeting
- Unleash, “Strategies”: https://docs.getunleash.io/reference/activation-strategies
- Flagsmith, “Segments”: https://docs.flagsmith.com/features/segments
Top comments (0)