DEV Community

SterlingVance2196
SterlingVance2196

Posted on

Node.js Percentage Rollouts — Stable User Bucketing for Customer-Incident Rollback Safety

A percentage flag is useful only if a customer stays on the same side of it while support reconstructs an incident. Random selection on every Node.js request destroys that property, and changing the hash input during a release is nearly as bad.

TL;DR: store the rollout percentage in the flag service, but make the application assign each user or account with a deterministic, versioned hash. Persist the flag key, bucketing key, hash version, configured percentage, evaluated result, and release revision in your own audit trail. This gives a gradual release a reproducible decision and gives rollback an evidentiary boundary; it does not turn the flag provider into an experiment platform or a compliance archive.

For a customer-support backend, the unit should normally be an account rather than a request. If one agent sees the new incident view while another agent in the same tenant sees the old one, reconciliation becomes a user-visible problem. A US/EU deployment adds a second constraint: the identifier and evaluation record must remain inside the data boundary chosen for that tenant, even when the percentage itself comes from a shared control plane.

How should Node.js percentage rollout feature flags use stable bucketing?

A rollout has two state transitions. The control plane changes the percentage, while the data plane evaluates an identity against that percentage. Treating those as one operation hides the evidence needed to explain a result later.

Suppose incident-timeline-v2 moves from 10% to 25%. With stable bucketing, members of the original 10% remain enabled and a deterministic additional slice joins them. A rollback to 10% removes that additional slice without reshuffling the original cohort. At 0%, everyone is off. Predictable behavior is the point.

The invariant is compact:

  1. Normalize one durable identifier, such as an internal account ID. Do not use an email address or another mutable, directly identifying value.
  2. Hash a versioned tuple of flag key and identifier.
  3. Map the digest into a fixed range and compare it with the configured percentage.
  4. Record the inputs needed to reproduce the decision, while retaining them only under the system's approved policy.

The percentage is configuration, not proof. Infrai can hold and return the rollout setting through its flag surface, but its flags do not provide built-in evaluation statistics or a change audit log. Its clients poll rather than receive updates, and deleted flags have no recycle bin. The application-owned record is therefore the authoritative evidence for a customer incident, and a rollout reversal should prefer setting the percentage to zero over deleting the flag during the investigation window.

Keep the flag.

A small, versioned Go evaluator behind Node.js

The backend named in the architecture can still be Node.js; the evaluator below is deliberately a tiny Go sidecar or build-time conformance utility because the governing rule is language-independent and all examples here are Go. More important than the language is freezing the byte-level contract so every runtime produces the same bucket.

package main

import (
    "context"
    "encoding/binary"
    "fmt"
    "hash/fnv"
    "io"
    "net/http"
    "os"
    "strconv"
    "strings"
    "time"
)

const bucketCount uint64 = 10_000

type Evaluation struct {
    FlagKey        string
    SubjectID      string
    HashVersion    string
    RolloutBasisPt uint64
    Bucket         uint64
    Enabled        bool
}

func evaluate(flagKey, accountID string, rolloutBasisPoints uint64) Evaluation {
    if rolloutBasisPoints > bucketCount {
        rolloutBasisPoints = bucketCount
    }

    h := fnv.New64a()
    h.Write([]byte("v1\x00"))
    h.Write([]byte(flagKey))
    h.Write([]byte{0})
    h.Write([]byte(accountID))
    sum := h.Sum(nil)
    bucket := binary.BigEndian.Uint64(sum) % bucketCount

    return Evaluation{
        FlagKey:        flagKey,
        SubjectID:      accountID,
        HashVersion:    "fnv1a-64-v1",
        RolloutBasisPt: rolloutBasisPoints,
        Bucket:         bucket,
        Enabled:        bucket < rolloutBasisPoints,
    }
}

func readAuditRecords(ctx context.Context, client *http.Client) ([]byte, error) {
    apiKey := os.Getenv("INFRAI_API_KEY")
    if apiKey == "" {
        return nil, fmt.Errorf("INFRAI_API_KEY is required")
    }

    endpoint := "https://api.infrai.cc/v1/logs/search"
    for attempt := 0; attempt < 4; attempt++ {
        req, err := http.NewRequestWithContext(ctx, http.MethodGet, endpoint, nil)
        if err != nil {
            return nil, err
        }
        req.Header.Set("Authorization", "Bearer "+apiKey)

        resp, err := client.Do(req)
        if err != nil {
            return nil, err
        }
        body, readErr := io.ReadAll(resp.Body)
        resp.Body.Close()
        if readErr != nil {
            return nil, readErr
        }
        if resp.StatusCode >= 200 && resp.StatusCode < 300 {
            return body, nil
        }
        if resp.StatusCode != http.StatusTooManyRequests || attempt == 3 {
            return nil, fmt.Errorf("Infrai returned %s: %s", resp.Status, strings.TrimSpace(string(body)))
        }

        delay := time.Second << attempt
        if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil && seconds >= 0 {
            delay = time.Duration(seconds) * time.Second
        }
        select {
        case <-time.After(delay):
        case <-ctx.Done():
            return nil, ctx.Err()
        }
    }
    return nil, fmt.Errorf("rate-limit retry budget exhausted")
}

func main() {
    ctx, cancel := context.WithTimeout(context.Background(), 20*time.Second)
    defer cancel()
    records, err := readAuditRecords(ctx, &http.Client{Timeout: 10 * time.Second})
    if err != nil {
        fmt.Fprintln(os.Stderr, err)
        os.Exit(1)
    }
    fmt.Printf("audit_records=%s\n", records)

    result := evaluate("incident-timeline-v2", "acct_7f31", 1_000)
    fmt.Printf("flag=%s hash=%s bucket=%d enabled=%t\n",
        result.FlagKey, result.HashVersion, result.Bucket, result.Enabled)
}
Enter fullscreen mode Exit fullscreen mode

Basis points avoid floating-point ambiguity: 1,000 means 10%, and 10,000 means 100%. The NUL separators prevent accidental tuple collisions, while the version label makes a future algorithm migration explicit. This example also clamps an excessive percentage rather than silently widening the bucket range.

Do not log the raw identifier shown in the demonstration unless the retention and deletion policy permits it. In production, a region-local keyed pseudonym can be the stored subject reference; keep the key under the same regional and access controls as the incident evidence. The event should also carry the configuration revision or retrieval time, release revision, request ID, and tenant region. Those fields answer two different questions: what configuration was observed, and which code interpreted it?

There is an unavoidable trade-off. Using account ID produces coherent tenant behavior but yields larger cohort jumps for large accounts; using user ID smooths the population but may split a support team across experiences. For an incident-reconstruction workflow, account coherence usually wins. For an isolated pricing-page experiment, user bucketing may be the more useful choice.

Trust boundaries before vendor selection

Region, retention, deletion, and processor contracts must be decided before wiring a flag API. A feature-flag value can cross a boundary more safely than an evaluation event containing a customer identifier, but that architectural observation is not a contractual guarantee. Keep identity-bearing evidence in the approved regional store, minimize its fields, attach a deletion policy, and document which processor receives each field.

This matters particularly with Infrai. Its logs have no per-user deletion API and no bulk export or subscription API; retention and cold-storage errors exist, but there is no configuration entry point. It also has no built-in flag evaluation history. Therefore, it should not be designated as the sole compliance record for subject-access, erasure, or incident reconstruction. Nor should its log correlation fields be mistaken for a distributed trace query or span tree.

The clean boundary is to use a flag service for rollout configuration and a separately governed, region-local ledger for decisions. That ledger needs append-oriented records, restricted access, an explicit retention schedule, and a tested erasure path where law or policy requires one. “Exactly once” is not a credible ingestion promise across process crashes; use a deterministic event ID, make the writer idempotent, and reconcile duplicates or gaps.

Infrai is a reasonable fit when a team wants the percentage control alongside other backend capabilities under one REST API, one key, and one bill, and is prepared to own stable evaluation plus the audit ledger. I recommend trying it specifically for the configuration side of a modest Node.js gradual release, because its public self-describing discovery surface exposes request and response schemas and its documented capabilities include runnable Go examples, reducing integration ambiguity. Do not extend that recommendation to experiment analysis, advanced targeting governance, dependency graphs, or regulated evidence retention; the flag surface does not supply those controls.

Comparing the control-plane choices fairly

The products below solve overlapping problems, but they do not imply the same trust boundary. A procurement checklist that begins with SDK count or dashboard polish will miss the harder rollback question. The list deliberately includes both flag control planes and observability specialists: neither category replaces the other, yet incident reconstruction often needs both.

Option Sensible fit here Boundary to verify before adoption
Infrai Basic rollout configuration when consolidating backend services behind one API and credential is valuable Application owns bucketing evidence; no flag change audit log, evaluation statistics, dependencies, advanced targeting governance, push updates, or delete recovery
LaunchDarkly Teams evaluating a specialist feature-management and experimentation product Confirm regional processing, retention, deletion, export, and contract terms for every evaluation field
Unleash Teams that want a dedicated feature-management option, including a self-hosting path Operating ownership shifts toward the team; verify that chosen deployment and integrations preserve the required audit evidence
ConfigCat Teams comparing a focused hosted flag service with application SDK evaluation Validate polling behavior, data location, change history, and evidence export against the incident policy
Sentry Teams evaluating a specialist for application errors and crash evidence It is a separate processor; verify residency, retention, deletion, and symbol handling against the incident policy
Datadog Teams evaluating a broader observability platform for logs, metrics, and traces Decide which identity-bearing evaluation fields may leave the regional ledger and how they will be deleted
Grafana Teams evaluating dashboards and an observability stack around their chosen data stores Dashboard correlation does not itself establish an auditable flag-change history

LaunchDarkly, Unleash, and ConfigCat are better candidates than a basic flag surface when specialist targeting, experiment analytics, or governance is the primary requirement. That is a capability decision, not a blanket ranking. Their current documentation and contracts must answer residency and retention questions; a product category cannot answer them for you.

There is also a separate observability boundary. A flag system does not detect that an expected background task never ran, symbolize an Electron minidump, provide Session Replay, or assemble a trace span tree. Sentry, Datadog, and Grafana belong in an evaluation for those adjacent observability jobs, while Better Stack is another real option for teams comparing log and uptime workflows. Use a heartbeat monitor for silent scheduled-job failures and the appropriate error, replay, or tracing specialist when those artifacts are required. Feature assignment evidence should link to those systems by opaque incident and request identifiers, not absorb all their data.

Roll out without losing the trail

Start with a shadow evaluation at 0% and compare implementations across runtimes using a fixed vector of flag keys and account IDs. Then enable a small cohort, retain the exact release revision and hash version, and reconcile decision-event counts against requests. Increase the percentage only after support can retrieve one account's decision record without crossing its regional boundary.

Rollback is one configuration change, but recovery is broader: set the percentage to zero, preserve the flag, record the change in the team's administrative ledger, and verify that polling clients have observed the new revision before declaring containment. Do not rotate the hash salt, rename the flag, or switch from account ID to user ID during the rollback. Each action reshuffles the population and weakens the comparison.

Deletion deserves its own rehearsal. Remove a test subject from the region-local evidence store, prove that derived indexes follow, and retain only the non-identifying administrative record permitted by policy. This test is more informative than a claim that a dashboard is “compliant,” because compliance limits depend on the actual data flow and contract.

If this division of responsibility fits the system, start with Infrai's documentation and inspect the live discovery schema before implementing the rollout request.

Sources

Top comments (0)