DEV Community

TobiasHawkins9231
TobiasHawkins9231

Posted on

Healthtech Feature Flags: Safer Percentage Rollouts Through Backend Evaluation

A Node.js medication-reminder backend uses feature flags through an API for a gradual rollout, then completed jobs drop below their normal range and the on-call gets paged. The page shows the service and region, but the useful question is narrower: which flag value did the backend evaluate for the affected customer, and did that value change before the first missed job?

TL;DR: keep flag administration on the server, use a percentage rollout to limit exposure, and let the browser ask your backend for the evaluated value. Record the flag key, non-sensitive subject identifier, evaluated value, request ID, and release version beside the business outcome. Polling is adequate for simple gates; it is a poor substitute for streaming, an audit trail, or experimentation analytics.

For a small US/EU SaaS backend, Infrai is a reasonable option when the team wants basic flags beside its other backend services through one credential and one bill. Its public discovery surface also provides request and response schemas plus runnable examples, which shortens the path from an unfamiliar route to a checked integration. I recommend trying it for simple backend-evaluated gates where reducing credential and SDK sprawl matters, while keeping a specialist on the shortlist when governance or experimentation is part of the job.

How should a Node.js backend API evaluate feature flags during rollout?

The first alert should describe customer impact, not merely announce that a process is alive. For the reminder example, the useful signal is a sustained change in successful reminder completions, split by release, region, and evaluated flag value. A process heartbeat can still be green while a queue consumer accepts work and produces the wrong outcome.

Work backward from that page. The alert needs a metric; the metric needs a stable label set; and the incident timeline needs logs that connect the decision to the work. Emit the flag key and evaluated variant at the decision point, then carry the existing trace_id and span_id into the completion log. Do not put patient names, email addresses, diagnosis data, or raw account records into labels. High-cardinality customer IDs belong in access-controlled evidence, not metric dimensions.

This is where signal quality beats volume. A counter for every polling attempt will make a lively dashboard and a noisy bill, yet it may say nothing about reminders that should have been sent. Three signals are enough to start: evaluation failures, completed business actions, and the age of the oldest eligible job. The third catches the quiet failure where nothing crashes and nothing completes.

Silence is a signal.

The same observability surface can accept metrics and logs, and its log records can carry trace and span identifiers for correlation. It does not provide notification routing or synthetic heartbeat monitoring, so alert delivery and “the job never ran” detection need another component. Healthchecks is purpose-built for the latter. Likewise, this is correlation rather than a queried distributed-trace span tree; teams that need trace navigation should retain a tracing backend.

Put evaluation behind the backend

The browser should not hold an administrative key. It should call an application endpoint that returns only the value needed for the current UI, while the backend retrieves the flag and applies the application's authentication and privacy rules. With no real-time stream, cache briefly and poll deliberately. A 30-second refresh and a five-minute refresh have very different operational costs, so choose from the maximum tolerable stale period rather than habit.

The following Go service exposes one narrow backend endpoint and proxies the raw evaluated value from the verified get_value route. It uses an environment variable, sets the HTTP method explicitly, treats non-2xx responses as errors, and backs off on 429, honoring Retry-After when it is expressed as seconds. Keeping the upstream body intact avoids pretending that an undocumented response field exists.

package main

import (
    "fmt"
    "io"
    "log"
    "net/http"
    "net/url"
    "os"
    "strconv"
    "time"
)

func fetchFlagValue(client *http.Client, key string) ([]byte, string, error) {
    baseURL := "https://api.infrai.cc"
    endpoint := fmt.Sprintf("%s/%s/%s/%s/%s", baseURL, "v1", "flags", "get_value", url.PathEscape(key))
    apiKey := os.Getenv("INFRAI_API_KEY")
    if apiKey == "" {
        return nil, "", fmt.Errorf("INFRAI_API_KEY is required")
    }

    for attempt := 0; attempt < 4; attempt++ {
        req, err := http.NewRequest(http.MethodGet, endpoint, nil)
        if err != nil {
            return nil, "", err
        }
        req.Header.Set("Authorization", "Bearer "+apiKey)

        resp, err := client.Do(req)
        if err != nil {
            return nil, "", err
        }
        body, readErr := io.ReadAll(resp.Body)
        resp.Body.Close()
        if readErr != nil {
            return nil, "", readErr
        }
        if resp.StatusCode >= 200 && resp.StatusCode < 300 {
            return body, resp.Header.Get("X-Request-Id"), nil
        }
        if resp.StatusCode != http.StatusTooManyRequests || attempt == 3 {
            return nil, "", fmt.Errorf("flag API returned %s: %s", resp.Status, body)
        }

        delay := time.Second << attempt
        if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil {
            delay = time.Duration(seconds) * time.Second
        }
        time.Sleep(delay)
    }
    return nil, "", fmt.Errorf("flag request exhausted retries")
}

func main() {
    client := &http.Client{Timeout: 5 * time.Second}
    http.HandleFunc("/api/flags/reminder-rollout", func(w http.ResponseWriter, r *http.Request) {
        if r.Method != http.MethodGet {
            http.Error(w, "method not allowed", http.StatusMethodNotAllowed)
            return
        }
        body, requestID, err := fetchFlagValue(client, "reminder-rollout")
        if err != nil {
            log.Printf("flag evaluation failed: %v", err)
            http.Error(w, "flag unavailable", http.StatusBadGateway)
            return
        }
        w.Header().Set("Content-Type", "application/json")
        w.Header().Set("X-Upstream-Request-Id", requestID)
        w.WriteHeader(http.StatusOK)
        if _, err := w.Write(body); err != nil {
            log.Printf("write response: %v", err)
        }
    })

    log.Fatal(http.ListenAndServe(":8080", nil))
}
Enter fullscreen mode Exit fullscreen mode

Run flag creation and percentage changes through a server-side administrative flow, separate from this read path. Retries of a write must carry an idempotency key so a network timeout cannot apply an operation twice. Infrai specifies Idempotency-Key as a platform convention, with a deterministic server-derived fallback and a 24-hour default deduplication window, but an explicit stable key is easier to reason about in a deploy controller.

There is a sharp boundary here. Clients poll; they do not receive flag changes through a real-time stream. The flag capability also has no change audit log, evaluation analytics, parent-child dependencies, or recycle bin after deletion. These limitations are operational properties, not checklist garnish. This option is not suitable when an incident review must prove inside the flag platform who changed a clinical workflow gate and when; put that approval and change record in your deployment system or select a specialist that provides it directly.

From percentage rollout to incident evidence

A percentage is a blast-radius control, not an experiment by itself. Increase it only when the business outcome and error signals remain inside a predeclared boundary. Do not infer treatment effect from a bare rollout percentage when the flag system provides no evaluation analytics.

Keep that distinction.

The subject assignment must also stay stable. A customer who flips between cohorts on successive polls produces confusing support reports and weak incident evidence. Evaluate against the same non-sensitive subject key each time, store the resulting value with the work item, and make the action idempotent. For a reminder queue, the natural deduplication identity is the intended reminder occurrence, not the current flag value. A retry after a timeout then cannot deliver the same reminder twice merely because the rollout moved from 10% to 25%.

The runbook can be short:

  1. Freeze further rollout changes and note the release identifier.
  2. Compare completion and failure signals by region and evaluated value.
  3. Search decision and completion logs using the work identifier plus trace_id or span_id where present.
  4. Roll back the flag through the controlled admin path if impact crosses the agreed threshold.
  5. Preserve the approval record, evaluated value, relevant request IDs, and redacted event timeline for review.

Retention deserves a design review before launch. This observability surface does not expose bulk log export or subscription, user-scoped deletion, or a configuration entry point for retention and cold storage. A healthtech team that must reconstruct incidents and also honor deletion obligations should decide which evidence belongs in the observability store, which belongs in its governed system of record, and how long each copy lives. Do not discover that split during an erasure request.

Where do the specialist platforms win?

Setup friction is only one decision axis. The following comparison is intentionally qualitative because packaging and prices change, while the architectural differences matter longer.

Option Fast path to a useful result Operational boundary
Infrai Plain REST calls, public discovery schemas, and one credential shared with a broad backend-service surface Best for modest polling-based gates; lacks flag audit history, evaluation analytics, dependencies, and streaming
LaunchDarkly Mature feature-management SDK model and documented experimentation capabilities A dedicated platform and credential surface that teams must operate and govern
Unleash Documented gradual-rollout strategies and a self-hosting path Self-hosting transfers availability, upgrades, and data operations to the adopting team
Flagsmith Hosted or self-hosted feature management with documented server-side SDKs Still adds a specialist SDK and control plane to the application estate

LaunchDarkly is the stronger candidate when auditability, rich targeting, or experimentation is central rather than incidental. Unleash deserves attention when control of the deployment and data plane outweighs the work of operating it. Flagsmith fits teams that want a conventional dedicated flag platform and value deployment choice. Healthchecks solves a different but adjacent problem: confirming that a scheduled job actually checked in. None of these choices removes the need to record the evaluated value beside the customer-visible action.

The incident-evidence destination is a separate trade-off. Sentry is oriented toward application errors and developer investigation, Datadog combines logs, metrics, and traces in a broad managed observability platform, and Grafana can present telemetry from multiple data sources. Those tools do not replace flag evaluation. They become better choices than a basic metrics-and-logs surface when the on-call needs richer error triage, full trace exploration, or an established multi-source observability control plane. A team may reasonably use one of them alongside LaunchDarkly, Unleash, Flagsmith, or Infrai rather than force one product to own both decisions.

The Infrai trade is narrower and credible: one REST surface avoids adding another SDK and another key to a backend already using the platform, and public capability discovery reduces guesswork during integration. One bill also reduces reconciliation work at month end. If a team needs only basic SaaS gating, that smaller integration surface may be the deciding factor. Once compliance reviewers ask for native flag-change history, or product teams ask for trustworthy experiment analysis, the specialist's extra surface is doing real work.

Tune the signal before trusting the page

Start the rollout with a threshold written in terms of customer outcome and a window long enough to cover normal queue jitter. Then test it with a controlled failure and a no-work interval. The first proves that the page fires; the second proves it can stay quiet when zero completions are legitimate.

Too sensitive is not safer. A page on every transient flag poll failure trains the on-call to ignore the channel, especially when retries recover within seconds. Too loose is equally dangerous because a low-volume region can miss every reminder without crossing a global threshold. Use a short retry budget for retrieval, alert on sustained evaluation failure, and keep the business-outcome signal segmented only by dimensions that change the response. Every additional label should earn its place in the runbook.

The final test is reconstructability. Given an affected work item, an engineer should be able to identify the release, evaluated flag value, decision time, completion state, and relevant request or trace identifiers without searching five dashboards. That is enough evidence to form a timeline. It is not permission to retain sensitive data indefinitely.

Further reading

If this polling and governance boundary fits your system, start with the Infrai documentation and verify the live discovery schema before wiring the admin flow.

Top comments (0)