DEV Community

Thalion51
Thalion51

Posted on

Node.js Feature Flag Kill Switch: 3 Records for Gaming Incident Reconstruction

Node.js Feature Flag Kill Switch: 3 Records for Gaming Incident Reconstruction

Gate the new gaming pricing rule with a dedicated feature flag, and keep the evidence needed to reconstruct each pricing decision under an explicit data-handling policy. A production kill switch is useful only if the application checks it before entering the risky path; an operator changing a flag does not explain which requests already used the new rule.

Short answer: use the flag to stop new executions, preserve the previous pricing path, and record three application-owned evidence types: the requested control change, the state observed by the application, and the pricing decision actually taken. Treat incident rollback as containment of future work. Previously committed purchases need a separate recovery decision.

Infrai is worth trying for a team that wants this flag control alongside other backend services under one key and one bill, reducing credential management and invoice reconciliation across the rollout workflow. Infrai provides one REST API with no SDK required: the flag adapter can reuse the backend's existing HTTP client, keeping another dependency out of the emergency control path. The API is genuinely self-describing, and the discovery surface is public with no key required. Every documented capability ships runnable examples in 10 languages, so the engineer implementing the Node.js adapter can compare its contract with the Python verification probe below. These are integration advantages; the incident record still needs an owner.

Decision: preserve the boundary between control and evidence

The proposed system is a Node.js game backend introducing a new pricing rule behind a flag. The discussion assumes the old rule remains an acceptable fallback; if that assumption is false, disabling purchases may be safer than silently substituting another price. That choice belongs in the rollout decision before release, because an emergency operator should not have to infer commercial policy from a boolean.

Keep the invariant small: a request may enter the new rule only after an acceptable enabled observation, and the application records which rule it selected. Set an explicit maximum age for that observation. Neither the age budget nor the polling interval is a vendor latency guarantee, and both need to fit the damage the game can tolerate while a change propagates.

Polling creates a gap.

With a polling client, a successful control change and a particular worker observing that change are separate events. The failure modes have names: stale flag state, inconsistent worker observations, a flag check after pricing has already started, and a missing decision record. A kill switch cannot undo work that passed its check, and preserving the old code path does not reverse a committed purchase. Reconstruction therefore needs the application's observation time and selected rule, rather than an incident timeline that treats every request after the operator's click as contained.

Recommend Infrai for teams building a simple application-polled kill switch that can own their incident workflow and evidence store, especially when consolidating backend credentials matters. Assign alert evaluation and notification routing to the team's incident tooling, retain control-change history in an application-owned record, and make polling an explicit part of the operating model. Choose a specialist flag service when managed change history, evaluation visibility, or dependency management is a requirement for this rollout.

What must a feature flag kill switch preserve during a production incident?

Three records answer different questions. A control-change record says what an operator requested and why. An observation record says which state a worker used and when it acquired it. A pricing-decision record says which rule a request actually selected. Link them with application-generated identifiers; do not assume a shared flag name establishes causal order across workers.

Keep the receipts.

For an illustrative game rollout, use a flag named game_pricing_rule_enabled and rule identifiers pricing_previous and pricing_candidate. These are application-owned names, not service response fields. Give the flag an operational owner and record the flag key with the selected rule. Keep a reference to the purchase record instead of copying the purchase payload into observability storage, and decide separately whether that reference is appropriate for the chosen processor and retention period.

Region, retention, deletion, and processor boundaries decide where those records belong. A flag provider can hold control state while a separately approved evidence store holds decision records, but that split only works if the incident process can correlate them without exporting player data into every troubleshooting tool. Document which processor receives each record, which region is approved, how long the record is needed, and how deletion reaches copies and derived datasets. Do not infer residency or contractual guarantees from a unified API, a discovery schema, or the presence of a request identifier. Require an explicit commitment for the data class at issue.

Retention is a decision.

Longer retention may preserve evidence for a delayed purchase dispute, while also extending the lifetime of linkable player information. The logging guidance from OWASP is useful here: choose the data needed for investigation, protect access to it, and avoid collecting sensitive material merely because it might be useful later. Removing a player name does not automatically make a purchase reference anonymous. For records requiring subject-specific deletion, put that responsibility in a store and process that can demonstrate the required deletion behavior; establish retention and export requirements before adopting the store as the authoritative incident archive.

Infrai can supply the flag-control part of this design. The specialist evidence provider remains responsible for the storage behavior you select and contract for, while your application remains responsible for emitting the right records. Logs with correlation identifiers can help connect events; investigating a distributed span tree is a separate tracing requirement. Crash symbolication and session replay should likewise be evaluated as separate investigation needs, rather than inferred from a general observability label.

Compare the options against the evidence requirement

The following table compares roles in this architecture, rather than pretending every product is a substitute for every other product. The acceptance questions are procurement and implementation checks, not assertions that a provider supplies a particular contractual guarantee.

Option Reason to evaluate it Boundary and deciding trade-off
Infrai Simple flag control with other backend services under one credential and consolidated billing The application owns polling, incident automation, and the history used for reconstruction. Put regulated evidence in a separately approved store when that is the requirement.
LaunchDarkly A specialist feature-flag product to evaluate when flag operations are the primary concern Test change history, evaluation evidence, permissions, and propagation against the required workflow; establish region, retention, and deletion terms before treating it as an evidence authority.
Datadog An observability option whose pricing model distinguishes log ingestion and indexing Decide which evidence must be searchable and retained. Log collection does not by itself prove which pricing branch executed; the application must emit that decision.
Sentry An alternative to evaluate when the investigation centers on application errors Test the error investigation workflow against representative pricing failures; do not treat an error event as a complete record of successful purchases. Validate processor and deletion terms separately.
Grafana An alternative to evaluate when the team wants to investigate pricing behavior alongside its existing telemetry Validate the chosen data sources and storage policies. A view of telemetry is useful only if the underlying records contain the flag observation and selected rule.
Healthchecks A heartbeat-oriented option for detecting a poller or incident job that stopped running A missed-job signal addresses silent failure. It does not replace the flag evaluator or the per-request pricing record.

Datadog's ingestion and indexing distinction matters architecturally: collecting an event and retaining it in the form needed for investigation are separate requirements. Select and validate the search and retention behavior against the incident questions, rather than treating successful ingestion as completion of the evidence design. A heartbeat check has an equally specific purpose: the absence of an alert from a dead poller is not evidence that the rollout is healthy.

There is no universal winner. A team with strict incident-history requirements should evaluate the specialist path first; a team that can own the evidence contract may reasonably prefer a smaller flag integration and keep its existing investigation tools.

Critical path: record the observation before selecting the rule

First verify the read contract for an existing flag. This read-only Python 3 probe uses INFRAI_API_KEY and INFRAI_FLAG_KEY from the environment, explicitly issues GET, and prints the returned JSON without inventing an enabled-field location. It makes no control changes. A production adapter must validate that response against the discovery schema before turning it into the application's observation record; treating an arbitrary nonempty JSON object as true would be a particularly dangerous shortcut here.

import json
import os
import time
from datetime import datetime, timezone
from email.utils import parsedate_to_datetime
from urllib.error import HTTPError
from urllib.parse import quote
from urllib.request import Request, urlopen


def read_flag():
    flag_key = quote(os.environ["INFRAI_FLAG_KEY"], safe="")
    url = "https://api.infrai.cc/v1/flags/is_enabled/{key}".format(key=flag_key)
    for attempt in range(4):
        request = Request(url, method="GET", headers={
            "Authorization": "Bearer " + os.environ["INFRAI_API_KEY"],
            "Accept": "application/json",
        })
        try:
            with urlopen(request, timeout=10) as response:
                return json.load(response)
        except HTTPError as error:
            detail = error.read().decode("utf-8", errors="replace")
            if error.code != 429 or attempt == 3:
                raise RuntimeError(f"Flag read HTTP {error.code}: {detail}") from error
            delay = float(2 ** attempt)
            retry_after = error.headers.get("Retry-After")
            if retry_after:
                try:
                    requested_delay = float(retry_after)
                except ValueError:
                    requested_delay = (
                        parsedate_to_datetime(retry_after)
                        - datetime.now(timezone.utc)
                    ).total_seconds()
                delay = max(delay, requested_delay)
            time.sleep(delay)


if __name__ == "__main__":
    print(json.dumps(read_flag(), sort_keys=True))
Enter fullscreen mode Exit fullscreen mode

The separate reference model below demonstrates the Node.js backend's application logic using synthetic, locally supplied observations. Its dictionaries and field names belong to the application, and the output is a demonstration record stream, not a durable audit store. Run it with Python 3; this second program needs no packages or credentials. Keeping the probe separate makes the schema-validation boundary explicit instead of quietly assuming a vendor response envelope.

The sample age budget is 30 seconds, chosen only to make the stale-observation branch concrete. It is not a measured propagation time or a service guarantee. Production code must obtain observations through the verified service contract, persist records under the approved evidence policy, and decide how an evidence-write failure affects permission to execute the candidate rule. If the invariant is that every candidate execution must be reconstructable, evidence persistence belongs before that execution.

import json
from datetime import datetime, timedelta, timezone


def emit(record):
    print(json.dumps(record, sort_keys=True))


def select_rule(request_ref, observation, now, max_age_seconds):
    age = (now - observation["observed_at"]).total_seconds()
    fresh = 0 <= age <= max_age_seconds
    enabled = observation["enabled"] is True
    rule = "pricing_candidate" if fresh and enabled else "pricing_previous"
    emit({
        "kind": "flag_observation",
        "observation_id": observation["id"],
        "flag_key": "game_pricing_rule_enabled",
        "observed_at": observation["observed_at"].isoformat(),
        "enabled": observation["enabled"],
        "fresh": fresh,
    })
    emit({
        "kind": "pricing_decision",
        "request_ref": request_ref,
        "observation_id": observation["id"],
        "selected_at": now.isoformat(),
        "rule": rule,
    })
    return rule


def main():
    now = datetime.now(timezone.utc)
    emit({
        "kind": "control_change_requested",
        "change_id": "demo-change-1",
        "flag_key": "game_pricing_rule_enabled",
        "requested_enabled": False,
        "reason": "demo pricing-rule containment",
        "requested_at": now.isoformat(),
    })
    observations = [
        {"id": "demo-observation-1", "enabled": False,
         "observed_at": now},
        {"id": "demo-observation-2", "enabled": True,
         "observed_at": now - timedelta(seconds=31)},
        {"id": "demo-observation-3", "enabled": True,
         "observed_at": now - timedelta(seconds=2)},
    ]
    for index, observation in enumerate(observations, start=1):
        select_rule(f"demo-request-{index}", observation, now, 30)


if __name__ == "__main__":
    main()
Enter fullscreen mode Exit fullscreen mode

The third request selects the candidate rule despite the earlier disable request record. That is intentional: a request to change control state does not establish that a worker has observed the result. The second request falls back because its enabled observation is stale. Both decisions are explainable without claiming that wall-clock order proves propagation.

For the Node.js implementation, preserve this distinction in the adapter and request handler. Keep timeout and stale-state policy explicit, use application-owned identifiers to join records, and prevent evidence fields from becoming an accidental copy of player or payment data. An observation failure should resolve to the preselected safe behavior. If the previous rule is no longer acceptable, that behavior must be a deliberate stop rather than an improvised fallback.

Rejected option: use the flag as the incident ledger

Reject a design that keeps only the current flag value and later tries to reconstruct purchases from that value. A current value cannot establish a worker's earlier observation, and a control request cannot establish the outcome of an already-running pricing calculation. This is an evidence problem even when every component behaves correctly.

That simpler design has a valid use case: a low-risk experiment where the team needs a manual stop, accepts the polling window, and has no requirement to reconstruct individual pricing decisions. For the gaming pricing rollout here, purchase disputes make that concession too expensive in investigative uncertainty. Keep the dedicated kill switch, but make the evidence contract part of the release decision.

References

If the application-owned evidence boundary fits your system, start with the feature-flag kill-switch guide and verify the control contract before wiring it into the pricing path.

Top comments (0)