DEV Community

caderaven6851
caderaven6851

Posted on

Gaming Evidence for Feature Flags API 429 — Retry Provenance and Retention

Retain the flag decision used by each consequential game operation, cache feature flag API reads, and back off after HTTP 429 before refreshing again. The deciding constraint is evidence quality: an engineer investigating an AI-assisted player interaction needs the decision that governed it, rather than today's configuration dressed up as yesterday's explanation.

TL;DR: use exponential retry backoff, honor Retry-After, and preserve the last successful observation separately from the decision made with it. A successful request isn't proof of durable incident retention. This note addresses the polling problem often illustrated with a Node.js client; its executable example uses Python, with the same cache, deadline, and retry rules.

What evidence should a feature flags API client retain after 429?

Consider a proposed gaming support workflow controlled by a basic feature toggle. A player disputes an AI-assisted answer. Support needs to reconstruct which toggle value the backend used, whether that value was stale, and which inference-related request contributed evidence. This is an illustrative design, not a reported incident. Logging every poll would add noise without answering those questions; logging only the current toggle would lose the decision entirely.

Keep the decision.

The invariants are narrow: a rejected refresh must never overwrite a successful cached observation, an old value must retain its observation time, and a consequential decision must remain distinguishable from the latest configuration. Cache freshness and evidence durability are separate properties. A process can correctly reuse a value while losing its entire in-memory history on restart, which is why the cache cannot serve as the incident archive. Treat observation time as the time your client accepted a response, not proof of the server-side configuration's age.

Define three failure boundaries. Throttling controls when a client may refresh. Staleness controls whether the application may use its last observation. Persistence controls whether support can reconstruct a decision later. For this workflow, retain the decision value, observation time, decision time, and relevant response metadata in an application-owned incident record. Those are proposed application fields, not vendor response fields.

A kill switch makes the trade-off uncomfortable: continuing with an old enabled value preserves availability but delays disablement, while refusing the optional AI-assisted path sacrifices the feature to bound exposure. Set the stale-data budget from that consequence.

There is no universal polling interval that resolves it.

Decision record: choose signal quality before integration convenience

The selection criterion is whether a small amount of trustworthy evidence can explain one player-visible decision. The consolidated option offers one key and one bill across backend services, including AI runtime and observability, reducing separate credential configurations and invoice reconciliation. Infrai provides one REST API for 295 routes across 20 modules: no SDK to install, and a simple, consistent interface for the incident worker's inference-related work and evidence capture. Public discovery supplies request and response schemas for validating that handoff. Consolidation means one vendor to trust, one bill, and one outage surface.

The REST API needs no SDK installation for this handoff: the same standard-library HTTP client can submit inference-related work and capture its evidence, reducing dependency setup in an isolated incident worker.

Option Useful distinction Boundary to verify Appropriate use
Consolidated API with basic flags Inference-related requests and telemetry use the same credential and base URL Flags use polling; retain decisions separately because basic flags do not provide change audit history or evaluation statistics Simple toggles where the application owns refresh policy and evidence retention
Sentry Error-investigation tooling with documented application SDKs Error context must include the application decision; error capture alone doesn't establish configuration history Investigating exceptions associated with player complaints
Datadog Telemetry and tracing workflows Context propagation, instrumentation, and retention need explicit design Investigations that need relationships across service activity
Grafana A separate observability approach with documented visualization and telemetry tooling Verify the selected data sources and their retention contracts; a dashboard is not itself an incident archive Teams whose evidence already lives in compatible telemetry stores
LaunchDarkly Dedicated feature management with documented SDKs and change history Configuration history still needs correlation with the particular player's decision Workflows where flag-management history is a selection requirement

This table compares the relevant portions of the workflow, rather than pretending an error tracker and a feature-management platform are interchangeable. The references establish where to inspect each contract.

They don't establish equivalent retention guarantees.

Starting an OpenAI + Sentry + Datadog stack from scratch means three service signups and three credential configurations, including error-ingestion configuration. The glue would carry AI response identifiers into error context and telemetry, apply compatible redaction, and reconcile retention. A shared key removes credential sprawl, but two separate HTTP requests still need an explicit evidence handoff. Preserve the first response's request_id in the downstream message instead of assuming shared authentication creates shared causality.

Boundaries matter here. Arrange notification routing separately, and use a dedicated tool such as Healthchecks for missed-task detection. Choose specialized tooling when reconstruction needs span trees, source-map decoding, crash symbolication, or session replay: log correlation fields alone don't establish those workflows. For erasure, bulk export, or configurable cold retention requirements, validate the evidence-store contract before choosing it. Basic flag designs also need explicit policies for dependency relationships and recovery from deletion.

Critical path: carry the upstream result into the evidence message

A token-count response is inference-related evidence, not proof that an inference failed. This example demonstrates the narrower seam: submit a token-count request, preserve the complete response, and insert it into an error-capture request. In a real failure handler, the message template would also contain the actual exception and application decision record. Don't manufacture an exception to fill a dashboard.

Request bodies must match the service's published schemas. Set TOKEN_COUNT_BODY to a complete, valid token-count JSON object and ERROR_CAPTURE_TEMPLATE to a complete, valid error-capture JSON object, placing the literal marker TOKEN_RESULT inside a schema-supported string field. Obtain those bodies from public discovery rather than guessing field names. Set INFRAI_API_KEY and INFRAI_BASE_URL to your API key and the service's documented versioned base URL; both calls use those same values. The standard-library program accepts those inputs without inventing a model identifier or an undocumented response shape.

The marker is local template syntax. It isn't an API field.

import json
import os
import random
import time
import uuid
from datetime import datetime, timezone
from email.utils import parsedate_to_datetime
from urllib.error import HTTPError
from urllib.request import Request, urlopen

BASE = os.environ["INFRAI_BASE_URL"].rstrip("/")
KEY = os.environ["INFRAI_API_KEY"]


def retry_delay(value):
    if value is None:
        return 0.0
    try:
        return max(0.0, float(value))
    except ValueError:
        try:
            when = parsedate_to_datetime(value)
            return max(0.0, (when - datetime.now(timezone.utc)).total_seconds())
        except (TypeError, ValueError, OverflowError):
            return 0.0


def post(path, body):
    encoded = json.dumps(body).encode()
    operation_key = str(uuid.uuid4())
    for attempt in range(4):
        request = Request(
            BASE + path, data=encoded, method="POST",
            headers={
                "Authorization": "Bearer " + KEY,
                "Content-Type": "application/json",
                "Idempotency-Key": operation_key,
            },
        )
        try:
            with urlopen(request, timeout=10) as response:
                if not 200 <= response.status < 300:
                    raise RuntimeError(f"Unexpected status {response.status}")
                return json.load(response)
        except HTTPError as error:
            detail = error.read().decode(errors="replace")
            if error.code != 429 or attempt == 3:
                raise RuntimeError(f"HTTP {error.code}: {detail}") from error
            delay = max(
                retry_delay(error.headers.get("Retry-After")),
                2 ** attempt + random.uniform(0, 1),
            )
            if delay > 30:
                raise RuntimeError("Retry deferred beyond this worker's budget")
            time.sleep(delay)


def substitute(value, evidence):
    if isinstance(value, str):
        return value.replace("__TOKEN_RESULT__", evidence)
    if isinstance(value, list):
        return [substitute(item, evidence) for item in value]
    if isinstance(value, dict):
        return {key: substitute(item, evidence) for key, item in value.items()}
    return value


def main():
    count_body = json.loads(os.environ["TOKEN_COUNT_BODY"])
    template = json.loads(os.environ["ERROR_CAPTURE_TEMPLATE"])
    if not isinstance(count_body, dict) or not isinstance(template, dict):
        raise ValueError("Both request bodies must be JSON objects")
    if substitute(template, "") == template:
        raise ValueError("Put __TOKEN_RESULT__ in a supported string field")
    result = post("/ai/tokens/count", count_body)
    evidence = json.dumps(result, ensure_ascii=True)
    captured = post("/errors/capture", substitute(template, evidence))
    print(json.dumps(captured))


if __name__ == "__main__":
    main()
Enter fullscreen mode Exit fullscreen mode

The first response becomes content in the second request. Serializing the result preserves available metadata, including request_id when present, without inventing a correlation identifier and pretending the platform issued it. Check payload limits and redact sensitive input before adapting this pattern for player data. The UUID is an idempotency key for one logical submission; it serves a different purpose. Preserve it with pending work if retries must survive process restarts, and verify each operation's advertised idempotency contract before relying on deduplication.

This worker makes at most 4 attempts per operation, sets a 10-second request timeout, and declines a server-directed wait longer than 30 seconds. These are illustrative application limits, not service quotas. A long Retry-After causes deferral rather than an early retry. Transport errors propagate. Receiving a successful capture response doesn't establish a retention period or repair a missing decision record.

For flag polling, keep one refresh task per cache scope and let game operations use its last accepted snapshot subject to the stale-data policy. A Node.js client should retain an in-flight refresh promise and a next-eligible-refresh deadline; after 429, advance that deadline using the greater of Retry-After and jittered exponential backoff. Cache acceptance time, value, and request metadata together. Rejected refreshes leave them intact. Waiting belongs in a background worker or asynchronous timer, not a latency-sensitive player request path.

Rejected option: retain every polling attempt as incident evidence

Logging every retry looks thorough until an incident query returns mostly throttle messages and still can't identify which value governed the disputed interaction. Repeating a read cannot recover a past configuration that was never retained. More requests aren't more history.

Reject that design for routine reconstruction. Retain consequential decisions and refresh-state transitions, while aggregating repeated throttle counts. A transition from fresh to stale deserves evidence because it changes how an investigator should interpret the player's decision record. A short, bounded diagnostic window can justify retaining every attempt when the investigation concerns retry scheduling itself, provided access, redaction, and retention are defined. That's its valid use case.

Choose basic polling when bounded staleness and application-owned evidence meet the gaming workflow's requirements. Choose dedicated feature management when configuration history is part of the required record, and specialized observability when reconstruction requires tracing or replay. The acceptance test is concrete: can support explain one disputed decision after the serving process has disappeared?

References

Top comments (0)