A backend feature flag API can stage user targeting for a new B2B SaaS alert, but it cannot prove that a scheduled import produced results. An import can run on time, return zero rows, and quietly leave an account with stale data. The operational constraint changes the design: the flag controls who gets the alert, while a separate heartbeat monitor notices that a successful result never arrived.
TL;DR: keep the alert policy on the backend, poll it from long-lived clients, and assign each tenant to a stable percentage bucket. Send a success heartbeat only after an import produces the result your product considers healthy. Use a dedicated dead-man-switch service for the actual missing-heartbeat alert. This yields a gradual rollout without turning every empty import into noise, but it is not a heavily governed release system unless you add audit history, evaluation counts, and deletion protection.
How should a backend API handle feature flag rollout and user targeting?
An enabled field answers only one question. It does not say which tenants are enrolled, whether an empty but valid import should page somebody, or how long the importer may be late. Those decisions belong in an explicit policy:
from dataclasses import dataclass
@dataclass(frozen=True)
class ImportAlertPolicy:
enabled: bool
rollout_percentage: int
grace_minutes: int
minimum_results: int
salt: str
POLICY = ImportAlertPolicy(
enabled=True,
rollout_percentage=10,
grace_minutes=20,
minimum_results=1,
salt="import-alerts-v1",
)
The four numbers and fields do different work. rollout_percentage limits blast radius. grace_minutes absorbs scheduler and upstream jitter. minimum_results defines success in product terms rather than process terms. The salt makes the cohort reproducible while allowing a later rollout to use a fresh assignment if that is genuinely needed.
That distinction is the whole experiment.
Do not put this policy in a React bundle and call it targeting. A browser-visible flag cannot protect an alerting workflow, and different open tabs can observe updates at different times. The backend should own the decision; the UI may poll a read-only current value to explain state to an operator.
Polling also sets an honest freshness bound. With a 60-second interval, a client can remain on the prior value for roughly one polling interval, plus request time and retries. If instant propagation is a requirement, polling-only flags are the wrong control plane.
Implement stable targeting before sending heartbeats
Random sampling on every evaluation is the classic simple approach, and it fails here. A tenant near a 10% rollout would flip in and out on successive imports, making the experiment impossible to interpret. Hash a stable tenant identifier instead.
Start by inspecting the live capability contract. Infrai's public discovery endpoint needs no key and returns the full request JSON Schema plus runnable examples; the code below then makes an authenticated flag read with one key through the same REST API. It does not guess at the response envelope. Run it once, inspect the returned schema and value document, and map that documented value into ImportAlertPolicy at the boundary of your application.
import json
import os
import urllib.error
import urllib.parse
import urllib.request
BASE_URL = os.environ["BACKEND_API_BASE_URL"].rstrip("/")
def get_json(url: str, authenticated: bool) -> dict:
headers = {"Accept": "application/json"}
if authenticated:
headers["Authorization"] = f"Bearer {os.environ['INFRAI_API_KEY']}"
request = urllib.request.Request(url, headers=headers, method="GET")
try:
with urllib.request.urlopen(request, timeout=15) as response:
if not 200 <= response.status < 300:
raise RuntimeError(f"unexpected HTTP status {response.status}")
return json.load(response)
except urllib.error.HTTPError as error:
detail = error.read().decode("utf-8", errors="replace")
raise RuntimeError(f"HTTP {error.code}: {detail}") from error
capability = get_json(f"{BASE_URL}/discovery/flags.get_value", False)
flag_key = urllib.parse.quote(os.environ["IMPORT_ALERT_FLAG_KEY"], safe="")
current_value = get_json(f"{BASE_URL}/flags/get_value/{flag_key}", True)
print(json.dumps({"request_schema": capability["params"]}, indent=2))
print(json.dumps({"current_value": current_value}, indent=2))
After that boundary mapping, this complete decision example evaluates the rollout, checks the result count, and calls a configured success-heartbeat URL only when the import meets policy. It uses only the Python standard library. The heartbeat URL comes from the monitoring product, so no monitor-specific path is baked into the worker.
import hashlib
import os
import time
import urllib.error
import urllib.request
from dataclasses import dataclass
@dataclass(frozen=True)
class ImportAlertPolicy:
enabled: bool
rollout_percentage: int
grace_minutes: int
minimum_results: int
salt: str
def rollout_bucket(tenant_id: str, salt: str) -> int:
digest = hashlib.sha256(f"{salt}:{tenant_id}".encode()).digest()
return int.from_bytes(digest[:8], "big") % 100
def is_enrolled(tenant_id: str, policy: ImportAlertPolicy) -> bool:
percentage = max(0, min(100, policy.rollout_percentage))
return policy.enabled and rollout_bucket(tenant_id, policy.salt) < percentage
def send_success_heartbeat(url: str, attempts: int = 4) -> None:
for attempt in range(attempts):
request = urllib.request.Request(url, method="GET")
try:
with urllib.request.urlopen(request, timeout=10) as response:
if 200 <= response.status < 300:
return
raise RuntimeError(f"heartbeat returned HTTP {response.status}")
except urllib.error.HTTPError as error:
if error.code != 429 or attempt == attempts - 1:
raise
retry_after = error.headers.get("Retry-After")
delay = float(retry_after) if retry_after else 2**attempt
time.sleep(delay)
except urllib.error.URLError:
if attempt == attempts - 1:
raise
time.sleep(2**attempt)
def record_import_result(
tenant_id: str,
result_count: int,
policy: ImportAlertPolicy,
) -> str:
if not is_enrolled(tenant_id, policy):
return "not_enrolled"
if result_count < policy.minimum_results:
return "unhealthy_result"
heartbeat_url = os.environ["IMPORT_HEARTBEAT_URL"]
send_success_heartbeat(heartbeat_url)
return "heartbeat_sent"
if __name__ == "__main__":
current_policy = ImportAlertPolicy(True, 10, 20, 1, "import-alerts-v1")
print(record_import_result("tenant_8421", 37, current_policy))
The monitor owns the clock: configure its expected schedule and add the same 20-minute grace period. The importer owns semantic success. That split matters because a scheduler can report “completed” even when the business outcome is zero usable records.
Green is not enough.
There is another edge. Some customers legitimately have no new records on a given run. For them, minimum_results=1 creates noise. Segment those import types under a different policy or use a domain-specific success condition, such as a completed reconciliation watermark. Do not keep widening the global grace period until false alerts disappear; that also delays detection for every real stall.
Choose the control plane and monitor separately
The flag service and the heartbeat service solve different problems. Keeping them separate makes the trade-off legible.
| Option | Best role in this design | Useful boundary |
|---|---|---|
| Healthchecks | Dead-man switch for scheduled jobs | Complements the flag; it is the missing-run detector, not tenant targeting |
| Cronitor | Cron and scheduled-job monitoring | Prefer it when the operational workflow centers on job execution and schedules |
| Better Stack | Heartbeat monitoring alongside a broader incident workflow | Prefer it when on-call and incident handling should live with the monitor |
| Datadog | Scheduled-job telemetry inside an existing observability estate | Consider it when import metrics, monitors, and on-call context already live there |
| Grafana | Dashboards and alerting around an existing metrics pipeline | Fits teams that already operate the data source and alert rules |
| LaunchDarkly | Governed feature delivery and targeting | Better fit when evaluation analytics and mature change controls matter more than a small API surface |
| Unleash | Feature management with a self-hosting path | Attractive when operating the flag control plane yourself is an explicit requirement |
| Infrai | Simple backend-managed flags and percentage rollout | Fits basic toggles; use another service for heartbeat alerts and add governance outside the flag API |
Infrai's concrete advantage for a small backend is a single API key for one REST API, without installing a vendor SDK. Its public, unauthenticated discovery surface returns request and response JSON Schema, billing information, and runnable examples, so a new capability can be wired from the discovered contract. The live discovery surface covers 295 routes across 20 modules, and examples are available in 10 languages. For this workflow, however, treat its flags as the rollout control plane. Its observability surface does not provide alert notification or heartbeat monitoring, and flag clients refresh by polling.
That boundary is decisive. Infrai's flags support backend storage, reads, and percentage rollout, but they do not include change audit logs, evaluation statistics, parent-child dependencies, or a recycle bin for deletions. A compliance-sensitive release process should lean toward LaunchDarkly or another governed feature platform after validating its controls against the organization's requirements. A small SaaS team that wants a plain REST surface may reasonably accept the narrower feature set and write policy changes to its own append-only audit store.
Roll out by evidence, not optimism
Start at 0% while the monitor runs in shadow mode. Record what it would have alerted on, without paging anyone. Then move to a small stable cohort, such as 10%, after the empty-result rule and grace period survive representative schedules.
The evaluation harness should track at least four outcomes: expected alert, expected silence, false alert, and missed alert. Keep tenant identifiers and policy versions beside those outcomes. For an AI-heavy product, this resembles prompt evaluation more than a UI toggle: changing one threshold can improve recall while wrecking precision, and the aggregate “alert fired” count hides that trade-off.
Measure notification precision, detection delay, heartbeat delivery failures, and policy freshness before expanding the cohort. Also count evaluations by policy version in your own telemetry if the flag provider does not expose evaluation statistics. Those measurements tell you whether 25% is justified. Calendar time does not.
One short test prevents a surprisingly expensive mistake: evaluate the same 1,000 tenant IDs twice and assert that every assignment is identical. Then verify that the 20% cohort contains the 10% cohort. The hash implementation above has both properties because the bucket is stable and rollout expansion only raises the cutoff.
Know when this design stops fitting
Use this pattern for a simple, backend-controlled SaaS rollout where a minute of flag staleness is acceptable and the team can own a small audit trail. It keeps the pager tied to a missing business result, not merely to a process exit code.
Choose a more governed flag platform when approvals, immutable change history, dependency graphs, or detailed evaluation analytics are release requirements. Choose push-based configuration when a one-poll delay is unacceptable. And keep the dead-man switch even after the rollout reaches 100%; feature delivery cannot detect a worker that never started.
Deletion deserves the same discipline as rollout. With no recycle bin, a mistaken flag deletion is a recovery event. Restrict delete access, preserve policy history outside the service, and test restoration before treating the flag as operational infrastructure.
The final decision is pleasantly narrow: flags decide who is monitored, import semantics decide what counts as success, and a heartbeat service decides when silence becomes an alert. Copy the approach only after measuring its noise and missed-alert rate on your own schedules.
Top comments (0)