Short answer: keep a small feature flag control plane beside the notification backend, resolve flags during server-side rendering, and attach a bounded decision record to delivery telemetry so every stored byte has an owner.
For a Next.js admin page, “cheap and simple” should describe the operating model, not the absence of controls. The page can call one authenticated backend API, while the server-rendered path reads the same authoritative state before composing a notification workflow. The deciding constraint is cost attribution: a toggle that changes delivery volume or failure shape must leave enough evidence to explain the observability bill, without copying an unbounded set of flag properties into every event.
This is an architecture decision record for a media notification service. It assumes a modest set of operational flags, such as pausing a delivery channel or enabling a revised retry policy. It does not assume a particular flag vendor, telemetry vendor, or deployment platform.
Put the telemetry byte budget before the toggle
Store runtime flag state in an authenticated control plane with revisioned writes. Render the admin page from server-fetched state, and make the notification backend evaluate the same revision close to the delivery decision. Emit one compact decision identifier with the delivery outcome; keep the descriptive flag metadata in a separate, low-volume change record.
The distinction matters. A flag value answers “what did the service decide?” A revision answers “which configuration did it decide from?” The actor and reason belong to an audit record, not to a high-volume delivery event. Mixing those fields makes each notification carry administrative history, even though the history changes far less often than notifications do.
The design has four invariants:
- A write is authenticated and authorized independently of the page that issued it.
- A write includes the revision the operator saw, so concurrent edits can be rejected rather than silently overwritten.
- A delivery decision records a stable flag key, evaluated value, and configuration revision.
- Telemetry dimensions are selected from a fixed allowlist; editor email, reason text, recipient ID, and notification ID do not become metric labels.
That last rule pays the bill. Suppose a service sends 12 million notifications in a retention window and records a 180-byte decision envelope on every delivery. The raw envelope contribution is about 2.16 GB before indexing, replication, or compression. Add a unique notification identifier as an indexed dimension and the cardinality can approach the number of attempts. By contrast, channel, outcome, and a bounded flag_key produce a small, explainable set of combinations. These numbers are an illustrative capacity model, not a benchmark; substitute measured encoded event sizes and the actual retention window before making a storage decision.
Keep the failure boundaries explicit. If the admin read fails, the page should show no editable state rather than guessing. If an update races with another update, the API should return a conflict and the operator should reload the current revision. If decision telemetry can't be exported, delivery behavior should follow the service's defined policy rather than treating observability as the source of truth. This keeps control, execution, and evidence related without making them the same subsystem.
Compare storage shapes before choosing controls
Start with the question the retained data must answer: did a flag decision alter delivery failures, and which team owns the resulting telemetry? From there, assign each field to one of three shapes. Metrics hold bounded aggregates for alerting. Delivery events hold sampled diagnostic context. Flag change records hold the full administrative explanation at very low volume.
| Option | Delivery-path evidence | Cardinality profile | Cost attribution | Main trade-off |
|---|---|---|---|---|
| Bounded metric labels | Counts by channel, outcome, and selected flag key/value | Predictable if all values are allowlisted | Directly assignable to a service and flag family | Cannot explain one recipient's path |
| Sampled delivery events | Decision revision plus diagnostic fields on a fraction of attempts | Controlled by sampling and field policy | Estimated from sampled bytes and event ownership | Rare failures may be missed |
| Full event on every attempt | Complete decision context for each delivery | Scales with attempt volume and any high-cardinality fields | Easy to sum, expensive to retain | Duplicates slowly changing context at high volume |
| Change records only | Actor, reason, old value, new value, and revision | Low because writes are rare | Administrative cost is clear | Cannot correlate a particular failure without a decision marker |
The recommended combination is bounded metrics, sampled delivery events, and complete change records. Sampling is a trade, not cleanup. A 1% uniform sample reduces common-path event volume by roughly two orders of magnitude, but it can discard the only example of a rare failure. Stratified sampling is usually more useful: retain all events for a narrowly defined failure class, sample successful deliveries, and impose a hard cap so a malformed producer cannot defeat the budget. I'm not sure a single sampling rate is defensible across email, push, and webhook channels; measured per-channel failure frequency and encoded event size would resolve that choice.
Retention math should appear in the design review. Let N be attempts per day, s the retained sample fraction, b the average encoded bytes per retained event, and d the retention days. The first-order stored volume is N × s × b × d. It excludes indexes, replicas, and compression, so measure those multipliers in the chosen storage system. For an illustrative workload of 400,000 attempts per day, a 2% sample, 600-byte events, and 30-day retention produce 144 MB of raw event bodies. The formula is more durable than the number.
Privacy changes the storage plan too. A notification address or recipient identifier is tempting during incident work, but it expands cardinality and creates erasure obligations. GDPR Article 17 defines a right to erasure under specified conditions. The practical response is data minimization: keep direct recipient data out of metrics, use short-lived access-controlled diagnostic events only where justified, and maintain a deletion path that can locate the data actually retained.
How can a backend API implement a feature flag admin page with server side rendering?
The backend API should expose a compact snapshot for reads and a revision-checked command for writes. During server-side rendering, the Next.js server requests the snapshot with its server credential and renders the current value plus revision into the admin page. Browser code submits an operator action to the application's server; it doesn't receive the service credential.
The names below are pseudonymous endpoints, but the interaction is concrete. A server-side read obtains the current state:
curl --fail-with-body --silent --show-error \
--request GET \
--header "Authorization: Bearer ${ADMIN_SERVICE_TOKEN}" \
--header "Accept: application/json" \
https://control.example.test/api/flags/notification-retry
An update carries the observed revision and a short reason. The API can return 409 Conflict when that revision is stale, a precise signal that the page must fetch again instead of overwriting somebody else's decision:
curl --fail-with-body --silent --show-error \
--request POST \
--header "Authorization: Bearer ${ADMIN_SERVICE_TOKEN}" \
--header "Content-Type: application/json" \
--data '{"enabled":true,"expected_revision":17,"reason":"controlled retry-policy rollout"}' \
https://control.example.test/api/flags/notification-retry/set
Don't turn the flag response into a telemetry payload. A useful read response may contain the current revision, value, update time, and display-safe audit summary. The delivery event needs less: for example, flag_key=notification_retry, flag_value=true, and flag_revision=18. Even there, revision should usually be an event field rather than a metric label because its distinct values only grow.
The server-rendered page is an administrative view, not an authorization boundary. The mutation endpoint must still validate the caller, the allowed flag, the requested value, and the expected revision. It should also produce an append-only change record. This is deliberately plain. Clever client state management won't repair a write path that accepts stale or unauthorized changes.
Secure mutations and preserve revision evidence
The critical path has three actions: read the revision, submit a compare-and-set update, and verify the new state before showing success. The delivery worker then evaluates locally cached or directly read state according to a documented freshness limit and records the decision revision with its outcome. The admin page should not claim success merely because a button was clicked.
One failure deserves special attention. Two editors can render revision 17, then choose opposite values. Without expected_revision, the later request silently wins and the audit trail looks valid while concealing the lost decision. With revision checking, one request advances the state to 18 and the other receives 409 Conflict. That's useful friction — the second editor now has to review current state before issuing another command.
Verification can remain a plain API read:
curl --fail-with-body --silent --show-error \
--request GET \
--header "Authorization: Bearer ${ADMIN_SERVICE_TOKEN}" \
--header "Accept: application/json" \
https://control.example.test/api/flags/notification-retry
Test the behavior as state transitions, not screenshots. Cover an authorized read, an unauthorized mutation, a valid revision update, a stale revision conflict, and a delivery decision made before and after a revision change. In deployment, migrate the control-plane schema before releasing code that writes the new shape; readers should tolerate the previous shape for the duration of the rollout. Error handling should distinguish retryable transport failure from a semantic conflict, because retrying a stale write unchanged only repeats the conflict.
For diagnostic grouping, avoid making each revision a separate issue by default. Sentry documents that event grouping uses grouping algorithms and that custom fingerprints can alter how events are grouped. The general lesson applies beyond one product: group on the stable failure mechanism, preserve the flag revision as context, and split by revision only when the revision itself defines a materially different failure. Otherwise, every administrative edit fragments the evidence.
Migrate only when runtime changes justify a control plane
The rejected option for this system is one environment variable per flag with a deployment required for every change. It has attractive properties: little runtime machinery, configuration reviewed with code, and no separate write API. The catch is that a delivery operator cannot make a narrow runtime change without invoking the deployment path, and the deployment record alone doesn't provide a compact decision marker for each notification outcome.
Stick with deployment-time configuration when changes are rare, every change should receive code review, request-time variation is unnecessary, and deployment latency satisfies the operational requirement. It is also preferable when the organization cannot support a secure administrative control plane. A runtime admin page is not suitable when the team lacks an authorization model, audit ownership, or a tested stale-write policy; adding a toggle UI under those conditions creates a second production change mechanism with weaker governance.
A hosted flag service can also be reasonable when flag volume, targeting rules, approval workflows, or multi-service propagation exceed what a small internal control plane should own. Its valid cost comparison includes evaluation traffic, retained audit data, telemetry duplication, operational labor, and exit complexity. No universal winner follows from the word “cheap.” For this bounded notification case, the small control plane wins because it makes revision semantics and cost ownership explicit, not because runtime flags are inherently better.
The final decision rule is narrow: adopt runtime administration only when the response-time value of a flag exceeds the control-plane and telemetry burden it creates. Count the labels. Measure the bytes. Then retain less, on purpose.
Top comments (0)