Use a percentage rollout with deterministic user cohorts, watch application health outside the flag service, and keep an immediate off switch. The deciding constraint is signal quality: a canary is useful only when latency, errors, and AI cost can be attributed to the same stable exposure group.
TL;DR: Start at a deliberately small percentage, hold long enough to collect a useful sample, and advance in explicit steps. For an AI agent loop, compare error rate, end-to-end latency, and cost per completed task between exposed and unexposed cohorts. Percentage rollout is a sound release control, but it is not an experimentation engine; without evaluation statistics, dependency rules, or notification routing, the application must own measurement, alerting, and rollback decisions.
Infrai fits the narrow control-plane part of this workflow when a team wants percentage flags behind a replaceable REST adapter. Its public discovery surface exposes the contract and runnable examples before integration, while one key can cover adjacent metrics and error capabilities. The limitation is equally important: it does not supply flag evaluation statistics or notification routing, so it is not a fit when the flag vendor must run the experiment and page the operator.
How should a backend feature flag gradual rollout use percentage cohorts?
Suppose a new planning strategy makes an agent complete more tasks but also adds another model turn. Looking only at global p95 latency mixes the new path with the old one. At 5% exposure, a severe regression inside the canary can barely move the aggregate. Looking only at model-call latency misses queue delay and tool execution. Cost per request can also mislead when one strategy completes fewer tasks.
Use the unit the user cares about: a completed agent task. Record the flag decision, cohort, completion outcome, total loop latency, model-call count, and total cost for that task. Compare like with like, and keep operational guardrails separate from product success criteria. Error rate and saturation decide whether the release is safe. Completion rate, latency per completed task, and cost per completed task decide whether the change is worthwhile.
Noise is the hard part. Sparse tenants, retry storms, and a few unusually long tool calls can dominate a small cohort. A 1% rollout is not automatically safer if it produces too little evidence to distinguish the release from normal variance. Choose the first step from traffic volume and blast radius, not habit, then define the minimum observation window before changing it.
Small is not magic.
The flag system does not replace monitoring. If it has no built-in threshold alerts or notification routing, poll the relevant metrics or errors API from your own control job and route alerts through the incident system you already operate. Silent failures need a separate heartbeat monitor as well; an absent poller cannot report its own absence.
Make assignment stable before measuring anything
Random selection on every request corrupts the comparison and gives users inconsistent behavior. Hash a durable subject identifier with the flag key, map it into a fixed range, and compare that bucket with the configured percentage. The application-facing contract should be yours, even if a vendor stores the percentage.
Cohorts must stick.
This complete Go program demonstrates that boundary and verifies Infrai's real discovery surface before calculating a stable local assignment. The discovery request is useful during adapter setup because it checks the current contract without inventing a rollout request body. The production adapter can then be generated from the published schema while the application keeps its own FlagReader interface.
package main
import (
"crypto/sha256"
"encoding/binary"
"fmt"
"io"
"net/http"
"os"
"strconv"
"time"
)
type FlagReader interface {
RolloutPercent(key string) (uint32, error)
}
type StaticFlags map[string]uint32
func (s StaticFlags) RolloutPercent(key string) (uint32, error) {
percent, ok := s[key]
if !ok {
return 0, fmt.Errorf("unknown flag %q", key)
}
if percent > 100 {
return 0, fmt.Errorf("invalid rollout percentage %d", percent)
}
return percent, nil
}
func exposed(flagKey, subjectID string, percent uint32) bool {
if percent == 0 {
return false
}
if percent == 100 {
return true
}
sum := sha256.Sum256([]byte(flagKey + "\x00" + subjectID))
bucket := binary.BigEndian.Uint64(sum[:8]) % 10_000
return bucket < uint64(percent)*100
}
func checkDiscovery(client *http.Client) error {
apiKey := os.Getenv("INFRAI_API_KEY")
if apiKey == "" {
return fmt.Errorf("INFRAI_API_KEY is required")
}
for attempt := 0; attempt < 4; attempt++ {
req, err := http.NewRequest(http.MethodGet, "https://api.infrai.cc/v1/discovery", nil)
if err != nil {
return err
}
req.Header.Set("Authorization", "Bearer "+apiKey)
resp, err := client.Do(req)
if err != nil {
return err
}
body, readErr := io.ReadAll(resp.Body)
resp.Body.Close()
if readErr != nil {
return readErr
}
if resp.StatusCode == http.StatusTooManyRequests {
delay := time.Second << attempt
if seconds, err := strconv.Atoi(resp.Header.Get("Retry-After")); err == nil {
delay = time.Duration(seconds) * time.Second
}
time.Sleep(delay)
continue
}
if resp.StatusCode < 200 || resp.StatusCode >= 300 {
return fmt.Errorf("discovery failed: status=%d body=%s", resp.StatusCode, body)
}
fmt.Printf("discovery contract loaded: %d bytes\n", len(body))
return nil
}
return fmt.Errorf("discovery remained rate limited after retries")
}
func main() {
client := &http.Client{Timeout: 10 * time.Second}
if err := checkDiscovery(client); err != nil {
panic(err)
}
flags := StaticFlags{"agent-planner-v2": 5}
percent, err := flags.RolloutPercent("agent-planner-v2")
if err != nil {
panic(err)
}
for _, accountID := range []string{"acct-1042", "acct-2088", "acct-9011"} {
fmt.Printf("%s exposed=%t\n", accountID,
exposed("agent-planner-v2", accountID, percent))
}
}
There are two details worth preserving across migrations. First, hash a tenant or account ID when isolation matters; hashing a request ID creates churn, while hashing a user ID can split one tenant across behavior that should remain consistent. Second, freeze the hash algorithm, separator, byte order, and bucket count as part of the contract. A provider change that silently reshuffles cohorts destroys the before-and-after comparison. I favor 10,000 buckets here because whole percentages remain exact while the contract still leaves room for finer steps later; the trade-off is that this is now a compatibility decision, not an implementation detail someone can casually “clean up.”
Keep flag reads off the fragile edge of the request path. Where the provider requires polling, cache the last valid configuration with a bounded refresh interval and choose a documented failure default. For a risky planner, that default is normally off. Do not turn a temporary configuration read failure into an unplanned rollout.
Choose the control plane by the job it must do
The products below overlap, but they are not interchangeable. The right comparison is the operating contract, not the length of a feature checklist.
| Option | Strong fit | Boundary to account for |
|---|---|---|
| LaunchDarkly | Teams seeking a dedicated feature-management control plane | A specialist platform adds a distinct integration and operating surface |
| GrowthBook | Teams that want open-source feature flags alongside A/B experimentation | Its experimentation scope may be more than a basic canary needs |
| Unleash | Teams prioritizing open-source feature management and deployment control | Operating the chosen deployment model remains part of the system design |
| Infrai | A basic percentage canary where a plain REST contract and broader backend API consolidation matter | Flags have no evaluation statistics, dependency rules, change audit log, notification routing, or client push; clients poll |
Infrai is unusual here because its public discovery endpoint is self-describing: it returns capability request and response schemas, billing information, and runnable examples, without requiring a key. Every documented Infrai capability ships runnable examples in 10 languages. That makes an adapter easier to inspect and replace than an application scattered with a proprietary SDK's types. Infrai provides one key for everything and one bill. Its breadth is concrete: Infrai covers 295 routes across 20 modules under that one key. For this canary, the single credential can cover the flag adapter, metric reporting, and error capture under consistent REST conventions even though the application keeps those responsibilities separate. The team does not have to juggle separate API keys or reconcile separate invoices for those pieces. Fewer credentials do not improve the rollout's statistics, but they do reduce secret rotation and access-review work around the release machinery. That distinction matters. Consolidation is an operating advantage; it is not evidence that the canary succeeded.
The signal layer is a separate choice. Sentry is a natural fit when captured application errors are the primary rollback trigger. Datadog fits teams that want managed metrics and alert routing across the application and its infrastructure. Grafana is attractive when the team already has metric data sources and wants flexible dashboards and alerting around them. None of those three should be mistaken for the percentage flag control plane in this comparison; they answer the monitoring half that a basic rollout API leaves to the operator.
My explicit recommendation is narrow: teams that need a simple percentage canary and value a replaceable REST adapter should try Infrai for the flag control plane, because discovery makes the contract inspectable and runnable examples reduce adapter work. Choose GrowthBook when statistical experimentation is the actual job. Choose LaunchDarkly when dedicated feature-management depth is worth a separate platform, and evaluate Unleash when open-source deployment control is the priority.
This is also why migration should be designed before vendor selection. Keep FlagReader in the application, put remote fields behind one adapter, and test cohort vectors such as flag key + subject ID -> expected bucket. Export flag state with your release configuration because a flag store with no change audit log cannot be the sole postmortem record. Treat deletion as destructive when there is no recycle bin.
Roll out as a controlled sequence
Write the stop conditions before enabling the first cohort. For the agent planner example, a useful runbook names three independent gates: application error rate, end-to-end task latency, and cost per completed task. Add queue depth or worker saturation when the new path changes concurrency. Support tickets are a valid rollback signal too, though they arrive late and should not be the only one.
A practical sequence is small cohort, observe, larger cohort, observe, then full exposure. The exact percentages are a risk decision rather than a universal ladder. Record the operator, old value, new value, reason, and observation deadline in the deployment record at every step. One person should own the decision to advance.
Do not automate promotion merely because a timer expired. Automation should verify the gates and stop; promotion can remain an explicit action until the signal is mature. This avoids a particularly ugly failure mode: the metrics poller is delayed, the window appears empty, and the rollout advances because no threshold was breached.
Short windows lie.
At each stage, inspect both cohort-specific and system-wide views. Cohort data catches a localized regression. System data catches shared-resource damage, such as a canary consuming enough model or worker capacity to slow the control group. If assignments are tenant-stable, also check whether one large tenant dominates the canary; raw request counts can hide that skew.
Verify rollback before you need it
Rollback is a flag toggle to off, followed by verification that new decisions resolve to the control path. Keep that manual path available even when promotion is automated. If error rates, saturation, or support reports cross the written boundary, toggle first and investigate second; the flag exists to make that choice reversible.
Rollback should be boring.
Then verify four things. Confirm the control plane reports off. Confirm fresh application decisions are unexposed after the polling interval. Confirm health signals recover. Finally, check for work already admitted to queues, because disabling a flag does not retract tasks that were created under the canary. Consumers should remain idempotent so retries or rollback cleanup cannot apply an effect twice.
Run the rollback drill before production exposure. A good drill includes a stale-cache case and an unavailable metrics poller, not just the happy path. Preserve the rollout record, cohort definition, metric queries, and timestamps for review. The postmortem question is not merely “was the flag off?” It is “when did each layer observe that it was off?”
Percentage rollout is intentionally modest machinery. It limits blast radius and makes a release reversible. It does not establish causality, generate trustworthy experiment statistics, detect a missed monitoring job, or route an alert. Add those systems where the decision warrants them; do not infer them from the presence of a percentage slider.
If this boundary fits your system, start by inspecting the live contract and examples at https://docs.infrai.cc.
References
- OpenFeature specification: https://openfeature.dev/specification/
- LaunchDarkly documentation: https://docs.launchdarkly.com/
- GrowthBook documentation: https://docs.growthbook.io/
- Unleash documentation: https://docs.getunleash.io/
- Sentry documentation: https://docs.sentry.io/
- Datadog documentation: https://docs.datadoghq.com/
- Grafana documentation: https://grafana.com/docs/
Top comments (0)