A staff member opens the new directory search, gets a blank page, and files a bug. You turn the pilot off. Their next refresh works, then the broken screen comes back. With two servers behind a load balancer, both requests can be handled correctly according to the flag rules each server has in memory. The servers simply disagree about which rules are current.
AI generated this article from public sources. The rollout and staging exercise are illustrative.
On November 10, Cloudflare announced an official Rust SDK for Flagship that evaluates flags through OpenFeature. The SDK calls the service over HTTP by default. With local evaluation enabled, it downloads definitions and evaluates them inside the process. When a background refresh fails, the provider reports a stale status and keeps serving the last downloaded definitions. A request can therefore get a successful “on” evaluation after an operator has turned the flag off. Cloudflare’s announcement · Rust SDK documentation
Give the pilot an expiration rule
The Garden’s public case study describes a staged replacement of Planning Center, with staff workflows and a directory that supports pagination, filters, and server-side counts. The migration remains ongoing. A read-only directory search pilot makes a good example of a workflow that needs a way back to an existing path. The Rust deployment below is a proposed example inspired by that staged approach. The Garden case study
Imagine that a new directory endpoint can send an eligible staff member to either of two search implementations. Keep the evaluation on the server so that a saved browser tab can’t choose the backend by itself. Use the same synthetic account and targeting context on both servers during the test. That removes differences in targeting from the explanation if the responses diverge.
Pick the fallback before opening the pilot. For this example, route to the existing search when the provider can’t initialize, when flag evaluation errors, or when the provider reports stale definitions. Keep existing access checks on both paths. If you can measure the last successful refresh reliably, you can also cap the age of the rules your app will use to admit a request to the pilot. A refresh interval schedules an attempt. When connectivity fails, it can’t guarantee that the new rules have arrived. Cloudflare’s local evaluation and refresh behavior
The tradeoff shows up during a vendor or network outage. A team may be happy to leave a cosmetic pilot running from the last known configuration. If the pilot can return a blank directory when staff need it, sending them back to the existing search may be worth an interruption to the experiment. Choose how long you can live with the wrong variant before choosing how much stale configuration to tolerate.
Run the rollback with one server cut off
Start two staging instances and verify that both send the same test account to the pilot. Cut only the second instance’s connection to the flag service, keeping its application traffic working. Turn the flag off. Probe each instance directly so a load balancer can’t hide a lagging instance. Record which search backend actually served each request, when it switched, and whether the user still got a usable result. Wait through a failed refresh and repeat. The healthy instance should pick up the change. The isolated one should fall back according to the policy you chose.
Run the exercise once more with the second instance starting cold while the service is unreachable. The Rust SDK’s initialization returns an error if the first definitions download fails. The cold case tells you whether your application can start the existing search independently or whether flag initialization can take down the whole endpoint. When connectivity comes back, check that both instances honor the current flag before you re-enable the pilot. Cloudflare’s Rust initialization behavior
Keep the backend selection visible in application metrics. Cloudflare says local evaluations don’t appear in Flagship analytics. For a rollout, a small counter broken down by instance, selected route, and fallback reason lets the person on call answer the question that matters after pressing Off. Which server still sent a request to the pilot, and how long did it keep doing that? Cloudflare’s local evaluation documentation
Top comments (0)