Originally published on the Featureflip blog.
Every feature flag platform can tell you which flags are dead. That part is a database query against evaluation traffic, and it has been solved for years. Almost none of them will delete the flag from your source code, and the gap between those two things is much wider than it looks.
A stale-flag report is a list. Lists do not get worked. The flag stays in the code because removing it means opening the file, reading the branch, deciding which arm survives, deleting the other one, and convincing yourself you did not just change behavior in production. That is twenty minutes of careful work for a change that ships nothing, and it loses to every other item in the sprint, every sprint, forever.
So the interesting question is whether you can leave a bot switched on to do it. Most of that answer lives in what the bot declines to touch.
The rewrite that compiles and is still wrong
The obvious bar for an automated rewrite is that the result compiles. It is nowhere near enough.
Here is a test that stubs a flag read with Mockito:
when(client.boolVariation("checkout-v2", ctx, false)).thenReturn(true);
The flag is dead and serves true, so the naive rewrite folds the read to its value:
when(true).thenReturn(true);
That compiles. Javac has no objection, the suite runs green, and an assertion that used to depend on a stubbed flag now depends on nothing at all. The stub silently stopped stubbing.
The permanent part is worse than the immediate part. The flag key is gone from the file, so no later run of the tool will ever look at this line again. Nothing will rediscover it. You get a test that passes forever while testing less than it claims.
Mocking libraries work this way because they stub by performing the call and capturing the invocation it registers. The text of the read is the payload. Fold it and you have deleted the payload while leaving the wrapper standing.
Correctness has to live outside the rewrite rules
It is tempting to fix this by writing a smarter rule. That approach runs out fast, because every fix is a hand-copied summary of some other tool's semantics, and the list is never finished.
The alternative is to check the result rather than predict it. Parse every changed file again after the rewrite, compare what came out against what went in, and throw the work away if the comparison looks wrong. A gate that runs afterward asks the real question instead of encoding an opinion about what should have happened.
Two things it catches that no rule would have:
Lost side effects. In Go an if can carry an initializer that runs regardless of the condition. Fold the condition away carelessly and if v, err := load(); err != nil loses the load() call entirely. The result compiles. A gate that counts initializers before and after refuses it.
Shadowed bindings. Deleting a flag-guarded block can change which declaration a later name resolves to. Nothing in the diff shows it.
The parser decides whether the tool works at all
An early version of ours used node --check as a syntax gate for TypeScript. It parses JavaScript, so it rejected 286 of 431 real .ts files on sight: generics, interface, JSX. tree-sitter rejected 4.
A gate with a high false-positive rate does not fail loudly. It produces zero diffs forever, which reads exactly like a clean codebase. You would run it every Monday for a year and conclude you had no flag debt.
Nobody writes a test for silence, which is why that gate stayed broken until someone counted the rejections.
The refusal list is the specification
Ask a cleanup tool what it will not touch, and read that answer before the feature list.
Ours declines mock wrappers, exported bindings, module-level Python where a fold would dedent a return out of its function, and expression trees where the value is consumed rather than branched on. A short refusal list usually means the hard cases have not been found yet.
A refusal is safe. A silent refusal is a bug, because a declined file and a clean repository read off the same line unless the tool separates them on purpose. Ours names the files it turned down and exits non-zero.
What running it looks like
name: Featureflip flag cleanup
on:
schedule:
- cron: '0 9 * * 1'
workflow_dispatch: {}
permissions:
contents: write
pull-requests: write
jobs:
cleanup:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: canopy-labs/featureflip-flag-cleanup-action@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
api-token: ${{ secrets.FEATUREFLIP_API_TOKEN }}
org: my-org
project: my-project
staleness: dead
dry-run: true
Three properties are worth copying whatever tool you use.
Start in dry run. It computes and prints every diff without contacting GitHub at all. Every refusal a real run can produce is reachable in a dry run and reported the same way, so the preview cannot approve something the real run turns down.
Cap the first run. A project with two hundred dead flags would otherwise get two hundred branches on the first morning, and if that run was mistaken, all two hundred were.
Give the scanning job a read-only token. Reading candidates and rewriting code needs nothing more. Archiving the flag afterward is a separate workflow on a different trigger with its own write-scoped secret, which keeps the permission escalation deliberate rather than incidental.
The archive step has its own refusal list, and it is the awkward one. A flag that another live flag lists as a prerequisite cannot be archived, and neither can one with a scheduled change still pointing at it. Both arrive after the removal has already merged, so the refusal is not a prompt to try something else — it is a note about work somebody still has to do.
The Action covers thirteen languages, runs in your own CI, and reads through the public API, which is on every plan including the free tier. The full post has the gate details and a couple of sections this one skips, and the docs have the input reference. Featureflip is a flat-priced feature flag platform.
Top comments (0)