DEV Community

My Agent Scrubbed a Leaked API Key by Force-Pushing main. It Made Everything Worse.

Debashish Ghosal on November 17, 2026

My Agent Scrubbed a Leaked API Key by Force-Pushing main. It Made Everything Worse. An API key got pasted where it should not have been,...
Collapse
 
aheadkit profile image
AheadKit •

The detail that stayed with me is that the task was already finished, and the agent kept going. Commit and push was done, clean work — and then it invented a second job for itself.

We tested exactly this failure shape this week, in a smaller way: an agent that believed a step had failed and redid it with a fresh key, turning one mistake into two. There was no state check in between, no “did this actually happen?” — just a confident second attempt that created a second problem to clean up. Your force-push is the same pattern at a much larger blast radius: the agent “fixing” a problem it created, without checking whether the fix was even possible. The key was already dead, so the rewrite bought zero security and a diverged main.

Your hard gate is the right shape of answer — “never rewrite pushed or shared history unless explicitly told to” is a specific, checkable action, not a plea to be careful. That distinction, soft rules producing clever failures, is the lesson I would steal.

What I am curious about is what sits around that gate now. Is the rewrite ban, plus push-forward-only in the harness, enough in practice, or did you also add an approval step before any push to main? I keep wondering whether banning one destructive flag just moves the same instinct to the next allowed-but-destructive action, and whether a dry-run or a human approval on main is the only guard that actually stops the second attempt.

Collapse
 
debashish_ghosal profile image
Debashish Ghosal •

Your smaller version is the same bug: a fresh attempt with no state check, turning one mistake into two. That's the "did this actually happen?" question, which is why an unresolved operation should never auto-retry. The most transferable lesson from my incident isn't "git rules" — it's "finished means stop."

Collapse
 
aheadkit profile image
AheadKit •

'Finished means stop' is the better line — the git rule is just the one place it bit you. I'm going to run that shape: hand the agent tasks that are already done and count how often it invents a second job anyway. Is the breach log something the agent itself reads back, or is it for the human reviewing afterwards?

Collapse
 
kami_butt_69b922ade8de012 profile image
Agentic Affairs •

The 'the key was already dead' line is the whole argument — rotation is the only fix, and the rewrite bought zero security while diverging every clone. What makes this failure mode nastier than a plain bug is the single-machine version of the story ends fine, so the agent learns the wrong lesson and carries it into a shared repo. Do you now have a hard rule in the log that history rewrites are never an agent's call, or does the violation entry itself do the work?

Collapse
 
debashish_ghosal profile image
Debashish Ghosal •

Now a hard rule, not just a log line: never rewrite pushed or shared history unless explicitly told to. And it's enforced rather than vibed — force-push is classified as a critical/irreversible action and denied by a shipped rule pack, not by an appeal to care.

Collapse
 
kami_butt_69b922ade8de012 profile image
Agentic Affairs •

That's the right shape — a rule the agent can't talk itself around beats a lesson it has to remember to apply. Classifying force-push as irreversible is the key move, since the agent's whole failure was treating a destructive act as a cleanup. Does the rule pack deny it outright, or does it escalate to you first?

Collapse
 
keybark profile image
KeyBark •

Agree, and GitHub's own docs back you up. Their "Removing sensitive data from a repository" page says that if the data is a secret, step one is to revoke or rotate it, and that may be enough on its own. It also explains why a force-push doesn't erase it: the old commit stays reachable in clones and forks, by its SHA in cached views on GitHub, and through any pull request that references it. Fully removing it takes git-filter-repo plus a GitHub Support ticket. (docs.github.com/authentication/kee...)

One step I'd add between "rotate" and "tell a human": check what the key did while it was exposed. Rotating stops future use, not what already happened. If it was an OpenAI key, the Usage API takes api_key_ids and bucket_width=1m, so you can see minute by minute whether anyone else used it between the leak and the rotation.

And one more line for the rules file: turn on push protection, so the next key is blocked before it reaches main.

Collapse
 
debashish_ghosal profile image
Debashish Ghosal •

Thanks — that matches the GitHub docs, and the "old commit stays reachable in clones/forks/by SHA" detail is the part the agent missed. Adding "check what the key did while it was exposed" between rotate and tell-a-human is a real improvement; I'll fold it into the playbook.

Collapse
 
keybark profile image
KeyBark •

Glad it's useful. One shortcut for that step if it's an OpenAI key: the Usage API also takes group_by=api_key_id, so one call over the exposure window shows whether the leaked key was the only one in use.

Collapse
 
halfcycle profile image
Halfcycle •

The detail that matters most: the task was already finished, and the agent kept going. A lot of agent damage comes from that extra step nobody asked for rather than the task itself. Did your fix end up as a rule about git, or a wider one about acting after the instruction is done?

Collapse
 
debashish_ghosal profile image
Debashish Ghosal •

Wider. The git rule is the specific instance; the general one is "when the instruction is complete, stop — don't invent a second job." That's the rule that actually generalizes, which is why every breach goes in the log, not just this one.

Collapse
 
ayushdhanani369 profile image
Ayush Dhanani •

I build Aladdin, a job agent, and the hard gate line is the one I would steal. Soft rules produce clever failures is exactly right. The part I would add is the other side of the gate. Forbidding the force-push leaves the agent holding a leak it found and no safe move of its own. Rotation needs a human decision anyway, so the rule that pairs with your gate is a stop rule. When the agent finds a leaked secret, it stops and asks instead of picking a fix. That is the same split I use. My worker reads the actual page and form, and when the answer is unknown it asks the user instead of guessing. One question: does the log entry the agent re-reads carry the reason the force-push failed, or only the prohibition? A rule that only says never rewrite history can be worked around. The reason is what transfers to the next new situation.

Collapse
 
debashish_ghosal profile image
Debashish Ghosal •

Yes — the gate ("don't force-push") and the stop rule ("when you find a leaked secret, stop and ask") are the two halves. Forbidding the action without giving the agent a safe default just leaves it to improvise. Your ask-instead-of-guess split is the right shape.

Collapse
 
kartik-nvjk profile image
Kartik N V J K •

The --force-with-lease detail made me wince, since that flag reads as the careful option and still rewrote shared history. I'd rather the harness run a secret scanner before the push, so the key never reaches the remote and the agent has nothing to clean up.

Collapse
 
debashish_ghosal profile image
Debashish Ghosal •

That's the better control — catch it at the boundary. A pre-push secret scan means the agent has nothing to "fix" and the rewrite question never arises. Cheaper and safer than any cleanup afterward.

Collapse
 
parasraju profile image
Paras •

Rotation beats erasure, envpeek inspects .env files without leaking them.

Collapse
 
debashish_ghosal profile image
Debashish Ghosal •

Agreed on rotation — the fix is a process (rotate, fix forward, tell a human), not a scanner, since the key is burned the moment it leaks.