Two projects, thirteen serious bugs, and one pattern that kept showing up everywhere.
Most engineers find out their code has problems when a user reports one. I found out earlier — by going back through two of my own projects and auditing them properly, on purpose, before anyone else had to.
The projects were different in almost every way. GitHub Autopilot is a Flask app that automates code review and PR management through GitHub webhooks. TokenMizer is a proxy that intercepts LLM API calls to give them persistent memory. One handles webhook traffic; the other handles model requests. Different languages of failure, you'd think.
They weren't. The same failure pattern showed up in both, wearing different clothes each time. Here's what I found, and what it taught me about the difference between code that works and code that's actually safe to ship.
The Pattern: Failure That Doesn't Announce Itself
In GitHub Autopilot, I found 27 places where the code caught an exception and did nothing with it — no log, no alert, just silent continuation. In TokenMizer, the same shape of bug appeared across 12 files as silent crash patterns, plus an async race condition where concurrent requests could read or write memory state out of order without anyone noticing until the data was already wrong.
Neither of these is a bug you catch by staring at the code. They're bugs you catch by asking a different question than "does this work?" The right question is: when this fails — and eventually it will — does the failure tell anyone?
Code that fails loudly is annoying but honest. Code that fails silently is comfortable to write and dangerous to run, because it lets small problems compound into large ones with no trail to follow back. Every fix I made across both projects, at its core, was about converting a silent failure into a visible one.
Three Specific Failures, One Underlying Habit
GitHub Autopilot's auth bypass: the MCP authentication check caught its own exceptions and let the request through — failing open instead of closed. A slow auth service became an accidental backdoor.
GitHub Autopilot's rate limiter: it tracked request counts per IP but never cleaned up stale entries, so the defense meant to stop abuse became a memory leak that could crash the app under sustained traffic.
TokenMizer's installer bug: an edge case in the MCP installer could overwrite a user's existing memory graph on install, instead of merging with it — a data-destruction bug hiding inside what looked like routine setup code.
Three different subsystems, three different consequences — a security hole, an outage vector, a data-loss bug. But look at the shared habit underneath: in all three cases, the code assumed the happy path would hold, and didn't plan for what happens when it doesn't. The auth check assumed the auth service stays up. The rate limiter assumed traffic stays bounded. The installer assumed there's nothing to lose.
Good defensive code doesn't assume the happy path. It asks "what's the safe default when I don't know what's going on?" — and for anything touching security or user data, the safe default is almost always: deny, don't proceed, don't overwrite.
Testing Isn't the Finish Line — It's How You Prove the Fix
Finding bugs is one skill. Proving they're actually fixed is a different one, and it's the part that's easy to skip under deadline pressure. For GitHub Autopilot, that meant writing targeted tests that reproduce each original failure — a forged request header, an auth-service timeout, sustained traffic from many IPs — and raising overall coverage from 62% to 76% along the way, with all 654 tests passing across an 834-test suite. For TokenMizer, it meant verifying all nine issues individually rather than assuming a broad refactor had swept them up.
The number that matters isn't the coverage percentage. It's whether each specific failure mode you found has a test that would catch it coming back. Coverage without targeted regression tests is a vanity metric; targeted tests without coverage tracking mean you don't know what you haven't checked. You need both.
Why I Wrote This Down
I could have fixed these bugs quietly and moved on — nobody was demanding an audit, no user had filed a report. I did it because the fixing wasn't actually the valuable part. The valuable part was noticing the pattern across two unrelated codebases, and writing it down so it's a reusable lesson instead of a one-off cleanup.
That's the habit I'd recommend to any engineer, and it's also, honestly, the reason I enjoy technical writing as much as building: a bug fix helps one project. A clearly explained pattern helps every project you touch after it. If there's one thing worth taking from two audits and thirteen bugs, it's this — the question that finds real problems isn't "does it work in the demo?" It's "what happens when the thing I'm depending on doesn't behave?" Ask that question early, and you fix bugs before they have users attached to them.
GitHub Autopilot and TokenMizer are both open source: github-autopilot · tokenmizer.
I build developer tools and write about what breaks while building them, at TechNova World. If you want documentation that comes from someone who's actually shipped and audited production code, let's talk.
Top comments (3)
This is the part I trust most in audits. The bugs that matter usually do not crash; they silently choose the wrong boundary. I like treating webhook payloads and model requests as the same class of problem too. Both need a small gate that records what it accepted and what it rejected, not just a happy-path test.
Thank you, Reid. I completely agree. The most expensive bugs are often the ones that quietly pass validation while violating assumptions. Treating webhook payloads and LLM requests with the same validation-first mindset has saved me from several subtle issues. Logging both accepted and rejected inputs has become a key part of my auditing process. Really appreciate your insight.
This pattern screams for AI-powered prevention. If TokenMizer remembers project context, it could flag silent failures before they ship. The tricky part: distinguishing between intentional exception-swallowing and actual bugs. Maybe codify your "13 bugs" as custom lint rules that Copilot enforces?