Most CI quality gates work the same way: diff the PR, check what changed. Simple — until you run the same tool locally, where there's no PR event to read a base SHA from. The naive fallback is HEAD^, the previous commit. It works... until it doesn't.
If you commit a debugger statement and then commit again — even a whitespace fix — a local run diffing against HEAD^ only sees the second commit. The debugger is gone from the diff. The tool reports a clean project. It isn't one.
This is exactly the trap I hit building PR CheckMate, a CI PR gate I run across 11 languages. The fix ended up being a scope split, not a smarter diff algorithm: in CI, with a real pull_request event, diff the PR as intended, falling back to HEAD^ only when no PR is attached to the CI run. Locally, with no CI environment detected at all, don't diff — review every tracked file. --full forces that same whole-repo review anywhere, including inside CI, and composes with any command.
The lesson wasn't about diffing better. It was that "no base to diff against" and "wrong base to diff against" need different fallbacks, and conflating them is how a tool ships a false sense of safety.
Curious if others building PR/CI gates have hit the same trap — or solved it differently.
Top comments (0)