Most AI code reviewers work the same way. They read the pull request diff, maybe a few files around it, and comment on what they see: a missing null check, an unhandled promise, a SQL string built by hand. They are good at this now, and if your team uses one, keep it.
But a whole class of bugs is invisible to that approach. The diff looks fine. The problem is somewhere the reviewer never looks.
They come in two kinds.
1. Regressions outside the diff
A function gets a new required parameter. The pull request updates the function and the two callers the author remembered. There are three more callers in other packages. They are not part of the diff, so the reviewer never reads them, the type checker doesn't help if one of them goes through an any-typed adapter, and tests pass if nobody tests that path.
The diff is correct. The change is not.
This gets worse with coding agents. An agent changing a function rarely goes looking for every caller unless you tell it to, and the pull request it opens is usually tidy and confident. A reviewer that reads only that pull request has nothing to object to.
To catch this, the reviewer needs one question answered before it reads the diff: who depends on the code that changed?
2. Decision drift
The second kind is quieter. Say your team decided, months ago and on purpose, that every LLM call goes through one shared client, so cost tracking and retries live in one place. The decision is in a merged pull request, a review thread and a line in CLAUDE.md.
A new pull request calls the provider SDK directly from a new file. It works. It has tests. It is also the start of the thing you decided not to do, and nothing in the diff says so.
A generic reviewer can't flag this, because the rule isn't a generic rule. It's yours. It lives in your history, not in the code being changed.
To catch this, the reviewer needs a second question answered: which of our decisions apply to the files that changed?
Answering "who depends on this?" is harder than it looks
The obvious tool for the first question is a code graph: index the repository, then ask for the callers of each changed symbol. We built our regression check on one, and then we measured it, because a missed caller is a missed regression.
We labelled ground truth with the TypeScript LanguageService (every real call site of a target function, attributed to the function that contains it) and compared a code graph's answers against it.
On our own TypeScript monorepo, with an off-the-shelf graph tool:
- 57.4% recall: it found 27 of 47 real callers.
- 17 of 26 functions came back with zero callers, even though every one of them had real ones.
- Some misses were same-file calls, not just cross-package ones.
- Every empty result was reported as exact, with risk LOW.
That last point is the one that matters. A tool that says "I don't know" is fine. A tool that says "nothing calls this, safe to change" when three things call it is how regressions get approved.
We ran the same kind of study on Vite with a different engine. It named the exact caller 67.9% of the time, but every one of the 185 call sites fell inside some caller it reported. The engine was naming the wrong enclosing function, not missing the call. That is a much better failure, because a reviewer reading the reported callers still finds the call.
What we took from this:
- An empty caller list is not evidence of safety. Treat it as "unknown", search the code anyway, and read likely consumers before deciding a change has no impact.
- Measure your graph against a real oracle, on a real repository. A small clean fixture will read 100% and tell you nothing. The misses only show up at scale.
- Pass the reviewer the uncertainty, not just the answer. Our impact results carry an explicit "approximate" label and a list of reasons they may be incomplete (dynamic dispatch, re-exports, reflective calls), and the reviewing model sees those.
Answering "which decisions apply?" needs a decision store
Decisions are scattered: merged pull requests, review threads where someone said "please don't do this, we use X", ADRs, READMEs, CLAUDE.md. To check a pull request against them you need them in one place, each with a scope (the paths it covers) and a link to where it came from.
We mine candidate decisions from that history automatically, but a mined decision is a guess. So they start as proposals, and someone on the team approves, edits or rejects each one. A finding that cites an approved decision can block a merge. One that cites an unconfirmed proposal is labelled as such and weighted lower.
The link back to the source matters more than we expected. "This breaks a decision" is easy to dismiss. "This breaks the decision from that pull request six months ago, here is the thread" is not.
Keeping the noise down
Both checks are easy to make noisy. A reviewer that flags every caller of every changed function is useless within a week. What worked for us:
- Two passes. A cheap model sorts candidate findings, then a stronger model has to confirm each one against the actual code before anything is posted.
- Evidence on every finding. The affected callers by name and file, or the decision and its source. No evidence, no comment.
- Feedback that sticks. A finding the team dismisses is not posted again on the next push.
Run it next to your current reviewer, not instead of it
Diff-level review and context-level review catch different bugs. We run both. The diff reviewer catches the null check. The context check catches the caller in the other package and the decision from six months ago.
We built this into Tently, an AI code reviewer for GitHub pull requests that checks every change against the code that depends on it and against your team's decisions. The same decision store is also available while the agent writes code, through an open-source MCP server (tently-team/tently-mcp), so Claude Code, Cursor or Codex can look up the rules before the pull request exists.
Whatever you use, the two questions are worth asking of any pull request, human or agent-written:
- Who depends on the code that changed?
- Which of our decisions apply to the files that changed?
If your reviewer can't answer them, someone on your team is answering them by memory.
Top comments (0)