After 15+ years of building software, reviewing pull requests, debugging production systems, and working with engineering teams, I've become increasingly convinced that we're asking the wrong question about AI code review.
The obvious question is:
How accurately can AI detect problems in code?
It's an important question.
But there's another question that becomes more important once a team uses the system for months:
How well does the system understand what this particular team considers worth reviewing?
Those are very different problems.
A reviewer can be technically accurate and still be frustrating to use.
It can identify real complexity, real duplication, real security risks, and real maintainability concerns—and still lose the trust of the engineers receiving its comments.
Why?
Because software engineering isn't just a collection of rules.
It's context, conventions, trade-offs, history, architecture, and judgment.
And much of that knowledge isn't written anywhere.
It's learned through interaction.
That is why I believe the next generation of AI code review needs to learn from developers.
Static Analysis Was Never the Whole Story
We've had static analysis for decades.
Linters are incredibly useful.
They catch formatting problems, suspicious constructs, unused variables, dangerous patterns, complexity issues, and countless other classes of problems.
Static analysis is deterministic and predictable.
That's exactly why engineers like it.
You can define a rule:
if condition X is true
report finding Y
You can then configure how that finding behaves.
- Warning
- Error
- Info
- Ignore
You can add thresholds.
You can exclude directories.
You can configure allowlists and denylists.
You can override rules for specific repositories.
This works remarkably well for problems that can be expressed clearly.
But there's a fundamental limitation:
The configuration describes what we think matters. It doesn't necessarily describe what the team actually considers useful.
Those aren't always the same thing.
Configuration Has a Ceiling
Consider a team working on a large backend system.
They configure a complexity threshold:
complexity > 10 → warning
Seems reasonable.
Then the system starts flagging a particular class of functions.
The team repeatedly dismisses them.
Why?
Maybe those functions are generated around a complex business domain.
Maybe they're deliberately structured that way because splitting them would make the code harder to understand.
Maybe the threshold is appropriate for most of the repository but not this subsystem.
Maybe the team has accepted the complexity because the alternative introduces a worse architectural trade-off.
You can solve this with another configuration rule.
Add an exception.
Then another exception.
Then another.
Eventually, your configuration starts looking like this:
flag X
unless repository Y
unless directory Z
unless pattern A
unless service B
unless annotation C
At some point, we're essentially trying to encode years of engineering judgment into a configuration file.
That's not necessarily the best abstraction.
Engineering Teams Are Dynamic Systems
The bigger problem is that repositories don't stay still.
Neither do teams.
An organization might move from a monolith to services.
A team might adopt a new framework.
A legacy subsystem might become effectively frozen.
A new architectural standard might be introduced.
Performance requirements might change.
Security requirements might become stricter.
The team might deliberately relax a particular style rule because it's slowing development without providing meaningful value.
Configuration tends to represent a snapshot.
Engineering practice is a moving target.
That's why I think AI code review needs something beyond configuration.
It needs a feedback loop.
The Most Interesting Signal Might Be a Dismissal
Here's a simple example.
Suppose an AI review system repeatedly reports:
This function is too complex.
The developer dismisses the comment.
The same pattern appears in another pull request.
The developer dismisses it again.
Then again.
What should happen?
A conventional system records three dismissed findings.
End of story.
But I think we're throwing away valuable information.
Repeated dismissal could mean several things.
1. The rule is too broad
Perhaps the complexity threshold isn't appropriate for this codebase.
2. The system lacks context
The function may look complex locally but be perfectly reasonable given the surrounding architecture.
3. The team has an intentional exception
The engineers may understand the trade-off and have consciously decided to accept it.
4. The feedback is technically correct but practically irrelevant
Something can be objectively improvable without being worth interrupting a developer about.
5. The team may actually be wrong
This possibility is important too.
Learning from developers shouldn't mean blindly accepting every dismissal.
Sometimes an engineer dismisses a legitimate security issue.
Sometimes a team has normalized technical debt.
Sometimes people ignore warnings because fixing them is inconvenient.
So the answer isn't:
"Developer dismissed it, therefore never report it again."
The better question is:
"What does this interaction tell us, and how confident should we be when adapting?"
That's a much more interesting engineering problem.
A Review Comment Is an Interaction
We usually model code review as something like:
Code → Analysis → Finding
But real code review is closer to:
Code
↓
Analysis
↓
Finding
↓
Developer judgment
↓
Accept / dismiss / modify / ignore
↓
Future behavior
That final step is where a lot of today's systems stop paying attention.
But it may be the most valuable part.
Every interaction provides information.
A developer accepts a finding.
That's a signal.
A developer dismisses it.
That's a signal.
A developer repeatedly modifies recommendations in the same way.
That's a signal.
An entire team consistently ignores a category of feedback.
That's a much stronger signal.
The challenge is turning those signals into useful adaptation without creating unpredictable behavior.
The Repository Shouldn't Be Treated as an Isolated Installation
One thing that bothers me about the current mental model is the idea that every repository starts from zero.
Install the tool.
Configure it.
Run it.
Done.
But an established codebase already contains enormous amounts of engineering knowledge.
You can find it in:
- Existing implementation patterns
- Pull request discussions
- Architectural decisions
- Test strategies
- Naming conventions
- Error-handling approaches
- Accepted technical debt
- Framework-specific patterns
- Exceptions to general standards
Some of this can be explicitly documented.
Much of it isn't.
An AI system operating inside that repository should have the opportunity to understand those patterns.
Not simply analyze the current pull request in isolation.
False Positives Are More Expensive Than They Look
False positives aren't just an accuracy problem.
They're a trust problem.
Imagine a developer receives ten review comments.
Two are genuinely useful.
Eight aren't.
After a while, the developer stops carefully evaluating each comment.
They start scanning.
Eventually they start dismissing automatically.
That's when the system has failed even if its underlying detection model is technically sophisticated.
The cost isn't just the time spent reading irrelevant comments.
The larger cost is signal degradation.
Once developers stop believing that a comment is likely to matter, even good findings become harder to trust.
This is why reducing noise isn't merely a UX improvement.
It is part of the intelligence of the system.
But Adaptation Needs Guardrails
There's an obvious danger here.
If we allow AI systems to learn from developer behavior without constraints, they could learn the wrong lessons.
Imagine a team consistently dismisses security warnings because fixing them is inconvenient.
Should the system become quieter?
Probably not.
Some categories should have stronger guarantees.
Some findings should remain visible regardless of preference.
Some adaptations should require explicit confirmation.
Some changes should be explainable.
For example:
"This finding has been repeatedly dismissed for this repository. Should future findings of this type be deprioritized?"
That's very different from silently changing behavior.
The goal shouldn't be autonomous customization at any cost.
The goal should be safe adaptation.
What Would an Adaptive Review System Look Like?
I don't think this requires some magical autonomous agent.
The underlying architecture could be relatively straightforward.
Imagine maintaining a feedback model for findings.
Something like:
Finding
├── category
├── location
├── repository context
├── confidence
├── severity
├── historical interactions
└── developer/team feedback
Then review decisions become part of the system's future reasoning.
For example:
Finding A
→ accepted 27 times
→ rarely dismissed
→ high relevance
Finding B
→ dismissed 18 times
→ accepted 1 time
→ low relevance in this repository
Finding C
→ dismissed frequently by one team
→ accepted frequently by another
→ highly context dependent
Now the system isn't simply asking:
"Is this technically a problem?"
It can also ask:
"How relevant has this type of finding historically been in this environment?"
That is a fundamentally different signal.
And it can become increasingly repository-specific without requiring engineers to manually encode every preference.
The Goal Isn't Fewer Findings
This distinction matters.
Adaptive code review shouldn't simply become a system that says less.
The objective is higher-value output.
There will be situations where the correct behavior is to become more aggressive.
Suppose a team repeatedly accepts findings related to a particular security pattern.
The system now has evidence that this category matters.
That should increase confidence that similar findings are worth surfacing.
Adaptation should work in both directions.
Developer behavior
↓
Observed preference
↓
Updated relevance
↓
Future review behavior
↓
More developer feedback
↓
Updated preference
That's a feedback loop.
And feedback loops are how systems become better aligned with their environment.
The Metric We Should Care About
This leads to a question I think the industry should spend more time discussing.
We measure code-review systems by things like:
- Detection accuracy
- Precision
- Recall
- False-positive rate
- Number of findings
- Vulnerability coverage
These metrics are useful.
But they're incomplete.
I'd also want to know:
What percentage of findings does a team consider useful?
And perhaps more importantly:
Does that percentage improve over time?
Imagine two systems.
System A finds 100 issues in the first month and continues finding roughly 100 every month.
System B finds 70 issues initially, learns from developer interactions, and eventually surfaces 30 highly relevant findings.
Which one is better?
The answer depends on what those findings contain.
But for an engineering team, I would strongly prefer the system that earns the right to speak less—and gets better at knowing when to speak.
This Is Bigger Than AI Code Review
There's a broader principle here.
Good developer tools don't merely perform tasks.
They fit into human workflows.
Git became useful because developers could build mental models around it.
IDEs became useful because developers shaped them around their workflows.
Linters became useful because teams could define what mattered.
The next generation of AI tools has another opportunity:
They can learn from the workflow itself.
That's different from traditional configuration.
Configuration says:
"Here is what we told the system."
Behavioral feedback says:
"Here is what the team actually does."
The combination could be much more powerful.
Signal Isn't a Feature. It's a Relationship.
After 15+ years in software engineering, I don't think developer trust comes from a feature checklist.
It comes from repeated evidence.
You make a judgment.
The system observes it.
It adapts.
You make another judgment.
It adapts again.
Over time, the system becomes less generic and more aligned with the environment in which it operates.
That's what makes the idea of adaptive AI code review compelling to me.
The next generation won't compete only on who can detect the most bugs.
They'll compete on who can understand which bugs, patterns, and risks are actually worth surfacing to a particular engineering team.
Because detection gets you into the workflow.
Relevance keeps you there.
And relevance comes from understanding context.
That is why I believe:
Signal isn't a feature. It's a relationship.
The best AI code-review systems will be the ones that earn that relationship carefully—through feedback, context, transparency, and repeated interaction.
Not during installation.
Over time.
And ultimately, the real measure of an AI reviewer won't be how much it can say.
It will be how quickly—and how safely—it earns the team's trust.
:::
Top comments (0)