Igor Pavlenko

Posted on Aug 30 Originally published at Medium

AI Code Review as a Quality Gate, Not a Chat: Pre-commit, Pre-push, PR

#programming #codereview #devops #ai

Today, in most teams, the path code takes to production looks roughly like this:

Linters and tests became gates a long time ago. Nobody argues with them: they stand on the road and refuse to let through what they don't like. AI review is being wired in everywhere now - into GitHub, into GitLab, as standalone bots. But in the role I keep running into, it stands in exactly one place: one more commenter on the pull request. I want to talk about a different role - one more gate, standing next to the linter and the tests rather than next to the reviewer. And about the properties a probabilistic check must have before you can let it into the pipeline alongside deterministic ones.

Here is the whole thing at once; the rest of the article walks through each step:

Conflict of interest, stated up front: I'll be showing all of this on my own tool. ReviewGate is a reviewgate binary for your terminal, your git hooks and your agent, plus a bot for merge and pull requests. It is free and closed source: there are no sources to read, it lives on donations, there is no account and no sign-up, and you pay for the model with your own key. "Self-hosted" here does not mean "open source" - better to learn that now than after you install it. (I plan to open it later. Not today.)

The pipeline today, and the hole in it

Take a small Angular project: an internal request panel that support managers keep open all day without closing the tab. The task of the week is search over requests - a search box, a status filter, a new service. Four files, about 140 lines. Lint is green, the build is green, search works in the browser. A client would sign it off.

The project with its full history lives on GitHub: ReviewGate/service-desk. The "before review" state is the tag search-unreviewed, and every snippet below comes from there.

Here is the method that fires on every keystroke in the search box (and on every filter change), verbatim from request-list.component.ts:

private runSearch(): void {
  if (!this.query() && this.statusFilter() === 'all') {
    this.found.set(null);

    return;
  }

  this.search.search(this.query(), this.statusFilter()).subscribe({
    next: (items) => {
      const now = new Date();

      this.found.set(items);
      this.foundAt.set(`${now.getHours()}:${now.getMinutes()}`);
    },
    error: () => this.found.set([]),
  });
}

What catches this today? Tests - if there were any - would pass: search finds things. The linter says nothing: there is nothing wrong with the line subscribe({…}), it is a legal call, and any Angular project has hundreds of them. What is wrong here is something else, and you can only see it by reading the whole file and the template next to it:

higher up, in the constructor of the same file, it is done properly - .pipe(takeUntilDestroyed()). So the author knows how it should be done. This is a slip, not ignorance;
the method is called on every keystroke - there isn't one subscription, there are as many as the manager typed letters, and the tab stays alive for days;
the error branch quietly turns a server failure into an empty result - the manager cannot tell a broken search from "nothing found".

This is how those two findings look in the report, verbatim from one run (I only trimmed the layout):

🔴 critical · request-list.component.ts:78 · rule team:no-leaking-subscriptions
subscribe() in runSearch() is not tied to the component lifecycle: if the manager leaves the screen before the response arrives, the callback still runs and writes into signals of a destroyed component. Team rule no-leaking-subscriptions requires takeUntilDestroyed() (or async pipe/toSignal) even for finite streams.
Judge: confirmed.

🟠 major · request-list.component.ts:85 · rule team:three-states-on-load
When the search request fails, error() sets found([]), which the template renders as 'No requests yet.' / 'Found: 0'. The manager cannot tell an error from an empty result, and there is no 'searching…' state.
Judge: confirmed.

Five severity levels, one scale across the whole tool - in findings, in gate thresholds and in your team's own rules:

	level in the config	what it means
⛔	`blocker`	unconditional block: a secret or credentials in the diff, irreversible data loss
🔴	`critical`	security, data loss, a crash
🟠	`major`	a probable bug, an unhandled error
🔵	`minor`	a small thing, a maintainability risk
⚪	`info`	an observation, no action needed

The gate threshold is written in those same words: --fail-on major means "block on major and above".

Note the team: prefix in front of the rule id. That is not "the model knows Angular". That is a rule of my team, written down in a file in the repository - and the finding shows which one. In the terminal and in --json it is the team: prefix; in the bot's comments on a pull request the same ownership shows up as a 📐 badge next to the id. You'll see it in the chapter about pull requests.

"Linters don't hallucinate"

True - and they don't see what AI review sees. And AI review does see it, and can be wrong about it. Everything else rests on that symmetry, so let me spell it out on the same diff. Six team-rule violations are planted in it, and the engine brought one more finding of its own, without a team: prefix. Here is which of those seven a linter catches and which it doesn't:

finding	caught by a linter?	why
`three-states-on-load` - an error rendered as an empty list	no	this is behavior, not syntax: you have to understand that the error branch renders "empty"
`debounce-user-input` - a request per keystroke	no	you have to connect the input event in the template → the handler → the HTTP call; an AST rule doesn't stitch three files together
an unencoded `query` in the request URL (the one without `team:`)	no	the semantics of the data, not the shape of a string
`format-through-pipes` - a time glued from strings, showing `14:5` at 14:05	no	a linter doesn't know this is a date rather than a string
`no-leaking-subscriptions` - that very `subscribe()`	partly	the "subscribe without takeUntil" pattern is caught by `eslint-plugin-rxjs-angular`; it won't see that the method fires on every keystroke (the part that makes the finding critical) or that the constructor above does it correctly
`http-only-in-api-service` - a new service injecting `HttpClient` around `ApiService`	yes	`no-restricted-imports`, and let the linter do it
`signals-in-new-code` - a new `BehaviorSubject` instead of a signal	yes	strict lint-staged on new files

Two out of seven are the linter's job, and the linter should do them: it is free, instant and deterministic. Which brings us to the main point of this whole article:

AI review belongs next to deterministic checks, not instead of them, and it belongs after them: on every step, first what is free and deterministic, then what costs money and can be wrong.

The contract for a probabilistic gate

If a check can be wrong, you cannot put it into the pipeline on the same terms as a linter. It needs different terms, and these are requirements, not caveats: drop any one of them and your "AI gate" turns into either a wall or noise, and the team will have every reason to switch it off. Here they are.

1. Rules come from the repository, not from "best practices" in general. A finding carries a rule id, and the id tells you who is complaining: your team (team:) or the engine - the stack preset and the shared core. You argue with those two differently.

2. It talks to the pipeline in exit codes, not prose.

0 - the threshold was not exceeded (and with the gate off, always);
2 - the threshold was exceeded;
1 - the run itself failed.

Only 2 blocks.

3. It knows how to stay silent. A single model call is a lottery: the model will confidently say something, and you will be the one reading it. So it isn't a call, it is a ladder:

a generator looks for problems;
a judge (up to two of them) checks every finding with refute as the default stance - not "rate this" but "prove it isn't true" - and looks not at the diff but at whole files and their neighbors by import;
if the judges disagree, an arbiter steps in, and only on the disputed ones; with no arbiter configured, a tie is resolved in favor of silence.

Yes, some findings die along the way. That is a deliberate price: what got through did not get through by accident.

4. It doesn't block out of the box, and turning blocking on shouldn't be day one. The gate is off in the default policy (severity_gate: off); strictness is something the team turns on explicitly - with a threshold in the config or a flag in a hook. But first comes calibration on your own pull requests, with no power to block:

watch, over a stretch of real work, what the review finds and how much of it you consider noise;
fix your rules and your list of deliberate trade-offs (more on that below);
only then move severity_gate to a blocking value.

A gate switched on before calibration gets switched off on the first false block - and never comes back.

5. When it breaks, it lets you through - and says so. No network, a revoked key, a model outage: none of that is a blocked push.

a broken reviewer must not block a developer in their own branch, hence exit code 1, "allowed unchecked";
but staying quiet about the skip is not allowed either - a line on stderr is mandatory.

6. You can see what ran and what it cost. Under the report there is a "🔬 Run diagnostics" block, switched on with one line in the config (diagnostics: true):

which model ran in which role, and for how many seconds;
how many candidates the judge dropped, and why;
the cost of the run right next to it (cost.show).

This is not debugging, it is trust: a gate you cannot inspect is not a gate.

7. The code never reaches the author of the tool - only where you sent it yourself:

to your model, with your key;
in a closed network, to a model on your own hardware.

Diffs and code are neither logged nor stored.

Here is what that looks like in practice - the 🔬 block under two runs of the very same diff, one fast and one full (in the call list the judge is labeled "validator", that's the same thing):

**🔬 Run diagnostics**

**Model context**: diff: 4 files · full files: 10 (12K chars) · environment: ✓ · from team standards (8 rules): 6 of 7 findings

**Calls**:

- generator `deepseek-v4-flash` — 10.8K→19.6K tokens · 173 s · findings: 7

_No judging took place: fast mode — a single generator, no judge._

**🔬 Run diagnostics**

**Model context**: diff: 4 files · full files: 10 (12K chars) · environment: ✓ · from team standards (8 rules): 6 of 8 findings

**Calls**:

- generator `deepseek-v4-flash` — 10.8K→12.6K tokens (cache: 10.8K) · 105 s · findings: 8
- validator `deepseek-v4-pro` — 10.7K→5.6K tokens · 114 s · dropped: 0 · downgraded: 0

**The judge** confirmed every finding (dropped 0, downgraded 0).

Two honest observations from those blocks.

The spread is real. Same diff, same generator - and it brought 7 findings in one run and 8 in the other. A table of "what's planted in the diff" is a reference point, not a promise.
The judge takes away more than findings. It can also strip the suggested fix off a finding it otherwise confirms - when the fix wouldn't compile as written, for instance because of a missing import. The "apply" button in a pull request only shows up on fixes the judge confirmed as mechanical.

I have no public accuracy metric to offer. Any "N % false positives" I printed here would be a number from one diff on one day. What I can show instead is the machinery that produces those percentages, and it is in the 🔬 block above every run.

Rules: from notes to config

Rules are the things you already agreed on and forget by month three. They live wherever they landed: in Confluence, in a pinned chat message, in an ADR, in the team lead's head; at best, in a .md next to the code. Mine are plain notes in docs/conventions.md. Wherever they come from, the move into a config looks like this:

reviewgate rules      # .reviewgate/config.yml not found at HEAD — default config
reviewgate init       # creates BOTH configs: the repo skeleton and your personal one

init detects the stack from the manifests (preset: angular is already in the file) and never touches what exists - run it twice and it says already exists — left untouched. The rules go into the repo skeleton. Here is the file from the sandbox, three rules out of eight, everything else as it is (the whole thing: .reviewgate/config.yml):

# Project rules for automated review — the machine-readable version of docs/conventions.md.
# Change a rule there — change it here too, in one commit: otherwise the docs and the policy drift apart.
version: 1
language: en
preset: angular

# The gate is off: I'm the only one here and I decide myself what blocks a push and what doesn't.
# For a particular run I set the threshold with the --fail-on flag when I need it (the pre-push hook).
severity_gate: off

# There are no tests in this project and none are planned — don't flag their absence as a problem.
tests: optional

llm:
  generators:
    - model: deepseek-v4-flash # hunts: fast and cheap
  judges:
    - model: deepseek-v4-pro   # judges on full files: stronger and more thoughtful

  full_file_context: true      # the generator gets whole files for findings, not just the diff
  environment_context: true    # stack versions into the prompt: Angular 21, zoneless, new control flow

# A collapsed block with timings and the judge's decisions. No code in it, only metadata.
diagnostics: true

# Unambiguous mechanical fixes — as a native suggestion block.
committable_suggestions: true

cost:
  show: true
  currency: "$"
  models:
    deepseek-v4-flash: { input: 0.44, output: 1.32 }
    deepseek-v4-pro:   { input: 1.32, output: 3.96 }

# Rules = only what ESLint can't see. The id ends up in the finding's ruleId.
rules:
  - id: no-leaking-subscriptions
    description: "Every subscription in a component dies together with it. In order of preference:
      async pipe in the template, toSignal, takeUntilDestroyed(). A bare subscribe() with no lifecycle
      management is a leak, even if the stream looks finite."
    severity: critical

  - id: three-states-on-load
    description: "Data loading handles three states: loading, error, empty result. A missing error
      branch is a defect, not an unfinished bit: the user stares at an eternal “Loading…”."
    severity: major

  - id: format-through-pipes
    description: "Dates and phone numbers are formatted with pipes (DatePipe, sdPhone), not by gluing
      strings together in a component."
    severity: minor
  # … five more rules in the file

# Deliberate trade-offs — don't flag them, the whole class of false positives goes quiet.
dont_flag:
  - "Localization: the panel is internal, UI strings sit right in the templates — that's a decision, i18n isn't planned"
  - "No unit tests — the project deliberately has none"
  - "Inline templates and styles in small presentational components (status-badge) — that's on purpose"
  - "BehaviorSubject in core/api.service.ts — legacy code, the move to signals is happening
     gradually. The exception is about THIS file: in new code BehaviorSubject is still a violation"

ignore:
  - "**/*.spec.ts"
  - "src/assets/**"
  - "**/*.svg"
  - "**/*.png"

review_prompt:
  mode: extend
  text: |
    The panel is internal, managers keep it open all working day without closing the tab.
    So on top of that, look at:
    - Long-lived subscriptions, timers and intervals: the tab stays alive for days.
    - How the screen behaves when the API returns an error or an empty list.
    - Actions that change data: what the manager sees if the request didn't go through.
    Leave pure styling and anything the linter catches alone.

Three things make this file worth having.

Only what the linter cannot see goes into the rules. The comment above rules: is not decoration: the rest is the linter's job, and the linter should do it.
dont_flag holds deliberate trade-offs. Missing tests and inline templates will not be commented on. That silences an entire class of false positives, not a single finding.
The config is code. It lives in the repository, gets committed, travels into the review with the branch, and changes in the same pull request as the agreement it describes.

You can check it without a model and without spending anything:

$ reviewgate rules
Config loaded: preset=angular, rules=8, gate=off
Stack preset: angular

Team rules:
- team:http-only-in-api-service (critical): Network calls live only in core/api.service.ts. …
- team:no-leaking-subscriptions (critical): Every subscription in a component dies together with it. …
- team:authorization-is-server-side (critical): A button or a menu item hidden by role is a UI convenience, not protection. …
- team:debounce-user-input (major): …
- team:three-states-on-load (major): …
- team:signals-in-new-code (major): …
- team:format-through-pipes (minor): …
- team:routes-from-constants (minor): …

Review guideline:
The panel is internal, managers keep it open all working day without closing the tab. …

That is exactly what the model will see: eight rules with ids and levels, the stack, and the text of review_prompt.

This list has a second use, as a lookup table for "where did this finding come from": if a rule is not on it, the finding did not come from your team but from the stack preset or the core, and you argue with it differently. Sonar and your linter are neighbors in this picture, not competitors: they gate on rules and metrics, this gates on agreements written in plain language, by meaning.

Gate 1: inside the AI agent, before the commit

Code isn't only typed by hand in an editor anymore - more and more of it is written by an AI agent (Claude Code, Cursor, Codex and their relatives). By the time a pull request is open it is too late: you are tired and you want to merge. So the first step of the ladder sits inside the agent's loop: the same rules before it starts writing, and the same review before it says "done".

It is set up with one line you paste into the agent's chat:

run reviewgate help agent-setup and do what it says

The instructions are written for the agent, not for a human, and that is deliberate: there are many clients out there, and the agent knows the shape of its own environment better than I can guess. It will check whether things are already configured (otherwise it writes a second configuration on top of a working one, and the final check passes against the old one), and it will put the MCP server where its client expects it - for Claude Code, .mcp.json in the repository root, at project level, so it travels with the repo:

{
  "mcpServers": {
    "reviewgate": { "command": "reviewgate", "args": ["mcp"] }
  }
}

…and it will add two lines to the project instruction file (CLAUDE.md, AGENTS.md, .cursorrules):

Project standards come from mcp__reviewgate__get_team_rules — call it BEFORE writing code.
Before finishing a task and before git push, check the changes with the mcp__reviewgate__review_changes tool.

Without those lines many clients load tool descriptions lazily, and the agent never finds out the tools exist. There are two of them: get_team_rules before the code, review_changes before "done", with the arguments scope: uncommitted | staged | {base, head} and mode: full | fast. Any other value is a tool error, not a silent default.

Two things I want to be straight about.

A human grants the permission to call tools, through their client's dialog. The agent does not grant it to itself, even knowing the file format. That is a barrier, not an inconvenience.
No tool call, no review. An agent without access to the tool will not say "I can't" - it will simply read the code itself and write its own remarks. In form that is a review; in substance it isn't: no team rules, no judge, no exit code behind that text. The only sign that it worked is that the call happened and the rules came back. Checking takes one phrase, same as setting it up:

  call get_team_rules and show me what came back

Besides MCP tools there is one more place to put a gate inside an agent: the client's hooks - commands the client runs itself on certain events. ReviewGate can be such a command (reviewgate review --hook-stdin), and two events are useful here:

before git push - the agent is about to send code out, the hook runs a review and refuses the push on blocking findings (in Claude Code this is the PreToolUse event for Bash);
before "done" - the agent considers the task finished, and the hook runs a review before it gets to announce that (the Stop event).

For Claude Code that is one entry in the hook settings:

{
  "hooks": {
    "PreToolUse": [
      { "matcher": "Bash", "hooks": [{ "type": "command", "command": "reviewgate review --hook-stdin" }] }
    ]
  }
}

A failure of the review itself (no network, a revoked key) is not turned into a block by the hook - it warns and lets through, like every other gate here.

Here is a real session, from the sandbox of this article:

Two things in that transcript are worth pausing on.

The team rule authorization-is-server-side made the agent write a server-side check (a 403 for non-admins in the dev stub) instead of settling for a hidden button. The rule did that, not the model's good taste - and the rule is four lines of YAML in the repository.
The judge downgraded the generator's own finding from major to minor, with its reason attached: "Missing in-flight guard on data-changing delete allows double-submit; real UX concern but severity major is too high for a non-bug guard gap." Then the agent fixed it and ran the review again - and on the second run the judge dropped another candidate as "no actual defect". The ladder works in both directions: it finds, and it takes back.

And a live epilogue to this step, which happened while I was preparing the sandbox for this article. The "Call the customer" button in it was written by an agent: by the rules, through get_team_rules, with review_changes before "done". I committed its work as it was - 18 lines in three files. Pre-commit (fast mode, threshold blocker) left one minor: telHref() re-implements phone normalization outside the existing sdPhone pipe. The commit went through. Then pre-push (full run, threshold major) put the same finding in front of a judge, and the judge took it away:

❌ format-through-pipes — "telHref builds a tel: URI (digits only), not display formatting; sdPhone's human format wouldn't fit, so false binding to format-through-pipes."

Note what happened here: the fast mode saw the same thing the full run saw, and the full run was not just stricter, it was more accurate. The push went through with a clean gate. Both the finding and the verdict are in the feat/call-button branch of the sandbox, as they happened.

Gates 2 and 3: pre-commit and pre-push

Here are the recipes you can copy. Strictness grows by steps - the further from the keyboard, the stricter:

On commit, fast mode with the threshold blocker. --fast is one model, no judge: twice as fast and three times cheaper. A commit is my draft: I want to see the findings immediately, but I don't block my own draft.
On push, a full run with a judge and the threshold major. This is the boundary where code leaves for the remote repository, and here I am willing to wait for a verdict.

The hooks go through husky - .husky/pre-commit:

npm run lint || exit 1

# The review is controlled without editing the hook — local .git/config, never committed:
#   turn it off for yourself: git config reviewgate.hook false · turn it back on: git config --unset reviewgate.hook
#   skip the whole hook once: --no-verify
if [ "$(git config --get reviewgate.hook)" = "false" ]; then exit 0; fi
command -v reviewgate >/dev/null 2>&1 || { echo "reviewgate: not installed — review skipped" >&2; exit 0; }

# A quick review of what I'm committing (--staged): I see the findings right at commit time.
# Blocks on blocker only; the full judge pass happens on push (.husky/pre-push).
code=0
reviewgate review --staged --fast --fail-on blocker || code=$?
case $code in
  0) ;;
  2) echo "reviewgate: blocker findings — commit stopped" >&2; exit 1 ;;
  *) echo "reviewgate: the review did not run (failure) — commit allowed unchecked" >&2 ;;
esac

…and .husky/pre-push:

# .husky/pre-push — a full review of what's leaving: from origin/main to the tip.
# Blocks ONLY on code 2 (threshold not met). Code 1 means the review itself failed
# (network, key): warn and let it through — a broken reviewer must not lock up the work.

if [ "$(git config --get reviewgate.hook)" = "false" ]; then exit 0; fi
command -v reviewgate >/dev/null 2>&1 || { echo "reviewgate: not installed — review skipped" >&2; exit 0; }

code=0
reviewgate review --refs origin/main --fail-on major || code=$?
case $code in
  0) ;;
  2) echo "reviewgate: blocking findings — push stopped" >&2; exit 1 ;;
  *) echo "reviewgate: the review did not run (failure) — push allowed unchecked" >&2 ;;
esac

The order inside the hook is the principle from the first part: lint first, paid review after. If lint fails, the model is never called. And the || exit 1 after lint is not redundant: under husky the hook already runs with -e, but in a plain git hook without it a failing lint blocks nothing - the review runs anyway and the commit goes through.

Three sentences without which this recipe is dangerous. Each of them I learned on myself.

The gate defaults to off. With no explicit --fail-on, the exit code is decided by the team gate from the config, and that one is off by default: review returns 0 whatever it finds, and such a hook never blocks. If I didn't say this, you would set up a dead hook and think you were protected. That's why both hooks pass the threshold explicitly.
Exit code 1 lets you through, and that is not an embarrassment. The provider drops the stream, the key gets revoked, the network blinks - "push allowed unchecked" goes to stderr and the push happens. The fail-open is deliberate; the silence is not part of it.
A home model scheme replaces the team one entirely. The role scheme arrives from the team config, but you can override it in your personal ~/.config/reviewgate/config.yml - and a declared home scheme replaces the team one rather than blending into it. Declare only a provider and a key at home and the roles come from the team, and doctor will say so: "from the team policy". To run exactly what the bot will run, use reviewgate review --team-llm. Declaring judges without generators at home refuses to start rather than quietly filling in the gap.

The switch at the top of both hooks is how you control the review without editing code. The reviewgate.hook key lives in the local .git/config of one clone: it is not versioned and never reaches the repository, so every developer decides for themselves while the hook in the repo stays the same for everyone. Review is on by default; git config reviewgate.hook false turns it off for you, git config --unset reviewgate.hook brings it back, and --no-verify skips the hook once. If your team wants the hook to be an opt-in instead, flip the comparison and it stays quiet for everyone who hasn't switched it on:

[ "$(git config --get reviewgate.hook)" = "true" ] || exit 0   # turn it on for yourself: git config reviewgate.hook true

And now about money, because you are going to ask: why pay for a review twice, locally and then again on the pull request? Because these are two different steps with two different owners.

Before the commit, the code is the developer's own business. If it matters to them to show good code before it lands in git history, they have a way to do that. If it doesn't, they are free not to use it.
On the pull request, the team pays. That is a normal appetite for product quality, not duplication.

Your own key, the company's key or a local model - on the developer's step, the one who chooses the step is the one who pays.

Here is gate 2 on the task of the week - a commit through pre-commit, a live run, output shortened with ellipses, the wording of the findings untouched:

$ git commit -m "search across requests: search box and status filter"
… (npm run lint — green) …
The MR adds a search/status toolbar to the request list and mostly follows modern Angular conventions
(signals, OnPush, control flow). The main risks are in the search pipeline: HTTP calls are placed outside
core/api.service.ts, every keystroke fires an unmanaged subscription without debounce/cancellation,
outdated responses can overwrite newer ones, and search errors are silently shown as an empty list.

🔴 `request-search.service.ts:25` — Network access is outside the single allowed place. This service injects
   HttpClient and hardcodes `/api/requests/search`, while the team rule says all HTTP calls live in
   core/api.service.ts … _(team:http-only-in-api-service)_
🔴 `request-list.component.ts:78` — runSearch() creates a new bare subscription on every keystroke without
   takeUntilDestroyed or any lifecycle management. The previous HTTP request is not cancelled, so an
   outdated response can overwrite a newer one … _(team:no-leaking-subscriptions)_
🟠 … four more major: debounce, three states on load, a dead BehaviorSubject in the new service,
   an unencoded query in the URL …
🔵 `request-list.component.ts:83` — foundAt is formatted by concatenating raw getHours()/getMinutes(),
   so times like 09:05 render as '9:5' … _(team:format-through-pipes)_

✅ Gate (blocker) passed. Findings: 7 (⛔ 0 · 🔴 2 · 🟠 4 · 🔵 1 · ⚪ 0).
⚙️ Run cost: ≈ 0.03 $

**🔬 Run diagnostics**
**Model context**: diff: 4 files · full files: 10 (12K chars) · environment: ✓ · from team standards (8 rules): 6 of 7 findings
**Calls**:
- generator `deepseek-v4-flash` — 10.8K→19.6K tokens · 173 s · findings: 7
_No judging took place: fast mode — a single generator, no judge. Findings are published as they are;
a full run (without `--fast`) confirms them with the second generate→verify pass._

🏷️ License: Community · ❤️ Support the project: https://reviewgate.dev/donate

Read the gate line carefully - everyone trips on it, myself included: there are two red findings on the screen and the commit went through. The parenthesis holds the threshold of this run: blocker. Critical is serious, but it is a question for the push, not for a draft; the push is where it gets stopped. And the second thing: fast mode brought both criticals and six of seven findings from team rules - but with no judge. Nobody checked them, and the 🔬 block says so in plain words. What you see at commit time is a draft of a review; the verdict comes at push.

Now the push of that same commit. Same diff, but a full run with a judge and the threshold major; live, shortened with ellipses, the remote address removed. The line numbers in findings come from the model and can drift by a few lines:

$ git push -u origin feat/search
The MR adds client-side search/filter UI backed by a new /api/requests/search call. Main problems:
the network call violates http-only-in-api-service, the component's search subscription is not
lifecycle-managed, search requests are neither debounced nor cancelled, errors are silently shown
as an empty list, and the URL is built without encoding.

🔴 `request-search.service.ts:15` — RequestSearchService injects HttpClient and calls
   '/api/requests/search' directly. Team rule http-only-in-api-service requires all network calls to
   live only in core/api.service.ts … _(team:http-only-in-api-service)_
🔴 `request-list.component.ts:78` — subscribe() in runSearch() is not tied to the component lifecycle:
   if the manager leaves the screen before the response arrives, the callback still runs and writes into
   signals of a destroyed component … _(team:no-leaking-subscriptions)_
🟠 … five more major: no debounce, no cancellation of stale requests, error rendered as an empty list,
   a new BehaviorSubject instead of signals, an unencoded query parameter …

❌ Gate (major) FAILED. Findings: 8 (⛔ 0 · 🔴 2 · 🟠 5 · 🔵 1 · ⚪ 0).
⚙️ Run cost: ≈ 0.06 $

**🔬 Run diagnostics**
**Calls**:
- generator `deepseek-v4-flash` — 10.8K→12.6K tokens (cache: 10.8K) · 105 s · findings: 8
- validator `deepseek-v4-pro` — 10.7K→5.6K tokens · 114 s · dropped: 0 · downgraded: 0
**The judge** confirmed every finding (dropped 0, downgraded 0).

reviewgate: blocking findings — push stopped
error: failed to push some refs to '…'

Compare it with the commit: there, seven findings with no judge and a green blocker gate; here, the same diff went through a judge that confirmed all eight, two of them critical - and the major gate is red. The push is stopped; next come the fixes and another push.

Gate 4: the pull request, a gate on merge rather than a chat

Now there are several of us, the repository is on GitHub, and the review has to stand on the road to the merge rather than in my terminal. Same engine, same policy from the same .reviewgate/config.yml - but a bot, and here the gate is not an exit code any more. It is a check run on the commit (or a plain commit status, if the bot runs on a token rather than as a GitHub App).

It goes up in an evening, without reading the docs. Two ways, pick one.

Way 1: hand it to your agent. The ai-setup.md playbook is written for an agent rather than for a human. Its step-by-step part is written around GitLab, with a GitHub section that sends the agent to /docs/github for the credentials and the webhook. The agent will ask you for the App credentials (or a token) and the model key, show you what it is about to write into .env and which webhook it will create, and verify every step. One phrase into the chat:

set up ReviewGate for our GitHub org following https://reviewgate.dev/ai-setup.md - ask me for the secrets

Way 2: by hand. The terminal:

mkdir reviewgate && cd reviewgate
curl -O https://reviewgate.dev/docker-compose.yml
# put an .env next to it (below), then:
docker compose up -d
curl http://localhost:3000/api/health        # {"status":"ok", …}

The .env holds the GitHub block and the model block (keep one of the variants). The full variable reference is at /docs/install, the model providers with all their keys at /docs/llm:

# --- GitHub, App mode (recommended) ---
GITHUB_WEBHOOK_SECRET=a-long-random-string
GITHUB_APP_ID=…
GITHUB_APP_PRIVATE_KEY_B64=…        # base64 -w0 key.pem  (macOS: base64 -i key.pem | tr -d '\n')

# --- or PAT mode (a quick trial) ---
# GITHUB_TOKEN=github_pat_…

# --- The model, variant A: Anthropic ---
LLM_PROVIDER=anthropic
ANTHROPIC_API_KEY=…
# ANTHROPIC_MODEL=claude-sonnet-5               # claude-opus-4-8 by default

# --- variant B: DeepSeek or any other OpenAI-compatible endpoint ---
# LLM_PROVIDER=openai
# LLM_BASE_URL=https://api.deepseek.com/v1
# LLM_MODEL=deepseek-v4-flash
# LLM_API_KEY=…
# LLM_JSON_MODE=none
# LLM_MAX_TOKENS=64000

# --- variant C: a local model, the code never leaves your network ---
# LLM_PROVIDER=ollama
# LLM_BASE_URL=http://your-ollama-host:11434/v1
# LLM_MODEL=qwen3-coder:30b-a3b-q8_0

On the GitHub side, a GitHub App is the proper way in: the bot gets an identity of its own, short-lived per-installation tokens and a gate through the Checks API. The permissions it needs are exactly five, and I measured them by running into each missing one (details at /docs/github):

permission	level	what for
Pull requests	read and write	the diff, the summary, inline comments, replies in threads
Contents	read-only	repository files: the team config and neighboring files for the judge
Checks	read and write	the severity gate as a check run, if you turned it on
Issues	read-only	only so that the Issue comment event exists - without it the bot never sees a question under the summary
Metadata	read-only	GitHub always requires it

Subscribe to four events: Pull request, Pull request review comment, Pull request review, Issue comment. Then the webhook: URL https://your-bot-host/api/webhooks/github, content type application/json, secret equal to GITHUB_WEBHOOK_SECRET. The bot verifies X-Hub-Signature-256 over the raw body in constant time; an unsigned request gets a 401, a valid one gets 202.

One trap worth knowing before it costs you an evening: changing an App's permissions takes two steps. Ticking a box changes the App; every installation then has to accept the new set (organization settings → GitHub Apps → Configure → "Accept new permissions"). Until that happens GitHub keeps sending the old set of events, and from the outside it looks exactly like "the bot is silent" - with not one error in the logs.

A personal access token works too (Pull requests write, Contents read, Commit statuses write), with three differences: comments come from your own account, the gate degrades from a check run to a commit status, and the bot never answers comments from the account whose token it runs on - that is the anti-loop guard. In PAT mode the conversation is open to everyone except you.

In the team config, the same file from the rules chapter changes in exactly these places:

# …
severity_gate: major         # was off: blocking findings → a red commit status

llm:
  generators: [{ model: claude-sonnet-5 }]               # was deepseek-v4-flash
  judges:     [{ model: claude-opus-4-8, effort: high }] # was deepseek-v4-pro
  # …

questions: true              # ❓ questions to the author, built from the judge's rejects
reply:
  enabled: true              # replies in threads (up to 3 bot replies per thread by default)
# …

With branch protection requiring checks to pass, a red check run locks the Merge button - and that is the gate. Here is what the team sees on the pull request:

one summary comment that lives and gets rewritten in place on new commits (not a second one, not a third - the same one), with a digest of the findings and a cost line;
inline threads anchored to diff positions, and on mechanical fixes a native suggestion block with a "Commit suggestion" button;
the line ❌ Severity gate (major) failed: N blocking findings → the check run turns red; after a fix comes an incremental run, previously open findings get closed, and the gate remembers them and only lets the pull request go when all of them are gone;
a conversation in the thread: the bot answers an objection under its own finding, and answers a mention of its account under the summary;
findings that come from your team's rules carry the 📐 badge next to the id - the one promised in the first chapter: 🔴 http-only-in-api-service 📐 — ExportCsvService injects HttpClient and calls '/api/requests' directly …

Here is that gate on a real pull request: #2 in the sandbox, a CSV export the managers asked for. Five findings, three of them blocking, Severity gate (major) failed, check run red, ≈0.25 $ for the run.

The conversation, where a text generator and a reviewer part ways

Agreement is the most probable next token. A reviewer weighs the argument instead: sometimes the argument wins, sometimes it doesn't.

Thread one, escaping the CSV. The bot's finding was the classic one: fields joined with a bare comma, no quoting, and a value starting with = becomes a formula when the file opens in Excel. My objection as the author:

Request subjects are typed by our own managers, there is no external data on this panel - isn't escaping overkill for an internal tool?

The bot did not fold. It split the answer in two: column corruption is a correctness bug that stands on its own even if every field were trusted, and clientName/clientPhone come from the customer, not from the manager, so untrusted input does reach an Excel file. It kept the finding.

Thread two, the date format. The finding was minor: the export glues the date together by hand instead of going through the app's pipes, so the file and the screen disagree. My objection:

The CSV is opened in Excel, which re-formats dates on its own anyway, and the file is a machine artefact rather than the screen. Does the export really have to match the table format?

This time the bot gave up half of its own finding and kept the half it could prove:

Fair point on dates — Excel does interpret and re-render M/D/YYYY on its own, so matching the on-screen format there is not important; that half of the comment can be dropped. Phone numbers are a different story though: Excel treats unquoted digit strings as numbers, so a phone like 0501234567 can lose the leading zero or get turned into scientific notation once opened.

There was a third exchange, later, and it is my favorite line the bot has produced. After the fixes I objected that the error handling now lives in the component, not in the service. The bot agreed - and drew the boundary of its own knowledge without being asked:

… no additional catchError is required in the service itself … I can't see request-list.component.ts in this diff, so I'm taking your description at face value; if that's accurate, this thread can be closed.

The most useful finding of the whole pull request was in the fix

I pushed the fixes - the request through ApiService, escaping, formatDate, the phone through the pipe, a busy state on the button. The summary was rewritten in place, Incremental review: 3 of 4 changed files checked. And the incremental run found this:

🟠 export-triggers-shared-cache-refetch — buildAndDownload() calls this.api.loadRequests(), which internally does tap((items) => this.requestsSubject.next(items)). That subject backs requests$, the same stream the request table's async pipe renders … So clicking 'Export CSV' silently triggers an extra network round trip and overwrites the visible list's data behind the manager's back, potentially reshuffling rows they're currently reading.

I had introduced that while fixing something else, which is where this class of tool earns its keep: not on the sins you plant on purpose, but on the ones you add while removing the previous ones. The judge, in the same run, dropped a speculative null-safety candidate: "ServiceRequest fields are typed string; null-safety concern is speculation not backed by code."

One more fix (a read-only fetchAllRequests() that doesn't touch the cache), all threads resolved, and the next run closed it:

✅ Severity gate (major) passed.
✅ No findings.
Open findings on this PR: 0.
🧾 Total for this PR (4 runs + 3 replies): ≥ 167.1K tokens in · 39.7K out · ≥ 1.01 $

The check run went green and Merge unlocked. One honest detail from that last stretch: resolving the threads does not recompute the gate by itself - the recount happens on the next run. So the green status arrived with the next commit, not the moment I clicked "Resolve conversation".

And the numbers, from the footer of that same summary (the cost line is switched on with cost.show, so the whole team sees it):

one review on the "sonnet finds → opus judges" scheme: generator 109 s, judge 23 s, about two minutes and ≈0.25 $. Note the judge: 23 seconds. An expensive model is not a slow one;
a reply in a thread: 15-23 s and 0.02-0.06 $;
the whole pull request: 4 runs and 3 replies, ≥1.01 $ by the counter in the summary.

That is a small feature in four files reviewed by an expensive pair of models. How the numbers change from step to step is in the cost table below.

Closed network: when cloud LLMs are off limits

The last check of the frame: does any of this survive in a place where source code must not leave the network? The question is really just "where does the model run". I switch the provider in my personal config to a local one (Ollama, an OpenAI-compatible endpoint, no keys involved) and declare both roles on the local model. Rules, thresholds and hooks are untouched:

llm:
  provider: ollama
  base_url: http://localhost:11434/v1
  generators: [{ model: qwen3-coder:30b-a3b-q8_0 }]
  judges:     [{ model: qwen3-coder:30b-a3b-q8_0 }]

That part works exactly as advertised: the same policy, the same gate, the same exit codes, and after four review runs nettop on the ollama serve process still reports zero bytes in and zero bytes out. And then the honest part, which I am not going to dress up.

I ran both diffs from this article through that setup on an Apple Silicon laptop. Here is what came back:

diff	mode	time	result
CSV export	fast	259 s	3 findings, all `info`, all noise
CSV export	full	374 s	1 `info` finding; the judge stripped its fix
task of the week	fast	374 s	3 findings, all `info`
task of the week	full	590 s	the same 3, and the local judge confirmed all of them

Not one of the real defects. On the search diff the model reported three times that HttpClient "is unused" in a service that uses it two lines below. The cloud pair found the leaking subscription, the missing debounce, the swallowed error and the CSV injection on those same diffs.

Three conclusions, in order of usefulness.

The machinery is not what degrades - the model is. Rules, judging, gates, diagnostics, exit codes: all identical down to the line. What collapsed was the quality of the findings.
A weak judge is worse than no judge. In the last run the local judge confirmed three worthless findings. A judge is only a filter while it is stronger than the generator; make both of them the same weak model and you get a rubber stamp. If you have one strong model and one weak one, the strong one belongs in the judge's seat.
A laptop is not what a closed network runs on. Mine is a 30-billion-parameter model in an 8-bit quant on a personal machine - a personal experiment on personal hardware. An organization that cannot use the cloud puts a GPU server under the model and runs something bigger; the gap narrows and "zero bytes out" stays exactly the same. How much of the gap your hardware closes is for you to measure - I don't have those numbers.

If that is your world, the working recipe is: keep the fast gate close to the keyboard, put the full run on push and on the pull request, and give the judge the strongest model you are allowed to run. vLLM, your own gateway, anything OpenAI-compatible: it needs an address and a model name.

What it costs, step by step

All the numbers are live runs from this article: two small diffs (the task of the week - 4 files, ~140 lines; and the CSV export), an 18-line agent diff and a one-paragraph documentation change, one to four runs each, measured on 24 August 2026. This is not a benchmark, it is an order of magnitude. The formula to remember is "seconds to minutes, cents to a dollar".

step / mode	models	diff	time	cost
gate 1, `review_changes` in the agent	sonnet-5 → judge opus-4-8	"Delete request" button, 5 files	165 s; the re-check after the fix 272 s	0.30 $; 0.52 $
gate 2, pre-commit, fast	DeepSeek flash	task of the week, 4 files	173 s	≈0.03 $
gate 2 → 3 on the agent's diff (18 lines)	sonnet-5 → judge opus-4-8	the call button	35 s / 70 s	0.06 $ / 0.17 $
gate 3, pre-push, full	flash → judge pro	task of the week, 4 files	222 s	0.06 $
gate 3, pre-push on a trivial change	sonnet-5	1 file, a paragraph of docs	12 s	≈0.01 $
gate 4, pull request review	sonnet-5 → judge opus-4-8	the CSV feature	~2 min (the judge: 23 s)	0.25 $; the whole PR (4 runs + 3 replies) ≥1.01 $
gate 4, a reply in a thread	generator sonnet-5	-	15-23 s	0.02-0.06 $
closed network, fast / full	qwen3-coder:30b-a3b-q8_0, Ollama, laptop	the CSV feature	259 s / 374 s	0 $
closed network, fast / full	the same	task of the week	374 s / 590 s	0 $

A full run takes minutes, and that is not a bug, it is the ladder: the further from the keyboard, the stricter and the slower. At commit time you wait seconds; at push time you are prepared to wait for a verdict; on the pull request you don't wait at all, it arrives on its own.

What is not here

What it doesn't do:

it doesn't find everything - you saw the spread;
it doesn't replace a human reviewer - the "human review" step never left the diagram;
it isn't free in tokens - the tool is free, the model is yours and on your bill;
it isn't open source - the binary is closed and lives on donations. I plan to open it later; not today.

There are no sources to read, so I am not asking you to take my word for it. Here is what you can check yourself:

the review policy is YAML in your repository, and the result is --json plus exit codes;
the code and the diff go only to your model with your key. They do not reach me, the author of ReviewGate, and are not stored anywhere;
on a local model, "zero bytes leaving the machine" is visible in nettop on the model's process;
SHA256SUMS sits next to the binaries.

Reproduce it yourself

The sandbox with that exact project is github.com/ReviewGate/service-desk, and its history is the scenario: the tag before-search (rules and hooks already in place, no feature yet) → search-unreviewed (search with its sins: the commit passed pre-commit, the push was stopped by the gate) → main (fixes from the review, and then the team stage). To repeat gate 3 on that same diff:

git clone https://github.com/ReviewGate/service-desk && cd service-desk && npm install   # the hooks install themselves
git checkout search-unreviewed
reviewgate review --refs before-search --fail-on major; echo "exit=$?"   # what pre-push does: 2 = push stopped

The model scheme in the repo config is DeepSeek. With a key from another provider, declare your own scheme at home (generators/judges) and it replaces the team one entirely - there is an example in the README there. Everything else is at reviewgate.dev/docs.

FAQ, before you ask in the comments

Linux / Windows? Six platforms, including musl for Alpine and Windows; the install command for each and the SHA256SUMS are on /docs/agents, and the binaries are mirrored on GitHub Releases.

How is this different from the AI review built into GitHub or GitLab? Four checkable things, as of this writing:

your team's rules, from your repository;
your key and your model, down to your own hardware;
one engine in three places - in the agent, in the hooks, on the pull request;
a gate you configure, rather than one you accept as given.

I won't take a position on "who is smarter" - that depends on the model, and the model is yours.

How many false positives? I haven't measured it publicly, and the number depends heavily on the provider and the model you pick: DeepSeek and Sonnet/Opus behave differently.

Who uses this? There is a pilot in a real team. No name and no numbers - I don't have permission for either.

Two AI gates, isn't that expensive? See the pre-commit and pre-push chapter: before the commit, the one who chooses the step pays (own key, company key, or a local model at 0 $); on the pull request, the team pays, the same way it pays for any other gate.

DEV Community