Every merge your team ships goes through a build, a test suite, and probably a linter. It does not go through an AI code reviewer unless someone explicitly posts the diff. That gap is the problem.
Asking a frontier model to review AI-generated code is good practice. It catches things. The issue is the word "asking": it only happens when someone remembers to ask, and it produces a different answer each time. You can't build a gate on a conversation.
The Automation Problem
BrassCoders's generation-mode benchmark ran six neutral code-generation tasks and asked the same model to self-review its output. The model warned about a performance problem it had just introduced in 0 out of 6 tasks without an explicit review prompt. The bugs were there — an insert(0) in a loop on one task, an unbounded poll on another — but the model's "next step" after writing code is never "wait, let me check this again." It writes and exits.
A CI system that runs only when a developer types a command is not CI. It's a suggestion box.
Non-Determinism Disqualifies a Tool from Gate Duty
BrassCoders's benchmark ran the same 12 Python files three times and returned identical results every time. An LLM asked to review those same files returned different findings each run — not wrong findings, but different ones. A finding that appeared in review #1 ("this SQL string is built with f-string interpolation and might be injectable") was absent from review #2. This is not a bug; it's how probabilistic sampling works.
A gate has to be reproducible. "Did this commit pass review?" has to have the same answer on Tuesday as on Thursday, for the same code. LLM reviews cannot provide that. Static analysis — rules applied to an AST, deterministically — can.
BrassCoders runs 12 scanners on the same source and produces identical output on every run. If perf_antipatterns/csv_export_builder.py has an O(N²) string-concatenation loop, BrassCoders flags it on every scan, not 92% of scans.
The Cost Compounds at Commit Volume
BrassCoders OSS core runs at $0 per scan — Apache 2.0, installs with pip install brasscoders, no outbound API calls. The same 500-line diff reviewed by claude-sonnet-4-6 at Anthropic's published June 2026 pricing costs roughly $0.003–0.005. At 50 commits per developer per month, that's $0.15–0.25 per developer per month in review tokens before tooling overhead.
The dollar figure isn't the point. Free scans run without anyone approving the budget line.
Code Egress Is a Hard Constraint for Many Codebases
BrassCoders's --offline flag sends zero bytes off the machine — not findings, not paths, nothing. For regulated industries, proprietary algorithm codebases, or any code under NDA, that's the difference between "can run on every commit" and "can't run at all."
You can't opt a proprietary codebase into LLM review on every commit. The LLM review requires an outbound API call with source in the payload. brasscoders --offline scan has no outbound call.
The optional Paid-plan enrichment sends already-redacted scanner findings through the BrassCoders gateway plus a project signature of at most 7,500 characters — never raw source code. The data handling page documents this precisely. For teams with strict egress policies, offline mode is the complete product.
What a Pre-Merge Gate Actually Requires
BrassCoders satisfies four constraints that together define gate status; an LLM reviewer satisfies none of them.
- Deterministic. Same input, same output, always. Reproducible failures. No sampling.
- Automated. Runs without being invoked by a human. On every commit, not on remembered ones.
- Free per run. No tokens, no API cost that scales to commit volume.
- Local or minimally egressing. Acceptable for codebases with data-handling requirements.
A linter satisfies all four but covers a narrow category. BrassCoders covers performance anti-patterns, secrets, PII leaks, and interprocedural taint flows that static linters miss — and it satisfies all four.
What AI Review Does Well (and Why You Still Want It)
BrassCoders makes no judgment calls. It finds O(N²) loops and SQL injection and hardcoded AWS keys and Luhn-valid credit card numbers in seed data. A frontier model reviewing the same PR does something different — it reasons about intent.
The model says "this abstraction is adding complexity without adding flexibility — consider collapsing these two functions." It catches intent-level problems: the code does what it says but the wrong thing overall. It spots naming that will confuse the next developer. These are not deterministic-rule problems. They're judgment calls BrassCoders doesn't make.
The categories are different. The tools are complementary. BrassCoders is the pre-merge gate (automatic, local, deterministic). A frontier model is the PR advisor (invoked by choice, reasoning about design). Neither replaces the other.
Setting Up the Gate Takes Three Minutes
Add BrassCoders to CI with the GitHub Actions workflow from the BrassCoders OSS repo. The file is eleven meaningful lines. It installs brasscoders, runs brasscoders --offline scan, and uploads .brass/ as an artifact. Push it and the gate runs on every PR.
pip install brasscoders
brasscoders --offline scan /path/to/project
The output lands in .brass/ai_instructions.yaml, ranked by severity and ready for Claude Code or any other AI assistant to read. The AI assistant gets the signal without seeing the source; the CI log shows pass or fail.
That's the loop. Not a conversation. A gate.
Top comments (0)