BrassCoders flagged a finding in all 15 AI-generated Python files in its published corpus — 53 findings total, 16 of them critical or high severity. On careful triage, 9 of 15 files carried a confirmed real issue. The other findings were false positives or style notes.
That 53-to-9 ratio isn't a problem. It's the scanner working as designed.
What the Triage Numbers Actually Show
BrassCoders scanned 15 AI-generated Python files and produced 53 findings; after triage, the confirmed real issues were: SQL injection in two files, command injection in two files, hardcoded credentials in two files, an unsafe yaml.load call, a requests call with no timeout, Flask debug=True, and one genuine O(N²) string-concatenation loop. That's 9 of 15 files with at least one real problem.
The rest — a mix of performance flags on integer counters, an MD5 hash in a deduplication context, and a few low-severity style notes — didn't survive triage. They weren't bugs. They were the scanner reporting the pattern, and the triage layer correctly dismissing them.
The full corpus and results are published in the BrassCoders OSS repo, reproducible by anyone with pip install brasscoders.
Why page += 1 Looks Like a String-Concat Loop
BrassCoders flagged four O(N²) string-concatenation findings across the corpus, but only one was real. The other three — in api_client.py, log_parser.py, and retry.py — were counter increments: page += 1, count += 1, a loop index.
The rule matches integer addition inside a loop when the accumulator variable follows the same AST pattern as a string-concatenation loop. It fires on result += chunk and page += 1 the same way, because the rule reads structure, not type. At the pattern level, they look identical. The scanner doesn't know whether page is a string or an integer.
This is the design constraint that makes false positives unavoidable in any structural analyzer. A rule specific enough to match only string operations wouldn't match bytearray += bytes_chunk, a genuine performance bug. The rule fires broadly; the triage layer reads context.
The MD5 Case: One Pattern, Two Meanings
In file_dedupe.py, BrassCoders flagged an MD5 call with a high-severity security finding. The file uses MD5 for content deduplication — computing a hash of a file's bytes to detect duplicates, where collision resistance doesn't matter and MD5 is perfectly fine. The finding was a false positive.
In a different context, the same MD5 call hashing a user password is a genuine security issue. The structural pattern is the same: hashlib.md5(data).hexdigest(). The meaning depends on what data is and what the result is used for.
A scanner that inferred context and suppressed the finding when the variable name was content_hash or the file was named deduplication.py would solve the false-positive problem for those cases. It would also suppress real credential-hashing bugs where the developer happened to name the variable content_hash by mistake, or where the file was refactored to handle passwords after the original deduplication logic. The suppression teaches the AI consumer to trust the demotion — and the one real bug in ten never surfaces.
The Case Where False Positives Reveal Real Bugs
The api_client.py false positive has a detail worth slowing down on. BrassCoders flagged a false string-concatenation finding in that file — the counter-increment pattern. Triage dismissed it. But one line above the flagged line, the file had a real O(N²) bug the scanner missed: items.insert(0, item) in a loop, a list prepend that shifts the entire list on every iteration.
The scanner reported a false positive and missed the real bug in the same file. Triage caught both: it dismissed the false positive and, while reading the surrounding code to confirm the dismissal, spotted the real insert(0) pattern. This is the case the CLAUDE.md architecture principle anticipates: "if brass infers wrong, the AI consumer trusts the demotion and the real bug ships." The inverse is also true — a scanner that reports broadly gives the triage layer a reason to read the code carefully, and the careful read finds what the rule missed.
How to Triage Efficiently
BrassCoders sorts findings by severity before writing .brass/ai_instructions.yaml, so the critical and high-severity findings lead the file. An AI assistant reading the file starts at the most important findings and works down. Triage the critical and high entries first; most low-severity findings can be reviewed after the dangerous ones are addressed.
For the false positives that do slip through, the remediation field gives the AI assistant the pattern it was matching. A remediation that says "replace with io.StringIO or ''.join() accumulation" is easy to confirm as wrong when the flagged line is page += 1. The model reads the remediation, reads the line, disagrees, and moves on. The conversation takes five seconds.
BrassCoders Paid's enrichment pass handles this layer automatically before findings reach the file. The OSS core leaves it to the triage session, which in practice means one pass with Claude Code before addressing any finding.
pip install brasscoders
brasscoders --offline scan /path/to/your/project
Top comments (0)