I ran the same security audit 12 times. The results contradicted each other.
Not because the tool was broken — because one run was never enough to say anything meaningful.
The Setup
I'm building NexaVerify, a multi-LLM code review tool that scans Python code for security vulnerabilities. It uses different AI models (via a cloud inference gateway) and compares their findings through a consensus pipeline.
When I first built the benchmark, I ran each configuration once and compared the F1 scores. Simple. Clean. And completely meaningless.
Here's why.
The Experiment
I took the same corpus of 4 vulnerable Python files (24 known findings), the same ground truth, and ran 3 independent evaluations on each of these 4 configs:
solo-flash — one model only (flash)
solo-llama — one model only (llama)
Union (min=1) — both models, accept findings from either
Quorum (min=2) — both models, require agreement
Every run starts fresh. No cached state. Same prompt, same code, same everything.
The Results
Config Run 1 Run 2 Run 3 Mean F1 StdDev
solo-llama 0.7368 0.7636 0.7143 0.7382 0.0247
Union (min=1) 0.6557 0.6250 0.6471 0.6426 0.0158
Quorum (min=2) 0.5263 0.6522 0.6190 0.5992 0.0653
solo-flash 0.2909 0.5357 0.4138 0.4135 0.1224
Read that solo-flash row again. Same tool. Same code. Same config. F1 ranges from 0.29 to 0.54. That's not a measurement — that's a coin flip with extra steps.
The Surprise Nobody Expected
Llama solo beats the Union pipeline on raw F1. 0.738 vs 0.643.
If you'd told me before the experiment that running two models and merging their findings would produce worse F1 than a single model, I'd have said your math was wrong.
But the math isn't wrong. Here's what's actually happening:
When you merge findings from two models (Union), you inherit the weaknesses of both. Model A finds something Model B doesn't → FP. Model B misses something Model A catches → FN from Model A's perspective gets diluted. The merge logic is conservative by design — it prioritizes not missing anything, which inherently trades precision for recall.
A single model, especially one that's good at this specific task, doesn't have that noise problem.
So Why Bother With Two Models?
Because F1 is not the only metric that matters.
What the Union pipeline gives you that solo can't:
Dual-source traceability. Every finding has two independent origins. You can show an auditor exactly which models agreed, which disagreed, and why. Try doing that with a single model — you get "the AI said so."
Forensic audit trail. Every LLM response is captured in raw JSONL format before parsing. If the output looks wrong, you can trace back to the exact provider, chunk, and timestamp. A single model gives you a result. Two models give you evidence.
Stability under variance. Union has the lowest standard deviation (0.0158) of all configs. Solo-flash has the highest (0.1224). When you need consistent results across runs, the pipeline wins — not on F1, but on reproducibility.
The impossibility of catching everything. No single model catches all vulnerability types. Flash is good at injection flaws. Llama is better at logic errors. The union covers both. The solo doesn't.
The Real Lesson
Someone independently reviewing this approach pushed back on my framing: the real question isn't whether the pipeline beats solo on F1 — it's whether it adds enough value in stability, coverage, and traceability to justify the extra complexity.
And the answer, backed by 12 runs of data, is: yes — but not for the reason most people assume.
The value isn't in a higher F1 score. It's in being able to prove your results are reliable, show where they came from, and demonstrate that no single model's blind spots are silently shipping into production.
What I'm Doing Differently Now
Every benchmark is 3 runs minimum. Reported as mean ± standard deviation. A single number without variance is a guess.
Raw logs are preserved. Every LLM response is captured in JSONL before any parsing. Post-hoc audits are possible on any run.
Cost is tracked per run. Total for this experiment: 17 DZD ($0.13 USD) on a cloud inference provider. 12 runs, 4 configs. Cheap enough to be rigorous.
The held-out corpus is synthetic by design. No real NexaVerify code, no customer data. Each vulnerability is hand-planted with a verified ground truth.
Next Steps
I'm now building a second corpus (T2) with 5 files and 35 findings — denser, more varied, and completely unseen by any model. The same 4 configs will be benchmarked again, same protocol, same rigor.
The question isn't whether the pipeline "wins." The question is whether the extra infrastructure is worth it for real-world audit scenarios where traceability matters more than a single F1 number.
If you're building LLM evaluation pipelines: measure your measurements. The tool is only as good as the data you use to validate it.
Building in public. All benchmarks, raw logs, and methodologies are documented internally. The tool is open for testing at nexaverify.netlify.app.
python #ai #testing #llm #security
Top comments (0)