This is a submission for the MLH x DEV Writing Challenge
What I Built
AI writes code faster than anyone can read it, so people end up just pressing Enter. And asking another AI "is this fine?" doesn't solve anything: "the AI said it was fine" is exactly what Tieto's challenge at Junction 2026, Earned Trust in AI Code, ruled out.
So with my team (GOATiers) we built Katse: a PR review tool that lets a named human sign every AI-written change without reading every line.
The idea comes from outside software:
- 🔬 Science doesn't trust a paper because a reviewer read it twice. It trusts it when another lab reproduces it.
- 🏦 Banks don't re-read every entry. They reconcile two independent ledgers and only look at the mismatch.
An agent's change is cheap to produce, so it's cheap to reproduce. Katse rebuilds the change blind with several agents from different model families, runs the tests, attacks the result and compares everything against the original requirement. What reproduces, is proved, or passes a test that actually bites is backed by evidence. What doesn't is the residue: the only part the human owner reads, questions and signs.
Two rules hold everywhere:
- Evidence is executable, or it isn't evidence.
- Consensus directs attention. It never approves. Katse has no approve button and never merges.
On Tieto's sample change (#2283, a 228-line agent-written crash fix) Katse compiled it on the pinned base, saw 3/3 new tests go red→green, ran 174/174 regression tests, caught 3/4 seeded bugs, and threw out a planted false accusation by running it. It also refused to decide one thing: the fix silently drops some diagnostics, which the requirement never asked for. That's a product decision, so it became a question for the owner. Result: the human reads 29 of 228 lines plus 2 questions.
Across the four Tieto branches, Katse said "don't trust this yet" as readily as "this is fine". One PR was sent back because 4 of its 10 new tests passed even without the change.
What I learned: "blind" is a property of the environment, not of the prompt. Every agent call is code execution, even when you only ask for text. And a reviewer that accuses falsely loses trust as fast as one that lets bugs through.
Demo
- 💻 Code: github.com/janvilaregut/Junction2026
- 📑 Slides (PDF) · 🃏 Concept card · 🔎 Evidence on #2283
In the UI you see every branch as a tree, a dependency graph where size = blast radius (PageRank) and colour = what maths can prove, the blind rebuilds judged against a quote from the requirement, and finally the screen where you sign.
Partner Technologies
Entire (we were also in the running for MLH Best Use of Entire). Entire saves the AI session behind every commit directly in git. For a review tool that's huge: Katse can walk back from a change to the original prompt and check whether the code does what was actually asked, not just what the agent decided. We install it inside the repo with a pinned SHA256, and npm run entire -- push only pushes checkpoints if no secret is found. Being honest: Tieto's branches had no saved prompts, so the demo branches use seeded Entire fixtures (marked as such in the code), and the Tieto reports say "original prompt: not available".
Rest of the stack:
- Gemma 4 (local, oMLX / Ollama) as a privacy gateway: in strict mode every cloud call must pass a local scan first. If local Gemma is down, nothing is sent. Fail-closed.
- Gemini 4 Hydrogen / Carbon for the contract, blind rebuilds and the judge (who must quote the requirement).
- NVIDIA Nemotron 3.5 Super as a second model family, so agreement isn't one model agreeing with itself.
- Gemini Live for voice ("Hey Katse").
- Node.js (no framework), Roslyn and .NET 8 for C# symbol analysis and running tests.
The maths layer (PageRank, proofs, entropy, complexity) has no model in it: no model can turn a node green.
Hackathon Experience
I was at Junction 2026 in Finland with team GOATiers, and we took 2nd place in Tieto's challenge 🥈.
What broke at 3am? Our own reviewer. In the first full run, the "blind" agents escaped their sandbox and worked inside the original repo, where they could see the answer. We had set each agent's working directory but not PWD, and the CLI trusted PWD. Later, the model that was only supposed to write text (the judge) decided it was a coding task and edited the repo. We only caught it because git status showed a file modified at the exact second the judge ran. Both are now in our lessons log, together with 18 other failure modes, and they became the best answer to the judges' question: "What would make you not trust your own output?"
But honestly, the prize was only half of it. ~1,800 hackers, ~800 projects and one sleepless weekend in Finland, building with three absolute legends: Meulén Cirillo Veltri, Jan Vilaregut and Fabián Birch Torrejón 🐐. We met so many amazing people from all over the world, and I'm leaving with new friends and a lot of inspiration.
Huge thanks to Tieto, the Junction organisers and MLH. 💜
Top comments (0)