Every change our AI app builder makes gets a second model’s review. A timeout is never a pass.

#ai #security #webdev #programming

I'm building SPOE, an AI app builder. You describe an app, a model writes it into a real Next.js, Fastify and Postgres project, and you can export the whole thing. Last time I wrote about how every export is signed. This post is about the other half: nothing the builder writes reaches your project until a second model has read it.

Why a second model, and why a different family

A model reviewing its own output tends to like it. Two sizes of the same model family share training data, so they tend to share blind spots too. So the rule is that the reviewer must come from a different model family than the writer.

That rule lives in code, not in a doc. The server reads the family off each model's id (Qwen, GLM, Llama, DeepSeek, Mistral and so on). If it can't tell, the operator has to state it. If the writer and the reviewer turn out to be the same family, the server refuses to start:

if (g === r) throw new Error(
  `Reviewer and generator are both "${g}" family. The security review must use a different model family.`
);

If you pick your own model for building, the reviewer is chosen from a different family than that one too. Both run on Venice's private, zero-retention inference.

What the reviewer sees

It gets two things: the original request and the proposed diff. Its instructions open with "You are a security reviewer. You did not write this code."

It looks for, roughly in this order:

Critical: injection, auth that can be bypassed, secrets in code, arbitrary file access, one user reaching another user's data. Also any import of an SPOE package, because the export has to run without us.
High: missing validation at the edges, weak crypto or password storage, SSRF, path traversal, XSS.
Medium: missing rate limits on auth, error messages that leak, insecure defaults.

It answers in JSON at temperature 0: a verdict, and findings that each name a file, a line, the problem and the fix.

Don't trust the model's own summary

A model will happily list a critical finding and then write "verdict": "pass" underneath it. So the verdict is worked out again on our side, from the findings:

const verdict = findings.some((f) => sev(f) === 'critical') ? 'block'
  : parsed.verdict === 'pass' && findings.some((f) => ['high', 'medium'].includes(sev(f))) ? 'warn'
  : parsed.verdict;

A critical finding blocks the change, whatever the reviewer wrote. A blocked change can't be approved at all. You regenerate, and the findings go back to the writer word for word as a checklist.

A timeout is never a pass

This is the rule I care about most. When a review errors, times out or returns something that doesn't parse, the easy move is to let the change through. SPOE doesn't. A network or provider hiccup gets one retry. After that, the change comes back marked as not reviewed, with a note in plain words saying exactly that. Approving it takes an explicit "I know this is unreviewed", and that approval goes into the audit log.

Fast, without skipping the check

At first, every change waited for a human in a diff view. Safe, and slow. Now, with auto-apply on (the default), a change the review passes goes in by itself, as a version you can undo in one click.

Four kinds of change never apply themselves: a blocked one, an unreviewed one, one with a high or critical finding, and one whose files you edited by hand since it was written. Those wait in the diff view for you. A manual edit is never silently overwritten.

Where it falls short

A second model is not a security audit. It catches common, obvious mistakes. It will miss subtle logic bugs, and it can be wrong in both directions.
It reads the diff and the request, not the whole running system.
Calibration never ends. A reviewer that calls everything critical gets ignored, so its instructions spell out what critical means: a human must not merge this, full stop. A missing rate limit is not that.
Two families can still share blind spots. Different beats same. It is not a guarantee.