I've been thinking about this idea for a few weeks now. Instead of fighting with AI agents to follow conventions, what if you just made the wrong thing impossible?
That's the gist of what Ryan Lopopolo's team did at OpenAI. They encoded their team's taste as failing tests. Curly quotes instead of straight quotes? Build fails. Component in the wrong package layer? Build fails. The agent can't produce slop because slop won't compile.
The thing that stuck with me is how much this changes the feedback loop. Normally you'd review a PR, leave comments, the author revises, repeat. With tests enforcing taste, you skip most of that entirely. The agent gets immediate signal. We get consistent output without micromanaging every decision.
I tried something similar on a side project. Added a few conventions as tests before letting an AI touch the codebase. The diff quality went up noticeably. Not because the AI got smarter, but because the guardrails got tighter.
Still figuring out where the line is though. Too few tests and things drift. Too many and you're fighting yourself. Anyone found good heuristics for how much to encode?
Top comments (0)