DEV Community

Cover image for The vibe-coding trap has a name, and the name is not "the model"
Levelbrook Consulting
Levelbrook Consulting

Posted on Originally published at ai.levelbrook.com

The vibe-coding trap has a name, and the name is not "the model"

A programming language shipped this week with a 99 percent AI-written compiler and no mention of the forty-year-old field it reinvents. The failure was not generation quality. It was that building got cheaper than reading, and nobody put a gate between them.

A language, a proof, and a field nobody looked up

Two stories ran side by side on Hacker News this week and they are the same story.

The first is Bend 2, a language pitched for the AI coding era: humans write "laws", the AI writes
implementations and proofs, and the compiler checks the proofs. It got six hundred points and a lot
of admiration. Then Liam Powell wrote a response that got three hundred more, and his point was not
that the language is bad. His point was that the demo on the home page takes 58 lines to state that
a player can never touch the flag, 442 lines of AI-written proof to establish it, and that the
phrase "formal verification" appears nowhere on the website or in the codebase. He then asked a
model to redo the demo in SPARK, a language built for exactly this, with no further guidance, and
it came back a fraction of the size. The README, a commenter noted, says the compiler is 99 percent
AI-written and has not been fully audited.

The second story is Dan Abramov's account of vibing a proof of a conjecture of Conway's with a
model, over days, in a long transcript he published in full. It is a good post and an honest one.
The most upvoted objection under it was a mathematician pointing to Gowers's essay from the same
week on why he did not sign the Fields medallists' letter, and the older point Gowers has been
making for twenty-five years: there is a difference between solving a problem and understanding a
field, and the second is what makes the first mean anything.

Powell names the mechanism precisely and we are going to steal his sentence: vibe coding makes it
possible to build a substantial solution before learning enough about the problem to recognise that
a much better solution exists.

Why this is new

It has always been possible to reinvent a field badly. Every senior engineer has watched a junior
build a job queue in a spreadsheet. What is new is the ratio.

For all of software's history, building was expensive relative to reading. Before you could produce
442 lines of anything, you had spent enough hours inside the problem that you had, almost by
accident, tripped over the prior art. You searched for the error message. You read the paper the
library cited. You asked the person at the next desk, who said "oh, that's just a Bloom filter".
The cost of building was a tax that paid for an education.

That tax is gone. A model will produce the 442 lines in the time it takes to make coffee, and it
will produce them competently enough that they work, and working code is the most persuasive
argument in the world against going back to read. Nothing in the loop ever forces you to discover
that the field exists. The model will not volunteer it unless you ask, and you do not know to ask,
because the whole point is that you do not know the field exists.

The old cost of building bought an education for free. The new cost does not. Time axis illustrative; the shape is the point.
The old cost of building bought an education for free. The new cost does not. Time axis illustrative; the shape is the point.

The Bend story is the pure case because a language is the most expensive thing you can build and
formal verification is one of the best-documented fields in computer science. If it can happen
there, at that scale, with that much talent, it is happening in your codebase this week at a
smaller scale where nobody will write a blog post about it. The agent that built your rate limiter
from scratch instead of reading the one in your framework. The retry logic that reinvented
exponential backoff without the jitter. The custom auth layer.

The gate goes before the build, not after

The instinct is to fix this with review, and review does catch some of it. But review happens after
the 442 lines exist, when the sunk cost is already arguing for them, and the reviewer usually
shares the author's blind spot. The place to put the gate is the fifteen minutes before anything
is built.

We run something we call the prior-art pass, and it is embarrassingly simple. Before an agent is
allowed to build anything with a name, it has to answer four questions in writing and a person has
to read the answers. What is this problem called by people who study it? What do they already use?
Why does the existing thing not work here? What is the smallest version of this we could build on
top of the existing thing instead?

The model is extremely good at answering these questions. It has read the field. It will tell you
about SPARK and Dafny and Lean and TLA+ in one paragraph if you ask it to, and it will tell you what
each is for. The trick is that somebody has to ask before the build starts, and that somebody has to
be willing to hear "this already exists" as good news rather than as an obstacle to the thing they
were excited to make.

The prior-art pass: four written answers, one human read, before an agent may build anything with a name.
The prior-art pass: four written answers, one human read, before an agent may build anything with a name.

Liam Nugent's piece from the same week, on why the most important product decision is what you do
not build, makes the organisational version of the same point. Nobody gets promoted for deleting
things. Those who create and launch are the ones rewarded. The models have made creating and
launching nearly free, which means the incentive that was already skewed towards building is now
skewed by another order of magnitude, and the only counterweight is a deliberate, slightly
unpopular gate that asks "does this need to exist" before the exciting part starts.

The pass in practice

A composite from our own work, because the pass sounds like a platitude until you watch it fire.

A team wanted a service that deduplicated inbound customer records, which arrive from four systems
with inconsistent formatting, so that the same person is not created four times. An agent, asked
directly, would have built it in an afternoon: normalise the fields, hash them, compare. The
prior-art pass asked the four questions first, and the agent's written answers were, in order: this
is called entity resolution or record linkage; the standard approaches are probabilistic matching
in the Fellegi-Sunter family and there are mature libraries in every major language; the naive
hash-and-compare approach fails on exactly the inconsistent formatting the team has, because it
treats a transposed digit as a different person; the smallest version is to run an existing
library with blocking on postcode and hand the ambiguous pairs to a human.

Fifteen minutes. The person reading the answers had never heard the phrase "record linkage". The
team built the small version on top of the library, spent the afternoon they saved on the human
review queue for ambiguous pairs, and did not spend the following quarter discovering, one support
ticket at a time, every way in which the hash approach silently merges or splits real people.

The point is not that the agent knew about record linkage; of course it did. The point is that
nobody would have asked, because the task looked simple and the build was cheap, and the cost of
the field not being known would have been paid by customers over months rather than by the team in
one visible failure. That is the shape of the trap every time. The wrong build does not fail. It
works, slightly worse than the right build, forever.

Where this is wrong

The honest caveat is that the prior-art pass has a failure mode of its own: it can become an excuse
never to build anything new, and some things genuinely are new. Bend's author may well have
considered SPARK and rejected it for reasons that are not on the website. Abramov's proof may be
a real contribution even if he cannot yet situate it in the field. The gate is not "never build".
The gate is "never build without having looked", and the output of looking is sometimes "nothing
here fits, build it, and say in the README what you looked at and why it did not fit". That
sentence in a README is worth more than the 442 lines under it, because it tells the next reader
that the author knew where they were standing.

What to do on Monday

Find the three most recent things your team or your agents built that have a name. A service, a
library, an internal tool, a pattern with a wiki page. For each one, ask the four questions now,
after the fact. Do it with a model; it will take ten minutes each. You will find that at least one of
the three is a smaller, worse version of something that already existed, and you will feel the
thing Powell's post is about, which is not embarrassment exactly. It is the realisation that the
cost of not knowing has gone up precisely because the cost of building has gone down.

Then put the pass in front of the next build. Fifteen minutes, four questions, one reader. The
model will do most of the work. The only thing it cannot do is want to know.

Sources


Originally published on the Levelbrook playbook. Levelbrook is a principal-led Rails and AI-systems consultancy; the playbook is where we write down what we see.

Top comments (0)