The anxiety of the last month has been about falling behind. The actual career risk sits in the other direction: the moment a model's output carries your signature. The military found that out this week. Law firms are about to.
The two fears, and which one is real
There are two fears in circulation right now and they point in opposite directions.
The first is the one Fireship and Theo and every LinkedIn feed are selling: you are behind. The
models moved again, the people using them properly are producing five times what you are, and if
you do not get on this you will be the person the organisation quietly stops giving interesting work
to. There is a version of this fear that is accurate and we will get to it. But as a career risk it
is slow. Nobody gets walked out on a Tuesday for having been sceptical of a tool.
The second fear is the one that actually ends careers, and almost nobody is articulating it because
it is less flattering to the tools. On Thursday CNN reported that the US military had what it called
a close call after an AI-generated intelligence report contained hallucinated content that made it
some distance up the chain before somebody caught it. We do not know the details beyond what was
reported and we are not going to pretend to. But read the top comment on the Hacker News thread,
because it is the whole essay in three sentences: AI is not responsible. People are responsible.
As soon as people choose to remove their own accountability, that is when the bad stuff happens.
The same week, OpenAI launched Astra for Law, a model tuned for legal work. The launch post, as
several commenters noted, does not mention hallucination once. The most upvoted question under it
was not about capability. It was: if I use this to write my contracts, who is getting sued when it
is wrong? That question has an answer, and the answer is you.
Where the signature lands
Every organisation has a small number of places where a person converts information into a
commitment. A lawyer signs the brief. An officer signs the assessment. A reviewer approves the
change. A controller approves the payment. A doctor signs the order. These are the seats where the
organisation has decided, in advance, that a named human stands behind the outcome.
For the whole history of office work, the person in the seat also produced most of the information
that went into the decision, or supervised the junior who did. That has quietly stopped being true.
The brief was drafted by a model. The assessment was summarised by a model. The pull request was
written by a model. The person in the seat is now signing for work they did not do, produced by a
process they did not watch, at a volume that does not leave time to check.
The approval seat has not moved. Everything feeding it has. The seat still carries the name, the liability and the memory.
This is the structural fact under the anxiety. The tools did not remove the seat. They removed the
slack around the seat, the four hours of drafting during which a person used to notice things. And
in a lot of organisations they removed it faster than anybody redesigned what the person in the
seat is supposed to do with their eight seconds.
Martin Fowler's short piece this week, the one titled simply that he does not like LLMs, is being
read as a grumpy-elder post. It is more useful than that. His complaint is that they confidently
bullshit him, often while giving useful answers, with the same assurance either way. That is not a
complaint about capability. It is a description of the exact property that makes the approval seat
dangerous: the output does not carry a signal about its own reliability. A junior who is unsure
looks unsure. A model that is unsure looks like a model that is sure.
What the people who survive this will have done
We build systems where a model is allowed to act, inside a gate a person controls, and the entire
design problem is the seat. So here is what we have learned about occupying one well, offered to
the lawyer and the officer and the reviewer as much as to the engineer.
The first thing is to refuse to sign for what you cannot see. That sounds obvious and it is
violated constantly, because the tools present a finished artefact and hide the process. Insist on
the provenance: what was the model given, what did it produce, what did it check, what did it say it
was unsure about. If the tool cannot show you that, the tool is asking you to carry liability for a
process it will not disclose. The correct response to that is not faster reading.
The second is to make the model's abstentions visible and to treat them as the most valuable line
in the output. A model that has been built to say "I could not verify this citation" or "this
clause is outside the examples I was given" hands the seat exactly the information it needs to spend
its attention well. A model that has been built to always answer hands the seat nothing. Most
vendors ship the second kind because it demos better. Buy the first kind, or add the gate yourself.
Two ways to build the seat. The left one is what most tools ship. The right one is the only one worth signing for.
The third is the record. When something goes wrong, and it will, the difference between a career
ending and a process improving is whether there is a log that shows what the seat was shown, what
it decided, and why. Approvals without a record are how organisations find scapegoats. Approvals
with one are how they find bugs. If you are the person in the seat, you want the record more than
your employer does.
The fourth is the one that answers the first fear honestly. You do need to understand the tools.
Not to produce more, though you will, but because the person who understands how a model fails is
the only person who can occupy the seat competently. A reviewer who has never watched an agent
confidently rewrite a retry loop it did not understand cannot review agent-written code. A lawyer
who has never seen a model invent a citation cannot sign a model-drafted brief. Falling behind is a
real risk, and the reason is not productivity. The reason is that the seat is about to be occupied
exclusively by people who know what they are looking at.
What it looks like when it works
A composite from the systems we run, with the details changed. A finance team processes vendor
invoices through a model that reads the document, matches it to a purchase order, and proposes a
payment. The person in the seat sees, for each proposal, four lines: the amount, the matched order,
the confidence of the match, and a list of anything the model could not reconcile. Most proposals
show an empty list and take three seconds. A handful show one line, "invoice quantity 12, order
quantity 10", and take thirty. Once or twice a day one shows "no matching order found" and gets
routed to a human who investigates.
The team approves several hundred payments a day this way, which is roughly six times what it did
by hand, and the person in the seat is not faster at reading invoices. They never read the invoice.
They read the abstentions. The model was built to say what it did not know, the seat was built to
show only that, and the record of every approval, with what the seat saw, is kept for the auditor.
When a duplicate payment went out in the spring, because a vendor had sent the same invoice under
two numbers, the investigation took twenty minutes. The record showed the seat had been shown two
proposals, each with an empty abstention list, three days apart. Nobody was blamed, because nobody
had been shown anything that should have stopped them. The model was changed to flag matching
amounts to the same vendor inside thirty days. That is what the seat is for: not to be infallible,
but to make failure legible enough that the next one does not happen.
Compare that with the version most organisations are building, where the same person is handed a
finished payment run and asked to approve it by five. That seat has the same liability and none of
the information, and when the duplicate goes out the investigation finds a name rather than a bug.
The uncomfortable version for managers
If you run a team, the obligation runs the other way. You have almost certainly increased the volume
flowing into your approval seats this year without increasing the time, the tooling, or the
authority of the people in them. You may have done it with a slide that said something about
productivity. When one of those seats signs something that goes wrong, the organisation will look at
the name on the signature, not at the slide.
The remedy is not to slow down. It is to treat the seat as the product. Decide, explicitly, which
decisions a model may take alone, which need a person, and which need a senior person. Write it
down. Give the seats the provenance, the abstentions and the record described above. Cap the volume
per seat per day, because a person who approves three hundred things approves none of them. Then,
and only then, let the models run.
The last month of discourse has been about who is behind. The next year is going to be about who
signed. Make sure that when it is your name, you were shown enough to deserve it.
Sources
- US military had close call after using AI for hallucinated intelligence report (CNN) (HN, 499 points, 18 Sep 2026)
- Astra for Law (OpenAI) (HN, 578 points and 678 comments, 17 Sep 2026)
- I Don't Like LLMs (Martin Fowler) (HN, 237 points, 17 Sep 2026)
- Levelbrook: the approval queue demo
Originally published on the Levelbrook playbook. Levelbrook is a principal-led Rails and AI-systems consultancy; the playbook is where we write down what we see.
Top comments (0)