DEV Community

Cover image for Everybody has lost their minds, and the boring companies are quietly winning
Levelbrook Consulting
Levelbrook Consulting

Posted on Originally published at ai.levelbrook.com

Everybody has lost their minds, and the boring companies are quietly winning

A senior security engineer wrote this week that he spends three quarters of his time on AI and it has robbed him of his enjoyment of the work. He is right about the symptom and wrong about the cause. The value from these tools is landing in the least glamorous places in the company, and the people looking at the frontier are looking the wrong way.

The vent, and the sentence buried in it

Jan Schaumann's post this week is a vent and he says so in the first line. A senior security
engineer, decades in, writing that he spends upwards of three quarters of his time directly or
indirectly dealing with AI and that it has robbed him of most of his enjoyment of the work. People
with no engineering background pitching industry-changing solutions from their agent-infested
homelab. Emails that read like influencer posts. Colleagues turned into meat proxies. It hit the
front page because a great many people feel exactly this and most of them are not allowed to say
it at work.

You can disagree with a lot of it, and the thread did. But there is one paragraph in the middle
that is not a vent. It is an argument, and it is the most important thing anyone wrote about AI
this week.

He describes the industry-wide effort, now many months old, to point frontier models at
vulnerability discovery. Dozens of highly paid engineers per organisation, priorities reshuffled,
harnesses built, pipelines built to shoehorn thousands of findings into vulnerability management.
Thousands of new vulnerabilities found. And then: I don't think we're any safer than before. Because
finding vulnerabilities has never been the bottleneck in information security. The bottleneck is
still, as ever before, getting the packages updated. Patching is still hard.

That is the whole essay. The models are extraordinary at the part that was never the constraint.

Meanwhile, at the frontier

Consider what the rest of the industry was looking at while he wrote that.

A researcher quit one of the labs and posted a warning that went, by ThePrimeagen's count on
stream, to well over a hundred million views in a day. A colleague who stayed put the odds of
catastrophe above ten percent in a decade. The lab's CEO published a three-point plan to pace the
frontier. A rival CEO agreed with him, which people found unusual. AI Explained spent a video on the
researchers' stated reason: a large gap, in one OpenAI researcher's phrase, between the internal
and external perception of the rate of progress. Six axes of improvement, none near saturation.
Fireship covered a 154-page threat report cataloguing eight months of misuse across seven
categories.

All of this is real and none of it is unimportant. But look at who is in the audience. Engineers,
managers, founders, people who have to decide on Monday what to do with a budget. And the frontier
discourse gives them precisely nothing to do on Monday, because it is about capabilities they do not
control, on timelines they cannot influence, at companies they do not work for. It produces
anxiety with no action attached, which is the most tiring kind, and it is why Schaumann is
exhausted.

Where the attention went this week, and where a Monday decision can actually land. Nothing in the top row is actionable by anyone outside a lab.
Where the attention went this week, and where a Monday decision can actually land. Nothing in the top row is actionable by anyone outside a lab.

Where the value is landing

We install AI systems in ordinary companies for a living, which gives us a view of this that the
frontier discourse does not have, and the view is this. The money is being made in the bottom row of
that figure, by people who are not on Hacker News, doing things nobody will write a threat report
about.

The shapes are always the same, and we describe them here as composites rather than clients, as
everything on this site is. A clinic group that uses a model to reconcile years of insurance
remittances against deposits and finds the pattern of underpayments a human was too busy to see. A
logistics firm whose intake now reads every inbound document and writes the structured record, so
the person who used to do that handles only the exceptions. A software team whose agent does not
write features but does read every production error, correlate it to a deploy, and open a ticket
with the likely cause before a human has seen the alert. And, in public this same week, Cloudflare
saving another hundred terabytes of RAM with what their post cheerfully calls maths.

None of that is at the frontier. All of it runs on models a generation or two behind the ones in
the headlines, because the constraint was never the model. Schaumann's line about vulnerability
discovery generalises: in almost every function of almost every company, the thing the model is
best at was not the bottleneck, and the bottleneck is some unglamorous downstream step that nobody
funded because it was boring. Patching. Inventory. The written specification. The approval seat.
The person who has to say yes.

The companies winning this year are the ones that noticed the bottleneck first and pointed the
tools at it, or more often pointed the tools at everything upstream of it and then spent the
savings on the bottleneck. They are not talking about it, partly because it is unglamorous and
partly because it is working.

Why the frontier framing hurts

There is a specific mechanism by which the frontier discourse makes ordinary organisations worse
at this, and it is worth naming because it is avoidable.

The frontier framing says the model is the variable. Wait for the next one; it will do what this
one could not. That framing is true for the lab and false for the buyer, and a buyer who believes it
does two bad things. They delay the boring work, because the boring work will surely be automated
by the next release. And they evaluate every tool on ceiling, on the impressive demo, rather than
on the floor, on what it does at two in the afternoon on a dull task with a tired reviewer.

The second thing the framing does is imply that the risk is exotic. Swarms, bioweapons, the
internet taken over. Meanwhile the actual risk in the actual company is the one CNN reported this
week from the military: a model produced a confident report with things in it that were not true,
and it got some distance up the chain before anyone checked. That failure does not require a
frontier model. It requires an unreviewed one. Every company installing these tools this year is
building that failure mode unless it builds the seat that catches it, and the seat is boring, and
the frontier discourse never mentions it.

The frontier framing versus the operator framing. Same tools, opposite conclusions about where to spend Monday.
The frontier framing versus the operator framing. Same tools, opposite conclusions about where to spend Monday.

How to find the boring bottleneck

The instruction "find the least interesting problem and fix it first" is easy to nod at and hard to
act on, so here is the exercise we run in the first week with any organisation.

Pick one process that produces money or stops it. Invoices out, claims in, orders through, tickets
closed. Walk it end to end with the people who do it, and at each step write down two numbers: how
long the step takes when it goes well, and how long the work waits before that step starts. Almost
nobody has the second number, and the second number is the process. A step that takes four minutes
and waits two days is not a four-minute step.

Then find the step where the wait is longest, and ask why the work is waiting. The answer is nearly
always one of three things. A person has to decide something and is busy. A piece of information is
missing and somebody has to go and find it. Or two systems disagree and a human has to reconcile
them by hand. Those three are the bottleneck, and none of them is "the model is not smart enough".

Now point the tools. Missing information is the easiest: a model that reads the inbound document
and fills the record removes the wait entirely. Systems that disagree is the next: a model that
reconciles the ninety percent that match and hands the rest to a person, with the mismatch
highlighted, turns a day of reconciliation into an hour. The busy person deciding is the hardest and
the most valuable, and it is the approval seat: give them the decision in a form that takes eight
seconds and they will make forty before lunch.

Nothing about this requires the frontier. It requires a whiteboard, an afternoon, and a willingness
to be interested in a process everyone in the building considers beneath them. The companies
winning this year did that afternoon a year ago.

Schaumann is also wrong, and it matters how

The honest paragraph. He is wrong that the technology is the reason the work stopped being
enjoyable, and the thread said so in the most upvoted reply: the models genuinely produce a lot of
good work, can debug in an afternoon what took a team a week, and the people running the companies
are, in that commenter's phrase, the most boring supervillains imaginable. All true at once.

What robbed him of the enjoyment is the framing, not the tool. Seventy-five percent of a senior
engineer's time spent on AI is seventy-five percent spent in the top row of the figure, on
harnesses for vulnerability discovery that was never the bottleneck, on pipelines to process
findings that will not be patched, on the frontier's priorities rather than the organisation's. Point
the same engineer and the same models at the bottom row, at the inventory and the patching he
himself names as the real work, and the time is not wasted and the work is not joyless. It is the
job he signed up for, done faster.

The frontier will keep moving. The researchers may well be right to be frightened; we are not
qualified to say and neither is most of the audience. But the companies that come out of this decade
ahead will not be the ones that watched it most closely. They will be the ones that found the
boring bottleneck in their own building, wrote it down, and put the tools to work on either side of
it while everyone else was reading the threat report.

Everybody has lost their minds. The way back is to find the least interesting problem in the
company and fix it first.

Sources


Originally published on the Levelbrook playbook. Levelbrook is a principal-led Rails and AI-systems consultancy; the playbook is where we write down what we see.

Top comments (0)