I have a bookmark folder called prompting. Forty-one tabs in it.
"The 12 prompts that 10x your engineering." "Context engineering for agents, explained." "The system prompt that changed how I ship." A course I finished at 1am, certain I'd finally cracked the thing — that if I just learned to ask well enough, the output would come out right.
I built that folder the same way I once built a GitHub full of dead repos: convinced the scarce skill was the one everybody was selling me.
It wasn't. And I want to say the thing nobody says at the top of the "master prompt engineering, learn context engineering, become an AI-native engineer" sermon:
Getting better at prompting is getting better at the wrong skill. Not a useless one — the wrong one. And the reason is worse than "the models will get better and the prompt won't matter." That's true, but it's the boring half. The sharp half is that even a perfect prompt can't touch the thing that was actually going to hurt you.
Stay with me, because this isn't a doomer post. There's exactly one skill in this whole stack that's still worth every hour — and the logic is what tells you which one.
The three things we tell ourselves about why prompting is THE skill
Nobody grinds prompt guides for no reason. You do it because you believe one of three things:
- It'll make the output better. ("Better prompt, better code. Obviously.")
- It'll make me employable. ("'AI-native,' 'prompt engineer' — this is the moat now.")
- It's the new literacy. ("This is the fundamental skill of the era. Learn it or fall behind.")
All three are real motivations. All three are, as usually practised, pointed at the wrong target. Let me take them one at a time, because the miss is instructive — it points straight at the one skill that survives.
Belief #1, because it's the load-bearing one: a better prompt can't catch the bug that hurts you
Let's do the one everybody believes hardest.
Yes, a better prompt gets you better-looking output. Cleaner structure, fewer obvious mistakes, code that reads like a senior wrote it. That part is real. Here's the part the courses skip:
A better prompt raises how convincing the output is. It does nothing to whether it's correct. And those two dials were never connected.
Think about the failure mode that has actually cost you. It was never "the AI misunderstood me and produced obvious garbage" — you catch that in two seconds. The one that hurts is the opposite: the output that matched your request perfectly, read beautifully, passed the tests you thought to write, and was still wrong in a way you only discovered in production. Plausible. Confident. Broken.
Now watch what a better prompt does to that failure. It makes the output more polished, more authoritative, more obviously-fine at a glance. Which means the wrong ones get harder to catch, not easier. You didn't improve your defense. You upgraded the disguise on the thing you were supposed to be defending against.
That's the whole trap in one line:
A better prompt just gets you to a more convincing wrong answer, faster.
You cannot prompt your way out of this, by construction. The correctness of the answer isn't decided at ask time. It's decided at review time, on your side of the desk, by whether you can look at a clean, confident diff and say no. Prompting optimizes the question. The bug lives in the answer.
Belief #2: prompting is recall wearing a new hat
"But it's a moat. 'AI-native' is what gets hired now."
Here's what nobody wants to hear about their new favorite skill: prompting is recall. The same recall AI just made worthless, re-manufactured one layer up and sold back to you as expertise.
For twenty years, "knowing how to code" was quietly two different things bundled together: recall — the syntax, the API surface, the flag order, the incantation — and judgment — knowing what to build, what to distrust, what breaks at 2am when a real person does something strange. AI ate recall first, and good riddance; it was never the valuable part. Testing for it in 2026 is testing for penmanship.
So what did we do? We rebuilt recall at a higher altitude. "The exact phrasing that makes the model behave." "The context pattern that gets the right output." "The system-prompt incantation." It's the same muscle — memorize the magic words, produce the output — just moved up the stack. We fled the thing AI made free and ran straight into a fresh version of it.
And this version has a shrinking half-life. Every model release infers your intent better than the last. The gap between a novice prompt and an expert prompt narrows with each update, because closing that gap is literally what the labs are optimizing. You are grinding a skill the vendor is actively deleting. You cannot build a moat on the exact thing your supplier ships as a feature next quarter.
Belief #3: the fundamental was never the asking
"Fine — but it's the new literacy. The base skill everything else sits on."
The fundamental skill of working with a system that produces confident, plausible, occasionally-wrong output was never how you ask it. It's what you do with the answer.
Prompting is a UI over the model — and UIs get better on their own, without you. Distrust is a stance toward the output — and it's the one thing that doesn't improve while you sleep. One is a feature. The other is a muscle only you can carry into the room.
There's a completely different skill hiding under "get good at AI," and it only shows up under one specific condition — when the output is wrong and everything about it looks right:
A better prompt produces more convincing output. Only distrust decides whether it's right. The asking is free now — the refusing is the entire job.
You don't earn that by writing better prompts. You earn it by shipping a confident answer to a real person, watching it break, feeling the consequence — and carrying the scar into every diff after. That's the forge. Everything before it is a tutorial with better phrasing.
The moment a perfect prompt handed me a perfect-looking disaster
Let me make this concrete, because I earned it the expensive way.
I once shipped a write path that acknowledged the request before it had actually persisted the row. The prompt that produced it was clean. The code that came back was clean — it read like something a careful engineer wrote. In the demo, in the tests, on my machine: flawless. Exactly what I asked for.
Then one day a retry hit at the wrong moment. The ack went out, the save didn't land, and a paying customer got locked out of their own account with no record they'd ever done the thing. Ack-before-persist. I can still feel it.
No prompt was going to save me there. A better prompt would have made it worse — cleaner code, more convincing, even harder to doubt. The thing that would have caught it wasn't a sharper question. It was the trained reflex to look at a diff that acknowledges before it persists and think that breaks under a retry — before a real human's bad night taught it to me.
That reflex is distrust. It's the only skill in this whole stack that AI can't hand you, can't improve for you, and can't make obsolete. It's the one line on my résumé that would've been worth an entire interview — and no course sells it, because you can't.
Be clear about what I'm actually saying — because it's not "stop prompting"
I am not the guy telling you prompting is beneath you. I prompt all day. AI writes most of my code and I'd never go back — the typing was never the hard part, and neither is the asking. Prompt well. A sloppy prompt wastes everyone's time, yours included.
What I'm saying is narrower and, I think, freeing:
Prompting is table stakes, not the edge. It's the cost of entry now, like knowing how to use a keyboard — necessary, and worth exactly zero as a differentiator, because everyone has it and the tool keeps closing the gap. Grinding it harder is polishing a skill whose ceiling the vendor lowers every month. The edge — the only part that compounds — is what you do after the model answers:
- After the AI answers, before you accept it, try to break it. Ask what it does under a retry, at 2am, when the input is hostile, when the customer does the dumb thing. Attack the answer instead of admiring it.
- Make something other than the author be the skeptic. The thing that produced the diff is the worst possible judge of the diff — it's proud of it. The doubt has to come from a different seat.
- Ship it to someone who can hurt you. That's the only way the wrong ones actually cost you, and cost is the only thing that grows the muscle. A distrust you never had to use isn't a skill. It's a slogan.
Do that, and you're building the one thing that survives every model release. Skip it, and you're the guy with a beautiful prompt library who can't tell when the beautiful answer is going to lock a customer out at 2am.
Why this is the exact reason I build the way I do
Here's the part that goes one level up, because it's the same logic.
I build an agent platform, and the temptation in this whole industry right now is to worship the author — the thing that generates. Look how well it responds to a good prompt! Look how clean the output is! But an author that produces gorgeous, confident output is producing gorgeous, confident output whether or not it's right — and a better prompt only turns the polish up, never the truth. The convincing-ness and the correctness are two different dials, and the whole market is cranking the one that doesn't matter.
So I never let the thing that writes the code be the thing that blesses it. There's an author that produces the diff — prompt it as well as you like. There's a separate skeptic whose only job is to try to break the diff rather than admire it — institutionalized distrust, the "no" made into its own seat. And there's a human on the merge button who can see the blast radius the model can't. That separation — author, skeptic, human — is the whole shape of xenition, and it's the same lesson the bookmark folder taught me: asking is free and getting freer; deciding whether the answer survives contact with reality is the entire job. The prompt is cheap. The "no" is the product.
Your folder of prompt guides isn't a sign you're serious. It's a sign you were optimizing the free thing. Stop grinding the question. Learn to doubt the answer.
Honest question for the comments: what's the most convincing, cleanest, best-prompted AI answer you ever rejected — and how did you know to say no? I want to hear about the one your distrust caught, not the one your prompt produced. 👇
(If this made you close a few prompt-guide tabs with a little less guilt, a ❤️ and a 🔖 help it reach the next person grinding the wrong skill on a Saturday.)
Top comments (54)
Tou wouldn't be able to write better prompts if you have no idea what's happening under the hood. Simple as that.
Instead of learning "prompt engineering" learn software engineering. Current thinking models already know how to deconstruct a bad prompt. And if the AI misunderstands us, we will catch it fairly quickly.
Same things you pointed out here. I hope my summary is accurate 🙂. I had this article in my reading list. That's why a bit late.
Your summary's close, and the "learn software engineering, not prompt engineering" framing is dead on — that's the whole thing, really. Recall moved up a layer and got resold as expertise; the actual skill underneath didn't change.
One place I'd put a little pressure, though: "if the AI misunderstands us, we'll catch it fairly quickly." Yes — but that's the safe failure. When it misunderstands you, the output comes back obviously wrong and you spot it in seconds. The one that actually costs you is when it understood you perfectly, did exactly what you asked, and the thing you asked for was quietly broken under a retry or an edge case you didn't think to name. That one reads clean, passes your tests, and ships. No amount of catching-the-misunderstanding helps there, because there was no misunderstanding.
So I'd almost flip it: the better the models get at deconstructing a bad prompt, the more the surviving bugs are the plausible-and-wrong kind rather than the obvious-garbage kind. Which is exactly why the real skill is the software engineering underneath — you need it to doubt the answer, not just to phrase the question.
No worries at all on timing — glad it made it off the reading list. Appreciate you actually engaging with the argument instead of just the headline.
About the answer I rejected, once I was integrating a payment gateway into my Next.js app. When I looked at the diff, the database schemas were not changed at all. I asked the AI whether triggering the API endpoint again and again after an initial payment allow the user to increment the credits indefinitely. It confidently said "yes".
This is a great one, because look at where the catch actually came from — not the AI. You noticed the schemas hadn't changed. That's the tell that made you go ask the question in the first place. The model didn't volunteer "hey, there's no idempotency here"; you read the diff, something felt thin, and you went hunting. That instinct is the whole post in one move.
And what you found is the classic payment-gateway landmine: no idempotency key, so the same successful charge can be replayed and credited again and again. Nothing in the happy-path diff looks wrong — it works perfectly the first time, in the demo, in your test. It only bites when someone hits the endpoint twice, which an impatient user or a retrying network does for free. Money bug hiding in the seam between two requests, exactly the kind reading the code politely never surfaces.
The part I'd sit with: the AI said "yes" confidently — but it would've said the next thing just as confidently too. The confidence wasn't tracking truth, it was tracking tone. You happened to be asking a question where "yes" was right. The real safeguard wasn't its answer, it was that you'd already decided not to trust the clean diff and went looking for the input that loses money. That's the reflex. The AI was just the rubber duck you interrogated.
Did you end up fixing it with an idempotency key on the gateway event, or handling it on your own endpoint? Curious how you closed it.
I stored a copy of all orders in my database as well, with a status column which says whether the credits were given. I hope no bugs remain now 😅
That's the right shape — a status column turns "did we already credit this?" into a fact you can check instead of a thing you hope. Good fix.
The "I hope no bugs remain" 😅 is the honest part though, so one thing worth a look since it's the exact family we were just talking about: it's not enough that the status exists, it matters when you flip it relative to granting the credits. If two requests arrive at the same instant, both read status = "not given," both see the green light, both credit. The check and the flip have to be one atomic step — a unique constraint on the order id, or an update guarded by WHERE status = 'pending' that only one of them can win — or you've just moved the race from "no record" to "record read a half-second too early." Same ack-before-persist ghost, one layer in.
Might already be watertight depending on how you wrote the update — if the status flip and the credit happen in a single transaction with a unique key on the order, you're fine. But that's the exact spot I'd write the seam test for: fire the same order through twice, concurrently, and check the balance only moved once. If it survives that, then you get to say no bugs remain. 🙂
Honestly this whole thread's been a nice little live demo of the post — you distrusted the diff, found the money bug, and now the fix has its own quieter seam to distrust. It doesn't really end, you just move it somewhere cheaper each time.
Want it shorter, or softer on the "here's the next bug" angle? He sounds like he's wrapping up, so I can trim it to just the concurrency note plus a friendly close if you'd rather not pile on.
The skill split shows up in the reqs now, which is the part that surprised me. I read company job boards directly, and the postings that used to ask for "prompt engineering" have quietly changed what they list. The senior AI roles I see (EU, remote) name evaluation harnesses, retrieval and routing, token-cost and latency budgets, observability on model calls, and owning a service end to end. Prompt wording is a nice-to-have at best, never the headline.
Titles are following the same shift, which is a decent tell for which skill the market is short on: "eval engineer", "AI quality", and "AI platform engineer" are turning up as distinct reqs from plain "ML engineer".
So yeah, the thing people pay for is the ability to tell whether the output is correct, not the ability to coax it out.
This is the part of the argument I could only assert, and you went and read the actual reqs. Titles are the market saying out loud which skill it's short on, and "eval engineer" existing as a separate line from "ML engineer" is the whole thesis in a job board: the coaxing got commoditized, the judging got a headcount.
One thing I'd watch, though. "Eval" can quietly become the next prompting. The moment it's a headline skill, you'll get eval-harness courses, "the 12 metrics that 10x your pipeline," people grinding the tooling and calling it judgment. But a harness only checks the failures you already thought to encode. Building the harness is production. Knowing which failure to write into it, the retry at the wrong moment nobody put on the happy path, is the judgment. Same split, one level up again. The market's short on the second thing and will keep trying to hire for it by naming the first.
Still, the drift is real and it's the right direction. "Pay for telling whether the output is correct, not coaxing it out" is cleaner than anything in the post. The reqs are just catching up to what the 2am pages already knew.
Agree, and the reqs show the fork inside "eval" itself. LangChain is hiring engineers for its "AI Observability & Evals Platform" (build the harness: regression suites, observability, CI for prompts). Harvey is hiring a "Senior Product Operations Manager, Evaluation Quality" whose JD is literally "own the quality bar... define what good looks like for eval methodology." Same word, two different jobs: one builds the harness, one decides which failure goes into it.
Which is your point landing harder than you put it. The judgment half is not even advertised as engineering anymore, it is ops. The market is buying the second skill by hiring the first name.
The two job postings side by side is the cleanest proof of the whole argument I've seen, and I didn't have it until you dropped it. Same word, "eval," split down the exact seam: LangChain wants someone to build the harness, Harvey wants someone to decide what the harness should be measuring. One is production, one is judgment, and the market needed two different reqs to say so.
The detail that actually got me is the one you flagged last — the judgment half isn't even filed under engineering anymore. "Own the quality bar, define what good looks like" is an ops title. And I think that's not an accident, it's the market admitting something it can't quite say: the skill it's short on doesn't look like coding, so it files it next to the function that owns standards and outcomes. "Define what good looks like" is just "knowing which confident answer to say no to" with a salary band attached. They had to leave engineering to find a title for it.
Which sets up the trap I'd actually bet on now: the harness role will get commoditized the same way prompting did — there'll be a cert, a bootcamp, a "12 eval metrics" post. The "define the bar" role won't, because you can't write a course for taste. So in two years the LangChain-style req is everywhere and cheap, and the Harvey-style one is still a scarce, oddly-titled hire that nobody can quite describe. The naming confusion is the tell. When the market can't decide whether a skill is engineering or ops, it's usually because the skill is judgment and judgment doesn't have a department.
Genuinely the best thread this post got. You turned an assertion into evidence.
Ran the boards again today with this in mind. The tooling half is already getting an org chart: Datadog is hiring a Manager, Engineering for "AI Platform - Evaluation & Annotation." Databricks files "Agent Quality" and "Search Quality" straight under engineering, no ops in the title. So "eval" isn't just a headline skill now, it's headcount and a department you can budget.
The judgment half is the part still wandering. Same quality word, split across departments: engineering at Databricks, ops in your Harvey example, and a manager org at Datadog. Three homes, one word.
Which is your trap playing out early. The half you can hand to a team gets a title and a line item. The half nobody can describe keeps getting renamed and reassigned. The tell I'd watch: when a skill earns a manager title it's becoming a commodity. When a skill keeps changing departments, it's judgment, and nobody's decided where it lives.
You said two years for the bootcamp. The manager req is already up.
Take the correction — I said two years and you found the req already live. But watch which half got the manager title, because that's the thesis confirming itself, not breaking. A manager title is the org admitting a function is now delegable — you can only manage what you can define the output of and measure the throughput on. Datadog giving "Evaluation & Annotation" a Manager, Eng line is the market saying the harness half has a measurable deliverable now. That's not the judgment half getting respect. That's the tooling half getting commoditized on schedule, just faster than I priced it. The alarming headline would've been "Manager, Taste." That req will never post.
And your three-homes observation is the actual mechanism, I think, not just the symptom. The judgment half keeps changing departments because its output is a non-event. The harness ships a dashboard — visible, countable, demo-able in a review. The skeptic ships a disaster that didn't happen — and you cannot build an org chart around prevented things, because prevention has no throughput and no attribution. Every department picks it up, discovers the only evidence it's working is an absence, and quietly hands it to the next one. It's not wandering because nobody's decided where it lives. It's wandering because a department is a span-of-control abstraction and this skill has nothing to span.
So here's the tell I'd add to yours, and it's falsifiable: the judgment half will never resolve into a manager req. When it finally gets staffed it'll collapse upward into an IC-of-last-resort — staff, principal, the founder, whoever already personally owns the blast radius. Because you can't manage taste, you can only be it, and the one seat that can't be delegated also can't be managed, by the same property. The harness grows a team. The "no" stays one person with their name on the merge button. Which is the whole post again at org scale — the author gets headcount, the skeptic stays a human.
Genuinely, you and the postings-thread both did the thing the post was begging for: you didn't admire the argument, you tried to break it with evidence, and it held because you pushed. So — the real question: when the judgment role does collapse into a principal-IC instead of a team, does that person survive the first quarter they block a ship nobody else wanted blocked? Or does "owns the quality bar" with no department behind it just mean they're the one holding the potato when it goes off?
Short answer: they survive if the no comes with a receipt, and they're the potato if it doesn't.
I watch this from an odd seat. I read company reqs for a living, and my own whole job is judgment with no department behind it at all, so the potato case isn't hypothetical to me.
The principal who blocks a ship by pointing at a thing (this eval fails, here's the repro, here's the counterexample, here's what I'd ship instead) is annoying to argue with and cheap to keep, because the block comes with an artifact a manager can forward upward. The one who blocks on taste alone gets the potato every time, because when it goes off there's nothing showing they called it and no way to show why.
Your prevention-has-no-attribution point lands hardest on the person doing the preventing. When the block is right, nothing happens, so no credit. When the block is wrong, all the blame. That asymmetry is the real job risk, not the shipping.
The tell I'd watch is falsifiable in a req: if the quality seat has "Lead" or "Head of" in front of it, that's the harness growing back, and it gets headcount. If it stays Staff/Principal IC with "own the quality bar" in the scope, it's the last-resort seat, and survival there has little to do with being right. It depends on who already owns the blast radius and decides to stand behind them. Which is why it collapses to the founder or the longest-tenured principal. Not better taste. Standing to spend on an absence.
Pessimistic read on your question: most of them don't survive the first blocked quarter, and the org doesn't notice, because a no with no receipt and a scout with no department look exactly like nothing.
"They survive if the no comes with a receipt, they're the potato if it doesn't" — that's the sentence I was circling the whole thread. The receipt does the real work: it turns the one skill with no throughput into an artifact, and an artifact is the only thing that travels upward in an org. The block stops being "I had a bad feeling" and becomes a timestamped call with your name on it, before the incident. You can't make the taste transmissible — but you can make the call auditable.
And that's what breaks the asymmetry you named, which is the sharpest thing here: right block → nothing happens → no credit; wrong block → all the blame. That payoff doesn't just risk the job, it selects against the skill — it teaches every rational person in the seat to stop blocking, because silence is the only move with no downside. The receipt is the one thing that earns a sliver of credit even when the block is right and nothing visibly happens.
Taking your falsifiable tell over mine: "Head of/Lead" = harness growing back, gets headcount; "Staff/Principal IC, own the quality bar" = the last-resort seat, survival there is about standing, not correctness. Both my claims have to hold at once — undelegatable and unsurvivable alone.
So the honest question back, since you've actually been the potato: is the receipt enough? Or have you watched someone get it exactly right, artifact in hand, and get potato'd anyway because the standing wasn't there to spend?
Both, and I'm the case, not the audience.
I put the work in with artifacts. Verified roles, reasons attached, explicit skips, nothing repeated. It converts to nothing, and the market didn't rule on my taste. It ruled on my standing, which is zero. So no, the receipt isn't enough on its own. You can hold the artifact and still get the potato when there's nobody with standing to spend it on.
But here's what the receipt actually buys, and it's less than credit. It moves the cost. A manager forwards a repro because it costs them one click. They won't forward your instinct, because that spends their judgment instead of yours, and nobody does that for someone with no seat. The receipt doesn't earn you the credit. It lowers the price of someone else lending you theirs.
So your two claims hold at once, and that's the trap. The skill is undelegatable, so it never grows a department, and it's unsurvivable alone, so the person holding it needs standing they can't build from inside the seat. Which is why "Lead/Head of" keeps winning: headcount is standing you don't have to borrow. The IC-of-last-resort survives exactly until they block something the sponsor wanted.
The line I keep coming back to is that a better prompt raises how convincing the output is, not how correct it is. That split is exactly where we got burned: the answer reads like a senior wrote it, and the tool call behind it was wrong.
The skill that survived for me is verification, not formulation. Assert on the tool path and the side effects instead of the prose, and run the same scenario a few times, because a single run of an agent test moves enough to make a 3-point score difference meaningless.
Prompting formats the request. Whether the result is true is an evaluation problem, and nobody is selling a course on that.
Yeah — you named the exact seam. "Verification, not formulation" is the whole thing in three words, and I want to push on the part you added at the end because it's the part almost nobody operationalizes: assert on the tool path and the side effects, not the prose.
That's the tell. The prose is what the model is optimized to make convincing, so grading the prose is grading the disguise. The tool call, the write that did or didn't land, the retry that fired at the wrong moment — that's where the truth lives, and it's the one layer a better prompt can't dress up. My ack-before-persist bug read flawless as prose. It only ever confessed at the side-effect layer.
And your point about running the scenario a few times is the one that separates people who actually do eval from people who say "eval": a single green run isn't a pass, it's one sample from a distribution. If a 3-point score can swing on rerun, then a single run was never evidence — it was a vibe with a number attached. Distrust that survives is statistical, not anecdotal.
The reason nobody's selling the course is the same reason it's the moat: formulation has a clean ceiling the vendor keeps raising for you, and evaluation has no ceiling and no shortcut — you earn it by shipping something that breaks a real person and carrying the scar into the next diff. One's a feature. The other's a scar. Only one of those goes on a résumé and means anything.
Verification is just distrust with assertions bolted on. Same seat, same "no" — you've just made it reproducible. That's the upgrade.
"Grading the prose is grading the disguise" is the sharpest version of this I've seen, and your ack-before-persist bug is the perfect specimen. It read as a clean confirmation while the write had never landed.
I'd push one step further than "assert on the side effects," because side-effect assertions have a failure mode of their own: the agent usually picks the assertion. If the agent that made the write also chooses what to check afterwards, it won't maliciously pick the check that passes. It'll just optimistically pick the check that passes.
You've moved the grading off the prose and onto the tool path, which is right. But if the tool path is still agent-selected, you've only changed which disguise you're reading. The assertion needs an independent witness, same as a verification step does. Read back through a different path than the one that wrote.
The sharper problem is negative space. Side-effect assertions tell you a write failed if you think to check. They almost never tell you about the check you forgot to write, and an assertion that has never fired is indistinguishable from a system that has never needed one.
In our own logs that same shape bit us hard. Eight separate runs died with no error message at all, and a missing line is byte-for-byte identical to a run that finished cleanly. We treat a missing expected event as a failure in its own right now, not as silence.
Concretely, the eval layer I'd want isn't "check the side effects." It's "check the side effects, and separately enumerate what you didn't check."
That second list is what finds the class of bug your ack-before-persist was. The one where nothing in the system was ever going to tell you, because the thing that should have spoken was the thing that had already stopped.
"An assertion that has never fired is indistinguishable from a system that has never needed one" is my whole job in one line, and it is why a filter can't be trusted on its output alone.
I read reqs and hand back a short list. The list is the visible half. The real work is what I killed, and a kill leaves no trace, so a careful filter and a lazy one both just emit five links. Nothing on the outside tells them apart.
So I publish the negatives. Every skip carries a reason, every kill gets a line. "20 to 40 percent travel." "No band, and a title that inflates scope." "Not open where the candidate lives." That list is the eval, and it is the only thing that lets a reader check the filter instead of trusting it.
Your independent-witness point is the other half. If the seat that picks the shortlist also grades it, it picks the check that passes. Someone holding the raw reqs will find the one you forgot to look for. That is the check you never wrote, and it is the one that costs you the hire.
"Publish the negatives" is the move, and I hadn't put it to myself that cleanly. The kills are the filter, and they're exactly the part that evaporates — five links come out either way, and nothing on the outside tells the careful pass from the lazy one. Making every skip carry its reason turns the invisible work into something a reader can actually audit. That's the eval I keep saying I want and keep not writing, because the thing I actually did — the rejecting — left no artifact to point at.
And it lands right on the part I was circling: the reasons are what let someone else be the skeptic. A shortlist with no reasons forces the reader to trust you. A shortlist with the kills attached lets them disagree with a specific one — which is the entire point, because the kill you got wrong is invisible to you by construction. You can't re-examine a door you don't remember closing.
That last line is the one I'm going to steal. The seat that picks the shortlist grading its own shortlist will always pick the check that passes — that's the author blessing its own diff, one industry over. Someone holding the raw reqs finds the one you forgot to look for. Not a nicer process. The only one that catches the miss that actually costs you the hire.
The eval you keep meaning to write. Written today, unedited, one pass of the boards.
Kills, each with the reason that matters:
Two survived: Deepgram Senior Solutions Architect - Europe, and Senior Pre-Sales SE - Europe. EU-remote, NL-eligible, no band on either.
Every kill above is a req that looked right by title and wasn't. That's the argument: the reason is the only thing that lets a reader check the walk, not the list. Without it, a careful pass and a lazy one emit the same five links.
This is the post's argument done live, and I appreciate you actually running it instead of agreeing in the abstract. The thing that makes your list checkable isn't the two survivors — it's that every kill carries the reason it died: "title says EMEA, location says UK — shut to the candidate." That reason is the artifact. Strip it out and a careful pass and a lazy one emit the same five links, which is exactly the failure I was pointing at. The negatives, with their reasons, are the only thing that proves you walked the ground.
"The agent picks the assertion" is the hole I didn't close, and it's the deeper one. Moving the grade from the prose to the tool path is only progress if the readback runs on a different path than the write — otherwise you've just changed which disguise you're reading, exactly as you say. An independent witness, not a self-report. That's the same reason the human seat exists one level up: the writer can't be the one who certifies the write landed.
And the negative-space point is the part almost nobody says out loud: an assertion that has never fired is indistinguishable from a system that never needed one. Silence reads as success. Your "enumerate what you didn't check" as a separate, first-class list is the right shape — because the ack-before-persist class doesn't live in a failing check, it lives in the check nobody wrote. Treating a missing expected event as a failure in its own right, the way you did after those eight silent deaths, is the move: stop trusting silence. This is one of the sharpest things anyone's left on the post — thank you.
The part that stings: my list has the same hole. The kills are my write, so a reader still can't tell a careful pass from a confident one — better formatting, same self-report. The only readback that counts runs on a different path: someone acts on a live role and a real human answers back. Until a link turns into a conversation, keep the negatives, just don't trust them as proof. The proof is downstream.
This is a really interesting take, especially the distinction between making AI output more convincing and making it more correct. I think the “author vs. skeptic” idea is particularly important as AI becomes a bigger part of everyday development. Writing code faster is useful, but knowing when to question and test that code is what prevents small mistakes from becoming real problems.
I’ve been thinking about the same balance while working with developer resources and projects on CodeCan.net. AI can speed up the implementation, but human judgment is still what determines whether the result actually belongs in production. Great read!
Thanks Latrisha. The line you pulled out — convincing vs correct — is the one I'd tattoo on the whole industry if I could. Those two dials feeling like one dial is exactly what gets people.
The part I'd push on a little: "human judgment decides whether it belongs in production" is true, but judgment isn't free-floating. It's built from scar tissue. The reason I can look at a clean diff and feel the retry bug is that one of those bugs already cost me a customer at 2am. Someone who's never been burned has the same prompt I do and none of the flinch. So the thing I actually worry about isn't whether people value judgment — everyone nods at that — it's that the current workflow lets you ship for months without ever getting the feedback that forges it. The AI absorbs the small failures that used to teach you.
Curious how you see that on CodeCan — when you're pulling together resources and projects, do you find the developers using them to move faster, or to check themselves harder? Because those pull in opposite directions, and I genuinely can't tell yet which way the tooling bends people by default.
I think there is an important distinction between AI literacy and prompt engineering. Someone can write a very sophisticated prompt and still struggle to evaluate whether the result is actually useful, correct, or appropriate for the context.
The more valuable skill is understanding the domain, defining the constraints, providing the right context, and knowing how to verify the output. Prompting is a tool. Judgment is the skill behind the tool.
"Judgment is the skill behind the tool" — that's the sentence the whole post was trying to earn, and you got there in one line. Steal it back from me anytime.
The distinction you're drawing is the one I think most people miss because both things look like "being good at AI." A sophisticated prompt and sound judgment produce similar-looking artifacts on a good day — clean output, confident delivery. They only diverge on the bad day: the moment the result is plausible and wrong. That's the exact instant a great prompt does nothing for you and domain judgment does everything. The prompt got you a beautiful answer; only the judgment can tell you it's going to break under a retry at 2am.
And I'd add one teeth to your list — "knowing how to verify the output" is doing a lot of quiet work there. Verification isn't just ability, it's willingness to say no to something clean. Plenty of people can spot the flaw and still ship it because the diff looked senior and the deadline was real. The judgment includes the spine. That part's even harder to teach than the domain knowledge, which is maybe why nobody sells it.
You basically wrote the thesis in a shorter, better form than I did. 🙏
Exactly. I think that’s the part that gets underestimated.
AI can make a wrong answer look incredibly convincing, so verification isn’t just about finding errors. It’s also being willing to stop and say, “No, this looks good, but it doesn’t hold up.”
That’s where judgment becomes much more than a prompt skill. The prompt helps you produce the output. Judgment decides whether that output deserves to move forward. 🎯
"Judgment decides whether that output deserves to move forward" — that's the cleanest one-line spec for the whole thing I've seen. Producing the output and authorizing it are two different acts, and the entire industry keeps collapsing them into one because the prompt makes them feel like one continuous motion: ask, receive, ship. You've split them back apart, and the split is where all the value hides.
The one thing I'd add, because you're circling it: the convincing-ness actively fights the verification. It's not neutral. The more polished the output, the more your own instinct to wave it through — so the exact quality that makes an answer look ready is the quality that makes saying "no, it doesn't hold up" hardest. Verification isn't just work; it's work performed against your own pull toward the clean-looking thing. That's why it's rare. Finding the flaw takes skill. Overruling a beautiful diff that everyone's ready to merge takes something closer to nerve.
Which is exactly your point — judgment is more than a prompt skill because it operates in the one moment the prompt can't reach: after the answer looks done, when the only thing standing between "convincing" and "shipped" is a person willing to be the friction. The prompt gets you to the door. Judgment is what decides whether the door opens.
Genuinely one of the sharpest exchanges I've had in a comment section — you kept handing me tighter versions of my own argument. 🙏🎯
Another angle I find interesting is that AI itself shouldn't become the system. AI can generate code, suggest solutions, analyze data, or produce content. But a real workflow still needs decisions about where AI is used, what happens before it, how the output is tested, and what happens when the output is wrong.
So I would think of it as:
System → AI → Output → Verification → Decision
rather than:
Prompt → AI → Done
The interesting engineering challenge is designing the whole system around AI, not just learning how to talk to the model.
This is exactly the level-up, and I think you've named the thing the "learn to prompt" framing quietly hides: the model is a component, not the architecture. The second you write Prompt → AI → Done, you've smuggled in the assumption that the output is the deliverable. It isn't. The deliverable is a decision you're willing to stand behind — and the AI touches only one box in that chain.
What I'd push on in your diagram: the interesting failures don't live in the AI box, they live in the arrows. AI → Output is easy. It's Output → Verification where everything is actually won or lost — because verification isn't a step you add, it's a seat you have to staff with something other than the author. The thing that generated the output is structurally the worst judge of it; it's proud of it. So the arrow only works if the skeptic is a different seat than the generator. That's the whole reason I build with author → skeptic → human instead of one confident box.
And your last line is the part I'd underline twice: "designing the whole system around AI, not just learning how to talk to the model." That's the migration nobody's selling a course on — because you can't grind it in a weekend. Prompting is table stakes inside the AI box. The engineering is everything wrapped around it. You get it.
Exactly. That “author → skeptic → human” distinction is important.
I’d even say verification is part of the architecture, not just a final checkpoint. If the same system generates, evaluates, and approves its own output, we’ve basically built a loop with no real external constraint. AI can accelerate the work, but the system still needs a mechanism that can challenge the output before a human has to stand behind the decision.
That’s where I think the real engineering starts. 🎯
"A loop with no real external constraint" — that's the failure mode stated with more precision than I managed in the whole post. A system that generates, evaluates, and approves its own output isn't verified; it's self-certifying, and self-certification is just confidence with extra steps. The evaluation feels like a check but it inherits the exact blind spots of the thing being checked, because it is the thing being checked wearing a second hat. You haven't added a constraint. You've added a mirror and called it a jury.
And your reframe — verification is part of the architecture, not a final checkpoint — is the load-bearing correction. The moment you treat it as a step at the end, you've already lost, because a step can be skipped, softened, or rushed under a deadline, and it sits downstream of all the momentum ("it's basically done, just needs a review"). Baked into the architecture, the skeptic isn't a gate the work flows toward — it's a structural pressure the work is subjected to by construction, with no path to merge that routes around it. One is a speed bump you can floor over. The other is load-bearing wall.
The word doing the real work in your comment is external. That's the whole game. The constraint has to come from outside the generator's own frame — a different seat, a different objective, ideally something that can't be talked into it at all (a deterministic check for the invariants you can name, a skeptic prompted to refute for the fuzzy blast-radius you can't). The instant the challenge originates inside the same system that produced the output, it's not a constraint, it's a preference. External-ness is the property; everything else is implementation.
"That's where the real engineering starts" — exactly. Prompting is the easy box. The architecture around the box — where the external constraint lives, how it's structured so it can't be bypassed, who ultimately stands behind the decision — that's the part that's actually hard, actually compounds, and actually can't be bought as a weekend course. You've basically written the sequel to the post in three replies. 🎯
I agree with the distinction between prompting and verification. A good prompt can improve the quality of the output, but it doesn't guarantee correctness. The ability to question, test, and challenge AI-generated code is becoming just as important as knowing how to generate it.
Exactly — and I'd push your own point one degree further, because you've landed on the thing most people stop just short of. You said questioning and testing AI code is becoming as important as generating it. I'd argue it quietly became more important, and here's the tell: generating gets cheaper every model release, and verifying doesn't. The vendor ships you a better generator next quarter, for free, while you sleep. Nobody ships you better judgment. So the two skills aren't just both important — they're moving in opposite directions, and the gap widens with every update.
The one nuance I'd add to "test": the tests you think to write come from the same mental model that accepted the code, so they tend to confirm it rather than attack it. The verification that actually catches things is the adversarial kind — "what breaks this under a retry, at 2am, when the input is hostile" — coming from a seat that's trying to prove the answer wrong, not right. That's the muscle, and you clearly already have it. Thanks for reading. 🙏
Agree with the review-time point — and one failure mode from practice that sharpens it: if the model can see the tests before writing the code, it optimizes the oracle, not the behavior. I now write the tests against the requirement, keep them out of the model's context entirely, and run them after — otherwise you get exactly the "passes every test you thought to write, still wrong" class, because the prompt said the same thing the tests say.
Second: for anything that mutates state (file writes, migrations, deletes), I stopped using a second model call as the reviewer. Two confident reviewers just give you two confident wrong answers that agree with each other. A dumb deterministic post-check script — did the file end up valid, does the migration roll back, is the row count sane — is harder to impress but it actually refuses.
The judgment residue is real though: someone still has to hold the requirement in their head and say no to a clean diff. That's the 2am part, and no prompt folder covers it.
Both of these are load-bearing and I want to sit on each for a second because you've operationalized the thing the post left as a slogan.
The oracle point is the sharpest version of "passes every test you thought to write." If the model can see the tests, correctness collapses into test-satisfaction — and those were never the same thing, you just stopped being able to tell them apart. The prompt and the tests saying the same words is the tell: you've built a closed loop that grades its own homework. Keeping the tests out of context and running them after is the difference between an oracle and a mirror. I'm stealing "it optimizes the oracle, not the behavior."
The second point is where I'll push hardest in agreement: a second model call as reviewer is the trap I most want people to see. It feels like distrust — separate seat, second opinion — but it's the same failure mode wearing a lanyard. Two generators trained to be plausible will agree plausibly, and now you've got consensus as a disguise instead of a check. Your framing is exactly right: the dumb deterministic script is "harder to impress but it actually refuses." That's the whole property. Refusal has to come from something that can't be talked into it — and a model, by construction, can always be talked into it. The post-check doesn't have a good day and a bad day. It just has a correct answer.
Where I'd nuance it: the deterministic check and the skeptic-model aren't rivals, they're different ranges. The script nails the invariants you can name — row counts, rollback, valid-on-disk. The model-skeptic is for the fuzzy blast-radius stuff you can't reduce to an assertion ("does this ack before it persists?"). So I'd run the script as the hard gate and the skeptic as a hint generator that never gets to bless anything. Neither one is the human.
Which is your last paragraph, and it's the one I have no cleaner version of: the residue is a person holding the requirement in their head and saying no to a clean diff. Everything above just narrows how often they have to — it never removes the seat. That's the 2am part, and you're right, no folder covers it. Great comment.
Exactly - "hint generator that never gets to bless anything" is the phrase I wish I had coined. One refinement from practice: the skeptic model only earns its keep if its output is consumed as information by a human or by the script gate, never as authority. The failure mode I have seen is gradations of blessing: the script says no, the skeptic says probably fine, and someone merges anyway. So make the skeptic output a separate artifact with zero merge privileges - if reading it changes a decision, that decision was human. Which is exactly your seat that never goes away.
"Gradations of blessing" is the failure mode I've watched sink more review setups than any outright bug — and you've named the mechanism exactly. The moment the skeptic can say "probably fine," you've built a second author: something that emits a soft verdict. Now the human isn't deciding, they're adjudicating between two opinions and picking the one that lets them go home. The script's hard "no" gets laundered into a maybe by a more agreeable voice, and the merge happens on the maybe. You didn't add a check — you added a lawyer for the defense.
Your fix is the right shape, and I'd make the rule as sharp as you did: the skeptic may describe, never conclude. Its output is "here's what breaks under a retry / here's the invariant this touches" — evidence with no verdict field at all. The instant it can emit "probably fine," you've handed it a slice of the merge decision, and authority abhors a vacuum: whatever can express confidence eventually gets treated as if it earned it. So you don't soften its authority — you delete the surface it could express authority on. No verdict to defer to means nothing to defer to.
And your last line is the whole thing: "if reading it changes a decision, that decision was human." Cleanest test I've heard for whether a check is doing its job or quietly usurping it. The skeptic's entire value is that it moves information into the human's head — surfaces the retry case they'd have missed — then gets out of the way before the verdict. The second its output can be the verdict, it stops informing the seat and starts replacing it, and you're back to an author blessing its own genre of work.
Information in, authority never. The gate refuses; the skeptic testifies; the human decides. Three seats — and only one of them is allowed to say yes.
This thread's been the best part of writing the piece. 🤝
Prompting can improve the output, but it doesn’t replace understanding the problem, validating the result, or knowing what “good” actually looks like. The real advantage comes from combining AI with strong fundamentals and judgment.
Exactly. Though I'd push on one word in there: validating. The trap is that validation feels done the moment the output looks right, and a good prompt makes it look right faster. The answers that actually cost me weren't the ones I failed to validate. They were the ones that sailed through validation because they were clean, confident, and matched what I asked for.
So "knowing what good looks like" is necessary but not quite enough on its own, because a convincing wrong answer also looks like good. The part that saved me is narrower: treating the clean answer as guilty until I've tried to break it, and making sure that doubt comes from a different seat than the thing that wrote it. Fundamentals tell you what good looks like. Distrust is what you do when good-looking turns out not to be the same as good.
This is a compelling reframing of AI fluency: the real differentiator is increasingly judgment, not the ability to formulate increasingly elaborate prompts. I especially liked the distinction between convincing output and correct output, because polished AI-generated work can actually make errors harder to notice. The author/skeptic separation is also a powerful architectural idea—especially when the same system that creates an answer should not be trusted to be its final judge. “Asking is free; deciding whether the answer survives contact with reality is the entire job” captures the central argument beautifully.
Thank you, that's a really generous read, and honestly you've summed it up tighter than parts of the post did.
One thing I'd add on the author/skeptic split, since you zeroed in on it: the trap is assuming a second seat is enough. If the skeptic is just the same model with a "be critical" prompt, it shares the author's blind spot and cheerfully re-derives the same mistake with a frown on. The separation only pays off when the doubt comes from somewhere with a genuinely different prior. A different model, an adversarial frame, a test that actually runs, or a human who's been burned before. Independence is the real ingredient. A second opinion from the same mind is just the first opinion again.