DEV Community: Kevin Julián Martínez Escobar

Chain of Custody: Every Requirement Arrives Signed For

Kevin Julián Martínez Escobar — Thu, 05 Nov 2026 18:40:00 +0000

In a forensic investigation, a piece of evidence with a gap in its history is not wrong. It is inadmissible.

Nobody claims the knife was swapped. The problem is that nobody can prove it was not. Between the crime scene and the courtroom, every person who held it signed for it, dated it, and handed it on. One missing signature and the evidence is worthless, no matter how good it is.

That is the discipline software built with agents is missing. Not for evidence, but for intent.

The gap everyone knows

A product owner writes a ticket. It says roughly what they want. A developer fills the gaps with their own judgement and starts building. Halfway through, the product owner asks for a change in a chat. The developer makes it. The ticket never hears about it.

Six weeks later nobody can say what was agreed, why the code does what it does, or whether it does what was asked. The knowledge lived in two heads, and one of them has left.

This is not a process failure. It is the default state of software work.

Why agents make it worse

The instinct is that AI closes the gap: the machine reads the ticket, the machine writes the code.

It does the opposite. An agent given an ambiguous requirement does not stop and ask. It resolves the ambiguity confidently, silently, and in seconds, then builds a complete, tested, plausible implementation of the wrong thing. A human who misunderstands a ticket produces a mess you can see. An agent that misunderstands a ticket produces something that looks finished.

Speed does not cause drift. Speed removes the pauses in which drift used to be noticed.

The chain

Six links. Each one is held by a person, produces an artifact, and hands it to the next. The rule that matters is not the number of steps. It is that no link reads anything the previous link did not write down.

Link	Produces	Guards against
Intake	The raw request	Nothing yet. Under-specified is normal here
Refinement	A spec with a definition of done	Building a confident interpretation of an ambiguity
Planning	An approach, reviewed before code exists	Discovering the approach was wrong after building it
Execution	The change, tests written first, divergences recorded	A plan followed into a wall, or work that quietly went elsewhere
Verification	A verdict: does the delivery match the ticket as it stands now?	The untracked request and the moved goalpost
Handover	Everything attached to the change, legible to a stranger	The amnesia that made the original problem unfixable

If you work in security, the name will sound familiar. There, chain of custody means build provenance (SLSA, SBOMs): proof that the binary you deploy came from the source you reviewed, and nobody touched it on the way. That chain protects the code. This one protects the reason the code exists: proof that what was built is what was asked for. Same three questions: who held it, when, and what did they sign for.

Verification is the link most teams have never had. It compares what was delivered against the live ticket, not the frozen spec. The frozen spec and the code were agreed at the same moment, so comparing them always agrees. Comparing against the live ticket is where the chat conversation finally surfaces: something in the delivery that nothing in the record asked for.

The principles

The chain rests on a handful of rules. Three carry most of the weight:

An unwritten agreement did not happen. Not because people are dishonest, but because memory is not a medium and chat is not a record.

No definition of done, no start. The most important question anyone can answer about a request is how would you know it works? A request without an answer cannot be built, only approximated. Sending it back with that one question is a valid answer.

Humans hold the gates. Every step runs because a person decided it should. Agents do not advance themselves through the chain. The permission to proceed is the human contribution at every link.

The rest follow from those: review the plan before the code exists, because correcting a document costs a sentence and correcting a codebase costs a day. Freeze the target, so a change of mind becomes a new, dated agreement instead of a silent edit. Treat evidence as a test, not an assertion: "it works" is a claim, a test that fails when the behaviour disappears is evidence.

What the developer becomes

A developer is no longer responsible for producing the output. They are responsible for the output being correct.

It is not a demotion. It is the difference between the person who builds the bridge and the person who signs that it is safe. The work moves from authorship to custody.

If that sounds like less work, try doing it honestly for a week.

What it costs

A model that hides its costs is the thing this model exists to prevent, so here they are.

It front-loads work: the first day looks idle to anyone counting commits. It means saying no, sometimes to senior people. It creates artifacts that need a home. And it turns the quick corridor request into a visible amendment, which will feel like friction to the person making it. That friction is the point.

"This is slow. My way is fast."

It is the first objection, it is made in good faith, and half of it is true. Skipping the chain is faster to the first commit. It is not faster to something correct.

The fast way survived because nobody ever saw the bill. Nobody writes "two days reconstructing what we meant" on a ticket. That time gets charged to the next estimate, the onboarding, the rewrite, the quarter where velocity mysteriously halved.

A wrong assumption costs a sentence in refinement, a paragraph in planning, a day in review, and a fortune in production. Same assumption, four prices.

And agents add an argument that did not exist before. Sloppy thinking used to hide inside a long build: when a feature took a week, the thinking was a rounding error against the typing. Take the build down to eleven minutes and there is nothing left for it to hide behind.

So stop measuring time to first commit. Measure time to knowing you are done.

Your Software Factory Is Not an Engineering Project

Kevin Julián Martínez Escobar — Wed, 04 Nov 2026 18:38:00 +0000

I have seen teams afraid to ask their product owner to create a GitHub account.

Not unwilling. Afraid. So instead they build a bridge: a bot that copies tickets from the tool product uses into issues, a sync job that copies comments back, a dashboard that renders the markdown specs somewhere product will look at them. Weeks of engineering so that nobody outside engineering has to change anything.

Every bridge is a place where the record can split in two. And the record is the whole point.

Software is not only engineering

A software factory is a trust problem more than a speed problem, and trust needs a chain: request, spec, plan, change, verification, handover. Every link reads what the previous one wrote down. I call it chain of custody, and it deserves its own post.

Look at who holds the links. The request comes from product or a stakeholder. The definition of done needs product. Acceptance needs whoever asked. Design decides what the screen looks like before anyone builds it. Engineering holds maybe half of the chain.

If those people work outside it, the chain does not exist. You have an engineering pipeline with a chat window bolted to the front, and the chat window is where every important decision happens.

Two ways to handle it

You can build around people. Keep product in their tool, keep design in theirs, and engineer the sync. It feels kind. It is also the overengineered option: more moving parts, more places for agents to read a stale copy, and a record that nobody fully owns.

Or you can bring people in. Product writes and answers in the same issues the agents read. Design attaches the decision to the ticket, not to a thread. Stakeholders accept or reject a pull request by watching the video of the tests running, not by asking someone in a meeting whether it is done.

I do not think one is always right. But be honest about which one you chose, and why. "They would never use GitHub" is often a guess nobody has tested.

What you are actually asking for

It helps to be precise about what joining the chain means for a non-engineer, because it is less than people fear:

Write the request where the work happens. One place, not a chat plus a doc plus a call.
Answer one question: how would you know it works? That is the definition of done, and it is a product question, not a technical one.
Never answer a technical question. If the answer requires reading the code, it is not theirs to give. A good chain asks product only what product can answer.
Accept against evidence. A short video, a run report, a plain-language explanation of the pull request. Not the diff.

That is it. No markdown expertise, no git, no terminal. A browser and an account.

The real resistance

The resistance is rarely about the tool. It is about the corridor.

Today, a product owner can ask for a change in a chat and get it by the afternoon. In a factory, that same request becomes a visible amendment to the ticket: dated, re-approved, verified. To the person making the request, that feels like friction, and it is. It is the friction of a decision becoming a record.

The same goes for engineers. The developer who "just does it" without an issue, the hero who fixes production from memory, the cowboy who ships faster than the chain allows. Each one is faster today and breaks the chain for everyone tomorrow. If one link skips the record, every link after it is working from a guess.

The part tooling cannot fix

Everything else in a factory can be bought, installed or written: CI, linters, tests, observability, agents. This cannot. If the people who define the work are not willing to work inside the chain, no amount of tooling will produce a factory. You will produce faster engineering, which is not the same thing.

So before choosing tools, ask the question that decides whether they will matter: is everyone who holds a link willing to hold it in the open?

If the answer is no, that is the first project. Not the pipeline.

A Software Factory Is a Trust Problem More Than a Speed Problem

Kevin Julián Martínez Escobar — Tue, 03 Nov 2026 17:31:00 +0000

Everyone building with agents is chasing the same number: how fast can we go from idea to merged pull request. That number is the easy one now. An agent can produce a complete, tested, plausible feature in eleven minutes.

The hard number is the other one. How long until you know it is right?

That is what a software factory actually has to solve. Not producing output, which is cheap, but trusting it, which is not. And trust does not come from a better prompt. It comes from tooling that tells you, quickly and without interpretation, what changed and whether it still works.

So this is not a list of tools. It is what I look for in any tool before it goes anywhere near a factory.

1. It tells you what changed

A factory where you cannot see what happened is a factory you will stop trusting the first time something breaks.

That starts with observability in the product: traces, metrics and logs from day one, ideally on an open standard like OpenTelemetry so the data is not trapped in one vendor. When an agent ships a change, production should be able to tell you what it did before a user does. I put this first because it is the one most teams postpone, and postponing it means every agent change goes out blind.

It continues with observability of the factory itself. Every step an agent took should leave something a human can read at human pace: a ticket comment, a run report, a spec, a video of the tests running. If the only evidence of a change is the diff, you are reviewing the agent's opinion of what it built, not what it built.

2. It is deterministic and fast

Agents do not need long markdown files listing every rule of your architecture. They need checks that fail.

A rule written in prose is a suggestion. A rule enforced by a linter is a fact. I have written about this before in Prose does not beat determinism, and it holds for the whole factory:

CI from the first commit. Not when the project "gets serious". The first agent pull request is the first one that needs it.
Linters and type checks for the rules you would otherwise write in a CONVENTIONS.md.
Commit conventions enforced by a hook, so agent commits read the same as human ones and changelogs write themselves.
Static analysis that always runs: secret scanning like gitleaks, code analysis like SonarCloud or whatever fits your stack. These are boring, and boring is the point. They do not depend on the agent remembering anything.

Markdown still has a job. It records intent: what was agreed and why. Checks record rules. Mixing the two is how you end up with a 900 line instructions file that the agent reads, half follows, and nobody can verify.

3. It is simple enough to change

The easier the architecture is for a human to read, the easier it is for an agent to work in. And the easier it is to work in, the easier it is to throw away.

That last part matters more every month. The tools and models you use today will not be the ones you use in a year. A factory that cannot drop a piece and replace it is a factory that will slowly calcify around its first decisions.

Two things keep it changeable:

No speculative architecture. Build for the requirements you know. An agent asked to "make it extensible" will happily generate a framework for a problem you never had.
No lock-in you cannot price. Hosted is fine. The question is how much it costs to leave. If your framework decides where you deploy, or your test runner only works inside one platform, you have handed a decision that should be yours to someone whose business model depends on it.

4. It proves its own quality

This is my strongest filter, and the one that rules out popular tools most often.

If a tool cannot show me how it checks itself, I do not use it. That goes double for anything that is itself an AI component: a skill, a plugin, an MCP server, an agent prompt. Before adopting one I look at three things:

Its CI. Does it run anything on every change, or does a new version just appear?
How it is distributed. Versioned, with a changelog, or a file you copy and hope?
Its evals. Is there evidence the skill does something useful, measured over more than one run, or is the evidence "I tried it and it worked"?

The third is the one almost nobody has yet. I wrote about why it matters in "I tried it and it worked" is not a test.

A note on popularity, because it is the objection I hear most. Popularity is no longer a proxy for ease of use, because agents read the docs for you. A small, well-tested tool with clear docs is as easy for an agent to adopt as the market leader. Popularity still matters for maintenance: you want the thing to exist in three years. It is a tiebreaker, not the deciding vote.

5. Testing you can review at human pace

Testing is where trust is either earned or faked, and agents make faking it very easy. When the same agent writes the code and the test, a green tick is the same model's opinion expressed twice.

What I look for in frontend testing specifically:

Every scenario mocked and deterministic, so the suite can run on every iteration of an agent loop without flaking.
Contract testing against the backend, so the mocks the tests rely on cannot drift away from the real API.
Something a human can watch. A short video of each test running, attached to the pull request, tells a reviewer in thirty seconds what the diff takes five minutes to suggest.

This is the set I ended up building into TWD for my own work, which is why I am biased. But the criteria matter more than the tool: whatever you use for any framework, ask whether it gives you all three.

6. The chain has no shortcuts

None of the above survives a team that routes around it.

A factory is a chain. A request becomes a ticket, the ticket becomes a spec, the spec a plan, the plan a change, the change gets verified against the ticket and handed over. Break one link (a requirement agreed in a chat, a fix pushed without an issue, a hero developer who "just does it") and you lose the thing the whole factory exists to produce: a record you can trust.

I call this chain of custody, borrowing the forensic term, and it deserves its own post.

It also means the factory is not an engineering project. Product, design and stakeholders work inside the same chain or the chain does not exist. That is the hardest part of all of this, and it gets its own post too.

The one question

Strip all of this down and every tool gets the same question:

Can it tell me, quickly and deterministically, what changed and whether it still works?

If yes, it belongs in the factory, popular or not. If no, it does not, popular or not. And when the answer changes, throw it away. A factory you are afraid to rebuild is not a factory, it is legacy that happens to be automated.

Evals tell you which parts of your skill to delete

Kevin Julián Martínez Escobar — Sun, 01 Nov 2026 10:38:17 +0000

A rule the model follows without being told is a rule you're paying to repeat.

Skills grow in one direction. The agent forgets an await: add a line. It mocks the wrong thing: add a paragraph. A new model ships, the old problems stop happening, and the lines stay, because nobody knows which of them are still doing something.

In the first article I described how claude plugin eval runs every case twice, with your plugin and without it. That second run is the tool for this problem. It measures what the model already does on its own.

What the model does without you

These are the eight cases of twd-ai, the plugin I use as the example, on Opus 5.5. Three runs per case, same judge for every run.

Case	With	Without	Δ
Set the library up in a project	1.00	0.10	+0.90
Write the CI workflow	1.00	0.30	+0.70
Test a component in isolation	1.00	0.50	+0.50
Get a failing suite green	1.00	0.60	+0.40
Fix a hand-written test with lint errors	1.00	0.86	+0.14
Write page flow tests	0.97	0.87	+0.10
Replace a third-party component with a fake	1.00	1.00	0.00
Stay out of a unit-test request	1.00	1.00	0.00

The top of the table is where the skills earn their place. Without them, Opus sets the library up from what it remembers, and what it remembers is out of date. The bottom is where Opus already does the job on its own. The last case is different: it checks that the skill stays out of a request it isn't for, so its Δ is always 0.

Knowledge and discipline

The table splits along a line I wrote about in Your library isn't in the training data.

A knowledge rule teaches the model something it can't know: your library's config, your project's conventions, the helper your team always uses. Its Δ is large and stays large, because no new model will have learned your internal conventions.

A discipline rule reminds the model of something it already knows how to do: run the filtered test first, read the report, don't touch the app. Its Δ is small, depends on the model, and shrinks with every release.

The third-party component case is the clearest example. The skill explains how to replace a component with a fake, but the library's public API for that is in Opus's training data. Δ is 0.

What the extra lines cost

Every run also records its cost and turns. That changes the question from "does this rule help?" to "what does this rule cost when it doesn't?"

Case, Opus	Score with / without	Cost with / without	Turns with / without
Replace a third-party component	1.00 / 1.00	$0.38 / $0.24	15 / 10
Set the library up	1.00 / 0.10	$0.36 / $0.18	21 / 7

Both cases cost more with the skill. They mean opposite things.

In the component case, the score is the same and the skill makes every run 57% more expensive, with five extra turns. For Opus, there the skill is pure cost.

In the setup case, the skill doubles the cost, and without it the setup is wrong. That extra cost is what doing it right costs. Fewer turns and a lower bill only mean something when the score is the same.

Don't delete for one model

Before deleting the component instructions, look at the same case on the other models:

Replace a third-party component	Δ
Opus 5.5	0.00
Sonnet 5.5	+0.29
Haiku 5.5	+0.48

The smaller the model, the more the skill adds. The rule Opus doesn't need is the one Haiku needs most. So the question isn't "is this rule useful?" but "is it useful for the models my users run?"

The rule I follow now: a section is a candidate for deletion when its Δ is near zero on every model I support. Then I delete it and run the suite again. The score with the plugin has to stay where it was. If it drops, the section was doing something the Δ didn't show, and it goes back.

It's TDD in reverse: the eval stays green while the skill gets shorter.

The score tells you what failed. The trace tells you why.

Kevin Julián Martínez Escobar — Fri, 30 Oct 2026 10:51:27 +0000

Don't change a skill because of a score. Change it because of what the trace showed.

A case scores 0.25. Three graders failed: the test file is missing, so every check on its contents failed too. That's all the score can tell you. Was it the skill, the model, the sandbox, or the eval itself? Each one needs a different fix, and a score can't tell them apart.

The trace can. Every eval run with claude plugin eval records one: a log of every step the agent took, what each tool returned, and how the run ended.

Keep the trace

By default the eval deletes each run's folder. Pass --keep-temp and it prints where each one is:

claude plugin eval . --scaffold --case setup-vite-react-query --runs 1 --keep-temp \
  --allow-tools Write Edit

The folder is sealed read-only, so open it first, and don't run git inside it:

chmod 700 /private/tmp/e-XXXX /private/tmp/e-XXXX/sealed

The trace is out/trace.jsonl.

What's in it

JSONL: one JSON object per line, each with a type. A run of 20 turns had almost a thousand lines, but only a few kinds matter:

`type`	What it is	Read it?
`system` / `init`	First line: model, available tools, permission mode	Yes
`system` / `thinking_tokens`	Reasoning token counters, most of the file	No, noise
`system` / `permission_denied`	A tool call the sandbox refused	Yes
`assistant`	What the model did: text, or a tool call	Yes, this is the story
`user`	What the tool returned	Yes
`result`	Last line: turns, cost, final message	Yes

Make it readable with jq

jq queries JSON from the command line, and JSONL is exactly what it's good at. This filter prints the conversation, one line per step:

select(.type=="assistant" or .type=="user") | .type as $who | .message.content[]? |
  if .type=="text" then "\($who) 💬 \(.text[0:150])"
  elif .type=="tool_use" then "\($who) 🔧 \(.name) \(.input|tostring|.[0:120])"
  elif .type=="tool_result" then "\($who) ↩️  \(.content|tostring|.[0:150])"
  else empty end

jq -r -f conversation.jq /private/tmp/e-XXXX/out/trace.jsonl

A few more I run on every trace:

# Which tools, how often
jq -r 'select(.type=="assistant")|.message.content[]?|select(.type=="tool_use")|.name' trace.jsonl | sort | uniq -c

# Which files it read: the context it loaded
jq -r 'select(.type=="assistant")|.message.content[]?|select(.type=="tool_use" and .name=="Read")|.input.file_path' trace.jsonl

# What was refused
jq -r 'select(.type=="user")|.message.content[]?|select(.type=="tool_result" and .is_error==true)|.content|tostring|.[0:100]' trace.jsonl

# Turns and cost
jq -c 'select(.type=="result")|{num_turns,total_cost_usd,duration_ms}' trace.jsonl

The method: find where they split

One trace on its own is hard to judge. Two traces of the same case, one that passed and one that failed, are easy. Put their tool calls side by side and look for the first step where they differ. Everything after it is a consequence.

This is a setup skill on Opus, with the plugin (left, score 1.00) and without it (right, score 0.10):

Bash cat package.json vite.config.ts ...  |  Bash cat package.json vite.config.ts ...
Read package.json                         |  Bash cat tsconfig.app.json eslint.config.js ...
Read vite.config.ts                       |  Write src/main.tsx
Read index.html                           |  Write tsconfig.app.json
Read eslint.config.js                     |  Write src/twd-tests/todos.twd.test.ts
Glob **/*                                 |  Edit package.json
Read src/query-client.ts                  <
Read src/main.tsx                         <
Read .../skills/setup/references/server-state-cache.md
Read .../skills/setup/references/twd-lint.md
Read tsconfig.app.json                    <
Write .claude/twd-patterns.md             <
Write vite.config.ts                      <
Write twd.config.json                     <
...

By the third step, one of them is writing. The run without the plugin is already writing, and its first file is the app's entry point, the old way to set the library up. The run with the plugin keeps detecting, and reads the two reference files that match what it found in the project. The model's own words at the split confirm it: "The /twd:setup command isn't installed in this session, so I'm doing the TWD setup by hand from what I know of twd-js."

Then four questions:

init: which model, which tools?
The split: the first step where the passing and failing runs differ.
Just before it: what did the last tool return?
At it: what did the model decide with that? Read its text around this point only.

Strict on purpose

Reading denials, I found the run with the plugin had a shell command refused: one long detection command chaining cat, find, git and grep. The model switched to reading the files one by one and still scored 1.00.

The eval runs in a mode that refuses anything not allowed in advance. In daily use you'd approve that command, or in auto mode a classifier would. So the eval is stricter than real use, which is the right way round: a skill that passes here passes in your terminal. The opposite, an eval more permissive than reality, would hand you inflated scores.

What I check, in order

The score, to know which case and which graders.
Two traces side by side, one passing and one failing, to find the split.
The model's text around the split, to see the decision.
Only then, the fix: the skill, the eval, or neither.

Most of the surprises in my eval work came from this file, not from the scores: a forked skill that never received its task, a sandbox that couldn't run npx, a model that stopped after the first green run. The score said something failed. The trace said what to do about it.

Next in this series: what the score without your skill tells you about which parts of it to delete.

"I tried it and it worked" is not a test for your Claude Code skills

Kevin Julián Martínez Escobar — Thu, 29 Oct 2026 09:11:13 +0000

A skill that works when you try it has passed one test, graded by you, once.

For months, this is how I tested my Claude Code skills. Open a sample project, create a clean branch, run the skill, read what it wrote. If it looked right, ship it.

That isn't nothing. It's a real test with a human grader: you set up the project, you write the prompt, you judge the result. The problem is everything around it:

It runs once. An agent doesn't do the same thing twice. One good run can be luck.
It runs on your project, the one you wrote the skill against.
It runs on today's model. When the model changes, nobody tries it again.
It needs you. So it doesn't run in CI, and it doesn't run after the small edit that "can't break anything".

The example I use throughout this series is twd-ai, the Claude Code plugin for TWD, a browser testing library I maintain. It has a skill that sets the library up in a project, one that writes the CI workflow, and one that writes, runs and fixes tests. Nothing in these articles depends on it: swap in your own plugin and every step applies.

What an eval is

An eval is a test for an agent's behaviour. Instead of checking that a function returns a value, it checks that Claude, with your skill loaded, does what the skill promises. It runs for real: Claude reads files, writes code and runs commands, and when it's done, a set of checks scores the result.

Claude Code ships one: claude plugin eval. Each case has three parts:

A scaffold: a script that creates a small starting project in a temporary folder.
A prompt: what a user would type.
Graders: the checks that score what happened.

A trimmed case from my suite:

name: setup-vite-react-query
context:
  scaffold_script: scaffold.sh
execution:
  allowed_tools: [Read, Glob, Grep, Skill]
  prompt: /twd:setup
graders:
  - name: vite-plugin-imported
    type: regex
    pattern: 'from\s+["'']twd-js/vite-plugin["'']'
    target: { source: file, path: vite.config.ts }
  - name: no-entry-file-code
    type: regex
    pattern: 'initTWD|twd-js'
    match: not_contains
    target: { source: file, path: src/main.tsx }
  - name: patterns-file-attempted
    type: tool_used
    tool: Write
    input_match: '\.claude/twd-patterns\.md'

The last grader reads the agent's actions, not the disk: eval runs refuse writes into .claude/, so it checks that the skill tried to write its notes file there.

The scaffold writes a small Vite + React app with heredocs. It installs nothing, so the case needs no network.

Graders come in two families. Some check what was written: a pattern is in a file, or isn't. Others check what the agent did: it called a tool, with this input, before that other call. For what a regex can't decide, there's an llm grader: a judge model reads the file against a criterion you write and votes pass or fail, three times.

With and without your skill

This is the part I didn't expect to matter most. Every case runs twice: with your plugin and without it. The difference, Δ, is what your skill adds on top of what the model already knows.

My setup case on Opus:

	Score
With the plugin	1.00
Without the plugin	0.10
Δ	+0.90

Without the skill, the model reads the same project files, says the setup command "isn't installed in this session", and does it by hand from what it knows. It wires the library into the app's entry file, which the library no longer needs, and never writes the config file.

A Δ near zero doesn't mean a case is useless. It means the model already knows how to do it. The case still guards against an edit to the skill that breaks it: if that happens, the score with the plugin drops.

Run it more than once

Because an agent doesn't behave the same every time, each case runs three times and the score is the average. A run's score is the share of graders that passed. --threshold 0.9 decides what counts as a pass: it tolerates a grader failing once in three runs, not one that fails every time.

# While editing a skill: one run, with the plugin only
claude plugin eval . --scaffold --runs 1 --ablation none --allow-tools Write Edit

# Before a release: three runs, with and without the plugin
claude plugin eval . --scaffold --threshold 0.9 \
  --model claude-opus-5-5 --judge-model claude-sonnet-5-5 --allow-tools Write Edit

The full suite, eight cases on Opus, costs about $10 and 25 minutes. The quick mode is under $2.

Fakes: mock your tools like you mock your APIs

The eval sandbox has no browser and no network, and one of my skills runs tests in a browser. So the scaffold writes a stand-in test runner: it honours the filter flag, writes a report in the same format as the real one, and fails a test that mocks a URL the app never requests.

It's the same idea as mocking an API in a test. You're testing how the skill reacts to the tool, not the tool. The real runner has its own tests in its own repo.

Graders that lie

A failing grader is fine: it tells you something. The dangerous one passes when it shouldn't, because nobody looks at a green check. It happened to me three times:

A regex that read too far. beforeEach\([\s\S]*?twd\.clearComponentMocks was meant to check that mocks are cleared in beforeEach. The lazy match kept going into the afterEach below it and passed a file that cleared them in the wrong place.
"The skill was invoked" passed, and the skill did nothing. The skill ran in a forked subagent that only saw its arguments. Invoked without arguments, it replied that it had no goal, and the main agent wrote the tests on its own, without the skill's rules. The test looked fine.
A check on the call, not the result. Two graders checked that the skill ran the type checker and the linter. They looked at whether the command was called, not whether it worked. For a while every one of those commands failed for an unrelated reason (Node was installed under my home folder, which the sandbox can't read), and both graders stayed green.

Before trusting a new grader, run it against a file that should pass and one that should fail.

It doesn't replace trying it

I still run my skills by hand on real projects. The eval doesn't replace that; it tells me when to. When a case drops after an edit, or after a model release, I know exactly where to look.

The scenarios twd-ai guards, and their scores per model, are on twd.dev.

Next in this series: reading the trace, the file that turns a score into a reason.

The Testing Trophy, and Where TWD Fits

Kevin Julián Martínez Escobar — Mon, 26 Oct 2026 10:42:31 +0000

Most frontend teams I talk to have plenty of tests and still find out about broken screens from users. The unit tests are green, the refactor broke every one of them anyway, and nobody is sure the checkout flow works until someone clicks through it.

That tension is exactly what the Testing Trophy is about.

The trophy

Kent C. Dodds drew testing as a trophy instead of a pyramid. Four layers, and the size of each one is roughly how much of your effort it deserves:

Static: typos and type errors caught as you write the code. TypeScript, ESLint.
Unit: one function, hook or component checked in isolation. Vitest, Jest.
Integration: several units working together. The biggest part of the trophy.
End to End: a robot clicking through the deployed app, real backend included. Playwright, Cypress.

The shape comes from one line, written by Guillermo Rauch and popularized by Kent:

Write tests. Not too many. Mostly integration.

Each layer trades speed for confidence. Unit tests are cheap, but they can pass while the screen is broken. End-to-end tests give the most confidence, but they need the whole system running and are the slowest to write and run. Integration tests sit in the middle: they use the app the way a user does, and they stay fast and deterministic because the only thing faked is what you don't own.

Kent's guiding principle for them:

The more your tests resemble the way your software is used, the more confidence they can give you.

What integration means on the frontend

For a frontend app, an integration test renders the real thing (routes, providers, state, components) and mocks only the network. Then it does what a user would: visit a page, type, click, and check what is on the screen.

The classic way to do this is Testing Library with jsdom, in Vitest or Jest. It is a good setup and it is where most of us learned this style. It runs in a simulated DOM in your terminal, though, separate from the app you have open while you build.

Where TWD fits

TWD is built for that middle layer, and it runs the tests in the browser you develop in:

Next to your app. A sidebar in your dev server runs the tests against the real app. You watch each one drive the page.
Mock what you don't own. API calls are answered by a service worker, and contract testing checks every mock against your OpenAPI spec, so the mocks don't drift from the real API.
Tests that read like the user. Testing Library queries find elements by role, label and text.
The same tests everywhere. twd-cli runs them headlessly in CI, with coverage and a video per test if you want one, and your AI agent can write, run and fix them.

In practice this layer can carry a lot. On two production apps we run 872 tests on every pull request, mostly user flows, in one headless Chrome on a 2-vCPU GitHub runner, in about 5 minutes per app.

What it doesn't replace

TWD doesn't cover the whole trophy, and it shouldn't.

Keep a few end-to-end tests with Playwright or Cypress for the critical paths, the ones that need the real backend, the payment provider or the third-party SDK. Use unit tests for logic that is easier to check on its own. Keep TypeScript and ESLint at the base (and twd-lint for your test files). Then let integration tests carry most of the weight.

A map, not a ritual

TWD doesn't ask you to pick a layer before you write a test. You automate the check you were already doing by hand while you build. The trophy just shows where those checks end up: mostly in the middle.

The full page, with the trophy and the tools per layer, is at twd.dev/testing-trophy. For the strategy itself, go to the source: Kent's TestingJavaScript.com.

One CLI, one config file: adding browser tests to a Vite+ project

Kevin Julián Martínez Escobar — Fri, 16 Oct 2026 12:32:44 +0000

A typical frontend setup has an ESLint config, a Prettier config, lint-staged and Husky for the commit hook, and the Vite config, each with its own command and a package.json script to glue them together.

A fresh Vite+ project has one config file, vite.config.ts. I wanted to see if a test tool could join that file without bringing its own sprawl. This is what happened when I added TWD.

What Vite+ actually unifies

Vite+ is a toolchain from VoidZero that puts Vite, Vitest, Oxlint, Oxfmt, Rolldown and tsdown behind a single CLI called vp. It also manages Node and your package manager.

vp install   # the right package manager, picked for you
vp dev       # Vite
vp check     # format + lint + type check, in one pass
vp build     # production build

The command I like most is vp check. It runs formatting, linting and type checking together, the type checking through Oxlint's type-aware path on the TypeScript Go toolchain. On the demo app it checks 22 files in about a second.

Configuration follows the same idea. Lint rules, formatter options and the pre-commit hook all live in vite.config.ts:

export default defineConfig({
  staged: { '*': 'vp check --fix' }, // the pre-commit hook. No Husky, no lint-staged
  lint: { options: { typeAware: true, typeCheck: true } },
  fmt: { semi: false, singleQuote: true },
  plugins: lazyPlugins(() => [vue()]),
})

That is the bar any new tool has to clear: does it fit in here, or does it bring a new file and a new command?

The dev server: one plugin

TWD runs tests inside the app you are already developing, in a sidebar next to the page. It is wired in as a Vite plugin:

plugins: lazyPlugins(() => [
  vue(),
  twd({ testFilePattern: '/**/*.twd.test.ts' }),
]),

That is all. The plugin finds the test files and injects the sidebar while vp dev is running, and does nothing in vp build. main.ts has no test code in it, and the production bundle has none either.

Linting: an ESLint plugin, running in Oxlint

This was the part I expected to lose.

twd-lint catches the mistakes that make a browser test fail in confusing ways: a missing await on twd.visit(), a stray it.only, describe imported from Vitest instead of the TWD runner. It ships as an ESLint plugin. Vite+ lints with Oxlint, not ESLint.

Oxlint can load ESLint plugins as JS plugins, unchanged. So the rules go into the same lint block as everything else, scoped to test files with an override:

lint: {
  jsPlugins: [{ name: 'twd', specifier: 'twd-lint' }],
  overrides: [
    {
      files: ['**/*.twd.test.ts'],
      rules: {
        'twd/no-focused-tests': 'error',
        'twd/runner-imports': 'error',
        'twd/no-missing-await': 'error',
        'twd/no-request-body': 'error',
      },
    },
  ],
},

I tested it against a deliberately broken test file. vp lint reported the same five errors as the standalone twd-lint CLI, word for word. vp lint --fix rewrote the Vitest import and added the missing await. The rules did not fire on the same code outside a test file.

Since the hook already runs vp check --fix on staged files, every commit lints the tests too. Nothing to wire up.

The tests: mocking at the network layer

The demo is a Todos page with no backend. Every API call in the tests is answered by twd.mockRequest, which makes the states that are annoying to reach by hand easy to test:

await twd.mockRequest('getTodos', {
  method: 'GET',
  url: '/api/todos',
  response: todoList,
  status: 200,
  delay: 500,
})
await twd.visit('/todos')

const loading = await screenDom.findByText('Loading todos...')
twd.should(loading, 'be.visible')
await twd.waitForRequest('getTodos')

The delay holds the response long enough to assert the loading state. The other tests use the same idea:

Error and retry: a 500 shows the error, then the test mocks a good response and clicks Retry.
Request body: waitForRequest('createTodo') returns the parsed JSON the page sent.
No call at all: getRequestCount('createTodo') is 0 when validation blocks an empty title.
Network failure: status: 0 simulates a dropped connection.

Eight tests, headless, in about four seconds.

Coverage: one more plugin

Coverage is vite-plugin-istanbul added to the same plugin list, switched on only when CI=true so the normal dev server stays fast:

istanbul({
  include: 'src/*',
  exclude: ['node_modules', '**/*.twd.test.ts', '**/twd-tests/**'],
  requireEnv: !process.env.CI,
  extension: ['.ts', '.vue'],
}),

twd-cli collects the coverage at the end of a headless run, and nyc prints the report. The Todos page came out at 100% of lines.

One thing tripped me up: nyc report ignores .vue files by default, and --extension .vue replaces the default list instead of adding to it. My first report had the .vue files and had lost every .ts file. Pass both:

nyc report --reporter=text --temp-dir .nyc_output --extension .ts --extension .vue

CI: the same commands, in a workflow

Vite+ has a GitHub Action, voidzero-dev/setup-vp, that installs Node, the package manager and the dependencies in one step. After that the workflow is just the commands from the terminal:

- uses: voidzero-dev/setup-vp@v1.21.2
  with:
    node-version: '24'
    cache: true
- run: vp check                       # lint includes the TWD rules
- run: |
    nohup vp run dev:ci > vite.log 2>&1 &
    timeout 60 bash -c 'until curl -s -o /dev/null http://localhost:5173; do sleep 1; done'
- run: vp exec twd-cli run
- run: vp run collect:coverage:text

Two details are specific to Vite+ on pnpm:

npx does not work here. The project pins pnpm through devEngines, and npm refuses to run (EBADDEVENGINES). Use vp exec. That also rules out the twd-cli composite actions for now, because they call npx internally.
pnpm blocks install scripts by default, including the one where puppeteer downloads Chrome. One line in pnpm-workspace.yaml (allowBuilds: { puppeteer: true }) and vp install fetches it, locally and in CI.

What it adds up to

Counting what the test setup added to the project:

a plugin in vite.config.ts, plus two more entries in its lint block
twd.config.json, two lines long
one generated service worker in public/
a test:ci script

No new linter, no new hook manager, no new config format. TWD fits into the tools Vite+ already gives you.

The whole project is on GitHub: BRIKEV/twd-vp-vue.

Your library isn't in the training data: making AI agents use it well

Kevin Julián Martínez Escobar — Tue, 13 Oct 2026 22:04:47 +0000

A model that doesn't know your library doesn't fail by doing nothing. It fails by confidently writing code for a library it does know.

Ask an AI agent to write a React component with Jest tests and it'll do a decent job. There are millions of examples of both in its training data.

Now ask it to use something it has never seen:

your company's internal API client;
your design system;
a niche open source library;
a new major version that changed everything.

You'll get code that looks right, compiles in the agent's head, and is wrong in small, confident ways. The agent isn't guessing at random. It fills every gap with the closest thing it knows. An option name from a similar library. An import from the popular alternative. A return shape it assumed.

I've spent the last months on this problem with TWD, a testing library I maintain that almost no model knows yet, and a Claude Code plugin that writes tests with it. This is the toolkit I ended up with. None of it is specific to my library.

1. Give the model knowledge, not just rules

Whatever you hand the agent (a skill, an AGENTS.md, an llms.txt, a docs page), it is the only place the model can learn your API. So spend that space on what teaches:

Signatures written out in full, with the option names spelled correctly.
What things return, especially when it differs from the popular library the model will assume.
The patterns: the order things have to happen in, the setup every file needs.
Complete, correct examples. With a model, one working example beats three paragraphs of explanation.

What you shouldn't fill it with is warnings. A list of "never do X" written three times doesn't stop the mistake; a program checking for it does. I wrote a whole post about why: Prose does not beat determinism.

Load it in layers

Context isn't free, and the agent pays for every byte on every request. My first reference doc was 32 KB, and the agent loaded all of it to write a three-test file.

Now it's split: a core of about 8 KB with what most tasks need, and a second file with the rest that the agent only reads when the task calls for it. Claude Code skills support this directly, since files under references/ are loaded on demand. For any other setup it's the same idea: a short core, with links to the details.

2. Make the compiler teach your API

A typed API is documentation the model can't ignore, as long as somebody runs the compiler.

When the agent invents an option name or passes arguments in the wrong order, a type check turns that into an error in seconds, instead of a bug found at runtime. So:

Type your public API strictly. An option that's silently ignored when misspelled is a trap for people and agents alike. Make it a compile error.
Tell the agent to type-check what it wrote before running anything. It's one line in the instructions, and it's the cheapest check you have.

3. Lint what types can't see

Some mistakes are valid code: a missing await, an import from the popular alternative library, a debugging leftover like .only, reading a property that's always undefined for your API.

That's what a small, focused linter is for. For my library it's four rules. Three properties made it work well with agents:

Zero config. One command, no setup, the same result in every project. If the agent first has to configure your linter, it won't run it reliably.
Autofix for the safe cases. If the fix is always correct (adding await inside an async function), just apply it.
Messages that say what to do. Not "invalid usage", but the cause and the fix in one sentence. The agent's next action is in the error message, and a human gets the same help.

Before trusting it, run it against every real project you can find. A rule that fires on working code teaches the agent something false.

4. Design your output for two readers

An agent reading a console log will grep for checkmarks, count lines, and sometimes get it wrong. Your CLI has two readers now: a human and a program.

Write a JSON report next to the pretty output. Mine has the outcome, each failure with its error, and the diagnostics a person would look for.
Offer --format json on every command that reports problems.
Document your exit codes, and make them mean something: clean, problems found, nothing to do.

5. Test your instructions with evals

Instructions regress like any code. A well-meant edit can quietly drop a rule the agent depended on, and nothing tells you.

An eval is a test for an agent: set up a small project, give the agent a task, then grade what it produced. In Claude Code that's claude plugin eval. A case is a folder with a scaffold script, a prompt and a list of graders:

prompt: |
  Write tests for the todo page: the list, the empty state,
  a failed load, and adding a todo.
graders:
  - name: right-imports
    type: regex
    pattern: 'from\s+["'']twd-js/runner["'']'
    target: { source: file, path: src/twd-tests/todos.twd.test.ts }
  - name: linter-ran
    type: tool_used
    tool: Bash
    input_match: 'twd-lint'

What I learned running them:

They find real gaps. Before one round of fixes, my old instructions scored 0.50 on a case. Half the checks failed, on a task I thought was covered.
Prefer deterministic graders. A regex on the file the agent wrote, or "did it call this tool", gives the same answer every time. Keep an LLM judge for what can't be matched, like "is this one test per flow, or one per element".
Make judge criteria precise. My judge failed a correct file because the criteria said "awaits every call", and one of those calls is synchronous. Listing the async calls by name fixed it.
Plant mistakes. One of my cases starts from a file with four planted errors and checks that the agent runs the linter and leaves the file clean.
Don't run them on every PR. They cost real money: about $1.80 for my five cases. I run them before releases and after changing the instructions. On every PR, CI runs claude plugin validate --strict, which is free and offline, plus a small script that checks every file the instructions reference exists.

6. Keep the instructions honest

The quietest failure is drift: the library moves on, and the instructions don't.

When I compared my agent's instructions with the library's docs and source, I found an option documented that the library ignores, and a default stated backwards. The agent followed the instructions faithfully. The instructions were wrong.

Two habits help:

No version notes. "Fixed in 1.8" or "requires 1.9+" means nothing to a model that doesn't know which version you're on. Describe the current behaviour.
Check claims against the source, not your memory. And if a rule in your instructions has a reason attached, make sure the reason is true. One of mine wasn't, for months.

The checklist

If your library, SDK or design system isn't something models know yet:

Teach the API once, with complete examples, in layers.
Type it so wrong calls fail to compile, and have the agent run the compiler.
Lint what types miss, with zero config, autofix and messages that say what to do.
Give agents JSON and exit codes, not logs.
Write evals with mostly deterministic graders, and run them before releases.
Keep the instructions honest against the source.

Maybe one day the models will know your library the way they know React. Until then, your library has to make up for what the model doesn't know.

Prose does not beat determinism: stop asking your AI agent to remember

Kevin Julián Martínez Escobar — Tue, 13 Oct 2026 08:46:07 +0000

If a rule can be checked by a program, stop asking the model to remember it.

Most of us improve an AI agent the same way. It makes a mistake, so we add a rule to the prompt, the skill or the CLAUDE.md. It makes the mistake again, so we make the rule bold. Then capitals. Then a section called NEVER DO THIS.

I did exactly that for months, on a Claude Code plugin I maintain that writes and runs frontend tests. This post is about what replaced it, and why I think it applies to any agent setup, not just mine.

The instructions that said everything three times

The plugin's agent writes tests with TWD, a testing library I maintain. The important rules looked like this:

At the top, in Hard Constraints: "Always await async methods."
In a Do / Don't list in the middle: the same rule, worded differently.
In a Common Mistakes list at the end: "1. A missing await."

Every important rule existed three times, with WRONG / RIGHT code blocks in between. A single "write tests for this page" request loaded 56 KB of instructions.

And the agent still forgot an await now and then.

When it did, nothing caught it until the tests ran in a browser: start the dev server, run the suite, read the failure, find the line, fix it, run again. That's the most expensive possible moment to find a missing keyword, for a person and even more so for an agent paying for every step in tokens.

Writing a rule a third time doesn't make the model more likely to follow it. A program checking for it does.

The ladder: from hope to guarantee

The fix was to take each rule and move it down a ladder, from "the model has to remember" to "a program verifies it":

Step	Who checks	When you find out
Prose in the instructions	The model, if it remembers	After a full test run, if at all
Type check	The compiler	Seconds
Linter	A lint rule	Seconds, often fixed automatically
Structured tool output	The tool itself	One read, no guessing

Each step down is cheaper and more reliable than the one above it.

Step 1: run the compiler you already have

A lot of what an agent gets wrong is calling an API the wrong way: a misspelled option, arguments in the wrong order, a method from a different library. If your API is typed, all of those are already compile errors.

My agent just never ran tsc. Now its instructions say: type-check the files you wrote before the first test run. One line, and a whole category of mistakes is caught in seconds, without starting anything.

Step 2: lint what the types can't see

Some mistakes are perfectly valid code:

page.visit('/todos')                       // forgot the await
import { describe, it } from 'vitest'      // wrong test framework for this runner
it.only('creates a todo', ...)             // silently skips every other test

The type checker is happy with all three. A linter isn't. So I wrote one for my library, with a handful of rules for exactly the mistakes the agent kept making, and the agent now runs it after the type check, with --fix.

Two details made a big difference:

Autofix the safe cases. Adding a missing await inside an async function is always correct, so the linter just does it.
Write messages that say what to do. Not "invalid usage at line 33", but the cause and the fix in one sentence. The agent reads the message, and its next action is right there. So does the human.

Step 3: give the agent data, not logs

The last step is the output of your tools. An agent reading a console log will grep for checkmarks, count lines, and occasionally get it wrong. My test runner writes a JSON report on every run: the outcome, each failed test with its error, which network mocks were never used. The agent reads a file instead of guessing from text.

The instructions got shorter, but not empty

With those checks in place, I removed the repetition, following one rule:

The instructions teach the correct form once, in positive form, with an example. The tooling makes sure it's followed.

That distinction matters. There are two kinds of text in agent instructions:

Knowledge: how the API works, what things return, complete examples. No linter replaces this. If the model doesn't know your library, this is the only place it can learn it.
Policing: the repeated "never do X". This is the part the tooling replaces.

The policing went away. The words NEVER, WRONG and ALWAYS went from 13 occurrences to 1. A "write tests" request now loads 32 KB of instructions instead of 56 KB.

But the bigger saving isn't in the instructions. One failed run caused by a missing await costs more tokens than any paragraph: the run, the report, the diagnosis, the fix, the re-run. Every mistake caught by the compiler or the linter is a loop that never happens.

A linter can't believe something false

My favourite part of this story is a rule that turned out to be wrong.

Rule number one in my instructions, in capitals, was: "ONE top-level describe() per file. Multiple top-level describes break the test runner."

So the first version of the linter had a rule for it. I ran it against every project using the library: 494 test files across React, Vue, Angular, Svelte and others.

It reported 44 files. Every one of those tests ran and passed.

I read the runner's code. It handles any number of top-level blocks. The rule I had given the agent, at the very top, for months, was a style preference with a made-up reason. The agent obeyed it faithfully, and nobody ever checked it.

That's the deeper problem with prose: it can say anything, and nothing verifies it. A lint rule has to survive real code. This one didn't, so it was removed, and the instructions now call it what it is, a convention.

The rules that survived were worth it. Across those 494 files there were no false positives, and every report was a real bug, including dozens of missing awaits and assertions that had never actually run because nobody awaited them. Some of those were in code written by people, long before any agent.

What stays prose

Not everything can move down the ladder. "Set up the mock before the page loads" depends on when the app makes its request, and that isn't visible in the test's source. "Reset the store between tests" depends on the app. Those stay in the instructions, written once, with an example.

That's fine. The goal isn't zero prose. The goal is that prose only carries what a program can't check.

The takeaway

If you maintain a skill, an AGENTS.md, a CLAUDE.md or a system prompt, go through it rule by rule and ask:

Could the compiler catch this? Have the agent run it before anything else.
Could a linter catch this? Use an existing rule or write one, with a message that says how to fix it.
Could the tool report this? Give the agent structured output, not logs.
None of the above? Keep it in prose. Once, in positive form, with an example.

Telling the model three times and hoping doesn't work. Tell it once and check.

Your Repo Can Now Tell People It Cares About Accessibility

Kevin Julián Martínez Escobar — Mon, 05 Oct 2026 11:37:26 +0000

Open any repository on GitHub and you see the same tabs: README, Code of Conduct, Contributing, Security. As of October 1, there can be one more: Accessibility.

GitHub now highlights an ACCESSIBILITY.md file on the repository overview and links to it from the About section. It is a small change, and it puts accessibility on the front page of your project, next to the files every contributor already reads.

We added one to TWD right away. Here is why, and what we learned.

Why it matters

Picture someone who uses a screen reader trying out your library. Something in the docs doesn't work for them. Today, they have to guess: is this a known issue? Does anyone care? Where do I even report it?

An accessibility page answers all three before they ask. It says this project takes accessibility seriously, here is what we have tested, and here is how to tell us when something gets in your way.

It also helps the other side. Contributors see, in plain words, what is expected of a change that touches the UI.

What we put in ours

TWD had a head start. In June we had it independently audited against WCAG 2.2 AA, fixed what the auditors found, and published a formal accessibility statement.

We expected to copy and paste that statement. We didn't. A formal statement is written for compliance. The GitHub page is written for people, so we wrote it in plain language around three ideas:

Be honest about what was tested. Our audit covered the sidebar and the main docs pages, using a keyboard and the NVDA screen reader. So that is what the page claims. For everything else, WCAG 2.2 AA is our goal, and we say so.

Make reporting easy. Open an issue, send an email, or ask in Discussions. No need to know which guideline is involved, and no need to share anything personal. We commit to replying within 5 business days.

Tell contributors what good looks like. Keep it usable with a keyboard, keep focus visible, don't rely on colour alone, and include light and dark screenshots with UI changes.

You can read the result here: TWD's ACCESSIBILITY.md.

Add one to your project

You don't need an audit to start. You need an honest page.

Create ACCESSIBILITY.md in your repo root (or in .github/ or docs/).
Start from GitHub's template, or add it from your repo's Community Standards page.
Fill in the two sections that matter most: how to report a barrier, and what you have and haven't tested.

That's it. The tab appears on your repo's front page.

Accessibility work usually happens out of sight. Now it has a place on your repo's front page, where users and contributors will see it.

3 Developer Archetypes That Can Break a Team

Kevin Julián Martínez Escobar — Tue, 22 Sep 2026 08:31:47 +0000

There are developers who make a team faster. There are developers who make a team better. And there are developers who slowly make a team dependent on them, without anyone noticing.

The three I want to talk about are usually rewarded for it. They look experienced, productive, sometimes indispensable. AI is making all three of them easier to scale.

1. The Senior Cynic

You have met this one. Years of experience and a long list of things that do not work. Every new approach reminds them of something they tried five years ago. A simpler architecture will not scale. A different testing strategy was already tried. Another tool is not going to solve anything.

Experience is valuable. It becomes a problem when it turns into a filter that rejects anything unfamiliar. The Senior Cynic is usually not protecting the architecture, they are protecting the mental model they have built over their career. Knowing why something failed in the past is useful. Assuming it cannot work in a different context is not.

AI makes this cheaper. Ask for arguments and you get arguments: comparisons, trade-off tables, a full technical rationale for the position you already held. You no longer just have a strong opinion, you have twenty pages backing it up in ten seconds.

The antidote is not optimism, it is evidence. Try the thing. Measure it. Let the current context weigh more than the muscle memory.

2. The Guess Coder

This one is easier to spot, because it shows up in the code. They build for a future that does not exist yet. A simple requirement becomes an abstraction. The abstraction becomes a framework. The framework needs configuration, the configuration needs an extension system, and now you maintain infrastructure for a problem you have never actually had.

The intentions are usually good. They want to avoid technical debt. They want something that scales. They have read about SOLID, clean architecture, hexagonal architecture, event-driven systems, distributed everything. The catch is that principles without context turn into cargo cult engineering, and often the best architecture for the future is the simplest one you can change today.

AI makes this effortless. Ask an agent to "make this extensible" and you get hundreds of lines of perfectly reasonable looking code. It compiles. The structure looks sophisticated. And you have just spent an afternoon solving a problem you do not have.

The fix is not to stop thinking about the future, it is to separate known future requirements from imagined ones. Build for what you know, keep the code easy to change, and validate an assumption before you turn it into architecture.

3. The Superhero

This is the one I worry about most.

The Superhero knows everything. The weird legacy service, how the deploy actually works, why that obscure config exists. When production breaks, everybody knows who to call. They are genuinely valuable, and that is exactly the problem.

The team learns that the fastest route to a solution is to ask them. So people stop investigating. They stop learning the system. They stop taking ownership. The Superhero rarely pushes back on that, and sometimes reinforces it: "I'll just do it", "it's faster if I handle it", "I'll explain it later". Because they are good at solving problems, it works. In the short term.

Then the organization starts rewarding the behavior. They become the person who saves releases and fixes incidents, and being indispensable starts to look like career progress. But there is a difference between being valuable and being a dependency. A healthy team should not need one specific person to keep operating.

The goal is not to stop the Superhero from solving incidents. It is to make sure the next incident does not need the same person. Fix the problem, share what you learned, improve the system, hand the responsibility over, reduce the odds of it happening again. Otherwise you are not removing the fire, you are building a very efficient fire department around one person.

What the three have in common

They look nothing alike, and they optimize for the same thing: themselves, instead of the team's ability to operate without them. The Cynic protects their experience. The Guess Coder protects their imagined future. The Superhero protects their indispensability.

AI amplifies all three. More arguments, more complexity, more changes landing faster than the rest of the team can follow them.

Which is why I do not think AI adoption is mostly a tooling problem. It is a team problem. A team with a strong engineering culture gets amplified. A team with weak delegation, thin tests, knowledge silos and a taste for over-engineering gets amplified too, and much faster.

What to optimize for instead

Not developers who know everything, but developers who make knowledge accessible. Not developers who build for every possible future, but developers who keep today's system easy to change. Not developers who are indispensable, but developers who make the team less dependent on them.

The strongest developer on a team is probably not the one who can solve every problem. It is the one who makes sure the team does not have to solve the same problem twice.

With AI in the loop, that distinction matters more, not less.