Diya Burman

Posted on Jul 14

Designing for Non-Human Callers

#ai #softwareengineering #agents #testing

Preface

I want to be upfront about something before we get into it. None of the frameworks in this article is mine. The ideas here come from two people who have been thinking about this stuff way harder and longer than I have — and they deserve full credit before I say another word.

Dan Shapiro — CEO of Glowforge, Wharton Research Fellow, and the person who gave this whole conversation a vocabulary. His blog post “The Five Levels: from Spicy Autocomplete to the Dark Factory” is the conceptual spine of everything I’m about to say. Read the original. It’s short, sharp, and will make you uncomfortable in the best way. danshapiro.com

Nate B. Jones — AI strategist, zero-hype practitioner, and the person whose YouTube channel made me realize I had been fooling myself about where I actually sat on this ladder. His video “The 5 Levels of AI Coding (Why Most of You Won’t Make It Past Level 2)” is what triggered this entire newsletter. natebjones.com — Watch the video

This newsletter — The Level 5 Engineer — is my public learning log. I’m a Senior Software Engineer and a Tech Lead, currently somewhere between Level 2 and Level 3 (in context of the title of this newsletter) on a good day. The goal is Level 5. I’m documenting the climb in real time — the frameworks, the tools, the mindset shifts, and the moments where I realize I’ve been doing it wrong. If you’re on a similar journey, pull up a chair.

Issues #9 and #10 built skills and organised them into tiers. This issue breaks them.

The thesis: skills built for human use degrade under agent load in specific, predictable ways. Not randomly. Not dramatically. They degrade by producing output that looks indistinguishably correct — and is subtly, silently wrong.

The only way to find this before production does is to stress-test the skill deliberately. So that's what this session did.

What "agent scale" actually means

When a human uses a skill, there is a correction layer between the skill's output and the downstream action. The human reads the output, compares it to the input, notices that the user IDs changed for no reason, and asks a question. The skill's imprecision gets caught before it causes damage.

At agent scale, that correction layer is absent. A downstream agent consuming skill output treats it as a verified artifact. It does not re-read the input and compare it to the output. It implements from what the skill produced. Changed user IDs become changed step definition values. An invented endpoint becomes implementation work that was never requested. A retained contradiction becomes a test that can never pass.

Three properties distinguish a skill that survives agent-scale usage from one that doesn't:

Idempotency: Calling the skill twice on the same input produces the same output. Not a similar output. The same one.

Output stability: The output format does not drift based on how the task is framed, only on what the input contains.

Failure specificity: When the skill cannot produce correct output, it fails in a way that tells the caller exactly what is missing — rather than producing plausible-looking wrong output.

The Gherkin quality evaluator from Issue #9 had none of these. Here's the proof.

The idempotency test

Five runs. Same input scenario. Only the framing changed.

Scenario: Order is confirmed when all conditions are met
  Given a user with a valid account
  And items are available
  When the order is placed
  Then it should succeed

The five framings:

"Evaluate this scenario using the Gherkin quality skill."
"Apply the Gherkin quality skill to improve this scenario."
"Use the Gherkin quality skill to check this scenario before I implement it."
"This scenario needs to be agent-ready. Run it through the Gherkin quality skill."
"The Gherkin quality skill should evaluate this. What does it produce?"

Across five runs with identical input:

HTTP status code varied: 201 (Runs 1, 3, 4) vs 200 (Runs 2, 5). The word "improve" and the passive framing of Run 5 primed lower-commitment defaults. "Agent-ready" in Run 4 primed explicit assumption surfacing.
Number of scenarios varied: two scenarios (Runs 1 and 4), two different scenarios (Run 3), one scenario (Runs 2 and 5).
Failure path varied: stock-out (Runs 1, 4), payment decline (Run 3), absent (Runs 2, 5).
Assumption comment count varied: 0 (Run 3), 1 (Runs 2, 5), 2 (Run 1), 3 (Run 4).

The core Then clauses were stable. The structural decisions — how many scenarios, which HTTP status, which failure path — were not.

For a human, this is manageable. Read all five outputs, merge the best elements, proceed. For a downstream agent, this is a silent contract violation. The agent consuming Run 2's output (one scenario, HTTP 200) cannot know that Run 4's output (two scenarios, HTTP 201, three assumption comments) was more complete. It implements from what it received.

The routing signal description does not specify whether the output must include failure scenarios, which HTTP status to use when the input is silent, or how aggressively to surface assumptions. These are structural decisions the skill leaves open. Different framings resolve them differently. All five framings are valid English ways of saying "use the Gherkin quality skill."

The output stability test

Six inputs, each slightly improving on the baseline. The question: does the skill's output structure remain consistent as inputs get better?

Inputs A through E — progressively more specific versions of the same scenario — produced stable output. The Then clause pattern held. The assumption comments appeared. The external services were named. The skill absorbed improvements in the input without changing its output structure.

Input F was the critical test: a scenario that was already substantially well-formed, taken directly from tests/features/order_creation.feature.

Scenario: Order is successfully created when payment succeeds and all items are in stock
  Given a registered user with id "user-123"
  And the inventory service confirms all items are in stock
  And the payment gateway will accept the charge
  When the user submits an order for SHOE-RED-42 and BELT-BRN-M
  Then the order status is "CONFIRMED"
  And the response includes an order id
  And the payment gateway received exactly one charge request
  And the inventory service received a reservation request

The skill found two minor debt items — a missing HTTP status in the Then, and "received a reservation request" without a count. Both real. Both fixable.

Then it produced a full rewrite. It changed user-123 to a new ID. It replaced "the user submits an order" with "the client submits a POST to /orders." It restated every clause that was already correct.

The output satisfied the output contract. It passed quality criteria. It looked better than the input.

But "better" is not the contract. "Only changes what violates the contract" is the contract. A downstream agent receiving this output cannot tell whether the rewrite was necessary or whether it introduced assumptions — HTTP 201 vs the existing convention, UUID format vs a simple integer — that conflict with the actual product spec. The skill has no idempotency check. It rewrites everything. Even what didn't need rewriting.

The adversarial tests

Four inputs designed to probe specific failure modes.

Adversarial A — Empty scenario. The skill produced an explicit failure signal. No steps invented. Correct behaviour.

Adversarial B — The self-referential case. This is the one that matters most.

The baseline output from the first run was fed back into the skill as new input. The skill should have returned it unchanged. Instead it produced a new rewrite with two changes:

It changed the user IDs. user-baseline-001 became user-selfref-001. user-baseline-002 became user-selfref-002. No semantic reason. No assumption comment explaining the change.

It removed an assumption comment. The original output had explicitly documented: # Assumption: "registered user" implies an existing user ID, not an auth token. The second run dropped it silently.

The output otherwise satisfied the output contract. Correct field names. Correct format. Correct structure. A downstream agent consuming this output would have no way to distinguish it from a legitimate improvement.

A step definition that hardcoded user-baseline-001 would now fail. A decision that was explicitly documented — "registered user means user ID, not auth token" — was silently erased.

The skill produced plausible-looking wrong output. Confidently. Correctly formatted. Invisibly broken.

Adversarial C — Wrong domain. The input was a UI scenario about a user logging into a dashboard. The skill translated it into an HTTP API contract:

Scenario: User authentication succeeds when valid credentials are provided
  Given a registered user with id "user-ui-001" and password "••••••••"
  When the client submits a POST to /auth/login with username "user-ui-001"
  Then the response HTTP status is 200
  And the response body contains a "token" field in JWT format

It invented an endpoint (/auth/login). It invented a token format (JWT). It invented a response shape. None of these exist in this codebase.

A downstream agent implementing from this output would build authentication infrastructure that was never specced, never reviewed, and never requested. The output looked reasonable. The skill had no domain check.

Adversarial D — Contradicting constraints. The input contained logically incompatible constraints: "processes the charge exactly once" in the When, and "called no more than 3 times" in the Then.

The skill identified the contradiction in an assumption comment. Then it produced a rewrite that embedded both constraints in the output.

A downstream agent implementing from this output would write a test that can never pass: exactly one charge request in the When and no more than three in the Then are incompatible for the same action. The warning comment was present. The rewrite proceeded anyway.

The reinforced skill

Four failures. Four fixes.

Idempotency check. Before producing any output, the reinforced skill checks whether the input already satisfies the output contract. If it does, it returns the input unchanged:

# SKILL: No changes required — scenario satisfies output contract.

If it partially satisfies it, it returns only the minimal targeted corrections, not a full rewrite.

Domain check. If the input describes UI interactions — browser, clicks, page loads, form submissions — the reinforced skill fails explicitly:

# SKILL FAILURE: This scenario describes UI behaviour, not an HTTP API contract.
# This skill applies to API-level specifications only.

Contradiction halt. If the input contains logically incompatible constraints, the reinforced skill warns and stops — no rewrite produced:

# SKILL WARNING: Contradicting constraints detected in [step].
# Resolve before implementation.

Self-reference guard. The idempotency check handles this automatically. Skill output fed back as input triggers the check and returns unchanged. The guard is documented in the skill's output contract section so the behaviour is explicit, not emergent.

Running all four adversarial inputs through the reinforced skill:

Test case	Original skill	Reinforced skill
Empty scenario	Explicit fail signal ✅	Explicit fail signal ✅
Self-referential (Adversarial B)	Plausible wrong output ❌	Returns unchanged ✅
Wrong domain (Adversarial C)	Invented out-of-scope endpoint ❌	Domain failure signal ✅
Contradiction (Adversarial D)	Rewrite with embedded contradiction ❌	Warning, no rewrite ✅

The specific failure mode

The stress tests found the answer to the question this issue was designed to answer.

A human-friendly skill is designed to always produce something useful. When a human asks "evaluate this," they always want an answer — even if the answer is "I couldn't evaluate this and here's why." A skill optimised for human use therefore has no termination conditions for edge cases. It produces output in all circumstances.

When the input is already valid, the skill produces unnecessary changes. When the input is out of domain, the skill translates it rather than rejecting it. When the input contains a contradiction, the skill documents the contradiction in a comment rather than refusing to proceed.

Each of these produces output that satisfies the output contract. Correct field names. Correct format. Correct structure. A downstream agent cannot distinguish this output from a legitimate improvement. The output looks like a skill succeeded. The downstream action proceeds. The error only becomes visible when a test fails for a user ID that was silently changed, or when an engineer asks why authentication infrastructure was built when it was never in scope.

A human-friendly skill is dangerous at agent scale not because it produces wrong output — it produces output that looks indistinguishably right — but because the mechanism by which it produces wrong output is exactly the same as the mechanism by which it produces correct output: it always gives you something useful, and never tells you when useful is the wrong thing to give.

Next issue: The Skill Review — what code review looks like when the review target is the skill, not the diff. A PR template, a checklist, and a real review of the reinforced skill from this issue.

Sources & Further Reading