A hands-on look at the Agent Harnesses standard, and why the word “harness” means three different things right now
A few days ago I read a short piece by Daniel Warfield called “Towards Self-Repairing and Repeatable AI Systems.” It’s a progress update on something he calls the Agent Harnesses standard, a way of organizing folders so that AI coding agents like Claude can find the right context without getting lost. The title promised something big: systems that repair themselves and behave the same way twice. The article itself, behind Medium’s paywall, gives you about a paragraph and a half before it cuts off.
That combination, a big claim and a locked door, is exactly the kind of thing I don’t like taking on faith. So instead of summarizing the abstract, I spent an afternoon doing the boring version of due diligence: I read the actual specification, installed the actual CLI tool, ran it in a sandbox, opened every file it generated, and went and found the research papers that use the phrase “self-repairing harness” in a much more literal sense than Warfield does. What follows is what I found, what worked, what didn’t, and where I think the framing oversells the mechanism a little.
If you only read one paragraph of this, read this one: Agent Harnesses is a genuinely useful, very young naming convention for organizing markdown files so an agent can navigate a large project without flooding its context window. It is not, today, a system that watches an agent fail and rewrites its own logic in response. That second thing exists too, it’s called Self-Harness and HarnessX in the academic literature, and it’s a completely different and much heavier piece of machinery. Conflating the two is easy to do because everyone is using the word “harness” for different things this year. Let’s untangle it.
Step one: what a “harness” even is
Before Agent Harnesses (capital A, capital H, a specific standard), there’s just “harness” as a generic term that’s been floating around AI engineering circles since early 2026. LangChain popularized a simple equation for it: Agent = Model + Harness. The model does the reasoning. The harness is everything else: the system prompt, the tool definitions, the memory management, the loop that decides when to call a tool and when to stop.
Birgitta Böckeler at Thoughtworks wrote a longer treatment of this idea for Martin Fowler’s site in April, and I think her framing is the clearest one I’ve come across. She splits a harness into two directions of control:
GUIDES (feedforward) SENSORS (feedback)
----------------------- -----------------------
Steer the agent BEFORE Observe the agent AFTER
it acts. Increase odds it acts. Let it self-
of a good first attempt. correct.
Examples: Examples:
- AGENTS.md / CLAUDE.md - Linters with LLM-
- Skills readable error messages
- Reference docs - Structural/architecture
- Bootstrap scripts tests
- Code review agents
and a second axis, computational versus inferential:
COMPUTATIONAL INFERENTIAL
----------------------- -----------------------
Deterministic, fast, run Semantic, slower, run by
by the CPU. Tests, type a model. AI code review,
checkers, static analysis. "LLM as judge," rules
Cheap enough to run on written from patterns.
every single change. Non-deterministic results.
Her point, which I think holds up, is that a good harness needs both directions and both types. An agent that only gets feedforward guidance repeats the same mistake forever because nothing ever tells it the mistake happened. An agent that only gets feedback sensors has no way to avoid the mistake on the first try.
This is the wide definition of “harness.” It covers everything from a CLAUDE.md file to a CI pipeline running mutation testing. Warfield’s Agent Harnesses standard is a much narrower thing living inside this wide definition: it’s specifically about the feedforward side, and specifically about the discovery problem, how does an agent find the right guide out of hundreds of possible guides without reading all of them.
Step two: the foundation it’s built on, Agent Skills
You can’t understand Agent Harnesses without understanding Agent Skills first, because Harnesses is explicitly an extension of it, not a replacement.
Agent Skills is a format Anthropic released as an open standard. A skill is a folder with a SKILL.md file in it, plus optionally some scripts, templates, or reference material:
send-email/
├── SKILL.md
└── scripts/
└── send_email.py
The trick is progressive disclosure, done in three stages:
- Discovery : at startup, the agent loads only the name and one-line description of every skill it has access to. This costs almost nothing in context.
- Activation : when a task matches a skill’s description, the agent reads the full SKILL.md into context.
- Execution : the agent follows the instructions, running bundled scripts or reading referenced files as needed.
I went and checked how far this has actually spread, because “open standard” claims are cheap and adoption is the thing that matters. As of this month, the client showcase on agentskills.io lists a genuinely long roster: Claude and Claude Code, Cursor, GitHub Copilot, VS Code, Gemini CLI, OpenCode, OpenHands, Goose, JetBrains’ Junie, Amp, Letta, Factory, Roo Code, ChatGPT and Codex, and around two dozen smaller tools. That’s a real ecosystem, not a one-vendor gimmick. So the layer underneath Agent Harnesses is solid ground.
Where Skills falls short, and this is the actual problem Warfield is solving, is sequencing and grouping. Claude Code organizes skills in a flat directory. If you have three or four skills that’s fine. If you have thirty, and five of them only make sense in a specific order (look up the user before you look up their purchase history before you send them an email), there’s no standard way to express that intent. The model has to infer it from descriptions alone, every time, from scratch.
Step three: what Agent Harnesses actually adds
The core idea is one file convention: routing files. Every top-level subdirectory in a harness gets a markdown file named after that directory, in all caps, that tells the agent what’s inside and when to look.
Here’s the example from the spec itself:
my-harness/
├── HARNESS.md
├── tools/
│ ├── TOOLS.md
│ ├── backend/
│ │ ├── TOOLS.md
│ │ └── create-api/
│ └── frontend/
│ ├── TOOLS.md
│ └── build-ui/
└── data/
├── DATA.md
├── schemas/
│ ├── DATA.md
│ └── table-definitions.md
└── quirks/
├── DATA.md
└── known-issues.md
HARNESS.md sits at the root and is always loaded in full when the harness starts. It's the agent's identity document, this is who you are, this is the role you're filling. From there, instead of reading every file in tools/ and data/, the agent reads TOOLS.md or DATA.md first, decides whether that branch is relevant to the current task, and only then goes deeper. It's progressive disclosure applied one level higher than Skills applies it, at the level of an entire branch of the project instead of a single capability.
Naming convention aside, this is a genuinely sensible pattern, and honestly it’s one a lot of engineers were probably already doing by hand with README files before anyone gave it a name. What the standard adds isn’t the idea, it’s a consistent name (always HARNESS.md at the root, always .md for the routing files) that a tool can be built around. Consistency is the entire value proposition of a standard, so that part checks out.
Step four: I actually installed it
Reading a spec only tells you what’s supposed to happen. So I spun up a clean sandbox, installed the CLI, and ran it for real.
pip install agentharnesses-cli --break-system-packages
The package installs a command called ahar. My sandbox happened to be running Python 3.10, and the version resolver quietly gave me agentharnesses-cli 0.1.3 instead of the current 0.1.6, because the newer release depends on a package called harnesses-ref that requires Python 3.11 or newer. That's a small thing, but it's the kind of small thing a "long investigation" is supposed to catch: the PyPI page documents four commands (init, validate, read, prompt), and on the version I could actually install, only init exists. If you're on an older Python and just skim the README, you'll go looking for ahar validate and find nothing. Worth knowing before you build a workflow around it.
ahar init itself worked cleanly:
mkdir my-harness && cd my-harness
ahar init customer-support-harness
It asks one interactive question, which Claude Code preset you want, then scaffolds this:
customer-support-harness/
├── HARNESS.md
├── README.md
├── .gitignore
├── .claude/
│ ├── settings.json
│ └── skills/agent-harnesses/ <- the "metaskill"
│ ├── SKILL.md
│ ├── .leaf-detectors
│ ├── scripts/
│ │ ├── disclose.py
│ │ ├── reverse_disclose.py
│ │ ├── summarize.py
│ │ └── map_references.py
│ └── sessions/
├── skills/
│ ├── SKILLS.md
│ └── maintenance/
│ ├── SKILLS.md
│ └── modify-harness/
│ └── SKILL.md
└── references/
└── REFERENCES.md
The generated HARNESS.md is mostly TODO placeholders, which is fine, it's a scaffold, not magic:
---
name: customer-support-harness
description: TODO: describe what this harness does and the role it gives Claude.
---
## Upon loading the Harness
TODO: write the entry message Claude should internalize when this harness loads.
## How to Find Information for Claude
Use the `agent-harnesses` skill to explore the harness just in time,
based on prompts from the user. Select only what is relevant and
repeat until the session is complete, then read the returned resources.
When **maintaining the harness** (adding, moving, or renaming files),
consult the `agent-harnesses` skill for reverse progressive disclosure
to keep routing files in sync.
That last paragraph is the actual interesting part, and it’s the closest thing to “self-repair” that exists in the tool today. Let’s look at it directly.
Step five: where the “self-repairing” part actually lives
The modify-harness skill that gets scaffolded in is short enough to quote in full:
---
name: modify-harness
description: Update harness structure files, HARNESS.md, SKILLS.md
indexes, REFERENCES.md, to keep routing and descriptions accurate
as the harness evolves.
---
## Role
Keep the harness self-consistent when skills or references are
added, renamed, or removed.
## What to do
1. Use reverse progressive disclosure (via the agent-harnesses skill)
to find which index files reference the target path
2. Read the current state of each affected file
3. Apply the change: add, update, or remove the relevant entry
4. Ensure descriptions remain accurate and routing summaries
reflect actual contents
“Reverse progressive disclosure” is implemented in a script called reverse_disclose.py, which does something simple and clever: given a file path, it walks upward through the directory tree and finds every routing markdown file that references it, so that when you rename or move something, Claude can find and fix every stale pointer instead of leaving broken references scattered through the project.
There’s also a leaf detection mechanism I hadn’t seen described anywhere else. A directory in a harness is either a plain group or a “leaf” of some named type (a skill, an MCP server, whatever you define). Classification happens two ways: an explicit .harnessleaf file inside the directory, or a .leaf-detectors config, inherited from the nearest ancestor, that maps a type name to a marker filename:
# .leaf-detectors
skill=SKILL.md
mcp-server=MCP-SERVER.md
If a directory contains SKILL.md, it's automatically treated as a skill leaf. This is a small piece of design that I actually like, it means the classification is structural and inspectable rather than something the model has to guess at from context every single session.
So here’s my honest assessment of the word “self-repairing” as it’s used in Warfield’s title: what’s happening is that Claude, prompted by a skill file, walks the tree, finds stale references, and edits them. That is a real and useful behavior. It is also entirely dependent on the model being told to do it and choosing to do it correctly in that session. There’s no automatic trigger, no test suite that fails and forces a repair, no verification step that confirms the repair was correct. It’s closer to “I gave my intern a very good checklist for tidying the filing cabinet” than to a system that detects its own damage and heals it. That’s not a knock, a good checklist is genuinely valuable, but it’s worth being precise about what kind of “self” and what kind of “repair” we’re actually talking about, especially because a very different and much more literal version of self-repair exists in the research literature right now, using almost identical language.
Step six: the other “self-repairing harness,” the one that actually rewrites itself
While I was digging around, I found two research frameworks that use “self-repairing” or “self-improving” to describe something structurally different from anything above: a system where an agent mines its own failure traces and edits its own execution logic based on measured evidence, then validates that the edit didn’t break anything else before keeping it.
The first is called Self-Harness. It runs a three-stage loop:
1. WEAKNESS MINING
Run the agent against an eval dataset.
Log every tool call, error, and response.
Identify model-specific failure patterns
(not generic bugs, patterns specific to
THIS model's behavior).
2. HARNESS PROPOSAL
The agent proposes a minimal, targeted
fix to its own scaffold: a prompt tweak,
a new rule, a code change to the harness.
3. PROPOSAL VALIDATION
Regression-test the new harness against
the FULL eval set, not just the failing
case. If the fix breaks something that
used to pass, reject it.
On Terminal-Bench-2.0, the researchers describe the loop discovering that a model kept issuing duplicate shell commands and losing track of files it had already created. Instead of a person noticing this and hand-writing a patch, the loop generated new executable rules on its own: a strict no-duplicate-command policy, a mechanism forcing the agent to recreate missing artifacts when it hit file errors, and instructions to persist environment variables across shell sessions. The reported result was a jump from a 40.5% to a 61.9% pass rate for MiniMax M2.5, with no change to the model’s weights at all, entirely from evolving the harness around it.
The second is HarnessX , out of a research group at Xiaomi, which goes a step further by treating the harness as a set of independently swappable software components (context assembly, memory management, tool ecosystem, control flow) and running a four-stage evolution engine over them called AEGIS:
DIGESTER -> finds exactly where the harness failed, from traces
PLANNER -> proposes a high-level fix strategy
EVOLVER -> writes the actual code edit, tests it in isolation
CRITIC -> checks for reward hacking, gates against regression
The genuinely novel piece here is what they call harness-model co-evolution: instead of only editing the harness, or only fine-tuning the model, they interleave both through a shared replay buffer using a reinforcement learning method called Group Relative Policy Optimization, which scores a batch of candidate outputs against each other’s average rather than against a fixed reward model. Every time the harness’s structure improves, the model gets trained on data that teaches it to actually exploit the new structure. The reported numbers: harness evolution alone produced a 14.5% average gain across ALFWorld, GAIA, and SWE-bench Verified, and adding the model co-evolution step on top of that added another 4.7%. Interestingly, the researchers found smaller open-weight models like Qwen 9B gained the most from this, which is a real point in favor of the “you don’t need a bigger model, you need a better-fitted harness” argument.
I want to be fair to Warfield here: he never claims his Agent Harnesses standard does trace mining or reinforcement learning. But the title “Towards Self-Repairing and Repeatable AI Systems” sits right next to research that does exactly that, using the same vocabulary, and a reader skimming Medium is going to walk away thinking they’re the same category of thing. They’re not. One is a file-naming convention with a cleanup checklist. The other is closed-loop reinforcement learning over execution traces with statistically measured regression gates. Both are useful. They are not the same weight class.
A side-by-side, because I think this is the actual confusion
| Böckeler / Fowler | Warfield's Agent | Self-Harness /
| "harness engineering"| Harnesses standard | HarnessX (research)
--------------------|----------------------|------------------------|----------------------
What it regulates | Code quality, | Context discovery, | The agent's own
| architecture, | which files the | execution logic and
| behavior | agent should read | (optionally) weights
--------------------|----------------------|------------------------|----------------------
Mechanism | Guides + sensors, | HARNESS.md + all-caps | Trace mining ->
| feedforward + | routing files, leaf | proposal -> automated
| feedback loops | detection | regression validation
--------------------|----------------------|------------------------|----------------------
Who does the | Human, with agent | Human sets structure, | The system itself,
"repairing" | assistance | agent (when prompted) | autonomously, gated
| | tidies references | by eval scores
--------------------|----------------------|------------------------|----------------------
Maturity | Practitioner | v0.1.x, single | Peer-reviewed papers,
| consensus forming | maintainer, weeks old | open-sourced code
--------------------|----------------------|------------------------|----------------------
Where to look | martinfowler.com | agentharnesses.io | arXiv 2606.09498
| /articles/harness- | | (Self-Harness),
| engineering.html | | arXiv 2606.14249
| | | (HarnessX)
Trying it without the CLI at all
One thing worth knowing: none of this actually requires the ahar tool. The whole standard is just markdown files in folders with a naming convention. If you're wary of adding a single-maintainer PyPI package to your toolchain (I would be, at v0.1.x), you can hand-roll the same structure and get most of the benefit. Here's a plain Python script that reproduces the "summarize" behavior of the metaskill, no dependencies, no network calls, works fully offline:
#!/usr/bin/env python3
"""Minimal local harness summarizer. No CLI package required.
Walks a harness directory and prints the routing tree with
descriptions pulled from frontmatter. Point any local model at
the output, including one served by Ollama."""
import sys
from pathlib import Path
def parse_frontmatter(text: str) -> dict:
if not text.startswith("---"):
return {}
end = text.find("---", 3)
if end == -1:
return {}
meta = {}
for line in text[3:end].strip().splitlines():
if ":" in line:
k, _, v = line.partition(":")
meta[k.strip()] = v.strip().strip('"\'')
return meta
def find_routing_file(directory: Path) -> Path | None:
upper_name = directory.name.upper() + ".md"
candidate = directory / upper_name
if candidate.exists():
return candidate
if directory.parent == directory:
return None
return None
def walk(directory: Path, prefix: str = ""):
harness_md = directory / "HARNESS.md"
if harness_md.exists():
meta = parse_frontmatter(harness_md.read_text())
print(f"{prefix}[harness] HARNESS.md - {meta.get('description', '')}")
routing = find_routing_file(directory)
if routing and routing.name != "HARNESS.md":
meta = parse_frontmatter(routing.read_text())
print(f"{prefix}[routing] {routing.name} - {meta.get('description', '')}")
skill_md = directory / "SKILL.md"
if skill_md.exists():
meta = parse_frontmatter(skill_md.read_text())
print(f"{prefix}[skill] SKILL.md - {meta.get('description', '')}")
return
for child in sorted(p for p in directory.iterdir() if p.is_dir()):
if child.name.startswith("."):
continue
print(f"{prefix}+-- {child.name}/")
walk(child, prefix + " ")
if __name__ == " __main__":
root = Path(sys.argv[1] if len(sys.argv) > 1 else ".")
walk(root)
Run it with:
python3 summarize_harness.py ./customer-support-harness
And if you want to test that a fully local model can actually navigate the structure, without sending anything to Claude or any hosted API, pipe the summary into a locally-served model through Ollama:
# one-time setup
ollama pull llama3.1
# feed the harness summary to a local model and ask it to route a task
python3 summarize_harness.py ./customer-support-harness | \
ollama run llama3.1 "Given this harness structure, which branch \
would you open first to handle a task about refund policy? \
Answer with just the path."
This is a genuinely useful sanity check before you commit to a bigger harness: if a 7B or 8B local model, with no special training on the standard, can correctly route a task just from reading the top-level descriptions, that’s decent evidence your routing files are actually doing their job of being self-explanatory rather than requiring a frontier model to disambiguate.
What I actually think after doing this
I like the underlying idea more than I expected to. The problem it’s solving is real, I’ve watched agents burn a huge chunk of a context window re-reading files it already half-understood, or confidently using the wrong tool because two similarly named skills sat in the same flat directory with no signal about which one applied when. A consistent naming convention for routing files, paired with a script that can walk the tree and fix broken references, is a legitimately useful pattern, and I’d rather more people converge on one naming convention than have fifty teams reinvent slightly incompatible versions of the same idea privately.
What I’m less convinced by is the framing. “Self-repairing” implies a closed loop: something breaks, the system notices, the system fixes it, ideally without a human in that particular loop. What Agent Harnesses actually ships today is a very well-designed prompt and a couple of scripts that make it easier for a human-in-the-loop Claude session to keep its own documentation honest, only when someone remembers to ask it to. That’s valuable. It is not the same claim as Self-Harness or HarnessX, which run unattended, measure outcomes against a fixed evaluation set, and mathematically reject changes that regress. If you came away from the original article thinking you could point this standard at a flaky production agent and walk away, I’d pump the brakes. What you can do is give your project a much better filing system, and give Claude a checklist for keeping that filing system honest when you ask it to clean up.
The other thing I’d flag, and this is a smaller point but a real one for anyone deciding whether to build on this today: the CLI is a single maintainer’s PyPI package, six releases old as of this writing, and the version most people will actually get depends on their Python version in a way that isn’t obvious until you hit it. None of that is disqualifying, every standard starts somewhere and Skills itself was a scrappy Anthropic release before two dozen tools adopted it. But “standard” is a word that implies more stability than a project has usually earned in its first six weeks, and I’d treat this one as promising and early rather than settled.
If you’re already using Skills and you’ve hit the flat-directory wall, where you have enough skills that the model starts guessing at which one to use, this is worth trying this week. It’s low-risk: it’s just folders and markdown, you can adopt it incrementally, and you can strip it back out with nothing but a git rm if it doesn't earn its keep. Just don't expect it to repair anything you haven't asked it to look at.
Tags: ai-agents, agent-harnesses, claude-code, llm-engineering, prompt-engineering, context-engineering, agentic-ai
Top comments (0)