When an AI agent repeatedly fails at a specific task, the default reaction is often to use a better model or fine-tune the current one.
That can be the wrong first move.
The model is only one part of an agent. Its harness controls the context it receives, tools it can use, environment it can inspect, feedback it gets, and whether one failure becomes a durable improvement.
I built Harneloop to make that improvement process structured and verifiable.
The loop
Harneloop asks the operating agent to:
- attempt a real task;
- inspect the resulting artifacts, state, logs, and traces;
- trace likely failures back through the recorded run;
- create an isolated candidate change to the harness;
- retest the candidate and relevant regressions;
- promote the change only when evidence supports an improvement.
The candidate can change more than a system prompt. It can add tools, retrieval, examples, validators, environment automation, skills, or any other task-specific infrastructure.
The working harness remains unchanged while the candidate is tested. Failed experiments do not silently become production state.
Why artifacts matter
A command can succeed while the result is wrong.
An interface can compile but fail visually. A scene can render but violate the requested spatial relationships. A document can be valid but communicate the wrong thing. A browser workflow can finish in the wrong state.
For these tasks, the artifact is part of the evidence.
Harneloop links artifacts and observations to the exact run that produced them. That gives the agent a way to move from “this looks wrong” to a trace-backed hypothesis about what should change in the harness.
Portable harness units
Each task-specific development environment is stored as a harness unit.
A harness unit carries the target definition, operational map, reusable harness material, candidates, evidence references, regression cases, and promoted versions. The agent can add custom scripts, research, retrieval data, infrastructure, or entirely new directories when necessary.
Harneloop protects the lifecycle rather than putting the agent in a rigid template.
Successful units can be paused, restored, moved, packaged, or exported into another agent or application.
Why harness-first?
Research has repeatedly shown that inference-time systems can change agent performance without changing model weights.
SWE-agent demonstrated the impact of a purpose-built agent-computer interface. Reflexion and Self-Refine showed gains from feedback and iterative refinement without additional training. The 2026 Self-Harness paper directly demonstrated agents mining weaknesses from traces, proposing harness changes, and validating them with regression tests.
This does not mean harnesses always beat fine-tuning. Fine-tuning can still be useful for behavior, style, latency, or capabilities that cannot be supplied effectively at inference time.
The practical rule is narrower:
When the bottleneck is context, tools, retrieval, feedback, validation, memory, or environment interaction, test the harness before changing the weights.
Initial evidence
Harneloop was first used to develop ViperMesh, a Blender harness for spatial reasoning and scene generation.
The benchmark held the acting model fixed at OpenAI GPT 5.5 High and compared ViperMesh with the Anthropic x Blender MCP server baseline.
Across seven comparable live tasks, ViperMesh:
- was faster on 6 of 7 tasks;
- achieved a 2.534x mean speedup;
- improved preliminary neutral LLM visual evaluation by 8.19 points.
Harneloop did not generate the scenes. It structured the process used to inspect failures, improve the harness, preserve evidence, and verify whether changes helped.
The full ViperMesh case study includes methodology and limitations. It is evidence from one task family, not a universal harness-versus-fine-tuning benchmark.
Try it on a real task
Harneloop is optimized for agent-led setup. Give the repository to a capable coding agent and describe what you want the harness to improve. The agent can install the framework, map the environment, create the first harness unit, and ask for missing information when needed.
I am looking for independent use cases, including unsuccessful ones. The most useful feedback concerns onboarding, environment mapping, evidence quality, runtime cost, candidate decisions, and tasks where the harness reaches its limit.
Repository: https://github.com/Ker102/Harneloop
Case study: https://www.kristoferjussmann.me/case-studies/vipermesh
Top comments (0)