I want to start with an admission. Six weeks ago, if you had asked me to explain the difference between “agent harness engineering” and “loop engineering,” I would have given you a confident, slightly wrong answer. I had read the phrases enough times on Twitter and in Medium posts that they felt familiar, and familiarity tricked me into thinking I understood them. I didn’t.
What broke the illusion was a small, embarrassing failure. I was building a research agent that pulled papers from arXiv, summarized them, and wrote a weekly digest. It worked beautifully in my first three test runs and then quietly started hallucinating summaries for papers it had never actually opened, because a rate-limited API call failed silently and nothing in my code noticed. The model wasn’t the problem. My prompt was fine. The plumbing around the model had no way to tell “I successfully read this paper” from “I made something up that sounds like I read this paper.” That gap, I eventually learned, has a name, and it isn’t a prompting problem at all.
So I did what I usually do when I get burned: I went and read everything I could find, then rebuilt the same agent three times, once optimizing purely for environment design, once for feedback loops, and once for explicit workflow control. This article is what came out of that process. It runs long on purpose, because the topic deserves more than a listicle, and because I kept finding that the popular explanations out there (including the one that sent me down this path in the first place) describe the shape of these three ideas without ever showing you what they cost to build or where they break.
I’m not going to pretend these three terms are perfectly settled vocabulary. The field is maybe eighteen months old in its current form and people are still arguing about naming. But the underlying distinctions are real, they map to real engineering decisions, and by the end of this piece you should be able to look at a broken agent and know which of the three layers to go fix first.
Why the model was never the whole story
A language model, by itself, is a function that turns text into text. It has no memory between calls unless you give it one. It cannot open a file, run a shell command, or hit an API unless you build the plumbing that lets it. It cannot tell whether its own output is correct unless something outside the model checks. Everyone building agents in 2024 discovered this the hard way, usually by watching a demo work perfectly and then fall apart the moment a real user did something unexpected.
The industry response has been to stop treating “the agent” as a single artifact and start treating it as three separable engineering problems stacked on top of the model:
LAYER QUESTION IT ANSWERS FAILS AS
----------------------------------------------------------------------
Harness What can the agent see and touch? Missing tools,
(environment, tools, memory, lost state, blown
permissions, execution limits) budgets, leaks
----------------------------------------------------------------------
Loop How does one attempt turn into a Confident wrong
correct one? (retry, verify, stop) answers, infinite
spinning, silent
failure
----------------------------------------------------------------------
Graph What is the actual shape of the work? Wrong step order,
(sequence, branches, parallel paths, no recovery path,
human checkpoints) no visibility
----------------------------------------------------------------------
I’ll go through each one the way I actually learned it: by building the smallest possible version, breaking it, and then reading the research and framework documentation to understand why it broke.
Layer 1: the harness, or “what is the agent actually allowed to do”
The clearest one-line definition I found, and I now think it’s the correct one, comes from a LangChain engineering note that reduces the whole idea to an equation: agent = model + harness. The model supplies reasoning. The harness supplies everything the reasoning needs in order to touch the real world: tools, memory, context, permissions, and the guardrails that keep it from doing something expensive or dangerous.
Before I understood this, my mental model of “giving an agent tools” was basically “write some Python functions and describe them in the system prompt.” That works for a demo. It does not survive contact with a task that takes longer than one context window, or a tool call that fails at 2am with nobody watching.
What a harness actually has to manage
Through trial and error (mostly error), I found the harness responsible for six things:
Context injection. Deciding what the model sees before each reasoning step. Not just the user’s message, but retrieved documents, prior conversation, project state, and organizational policy. I over-corrected on this at first, dumping my entire file tree and git log into every call, and watched the model’s accuracy degrade because it was drowning in irrelevant tokens. Less, curated context consistently beat more, raw context.
Action surfaces. The actual mechanism by which the model’s decision becomes a real-world effect: an API call, a shell command, a database write, a browser click. This is where the Model Context Protocol (MCP) lives, and it’s worth pausing on because it changed meaningfully this year. MCP started in late 2024 as a way to standardize how an agent discovers and calls tools, so you didn’t have to write a bespoke integration for every API. As of the July 2026 specification, MCP moved to a stateless core that runs behind ordinary load balancers instead of requiring sticky sessions, added a Tasks extension (contributed by AWS) for long-running work that outlives a single request, and tightened its authorization model around standard OAuth and OpenID Connect flows. There are now more than ten thousand public MCP servers running in production, and monthly SDK downloads are past 97 million. If you’re designing a harness today and you’re not building your tool layer on top of MCP or something with equivalent guarantees, you’re probably reinventing a worse version of it.
Persistence. Anything the agent needs to survive a restart, a crash, or simply the end of a context window: checkpoints, session state, vector memory, git history. My research agent had none of this in its first version, which meant a dropped connection mid-run silently threw away forty minutes of work. Adding a simple JSON checkpoint after every completed sub-task fixed more reliability problems than any prompt change I tried.
Execution control. Retry policy, timeouts, token and cost budgets, model routing, how many sub-agents can spawn, and where a human has to approve before the agent proceeds. This is the layer that keeps an agent from turning a $2 task into a $200 one because it got stuck in a call-and-fail cycle overnight.
Safety and governance. Least-privilege tool access, secret handling, sandboxing, audit logs, allowlists. I sandbox anything that executes code the model wrote, no exceptions, because I’ve personally watched a coding agent try to rm -rf a directory it misidentified as a build artifact. It wasn't malicious. It was just wrong, and it had permission to be wrong destructively.
Observability. Traces, tool call logs, token usage, latency, and evaluation results. This one sounds boring until the day your agent starts behaving strangely in production and you have no record of what it actually did versus what you assumed it did.
A minimal harness, built to actually run
Here’s a stripped-down but functional harness pattern in Python. It uses a local Ollama model so you can run the whole thing without an API key, but swapping in a hosted model is a one-line change.
import json
import subprocess
import requests
from datetime import datetime
OLLAMA_URL = "http://localhost:11434/api/chat"
MODEL = "llama3.1"
# --- Action surface: the only two things this agent is allowed to do ---
def read_file(path: str) -> str:
with open(path, "r") as f:
return f.read()[:4000] # cap it, don't blow the context window
def run_tests() -> str:
result = subprocess.run(
["pytest", "-q"], capture_output=True, text=True, timeout=60
)
return result.stdout[-2000:] + result.stderr[-2000:]
TOOLS = {"read_file": read_file, "run_tests": run_tests}
TOOL_SCHEMA = [
{
"type": "function",
"function": {
"name": "read_file",
"description": "Read a file from the project",
"parameters": {
"type": "object",
"properties": {"path": {"type": "string"}},
"required": ["path"],
},
},
},
{
"type": "function",
"function": {
"name": "run_tests",
"description": "Run the project's pytest suite",
"parameters": {"type": "object", "properties": {}},
},
},
]
# --- Persistence: cheap but real ---
def checkpoint(state: dict, path: str = "checkpoint.json"):
with open(path, "w") as f:
json.dump({**state, "saved_at": datetime.utcnow().isoformat()}, f)
# --- Execution control: hard ceilings, not suggestions ---
MAX_TOOL_CALLS = 8
def harness_run(user_task: str):
messages = [{"role": "user", "content": user_task}]
calls_made = 0
while calls_made < MAX_TOOL_CALLS:
response = requests.post(
OLLAMA_URL,
json={"model": MODEL, "messages": messages, "tools": TOOL_SCHEMA, "stream": False},
).json()
msg = response["message"]
messages.append(msg)
if not msg.get("tool_calls"):
return msg["content"] # model is done, no more tools needed
for call in msg["tool_calls"]:
name = call["function"]["name"]
args = call["function"].get("arguments", {})
if name not in TOOLS:
result = f"error: tool {name} is not permitted"
else:
result = TOOLS[name](**args)
messages.append({"role": "tool", "content": str(result)})
calls_made += 1
checkpoint({"messages": messages, "calls_made": calls_made})
return "stopped: hit the tool call ceiling before finishing"
Nothing here is exotic. That’s the point. A harness isn’t a clever trick, it’s disciplined bookkeeping around a model that has none of its own. If you want to run this against a hosted model instead of Ollama, swap the requests.post block for the OpenAI or Anthropic SDK call and keep everything else identical, since the tool schema and the checkpoint logic don't care which model is answering.
Where I’ve seen harnesses fail in practice
Almost every “the model got dumber” complaint I’ve debugged turned out to be a harness problem instead: a tool that silently returned stale data, a context window stuffed with irrelevant history, a permission that was too broad or, just as often, too narrow so the agent kept trying and failing to do something it was never allowed to do in the first place. Before you touch your prompt or swap your model, check the harness. It’s usually the harness.
Layer 2: the loop, or “how does one attempt become a correct one”
If the harness is what the agent can touch, the loop is how it gets from a first attempt to a good one. This is the layer I underestimated the most going in, because “just let it retry” sounds trivial until you actually watch an agent retry.
The trap: confidence instead of evidence
Here’s the failure mode that got me. My first version of a “self-correcting” loop asked the model, after producing an answer, “are you confident this is correct?” It always said yes. Of course it did. A model grading its own homework with no external signal will pass itself almost every time, because it has nothing to fail against. This turns out to be a well-documented problem, not just my own mistake: a loop with no evidence to fail against will always think it succeeded.
The fix is to stop asking the model to judge itself and instead give the loop something outside the model to check against: a test suite that passes or fails, a schema validator, a compiler, a lint rule, a second independent retrieval that either confirms or contradicts the first answer. Evidence-driven loops check the state of the world. Confidence-driven loops check the model’s mood.
The Ralph Wiggum technique, and why something that crude actually works
One of the more interesting things I ran into while researching this was a pattern that has genuinely become popular under the name “Ralph Wiggum,” first described by engineer Geoffrey Huntley in 2025. In its purest form it’s almost insultingly simple: a bash loop that feeds the same prompt to a coding agent over and over until the task is done, with progress persisted in files and git commits rather than in the model’s own memory of the conversation.
#!/bin/bash
# ralph.sh - crude but surprisingly effective
while true; do
claude -p "$(cat PROMPT.md)" --dangerously-skip-permissions
if git diff --quiet HEAD~1 HEAD -- STATUS.md && grep -q "DONE" STATUS.md; then
echo "Task complete."
break
fi
sleep 5
done
I was skeptical the first time I read about this, because it looks like it shouldn’t work. What I found after trying it on a small refactor is that its crudeness is the feature. By throwing away the conversational context every loop and forcing progress to live in git history and status files, the technique sidesteps context rot and prevents the agent from talking itself into believing it already finished something it didn’t. It’s not elegant. It’s evidence-driven by accident, because git and the filesystem don’t lie to flatter the model.
A loop that actually verifies
Here’s a small evidence-driven loop, again runnable locally against Ollama, that treats a failing test as the only acceptable definition of “not done yet”:
def evidence_loop(task_description: str, max_attempts: int = 5):
attempt = 0
history = []
while attempt < max_attempts:
attempt += 1
code = generate_patch(task_description, history) # calls the model
apply_patch(code)
test_output = run_tests() # external evidence
passed = "failed" not in test_output.lower()
history.append({
"attempt": attempt,
"patch": code,
"test_output": test_output,
"passed": passed,
})
if passed:
return {"status": "success", "attempts": attempt}
# feed the *actual failure*, not a vague "try again"
task_description = (
f"{task_description}\n\nPrevious attempt failed with:\n{test_output}\n"
"Fix the specific failure above."
)
return {"status": "gave_up", "attempts": attempt, "history": history}
The important detail is the last line before the return: the loop doesn’t say “try harder.” It hands the model the actual test failure, because specific, external evidence is what turns attempt two into an improvement on attempt one instead of a random re-roll. I’ve found the single biggest reliability jump in any agent I’ve built came from replacing “did that seem right?” with a real external check, whatever form that check takes for the task at hand.
And know when to stop. A loop without a hard ceiling on attempts, cost, or wall clock time is not a robust agent, it’s a bug waiting for a bill.
Layer 3: the graph, or “what is the actual shape of the work”
The harness gives the agent a body. The loop gives it a way to improve one attempt at a time. Neither one tells you anything about the shape of a task that has real branches in it: do this, then depending on the result do one of three different things, then wait for a person to sign off, then run two independent checks in parallel and merge their results.
That’s graph engineering, and it’s the layer I resisted the longest, because for a while I thought “just let the loop handle it” was good enough. It isn’t, once the task stops being a single linear grind toward one test passing.
Why “just loop harder” breaks down
A loop is good at one thing: repeating attempt, verify, revise until a single condition is met. It’s a poor fit for a task where the correct next step depends on which of several outcomes just happened, where two subtasks can run at the same time, or where a human needs to approve something before the agent is allowed to continue. You can jam all of that into a single prompt and hope the model tracks it in its head, and for a while people did exactly that. It’s fragile. The moment the task is complex enough to need real branching, you want the control flow written down somewhere the model isn’t inventing it fresh each turn.
Graph engineering makes the workflow explicit: nodes are steps (an agent call, a tool call, a validator, a human checkpoint), edges are the conditions that move you from one node to the next. You can look at the graph and know exactly which path an execution took, which matters enormously the first time you’re debugging a production incident at midnight and need to know precisely what happened, not what you assume happened.
What this looks like in practice
LangGraph is the framework most people reach for here, and its model is a directed graph with typed state passed between nodes, conditional edges for branching, and built-in checkpointing so a run can pause, resume, or even roll back to an earlier state.
from langgraph.graph import StateGraph, END
from typing import TypedDict
class AgentState(TypedDict):
task: str
draft: str
review_passed: bool
attempts: int
def draft_node(state: AgentState) -> AgentState:
state["draft"] = generate_draft(state["task"])
return state
def review_node(state: AgentState) -> AgentState:
state["review_passed"] = run_validator(state["draft"])
state["attempts"] += 1
return state
def needs_human(state: AgentState) -> str:
if state["review_passed"]:
return "approved"
if state["attempts"] >= 3:
return "escalate" # branch: give up automating, ask a human
return "retry"
graph = StateGraph(AgentState)
graph.add_node("draft", draft_node)
graph.add_node("review", review_node)
graph.set_entry_point("draft")
graph.add_edge("draft", "review")
graph.add_conditional_edges(
"review",
needs_human,
{"approved": END, "retry": "draft", "escalate": "human_approval"},
)
Notice what this buys you that a plain loop can’t express cleanly: three distinct outcomes from a single check (done, try again, stop and ask a person), each with its own path. That “escalate” branch is doing real work. It’s the difference between an agent that silently fails after its third bad attempt and one that hands the problem to a human with full context on what it already tried.
Microsoft’s AutoGen shipped a comparable capability called GraphFlow this year for the same reason: once you’re coordinating more than one agent, or a workflow with real branches, an implicit loop stops being legible, and a graph gives you something you can actually inspect.
Picking a framework
I tried four of these on the same small project (a document-review pipeline with a validation branch and a human approval gate) to get a feel for where each one is actually strong, rather than trusting the marketing copy.
FRAMEWORK ORCHESTRATION MODEL STRONGEST AT STATE / PERSISTENCE
--------------------------------------------------------------------------------------------
LangGraph Directed graph, typed Complex branching, Built-in checkpointing,
state, conditional edges long-running workflows, time-travel debugging
human-in-the-loop
--------------------------------------------------------------------------------------------
CrewAI Role-based "crew" of Fast prototyping, Sequential task output
agents, sequential or multi-agent brainstorm passing, lighter state
hierarchical process style collaboration model
--------------------------------------------------------------------------------------------
OpenAI Agents SDK Explicit agent handoffs Minimal-friction start Ephemeral context
inside the OpenAI variables by default
ecosystem
--------------------------------------------------------------------------------------------
AutoGen / GraphFlow Multi-agent conversation Agent-to-agent Conversation history
plus optional graph layer negotiation, graph- plus graph state
for explicit control based orchestration
--------------------------------------------------------------------------------------------
My honest takeaway: if your workflow fits on one page as a flat list of steps, you don’t need a graph framework, a well-built loop inside a solid harness will do the job with less code to maintain. The moment you draw your workflow and it has a fork in it, a graph framework starts paying for itself immediately, mostly through the debugging time it saves you rather than anything it does at run time.
How the three layers actually sit on top of each other
It took me longer than I’d like to admit to see that these aren’t competing approaches, they’re vertically stacked. Here’s how I’d draw the stack after having now built with all three:
+--------------------------------------------------------------+
| GRAPH - decides which node runs next, where branches |
| are, where a human has to approve |
+--------------------------------------------------------------+
| LOOP - inside a given node, drives attempt -> verify -> |
| revise until the evidence says stop |
+--------------------------------------------------------------+
| HARNESS - underneath both, provides the tools, memory, |
| permissions, and limits that make any of it |
| possible to execute safely |
+--------------------------------------------------------------+
| MODEL - the reasoning engine none of the above exists |
| without |
+--------------------------------------------------------------+
A single “draft” node in a LangGraph workflow might internally run an evidence-driven loop for a few attempts before handing control back to the graph. That loop, in turn, is only possible because the harness underneath it gave the model a way to run tests, read files, and remember what it tried last time. Pull any one layer out and the other two stop working, which is exactly why treating “the agent” as a single thing to prompt harder was always going to hit a ceiling.
A quick diagnostic for when your agent is misbehaving
Once I started thinking in these three layers, debugging got noticeably faster. This is roughly the checklist I now run through, in order, before I touch a prompt:
SYMPTOM LIKELY LAYER TO CHECK FIRST
----------------------------------------------------------------------
Agent doesn't know something it should Harness -> context injection
Agent "can't" do something it should be Harness -> action surface /
able to do permissions
Agent forgets earlier progress after a Harness -> persistence
restart or long run
Costs or run time spiral unexpectedly Harness -> execution control
Agent confidently reports success on Loop -> swap confidence checks
work that's actually wrong for external evidence
Agent retries the same mistake Loop -> is it seeing the actual
repeatedly without improving failure, or just "try again"?
Agent takes the wrong path when a task Graph -> is the branching logic
has more than one possible outcome explicit, or implicit in a
single prompt?
No human ever gets a chance to catch a Graph -> is there an approval
bad decision before it ships node, or is it all automatic?
I wish I’d had this table six weeks ago. It would have saved me a full weekend of blaming my prompt for what was actually a missing checkpoint.
Where I still have doubts
I don’t want to wrap this up sounding more certain than I am. A few things I’m genuinely unresolved on, having now lived in this space for a while:
The vocabulary is still shifting under our feet. “Harness engineering” as a phrase barely existed at scale before this year, and I would not be surprised if in another year it’s absorbed into a broader “agent engineering” umbrella term, with harness, loop, and graph becoming subheadings rather than standalone disciplines people specialize in.
The Ralph Wiggum technique bothers me a little even though I’ve seen it work. Feeding a model the same prompt in a dumb loop and leaning on git history as the only real memory feels like it’s papering over a harness that should have proper persistence in the first place. I use it for small, well-scoped refactors. I would not trust it, yet, for anything with real branching or real stakes.
And I’m still not sure graph frameworks have found their final shape. LangGraph’s checkpointing is genuinely good, but I’ve watched teams reach for a full graph framework on a workflow that was three steps long and never actually branched, just because it was the trendy choice. If your workflow is a straight line, a straight line is fine.
None of that changes the core claim I’d defend confidently: if your agent is unreliable, the fix is almost never a better prompt. It’s almost always a gap in one of these three layers, and now you know which questions to ask to find out which one.
What I’d build differently next time
If I were starting my research agent over today, knowing what I know now, I’d build the harness first and treat it as boring, load-bearing infrastructure rather than an afterthought. I’d wire in an evidence-driven loop from day one instead of trusting the model’s self-assessment, because that one change alone fixed more bugs than anything else I tried. And I’d only reach for a graph framework once I could actually draw the workflow on paper and see a fork in it, rather than assuming I needed one from the start.
The model got all the attention for the last three years. The engineering around it is where production reliability actually comes from, and it’s a much less glamorous, much more solvable problem than “the model isn’t smart enough.” In my experience, it usually is smart enough. It just wasn’t given the environment, the feedback, or the map it needed to prove it.
Tags: AI Agents, Agent Engineering, LangGraph, MCP Model Context Protocol, Software Architecture, Machine Learning, LLM Development
Top comments (0)