why your ai agent fails: it is a harness problem, not a prompt problem

The most common mistake in agent development is believing the prompt is the product.

An agent gets stuck in a loop. The team edits the system prompt.

The agent clicks the wrong thing. The team adds another sentence about being careful.

The agent says the task is done, but nothing actually changed. The team writes, "Before saying you are finished, verify that you are finished."

Then it fails again, just with a more polite explanation.

I get why this happens. Prompts are visible. Prompts are easy to change. Prompts feel like the part of the system where intelligence lives. If the agent behaves badly, it is tempting to assume the instruction was not clear enough or the model was not powerful enough.

But after you build with agents for a while, a different pattern shows up.

The expensive failures are usually not language failures. They are runtime failures.

The model did not know the browser had been redirected to login. The tool returned a partial success and nobody checked the side effect. The loop had no stop condition. The context window filled with junk. The agent retried the same broken action until it ran out of budget. The final answer sounded confident because the model was doing what models do: producing a plausible continuation.

That is not a prompt problem.

That is a harness problem.

the harness is the product

An AI harness is the deterministic software around the model.

It is the loop that decides when the model runs. It is the tool registry. It is the state store. It is the retry policy. It is the permission boundary. It is the verifier that checks whether the intended action really happened. It is the boring code that turns a stochastic reasoning engine into something you can put near production.

If the LLM is the brain, the harness is everything that stops the brain from having to be the operating system, database, browser driver, security policy, and QA team at the same time.

This distinction matters because LLMs are good at interpretation, planning, and fuzzy judgment. They are bad places to put deterministic guarantees.

The model can decide that "upvote this Hacker News post" probably means it should navigate to Hacker News, find the post, and click the upvote arrow. Fine.

The model should not be the only thing responsible for knowing whether the user is logged in, whether the click actually registered, whether the site returned a validation error, whether the same step has already failed five times, and whether the final state matches the original goal.

That is harness work.

Tejas Kumar makes this point nicely in his AI Engineer talk, "Harnesses in AI: A Deep Dive." The interesting part of the framing is not that agents need better tools. Everyone knows agents need tools. The interesting part is that the tools are not enough either. You need the runtime that governs how the tools are selected, executed, observed, and verified.

Without that runtime, even a strong model becomes a very expensive intern with browser access and no checklist.

the demo failure is painfully familiar

The Hacker News example is a good one because it is small enough to understand and real enough to hurt.

The task is simple: log into Hacker News and upvote a specific post.

An unharnessed agent can look competent for the first few steps. It opens the page. It sees the post. It clicks upvote. Then Hacker News redirects it to a login page.

This is where the illusion breaks.

To a human, the state change is obvious. We are not on the post anymore. We are on an authentication screen. The task is now blocked on session state.

To a weak agent loop, this may just look like another page. The model tries to continue. It may click around. It may navigate back. It may retry the upvote. It may burn tokens describing what it thinks happened. Eventually, if the loop has no hard verifier, it may claim success because it has reached the narrative shape of success.

"I have successfully upvoted the post."

No, you have not.

You clicked a thing, got redirected, lost the state, and narrated the happy path.

This is the exact failure mode I see in coding agents too. The agent says it fixed the test, but the test was never run. It says it updated the issue, but the API call failed. It says it opened a PR, but the branch was not pushed. It says it deployed, but the deployment job is red. The English output has no authority unless the system can prove the side effect.

Natural language completion is not completion.

prompts express intent; harnesses enforce reality

A prompt can tell the model what you want.

A harness decides what is allowed, what happened, and whether the job is done.

That is the line I would draw for any production agent.

Use prompts for judgment-heavy work:

• interpret the user's request
• choose between reasonable next actions
• summarize a failure
• decide which file or page looks relevant
• explain tradeoffs

Use harness code for anything that must be true:

• maximum number of steps
• tool permissions
• schema validation
• authentication handling
• retries and backoff
• idempotency
• budget limits
• state persistence
• success verification
• audit logs

The more important the guarantee, the less I want it living only in prose.

This is not anti-prompt. A good prompt still matters. Bad instructions create bad work. But prompts should not carry the weight of infrastructure.

"Do not loop forever" is not a loop control strategy.

"Be careful with credentials" is not a secret management strategy.

"Verify before finishing" is not a verifier.

Those are intentions. Useful intentions, but still intentions.

what a real harness should own

The first thing a harness needs is a controlled loop.

Every agent loop should have a budget: maximum steps, maximum tokens, maximum wall-clock time, and ideally maximum repeated failures for the same action. If the agent tries the same thing three times and gets the same result, the harness should force a state change: inspect, escalate, try a different path, or stop with evidence.

The second thing is external state.

Do not rely on the context window as your source of truth. The context window is a working set, not a database. Keep durable state in a place the harness controls: task status, tool calls, browser URL, auth state, IDs created, files changed, commands run, errors observed, and verification results.

The third thing is tool discipline.

Tools should have typed inputs, typed outputs, clear error states, and narrow permissions. A browser tool should not pretend a click succeeded if the page navigated to an error. A GitHub tool should return the PR URL only after the API confirms creation. A file-editing tool should expose the diff, not just a cheerful message.

The fourth thing is interception.

Some states are not model problems. They are runtime events.

If the browser hits a login page, the harness should recognize that. If an API returns 401, the harness should classify it as auth, not ask the model to philosophize about why access failed. If a rate limit appears, the harness should apply a policy. If a destructive action is requested, the harness should require the right approval path.

The fifth thing is verification.

This is the big one.

The harness should never accept "done" because the model says "done." It should verify against the thing that matters.

For a coding agent, that might mean tests passed, typecheck passed, the diff only touches expected files, and the application still starts.

For a browser agent, that might mean a DOM state changed, a network request returned success, or a record appeared in the backend.

For a messaging agent, that might mean the message was actually sent by the platform API and the returned ID was recorded.

For a deployment agent, that might mean the new version is live and the health endpoint returns the expected response.

If you cannot verify it, the correct status is not success. It is unverified.

smaller models get much better when the harness is good

One of the more uncomfortable truths for AI product teams is that some "model quality" problems are really "we built a sloppy runtime" problems.

A stronger model can mask bad harness design for a while. It may recover from more weird states. It may infer missing context. It may be less likely to repeat itself. It may write a better apology when things go wrong.

But that is an expensive way to compensate for missing engineering.

If the harness handles auth, state, retries, tool schemas, and verification, the model's job gets smaller and cleaner. Smaller models become viable for more workflows. Latency drops. Cost drops. Behavior becomes easier to reason about because the deterministic parts are actually deterministic.

This is the part many teams miss. They jump from "the agent failed" to "we need a frontier model" when the real fix is often:

• stop relying on context as memory
• stop treating tool output as trustworthy without validation
• stop letting the loop run until vibes improve
• stop allowing natural language to define completion
• stop giving the model responsibility for infrastructure state

Use the expensive model where judgment matters. Use code where guarantees matter.

That is not a philosophical preference. It is just cheaper and less fragile.

the architecture I would start with

If I were building a production agent from scratch, I would start with a small harness before adding cleverness.

At minimum:

• a task object with explicit goal, constraints, status, and verifier
• a step loop with hard limits
• a tool registry with schemas and permission scopes
• an execution ledger that records every tool call and result
• state detectors for auth, navigation, rate limits, and common failure modes
• a verifier that checks real side effects
• a finalizer that can only report success after verification passes

The implementation does not need to be fancy. In many systems, this can be boring TypeScript or Python. The important thing is that the model is no longer floating in a chat transcript pretending to be a runtime.

A minimal mental model looks like this:

task -> plan -> tool call -> observe -> update state -> verify -> continue or stop

The model participates in the plan and maybe in choosing the next tool call.

The harness owns observation, state, limits, and truth.

That division of labor is what makes the system debuggable. When something fails, you can inspect the ledger. You can see the last known state. You can see whether verification ran. You can see whether the model misunderstood the page or the tool lied about the result.

Without that, you are debugging a transcript.

Debugging transcripts is miserable.

the punchline

Prompts are where agent design starts. They are not where agent reliability ends.

If your agent loops forever, misses auth walls, loses state, or claims work it did not do, adding another sentence to the system prompt may make you feel productive. It probably will not fix the system.

The fix is to build the harness.

Give the agent a controlled loop. Give it typed tools. Give it external state. Give it policies for auth, retries, budgets, and permissions. Give it verification that does not care how confident the model sounds.

Then the model can do the part it is actually good at: interpreting messy goals and choosing useful next actions.

Most agent failures are not proof that models are too dumb.

They are proof that we keep asking models to compensate for missing software architecture.

And software architecture, annoyingly enough, is still our job.

references

• Tejas Kumar: Harnesses in AI: A Deep Dive
• AI SDK documentation: Agents
• OpenAI Agents SDK documentation

To test my projects, I use Railway. If you want $20 USD to get started, use this link.