I Built a Web Search Agent Harness. Then I Checked If It Actually Deserved the Name.

Saurabh Singh — Fri, 31 Jul 2026 10:53:04 +0000

I Built a Web Search Agent Harness. Then I Checked If It Actually Deserved the Name.

Part 1 of my agent build series

Let me set the scene.

I wanted a Perplexity-style search assistant — type a question, watch it decide to go search the web mid-answer, get back a real answer with numbered citations you can click. Not a chatbot wearing a search icon. Something that actually reasons about whether it needs to look something up before it does.

So I built it. Bun backend, React 19 frontend, Tavily for search, OpenRouter for the model, Postgres underneath. A few weeks in, it worked. Streaming answers, clickable sources, follow-up questions, the whole thing.

Then I went to write this post, typed the words "agent harness" into the title, and stopped.

Was that actually true? Or was I about to publish a buzzword on top of a fetch call with a nice system prompt?

The Claim I Almost Made Without Checking

Here's the ambitious version of this post I almost wrote: "I built a multi-tool agentic harness with full observability and provider failover."

None of that is true. There's one tool. One model provider path. No retries if a tool call fails mid-turn.

So before publishing, I went back through my own agent-loop.ts and agent-runner.ts like I was reviewing someone else's PR, and asked the boring question: what actually makes something a "harness" instead of a script that calls an API?

A harness is the code around the model — the part that decides when the model gets to call a tool, manages the back-and-forth, keeps context sane, and turns the mess into something a frontend can render. Not the model itself. The scaffolding.

Checked against that definition, line by line:

The model decides, not my code. There's no if (needsSearch) gate before the LLM sees the query. It gets handed a web_search tool in its schema and chooses whether to use it.
The loop actually loops. agentLoop() keeps alternating — ask the LLM, execute whatever tool it called, feed the result back, ask again — until the model returns a stop reason instead of a tool-use reason.
Context is trimmed, not just piled on. Only the last 20 messages go to the model. Citation markers get stripped out of history before the next turn, or every follow-up question would silently re-send every source block forever.
Output is a structured event stream, not a blob dropped on the frontend when it's finally done.

So — yes, it's a harness. A real one. Just a small one.

The honest caveat: it's a single-tool harness. web_search, full stop. The multi-tool version — file access, code execution, retries, provider fallback — is a different tier of engineering, and I'm not there yet. I'd rather say that plainly than have someone read the repo and feel oversold by the title.

The Loop That Makes It an Agent

This is the part that turns "an LLM with instructions" into something that behaves like an agent instead of a script:

// backend/src/agent/agent-runner.ts
const context: AgentContext = {
  systemPrompt: getSystemPrompt(),
  messages: [...(options.history ?? [])],
  tools: [webSearchTool],
};

const stream = agentLoop([userMessage], context, config, signal, getDefaultStreamFn());

for await (const event of stream) {
  if (event.type === "tool_execution_end" && event.toolName === "web_search") {
    options.onEvent({ type: "sources", items: registry.list() });
  }
  if (event.type === "message_update" && event.assistantMessageEvent.type === "text_delta") {
    options.onEvent({ type: "delta", text: event.assistantMessageEvent.delta });
  }
}

My code never decides whether to search — it just watches the event stream coming out of the loop and forwards the right pieces downstream. Model decides, harness routes. That split is the entire reason I'm comfortable calling this a harness instead of a script with a search button bolted on.

The One Tool It Has (And Why the Numbering Almost Broke Me)

// backend/src/tools/web-search.ts
export function createWebSearchTool(defaultDepth, registerResult) {
  return {
    name: "web_search",
    parameters: Type.Object({
      query: Type.String(),
      searchDepth: Type.Optional(Type.Union([
        Type.Literal("basic"),
        Type.Literal("advanced"),
      ])),
    }),
    async execute(_id, params) {
      const results = await webSearch(params.query, params.searchDepth ?? defaultDepth);
      const text = results.map((r) => {
        const { index, title, url } = registerResult?.(r) ?? r;
        return `[${index}] ${title}\nURL: ${url}\n${r.content}`;
      }).join("\n\n");
      return { content: [{ type: "text", text }], details: { results } };
    },
  };
}

The Tavily call itself is the boring part. The part that actually bit me is registerResult — every result gets numbered and deduped through a shared registry the instant it comes back, so when the model writes [1] in its answer, that index has to map to a source the frontend already has sitting in its sidebar. Get the timing wrong — number it after the model already referenced it, or dedupe against the wrong run — and citations silently point at nothing. No error, no crash. Just a [3] in the answer with nothing behind it. That one took a full evening to notice, because it doesn't fail loud.

Frontend `searchMode`	Tavily depth
`"search"`	`basic`
`"research"`	`advanced`

Why Not Just Force a Search Every Turn?

I want to address this directly because it's the obvious simpler design, and I genuinely started building it that way first.

Force a search before every LLM call. No decision logic, no tool schema, no risk of the model "forgetting" to search when it should. Easier to reason about, easier to test.

It's wrong for a chat agent, and here's the concrete reason: half of what people type into a thread is a follow-up — "what about for React 19 instead?" — that needs the model to reread its own last answer, not burn a fresh Tavily call and 2+ seconds of latency for a question it can already answer from context.

Let the model decide when it actually needs fresh information. That's the whole reason this is tool-first instead of pipeline-first. It's also the design choice that makes the word "harness" earn its keep here — the intelligence about when to act lives inside the loop, not hardcoded in front of it.

The Stack, Complete

Client (React 19 + assistant-ui)
        │
        ▼  POST /ask  +  Bearer JWT
┌─────────────────────────────────────────────┐
│           Express 5 · /ask route             │
│    auth middleware → credit check → runner   │
└───────┬───────────────────────┬──────────────┘
        ▼                       ▼
   agent-runner.ts         Postgres (Prisma 7)
   systemPrompt + tools     history, credits
        │
        ▼
   agent-loop.ts  ── tool-first loop ──┐
        │                              │
        ▼                              ▼
   OpenRouter (LLM)               Tavily (web_search)

Layer	Technology
Runtime	Bun, Express 5
Agent	`@earendil-works/pi-ai` + OpenRouter
Search	Tavily API
Database	PostgreSQL + Prisma 7
Auth	Supabase (JWT)
Frontend	React 19, Vite 8, assistant-ui, Tailwind CSS 4

Two independent apps, no Docker, no shared infra to babysit. The backend has to know about exactly one thing outside itself at request time: which model to call and which tool to run.

What v0 Actually Buys You

	Plain LLM chat	v0 (this harness)
Freshness	Frozen at training time	Live web results, only when the model asks for them
Citations	None, or hallucinated	Numbered, deduped, clickable, persisted to the DB
Follow-ups	Re-explains from scratch	Reuses trimmed history, no repeat search
Cost control	None	Per-user credit gate (`creditLimit` default 10), search depth toggle
Delivery	Wait for the full response	NDJSON stream, renders as it's generated

None of that is exotic engineering. That's kind of the point — a harness doesn't need to be clever, it needs to be correct about the loop. The clever part is knowing which one line-item on that table you're actually solving before you write any code.

What's Next

This is v0 on purpose. A few things actually on deck for v1, not just aspirational bullet points:

A second tool — a "fetch and read this page" tool, so the model can go from a search snippet to the real page instead of reasoning off three lines of preview text.
Tool-call resilience — right now a failed Tavily call just fails the turn. A harness that holds up needs retry/backoff at the tool layer, not a bare try/catch.
Proving the provider seam actually holds — search.ts, models.ts, and stream-fn.ts are already written as swap points, but I've only ever run this against Tavily + OpenRouter. An abstraction nobody's swapped is just a hope.

Code's here if you want to see the whole thing: github.com/Saurabhsing21/Lumina

Loop internals specifically: docs/AGENT_LOOP.md

If you think one tool doesn't earn the word "harness," or you've hit the same silent-citation bug I did — tell me in the comments. I check daily.

#ai #agents #opensource #llm #webdev #buildinpublic

DEV Community: Saurabh Singh

I Built a Web Search Agent Harness. Then I Checked If It Actually Deserved the Name.

I Built a Web Search Agent Harness. Then I Checked If It Actually Deserved the Name.

The Claim I Almost Made Without Checking

The Loop That Makes It an Agent

The One Tool It Has (And Why the Numbering Almost Broke Me)

Why Not Just Force a Search Every Turn?

The Stack, Complete

What v0 Actually Buys You

What's Next