DEV Community: Saurabh Singh

My search agent reads 20 messages to answer one question

Saurabh Singh — Fri, 14 Aug 2026 19:04:16 +0000

TL;DR: I am at the very beginning of building agent infrastructure. I built Lumina, a search agent that streams cited answers, and gave it memory with one line: messages.slice(-20).
It worked, for a while. Then I noticed it citing sources that did not exist. Chasing that one bug took me through token budgets, a hidden formatting marker, an ordering bug, and eventually a question I hadn't actually answered for myself: does a search agent need "real" memory at all? You do not need a vector database to give an agent memory. You do need to know exactly what you are putting inside the window you send it, and that turned out to be the harder problem.

I was building Lumina, a search agent that streams cited, sourced answers back to the user (think a small, self-hosted version of the "ask a question, get an answer with [1][2] next to it" pattern). Solo project, no infra budget, no team to argue with about architecture. When I asked around about how to give an agent memory, almost everyone said some version of the same thing: embeddings, a vector store, retrieval over past turns. That's the default answer now. It's treated less like a design decision and more like a checkbox.

Which sounded to me like: you need a second piece of infrastructure before your agent is allowed to remember a follow-up question.

So I ignored that and wrote the simplest thing I could defend:

// backend/src/agent/agent-runner.ts
export const MAX_HISTORY_MESSAGES = 20;

export function historyFromDbMessages(messages) {
  return messages.slice(-MAX_HISTORY_MESSAGES).map((m) => ({
    role: m.role === "User" ? "user" : "assistant",
    content: m.content,
  }));
}

Last 20 messages, sent straight through. No ranking, no similarity search, no separate memory store. I asked a question, then a follow-up, and it resolved correctly. I asked a second follow-up. Also fine. I assumed memory was, more or less, a solved problem for this project, and I moved on to streaming and citation UI.

A few days later I was reading through a longer thread and noticed the model citing [1] and [2] on a turn where it had not called web_search at all.

That stopped me. Not "why is this slow," not even "why is this wrong," specifically: where did that citation come from? The agent hadn't retrieved anything this turn. It had no sources this turn. And yet there they were, formatted exactly like real citations, sitting in the answer.

That question took over the next few days. This is what I found.

Question 1: Why cap the history at all?

The obvious answer is tokens. That's true, but it's not the whole answer, and it wasn't actually the reason I picked 20 specifically.

Lumina's agent loop is tool first: on every turn, the model looks at the system prompt, the conversation history, and a web_search tool definition, and decides for itself whether it needs to search or can answer from what's already there. It makes that decision by reading the history directly. There's no separate classifier or router deciding "this needs a search," it's the same LLM call making the judgment inline.

That means an unbounded history doesn't just cost tokens. It changes the decision surface. A ten turns ago tangent sitting in context can quietly shift whether the model thinks the current question is "already answered" or needs fresh information. Rough math: if an average exchange (user question plus assistant answer) runs somewhere around 150 to 300 tokens once you include the citation formatting, a 20 message window lands around 3,000 to 6,000 tokens of history before the current question is even added. That's a manageable, predictable slice to reason over. Send the full history of a long running thread instead, and you're asking the model to weigh a hundred turn old detail against the actual question in front of it, on every single turn, forever.

So the cap isn't really a memory size decision. It's a decision about how much of the past gets a vote in what the agent does right now. Twenty was a guess, enough turns to resolve a follow-up cleanly, not so many that old context starts leaking into a decision it has no business influencing.

That part turned out to be fine. It wasn't where the bug was. But it's worth stating plainly, because it's the assumption everything else in this post sits on top of: the window itself was never the problem. What was inside it, was.

Question 2: Where were the fake citations actually coming from?

Here's where I lost real time.

Every assistant message Lumina saves to Postgres gets a hidden marker appended to the end of it. The reason is mundane: the frontend needs to re render citation cards when you reload a conversation, and rather than add a second table and join it back to messages on every read, I just stuffed the source data into the message itself, wrapped in an HTML comment so it wouldn't render:

${answer}\n\n<!--SOURCES:${JSON.stringify(sources)}-->

A real saved message looks something like this on disk:

Ice is less dense than water[1][2], which is why it floats instead of sinking.

<!--SOURCES:[{"index":1,"title":"Why Ice Floats","url":"https://...","domain":"..."},{"index":2,"title":"Density of Water","url":"https://...","domain":"..."}]-->

Invisible in the UI. Just a comment. I didn't think twice about it when I wrote it, because comments are, by definition, meant to be skipped.

That's true for a browser parsing HTML. It is not true for a language model reading a plain text message history. The model has no concept of "this part is a comment, ignore it." There's no DOM, no parser, just tokens. And those particular tokens happen to contain a JSON array with fields called index, title, and url, which is structurally identical to the citation data the model is supposed to be producing on its own.

So a few turns later, mid follow-up, the model had, sitting right there in its own context, indistinguishable from anything else, a fully formed, entirely plausible citation list from three turns earlier. It used it. The [1][2] pattern matched what it had produced before, syntactically. It just didn't correspond to anything registered in the current turn's actual source list, because no search had happened this turn.

It wasn't forgetting. If anything it remembered too accurately. It remembered formatting that was only ever meant for my frontend to parse, and treated it as legitimate conversational content because, from where it was sitting, there was no reason not to.

The test that convinced me

I didn't fully trust this explanation until I could reproduce it on demand, so I ran the same three turn conversation twice: once with the marker left in the history as is, once with it stripped out before the messages were sent to the model.

	Marker left in history	Marker stripped from history
Turn 1, search + answer	citations `[1][2]` correct, match this turn's sources	citations `[1][2]` correct, match this turn's sources
Turn 2, follow-up with new search triggered	citations `[1][2]` correct, match this turn's sources	citations `[1][2]` correct, match this turn's sources
Turn 3, follow-up with no new search	cites `[1][2]` again, but they resolve to turn 1's sources, which no longer exist in this turn's registry	no stale citations. Model either answers without citing or asks a clarifying question

Same model, same three prompts, in the same order. The only variable was whether that one hidden comment made it into the message list the LLM actually received. That's the point where I stopped calling this a hallucination and started calling it what it actually was: correct model behavior on input I hadn't bothered to clean.

I ran it a second time with a longer, five turn conversation to make sure it wasn't a fluke specific to a three turn setup, and got the same pattern: any turn that didn't trigger a fresh search would occasionally reuse whatever source indices happened to be sitting in the most recent marker still inside the 20 message window.

The fix was one line, applied before anything goes back into history:

const content = m.content
  .replace(/\n*\n?<!--SOURCES:[\s\S]*?-->\s*$/, "")
  .trimEnd();

Strip the marker, keep the visible answer text, then hand it to the model. No new tools, no new infrastructure. Just don't let the model see something it was never meant to read.

Question 3: Why did truncating before stripping still cause problems?

I got the regex right almost immediately. I got the order of operations wrong for longer than I'd like to admit.

My first pass truncated to the last 20 messages first, then stripped markers from whatever remained:

// what I had, roughly
const recent = messages.slice(-MAX_HISTORY_MESSAGES);
const cleaned = recent.map((m) => stripMarker(m.content));

That looks harmless. On short test threads, it was. On a couple of longer, real threads, a broken comment fragment started leaking through into what the model received. Not because .slice() cuts a message in half, it doesn't, message boundaries stay intact, but because of how I'd chained the two transformations, a message sitting right at the 20 message boundary occasionally had its marker only partially matched by the regex, depending on what else was on either side of it after truncation.

The practical effect: on certain threads, the model would receive a message ending in something like ...ice is less dense than water[1][2]. followed by a stray, unclosed . That's not just noise. It's a dangling, structurally broken tag sitting in the model's context, which is arguably worse than the clean, well formed version of the same bug, because now the model is trying to make sense of malformed markup instead of just ignoring it.

Reordering, strip the marker first, from the full message set, then truncate to the last 20, removed the ambiguity entirely, because truncation now happens after every message is already clean. Small fix, one line moved above another. But it's exactly the kind of bug that hides from every quick manual test you'll run while building the feature, and only shows up once a real conversation is long enough to actually reach the boundary you capped it at.

Question 4: Does a search agent need "real" memory at all?

This is the question that made me stop feeling behind on architecture, and it took the citation bug to get me to actually ask it properly instead of assuming the answer.

A vector store retrieves by semantic similarity. It ranks past messages by how related they seem to the current query, and pulls back the top few. For a follow-up like "what about the second one," the message that actually matters isn't the most topically similar one across the whole conversation. It's specifically the literal previous turn, in order, regardless of how "similar" it scores. Semantic retrieval can rank an earlier, more keyword heavy message above the one that actually matters, which would make a follow-up agent worse, not better. A sliding window doesn't have this failure mode, because it doesn't rank anything, it just preserves order.

The other half of what people usually mean by "agent memory," not re searching something you've already answered, turned out to be a prompt level decision, not a retrieval level one. Lumina's system prompt instructs the model not to call web_search again for follow-ups that are answerable from prior turns in the visible history. That decision lives entirely in prompts/prompt.md and the instructions appended in prompt-loader.ts, not in any retrieval layer. No embeddings involved, no similarity threshold to tune.

Where I do think retrieval based memory would actually matter is a fundamentally different shape of product: something that has to remember a specific user across sessions, days apart, where the fact you need genuinely isn't in the last 20 messages because it isn't even in this conversation. That's a real, well studied problem. It's what tools like mem0 and long term memory layers in agent frameworks are built for. Lumina isn't that product right now. It's a single thread search agent. Reaching for that architecture before I had that specific problem would have meant building and maintaining infrastructure to solve something I don't currently have.

Things I got wrong, collected in one place so you can skip them

"Agent memory means a vector database, that's the default now." Not for a single thread, tool first search agent. A sliding window solved the actual failure mode I had, which was resolving the previous turn correctly, not long horizon recall.

"What I store for my own UI doesn't matter to the model." It does. Anything that ends up back inside the message array is language, as far as the model is concerned, formatting conventions, HTML comments, and all.

"Truncate first, clean up second, it's just an ordering detail." It isn't just a detail. Cleaning up after truncating leaves you exposed to whatever happens to sit exactly at the cutoff, and that's the one case a quick manual test will never happen to hit.

"This is the model hallucinating, models do that." It looked exactly like that from the outside. It wasn't. The model was doing correct, expected inference on input that I had failed to sanitize.

"Twenty is an arbitrary number I should tune later." It's a real design decision about how much of the past gets to influence the agent's next action, not just a token budget knob. I hadn't actually thought about it that way until I had to defend it to myself while writing this.

What actually helped

A short list, in case you're building something similar and want to skip some of the detours I took.

Read the loop itself first

docs/AGENT_LOOP.md: the internal architecture document I ended up writing while debugging this, including the full sequence diagram from HTTP request to streamed, cited answer, and the state machine the loop runs on.

Read when you're deciding how much memory you actually need

Anthropic's writing on context and retrieval, for thinking clearly about what actually belongs inside a model's context window versus what should just live in your own storage layer.
LangChain's memory documentation, mainly for the general window versus retrieval framing, useful even if you don't end up using their implementation, which I didn't.
mem0's docs, as a reference point for what a genuine cross session memory layer looks like when you actually need one, so you can tell the difference between that problem and the one I had.

Read as code

backend/src/agent/agent-runner.ts in the Lumina repo: the entire "memory system" discussed in this post is about fifteen lines inside that one file.
backend/src/agent/agent-loop.ts: for how the tool first turn cycle actually decides, per turn, whether to search or answer, which is the mechanism the history window feeds into.

Where I'm taking this next

Right now, truncation is a hard cutoff. Message 21 stops existing to the model, silently, with no summary and no warning. The next thing I want to try is summarizing whatever falls out of the window instead of dropping it outright, so a very long thread degrades gracefully instead of the agent abruptly forgetting how the conversation started.

I also want to write a test that pushes a conversation past the 20 message boundary automatically as part of CI, so an ordering bug like the one in this post shows up in a failing test instead of in a production screenshot three weeks later, after I've forgotten exactly why I wrote the truncation logic the way I did.

Longer term, if Lumina ever needs to remember something about a specific user across separate sessions, not within a thread, but genuinely days apart, that's the point where a retrieval layer stops being premature and starts being the right tool. I don't think I'm there yet. But now I at least have a clearer test for when "yet" arrives: the question isn't "do agents typically have memory," it's "is the fact I need actually inside the window I'm already sending."

I am a builder, not an infra expert, and I've been wrong at least five documented times in this post alone. If something here is off, I'd rather be corrected than confident, tell me.

I Gave My Search Agent Memory and a Team. Then I Checked If That Was Actually True

Saurabh Singh — Sat, 08 Aug 2026 06:08:33 +0000

Part 2 of my agent build series

Let me set the scene again.

Part 1 shipped. One tool, one loop, no memory — and I was honest about all three limits in the post. A few days later I ended up reading through Google's writeup on how they built Dev Signal, an internal multi-agent system that goes from "scan Reddit for trending questions" to "draft a technical blog post" to "remember your writing style for next time." Root orchestrator, three specialist agents, and a long-term memory layer sitting underneath all of it.

I closed the tab and immediately started drafting a title in my head: "I gave Lumina memory and a team of agents."

Same instinct as last time. Same problem as last time — I hadn't written a line of it yet.

So instead of the announcement post, here's the actual plan — architecture, code, and the parts of Google's pattern I'm not stealing as-is because their own comment section already poked holes in them.

![Lumina multi-agent architecture: a supervisor node routes to search, verify, and synth agents sharing short-term state, with the synth agent reading and writing long-term memory in a vector store]

Infrastructure and Model Setup

Lumina today is Bun + TypeScript. The multi-agent version is a separate Python service sitting next to it — LangGraph for the graph, LangChain for the memory tools, same OpenRouter model underneath so I'm not paying for two providers.

# lumina_agents/graph.py
from langgraph.graph import StateGraph, END
from langchain_openai import ChatOpenAI
from typing import TypedDict, Annotated
import operator

llm = ChatOpenAI(
    model="anthropic/claude-sonnet-4-6",
    base_url="https://openrouter.ai/api/v1",
    temperature=0,
)

class AgentState(TypedDict):
    messages: Annotated[list, operator.add]
    query: str
    sources: list
    verified_claims: list
    final_answer: str

AgentState is the short-term working memory every node reads and writes. It's the thing that replaces the ad-hoc object I was passing through agent-runner.ts by hand in the TS version — LangGraph threads it through the graph for me instead.

Memory Ingestion Logic

The goal isn't "store everything Lumina ever sees." It's: capture the handful of things a user actually corrects or repeats — preferred source recency, terse vs. detailed answers — and make those available on the next session without re-asking.

Long-term Memory

A vector store, written to explicitly, never automatically:

# lumina_agents/memory.py
from langchain_chroma import Chroma
from langchain_openai import OpenAIEmbeddings
from langchain_core.tools import tool

embeddings = OpenAIEmbeddings(model="text-embedding-3-small")
memory_store = Chroma(
    collection_name="lumina_user_memory",
    embedding_function=embeddings,
    persist_directory="./memory_db",
)

@tool
def save_preference(user_id: str, preference: str) -> str:
    """Save an explicit user preference to long-term memory."""
    memory_store.add_texts(
        texts=[preference],
        metadatas=[{"user_id": user_id}],
    )
    return f"Saved: {preference}"

@tool
def load_preferences(user_id: str, query: str, k: int = 3) -> list[str]:
    """Retrieve the most relevant stored preferences for this user."""
    results = memory_store.similarity_search(
        query, k=k, filter={"user_id": user_id}
    )
    return [r.page_content for r in results]

Two tools, same split Google's pattern uses — one for writing, one for reading. The difference is save_preference only fires when a node explicitly calls it after an unambiguous signal ("always sort by recency"), not automatically at the end of every turn. Given the retrieval-precision problem below, I'd rather have less memory that's trustworthy than more memory I have to second-guess.

Short-term Memory

This is just AgentState. No separate service, no persistence — it resets when the graph run ends:

def verify_node(state: AgentState) -> AgentState:
    # reads what search_node already put in state, no DB round-trip
    sources = state["sources"]
    overlap = check_source_overlap(state["query"], sources)
    return {"verified_claims": overlap}

The boundary matters for the same reason it did in Google's writeup: state answers "what happened this run," memory answers "what do I know about this user across every run." Conflating them is how you end up either re-asking the same question every session or leaking one session's context into a completely different conversation.

Specialist 1: Search Agent

This one's not new — it's Lumina's existing web_search tool, ported over as a node instead of the whole loop:

# lumina_agents/agents/search_agent.py
from langchain_core.tools import tool
from lumina_agents.tools.tavily import tavily_search

@tool
def web_search(query: str, depth: str = "basic") -> list[dict]:
    """Search the web and return numbered, deduped results."""
    results = tavily_search(query, depth=depth)
    return [{"index": i + 1, "title": r["title"], "url": r["url"], "content": r["content"]}
            for i, r in enumerate(results)]

def search_node(state: AgentState) -> AgentState:
    response = llm.bind_tools([web_search]).invoke(state["messages"])
    if response.tool_calls:
        results = web_search.invoke(response.tool_calls[0]["args"])
        return {"sources": results}
    return {}

Same citation-numbering rule I learned the hard way in Part 1 applies here — the index gets assigned the moment a result comes back, not whenever the model gets around to referencing it.

Specialist 2: Verify Agent

New. Lumina v0 trusted whatever came back first. This node checks whether the sources actually agree before anything gets synthesized into an answer:

# lumina_agents/agents/verify_agent.py
def verify_node(state: AgentState) -> AgentState:
    sources = state["sources"]
    prompt = f"""Given these sources: {sources}
Identify which claims are supported by 2+ independent sources.
Flag any claim supported by only one source as UNVERIFIED."""
    response = llm.invoke(prompt)
    return {"verified_claims": response.content}

Deliberately dumb prompt, on purpose — the point of this node existing is that "did I check" becomes a visible step in the graph instead of an implicit assumption baked into the synthesis prompt.

Specialist 3: Synth Agent

Writes the final answer, and is the only node that reads from long-term memory — because tone and format preferences only matter at the point where you're producing output, not while you're gathering it:

# lumina_agents/agents/synth_agent.py
def synth_node(state: AgentState, user_id: str) -> AgentState:
    prefs = load_preferences.invoke({"user_id": user_id, "query": state["query"]})
    prompt = f"""Verified findings: {state['verified_claims']}
User preferences on record: {prefs}
Write the final answer, following those preferences where they apply."""
    response = llm.invoke(prompt)
    return {"final_answer": response.content}

The Root Orchestrator

The supervisor is the piece I'm most cautious about. An LLM deciding "route this to search vs. verify vs. synth" is classification with a probabilistic model attached — it will misroute sometimes, and that's not a bug to patch, it's a property to design around. So the routing function checks explicit signal first and only falls back to the model for genuinely ambiguous cases:

# lumina_agents/graph.py (continued)
def route(state: AgentState) -> str:
    if not state.get("sources"):
        return "search"
    if not state.get("verified_claims"):
        return "verify"
    if state.get("verified_claims") and not state.get("final_answer"):
        return "synth"
    return END

graph = StateGraph(AgentState)
graph.add_node("search", search_node)
graph.add_node("verify", verify_node)
graph.add_node("synth", synth_node)
graph.set_entry_point("search")
graph.add_conditional_edges("search", route)
graph.add_conditional_edges("verify", route)
graph.add_conditional_edges("synth", route)

app = graph.compile()

No LLM call in route() at all, for now. State shape decides the next node deterministically. I'll only reach for LLM-based routing if a real case shows up that state shape can't disambiguate — not by default, because it's less code to write upfront.

What I'm Not Copying As-Is

Google's own comment section did the work of stress-testing this pattern before I had to, and two points are worth taking seriously instead of shipping the architecture and hoping:

Embedding similarity isn't a precision instrument. One commenter had actually measured it — semantic search struggled to cleanly separate stylistic preferences that were near-opposite of each other. A load_preferences call can hand back the wrong preference with a confident-looking score. Nothing pulled from memory changes the final answer's substance without at least a recency check behind it.
Unstructured content flowing into long-term memory is a prompt-injection surface. If save_preference ever gets called on text the model read off the open web instead of something the user explicitly typed, that's an open door. For now, only user-authored turns can trigger a memory write — search results never do.

Summary

Three specialists — search, verify, synth — routed by explicit state instead of LLM judgment, with a short-term AgentState for in-run handoffs and a long-term vector store for anything a user explicitly wants remembered across sessions. Not the full Dev Signal pattern, and not meant to be — the parts I kept are the ones that held up under scrutiny; the parts I changed are the ones that didn't.

Code's still at github.com/Saurabhsing21/Lumina — the memory branch goes up once this is running end-to-end, not before.

If you've run embedding-based memory retrieval in production and it held up better than I'm expecting, or you've got a cleaner way to handle explicit-vs-LLM routing — tell me in the comments. I check daily.

#ai #agents #opensource #llm #langgraph #buildinpublic

I Built a Web Search Agent Harness. Then I Checked If It Actually Deserved the Name.

Saurabh Singh — Fri, 31 Jul 2026 10:53:04 +0000

I Built a Web Search Agent Harness. Then I Checked If It Actually Deserved the Name.

Part 1 of my agent build series

Let me set the scene.

I wanted a Perplexity-style search assistant — type a question, watch it decide to go search the web mid-answer, get back a real answer with numbered citations you can click. Not a chatbot wearing a search icon. Something that actually reasons about whether it needs to look something up before it does.

So I built it. Bun backend, React 19 frontend, Tavily for search, OpenRouter for the model, Postgres underneath. A few weeks in, it worked. Streaming answers, clickable sources, follow-up questions, the whole thing.

Then I went to write this post, typed the words "agent harness" into the title, and stopped.

Was that actually true? Or was I about to publish a buzzword on top of a fetch call with a nice system prompt?

The Claim I Almost Made Without Checking

Here's the ambitious version of this post I almost wrote: "I built a multi-tool agentic harness with full observability and provider failover."

None of that is true. There's one tool. One model provider path. No retries if a tool call fails mid-turn.

So before publishing, I went back through my own agent-loop.ts and agent-runner.ts like I was reviewing someone else's PR, and asked the boring question: what actually makes something a "harness" instead of a script that calls an API?

A harness is the code around the model — the part that decides when the model gets to call a tool, manages the back-and-forth, keeps context sane, and turns the mess into something a frontend can render. Not the model itself. The scaffolding.

Checked against that definition, line by line:

The model decides, not my code. There's no if (needsSearch) gate before the LLM sees the query. It gets handed a web_search tool in its schema and chooses whether to use it.
The loop actually loops. agentLoop() keeps alternating — ask the LLM, execute whatever tool it called, feed the result back, ask again — until the model returns a stop reason instead of a tool-use reason.
Context is trimmed, not just piled on. Only the last 20 messages go to the model. Citation markers get stripped out of history before the next turn, or every follow-up question would silently re-send every source block forever.
Output is a structured event stream, not a blob dropped on the frontend when it's finally done.

So — yes, it's a harness. A real one. Just a small one.

The honest caveat: it's a single-tool harness. web_search, full stop. The multi-tool version — file access, code execution, retries, provider fallback — is a different tier of engineering, and I'm not there yet. I'd rather say that plainly than have someone read the repo and feel oversold by the title.

The Loop That Makes It an Agent

Before looking at the implementation, here's the high-level architecture of how the agent executes a request. The core idea is that the application doesn't decide when to search—the model does.

The harness loads the system prompt, conversation history, and available tools, then hands control to the LLM. If the model decides it needs external information, it calls the web_search tool. The tool executes, registers and deduplicates sources, and returns the results back to the model. This LLM → Tool → LLM loop continues until the model has enough information to produce a final answer.

Once the response is complete, the harness extracts follow-up questions, persists the conversation, citations, and state to the database, and streams the final response back to the frontend.

The important part isn't the search itself—it's the orchestration loop. The harness manages context, tool execution, streaming, citations, and persistence, while the LLM decides when external information is actually needed.

*Figure 1: High-level execution flow of the AI agent harness. The LLM repeatedly reason

This is the part that turns "an LLM with instructions" into something that behaves like an agent instead of a script:

// backend/src/agent/agent-runner.ts
const context: AgentContext = {
  systemPrompt: getSystemPrompt(),
  messages: [...(options.history ?? [])],
  tools: [webSearchTool],
};

const stream = agentLoop([userMessage], context, config, signal, getDefaultStreamFn());

for await (const event of stream) {
  if (event.type === "tool_execution_end" && event.toolName === "web_search") {
    options.onEvent({ type: "sources", items: registry.list() });
  }
  if (event.type === "message_update" && event.assistantMessageEvent.type === "text_delta") {
    options.onEvent({ type: "delta", text: event.assistantMessageEvent.delta });
  }
}

My code never decides whether to search — it just watches the event stream coming out of the loop and forwards the right pieces downstream. Model decides, harness routes. That split is the entire reason I'm comfortable calling this a harness instead of a script with a search button bolted on.

The One Tool It Has (And Why the Numbering Almost Broke Me)

// backend/src/tools/web-search.ts
export function createWebSearchTool(defaultDepth, registerResult) {
  return {
    name: "web_search",
    parameters: Type.Object({
      query: Type.String(),
      searchDepth: Type.Optional(Type.Union([
        Type.Literal("basic"),
        Type.Literal("advanced"),
      ])),
    }),
    async execute(_id, params) {
      const results = await webSearch(params.query, params.searchDepth ?? defaultDepth);
      const text = results.map((r) => {
        const { index, title, url } = registerResult?.(r) ?? r;
        return `[${index}] ${title}\nURL: ${url}\n${r.content}`;
      }).join("\n\n");
      return { content: [{ type: "text", text }], details: { results } };
    },
  };
}

The Tavily call itself is the boring part. The part that actually bit me is registerResult — every result gets numbered and deduped through a shared registry the instant it comes back, so when the model writes [1] in its answer, that index has to map to a source the frontend already has sitting in its sidebar. Get the timing wrong — number it after the model already referenced it, or dedupe against the wrong run — and citations silently point at nothing. No error, no crash. Just a [3] in the answer with nothing behind it. That one took a full evening to notice, because it doesn't fail loud.

Frontend `searchMode`	Tavily depth
`"search"`	`basic`
`"research"`	`advanced`

Why Not Just Force a Search Every Turn?

I want to address this directly because it's the obvious simpler design, and I genuinely started building it that way first.

Force a search before every LLM call. No decision logic, no tool schema, no risk of the model "forgetting" to search when it should. Easier to reason about, easier to test.

It's wrong for a chat agent, and here's the concrete reason: half of what people type into a thread is a follow-up — "what about for React 19 instead?" — that needs the model to reread its own last answer, not burn a fresh Tavily call and 2+ seconds of latency for a question it can already answer from context.

Let the model decide when it actually needs fresh information. That's the whole reason this is tool-first instead of pipeline-first. It's also the design choice that makes the word "harness" earn its keep here — the intelligence about when to act lives inside the loop, not hardcoded in front of it.

The Stack, Complete

Client (React 19 + assistant-ui)
        │
        ▼  POST /ask  +  Bearer JWT
┌─────────────────────────────────────────────┐
│           Express 5 · /ask route             │
│    auth middleware → credit check → runner   │
└───────┬───────────────────────┬──────────────┘
        ▼                       ▼
   agent-runner.ts         Postgres (Prisma 7)
   systemPrompt + tools     history, credits
        │
        ▼
   agent-loop.ts  ── tool-first loop ──┐
        │                              │
        ▼                              ▼
   OpenRouter (LLM)               Tavily (web_search)

Layer	Technology
Runtime	Bun, Express 5
Agent	`@earendil-works/pi-ai` + OpenRouter
Search	Tavily API
Database	PostgreSQL + Prisma 7
Auth	Supabase (JWT)
Frontend	React 19, Vite 8, assistant-ui, Tailwind CSS 4

Two independent apps, no Docker, no shared infra to babysit. The backend has to know about exactly one thing outside itself at request time: which model to call and which tool to run.

What v0 Actually Buys You

	Plain LLM chat	v0 (this harness)
Freshness	Frozen at training time	Live web results, only when the model asks for them
Citations	None, or hallucinated	Numbered, deduped, clickable, persisted to the DB
Follow-ups	Re-explains from scratch	Reuses trimmed history, no repeat search
Cost control	None	Per-user credit gate (`creditLimit` default 10), search depth toggle
Delivery	Wait for the full response	NDJSON stream, renders as it's generated

None of that is exotic engineering. That's kind of the point — a harness doesn't need to be clever, it needs to be correct about the loop. The clever part is knowing which one line-item on that table you're actually solving before you write any code.

What's Next

This is v0 on purpose. A few things actually on deck for v1, not just aspirational bullet points:

A second tool — a "fetch and read this page" tool, so the model can go from a search snippet to the real page instead of reasoning off three lines of preview text.
Tool-call resilience — right now a failed Tavily call just fails the turn. A harness that holds up needs retry/backoff at the tool layer, not a bare try/catch.
Proving the provider seam actually holds — search.ts, models.ts, and stream-fn.ts are already written as swap points, but I've only ever run this against Tavily + OpenRouter. An abstraction nobody's swapped is just a hope.

Code's here if you want to see the whole thing: github.com/Saurabhsing21/Lumina

Loop internals specifically: docs/AGENT_LOOP.md

If you think one tool doesn't earn the word "harness," or you've hit the same silent-citation bug I did — tell me in the comments. I check daily.

#ai #agents #opensource #llm #webdev #buildinpublic