Built a Hand-Rolled Agent Loop and Found a Weird Retrieval Bias

Every "agent" framework boils down to the same three-step loop underneath: look at the world, decide what to do, do it. I wanted to build that by hand instead of reaching for a framework, mostly because I've never actually seen the seams of one up close, and frameworks are good at hiding exactly the parts I wanted to look at.

Started with a completely fake skeleton, hardcoded task and all, just to prove the shape works before adding anything real:

def observe():
    return "task: find out how to check pod status with oc"

def decide(observation):
    return f"search_notes: {observation}"

def act(decision):
    print(f"Would call: {decision}")

obs = observe()
plan = decide(obs)
act(plan)

Fine, obviously — it's just string formatting at this point. The real test was replacing decide() with an actual model call:

import ollama

def decide(observation):
    prompt = f"""You are an agent with one tool available: search_notes(query).
It searches a local knowledge base and returns relevant notes.

Given this task: {observation}

Respond with ONLY the exact search query you'd pass to search_notes — nothing else, no explanation."""
    response = ollama.generate(model="llama3.2:1b", prompt=prompt)
    return response["response"].strip()

First two runs gave me the identical output, which made me assume it was deterministic. It wasn't — a third run proved that wrong immediately. Three runs, three different behaviors:

1. A full oc command wrapped in backticks — the model ignored the "respond with only a query" instruction entirely and answered the underlying question instead. Worth noting: it's the exact same flawed command from Entry 02 (--all-namespaces instead of scoping to the actual namespace).
2. find -f 'oc status pod*' -A -v — not a real command in any tool, just a strange mashup of find syntax and oc concepts.
3. How to check pod status with oc — the one time it actually did what was asked.

That instability is itself the finding. A "decide" step built on a small local model with no real constraint around it isn't reliable even for one fixed task run back to back — which is a pretty direct argument for why Entry 02's system-prompt approach exists in the first place.

Wired act() to the real MCP tool from Entry 08 next, to see what each of those three outputs actually retrieved when fed to real search — not through a full MCP client this time, just calling the underlying function directly:

import sys
sys.path.insert(0, ".")
from mcp_search_server import search_notes

def act(decision):
    result = search_notes(decision)
    print(result)

act('`oc get pods --all-namespaces -o jsonpath=\'{.items[*].metadata.name}\'`')
print("---")
act("find -f 'oc status pod*' -A -v")
print("---")
act("How to check pod status with oc")

Worth noting: search_notes is wrapped in @mcp.tool(), and I genuinely wasn't sure that would still work as a plain callable outside a real MCP session. It did — no errors, clean results, all three queries went straight through to the Chroma index.

And this is where it got genuinely strange. All three queries surfaced the same top document — 02-oc-cli-mentor-system-prompt.md, which makes sense, it's the only stored content actually about oc. But the distances ran backwards from what I expected:

Decide() output	Distance to correct match
Garbled backtick command	324.3 (best)
Nonsensical find/oc hybrid	378.7
Clean, correct query	430.1 (worst)

The most broken query retrieved the correct document with the tightest confidence. The one time the model actually did what I asked — produced a clean, sensible search query — retrieved the same correct document, but with noticeably weaker confidence than either of the broken attempts.

I don't have a confirmed reason for this, and I'd rather say that plainly than invent one. My best guess: Entry 02's actual content contains real oc command syntax, and the garbled query text — being made of similar command-like tokens — may be matching on surface-level lexical overlap rather than pure semantic meaning. That's a testable claim, not a settled one, and it's worth checking directly before trusting it.

So the loop works end to end — a real model decision, feeding a real search tool, returning real results. But this entry didn't end where I expected. The interesting result wasn't "the agent works," it's that the search layer built across Entries 05-08 might be rewarding queries that look like the stored content syntactically, more than queries that actually match its meaning. If that's true, it's a real weakness in the retrieval system itself, not just in this one flaky decide() step — worth its own entry to actually test properly.