How I Built a Research Agent That Reads the Docs Before My Coding Agent Ships

TL;DR

My autonomous coding agent kept shipping code against library APIs that didn't exist, because it wrote first and read the docs never. I fixed it by adding a read-only research sub-agent that runs before implementation, returns a short structured brief, and blocks the implementer until that brief exists. Hallucinated-API bugs dropped from roughly one in five tasks to about one in forty over three months, and the extra cost was under 10% of tokens. Here's how it works and what I got wrong along the way. 🚀

The Problem

I run a fully autonomous implementation system on a Mac mini. An orchestrator picks up tasks, hands them to parallel implementation agents built on Claude Code, and a self-healing agent cleans up whatever fails. It ships real code to real projects, mostly unattended.

For the first few months, the single most annoying failure mode was this:

The implementer confidently calls client.batchUpsert(). The library has never had a batchUpsert(). It has upsertMany(), added in v4.2, with a different argument shape.

Tests fail, the self-healing agent tries three fixes, burns 40k tokens, and finally gives up and marks the task blocked. I'd wake up to a pile of "blocked" tasks that all shared one root cause: the agent guessed an API instead of reading it.

When I audited two weeks of failed tasks, the numbers were ugly:

Failure cause	Share of failed tasks
Hallucinated or outdated library API	38%
Misread existing internal code	22%
Genuinely hard bug	19%
Everything else	21%

Sixty percent of failures were "didn't look before writing." That's not an intelligence problem. That's a process problem.

What made it interesting: the implementer could read docs. It had web fetch and file tools. It just didn't, because the moment you give a capable model a task, its instinct is to start producing code. Prompting "please read the docs first" helped for about two days, then the instruction got crowded out by everything else in the context window.

How I Solved It

The fix was structural, not prompt-based. I split "figure out how this works" from "build it" into two different agents with two different tool sets.

The shape of it

flowchart LR
    O[Orchestrator] -->|task + question list| R[Research agent<br/>read-only]
    R -->|research brief| O
    O -->|task + brief| I[Implementation agent]
    I -->|diff| V[Verifier]
    V -->|pass / fail| O
    O -.->|brief missing or low confidence| R

Three rules make this work:

1. The research agent cannot write. It gets Read, Grep, Glob, web search, and web fetch. No Edit, no Write, no Bash. It physically cannot "just fix it real quick."
2. The implementer cannot start without a brief. The orchestrator refuses to dispatch an implementation task unless a research brief file for that task exists and has a confidence score.
3. The brief is short and structured. Not a doc dump. A contract.

The research brief contract

Every brief is a small Markdown file with a fixed shape. Here's a real one, lightly anonymized:

# Research brief: task-0417

## Questions
1. How do we bulk-insert rows with the ORM at the pinned version?
2. Does the existing repo already wrap this anywhere?

## Findings
### Q1
- Pinned version is 4.3.1 (from lockfile).
- Bulk insert is `Model.bulkCreate(rows, { updateOnDuplicate: [...] })`.
- `upsertMany` does NOT exist in this version. It was a community plugin.
- Source: node_modules/<orm>/lib/model.js lines 2210-2290, and the 4.x changelog.

### Q2
- Yes. `src/db/batch.ts` exports `insertBatch()` which already handles chunking at 500 rows.
- Two callers use it. Reuse it; don't add a second path.

## Verdict
Use the existing `insertBatch()` helper. Do not touch the ORM directly.

## Confidence: 0.9
## Sources read: 4 files, 1 changelog, 0 web pages
## Staleness risk: low (pinned version, checked lockfile)

The four fields that matter are Questions, Verdict, Confidence, and Staleness risk. Everything else is supporting evidence. The implementer reads the verdict first. The orchestrator reads the confidence.

The research agent definition

I define sub-agents as Markdown files with frontmatter. The research one looks roughly like this:

---
name: researcher
description: Read-only investigation. Answers "how does X actually work"
  by reading source, lockfiles, changelogs, and official docs. Returns a
  research brief, never code.
tools: Read, Grep, Glob, WebSearch, WebFetch
---

You answer questions about how things actually work. You never propose
an implementation.

Rules:
- Check the pinned version in the lockfile BEFORE reading any docs.
  Docs for the wrong major version are worse than no docs.
- Prefer reading node_modules / site-packages source over web docs.
  Source can't be out of date for the installed version.
- Search the repo for existing wrappers before answering "how do I call X".
- If two sources disagree, report both and lower your confidence.
- Output ONLY the brief format. No preamble.

That "check the lockfile first" line was added after a painful week. More on that below.

How the orchestrator gates on it

The gate itself is boring shell. That's the point. It runs before every dispatch:

brief="state/briefs/${task_id}.md"

if [[ ! -f "$brief" ]]; then
  dispatch_agent researcher "$task_id"
  exit 0   # come back next tick
fi

confidence=$(grep -E '^## Confidence:' "$brief" | awk '{print $3}')
if (( $(echo "$confidence < 0.6" | bc -l) )); then
  # Re-research with the low-confidence sections as new questions
  dispatch_agent researcher "$task_id" --refine
  exit 0
fi

dispatch_agent implementer "$task_id" --brief "$brief"

Two things I like about this:

• Deterministic gating. I don't ask the model whether it feels ready. A file exists with a number in it, or it doesn't.
• Refinement is a loop with a cap. Low confidence triggers a second research pass focused on the weak questions. After two refinements, the task is marked "needs human" instead of looping forever.

What the implementer sees

The implementer's prompt gets the brief injected verbatim at the top, followed by one line:

The brief above is the source of truth for library APIs and existing helpers. If you believe it is wrong, stop and write why to the task file. Do not work around it.

That last sentence matters. Before I added it, the implementer would occasionally read the brief, disagree silently, and do its own thing. Now disagreement is a visible event I can grep for.

Results after three months

Across about 1,100 tasks on four projects:

Metric	Before	After
Tasks failed on hallucinated API	~20%	~2.5%
Self-healing retries per task (avg)	1.8	0.6
Tokens per completed task	baseline	+8%
Tasks marked "needs human"	11%	6%

The 8% token overhead surprised me. I expected 25% or more. It turned out that the retries I eliminated were far more expensive than the research pass I added. A failed implementation plus three self-healing attempts costs more than reading four files.

Lessons Learned

1. Separate "understand" from "build" with tools, not prompts

Telling a single agent "research first, then code" decays. It works until the context fills up, then the instruction loses to the task. Giving the research agent no write tools makes the separation physical. It can't drift into implementation because implementation isn't possible.

This generalizes: if you want a behavior to be reliable, remove the ability to do the alternative rather than asking nicely.

2. The lockfile is the first document, not the last

My worst week with this system was when the research agent read the latest web docs for a library, wrote a confident brief, and the implementer built against a major version we didn't have installed. Confidence was 0.95. The brief was completely correct for the wrong version.

Now the agent's first action is always reading the lockfile or pip freeze output, and the brief has a Staleness risk field. Source in node_modules beats docs on the web every time, because installed source cannot be out of date for the installed version.

3. Confidence scores are only useful if something acts on them

I initially added the confidence field for my own reading. It was decorative. It became useful the day the orchestrator started gating on it. The moment a number has a consumer, the agent producing it gets noticeably more careful about calibration, because low confidence triggers visible re-work.

Same lesson applies to any structured output from an agent. If nothing reads the field, delete it or wire it up.

4. Make disagreement a logged event

"If you think the brief is wrong, stop and write why" turned a silent failure mode into data. In three months the implementer disagreed with a brief 23 times. It was right 9 times. Those 9 cases became test cases for the research agent's instructions. The 14 wrong disagreements were almost always the implementer wanting to use a newer API it "remembered" from training.

5. Skip research when the task is small, and be explicit about the threshold

Not everything needs a brief. Renaming a variable, adjusting a CSS margin, bumping a copyright year. Running research on those wasted tokens and, worse, made the system feel slow for trivial work.

I added a size heuristic: if the task touches fewer than 2 files and mentions no external library, the orchestrator writes a one-line auto-brief with confidence 1.0 and moves on. The threshold is dumb. Dumb thresholds that exist beat smart thresholds I never got around to building.

What's Next

Two things I'm working on:

• Brief reuse across tasks. Right now every task gets fresh research even if the last task answered the same question. I'm adding a simple index keyed on library plus version so briefs get reused for 7 days, then expire.
• Research as a first-class MCP server. I want other tools to be able to ask "how does X work in this repo at this version" and get the same brief format back, without going through the orchestrator.

I also want to try the same split for debugging: a read-only "diagnose" agent that produces a hypothesis brief before any fix agent touches the code. Early signs are promising, but I don't have enough data to write about it yet.

Wrap-up

If your coding agent keeps inventing APIs, the fix probably isn't a better model or a longer prompt. It's a process boundary. Give one agent eyes and no hands, give the other hands and a contract, and make the orchestrator refuse to skip the first step.

Versions for anyone reproducing this: Claude Code on the 2.x line as of September 2026, Node.js 22.x, Python 3.13, orchestrator in plain Bash on macOS.

If this was useful, follow me here on Dev.to for more build logs from running a fully autonomous implementation system 24/7. And if you've built a different kind of research or planning gate for your agents, I'd genuinely like to hear how it behaves in the comments. 💡

Comments

[Vlad Zoff]

I like the separation between “research” and “implementation”.

One thing I've been thinking about is that there are really two different kinds of context here: facts that can be re-derived from the current project, and reasoning that actually needs a model.

I’d much rather derive things like project structure, dependencies and contracts from the repo itself, then let the model reason on top of that, instead of asking it to rediscover those facts every time.