Building a self-hosted deep-research agent with Claude Code

Wouter Van Schandevijl — Fri, 01 May 2026 20:01:10 +0000

Every year the same problem... what to buy for her birthday. While standing in line at the bakery, I started a GitHub Issue. By the time I got back to my desk, a fancy html overview was published to my GitHub Pages with a fallback to the dry markdown research. In there was this gem "Hunt A Killer: a six-month serialized murder-mystery" (she'll love that)!

I built Scout, a MIT open-source research engine that runs Claude Code on your hardware and takes your one-line topic from a GitHub Issue to a cited markdown on a site you own. This post is about the four design decisions that made it actually useful, not just a demo.

The pain

I wanted a research loop that fits the way ideas arrive.

Most of my research curiosities show up on a phone — on a train, in line, mid-conversation. I'd then act on the result at my desk, often inside a Claude Code session, using the research as context for whatever I'm building. The loop I wanted: on the go capture, desk-side payoff.

What it is, in one diagram

→ Open a GitHub Issue
→ Sharpen & Sub-Topics selection (optional)
→ Fancy HTML pages selection (optional)

Decision 1: GitHub Issues as the UX

Thank you GitHub for not having to build a UI!

I already have the account, it's tightly linked with the source code,
gave me Auth, and a way to converse with Claude Code and checkboxes to
get Claude started.

Decision 2: Sharpen before research

The biggest single quality lever in the system isn't the model or the search backend — it's the sharpening step that runs before any research starts.

A user types "best NAS to buy in 2026." That's three different questions in a trench coat: best for what budget? Replacing what? Optimizing for storage capacity, idle wattage, or both? Without sharpening, Claude charges off in one direction and the output ignores two thirds of what the user actually meant.

So skills/scout/sharpen.md runs first: read the topic, propose a tightened framing as a comment ("Survey for replacing a Synology NAS in 2026, optimised for AVX2 + ECC, comparing UGREEN / Beelink / Minisforum and TrueNAS / Unraid / Proxmox…"), wait for tick ✓ or a steering reply. Only then does research start.

The lesson generalizes: agents that ask one clarifying question before working produce output an order of magnitude better than agents that don't. Sharpening is cheap (one model call), and it eliminates the most common (and expensive!) failure mode — answering the wrong question.

Decision 3: Decompose wide topics into parallel sub-agent expeditions

Some questions are atomic ("best static-site generator in 2026"). Some are containers ("everything I need to leave Synology"). Trying to answer the latter in one pass produces a 4-page jumble.

For expedition depth, the sharpener also proposes 2–8 sub-topics ("hardware shortlist", "OS/storage stack", "DSM migration playbook", "remote access"). Each becomes its own independent expedition, dispatched as a sub-agent in parallel. After they all finish, a synthesis pass produces a parent overview page that links each child.

This is map-reduce, but applied at the research-question level rather than the data level. Two non-obvious things matter:

Sub-topics must be independently coherent. If sub-topic 3 needs sub-topic 1's conclusions, you've decomposed wrong — collapse them.
The synthesis pass is where most value lives. Without it, you have N disconnected pages. With it, the parent page reconciles disagreements between children and ends with a recommendation.

Decision 4: "Every claim carries its URL inline" as a skill rule

The single rule that does the most work: every factual claim, number, or quote must have its source URL next to the claim.

Why: when claims and citations are decoupled, LLMs hallucinate. The citations look real, but they don't actually back the specific sentence above them. Forcing inline URLs ([[1]](https://...)) turns hallucination into a typing constraint the model can't fudge. If it can't find a URL, it can't make the claim.

This rule is enforced in skills/scout/SKILL.md and re-injected on every output pass. It's the cheapest, most effective hallucination-defense I've shipped in any agent.

What it costs

Measured per-run from my own Atlas (Sonnet 4.6 API):

Tier	Mode	Time	Cost median (range)	n
`recon`	one-page	~2 min	$0.83 ($0.73–$0.93)	4
`survey`	single page	5–10 min	$2.56 ($1.99–$3.99)	24
`expedition`	single deep	12–20 min	$8.64 ($7.23–$14.90)	16
`expedition`	multi-angle	35–70 min	$28 ($12–$35)	9

On a Pro/Max subscription, runs consume your existing quota — no incremental cost.

Try it

curl -fsSL https://raw.githubusercontent.com/Laoujin/Scout/main/install.sh \
  | bash -s -- --config=s5.cartography.v1

Repo: github.com/Laoujin/Scout
Live example Atlas: laoujin.github.io/Atlas
Landing page & config picker: laoujin.github.io/Scout

What's reusable

Sharpen first. Decompose wide topics. Inline citations as a hard rule. Use GitHub Issues so you don't build a UI. Those four ideas port to any agent that produces long-form output — the rest of Scout is plumbing.

MIT-licensed, three-repo split (Scout / Atlas / Compass), one-line install. Fork it, take the patterns, make it your own.

⭐ github.com/Laoujin/Scout if you'd find this useful — that's the signal that tells me whether to keep building.

DEV Community: Wouter Van Schandevijl