Every agent I've built eventually needs to search the web and feed a result into context. The naive approach is a SERP scraper — hit Google, parse the HTML, extract snippets, hope the selectors don't change. It works for about a week, breaks, and you fix it. Repeat until you give up and pay for a search API.
But even paid search APIs return blue links — titles, URLs, and a paragraph of excerpt. That's fine for a human deciding which link to click. For an agent trying to answer a question, it means another round-trip: fetch each link, extract the content, synthesize. The loop compounds latency.
What I actually want is: ask a question, get back a grounded answer with citations, in one call, as structured JSON.
The problem with SERP-for-agent pipelines
Here's what most agent search pipelines look like in pseudocode:
# Step 1: search (returns 10 blue links)
results = serp_api.search("what is the latest version of Python?")
# Step 2: for each result, fetch the page
for r in results[:3]:
page = fetch_page(r.url) # slow — 3 sequential HTTP calls
markdown = html_to_md(page) # fragile — per-site extraction
# Step 3: feed all of it to the LLM
context = "\n\n".join(markdowns)
Three round-trips minimum. Every step is a maintenance surface: the SERP API changes its schema, a target site changes its HTML, a page fails to load. The agent sits idle waiting for network I/O.
What if step 1 returned an answer — not links?
Grounded search with citations: cosift.answer
Pilot Protocol's app store ships cosift — a web search / answer / research adapter that returns clean structured JSON. It has three latency tiers so you pick the cheapest one that fits:
-
cosift.search— fast (~0.5s): keyword + semantic search, returns ranked hits with URL, title, score, excerpt. Drop-in replacement for a SERP API. -
cosift.answer— medium (~3s): LLM-synthesized answer with inline citations, grounded in the search corpus. This is what changes the game. -
cosift.research— slow (~8-30s): multi-step research report over the corpus, with planning and synthesis.
The one that matches "grounded web search API with citations" is cosift.answer. One call, structured JSON back:
pilotctl appstore call io.pilot.cosift cosift.answer '{"q": "what is the latest version of Python?"}'
Reply is something like:
{
"answer": "The latest stable version of Python is 3.13.x, released in October 2025...",
"citations": [
{"url": "https://python.org/downloads/", "title": "Python Releases"},
{"url": "https://docs.python.org/3/whatsnew/3.13.html", "title": "What's New In Python 3.13"}
]
}
Your agent gets an answer and the sources it can cite — both in one response. No separate fetch step. No HTML parsing. No SERP schema maintenance.
Discover → Install → Call
The loop is three commands, install once:
# 1. Discover
pilotctl appstore catalogue | grep cosift
# 2. Install once
pilotctl appstore install io.pilot.cosift --force
# 3. Call any time
pilotctl appstore call io.pilot.cosift cosift.answer '{"q": "latest research on agentic RAG patterns"}'
The app runs locally on your daemon as a typed IPC service — JSON in, JSON out. The daemon auto-spawns it, supervises it, and re-checks its signature on every spawn. No REST plumbing, no API key to rotate, no rate-limit errors to handle.
Use-case recipe: an agent that researches and cites
Here's the concrete pattern. Your agent needs to write a report comparing cloud GPU providers. Instead of a multi-step scrape pipeline, it calls cosift directly:
# One call. One result. Ready to cite.
result = terminal("pilotctl appstore call io.pilot.cosift cosift.answer '{\"q\": \"compare cloud GPU pricing 2026 AWS vs GCP vs Azure\"}'")
# result contains: answer field with synthesized comparison + citations
# Feed into LLM context immediately
If you need more depth, chain the methods: use cosift.search to find the top 10 relevant pages, then cosift.contents to fetch the full text of the 3 most relevant ones, then synthesize yourself. All structured JSON, all via the same local IPC interface.
Why this beats the alternatives
| Approach | Returns | Round-trips | Maintenance |
|---|---|---|---|
| SERP scraper | Blue links | 1 search + N fetches | Breaks weekly |
| Google Custom Search API | Links + excerpts | 1 search + N fetches | API key, quotas |
| Jina Reader / Firecrawl | Page markdown | N fetches (no search) | Per-site quirks |
| cosift.answer | Answer + citations | 1 call | None (app maintained) |
The structural difference: cosift returns a synthesized answer, not a list of URLs. Your agent doesn't need to choose which link to follow and parse the result — the answer arrives with citations attached. For an agent loop, this collapses three steps into one.
The bigger picture
cosift is one app in a growing catalogue. The pattern — discover → install → call — is the same for all of them. When your agent needs web data, it calls an app instead of building infrastructure. That distinction matters: one is work that compounds, the other is a tax you stop paying.
Grounded retrieval in one call. Citations included. No scraper to maintain.
curl -fsSL https://pilotprotocol.network/install.sh | sh
pilotctl appstore install io.pilot.cosift --force
pilotctl appstore call io.pilot.cosift cosift.answer '{"q": "grounded web search api with citations"}'
.
Top comments (0)