Cheap RAG in Go with Gemini File Search: no vector DB, two calls, one hosted store

Hello, I'm Maneshwar, and I'm building LiveReview — a blast-radius aware AI code review built for your business-critical systems. Star us to help devs discover the project, give it a try, and share your feedback to help improve the product.

Every team I have been on has the same shelf.

Nine engineering books somebody swears by, a couple of long essays that get linked in every third design review, and sixty-odd internal blog posts and postmortems that nobody rereads.

About 1.4 million tokens of "we already learned this once."

I wanted a tool where you paste a design proposal, an ADR, a postmortem draft, and it comes back with: here is what is actually being proposed, here is the pattern underneath it, and here are the three places on the shelf where we, or someone smarter, already ran into this exact shape.

Grounded. With citations. Not "an LLM read your doc and had feelings about it."

The obvious build is a vector database, an embedding pipeline, a chunker, a reranker, and a weekend.

I built it with none of those, in Go, on Gemini's free tier, and the retrieval side of it costs nothing to run.

This post is about how, and about the four things that bit me on the way.

RAG without owning the R

Gemini has a thing called File Search.

You create a "store", upload documents into it, and Google chunks them, embeds them, and indexes them.

Then you attach that store to a normal generateContent call as a tool, and the model searches it by itself, mid-answer, and hands you back the chunks it used as groundingMetadata.

No pgvector. No Pinecone bill. No embedding model to pick and then regret.

The pricing is the part that made me sit up.

Storage is free. Query-time embeddings are free. You pay once at indexing time, at embedding prices, and then the chunks the model pulls in are billed as ordinary context tokens on the call you were already making.

On the free tier the store caps at 1 GB. My entire shelf, every book and every post converted to markdown, is 5.3 MB.

So the architecture is embarrassingly short.

One Go binary. SQLite via modernc.org/sqlite, so no cgo. A REST client for Gemini with no SDK, because I wanted to log the exact request and response bodies verbatim, and SDKs love to hide those.

Two model calls per question. One hosted store. That is the whole thing.

Step zero: getting books into markdown without losing the spaces

Before any of the clever parts, the corpus has to exist as text, and this cost me an evening.

The first tool I reached for was markitdown. It gave me headings, sort of, and it also gave me this:

Theubiquityoffrustrating,unhelpfulsoftwareinterfaceshasmotivateddecadesofresearch

Every word in the PDF glued to its neighbour. The embedding model does not know what "Theubiquityoffrustrating" is, and neither does the retrieval.

pdftotext fixed the spacing and threw away every heading, so a 300-page book became one undifferentiated scroll.

The one that won was pymupdf4llm. It looks at font sizes to decide what is a heading, keeps bold and italics, and its text extraction handled the spacing correctly on the same PDF.

uv run --with pymupdf4llm python -c "
import pymupdf4llm, pathlib
pathlib.Path('out.md').write_text(pymupdf4llm.to_markdown('in.pdf'))
"

Forty-four real headings out of one essay, versus zero.

Headings matter more than they look like they should here, because File Search chunks on whitespace with a token budget. A chunk that starts at a heading is a chunk that means something on its own. A chunk that starts mid-sentence in a wall of text is noise with an embedding attached.

I put the conversion behind make process-data so nobody on the team has to rediscover this. EPUBs, PDFs, and a sync of our blog repos, all into one post_processed_data/ tree of markdown.

The store belongs to the key, not to you

This is the first thing that bit me, and it is not in the big print.

A File Search store lives inside the Google Cloud project behind the API key that created it.

Create a store with key A, try to search it with key B from a different project, and it does not error in a helpful way. It just is not there.

That matters the moment you have more than one key, which on the free tier you will, because the per-minute caps are real and the fix everyone reaches for is "rotate across a few keys."

You cannot rotate a tool-attached call across keys. The store pins you.

So keys got roles.

keys/store.txt is the short list. Each key in it owns a complete copy of the corpus in its own store. Ingest uploads to every one of them. The search call tries store 0 with key 0, and only on a quota or auth failure falls through to store 1 with key 1.

keys/rotate.txt is the long list, for any call with no tool attached: the analysis call, the corpus-index summaries. Those genuinely do not care which key answers, so they rotate on 429, 401, 403, and anything 5xx.

Two things I got wrong before I got them right:

503 is not a key problem. Gemini returns 503 "high demand" when Google is busy, and switching keys does nothing except burn another key's quota. So the client waits and retries the same key a couple of times before falling over.

The key goes in the x-goog-api-key header, never the URL. Put it in the query string and the first transport error prints your key into your own logs. Ask me how I know.

Yes, two stores means the one-time indexing cost happens twice. It is the price of the search call never dying on a single quota, and at embedding prices for 5 MB, it is coffee money.

Ingest is a diff, not an upload

The ingest step is where "just upload the folder" turns into an actual program.

Every file gets a sha256. A manifest in SQLite records path -> sha -> document id for each store. Then:

flowchart TD
    A[walk data/**/*.md, sha256 each] --> B{path in manifest?}
    B -- no --> U[upload to store]
    B -- yes --> C{sha changed?}
    C -- no --> S[skip, unchanged]
    C -- yes --> D[delete old doc id] --> U
    U --> P[poll the indexing operation until done]
    P --> M[upsert manifest: path, sha, doc id]
    A --> R{manifest path missing on disk?}
    R -- yes --> X[delete doc from store, drop manifest row]
    R -- no --> S
    M --> I{set of path+sha changed?}
    S --> I
    X --> I
    I -- no --> K[corpus index unchanged]
    I -- yes --> G[summarize new files with flash-lite, one line each] --> J[store corpus_index]

    classDef decision fill:#f4d35e,stroke:#b8991f,color:#1a1a1a
    classDef start    fill:#e9ecef,stroke:#6c757d,color:#1a1a1a
    classDef net      fill:#5ee6c8,stroke:#1f9c86,color:#1a1a1a
    classDef local    fill:#6ea8ff,stroke:#2f5fc4,color:#1a1a1a
    classDef idx      fill:#9d8cff,stroke:#5b4bcc,color:#1a1a1a

    class B,C,R,I decision
    class A start
    class U,D,P,X net
    class S,M,K local
    class G,J idx

Unchanged files are skipped. Changed files have their old document deleted first, then get re-uploaded. Files that vanished from disk get deleted from the store.

Each upload is a multipart POST that returns a long-running operation, and you poll it until done before you trust the document id. Four uploads run in parallel. Serial, a 71-file corpus takes about ten minutes. Parallel, a few.

The chunking is set per upload, and I landed on 400 tokens with 60 of overlap. The docs' example is 200 and 20, which for a book felt like reading through a letterbox.

meta := map[string]any{
    "displayName": displayName,
    "chunkingConfig": map[string]any{
        "whiteSpaceConfig": map[string]int{
            "maxTokensPerChunk": 400,
            "maxOverlapTokens":  60,
        },
    },
}
// multipart/related: part 1 is this JSON, part 2 is the markdown bytes
// POST /upload/v1beta/{store}:uploadToFileSearchStore
// with X-Goog-Upload-Protocol: multipart

The last box in that flowchart is the corpus index, and it is small but it is the thing that makes the next section work.

For every file, a one-sentence summary from the cheapest model available (gemini-3.5-flash-lite), keyed by the file's sha so it is only ever generated once. All the lines get joined into one block, "path, summary", that gets pasted into the analysis prompt.

That way the model deciding what to search for knows what is actually on the shelf. It aims at sources that exist instead of guessing.

Search with the pattern, not the post

This is the idea in the post that I would keep if I had to throw everything else away.

Embedding search finds things that sound alike.

Paste a proposal that says "we should standardize on one MCP transport across all our agents" straight into retrieval, and you get back every chunk that contains the words "MCP", "agents", and "standardize". Which is your own recent blog posts about MCP and agents.

That is not precedent. That is a mirror.

What you actually want is the shape of the situation with the nouns removed.

"Fragmented, incompatible implementations, then one open standard, then mass adoption."

Search the shelf with that, and the browser wars come back. Rickover on standardizing the nuclear navy comes back. A 2,300-year-old Legalist essay on uniform law comes back.

Same pattern, different clothes. That is what a reviewer with thirty years of reading brings, and it is what the model cannot do if you let it search with the raw text.

So the pipeline never lets it.

Call 1 gets the lawbook, the corpus index, and the proposal, with no tools attached. It returns JSON: the motive, what is actually happening, who the actors are, the generalization, and two or three retrieval queries derived from that generalization.

Call 2 gets the analysis from call 1, the File Search tool pointed at the store, and the instruction to search with those queries and then draft.

The raw document is in call 2's context, so the draft can quote it. But the search terms came from call 1, and they are about the pattern, not the subject.

{
  "systemInstruction": { "parts": [{ "text": "...the lawbook..." }] },
  "contents": [{ "role": "user", "parts": [{ "text": "PROPOSAL:\n...\n\nANALYSIS FROM CALL 1:\n{...}\n\nSearch the corpus with the retrieval queries above, then reply with the JSON contract." }] }],
  "tools": [{ "fileSearch": { "fileSearchStoreNames": ["fileSearchStores/abc123"] } }],
  "generationConfig": { "responseMimeType": "application/json", "temperature": 0.7 }
}

The response carries candidates[0].groundingMetadata.groundingChunks, each with the file's display name and the retrieved text. Those get shown to the user, in full, next to the draft. If the "authority" the review leans on is not in that list, it did not come from the shelf, and the checks catch that.

Every instruction is a law, every reply cites its laws

There is no free-text system prompt anywhere in this thing.

The prompts are an AgentLaws lawbook: a folder of markdown where every instruction the model sees is a numbered law, grouped into chapters, versioned in git, and compiled once at startup.

The model is required to return applied_laws, the numbers it relied on, and the pipeline resolves each one back to file:line in the lawbook.

A number that does not resolve fails the run. Which sounds pedantic until the first time the model confidently cites law 7.2.3 in a lawbook with six chapters.

The practical win is that "why did it do that" has an answer that is a file and a line number, and "make it stop doing that" is a pull request against a markdown file, not archaeology in a Go string.

The output contract is the last thing in the prompt, because that is where the model weights it most. Laws about tone and structure go first, the JSON schema goes last.

Trust, but run the checks

The model also fills a checks block in its own reply, a self-audit. I do not trust it, but it is useful as a second signal.

Before that gets read, a small pile of deterministic checks in plain Go runs on every draft: length bounds, no markdown, at most one question, no "see point N" references, every cited law resolved.

flowchart TD
    A[call 2: search + draft, JSON mode] --> B{reply parses as JSON?}
    B -- no --> F[resend once without JSON mode, extract first object] --> C
    B -- yes --> C[resolve every cited law number to file:line]
    C --> D{all citations resolve?}
    D -- no --> R
    D -- yes --> E[deterministic checks: length, no markdown, one question max, distinct openers]
    E --> G{checks pass?}
    G -- no --> R{attempts left?}
    R -- yes --> H[append the failures to the user message] --> A
    R -- no --> S[ship the draft with failures listed]
    G -- yes --> V[call 3: reviewer pass, verdict + objections]
    V --> W{verdict?}
    W -- ship --> O[return JSON with sources and applied laws]
    W -- revise, rounds left --> H
    W -- revise, no rounds --> S

    classDef decision fill:#f4d35e,stroke:#b8991f,color:#1a1a1a
    classDef llm      fill:#5ee6c8,stroke:#1f9c86,color:#1a1a1a
    classDef local    fill:#6ea8ff,stroke:#2f5fc4,color:#1a1a1a
    classDef bad      fill:#ff9a5c,stroke:#c4602a,color:#1a1a1a
    classDef good     fill:#9d8cff,stroke:#5b4bcc,color:#1a1a1a

    class B,D,G,R,W decision
    class A,F,V llm
    class C,E,H local
    class S bad
    class O good

A failure gets appended to the user message as a plain list, "the previous attempt failed these checks, fix them", and call 2 runs again. Two retries, then the draft ships anyway with the failures listed, because a draft with a warning is more useful than a spinner that gave up.

One box in there deserves its own sentence: JSON mode plus a tool is not guaranteed. responseMimeType: application/json together with File Search worked most of the time and then, occasionally, returned prose with a JSON object somewhere in it. The fix is boring: if the reply does not parse, resend once without JSON mode and pull the first {...} out of the text. Both attempts get logged.

The bug that made me split the database

The tool is used by five people, and each of them runs it against their own SQLite file, so history is per person.

The first version put everything in that one file: sessions, steps, and the ingest manifest.

Spot it?

A new teammate makes a new database. New database, empty manifest. Empty manifest means every file on disk is "new", so all 71 get uploaded again. And because the manifest is empty, ingest does not know the old document ids, so nothing gets deleted.

The store now has two of everything. Do it five times and search quality quietly rots, because every query returns the same chunk five times and crowds out the second-best hit.

The fix is one sentence: the manifest is a property of the store, not of whoever is asking questions today.

So there are two databases now. corpus.db holds the store names, the manifest, and the corpus index, and every session database reads from it. alice.db holds Alice's sessions and nothing else.

Delete corpus.db and you get a full re-ingest. Delete alice.db and nothing about the corpus changes.

What the free tier actually gives you

Since "cheap" is in the title, the honest numbers, as of when I built this.

Models. Flash only. gemini-3.5-flash for the real calls, gemini-3.5-flash-lite for the throwaway summaries. Pro returns a 429 with limit: 0 on free keys. A paid key unlocks it with one env var, and I have not needed to.

Rate limits. Per-minute caps are the thing you hit, not per-day. Ingest backs off with a growing wait when every key is rate limited, and the UI shows each key fallover as it happens instead of hiding it behind a spinner.

Storage. 1 GB per store. My 5.3 MB corpus does not register.

Cost per question. Two Flash calls, sometimes three with the review pass, around 10k tokens in and 2k out. The retrieved chunks are already counted in that. On the free tier, zero. On a paid key, well under a cent.

Latency. 30 to 90 seconds end to end, most of it call 2 doing the search and the draft. That is fine for "paste a design doc, get a review" and would be wrong for a chat box.

The catch. On the free tier, Google may train on what you upload. My shelf is published books and public blog posts, so I do not care. If your corpus is your customer contracts, read the terms before you make ingest.

What I would tell you to steal

If you have a shelf, and a question shaped like "have we seen this before", you do not need a vector database to answer it.

You need markdown with real headings, a hosted store you diff against instead of re-uploading, keys with roles because the store pins you to one, and a first model call whose only job is to turn the question into the pattern underneath it.

The rest is checks and logging, and the checks are the part you will thank yourself for.

---

Your team's attention is limited, and the deluge of AI-generated code is making it harder to keep production secure and reliable without slowing you down.

I'm building LiveReview, a blast-radius aware AI code review built for your business-critical systems.

Instead of presenting every diff with equal emphasis, LiveReview scores each change by blast radius — how far its impact reaches through your call graph — so you can focus attention where it actually matters.

Spend code review effort where business risk is highest — not spread evenly across every diff.

⭐ Star it on GitHub:

HexmosTech / LiveReview

Blast-Radius Aware AI Code Review for Business-Critical Systems

     

LiveReview: Blast-Radius Aware AI Code Review for Business-Critical Systems

LiveReview is an AI code reviewer that scores every hunk of a diff by blast radius: how far a change reaches through your call graph, how much persistent state it touches, and how well-tested it is. A 3-line change to a shared auth check can outrank a 300-line UI tweak. Your team's attention goes to the highest-risk code first, not spread evenly across every diff.

blast-radius-demo.mp4

LiveReview's Blast Radius & Review Priority scoring, live in the diff viewer.

The exact math, not a black box	Visualize blast radius at a glance	Every factor that feeds the score

How does Blast Radius scoring work? (a more technical explanation)

Here's the goal:

• A 3-line fix in a function used by 40 other files, that also writes to a database, should score high.
• A 300-line UI change in one file, fully covered by…

View on GitHub

Click below to try LiveReview with your codebase:

Comments

[FreyaLi]

excellent!