Agent reading every files to answer one simple question?

Le Huy Hiep — Tue, 18 Aug 2026 05:30:57 +0000

My knowledge base grew across thousands of markdown files. Finding one fact meant reading whole files. Keyword search found words, not meaning. And reading whole files burns tokens: tens of thousands of tokens to surface one fact. The real metric is not storage. It is tokens per answer.

Why not the usual tools?

grep finds the file, but you still read it. Cost unchanged.
A cloud vector DB sends private notes out and adds infra for a 2 MB corpus.
Semantic-only search cannot do exact filters.

What I built: a local-first retrieval layer.

SQLite FTS5 index: instant keyword search, exact filters, real file:line references.
Local ONNX embeddings (384-dim): semantic ranking. Every hit points at a real file.
One-command rebuild. The index is derived, so it never goes stale.

Result: 2,000 chunks, one CLI, zero servers. An answer now costs a few hundred tokens, not a sweep through thousands of files.

Start deterministic. Add semantic where keywords fail. Rebuild often.

How to add UI for your RAG?!?

Le Huy Hiep — Mon, 03 Aug 2026 14:53:12 +0000

Just built some FastAPI SSE backend streaming LLM responses token-by-token,
after i got some free time.

Two things broke in production that worked fine locally 🙃:

nginx buffers SSE by default. The stream was delivering locally, dead
silent on Railway. proxy_buffering off, two hours later, fixed.
The LLM API returns an empty_retrieval event when no docs match. I wasn't
handling it, so the frontend sat on a loading spinner indefinitely. Added
the handler, resolved.

https://zooming-charm-production-547e.up.railway.app/

DEV Community: Le Huy Hiep

Agent reading every files to answer one simple question?

How to add UI for your RAG?!?