A retrieval system over the SEPA payment rulebooks. Plain Python, Postgres, two LLM vendors. Three independent decisions from it are worth writing down, because in each case the decision was cheap and the thing it taught was not the thing it was made for.
1. The vector database never got installed
Every RAG guide starts in the same place. LangChain's own documentation
puts it plainly: to build RAG, you first need to create a vector store. Then the shortlist writes itself: Pinecone, Qdrant, Weaviate, Chroma.
The vectors live in Postgres instead, through pgvector.
Not because Postgres is faster. Because a new datastore is a new failure surface, with its own consistency model, its own operational habits and its own way of going wrong at 2am. Postgres was already there holding the documents, and I already knew how to debug it. Scaling was never the plan for a corpus of one rulebook.
The whole of the retrieval query is this:
SELECT id, source, text, page, embedding <=> %s::vector AS distance
FROM chunks
ORDER BY distance ASC
LIMIT %s
<=> is cosine distance: 0 is identical, 2 is opposite. The ::vector cast is not decoration. A bare Python list arrives as double precision[], which that operator refuses, so the vector travels as a text literal and is cast on arrival.
One operator, one cast, one index. That is the entire integration.
Convenience is not proof, though. Would the real vector database have bought back time worth having?
One question, end to end, takes about 2,823 ms. The vector search inside it takes 4.9 ms. Replacing pgvector with something infinitely fast would return 0.17% of the wait.
None of this says a vector database is wrong. It says the unfamiliar one would have been paid for in operations and returned a line nobody was waiting on.
2. You can only measure what you own
No LangChain, no LlamaIndex either. The pipeline is plain Python calling two LLM vendors and a Postgres.
The usual defence is YAGNI, and it holds: the flow is five fixed steps with no branching. Embed the question, search, build a prompt, call the model, parse the answer and its citations. Orchestrating five fixed steps is the easy part. A framework earns its keep on branching, retries and swappable backends, and none of those were in play.
But that is not the reason worth giving.
Every result worth publishing from this project came out of a layer a framework would have owned.
I wrote the chunker, so the repeated headers could be stripped from every page and the effect measured: retrieval did not move at all. I wrote the rank fusion, so it was visible that hybrid search scored exactly the same as dense-only, and that the corpus was too small to show a difference rather than the technique being wrong. I wrote the database call, so it could be timed alone and found to be 4.9 ms.
Three findings, all negative, all only visible from inside.
The cost is real and there is a file listing it: mature libraries exist for the chunker, the fusion, the metrics, the eval harness. That file is the off-ramp for when this stops being worth it.
You cannot instrument an abstraction you did not build.
3. Two dependencies called "model", two very different bills
The model this system was planned around was retired before the first commit. The decision record replacing it carries the same date as that commit. The project lost its LLM before it had a second file.
The swap itself was trivial. The responder is one environment variable:
LLM_MODEL: str = os.environ.get("LLM_MODEL", "claude-haiku-4-5")
One line, one deploy, done. Now the other model.
The embedding model cannot move like that. The table declares its vectors as 1536 dimensions and an insert guard rejects anything else, because vectors from different embedding models do not share a space. Changing it means re-embedding the entire corpus before a single query works again. An afternoon at this size. A migration at a real one.
Same word, "model". One is a config value. The other is infrastructure wearing a config value's clothes.
The vendor's deprecation schedule does not read your roadmap. Decide which of your dependencies is which before it decides for you.
Top comments (0)