DEV Community

Great Sage
Great Sage

Posted on

A self-hosted vector database for RAG, without the per-vector bill

#database #opensource #rag #rust

If you're building anything RAG-shaped — a chatbot with long-term memory, semantic search over docs, "find similar" for products or support tickets — you eventually need a vector database, and the default reflex right now is Pinecone or a managed Weaviate cluster. Both are fine, but they charge per vector and per query, and your embeddings live on someone else's infrastructure whether you like it or not.

Qdrant is the open-source alternative that's actually good enough to run this in production yourself. It's written in Rust, does sub-millisecond nearest-neighbor search, supports payload filtering (so you can combine "similar to this" with "where category = X"), and speaks REST + gRPC with official clients for Python, JS/TS, Rust, Go, Java, and .NET. If you've used Pinecone's SDK, Qdrant's client feels familiar on purpose.

What self-hosting actually looks like

The stock image is qdrant/qdrant. Two ports matter: 6333 for REST, 6334 for gRPC. Data — collections, the vector index, everything — lives under /qdrant/storage, so that path needs a real volume or you lose your embeddings on every restart. That's the part people miss first: spin up Qdrant in a container without a persistent volume mounted there and it "works" until the container recycles, then your collections are gone.

Second thing that's easy to get wrong: Qdrant ships with no auth by default. Every route — including the ones that let you delete a whole collection — is wide open unless you set QDRANT__SERVICE__API_KEY. If you're exposing it to the internet at all (vs. keeping it on a private network next to your app), set that key before you do anything else. Only the healthcheck routes (/healthz, /livez, /readyz) stay open with the key set, which is what you want for a load balancer or platform healthcheck to still work without auth.

Quick raw-Docker version if you want to try it locally first:

docker run -p 6333:6333 -p 6334:6334 \
  -v $(pwd)/qdrant_storage:/qdrant/storage \
  -e QDRANT__SERVICE__API_KEY=your-key-here \
  qdrant/qdrant:v1.18.3
Enter fullscreen mode Exit fullscreen mode

Then create a collection, add a vector, search it:

curl -X PUT 'http://localhost:6333/collections/test' \
  -H 'api-key: your-key-here' -H 'Content-Type: application/json' \
  -d '{"vectors":{"size":4,"distance":"Dot"}}'

curl -X PUT 'http://localhost:6333/collections/test/points' \
  -H 'api-key: your-key-here' -H 'Content-Type: application/json' \
  -d '{"points":[{"id":1,"vector":[0.1,0.2,0.3,0.4],"payload":{"t":"example"}}]}'

curl -X POST 'http://localhost:6333/collections/test/points/search' \
  -H 'api-key: your-key-here' -H 'Content-Type: application/json' \
  -d '{"vector":[0.1,0.2,0.3,0.4],"limit":1}'
Enter fullscreen mode Exit fullscreen mode

There's also a built-in web dashboard at /dashboard for poking around your collections visually, which is nice when you're debugging why a search isn't returning what you expect.

One-click if you'd rather not manage the box

I maintain a Railway template for this — full disclosure, I get a kickback if you deploy through it: https://railway.com/deploy/qdrant-vector-data-1?referralCode=Z1xivh&utm_medium=integration&utm_source=template&utm_campaign=inventory

It's the stock qdrant/qdrant:v1.18.3 image, nothing patched, with the volume and API key auth already wired up — QDRANT__SERVICE__API_KEY auto-generates on deploy, PORT is set to 6333 so Railway's proxy hits the REST port, and the volume is mounted at /qdrant/storage so collections survive redeploys. Same functional setup as running it yourself, just skips the "wait, why did my data disappear" step.

Honest limitations

Qdrant is memory-resident — vectors get held in RAM for fast search, so your RAM bill scales with vector count × dimensionality, not with disk usage. A few million 768-dim embeddings is a meaningfully different sizing question than a few thousand. If you're doing pure key-value-style similarity search at small scale, something like pgvector bolted onto a Postgres you already run might be less operational surface than a dedicated service. Qdrant earns its keep once you need payload filtering, hybrid search, or you're past the point where "just add a vector column" scales.

Upstream project, all credit to the actual authors: https://github.com/qdrant/qdrant

Top comments (0)