Self-Hosted Vector Databases in 2026: Qdrant vs pgvector vs Milvus vs Chroma for Production AI

Managed vector databases like Pinecone and Weaviate Cloud charge eye-watering sums once your embedding collections grow into the millions of vectors. For startups and engineering teams deploying local LLMs, retrieval-augmented generation (RAG), and semantic search, self-hosting your vector database eliminates vendor lock-in and slashes monthly infrastructure bills from hundreds of dollars to a $10–$20/mo VPS.

In this deep dive, we compare the top 4 self-hosted vector databases in 2026—Qdrant, pgvector (PostgreSQL), Milvus, and Chroma—benchmarking query throughput, memory footprints, filtering capabilities, and providing production Docker Compose stacks.

---

Quick Comparison: The 2026 Vector Landscape

Database	Primary Language	Index Types	Ideal Use Case	RAM Footprint (1M 1536-dim vectors)
Qdrant	Rust	HNSW, Inverted Index	High-throughput production RAG & fast payload filtering	~2.4 GB (in-memory) / ~800 MB (quantized)
pgvector	C / SQL	HNSW, IVFFlat, HNSW with halfvec	Existing Postgres stacks needing vector search without extra infra	~3.0 GB (HNSW index)
Milvus	Go / C++	HNSW, IVF_FLAT, SCaNN, DiskANN	Large-scale enterprise (>10M+ vectors, distributed clusters)	~4.5 GB+ (Distributed overhead)
Chroma	Python / Rust	HNSW	Rapid prototyping, local Python pipelines, small-to-medium collections	~2.8 GB

---

1. Qdrant: The Production Workhorse for RAG

Written in Rust, Qdrant is our top recommendation for standalone vector search in 2026. It features native payload filtering (combining metadata boolean filters directly during vector search rather than post-filtering), extreme concurrency, and native scalar/product quantization that reduces RAM consumption by up to 80% with negligible recall drop.

Why Qdrant Wins:

• Zero GC Pauses: Rust runtime guarantees deterministic sub-10ms query latency under heavy concurrency.
• Payload Indexing: Index metadata fields (e.g., tenant_id, created_at, status) so queries with complex filter conditions execute with zero latency penalty.
• Built-in Web Dashboard: Comes with an out-of-the-box web UI on port 6333/dashboard for exploring collections and running test vector queries.

Production Docker Compose Stack:

version: '3.8'

services:
  qdrant:
    image: qdrant/qdrant:v1.11.0
    container_name: qdrant_engine
    restart: unless-stopped
    ports:
      - "127.0.0.1:6333:6333" # HTTP REST API & Web Dashboard
      - "127.0.0.1:6334:6334" # gRPC API (Fastest for Python/Go)
    environment:
      - QDRANT__SERVICE__API_KEY=${QDRANT_API_KEY:-super_secret_api_key_here}
      - QDRANT__STORAGE__PERFORMANCE__MAX_SEARCH_THREADS=4
    volumes:
      - qdrant_data:/qdrant/storage
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:6333/readyz"]
      interval: 15s
      timeout: 5s
      retries: 3

volumes:
  qdrant_data:
    driver: local

---

2. pgvector: The Zero-Extra-Infrastructure Choice

If your application already relies on PostgreSQL (via Supabase, RDS, or self-hosted Postgres 16+), adding pgvector is often the cleanest architecture choice. You get ACID transactions, relational foreign keys, standard SQL queries, and vector similarity all in a single database container.

Why Choose pgvector:

• Single Source of Truth: No data syncing or double-writing between your application database and a standalone vector engine.
• HNSW Index Support: With pgvector 0.7+, Postgres supports both fast HNSW graph indexing and memory-saving halfvec (16-bit float) indexing.
• Relational Joins: Join similarity queries directly against user tables, permission tables, and tenant schemas in standard SQL:

SELECT documents.id, documents.title, 1 - (embedding <=> '[0.012, 0.045, ...]') AS similarity
FROM documents
WHERE documents.tenant_id = 'acme_corp'
ORDER BY embedding <=> '[0.012, 0.045, ...]'
LIMIT 5;

Production PostgreSQL 16 + pgvector Compose:

services:
  postgres:
    image: pgvector/pgvector:pg16
    container_name: postgres_vector
    restart: unless-stopped
    environment:
      POSTGRES_DB: app_db
      POSTGRES_USER: postgres
      POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-secure_db_password}
    ports:
      - "127.0.0.1:5432:5432"
    volumes:
      - pgdata:/var/lib/postgresql/data
    command: >
      postgres
      -c shared_buffers=1GB
      -c work_mem=64MB
      -c maintenance_work_mem=512MB
      -c max_parallel_workers=4

volumes:
  pgdata:

---

3. Milvus: Enterprise-Scale & Massive Partitioning

Milvus is designed for large-scale distributed deployments. If your vector collection exceeds 10 million embeddings and requires multi-node clustering or DiskANN storage (querying vectors stored directly on NVMe SSDs to reduce RAM requirements), Milvus Standalone is an impressive engine.

However, for single-server hobbyist or small startup stacks, Milvus has a heavier footprint due to its dependency on etcd and MinIO object storage.

---

4. Chroma: The Easiest Embedded Vector Store

Chroma has become the default vector database for LangChain and LlamaIndex tutorials. With its lightweight server container and native Python client integration, Chroma is unbeatable for rapid prototyping and local development.

For multi-tenant SaaS with millions of vectors, Qdrant or pgvector offer better concurrency and memory management, but Chroma remains exceptional for lightweight internal agents and personal automation.

---

Memory & Sizing Guide: How Much VPS RAM Do You Need?

Vector embeddings take more RAM than traditional tabular data because HNSW graphs must reside in memory for fast nearest-neighbor traversal.

Here is the hardware sizing rule of thumb for 1,000,000 vectors (using OpenAI text-embedding-3-small / 1536 dimensions):

• Raw Floats (FP32): $1,000,000 \times 1536 \times 4\text{ bytes} \approx 6.14\text{ GB}$
• With HNSW Graph Overhead ($M=16, ef=64$): $\approx 7.5\text{ GB RAM}$
• With Scalar Quantization (INT8 - Qdrant): $\approx 1.8\text{ GB RAM}$ (4x memory reduction!)
• Recommended VPS: A €6/mo Hetzner CX32 (4 vCPU, 8 GB RAM) or $12/mo DigitalOcean Droplet can comfortably serve over 1.5 million vectors with sub-15ms response times.

---

Explore More Open-Source Architecture

Looking to assemble a complete production self-hosted stack (Vector DB + Ollama + Open WebUI + Postgres + Traefik)?

👉 Check out interactive architectural stacks, hardware calculators, and pre-built Docker Compose templates on SelfHostStack.