Most teams configure their vector index once during initial setup, run some tests, see good results, and never touch it again. This works fine until query volume grows, the document corpus expands significantly, or retrieval quality starts degrading in ways that are difficult to diagnose.
The index configuration decisions that seem unimportant at small scale become load-bearing at production scale. I want to describe the specific decisions that matter and why, because the documentation for most vector databases explains what the parameters do but not how to think about setting them for a real enterprise deployment.
The most consequential configuration decision is the choice between exact nearest neighbor search and approximate nearest neighbor search. Exact search always returns the true top-k most similar vectors. Approximate search returns results that are similar but not guaranteed to be the exact top-k, in exchange for dramatically faster query times.
At a corpus of ten thousand documents, exact search is fast enough that the choice does not matter much. At a million document chunks, exact search becomes prohibitively slow and approximate search is the only practical option. The transition point where this matters is lower than most teams expect because enterprise knowledge bases grow faster than anticipated when you include email archives, Slack history, meeting transcripts, and support tickets alongside structured documentation.
Understanding which approximate nearest neighbor algorithm your vector database uses and what its quality-speed tradeoffs are is not optional for production deployments. Pinecone uses HNSW internally. Weaviate uses HNSW. Qdrant supports both HNSW and a scalar quantization variant. Chroma uses HNSW by default. The key parameters for HNSW are ef_construction (controls index build quality, higher is better but slower to build) and ef (controls search quality at query time, higher is better but slower to search).
# Weaviate HNSW configuration example
schema = {
"class": "EnterpriseDocument",
"vectorIndexConfig": {
"distance": "cosine",
"ef": 256, # search quality parameter, higher = better recall, slower queries
"efConstruction": 256, # build quality parameter, higher = better index, slower builds
"maxConnections": 64, # graph connectivity, affects memory and quality
},
"vectorIndexType": "hnsw"
}
# Rule of thumb starting points:
# ef: 2x to 4x your k value (if retrieving top 10, start with ef=64-128)
# efConstruction: 2x ef minimum, often 2-4x ef for better quality
# maxConnections: 16 for most use cases, 32-64 for high-accuracy requirements
The ef parameter at query time is the one with the most direct impact on the quality-speed tradeoff and the one most worth tuning empirically against your actual query patterns. The relationship between ef and recall is not linear: going from ef=64 to ef=128 typically improves recall significantly, while going from ef=256 to ef=512 often produces diminishing returns that are not worth the latency cost.
The distance metric choice matters more than most teams realize. Cosine similarity measures the angle between vectors, making it appropriate for most text embedding use cases where the magnitude of the vector is not meaningful. Dot product similarity measures both direction and magnitude, which is appropriate for models specifically trained for dot product retrieval. Euclidean distance measures geometric distance and is less common for text retrieval.
Using the wrong distance metric for your embedding model is one of the most common silent quality problems in RAG deployments. The embedding model's documentation specifies which distance metric it was trained for. If you are using a model trained for cosine similarity with a dot product configuration, or vice versa, your retrieval quality will be systematically lower than the model is capable of. Check this explicitly; do not assume the default is correct for your model.
Quantization is the configuration area where teams most often trade quality for performance without fully understanding the tradeoff. Scalar quantization reduces each float32 embedding value to an int8, cutting memory requirements by 4x at the cost of some retrieval quality. Product quantization goes further, achieving much higher compression ratios at more significant quality cost.
For most enterprise deployments where retrieval quality matters and hardware cost is not the binding constraint, I recommend avoiding quantization or using only scalar quantization with careful evaluation of the quality impact. The memory savings from quantization are appealing, but the retrieval quality impact is real and should be measured against your evaluation set before enabling it in production.
# Qdrant scalar quantization configuration
from qdrant_client.models import ScalarQuantizationConfig, ScalarType, QuantizationConfig
quantization_config = QuantizationConfig(
scalar=ScalarQuantizationConfig(
type=ScalarType.INT8,
quantile=0.99, # clip outlier values at the 99th percentile
always_ram=True # keep quantized vectors in RAM for faster access
)
)
# Measure quality impact before enabling:
# Run your eval suite with and without quantization
# Accept only if recall@5 degrades by less than your acceptable threshold
The index segment size is a configuration parameter that rarely appears in getting-started guides but matters significantly for write-heavy workloads. When you continuously ingest new documents, the index is updated incrementally. If the segment configuration is not tuned for your ingestion rate, you can end up with a fragmented index that serves queries from many small segments rather than a few large optimized ones, degrading query performance progressively over time.
Most vector databases handle this with periodic index optimization or merging. Make sure this is configured to run automatically rather than relying on manual intervention. The performance degradation from a fragmented index is gradual and easy to miss in monitoring until it becomes significant.
The configuration I described above represents the decisions worth understanding for any production enterprise RAG deployment. The specific values to use for your deployment depend on your corpus size, your query volume, your latency requirements, and your hardware. The right approach is to treat these as parameters to tune empirically against your evaluation set rather than values to set once and forget.
Index configuration is infrastructure. Like all infrastructure, it requires ongoing attention as the system scales and as usage patterns evolve. The teams that build and maintain excellent RAG systems treat index tuning as a recurring operational practice, not a one-time setup task.
Top comments (0)