If you've built a RAG pipeline before, you know the pattern: hook up an embedding model, dump vectors into Pinecone, and forget about it until the invoice shows up. That invoice is where most people first realize embedding dimensionality isn't just a technical detail — it's a direct line item on your bill.
Here's what we found while building FastRAG, and why we ended up forcing 1024 dimensions instead of letting the default ride.
The problem: dimension count is a hidden cost multiplier
Pinecone (like most vector databases) charges based on storage, and storage scales linearly with vector dimensionality. A lot of popular embedding models default to 1536 or higher dimensions. That's not wrong, but it's often more resolution than the retrieval task actually needs — especially for the kind of document-chunk semantic search most RAG apps are doing.
The math is simple: every vector at 1536 dimensions costs roughly 50% more to store than the same vector at 1024 dimensions. Multiply that across every chunk of every document a user uploads, and it adds up fast once you have real usage.
Why 1024 and not lower
We didn't pick 1024 arbitrarily. A few considerations:
- Retrieval quality holds up. For chunk-level semantic search (as opposed to fine-grained tasks like clustering or classification), 1024 dimensions preserves enough of the embedding space's structure that nearest-neighbor retrieval quality doesn't meaningfully degrade for most document types.
- It's a clean truncation point. Many embedding models support Matryoshka-style representation learning or clean dimensionality reduction to 1024 without retraining, which means you're not fighting the model to get there.
- Diminishing returns above it. Going from 512 → 1024 tends to show a noticeable jump in retrieval quality. Going from 1024 → 1536 shows a much smaller one, for most general-purpose RAG use cases. You're paying for resolution you can't fully use.
What this actually saved
Forcing this dimensionality across our ingestion pipeline reduced Pinecone storage costs by about a third compared to running with the un-truncated default. That's not a marginal optimization — for anyone running a document-chat product with meaningful upload volume, it's the difference between a Pinecone bill that scales sublinearly with growth and one that doesn't.
How it fits into the pipeline
In FastRAG's ingestion flow, this is enforced at the point of embedding generation, before anything touches the vector store — so it's not a post-hoc cleanup step, it's baked into lib/vector-store.ts from the start. Every chunk, whether it came from a scraped URL or an uploaded PDF, gets embedded and truncated consistently, which also avoids a subtler bug: mixing dimensions across your index, which some vector DBs won't even let you do without a full re-index.
The takeaway
If you're building a RAG app and haven't looked at your embedding dimensionality, it's worth five minutes to check. It's one of the few places where a config-level decision has a direct, compounding effect on unit economics — the kind of thing that's easy to ignore early and expensive to fix later once you have real data volume in the index.
If you want this pre-configured rather than tuning it yourself, that's exactly what FastRAG does out of the box — Pinecone and LangChain wired up with sane defaults, including this one.
Questions about the tradeoffs, or how this interacts with specific embedding models? Drop them in the comments — happy to go deeper on the retrieval-quality side too.
Top comments (0)