DEV Community

Cover image for Choosing an embedding model for RAG (and when to switch)
PRANJUL RATHOUR
PRANJUL RATHOUR

Posted on Originally published at pranjulrathour.scult.in

Choosing an embedding model for RAG (and when to switch)

#rag #embeddings #retrieval #guide

The embedding model decides what "similar" means in your retriever. Swap it and the same query returns different chunks. Yet most projects pick one from a tutorial and never revisit it. Here is how I choose, and how I keep the option to change later.

What actually differs between models

  • Dimension — 384 versus 1024 dimensions changes index size and search cost more than it changes quality on small corpora.
  • Context length — a model that truncates at 512 tokens silently ignores the end of long chunks. Match chunk size to the model.
  • Domain — general web models handle policy PDFs well; code and legal text benefit from models trained on them.
  • Language — Hindi and Hinglish queries against English documents need a multilingual model, or you will watch retrieval fail on your own users.
  • Hosting — an API model is simplest; a local model keeps documents on your server, which some clients require.

Test before you commit

Take the evaluation set from how to evaluate a RAG system and index the corpus with two or three candidate models. Compare recall@5 and the rank of the correct passage. Differences are often smaller than tutorials suggest — and when they are small, pick the cheaper, faster or more private model.

Design for switching

Store the model name and dimension alongside each index, keep the embedding call behind one function, and make re-indexing a command rather than a migration. RAG.NextUpgrad's multi-provider fallback exists for generation, but the same discipline applies to embeddings: never let a single vendor's model become load-bearing for your architecture.

When to switch

Switch when recall on your own questions is the bottleneck and hybrid search plus reranking have already been added — not because a leaderboard changed. Leaderboards measure someone else's queries.

About Pranjul Rathour

Pranjul Rathour in a shirt and tie holding a microphone in front of a career-opportunities slide
A career session for students

Pranjul Rathour presenting with a microphone in front of a slide reading 'Now what's the conclusion?'
Presenting to a room

Pranjul Rathour on stage presenting a requirements-gathering and user-flow slide
Requirements gathering, on stage

Pranjul Rathour seated in a black jacket and white turtleneck with an event lanyard
Pranjul Rathour

Pranjul Rathour presenting KrishGyan — farming advice in your voice and language — in front of a projector screen
Presenting KrishGyan

Pranjul Rathour is a GenAI engineer from Kanpur, India, and CTO at SCULT INDIA, currently shipping production RAG,
fine-tuning and agentic AI systems, mentoring 200+ students through TechVerse Enclave, and judging and speaking at
student hackathons across India. Updated 2026-09-06.

Reach out if you want to talk GenAI, book a campus session, or invite him to judge:

Pranjul Rathour · GenAI engineer, 3x hackathon winner, campus mentor. Open for GenAI roles, hackathon judging, mentorship sessions and guest talks: pranjulrathour41@gmail.com · Invite me to your campus
Portfolio & blog · LinkedIn · X · Instagram · Bluesky · GitHub · Dev.to

Top comments (0)