DEV Community: Arvid Andersson

RAG Retrieval Architectures: When Better Embeddings Stop Helping

Arvid Andersson — Sun, 16 Aug 2026 18:02:59 +0000

Most RAG projects start vector-first: embed the documents, store them, retrieve by similarity. It works in the demo. Then a user searches for an exact thing, a product code, an error number, a specific name, and the system misses it, because vector search ranks by meaning, not by literal tokens. This post is about that failure and the retrieval architectures that fix it, as of June 2026.

The failure pure vector search hides

An embedding turns text into a point in a space where nearby points mean similar things. That is exactly what you want for "find documents about return policies" and exactly what you do not want for "find SKU-4471". The model encodes "SKU-4471" as something close to other codes that look and read like it, so the literal one a user typed can sit just outside the top results. The same happens with error codes, part numbers, ticket IDs, acronyms, and rare proper nouns.

This failure stays hidden through testing. Semantic queries work, the demo looks done, and then a user types the one exact thing they expected to match and loses trust in the whole system. It compounds in chat, where users phrase things literally and expect literal matches. A bigger or better embedding model does not fix it, because the problem is not embedding quality, it is that exact-match is a different job from semantic similarity.

A retrieval ladder, in order

Climb only as far as your evaluation says you need to. Each rung costs more.

Hybrid search. Run BM25 (keyword) and vector queries together, merge with rank fusion. Recovers exact tokens without losing semantic recall. The biggest single win for most projects.
Reranking. A cross-encoder re-scores the top candidates against the query and reorders them. Usually the next biggest quality jump after hybrid.
Query and metadata work. Query expansion, metadata filters, and better chunking. Helps when the right chunk exists but is not surfaced.
Structure-aware retrieval. Entity-aware retrieval or a graph layer (GraphRAG) for questions about entities, relationships, and identities that no single chunk answers.

Step 1: hybrid search

Hybrid search runs a lexical query (BM25 or full-text) and a vector query, then merges the two result sets, commonly with Reciprocal Rank Fusion. Lexical search nails exact terms and rare tokens; vector search nails paraphrase and meaning. The merge recovers what either alone would miss. This is the rung most projects should reach for first.

Reciprocal Rank Fusion is worth seeing in full, because it is simpler than it sounds. It ignores the raw scores from each system (which are not comparable anyway) and uses only the rank position:

def reciprocal_rank_fusion(result_lists, k=60):
    """Merge ranked ID lists. k dampens the weight of top positions."""
    scores = {}
    for results in result_lists:
        for rank, doc_id in enumerate(results):
            scores[doc_id] = scores.get(doc_id, 0) + 1 / (k + rank + 1)
    return sorted(scores, key=scores.get, reverse=True)

# bm25_hits and vector_hits are ranked lists of document IDs
merged = reciprocal_rank_fusion([bm25_hits, vector_hits])

That is the whole algorithm. A document ranked well by either system rises; a document ranked well by both rises further. Because it never compares a BM25 score to a cosine similarity, you avoid the score-normalisation problem entirely.

You can get hybrid two ways. A vector database with native hybrid support handles both signals in one query: Weaviate and Qdrant use BM25-based sparse plus dense vectors, and Pinecone pairs dense vectors with its own sparse model. Or you use a search engine built around hybrid from the start: Typesense and Meilisearch combine full-text and vector search with typo tolerance, and Azure AI Search offers full-text, vector, and hybrid in one managed service. The right choice depends on whether you already run a vector DB or want search-engine ergonomics (faceting, typo tolerance, geo) alongside retrieval.

Step 2: reranking

Hybrid search widens the net for recall, but the merged ranking is approximate. A reranker fixes the ordering: a cross-encoder reads the query and each candidate together and scores true relevance, then reorders. It is usually the highest-leverage quality change after hybrid, the lever to reach for once recall is good but precision lags.

Rerankers ship as hosted APIs alongside embeddings. Jina AI (part of Elastic since October 2025) offers a multilingual reranker next to its embeddings and reader; its reranker weights on Hugging Face are CC-BY-NC licensed, so commercial use goes through the API. Voyage AI (now part of MongoDB) provides rerank models with a free tier, focused on retrieval quality. Cohere offers a Rerank endpoint. The cost note that matters: rerank only the top candidates (often 50 to 100), not the whole result set, since a cross-encoder is far more expensive per document than a vector lookup.

Step 3: query and metadata work

If the right chunk exists but does not surface, the lever moves upstream. Query expansion generates variants of the incoming query (synonyms, expanded acronyms, alternate phrasings) and searches them in parallel, then merges, useful when users type terse or ambiguous queries. Metadata filters narrow the search space before retrieval (by date, source, language, or a field like country) which improves both precision and latency. And chunking strategy decides whether a relevant passage is even retrievable as a unit. These are worth tuning once hybrid and reranking are in place, not before.

Step 4: structure-aware retrieval

Some questions cannot be answered by ranking chunks at all. "Who is X" or "how is X related to Y" over a corpus where the answer is spread across many passages, with no single chunk acting as a summary, is a retrieval-structure problem. Entity-aware approaches build profiles or summaries per entity; GraphRAG-style approaches build a graph of entities and relationships and traverse it before or alongside chunk retrieval. RAG frameworks like LlamaIndex and Haystack provide building blocks for these patterns. This rung is real work and worth it only when evaluation shows ranking is not the bottleneck.

Build the eval set before you climb

The order above is a default, not a prescription. The way to know which rung you actually need is to build a small evaluation set first: a few dozen real queries with the documents that should answer them, then sort the failures into buckets, wrong documents retrieved, right document retrieved but answer missed it, query too vague. Each bucket points at a different rung. Adding features without measuring just moves the failure around. For the generation side of this (catching answers that cite the right chunks but draw the wrong conclusion), see the companion post on evaluating RAG quality beyond RAGAS.

Tools by step

Step	What it does	Tools
Hybrid (native)	Vector DB with built-in keyword + vector	Weaviate, Qdrant, Pinecone
Hybrid (search engine)	Full-text + vector search engine	Typesense, Meilisearch, Azure AI Search
Reranking	Cross-encoder re-scoring of candidates	Jina AI, Voyage AI, Cohere
Structure-aware	Entity and graph retrieval patterns	LlamaIndex, Haystack

Frequently asked questions

Why does my RAG system miss exact matches like SKUs or error codes?

Vector search ranks by semantic similarity, not literal token match. An embedding of "SKU-4471" or "error E-1042" lands near other codes that look similar in meaning, so the exact one a user typed can fall outside the top results. Embeddings are strong at meaning and weak at literal identifiers. The fix is hybrid search: run a keyword (BM25) query alongside the vector query and merge the results, so exact tokens are matched exactly while semantic recall is preserved.

What is hybrid search in RAG?

Hybrid search combines lexical search (BM25 or full-text) with vector search and merges the two result sets, usually with Reciprocal Rank Fusion. Lexical search catches exact terms, identifiers, and rare words; vector search catches paraphrases and semantic matches. Together they recover cases that either method alone would miss. Hybrid search is a common default in production RAG systems once they hit real queries.

Do I need a reranker if I already use hybrid search?

Often yes. Hybrid search widens the candidate set for recall, but the top of that merged list is not necessarily ordered by true relevance. A reranker (a cross-encoder model) re-scores the top candidates against the query and reorders them, which usually gives the largest precision gain after hybrid search. It adds latency and cost, so rerank a small candidate set (for example the top 50 to 100), not the whole index.

When should I move beyond hybrid search and reranking?

When evaluation shows the failures are no longer about ranking. If the right chunk is never retrieved regardless of method, the problem is upstream: chunking, metadata filters, or missing structure. Questions about entities, relationships, and identities across a corpus often need entity-aware retrieval or a graph layer (GraphRAG) rather than better embeddings. Build an eval set first so you change the layer that is actually failing.

Is hybrid search worth it for a small RAG project?

It depends on the documents. If your corpus contains identifiers, product names, codes, or rare technical terms that users search for literally, hybrid search pays off quickly. If the content is prose where meaning dominates and exact tokens rarely matter, pure vector search may be enough. The practical test is to build a small eval set with the queries you actually expect, then compare pure vector against hybrid on it.

Originally published on Infrabase.ai, where I maintain a hand-verified directory of AI infrastructure tools.

Translate i18n strings in CI with GitHub Actions: tools compared

Arvid Andersson — Sun, 14 Jun 2026 22:04:04 +0000

If your team ships a localized product continuously, translations have a way of becoming the step everything waits on. The feature is done, the PR is green, and then it sits because the German and Japanese copy is blank. The fix a lot of teams reach for is to move translation into CI: when a PR changes the source locale, something translates the new keys and commits them back before merge.

That loop, source PR in, localized product out, is the whole goal. The rest is which tool runs the middle step.

A quick summary of the options: the tools worth knowing for this are ai-i18n if you want open source and don't mind wiring it up; hosted actions like Lingo.dev for a quick CI hook; the big TMS platforms (Crowdin, Lokalise, Transifex) if you have a localization team; and Localhero.ai (ours), built around the translate-on-PR workflow with glossary consistency and a review UI for non-developers. What separates them isn't the API call. It's how consistent the output stays over time and whether a non-developer can review it.

One thing to keep in mind while you compare: getting an LLM to translate one string is easy. Getting consistent, on-brand translations across hundreds of keys and many languages, PR after PR, so "Workspace" is the same word in German every time and the tone doesn't drift, is not. And whatever you pick, someone usually still wants to review the copy that matters before it ships.

There are quite a few ways to run that step now, from open-source GitHub Actions you wire up yourself to hosted services that run as an Action. Before the tools themselves, here's what actually separates them.

What "good" looks like for CI translation

Most of these tools look similar in a demo. The differences show up a few months in, on a real product with real churn. The things that matter by then:

Delta translation. Translate only the keys that changed in the PR, not the whole file every time. It keeps CI fast and gives the model tighter context.
Glossary and tone consistency. "Workspace" should be translated the same way in PR #200 as it was in PR #5. It helps when a tool remembers the edits you've made and picks up the patterns already in your codebase, so corrections stick instead of getting re-litigated every PR.
A review surface. A YAML diff is not a place a PM or native speaker can sanity-check copy. Either you trust the output fully, or you need somewhere to review it.
Commit-back vs PR. Committing to the same PR is convenient; opening a separate translation PR is safer for some teams.

i18n translation tools that run as a GitHub Action

ai-i18n (open source GitHub Action). A solid choice if you want full control and don't mind wiring it up. It translates i18n files with an LLM provider you choose (Anthropic, OpenAI, or self-hosted Ollama), uses content hashing to translate only changed strings, and commits results back. It handles XLIFF and JSON (flat and nested). You own the config and the maintenance. Best when you want to tune everything yourself and keep the model choice in your hands.

Hosted CI-action tools (Lingo.dev and a few others). A handful of hosted services run their own GitHub Action that translates on the delta and commits back. Lingo.dev is one of the more visible ones. They cover the basic translate-in-CI loop across a range of CI providers. Where they differ from each other (and from the option below) is less the CI mechanics and more what happens around the strings: how consistent terminology stays over time, and whether a non-developer can review the output. Worth a look if you mainly want the CI hook itself.

Crowdin / Lokalise / Transifex (TMS with GitHub sync). The established translation management systems. They sync with GitHub (Crowdin opens a PR when translations update) and are built for teams with dedicated translators: assigning work, tracking who translated what, managing big translation projects. Crowdin has documented support for both react-i18next and LinguiJS. If you have a localization team and a lot of long-form content, this is the category for you. If you're a product team that just wants UI strings translated on every PR, it's usually more platform than the job needs.

Tolgee. Open source with a hosted tier, an in-context editor, and CI integration. Supports the major JS frameworks (React, Vue, Angular, Svelte, Next) plus JSON and PHP .po. Good middle ground if you want an open-source core plus a UI.

Localhero.ai (ours). Built for product teams that ship continuously. It runs as a GitHub Action, translates the changed i18n keys on each PR, and focuses on the two things that make this hold up on a real product: a glossary and brand voice that stay consistent across PRs via translation memory, and a review UI a PM or native speaker can use instead of reading a YAML diff. Works with react-i18next, LinguiJS, Rails (YAML), and Django (.po). We're also adding review for translations a PR already includes, not just the ones we generate, so teams bringing their own translations get the same review surface. It's narrower than a full platform on purpose. If you want a focused tool for the translate-on-PR workflow rather than a large platform to configure, it's worth a look. If you want to self-host or stay fully open source, ai-i18n or Tolgee fit better.

A rough decision guide

Want full control, open source, tune it yourself: ai-i18n or Tolgee.
Have a dedicated localization team and long-form content: Crowdin / Lokalise / Transifex.
A product team that ships continuously and wants UI strings translated per-PR, with glossary/brand-voice consistency and a review surface for non-devs: try Localhero.ai.

The pattern, regardless of tool

Whatever you pick, the setup that holds up is the same: translate only changed keys, protect placeholders before sending to the model, pass the glossary in context so terms don't drift, validate output before committing, and give someone a place to review brand-sensitive copy. The tool matters less than getting those pieces right.

The DIY route is genuinely viable, and we wrote up that pattern in detail here. The cost that's easy to undercount isn't the first build, it's the upkeep: the placeholder edge case that shows up six months later, the glossary logic, the model swap when prices change, the review surface someone eventually asks for. That work competes with your actual product for engineering time and attention. For some teams that ownership is worth it. For others, the reason to use a tool here is less the features and more not carrying that maintenance yourself.

Automating Translations With Your Coding Agent

Arvid Andersson — Wed, 08 Apr 2026 13:08:49 +0000

If you've been building features with Claude Code or Cursor, you know the feeling. You're in flow, the agent is writing components, wiring up routes, adding tests. Then you hit the translation strings. What's the German word for "workspace" that we should use? Is it "Arbeitsbereich" or did we decide to keep it as "Workspace"? Who should review this?

And even when you get the source strings right, the translations themselves are often a separate step. Someone coordinates them, someone else reviews them, the feature sits in a PR waiting. With AI coding agents speeding up development, this gap only gets wider.

Been working in this space for a while, and the thing that made the biggest difference was simple: just take the translation knowledge your team already has collectively, make it structured, and ensure your coding agent has access to it.

The problem isn't translating, it's consistency and brand voice

AI can translate. That's not the hard part any more. The hard part is getting translations that sound like your product, consistently, when multiple people and agents are all contributing to the same codebase.

Think about what happens without a shared glossary and style guide:

Developer A's agent translates "Workspace" as "Arbeitsbereich", Developer B's agent keeps it as "Workspace"
One PR uses formal German ("Sie"), another uses informal ("du")
A contractor pastes in ChatGPT translations with a completely different tone
Your native speaker on the team corrects the same term for the third time this month

The app ends up feeling like it was translated by five different people, because it was. Your brand voice gets lost the moment it crosses a language boundary. And every new language multiplies the problem.

This is why a glossary and style guide matter more than the translation engine. They're the single source of truth that keeps your brand voice consistent regardless of who or what writes the code.

Start with a glossary and style guide

Before you automate anything, sit down with and make some decisions. These are the things that no one, not your coding agent, not a new team member, not a translation tool, can figure out on their own easily:

Which terms should never be translated? Product names, feature names, technical terms your users know in English. "Workspace", "Dashboard", "API" might all stay as-is in German.
Which terms have a specific translation? Maybe "Save" is always "Speichern", not "Sichern". Maybe "team member" becomes "Teammitglied", not "Mitarbeiter". These are the terms your native speakers have opinions about.
What's the tone? Formal or informal? In German that's the difference between "Sie" and "du", in French between "vous" and "tu". This needs to be consistent across your whole app.
Any language-specific decisions? Scandinavian languages often sound better with a natural spoken register rather than formal written style. Japanese needs the right politeness level. These are things worth writing down once.

This is a conversation with your PM, your content person, the native speakers on your team. It doesn't take long, but the decisions need to be explicit, not just in someone's head. People usually have opinions and examples ready once you ask.

Once you have this in place, document it. Agents or tools that use it will produce translations that actually sound like your product. Without it, even the best AI will just guess differently every time.

A nice thing this opens up for is automating. The actual translations into target languages can then happen in CI. For example, a GitHub Action picks up new strings on the PR, translates them with glossary enforcement, and commits them back. By the time someone reviews, all languages are there. Think of it like linting or tests, but for translations.

You could wire this up yourself

And for a small project with a few languages, you should. Call an LLM API, write a validation script, keep a glossary file in your repo. There are open source tools like i18n-ai-translate and attranslate that let you bring your own API key and run translations locally. Good starting points.

But the translation itself is only one piece. The harder parts come later:

Consistency over time: How do you make sure translations stay consistent as your glossary evolves and new people join the team?
Quality validation: Who catches broken placeholders, wrong formality levels, glossary violations, or tone drift?
Compound learning: When someone on your team corrects a translation, does that correction inform future translations?
Non-technical review: Can your PM or content person review and edit translations without opening a JSON file?
Keeping it running: You built it, now you maintain it. Every edge case, every new language, every LLM API change is on your team. That's time spent on translation infrastructure instead of your product.

What I'm building to solve this

It's not worth rebuilding this infrastructure for every project. Localhero.ai is what I decided to build, a translation service for real product teams, not translators. The core idea is that everything compounds: every translation you ship, every correction someone makes, every glossary term you add feeds into making the next translation better.

One cool part is how it connects to coding agents. There's an agent skill that loads your glossary, tone settings, and naming conventions dynamically every time your agent works on translation-related code. Install it with npx skills add localheroai/agent-skill and it works with Claude Code, Cursor, and any agent that reads skills.sh skill files. If your PM adds a new term to the glossary or adjusts the style guide, every developer's agent picks that up on the next task. No syncing, no Slack messages.

On the CI side, a GitHub Action ensures everything is in sync. Every PR that touches locale files gets translated automatically with glossary enforcement, translation memory, and language-specific rules for things like German formality or Scandinavian natural register. Just write the source language and let CI handle the localization.

There's a lot more going on under the hood than just calling an LLM. Quality checks catch broken placeholders, glossary violations, and tone drift before anything lands in your PR. Every PR gets a dedicated review page where anyone on the team can edit translations inline or apply suggested fixes. And when someone corrects a translation, that decision feeds back into translation memory, so the system learns from your team's choices over time. It's like working with a translator that actually remembers what you decided last month.

It works with JSON (React, Next.js, Vue), YAML (Rails), and PO files (Django, Python). The agent skill and CLI are open source, and there's a free plan if you want to try it on a real project.

The compounding part

The thing I like most about this setup is that it gets better on its own. Every translation you ship, every correction your team makes, every glossary term you add feeds forward. Six months in, the system knows your voice better than any new hire would. That's the part you can't get from a script and an API call.

If you're shipping in multiple languages and it still feels like a separate workstream, start with the glossary. Get your team's decisions documented, give that context to your agent, and see what happens. And if you want help setting it up, reach out, happy to help.