DEV Community: Yogesh23012001

[Boost]

Yogesh23012001 — Sun, 28 Jun 2026 09:58:37 +0000

Yogesh23012001

Jun 28

Advanced RAG Techniques Aren't Better. They're Better Sometimes.

#ai #rag #webdev #programming

5 min read

Advanced RAG Techniques Aren't Better. They're Better Sometimes.

Yogesh23012001 — Sun, 28 Jun 2026 06:04:24 +0000

I added five retrieval techniques to a RAG pipeline and measured each one. The most useful result was the technique that backfired.

The technique that made retrieval worse

I expected HyDE to improve retrieval.
On one query, it took context precision from 0 to 0.80 — surfacing a chunk my baseline had missed entirely, and ranking it first. A clean win.

On another query, it did the opposite. Recall collapsed from 0.80 to 0.17. The same technique didn't just fail to help — it actively dragged the right chunks out of the results.

Same pipeline. Same technique. Opposite outcomes.

That's the thing nobody tells you under the blog posts titled "Add HyDE to boost your RAG." HyDE isn't better. It's better sometimes — and knowing which times is the actual skill. Every "advanced RAG technique" I added this week turned out to be exactly this: a tool with a tradeoff, not a free upgrade. The work was never adding them. The work was measuring which one earned its complexity, on which query, and why.

This is a post about that measurement.

What I built (and the only question that mattered)

I built a RAG pipeline over Anthropic's own documentation — 15 doc pages, 667 chunks, Postgres + pgvector with an HNSW index — then bolted on five retrieval modes I could switch between and compare head-to-head:

Dense — plain vector similarity (bge-small embeddings).
Hybrid — dense + BM25 keyword search, fused with Reciprocal Rank Fusion.
Reranking — pull a broad candidate set, re-score with a cross-encoder.
HyDE — embed a hypothetical answer instead of the question.
Contextual retrieval — prepend an LLM-written, document-aware summary to each chunk before embedding.

I came at this as a backend engineer, not an ML researcher. And the backend engineer's question about any of these isn't "is it state of the art?" It's the same question you'd ask about a cache layer or a message queue: which of these actually earns its complexity? A RAG system is a distributed system with a model attached. Every component you add is something you have to operate, debug, and pay for. So — which ones pay you back?

Measure the boring baseline first

Before measuring anything fancy, I measured plain dense retrieval over a 28-question eval set:

Faithfulness: 0.96
Context precision: 0.60
Those two numbers point at two completely different problems, and conflating them is the most common RAG mistake I see. RAG fails in two independent places:

Retrieval — did you fetch the right chunks? (context precision / recall)
Generation — did the model answer faithfully from what you fetched? (faithfulness)
My faithfulness was already 0.96. The generator was not the problem — given good context, the model grounded its answer fine. The weak spot was precision at 0.60: roughly 40% of what I was feeding the model was noise. The right chunk was usually in the top-k, just buried in strays.

That reframes the whole project. Every advanced technique I was about to add targets retrieval — and retrieval was exactly the failing half. If faithfulness had been the low number, none of this would have helped; I'd have been tuning prompts instead. You can't know that until you split the metric and look. Look at the data first, then pick the tool.

Where each technique earned its keep — or didn't

Hybrid search (BM25 + dense). Dense retrieval has a blind spot: exact terms. Ask "what does cache_control: {"type": "ephemeral"} do?" and pure semantic similarity drifts toward vaguely-related caching prose. BM25 nailed the exact chunk dense missed entirely — but naive Reciprocal Rank Fusion then demoted it, because the dense ranker outvoted the one sparse ranker that got it right. Lesson: exact-term matching is a real, distinct failure mode, but fusion isn't free — RRF needs weighting, or it averages away the very signal you added it for.

*HyDE *— the rescue and the backfire. On casually-phrased, mismatched queries ("why does Claude keep forgetting what we talked about earlier?"), HyDE is magic: it writes a hypothetical answer full of the docs' actual vocabulary — "context window," "tokens," "compaction" — and embeds that, landing in the cluster the question's own words could never reach. Precision 0 → 0.80. But on queries already well-matched to the corpus, that same hypothetical answer invents detail that pulls retrieval toward the wrong region — recall 0.80 → 0.17. The fix isn't "use HyDE" or "don't." It's query-adaptive routing: apply HyDE only when the query and the corpus speak different languages.

Reranking. A cross-encoder reads query and chunk together, instead of comparing two independently-made vectors. It answered a question hybrid alone had declined — pulling the supporting chunk high enough that the generator finally had what it needed. The throughline: retrieval quality directly controls faithfulness. A faithful generator is downstream of good retrieval; you cannot prompt your way out of bad context.

Contextual retrieval. Anthropic's technique — an LLM writes a per-chunk summary situating it in its document, prepended before embedding. The catch is cost: one LLM call per chunk. Prompt caching is what makes it viable (cache the document, ~$1 per million chunks). It helped least on easy queries and most exactly where dense failed hardest — the starved, context-poor chunks. Situational, like all the rest.

Technique Earns its complexity when…
Hybrid + BM25 : the query hinges on an exact term/param dense smooths over
HyDE : the query is phrased nothing like the docs — and hurts when it already matches
Reranking : the right chunk is retrieved but ranked too low to use
Contextual retrieval : chunks are short/context-starved; you can afford the ingest cost

The detour where I stopped writing a tutorial and started doing infra

I reached for Ragas, the standard RAG eval library. With a Haiku judge, it projected ~11 hours for the full eval matrix. The culprit: Ragas wraps its judge calls in a structured-output layer that retried — and retried — every time the judge's JSON didn't validate, turning each question into an ~8-minute storm of failed calls.

So I read what the four metrics actually compute and built my own async harness. Every judge call returns a single boolean under a trivial schema — so it succeeds first try, no retries — and they all run concurrently under a semaphore. Same metrics. 221 seconds instead of 11 hours. ~50× faster.

That's the line between using a tool and understanding it. When you know your metrics well enough to implement them, you stop being hostage to a black box that's slow for reasons you can't see. Open the box, measure the thing yourself — that instinct is the whole job.

The honest close

None of this is production-ready, and I want to be precise about the gap.

For production I'd build the query router the HyDE backfire demands — classify each query, then pick the retrieval mode instead of applying one blindly. I'd run the eval matrix at real scale (every mode × every category), not the slice I have. The multi-tenant isolation I added (tenant + sensitivity filtering) needs adversarial testing, not just a passing happy path. And plenty is still unmeasured: p99 latency, cost per query, behavior on a corpus I didn't hand-pick.

But the takeaway is one every backend engineer already lives by: these techniques are tools with tradeoffs, and measuring which one helps — for your data, your queries, your failure mode — is the work. The model is new. The discipline isn't.

I expected the cheaper model to be cheaper. It cost 8.6 more.

Yogesh23012001 — Sat, 13 Jun 2026 07:44:50 +0000

I'd routed the same one-word prompt to Claude Haiku and to Gemini 2.5 Flash. Flash has the lower per-token price, so this should have been an easy win. It wasn't. Flash is a thinking model: before it answered "Paris," it spent a few dozen tokens reasoning, and reasoning is billed as output. Haiku answered in 4 tokens. Flash spent about 28 — lower unit price, far more units, ~8.6× the bill per request. I only caught it because I'd instrumented every call: tokens, cost, latency, written to Postgres. And I instrument every call because that's what you do when you've spent years keeping payment systems honest.

That instinct is the whole point. For two and a half years I built cross-border real-time payments at NPCI — the kind of system where a rounding error is an incident and a downstream going dark is a Saturday night. This year I built an LLM gateway, and I kept reaching for the same tools. That's not a coincidence.

AI infrastructure looks like a new field. Underneath, it's the systems work backend engineers have always done. A model API is a downstream dependency: it's slow, it's occasionally down, it rate-limits you, and it bills you per call. You have integrated a dependency exactly like that before — a payment processor, a KYC provider, a partner bank. The model isn't magic. It's a new kind of expensive, flaky downstream, and the hard parts — reliability, cost control, failover — are the parts we already solved.

The gateway needed a circuit breaker per provider, and I'd built exactly that at NPCI — on a payments platform where every partner integration was a downstream in a different country, each with its own SLA and its own definition of "up." My fault-tolerant Go processors wrapped those partners in circuit breakers and rate limiters for one reason: when a partner started failing, the worst move was to keep hammering it while your own worker pools filled with stuck calls. So you trip, fail fast, and let a trickle of traffic probe for recovery. CLOSED, OPEN, HALF_OPEN. Wiring those same three states around Anthropic and Gemini, the only thing that changed was the noun — "partner bank" became "model provider."

The gateway needed to meter every call into Postgres. That's an audit log, and payments runs on audit logs. Across 20+ Go services I'd keyed everything on idempotency so a retried request never double-counted; the cost log keys on request_id for the identical reason. And money was always fixed-precision, never a float — which is why cost_usd is a NUMERIC, not a float. You don't approximate money, and you don't approximate spend.

Retries versus the breaker — transient versus sustained — was muscle memory. In payments a careless retry is a double-debit, so one timeout means an idempotent retry, never a blind resubmit; sustained failure means you stop sending. I wrote that distinction into financial workflows for years. Rewriting it for an LLM call, the cost of getting it wrong fell from someone's money to a wasted token — but the shape of the problem didn't move an inch.

I won't pretend it was all familiar. A few things were genuinely new, and they're all about the model itself. Token economics is a real discipline — the 8.6× surprise up top doesn't exist in any database I've ever billed for. A model is non-deterministic in a way a datastore isn't: the same input can return a different output, so you can't assert on exact strings, and "testing" becomes evals and distributions instead of equality. And a model quietly spending your output budget on its own reasoning has no equivalent in a payment switch. That's the new surface area — but it's a few weeks of new sitting on years of foundation, not a new career.

So if you're a backend engineer wondering whether your distributed-systems experience transfers to AI: it doesn't just transfer — it's the scarce part. Anyone can call an LLM API in an afternoon. Far fewer people can make that call reliable when the provider degrades, cheap when the token math turns against you, and observable when someone asks you to explain the bill. That's not AI expertise. That's the work you've already been doing, with a model attached.

The gateway is live at https://llm-gateway-python.onrender.com, and the code is on GitHub: https://github.com/Yogesh23012001/llm-gateway-python/tree/main.

What a Go Engineer Learns Building Their First Real Python Service

Yogesh23012001 — Sun, 31 May 2026 16:27:36 +0000

Yogesh23012001

May 31

What a Go Engineer Learns Building Their First Real Python Service

#ai #webdev #python #career

5 min read

What a Go Engineer Learns Building Their First Real Python Service

Yogesh23012001 — Sun, 31 May 2026 16:18:04 +0000

I spent the last three years writing Go. At NPCI I built payment systems where the wrong defaults cost real money. At ShopUp I work on backend services that have to be right, fast, and observable in that order.

This weekend I built my first real Python service: an idempotent task queue with a Postgres-backed worker, retries, dead-letter queue, full Prometheus observability, and a 16-test suite. From mkdir to GitHub release in eight hours.

I want to write about what surprised me. Some of it was the language. Most of it wasn't.

What I built

The repo is here. The short version: an HTTP API that accepts tasks with an Idempotency-Key header (Stripe style), persists them to Postgres, and a separate async worker process that picks them up using SELECT ... FOR UPDATE SKIP LOCKED, runs them, and writes the outcome back.

Same key + same body returns the cached response. Same key + different body returns 422. Failed tasks retry up to max_attempts; tasks that exhaust the budget go to a DEAD_LETTER state for operator review.

This is the pattern most payment systems are built on. I've consumed it through Go libraries at NPCI; I'd never implemented it from scratch.

Benchmark on my MacBook Air M2: 590 req/s on GET /tasks at concurrency 50, p50 67ms, p99 228ms. The latency tail is dominated by Postgres connection-pool contention — pool size 10 versus 50 concurrent requests means 40 of them are waiting. That's not a Python problem; that's the same problem I'd have in any language with the same pool configuration. Production fix is PgBouncer or a larger pool.

What transferred from Go

More than I expected.
The mental model for state machines transferred cleanly. A task is PENDING, becomes PROCESSING when a worker claims it, ends as SUCCEEDED, FAILED, or DEAD_LETTER. The state guards are enforced in code the same way I'd enforce them in Go — return a 409 if someone tries an illegal transition. SQLAlchemy's enum type even maps to a Postgres task_status enum, so the database rejects invalid states too. That's the same belt-and-braces I'd build in Go.

Hexagonal architecture maps one-to-one. Models in one package, persistence in another, handlers in a third, transport (HTTP) at the edge. Pydantic models play the role of Go structs with validation tags; SQLAlchemy ORM models play the role of sqlx row types. The boundaries are identical; only the syntax differs.

Async-first thinking transferred without much friction. I expected this to be the hard part — goroutines feel native; Python's event loop is a thing you have to think about. In practice, asyncio with httpx and SQLAlchemy 2.0's async support gave me code that reads almost identically to Go. The big difference is await everywhere instead of implicit goroutine scheduling.

Database transactions as the contract. This is where I felt most at home. The race condition that hits every idempotent endpoint — two requests with the same key racing past the existence check — is handled with the same primitive in both languages: a unique constraint on the database, an IntegrityError on conflict, a re-read to find the winner. Postgres's correctness guarantees don't care what language is calling it.

What surprised me about Python

This is the section I'd been warned about. Most of the warnings were wrong.
Dependency injection feels heavier than Go's interfaces — but only at first. In Go I write a constructor, take an interface, done. In FastAPI I write a Depends() function, wrap it in Annotated, type-alias for readability, then reference it in every handler signature. It's more verbose. But after writing it a dozen times, I noticed something: the test ergonomics are actually better.

mypy --strict is a compiler — if you run it. This is the part Go engineers underestimate. Modern Python with mypy --strict plus Pydantic plus ruff catches almost everything the Go compiler would catch. The catch is "if you run it." Go enforces this on every build; Python relies on you to set up pre-commit hooks and CI. I built the hooks on day one and they paid for themselves within hours.

What I had to learn from scratch

A short list, but each item was real work:

Python's async model isn't Go's. Goroutines are preemptive, scheduled by the runtime, cheap. Python's coroutines are cooperative — they only yield at await points. If you write a CPU-bound function inside an async handler, the entire event loop blocks. This is the kind of thing the Go runtime saves you from. In Python you have to know it.
Connection pool sizing matters from minute one. When I ran my first load test (hey -n 1000 -c 50) I saw a long latency tail. I almost wrote it off as "Python being slow." Then I looked at my SQLAlchemy pool configuration: pool_size=10. With concurrency 50, 40 requests were waiting for connections. The same exact issue exists in Go; I just had production frameworks at NPCI that pre-tuned it for me. Building from scratch in Python forced me to learn what the framework was doing.

Alembic is genuinely better than go-migrate. Autogenerating migrations by diffing your ORM models against the live schema is a workflow Go doesn't really have. I was skeptical (autogen feels like magic) but reading the generated SQL before applying it is the right safety valve. I'll miss this when I'm back in Go.
Pydantic-settings makes config a non-issue. Type-safe, env-file-aware, validated at startup. Go has Viper or hand-rolled struct unmarshaling; both feel ad-hoc by comparison.

What I'd tell another Go engineer

Four things I wish I'd internalized on day one:

Don't try to make Python feel like Go. Lean into the idioms — Depends(), Annotated, Pydantic models, async context managers. Fighting the idioms wastes a week and produces unidiomatic code that other Python engineers will hate.
Set up mypy --strict and pre-commit hooks before writing a single business-logic line. This is the closest you'll get to Go's compile-time guarantees. Without it, you're writing JavaScript with type comments.
Build observability before features. I put structlog, OpenTelemetry, and Prometheus. Go engineers know this instinctively; Python tutorials skip it.
Python is faster to write than you think; slower to run than you'd hope. My benchmark hit 590 req/s. A Go equivalent would do 3-5x that on the same hardware. For most services this doesn't matter — you'll be bottlenecked on the database or the LLM API anyway. For some services it absolutely matters. Know which you're building.

The repo is here. The other repo with the FastAPI + observability + LLM gateway prototype is here. Both are pinned on my GitHub.

If you found this useful, follow me here or on GitHub — I'll be writing weekly through the rest of this transition.