DEV Community: Ameer Hamza

Retrieval-Augmented Generation (RAG) for Backend Engineers

Ameer Hamza — Sun, 05 Jul 2026 20:00:37 +0000

Introduction

RAG does not make your LLM smarter. It gives your LLM a reference sheet.

Retrieval-Augmented Generation is simple in concept: search your documents, inject the relevant chunks into the prompt, and let the model answer using that context. In production, it is a distributed query pipeline where chunking, embedding model choice, index freshness, re-ranking, and context window limits each can silently degrade answer quality.

Most RAG failures are not generation failures. They are retrieval failures. The model answered correctly based on the wrong context you gave it.

Why This Matters

If you have debugged a slow SQL query or a stale cache, you already understand RAG failure modes. The generation step is the visible symptom. The retrieval step is often the root cause.

A support bot that cites the wrong policy version did not necessarily hallucinate. It may have retrieved an outdated chunk from a vector index that was never re-embedded after a docs update. Your job as a backend engineer is to treat RAG like any other data pipeline: measurable, observable, and testable at every stage.

Prerequisites

This article assumes you have read Blog 001. You should understand that LLMs are probabilistic token predictors without guaranteed grounding.

The Problem

The naive pattern:

User question -> LLM -> Answer

fails for domain-specific factual questions because the model has no access to your private, current documentation at inference time.

The naive RAG pattern:

User question -> Vector search -> Stuff top-K chunks -> LLM -> Answer

often fails silently because:

Chunks are too large or too small, splitting tables across boundaries or burying the answer in noise.
Embeddings miss semantic intent, especially for short queries or domain jargon.
The index is stale after documentation updates.
Top-K without re-ranking returns plausible but wrong passages.
Context overflow truncates the one chunk that contained the answer.

Understanding the Core Concept

RAG separates knowledge storage from language generation:

Component	Role	Failure mode
Ingestion	Parse, chunk, embed documents	Bad chunks, lost structure
Index	Store vectors for similarity search	Stale vectors, wrong metric
Retrieval	Find candidate passages for a query	Low recall, wrong neighbors
Re-ranking	Re-order candidates by relevance	Skipped step, latency spike
Generation	Synthesize answer from context	Ignores context, hallucinates beyond it

The LLM is the last mile. Search quality is the first mile.

Chunking strategies

Fixed-size chunks (for example, 512 tokens with overlap) are simple but may split sentences and tables. Semantic chunks split on paragraph or section boundaries for better coherence. Parent-child chunking retrieves small chunks for precision but injects larger parent context for generation.

Overlap, typically 10-20% of chunk size, reduces boundary artifacts where the answer spans two chunks.

Embedding model choice

The embedding model maps text to vectors where cosine similarity approximates semantic relatedness. A mismatch between embedding model and domain (legal, medical, code) hurts recall. For RAG quality, embedding choice often matters more than generator model choice.

Hybrid retrieval

Dense vector search alone struggles with exact identifiers (SKUs, error codes, function names). Combining BM25 keyword search with dense retrieval (hybrid search) improves recall on production workloads.

How It Works Internally (High Level)

Offline: Documents are parsed, chunked, embedded, and stored in a vector index with optional metadata filters.
Online: User query is embedded with the same model used at ingest time.
Search: Approximate nearest neighbor (ANN) search returns top-K candidates.
Re-rank (optional): A cross-encoder or lightweight reranker scores query-passage pairs.
Prompt assembly: System instructions plus retrieved passages plus user question.
Generation: LLM produces an answer constrained by provided context.

Step-by-Step Example

Question: "What is the refund window for annual plans?"

Embed the query.
Search index for top-5 chunks by cosine similarity.
Re-rank so the passage about annual billing rises to the top.
Assemble prompt with system rule: "Answer only from context. If unknown, say so."
Generate with low temperature (0.1-0.3).
Log query hash, chunk IDs, scores, and latency per stage.

If retrieval returns a chunk about monthly plans only, the model will answer confidently about monthly plans. Debug retrieval first.

Architecture

Highlight retrieval in your monitoring. That is where most failures happen.

Python Example

Minimal RAG retrieval loop using sentence embeddings and cosine similarity. For production, use a dedicated vector database.

"""
Minimal RAG retrieval demo.
Requires: pip install sentence-transformers numpy
"""
import numpy as np
from sentence_transformers import SentenceTransformer

model = SentenceTransformer("all-MiniLM-L6-v2")

DOCUMENTS = [
    "Annual plans: refunds available within 14 days of purchase.",
    "Monthly plans: no refunds after billing cycle starts.",
    "Enterprise plans: custom refund terms per contract.",
]

def embed_texts(texts: list[str]) -> np.ndarray:
    vectors = model.encode(texts, normalize_embeddings=True)
    return np.array(vectors)

def retrieve(query: str, docs: list[str], top_k: int = 2) -> list[tuple[str, float]]:
    doc_vectors = embed_texts(docs)
    query_vector = embed_texts([query])[0]
    scores = doc_vectors @ query_vector
    ranked = sorted(zip(docs, scores), key=lambda x: x[1], reverse=True)
    return ranked[:top_k]

def build_prompt(query: str, contexts: list[str]) -> str:
    joined = "\n\n".join(f"- {c}" for c in contexts)
    return (
        f"Context:\n{joined}\n\n"
        f"Question: {query}\n\n"
        "Answer using only the context above."
    )

if __name__ == "__main__":
    query = "Can I get a refund on an annual subscription?"
    hits = retrieve(query, DOCUMENTS, top_k=2)
    for doc, score in hits:
        print(f"[{score:.3f}] {doc}")
    prompt = build_prompt(query, [d for d, _ in hits])
    print("\n--- Prompt ---\n", prompt)

Add chunking pipeline, metadata filters, reranker, and an eval set with recall@K metrics.

Real-World Applications

Internal documentation Q&A over wikis and PDFs
Customer support bots grounded in help center articles
Code assistants retrieving relevant files from a repository index
Compliance workflows requiring citations to source documents
Hybrid search combining BM25 keyword match with dense embeddings

Performance Considerations

Latency: Retrieval adds 50-300ms depending on index size, reranker, and filters. Budget it in your SLA.
Cost: Embedding at ingest time plus query-time embedding. Re-embedding an entire corpus on model swap is expensive.
Freshness: Event-driven re-index on document change beats nightly batch for accuracy.
Recall vs precision: Higher top-K improves recall but increases prompt tokens and noise.

Common Mistakes

Debugging prompts before debugging retrieval.
No eval set with labeled query-document pairs.
Chunking PDFs naively, destroying tables and lists.
Skipping re-ranking when top-K ANN results are noisy.
Assuming the LLM will ignore irrelevant context.

Interview Questions

Q1: What is RAG in one sentence?

A: A pattern that retrieves relevant documents at query time and injects them into the LLM prompt so answers are grounded in external knowledge.

Q2: Where do most RAG failures occur?

A: In retrieval: wrong chunks, stale index, poor embeddings, or insufficient recall.

Q3: How does RAG differ from fine-tuning for knowledge?

A: RAG injects facts at query time from an updatable index. Fine-tuning changes model weights and behavior but does not reliably store volatile facts.

Q4: What metrics would you track for a RAG pipeline?

A: Recall@K, MRR, retrieval latency, rerank latency, faithfulness, citation accuracy, and end-to-end answer correctness.

Q5: Why use chunk overlap?

A: To prevent answers from being split across chunk boundaries where neither chunk alone contains the full answer.

Q6: When would you add a re-ranker?

A: When ANN search returns semantically nearby but task-irrelevant passages, especially with short queries or large corpora.

Production Checklist

Before shipping RAG to production, verify each layer:

Ingestion idempotency: Re-running ingest on the same document produces the same chunk IDs or upserts cleanly.
Version tags: Store embedding model name and version on every vector.
Retrieval eval: Recall@5 above your threshold on a labeled set of at least 100 queries.
Faithfulness eval: Answers cite only retrieved text on a held-out Q&A set.
Latency budget: p95 retrieval under your SLA (often 200ms excluding LLM).
Failure logging: Log empty retrieval, low scores, and truncated context.

Index freshness patterns

Pattern	Freshness	Complexity
Nightly batch	Hours stale	Low
Event-driven on doc update	Minutes	Medium
Write-through on publish	Near real-time	High

For policy and pricing docs, event-driven re-index is usually worth the engineering cost.

When RAG is not enough

RAG handles lookup over static or slow-changing text. It struggles with:

Real-time transactional data (use tools or SQL instead)
Multi-hop reasoning across many documents (consider agent workflows or graph retrieval)
Computations (the model may hallucinate arithmetic; use a calculator tool)

Combine RAG with MCP tools (Blog 003) when answers require live system state.

Design Tradeoffs: Chunk Size

Chunk size	Pros	Cons
128-256 tokens	Precise retrieval	May lack surrounding context
512-1024 tokens	Good default for prose	Tables may split awkwardly
2000+ tokens	Full section context	Lower precision, higher noise

Tune on your eval set. Legal and API docs often need structure-aware chunking (by heading or OpenAPI operation), not fixed token windows.

Extended Walkthrough: Debugging a Wrong Answer

A user asks: "Do annual plans include phone support?"

Check generation: model cited "Priority email support for all plans."
Check retrieval logs: chunk support_tiers_v3.md#chunk-14 scored highest.
Open chunk: it describes monthly plans only; annual tier is chunk-22.
Root cause: embedding confused "annual" and "monthly" in short query.
Fix: add metadata filter plan_type=annual when query classifier detects billing intent; add reranker; expand eval queries.

Without retrieval logs, you would have tweaked the system prompt for days.

Observability Fields

Log per request:

query_text_hash
embedding_model_version
retrieved_chunk_ids with scores
rerank_scores if applicable
prompt_token_count
answer_faithfulness_score if automated

These fields make RAG debuggable like any distributed pipeline.

Eval Metrics Reference

Metric	Measures	Target direction
Recall@K	Relevant doc in top K	Higher
MRR	Rank of first relevant	Higher
nDCG	Graded relevance ranking	Higher
Faithfulness	Answer supported by context	Higher
Answer correctness	End-to-end vs gold	Higher
Latency p95	Retrieval plus generation	Lower

Run retrieval and generation evals separately before combining. A 10% retrieval recall gain often beats switching to a more expensive LLM.

Anti-Patterns in RAG Products

Dump entire wiki: Retrieval exists but pipeline sends 50 random pages.
No citations: User cannot verify answers; trust erodes on first error.
Single embedding for code and prose: Split indexes or use hybrid search.
Ignore ACLs: Vector index returns docs the user cannot access.
Skip ACL sync on delete: Removed permissions still retrievable until reindex.

Enforce document-level permissions at retrieval time, not only at the UI.

Summary

RAG is a search pipeline with an LLM at the end. Treat chunking, embedding, indexing, and retrieval as first-class engineering problems. Measure retrieval quality before tuning generation. Your RAG system is only as good as the search layer underneath it.

LLMs Explained for Backend Engineers

Ameer Hamza — Sat, 04 Jul 2026 19:37:44 +0000

Introduction

If you have built APIs, databases, and distributed systems, you already have the mindset needed for AI engineering. The missing piece is a clear mental model of what a Large Language Model (LLM) actually is.

An LLM is not a search engine with better grammar. It is not a database of facts. It is a probabilistic token prediction engine trained on massive text corpora. You give it a sequence of tokens. It predicts the next token. Repeat that thousands of times and you get text that reads like an answer.

That single distinction explains both the power and the unreliability of modern AI applications.

Why This Matters

Backend engineers are comfortable with unreliable dependencies. Caches go stale. Third-party APIs return 500s. Queues back up. We design retries, circuit breakers, fallbacks, and observability around those failures.

LLMs are another unreliable dependency, but with a twist: they fail confidently. A wrong answer often looks as polished as a right one. There is no HTTP status code that says "this paragraph is hallucinated."

Production AI engineering is therefore the discipline of wrapping a non-deterministic core inside a deterministic system: retrieval, guardrails, parsers, eval gates, and human escalation paths.

Prerequisites

This is the first article in the AI Engineering Handbook. You should be comfortable with:

Basic Python
HTTP APIs and request/response flows
The idea of latency, throughput, and error rates in production services

No prior machine learning coursework is required.

The Problem

Teams often treat the LLM as the entire product:

User -> LLM -> Answer

That mental model breaks in production:

No grounding: The model invents facts not present in training data for your domain.
No memory contract: Stateless APIs do not remember prior sessions unless you build memory.
No permission boundary: The model will attempt any completion the prompt allows.
Unbounded cost: Token usage scales with input and output length.
Variable latency: Time to first token and total generation time depend on model size and load.

The model is the easy part. The system around it is where engineering begins.

Understanding the Core Concept

Tokens, not words

LLMs operate on tokens, subword units produced by a tokenizer. The phrase "unhappiness" might be one token or three depending on the tokenizer. This matters for:

Billing: API pricing is per token.
Context limits: Windows are measured in tokens, not characters.
Domain failures: Rare product names may split into many tokens or map to unknown tokens, hurting quality and cost.

Next-token prediction

At each step, the model outputs a probability distribution over the vocabulary. A decoding strategy (greedy, temperature sampling, top-p) selects the next token. The process repeats autoregressively until a stop condition.

There is no separate "fact checking" step. There is no guaranteed retrieval from a knowledge base unless you add retrieval (covered in Blog 002).

Training vs inference

Phase	What happens	Engineering concern
Pretraining	Learn language patterns from huge corpora	Model choice, license, capability ceiling
Fine-tuning / alignment	Adapt behavior to instructions or domain	Data quality, forgetting, eval
Inference	Generate tokens for your prompt	Latency, cost, guardrails

As a backend engineer building applications, you mostly live in inference. You choose models, assemble context, and enforce policies around the call.

How It Works Internally (High Level)

Tokenization: Raw text becomes integer token IDs.
Embedding: Token IDs map to dense vectors.
Transformer layers: Self-attention lets each token attend to others; feed-forward layers transform representations.
Output head: Final layer projects to vocabulary logits.
Sampling: Decoding strategy picks the next token.
Repeat: Append token, update KV cache (Blog 013), continue until stop.

You do not need to implement a transformer to ship a product. You do need to know that latency and memory grow with context length and output length.

Step-by-Step Example

User question: "What is our refund policy for annual plans?"

Naive approach: Send the question directly to the LLM.

Likely failure: The model produces a plausible refund policy that does not match your actual terms.

Production approach:

Authenticate the user.
Retrieve policy chunks from your knowledge base (RAG, Blog 002).
Assemble a prompt with system rules, retrieved context, and the user question.
Call the model with temperature appropriate for factual tasks (low).
Parse structured output if needed.
Run output guardrails (no legal commitments beyond retrieved text).
Log prompt hash, retrieval IDs, latency, and token counts.

The LLM generates language. Your system decides what it is allowed to say.

Architecture

The LLM is one box in a larger service. Treat it like you would treat an external API with no SLA on correctness.

Python Example

Minimal illustration: call an OpenAI-compatible chat API and measure tokens. This is inference-only, no retrieval.

"""
Minimal LLM inference wrapper with token usage logging.
Requires: pip install openai
Set OPENAI_API_KEY in environment.
"""
import os
from openai import OpenAI

client = OpenAI(api_key=os.environ["OPENAI_API_KEY"])

SYSTEM_PROMPT = (
    "You answer using only the context provided. "
    "If the context is insufficient, say you do not know."
)

def complete(user_message: str, context: str = "") -> dict:
  messages = [
      {"role": "system", "content": SYSTEM_PROMPT},
      {
          "role": "user",
          "content": f"Context:\n{context}\n\nQuestion:\n{user_message}",
      },
  ]
  response = client.chat.completions.create(
      model="gpt-4o-mini",
      messages=messages,
      temperature=0.2,
      max_tokens=512,
  )
  choice = response.choices[0].message.content
  usage = response.usage
  return {
      "text": choice,
      "prompt_tokens": usage.prompt_tokens,
      "completion_tokens": usage.completion_tokens,
      "total_tokens": usage.total_tokens,
  }

if __name__ == "__main__":
  result = complete(
      user_message="Summarize the refund window.",
      context="Annual plans: refunds within 14 days of purchase.",
  )
  print(result["text"])
  print(f"Tokens: {result['total_tokens']}")

In production, add: timeouts, retries with idempotency keys, structured logging, and budget caps per user.

Real-World Applications

Customer support assistants with retrieval over help docs
Code assistants with repo context and sandboxed execution (Blog 004)
Document Q&A over internal PDFs and wikis
Classification and extraction with constrained output schemas
Agent workflows that call tools via standardized protocols (Blog 003)

Performance Considerations

Latency: Dominated by model size, context length, and output length. Measure TTFT and tokens per second (Blog 015).
Cost: prompt_tokens + completion_tokens at model-specific rates. Long system prompts are not free.
Concurrency: GPU memory limits concurrent sequences. Queue or route when saturated.
Caching: Identical prefix prompts may benefit from prompt caching on some providers (later in handbook).

Common Mistakes

Treating the model as source of truth for dynamic business data.
Omitting observability on prompts, retrieval IDs, and token usage.
Using high temperature for factual tasks.
Ignoring tokenizer effects on domain-specific vocabulary.
No failure mode when the model refuses or returns empty output.

Interview Questions

Q1: What is an LLM in one sentence?

A: A neural network trained to predict the next token in a sequence, used at inference time to generate text autoregressively.

Q2: How is an LLM different from a search engine?

A: Search retrieves existing documents by matching queries. An LLM generates new text from learned patterns without guaranteed grounding unless you add retrieval.

Q3: Why do LLMs hallucinate?

A: They optimize for plausible continuations, not verified truth. Without external grounding or constraints, they may invent facts.

Q4: What belongs in the system around the model?

A: Retrieval, auth, rate limits, guardrails, parsers, evals, logging, and escalation paths.

Q5: What drives LLM API cost?

A: Total tokens processed (input + output), model tier, and optional features like tool calls or vision.

Q6: When should you not use an LLM?

A: When deterministic rules suffice, when strict correctness is required without verification, or when latency and cost cannot tolerate probabilistic generation.

Summary

An LLM is a probabilistic token predictor, not an oracle. Backend engineers succeed with LLMs when they design systems: context assembly, retrieval, policy enforcement, and observability. The model generates language. Your architecture decides whether that language is safe, grounded, and useful.

Next in Series

Blog 002: Retrieval-Augmented Generation (RAG): Architecture and Tradeoffs

HTTP Protocol Deep Dive: Everything Every Backend Engineer Must Know

Ameer Hamza — Wed, 13 May 2026 10:27:35 +0000

This post is an in-depth explanation of HTTP (Hypertext Transfer Protocol), the core foundation of the internet. Whenever your frontend (browser or mobile app) communicates with a backend server, it happens through HTTP. The post explains that HTTP is inherently stateless — meaning the server does not remember previous interactions and treats every new request as independent. It also covers the different parts of an HTTP message (method, URL, headers, body) and their roles in detail. Headers are compared to address labels on a courier parcel that carry important extra information (metadata). The post further discusses API design best practices, including the correct use of HTTP methods (GET, POST, PUT, PATCH), CORS (Cross-Origin Resource Sharing), the importance of status codes (200, 404, 500), and techniques like caching and compression to improve server performance.

Key Concepts Breakdown

1. Statelessness

Simple Explanation: The server does not store any memory of previous requests. Every request must be self-contained, meaning it should include all necessary information (such as authentication tokens) to be processed.
Why It Matters: This makes backend architecture simple and highly scalable. If one server goes down, another server can handle the request without losing any session data.

2. HTTP Headers (Metadata)

Simple Explanation: Headers are key-value pairs that provide additional information about the request or response. Examples include which browser the client is using (User-Agent), the expected response format (Accept: application/json), or whether the user is authenticated (Authorization).
Why It Matters: They make communication between frontend and backend flexible without modifying the actual data in the body.

3. Idempotent vs Non-Idempotent Methods

Simple Explanation: A method is idempotent if calling it once or multiple times produces the same result on the server (e.g., GET, PUT, DELETE). If each call creates a different result, it is non-idempotent (e.g., POST).
Why It Matters: When the network fails and the client retries a request, you need to know whether retrying is safe or if it might create duplicate entries.

4. CORS & Pre-flight Requests (OPTIONS)

Simple Explanation: For security reasons, browsers block direct API calls from one domain to another. For complex requests (such as those with JSON data or custom headers like Authorization), the browser first sends an OPTIONS request (pre-flight) to ask the server for permission.
Why It Matters: Without proper CORS setup, browsers will reject third-party API calls — one of the most common issues in frontend-backend integration.

5. HTTP Status Codes

Simple Explanation: These are three-digit numbers that indicate the result of a request: 2xx (Success), 3xx (Redirection), 4xx (Client errors like 400 Bad Request or 404 Not Found), and 5xx (Server errors).
Why It Matters: Frontend applications can use these codes to show appropriate messages or update the UI without always parsing the response body.

6. Caching (E-Tags & 304 Not Modified)

Simple Explanation: If the data has not changed, the server sends a 304 Not Modified status instead of the full response. The browser then uses its locally cached version.
Why It Matters: This saves bandwidth and can improve application speed by up to 10 times.

Real Job Scenario

Problem Context:

You are working on an AI-driven Document Intelligence platform called LexAI. Your frontend (React/Vite) is running on http://localhost:5173 and your backend API (FastAPI) is on http://localhost:8000. When a user uploads a PDF, the frontend sends a POST request with Authorization: Bearer and Content-Type: application/json, but the request fails. The browser console shows a red "CORS Error", while the backend logs show no incoming request.

Why It’s Difficult:

The frontend engineer thinks the backend is down, while the backend engineer insists the code is correct and no request is reaching the server. Hours are wasted in debugging.

How This Concept Helps:

Understanding CORS and pre-flight requests solves this issue directly. Since the frontend and backend are on different ports (5173 to 8000) and the request includes non-simple headers (Authorization) and content type (application/json), the browser sends an OPTIONS request first to check permissions.

Step-by-Step Solution:

Check Network Tab: Open browser developer tools and look at the Network tab. You will see an OPTIONS request failing before the actual POST request.
Identify the Block: The backend is not properly handling the OPTIONS request or not returning the correct Access-Control-Allow-Origin headers.
Backend Fix: Configure CORS middleware in your backend (FastAPI, Django, etc.).
Configure Origins & Headers: Add http://localhost:5173 to the allowed origins and explicitly allow Authorization and Content-Type in Access-Control-Allow-Headers.
Cache Pre-flight: Set the Access-Control-Max-Age header so the browser caches the pre-flight response (e.g., for 24 hours) and avoids sending OPTIONS on every request.

Final Result:

The pre-flight check passes (returning 204 No Content), the original POST request succeeds, and the blocker between the API and UI teams is resolved.

Practical Implementation Guide

Step 1: Always use the correct HTTP methods. Use POST to create new records, PATCH for partial updates, and PUT only when completely replacing a resource.
Step 2: Standardize status codes. Return 401 Unauthorized for unauthenticated users, 403 Forbidden for permission issues, and 500 for server errors. Avoid sending all errors as 200 OK with custom JSON messages.
Step 3: Enable data compression. For APIs returning large JSON responses, turn on Gzip or Brotli compression at the server level (NGINX or API Gateway). This can reduce a 25MB payload down to 3MB.
Step 4: Handle large file uploads properly. For images, videos, or PDFs, use multipart/form-data instead of JSON to allow streaming and prevent server crashes.

Technical Insights & Tradeoffs

HTTP’s stateless nature is perfect for horizontal scaling and load balancing because no server needs to remember user sessions. The tradeoff is that every request becomes larger as you must send authentication tokens (like JWTs) with each call.

Common Mistake: Many developers use PUT for partial updates. PUT means complete replacement and is idempotent. For updating just one field (like phone number), PATCH should be used.

When NOT to use HTTP Caching: Avoid it for real-time dashboards or AI-generated streaming content (like ChatGPT). In such cases, WebSockets or Server-Sent Events (SSE) are better choices.

Workplace & Career Impact

Deep knowledge of HTTP helps you make better technical decisions in system design and confidently justify your choices during architecture discussions. Status codes act as a universal language that improves collaboration between frontend and backend teams. Mastering these fundamentals moves you from being just a framework developer to a true Software Architect.

Quick Recap

HTTP is a stateless protocol that manages data transfer between client and server.
Every request is self-contained and carries important metadata through headers.
HTTP methods tell the server the intended action, while status codes define the outcome.
Browsers enforce CORS and pre-flight requests for security.
Caching, compression, and proper use of methods and codes are essential for building efficient and scalable backends.

Understanding Check Questions

You are building a profile page backend. The user wants to update only their phone number. Should you use PUT or PATCH for this action, and why?
Your system is moving from a monolith to a microservices architecture with 5 different API servers. How does HTTP’s stateless property help you manage user login sessions?
A mobile app (non-browser client) and a web frontend are both sending the same POST request (with JSON and auth token) to the backend. The web app gets blocked while the mobile app works fine. What is the reason behind this difference in the context of CORS and OPTIONS requests?

Thank you for reading!

I hope this deep dive helped you build a stronger understanding of HTTP and how it powers modern applications. If you found it useful, feel free to share it with other developers.

Now it’s your turn — try answering the Understanding Check Questions in the comments below. I’d love to read your answers and discuss them with you. This is one of the best ways to truly internalize these concepts.

Happy coding! 🚀

Authentication vs. Authorization: A Deep Dive Every Backend Engineer Must Know

Ameer Hamza — Wed, 13 May 2026 07:19:07 +0000

This post is an in-depth breakdown of Authentication (Who are you?) and Authorization (What are you allowed to do?). In the early days, identity was based on simple trust. In modern web applications, we rely on complex and secure systems. As a backend engineer, it is essential to understand the key differences between stateful (Sessions) and stateless (JWTs) authentication, when to use API keys, and exactly how “Sign in with Google” (OAuth 2.0 / OIDC) works behind the scenes. The post also covers practical security risks such as how hackers exploit timing attacks and detailed error messages, along with ways to keep your systems secure.

Key Concepts Breakdown

1. Authentication vs. Authorization

Simple Explanation: Authentication (AuthN) is the process of verifying identity (like showing your ID card). Authorization (AuthZ) is the process of checking permissions (like whether that ID card allows you to enter the server room).
Why It Matters: Mixing up the two leads to insecure systems. Always verify who the user is first, then decide what they can access.

2. Stateful Authentication (Sessions)

Simple Explanation: The server stores the user’s session data (whether they are logged in, their user info, etc.) in a database or cache like Redis. The browser only receives a small Session ID stored in a cookie.
Why It Matters: You retain full control. You can instantly log out any user by revoking their session.

3. Stateless Authentication (JWT)

Simple Explanation: JSON Web Tokens (JWTs) are self-contained ID cards that include user data and a cryptographic signature. The server doesn’t need to query a database. It only verifies the signature.
Why It Matters: When you have thousands of users and multiple microservices, this approach allows your system to scale easily without slowing down due to database lookups.

4. OAuth 2.0 & OpenID Connect (OIDC)

Simple Explanation: OAuth 2.0 is a protocol that lets one application access another application’s data without sharing passwords (delegation). OpenID Connect (OIDC) builds on top of it to provide user identity and authentication (e.g., “Sign in with Google”).
Why It Matters: Users don’t want to create new passwords for every app. This is the industry standard for secure third-party logins and integrations.

5. API Keys

Simple Explanation: These are passwords used for communication between servers and machines. No human or UI is involved. It’s pure machine-to-machine communication.
Why It Matters: When your backend needs to talk to third-party services (like OpenAI, Stripe, etc.), API keys are the standard way to authenticate.

6. Role-Based Access Control (RBAC)

Simple Explanation: Assign each user a role (Admin, Editor, Viewer, etc.) and grant permissions based on that role.
Why It Matters: It keeps your authorization logic clean. Instead of writing if-else checks in every API route, you can handle role validation in middleware.

7. Security Flaws: Error Leaks & Timing Attacks

Simple Explanation: Never give hackers hints. Always return generic error messages (“Invalid credentials”) whether the email is wrong or the password is incorrect. Password verification should also take the same amount of time for every input (constant-time comparison).
Why It Matters: Small details can reveal whether an account exists, allowing attackers to focus on cracking the password.

Real Job Scenario

Problem Context:

You are an SDE at a fast-growing e-commerce startup. The team is migrating from a monolith to microservices (User Service, Cart Service, Order Service, etc.).

Why It’s Difficult:

Previously, the User Service stored sessions in Redis. Now, every time a user interacts with the Cart Service, it has to call the User Service to verify if the user is logged in. This cross-service communication dramatically increased latency.

Solution Using Concepts from This Post:

Shift from Stateful (Sessions) to Stateless (JWT) authentication.

Step-by-Step Implementation:

Login: User sends email/password to the User Service.
JWT Creation: After verification, the User Service generates a JWT (containing user_id, role, etc.), signs it with a secret key, and sends it to the browser.
Subsequent Requests: The browser includes the JWT in the Authorization header when calling the Cart Service.
Local Verification: The Cart Service uses the shared secret (or public key) to verify the token’s signature and extract the user_id and role locally. No need to call the User Service or Redis.

Final Result:

Internal network calls and database queries are drastically reduced. APIs respond extremely fast, and adding new microservices becomes much easier because authentication is now fully decentralized and stateless.

Practical Implementation Guide

Step 1: For simple web apps (e.g., internal admin dashboards), start with Stateful Sessions. They are secure and easy to manage.
Step 2: For mobile apps or microservices architectures, implement JWTs, but always store secret keys securely (.env files, AWS Secrets Manager, etc.).
Step 3: Always return generic error messages during login/signup: return res.status(401).json({ error: "Invalid email or password" });
Step 4: Protect routes using RBAC middleware (e.g., @require_role('admin') in Django or authorizeRole(['admin']) in Express).
Step 5: Never store plain-text passwords. Always hash them using bcrypt or argon2.

Technical Insights & Tradeoffs

JWTs are fast and highly scalable, but they come with a major challenge: Revocation. Since the token is stateless, you cannot instantly log out a user until the token expires. Sessions allow instant revocation by simply deleting the session record.

Use Sessions for B2B SaaS web applications where tight security control and immediate revocation are critical.
Use JWTs for high-traffic mobile backends and microservices architectures.
Never share usernames/passwords with third parties. Use API Keys or OAuth instead.
Common Mistake: Storing JWTs in localStorage (vulnerable to XSS attacks). Prefer HTTPOnly + Secure cookies when possible.

Workplace & Career Impact

Understanding these concepts deeply helps you make confident architectural decisions in system design discussions. Security and access control directly impact revenue and company reputation. Engineers who master these fundamentals progress faster to Senior and Staff-level roles.

You’ll also be able to push back effectively against Product Managers who request user-unfriendly but insecure flows (like revealing whether an email exists), explaining clearly why security tradeoffs matter.

Quick Recap

Authentication verifies identity. Authorization determines permissions.
Sessions (Stateful) keep data on the server. JWTs (Stateless) are self-contained, scalable tokens.
OAuth 2.0 & OIDC enable secure third-party logins without sharing passwords.
API Keys are for machine-to-machine communication.
Always use generic errors and constant-time operations to prevent information leaks and timing attacks.

Understanding Check Questions

Scenario: A user’s account is compromised and you’re using JWT (Stateless Authentication). How do you immediately revoke/block that specific user’s session without logging out everyone else?
Scenario: You’re designing a B2B SaaS dashboard with strict compliance requirements. Admins must be able to revoke any user’s access in milliseconds. Would you choose JWT or Session-based auth? Why?
Scenario: Your frontend developer wants to show “Account not found, please sign up” if an email doesn’t exist in the database. As a backend engineer, how would you explain the security implications from a technical and security perspective?

Thank you for reading!

I hope this deep dive helped you strengthen your understanding of Authentication and Authorization. If you enjoyed the post, feel free to share it with fellow developers.

Now it’s your turn — try answering the Understanding Check Questions in the comments below. I’d love to read your responses and discuss them with you. This is the best way to truly internalize these concepts.

Happy coding! 🚀

DEV Community: Ameer Hamza

Retrieval-Augmented Generation (RAG) for Backend Engineers

Introduction

Why This Matters

Prerequisites

The Problem

Understanding the Core Concept

Chunking strategies

Embedding model choice

Hybrid retrieval

How It Works Internally (High Level)

Step-by-Step Example

Architecture

Python Example

Real-World Applications

Performance Considerations

Common Mistakes

Interview Questions

Production Checklist

Index freshness patterns

When RAG is not enough

Design Tradeoffs: Chunk Size

Extended Walkthrough: Debugging a Wrong Answer

Observability Fields

Eval Metrics Reference

Anti-Patterns in RAG Products

Summary

Further Reading

LLMs Explained for Backend Engineers

Introduction

Why This Matters

Prerequisites

The Problem

Understanding the Core Concept

Tokens, not words

Next-token prediction

Training vs inference

How It Works Internally (High Level)

Step-by-Step Example

Architecture

Python Example

Real-World Applications

Performance Considerations

Common Mistakes

Interview Questions

Summary

Further Reading

Next in Series

HTTP Protocol Deep Dive: Everything Every Backend Engineer Must Know

Key Concepts Breakdown

Real Job Scenario

Practical Implementation Guide

Technical Insights & Tradeoffs

Workplace & Career Impact

Quick Recap

Understanding Check Questions

Authentication vs. Authorization: A Deep Dive Every Backend Engineer Must Know

Key Concepts Breakdown

Real Job Scenario

Practical Implementation Guide

Technical Insights & Tradeoffs

Workplace & Career Impact

Quick Recap

Understanding Check Questions