DEV Community: TechSimPlus Learnings

How to Build a Production RAG System Step by Step (Python, pgvector, Hybrid Search, Reranking)

TechSimPlus Learnings — Mon, 05 Oct 2026 01:15:00 +0000

This tutorial builds the retrieval core of a production RAG system. Not a notebook toy, but the pieces you actually need: tenant-safe vector search, hybrid retrieval, reranking and grounded answers.

Stack: Python, Postgres + pgvector, OpenAI embeddings, rank_bm25, sentence-transformers.

Step 1: Schema with tenant isolation

CREATE EXTENSION IF NOT EXISTS vector;

CREATE TABLE chunks (
    id          BIGSERIAL PRIMARY KEY,
    tenant_id   TEXT NOT NULL,
    doc_title   TEXT,
    section     TEXT,
    content     TEXT NOT NULL,
    embedding   VECTOR(1536)
);

CREATE INDEX ON chunks USING hnsw (embedding vector_cosine_ops);
CREATE INDEX ON chunks (tenant_id);

tenant_id lives in the table from day one. Permissions are enforced in SQL, not in the prompt.

Step 2: Structure-aware chunking

import re

def chunk_by_headings(text: str, doc_title: str, max_chars: int = 1500):
    sections = re.split(r"\n(?=#+ |\d+\.\s)", text)
    chunks = []
    for sec in sections:
        heading = sec.strip().split("\n")[0][:120]
        for i in range(0, len(sec), max_chars):
            chunks.append({
                "doc_title": doc_title,
                "section": heading,
                "content": f"{doc_title} > {heading}\n{sec[i:i + max_chars]}",
            })
    return chunks

Prefixing each chunk with its title and heading gives the embedding context that a raw slice of text does not have.

Step 3: Embed and store

import numpy as np
import psycopg
from pgvector.psycopg import register_vector
from langchain_openai import OpenAIEmbeddings

emb = OpenAIEmbeddings(model="text-embedding-3-small")  # 1536 dims
conn = psycopg.connect("postgresql://localhost/rag")
register_vector(conn)

def ingest(tenant_id, chunks):
    vectors = emb.embed_documents([c["content"] for c in chunks])
    with conn.cursor() as cur:
        for c, v in zip(chunks, vectors):
            cur.execute(
                "INSERT INTO chunks (tenant_id, doc_title, section, content, embedding) "
                "VALUES (%s, %s, %s, %s, %s)",
                (tenant_id, c["doc_title"], c["section"], c["content"], np.array(v)),
            )
    conn.commit()

Step 4: Hybrid retrieval with Reciprocal Rank Fusion

from rank_bm25 import BM25Okapi

def vector_search(tenant_id, query, k=30):
    qv = np.array(emb.embed_query(query))
    rows = conn.execute(
        "SELECT id, content FROM chunks WHERE tenant_id = %s "
        "ORDER BY embedding <=> %s LIMIT %s",
        (tenant_id, qv, k),
    ).fetchall()
    return rows

def keyword_search(tenant_id, query, k=30):
    rows = conn.execute(
        "SELECT id, content FROM chunks WHERE tenant_id = %s", (tenant_id,)
    ).fetchall()
    bm25 = BM25Okapi([r[1].lower().split() for r in rows])
    scores = bm25.get_scores(query.lower().split())
    ranked = sorted(zip(rows, scores), key=lambda x: x[1], reverse=True)
    return [r for r, _ in ranked[:k]]

def rrf(*result_lists, k=60):
    scores, docs = {}, {}
    for results in result_lists:
        for rank, (doc_id, content) in enumerate(results):
            scores[doc_id] = scores.get(doc_id, 0) + 1 / (k + rank + 1)
            docs[doc_id] = content
    return [(i, docs[i]) for i in sorted(scores, key=scores.get, reverse=True)]

In production, use Postgres full-text search or a search engine for the keyword side instead of loading rows into memory. The fusion logic stays the same.

Step 5: Rerank with a cross-encoder

from sentence_transformers import CrossEncoder

reranker = CrossEncoder("cross-encoder/ms-marco-MiniLM-L-6-v2")

def rerank(query, candidates, top_n=5):
    scores = reranker.predict([(query, c[1]) for c in candidates])
    ranked = sorted(zip(candidates, scores), key=lambda x: x[1], reverse=True)
    return [c for c, _ in ranked[:top_n]]

Step 6: Generate a grounded, cited answer

from langchain.chat_models import init_chat_model

llm = init_chat_model("openai:gpt-4.1-mini", temperature=0)

def answer(tenant_id, question):
    fused = rrf(vector_search(tenant_id, question), keyword_search(tenant_id, question))
    top = rerank(question, fused[:40])
    context = "\n\n".join(f"[{i}] {c}" for i, c in top)
    prompt = (
        "Answer ONLY from the sources below. Cite sources like [id]. "
        "If the answer is not in the sources, say you don't know.\n\n"
        f"Sources:\n{context}\n\nQuestion: {question}"
    )
    return llm.invoke(prompt).content

Step 7: Evaluate before you ship

Create 50 to 100 real questions with expected sources. Track faithfulness, answer relevance and context recall using RAGAS or DeepEval, and fail your CI pipeline when scores fall below your baseline.

What's next

The production version adds Docling for PDFs and tables, background ingestion workers, query rewriting, answer caching in Redis and observability with Langfuse.

That full system, RegRadar, is Project 2 in Vector 2.0, the live Gen-AI developer cohort by TechSimPlus taught by Prateek Mishra

https://vector.techsimplus.com

Visit the website and see the full curriculum.

Best Online GenAI Course for Developers and Working Professionals

TechSimPlus Learnings — Sun, 04 Oct 2026 00:30:00 +0000

Developers are skeptical by default, and that is a good thing when choosing a Gen-AI course. Here is a technical checklist to score any online GenAI course before you enrol.

The scoring checklist

Give each item 1 point. A strong course should score 10 or more out of 12.

#	Check	Why it matters
1	Starts with API engineering (FastAPI, Pydantic, async)	Gen-AI ships as APIs
2	Covers structured output, tool calling, fallbacks, cost control	First topics in Gen-AI interviews
3	RAG beyond basics: hybrid search, reranking, multi-tenancy	Where real systems break
4	RAG evaluation in CI (RAGAS, DeepEval)	Separates demos from production
5	LangGraph with checkpointing and human-in-the-loop	Needed for real workflows
6	Agents with MCP and A2A, plus guardrails and red-teaming	Appears in 2026 job descriptions
7	AWS Bedrock or Azure AI hands-on	Enterprises run on these
8	Fine-tuning with LoRA/QLoRA, and when not to fine-tune	Common senior interview question
9	LLMOps: vLLM, gateways, caching, rate limits, load tests	Senior-level differentiator
10	Every project deployed and on your GitHub	Proof beats certificates
11	One-on-one mock interviews with feedback	Converts skills into offers
12	Live schedule that fits a full-time job, with recordings	Completion matters

Red flags

Syllabus is mostly prompt engineering and no-code tools
Projects are "chat with PDF" notebooks with no deployment
No mention of evaluation, observability or security
Instructor has no visible production work
Promises of guaranteed jobs or salaries

How Vector 2.0 scores

Full disclosure: I am writing this for TechSimPlus. Vector 2.0, taught by Sr. Gen-AI Developer Prateek Mishra, was designed against this checklist and covers all 12 items:

Sprint 0: FastAPI foundations, ending with a reusable Launchpad starter kit
Sprints 1 to 4: LangChain, production RAG, LangGraph, agents with MCP and A2A
Sprint 5: fine-tuning, AWS Bedrock and Azure AI
Sprint 6: LLMOps with vLLM, LiteLLM, Redis, Kubernetes and k6 load tests
Sprint 7: capstone, Demo Day and the final senior mock loop

The stack includes FastAPI, LangChain, LangGraph, Pinecone, Qdrant, pgvector, RAGAS, DeepEval, CrewAI, vLLM. Prerequisites are only Python, Docker and Git.

It does not promise a guaranteed job. It does give you 5 deployed projects, 5 mock interviews, resume and LinkedIn reviews, and referrals when openings come through the network.

Next cohort: 21 Nov 2026, early-bird ₹16,800 till 30 Oct 2026.
Visit: http://vector.techsimplus.com

Use the checklist on any course you are considering, and share your score in the comments.

The GenAI Developer Roadmap for 2026: A Step-by-Step Path for Backend Developers

TechSimPlus Learnings — Sat, 03 Oct 2026 12:07:52 +0000

If you already write backend code, you are closer to a GenAI developer role than you think. The job in 2026 is mostly engineering: APIs, data pipelines, reliability and scale, with an LLM in the middle.

This roadmap is ordered by what you need to ship, not by what sounds impressive.

Level 0: The foundation you already half-have

Python (async/await, typing)
FastAPI + Pydantic
Postgres, Redis
Docker and Git

If async def and dependency injection feel shaky, fix that first. Every LLM call is slow I/O, and one blocking call slows every user.

Level 1: Call an LLM like an engineer
The first skill interviewers test is reliable structured output. Not "chat with a PDF", but "return valid data every time".

from pydantic import BaseModel
from langchain.chat_models import init_chat_model

class Invoice(BaseModel):
    vendor: str
    gstin: str | None
    total_amount: float

llm = init_chat_model("openai:gpt-4.1-mini")
extractor = llm.with_structured_output(Invoice)

result = extractor.invoke("Extract the invoice fields: ...")
print(result.total_amount)

Learn: tokens, context windows, temperature, retries, fallbacks, streaming, cost per request.

Level 2: Retrieval-Augmented Generation (RAG)
Learn embeddings, a vector database (pgvector, Qdrant or Pinecone), chunking, hybrid search, reranking and evaluation. The gap between a demo and production RAG is evaluation: build a test set of real questions and measure faithfulness and context recall before you ship.

Level 3: Agents and workflows
LangGraph: state, nodes, conditional edges, checkpointing, human-in-the-loop
MCP: expose tools to any agent through a standard protocol
A2A: let agents from different systems collaborate
Rule of thumb: most production "agents" are 80% predictable workflow and 20% LLM decision-making.

Level 4: Evals, observability, security
Tracing (LangSmith, Langfuse), quality gates in CI (RAGAS, DeepEval), prompt-injection defence, red-teaming with promptfoo.

Level 5: Cloud and fine-tuning
AWS Bedrock or Azure AI, plus LoRA/QLoRA fine-tuning with Hugging Face PEFT. Know the decision: prompt first, RAG for knowledge, fine-tune for behaviour or format.

Level 6: LLMOps at scale
vLLM, LiteLLM gateway, Redis caching and rate limiting, circuit breakers, Kubernetes, k6 load tests.

A 12-week plan if you work full-time

Weeks	Focus	Ship this
1	FastAPI foundations	Dockerised API with auth and tests
2 to 3	LLM basics + structured output	Document extraction API
4 to 5	Production RAG	Q&A over your company docs with citations
6 to 7	LangGraph workflows	Approval workflow with human-in-the-loop
8	Agents, MCP, A2A	Multi-agent assistant with one MCP server
9	Cloud AI + fine-tuning	Same RAG on Bedrock or Azure
10 to 11	LLMOps	Gateway with caching, rate limits, load test
12	Portfolio + interview prep	READMEs, demo videos, mock interviews

Want to do this with a cohort?
This plan mirrors the 8 sprints of Vector 2.0, a live Gen-AI developer cohort by TechSimPlus taught by Prateek Mishra. You build 5 production projects (LedgerLens, RegRadar, ClaimSense, TripPilot, TokenGrid) and do 5 mock interviews.

Next batch starts 21 Nov 2026: https://vector.techsimplus.com

What level are you at? Drop it in the comments and I will suggest your next project.