Ship a Production RAG Chatbot in a Weekend with Claude, pgvector, and FastAPI

You can stand up a genuinely useful retrieval-augmented chatbot in a weekend with three moving parts: Postgres (plus the pgvector extension) as your vector store, a FastAPI service as the glue, and Claude for the generation step. The one thing that trips people up on day one is that Claude has no embeddings endpoint — you bring your own embedding model, and everything else is standard web plumbing. What follows is the shape of that build, the code that matters, and the honest limits of doing it this cheaply.

What is each piece actually doing?

RAG is less mysterious than the acronym suggests. At query time you embed the user's question into a vector, find the most similar chunks of your own documents, paste those chunks into a prompt, and ask Claude to answer using only that context. Three responsibilities map cleanly onto three tools:

• pgvector stores chunk embeddings and answers "which chunks are closest to this question?" with a single SQL ORDER BY. If you already run Postgres, you don't need a separate vector database to start.
• FastAPI exposes an HTTP endpoint, validates input with Pydantic, and orchestrates the retrieve-then-generate flow.
• Claude does the final reasoning over the retrieved context.

The takeaway: nothing here is exotic — RAG is a database lookup wearing a trench coat, and Postgres is a perfectly good place to start.

Wait — does Claude have an embeddings API?

No, and this is the sharpest edge of the whole build. The Anthropic API generates text; it does not turn text into vectors. For embeddings, Anthropic points you to Voyage AI, and that's what I use below. Your other options are OpenAI's embedding models or a local model like sentence-transformers if you want zero external embedding calls. Whatever you pick, the embedding model and its output dimension are a hard dependency of your schema — your pgvector column width has to match, and you cannot mix vectors from two different models in the same table.

I'll use Voyage's voyage-3 (1024 dimensions) here because it pairs naturally with Claude and the API is trivial:

import voyageai

vo = voyageai.Client()  # reads VOYAGE_API_KEY

def embed(texts: list[str], input_type: str) -> list[list[float]]:
    # input_type is "document" when indexing, "query" when searching —
    # asymmetric embeddings improve retrieval quality noticeably.
    result = vo.embed(texts, model="voyage-3", input_type=input_type)
    return result.embeddings

The takeaway: budget for a second API key — the embedding provider is a first-class part of the stack, not an afterthought.

How do I set up pgvector?

Enable the extension and create a table whose vector column width matches your embedding model. The HNSW index makes nearest-neighbor search fast enough for interactive use.

CREATE EXTENSION IF NOT EXISTS vector;

CREATE TABLE chunks (
    id        bigserial PRIMARY KEY,
    doc_id    text NOT NULL,
    content   text NOT NULL,
    embedding vector(1024) NOT NULL
);

-- Cosine distance index; matches the <=> operator used at query time.
CREATE INDEX ON chunks USING hnsw (embedding vector_cosine_ops);

Ingestion is: split each document into chunks, embed them in a batch, and insert. A naive fixed-size chunker is fine for a first version — reach for something smarter only once retrieval quality actually disappoints you.

import psycopg
from pgvector.psycopg import register_vector

def chunk(text: str, size: int = 800, overlap: int = 100) -> list[str]:
    words = text.split()
    step = size - overlap
    return [" ".join(words[i:i + size]) for i in range(0, len(words), step)]

def ingest(dsn: str, doc_id: str, text: str) -> None:
    pieces = chunk(text)
    vectors = embed(pieces, input_type="document")
    with psycopg.connect(dsn) as conn:
        register_vector(conn)
        with conn.cursor() as cur:
            cur.executemany(
                "INSERT INTO chunks (doc_id, content, embedding) VALUES (%s, %s, %s)",
                [(doc_id, p, v) for p, v in zip(pieces, vectors)],
            )
        conn.commit()

The takeaway: if your data already lives in Postgres, adding pgvector is one extension and one index — not a new piece of infrastructure to operate.

How do I retrieve context and generate an answer?

Retrieval is one query. The <=> operator is cosine distance (smaller is more similar), so ORDER BY embedding <=> query LIMIT k gives you the top-k chunks. Then you build a grounded prompt and hand it to Claude.

import anthropic

client = anthropic.Anthropic()  # reads ANTHROPIC_API_KEY

SYSTEM = (
    "You answer strictly from the provided context. "
    "If the context does not contain the answer, say you don't know. "
    "Do not use outside knowledge."
)

def retrieve(dsn: str, question: str, k: int = 5) -> list[str]:
    [qvec] = embed([question], input_type="query")
    with psycopg.connect(dsn) as conn:
        register_vector(conn)
        with conn.cursor() as cur:
            cur.execute(
                "SELECT content FROM chunks ORDER BY embedding <=> %s LIMIT %s",
                (qvec, k),
            )
            return [row[0] for row in cur.fetchall()]

def answer(dsn: str, question: str) -> str:
    context = "\n\n---\n\n".join(retrieve(dsn, question))
    prompt = f"Context:\n{context}\n\nQuestion: {question}"
    resp = client.messages.create(
        model="claude-haiku-4-5",  # cheap and fast for grounded QA
        max_tokens=1024,
        system=SYSTEM,
        messages=[{"role": "user", "content": prompt}],
    )
    # content is a list of blocks; grab the first text block defensively.
    return next(b.text for b in resp.content if b.type == "text")

The system prompt doing the "answer only from context, otherwise say you don't know" work is what separates a RAG bot from a confident hallucinator. It won't eliminate hallucination, but it moves the needle hard.

The takeaway: retrieval is a single SQL statement, and grounding discipline lives in the system prompt — not in clever model settings.

Which model and embedding size should I pick?

You're trading cost against reasoning quality at two independent knobs. Here's how I'd choose for a first deployment:

Decision	Cheap / weekend default	When to upgrade
Generation model	claude-haiku-4-5	Move to claude-sonnet-5 when answers need multi-step reasoning over the retrieved chunks; claude-opus-4-8 for the hardest synthesis
Embedding model	voyage-3 (1024-dim)	Larger/most-accurate Voyage tier, or a local model, if retrieval quality is your bottleneck
Vector store	pgvector on your existing Postgres	A dedicated vector DB only once you outgrow single-node Postgres
Chunk size	~800 words, 100 overlap	Tune after you can measure retrieval hits, not before

Note that model pricing and the exact Voyage model lineup shift over time — check current rates before you commit a budget, as of late 2026.

The takeaway: start with Haiku plus a mid-tier embedding model; upgrade one knob at a time based on where quality actually breaks.

Wrapping it in FastAPI

The web layer is thin on purpose. Validate the input, call answer, return JSON.

from fastapi import FastAPI
from pydantic import BaseModel

app = FastAPI()
DSN = "postgresql://localhost/rag"

class Query(BaseModel):
    question: str

@app.post("/chat")
def chat(q: Query) -> dict[str, str]:
    return {"answer": answer(DSN, q.question)}

Run it with uvicorn main:app --reload and you have a working endpoint. Opening a fresh connection per request keeps the example readable, but it's also the first thing you'd replace — see the limitations below.

The takeaway: the API surface should stay boring; all the interesting decisions live in retrieval and prompting.

What are the honest limits of the weekend version?

This gets you a demo you can defend, not a system you can walk away from. Real gaps:

• Connection handling. psycopg.connect per request will fall over under load. Add a pool (psycopg_pool) before you show anyone.
• No evaluation. You have no way to know if retrieval is returning garbage. Even ten hand-written question/expected-answer pairs beat vibes.
• Naive chunking loses structure. Splitting on word count cuts through tables and code blocks. Fine to start, worth revisiting.
• No streaming or auth. The endpoint returns the whole answer at once and is wide open. Both are straightforward additions, neither is done here.
• Grounding is best-effort. The system prompt reduces hallucination; it does not guarantee the model ignores its own priors.

The takeaway: the weekend build is a real foundation, but "production" in the title means the architecture — the hardening is a second weekend.

Bottom line

If you already run Postgres, use pgvector and skip the separate vector database until you have a reason not to. Pair Claude Haiku 4.5 with a Voyage embedding model for the cheapest workable combination, and remember that the embedding step is a distinct provider — Claude won't do it for you. Keep FastAPI thin and put your energy into retrieval quality and the grounding prompt. Ship the weekend version, then spend the second weekend on pooling, evaluation, and auth before you call it production for real.

Related reading

• Cursor vs GitHub Copilot vs Windsurf: Which AI Editor Fits How You Actually Work?
• Postman vs Bruno vs Hoppscotch: Does Your API Client Really Need a Cloud Account?
• Vercel vs Netlify vs Cloudflare Pages: Where Your Side Project Should Actually Live