Sammishra Kh

Posted on Oct 23 Originally published at sammishra.hashnode.dev

Build a Free AI Support Chatbot with FastAPI, Groq, and Vercel (Complete Guide)

#python #fastapi #ai #tutorial

If you've wanted to add an AI chatbot to a website but assumed it needs a paid OpenAI account, a vector database cluster, and a weekend of DevOps — this guide is for you. We'll build a customer support chatbot that answers questions only from your own documents, refuses to make things up, and deploys to a public URL. Total cost: ₹0 / $0.

Here's the finished thing, live: try the demo — ask it about shipping, then ask it for a pizza menu and watch it politely decline to hallucinate.

What we'll use:

FastAPI — the backend (one file)
RAG (Retrieval-Augmented Generation) — a lightweight version with no vector database
Groq — free-tier LLM API, extremely fast
Vercel — free serverless hosting, deployed from your terminal

Time to complete: about 30 minutes.

Why "answers only from your documents" is the whole product

A raw LLM will cheerfully invent your refund policy. That's not a chatbot a business can use. The fix is RAG: before answering, we retrieve the most relevant sections of the company's real documentation and tell the model: answer from these excerpts only — if it's not here, say so.

That one constraint is the difference between a toy and a product.

Prerequisites

Python 3.10+
Node.js (only for the Vercel CLI at deploy time)
A free account at console.groq.com

Step 1 — Project structure

ai-support-chatbot/
├── api/
│   ├── index.py      # FastAPI app: retrieval + LLM call
│   └── kb.py         # your knowledge base, as a Python string
├── public/
│   └── index.html    # the chat widget
└── requirements.txt

requirements.txt is two lines:

fastapi>=0.110
openai>=1.30

(We use the openai client library, but pointed at Groq — it speaks the same API.)

Step 2 — The knowledge base

Create api/kb.py. This is your company's help docs as one markdown string, where every ## Heading becomes a retrievable section:

"""Knowledge base content, bundled with the serverless function."""

FAQ_MD = '''
# Verdant Coffee Co. — Help Center

## Shipping times and costs
Orders within India arrive in 2–4 business days and shipping is free over
₹999. International orders arrive in 7–14 business days; rates are
calculated at checkout. All orders ship with tracking.

## International shipping
Yes — we ship to over 40 countries, including the US, UK, EU, UAE,
Singapore, and Australia. Customs duties, where applicable, are the
recipient's responsibility.

## Returns and refunds
Coffee is perishable, so we can't accept returns of opened bags. If your
order arrived damaged or incorrect, email us a photo within 14 days and
we'll replace it or refund you in full.

## Contacting human support
Our human support team is available Monday–Saturday, 9:00–18:00 IST at
support@verdantcoffee.example.
'''

I'm using a fictional coffee company — swap in your own docs. More sections = smarter bot.

Step 3 — Retrieval without a vector database

For a knowledge base this size, you don't need embeddings or a vector store. Keyword overlap with one clever trick does the job. Create api/index.py and start with the retrieval engine:

import os
import re
from collections import defaultdict

from fastapi import FastAPI, HTTPException, Request
from openai import OpenAI
from pydantic import BaseModel

try:  # Vercel may load this as a top-level module or as a package
    from .kb import FAQ_MD
except ImportError:
    from kb import FAQ_MD

LLM_API_KEY = os.getenv("LLM_API_KEY", "")
LLM_BASE_URL = os.getenv("LLM_BASE_URL") or None
LLM_MODEL = os.getenv("LLM_MODEL", "openai/gpt-oss-20b")
MAX_MESSAGES_PER_VISITOR = int(os.getenv("MAX_MESSAGES_PER_VISITOR", "10"))
BOT_NAME = os.getenv("BOT_NAME", "Support Assistant")

SYSTEM_PROMPT = """You are {bot_name}, a friendly customer support assistant.
Answer the user's question using ONLY the knowledge-base excerpts provided.
If the answer is not in the excerpts, say you don't have that information
and suggest contacting human support. Keep answers short, warm, and concrete.

Knowledge-base excerpts:
{context}
"""

_STOPWORDS = set(
    "a an the is are am do does did you your yours i we our of to in on for "
    "and or but how what when where which who why my it its with can could "
    "would should this that these those be been have has had will there".split()
)


def _tokens(text: str) -> set[str]:
    return {w for w in re.findall(r"[a-z']+", text.lower()) if w not in _STOPWORDS}


def _overlap(query_tokens: set[str], chunk_tokens: set[str]) -> int:
    """Count query tokens present in the chunk, with prefix matching so
    'ship' matches 'shipping' and 'internationally' matches 'international'."""
    count = 0
    for q in query_tokens:
        for c in chunk_tokens:
            if q == c or (len(q) >= 4 and c.startswith(q)) or (len(c) >= 4 and q.startswith(c)):
                count += 1
                break
    return count


def _build_chunks() -> list[dict]:
    chunks, title, lines = [], "General", []

    def flush():
        body = "\n".join(lines).strip()
        if body:
            chunks.append(
                {"title": title, "text": body, "tokens": _tokens(body), "title_tokens": _tokens(title)}
            )

    for line in FAQ_MD.splitlines():
        if line.startswith("## "):
            flush()
            title, lines = line[3:].strip(), []
        else:
            lines.append(line)
    flush()
    return chunks


_CHUNKS = _build_chunks()


def retrieve(query: str, k: int = 3) -> list[dict]:
    q = _tokens(query)
    if not q:
        return _CHUNKS[:k]
    scored = []
    for c in _CHUNKS:
        score = _overlap(q, c["tokens"]) + 2 * _overlap(q, c["title_tokens"])
        scored.append((score, c))
    scored.sort(key=lambda s: s[0], reverse=True)
    top = [c for score, c in scored[:k] if score > 0]
    return top or _CHUNKS[:1]

The clever trick — prefix matching. My first version used exact token matching, and when I tested "do you ship internationally?", the wrong section won. Why? "internationally" ≠ "international" and "ship" ≠ "shipping" as exact strings. The _overlap function fixes this: tokens match if one is a prefix of the other (minimum 4 characters). Simple, no stemming library, works.

Title matches get double weight — if your question's words appear in a section heading, that section is almost certainly the right one.

Step 4 — The chat endpoint

Append to api/index.py:

app = FastAPI(title="AI Customer Support Chatbot")
_message_counts: dict[str, int] = defaultdict(int)

_llm_client: OpenAI | None = None


def _llm() -> OpenAI:
    """Create the LLM client on first use, with a clear error if unconfigured."""
    global _llm_client
    if _llm_client is None:
        if not LLM_API_KEY:
            raise HTTPException(
                status_code=500,
                detail="Server not configured: the LLM_API_KEY environment variable is missing.",
            )
        _llm_client = OpenAI(api_key=LLM_API_KEY, base_url=LLM_BASE_URL)
    return _llm_client


class ChatRequest(BaseModel):
    message: str
    history: list[dict] = []


@app.get("/api/health")
def health() -> dict:
    return {"status": "ok", "model": LLM_MODEL, "chunks": len(_CHUNKS)}


@app.post("/api/chat")
def chat(req: ChatRequest, request: Request) -> dict:
    visitor = (request.headers.get("x-forwarded-for") or "unknown").split(",")[0].strip()
    if _message_counts[visitor] >= MAX_MESSAGES_PER_VISITOR:
        raise HTTPException(status_code=429, detail="Demo message limit reached. Thanks for trying it out!")
    _message_counts[visitor] += 1

    question = req.message.strip()
    if not question:
        raise HTTPException(status_code=400, detail="Empty message.")
    if len(question) > 1000:
        raise HTTPException(status_code=400, detail="Message too long.")

    chunks = retrieve(question, k=3)
    context = "\n\n---\n\n".join(f"[{c['title']}]\n{c['text']}" for c in chunks)

    messages = [
        {"role": "system", "content": SYSTEM_PROMPT.format(bot_name=BOT_NAME, context=context)},
        *req.history[-6:],
        {"role": "user", "content": question},
    ]

    try:
        completion = _llm().chat.completions.create(
            model=LLM_MODEL, messages=messages, max_tokens=400, temperature=0.3
        )
    except Exception as exc:
        raise HTTPException(status_code=502, detail=f"LLM error: {exc}") from exc

    return {
        "answer": completion.choices[0].message.content,
        "sources": [{"title": c["title"], "source": "faq"} for c in chunks],
    }

Three production touches worth noticing:

Per-visitor message cap — a public demo calls your API key on strangers' behalf. Cap it.
Rolling history window (req.history[-6:]) — context stays short, token costs stay tiny.
Lazy LLM client — if the API key is missing, you get a clear error message instead of a mysterious crash at startup. (I learned this one the hard way.)

Step 5 — The chat widget

Create public/index.html. The full file is ~190 lines of plain HTML/CSS/JS — no framework, no build step. The important part is the fetch call:

const res = await fetch("/api/chat", {
  method: "POST",
  headers: { "Content-Type": "application/json" },
  body: JSON.stringify({ message: question, history: history.slice(0, -1) }),
});
const data = await res.json();

Everything else is styling: a message log, a typing indicator, suggestion chips, and a sources line under each bot reply. Style it however you like — or grab my complete widget from the live demo (right-click → View Source).

Step 6 — Get a free LLM key

Sign up at console.groq.com
API Keys → Create API Key → copy it somewhere safe

Groq's free tier runs open models at absurd speed. We'll use openai/gpt-oss-20b — around 1,000 tokens/second, which makes the bot feel instant.

Step 7 — Run it locally

python -m venv .venv
source .venv/bin/activate        # Windows: .venv\Scripts\activate
pip install -r requirements.txt fastapi uvicorn

set LLM_API_KEY=your-key-here            # Windows
export LLM_API_KEY=your-key-here         # Mac/Linux
set LLM_BASE_URL=https://api.groq.com/openai/v1
set LLM_MODEL=openai/gpt-oss-20b

uvicorn api.index:app --reload

Open http://localhost:8000 — wait, one catch: locally, FastAPI doesn't serve public/ automatically. Either open public/index.html directly and point the fetch at http://localhost:8000/api/chat, or add two lines to serve it. On Vercel (next step) this is handled for you — files in public/ are served at the site root automatically.

Test with a question your docs cover, then one they don't. The refusal is the feature.

Step 8 — Deploy to a public URL (free)

npm i -g vercel
vercel login
vercel

Accept the defaults — Vercel auto-detects FastAPI. Then add your environment variables in the Vercel dashboard (your project → Settings → Environment Variables):

Name	Value
`LLM_API_KEY`	your Groq key
`LLM_BASE_URL`	`https://api.groq.com/openai/v1`
`LLM_MODEL`	`openai/gpt-oss-20b`
`MAX_MESSAGES_PER_VISITOR`	`10`

Finally:

vercel --prod

That's it. Your chatbot is live at https://your-project.vercel.app — a real URL you can put on a portfolio, send to a client, or open on your phone.

Common issues (I hit most of these myself)

model_not_found error. LLM providers retire model names. Open your provider's model list (Groq: the Playground dropdown) and use an exact current ID.

Bot replies "Not Found". If you added a vercel.json with rewrites from an older tutorial — delete it. Vercel's FastAPI auto-detection routes everything through your app already; a rewrite renames the path before your routes see it.

"LLM_API_KEY is missing" after deploy. Environment variables only apply to deployments made after you add them. Add the variables, then deploy once more.

429 "Demo message limit reached." Working as intended — that's your cost cap. Raise MAX_MESSAGES_PER_VISITOR if you want longer conversations.

Wrong section retrieved. Check your ## headings — they carry double weight. A section titled "International shipping" gets found by "do you ship internationally?"; one titled "FAQ item 7" doesn't.

Where to take it next

This is deliberately the minimal production-shaped version. Natural upgrades:

Vector search — swap keyword retrieval for ChromaDB embeddings when your docs outgrow a few dozen sections
Multiple documents — index a folder of markdown files instead of one string
Streaming responses — token-by-token replies for the ChatGPT feel
Analytics — log questions people ask; it's free product research

Want the code without the copy-pasting?

Download this tutorial's complete project — free
AI Chatbot Starter Kit ($29) — the production version: ChromaDB vector search, Docker deployment, one-command setup, full guides, and a commercial license

I'm Sammishra — an AI engineer and published researcher. I build chatbots, RAG pipelines, and AI automations for businesses. See my work and get in touch at *sammishra.hashnode.dev/work-with-me** — or just talk to the demo this guide builds.*

DEV Community