How I Automated 70% of Customer Support with an AI Chatbot (Django + OpenAI)

#ai #django #python #chatbot

I'm Kirill Strelnikov, a freelance Python/Django developer based in Barcelona, Spain. I build AI-powered products for businesses across Europe. This is a real case study from a client project.

The Problem

An e-commerce clothing store was drowning in repetitive support tickets: "Where is my order?", "What's your return policy?", "Do you have size X in stock?" — the same 15-20 questions making up 80% of all inquiries. Two full-time support agents were spending most of their day copy-pasting answers.

The client asked me to build an AI chatbot that could handle these repetitive queries automatically, while escalating complex issues to human agents.

The Solution: RAG Chatbot with Django + OpenAI

I built a Retrieval-Augmented Generation (RAG) chatbot — not a simple FAQ bot, but one that understands context and generates natural answers from the store's actual data.

Architecture

User message
  → Django backend (receives via REST API)
  → Vector search (pgvector) finds relevant docs
  → OpenAI GPT-4 generates answer using retrieved context
  → Response sent back to user
  → If confidence < threshold → escalate to human

Key technical decisions:

pgvector for embeddings — stored product catalog, FAQ, and policy documents as vector embeddings directly in PostgreSQL. No separate vector DB needed.
Chunking strategy — split documents into 300-token chunks with 50-token overlap. This gave the best retrieval accuracy for product-related queries.
Confidence scoring — each response gets a confidence score. Below 0.7 → automatic escalation to human agent with full conversation context.
Django admin integration — the client's team can update FAQ entries, add product info, and review chatbot conversations through Django admin. No developer needed for content updates.

The code pattern (simplified)

from openai import OpenAI
from pgvector.django import CosineDistance

def get_chatbot_response(user_message: str) -> dict:
    client = OpenAI()

    # 1. Create embedding for user's question
    embedding = client.embeddings.create(
        model="text-embedding-3-small",
        input=user_message
    ).data[0].embedding

    # 2. Find relevant documents via vector search
    relevant_docs = (
        Document.objects
        .annotate(distance=CosineDistance("embedding", embedding))
        .order_by("distance")[:5]
    )

    context = "\n".join(doc.content for doc in relevant_docs)

    # 3. Generate response with context
    response = client.chat.completions.create(
        model="gpt-4",
        messages=[
            {"role": "system", "content": f"Answer using this context:\n{context}"},
            {"role": "user", "content": user_message}
        ],
        temperature=0.3
    )

    return {
        "answer": response.choices[0].message.content,
        "confidence": relevant_docs[0].distance if relevant_docs else 0,
        "sources": [doc.title for doc in relevant_docs]
    }

Results After 3 Months

Metric	Before	After
Automated inquiries	0%	70%
Avg response time	4 hours	8 seconds
Conversion rate	baseline	+35%
Support staff needed	2 full-time	1 part-time

The 35% conversion increase was unexpected — turns out, instant answers to "is this in stock?" and "what size should I get?" directly drive purchases. Customers who got instant chatbot answers were far more likely to complete checkout.

What I Learned

RAG beats fine-tuning for business chatbots. The client's product catalog changes weekly. With RAG, you just re-embed the new data. No retraining needed.
Confidence-based escalation is critical. The chatbot knows when it doesn't know. This prevents hallucination and maintains customer trust.
Django admin is your secret weapon. Non-technical staff can manage the knowledge base. This makes the chatbot self-sustaining.
Start with the top 20 questions. Don't try to automate everything. The Pareto principle applies: 20% of question types cover 80% of volume.

Cost Breakdown

Development: ~EUR 2,500 (3 weeks)
OpenAI API: ~EUR 50-80/month at their volume
Hosting (Django + PostgreSQL): ~EUR 30/month
ROI: Paid for itself in month 1 (saved 1.5 FTE support salary)

Tech Stack

Python 3.12, Django 5.0, Django REST Framework
PostgreSQL with pgvector extension
OpenAI GPT-4 + text-embedding-3-small
Celery for async embedding updates
Docker for deployment

I'm Kirill Strelnikov — I build AI chatbots, SaaS platforms, and Telegram bots as a freelance developer in Barcelona. If you have a similar project, feel free to reach out: