DEV Community: Shahzaib S

I Built a Support Bot That Knows When to Stop and Call a Human

Shahzaib S — Mon, 05 Oct 2026 06:48:25 +0000

Everyone has been stuck with a support bot that keeps saying "Sorry, I didn't understand that" while you get angrier and angrier.

I wanted to build the opposite: a bot that answers what it can, and quietly steps aside when a real person is needed. It works on web, Discord, and WhatsApp, and the code is open source: github.com/Shahzaib30/ai-support-agent

The Big Picture

The system has four layers. Customers talk to it from any channel, one FastAPI backend handles everything, a few services store state and send alerts, and monitoring watches it all.

Inside the FastAPI core there are three gates in front of the answer engine: a HITL gate (human-in-the-loop, which checks whether a human is already handling the chat), an explicit escalation check, and a sentiment escalation check. PostgreSQL remembers conversations, Redis caches answers, and everything runs with one Docker Compose command.

How the Bot Finds Answers

Most RAG bots search once and hope. Mine runs every question through six stages:

The question arrives.
A query condenser rewrites it. Customers say things like "what about sale items?", which means nothing alone. A DeepSeek step turns follow-ups into standalone questions first.
Two searches run at once. FAISS with BGE-small embeddings finds passages by meaning. BM25 finds passages by exact words. Each returns its top 20.
Reciprocal Rank Fusion merges them into one list of 20 candidates.
A cross-encoder reranks them. It reads the question and each passage together, and only the top 5 move on.
DeepSeek writes the answer using those 5 chunks, the chat history, and long-term memory.

Why two searches? Vector search understands meaning but is weak on exact things like prices, emails, and phone numbers. Ask "What does the Pro plan cost?" and it may return a nice passage about pricing in general instead of the one with the number. BM25 is the opposite. Together they cover each other's weak spots, and for questions about fixed facts, hybrid clearly beat vector search alone.

When Does the Bot Give Up?

There are two triggers, and I built them differently on purpose.

1. The customer is clearly upset (checked by an LLM)

Every message goes to the DeepSeek API with a strict prompt that replies in JSON, a label and a score from -1.0 to 1.0:

Rules:
- negative: customer is CLEARLY angry, frustrated, or complaining
- neutral: casual phrases, short responses, confusion, greetings
- positive: happy, satisfied, thankful

"oh no" → neutral
"wtf" → neutral (not clearly angry enough)
"this is terrible I want a refund NOW" → negative

Reply ONLY with JSON: {"label": ..., "score": ...}

The examples are what make it work. People say "oh no" and "wtf" casually, so those count as neutral. Only clear anger counts as negative.

One angry message doesn't trigger anything. The bot escalates after three negative messages in a row, so a customer who calms down isn't punished for an early bad moment.

2. The customer asks for a human (plain code, no LLM)

def wants_human(text: str) -> bool:
    text = text.lower()
    return any(p in text for p in HUMAN_REQUEST_PHRASES)

# "talk to a human", "real person", "live agent", ...

If the message contains a phrase like "talk to a human" or "real person," the bot escalates immediately. An API call to understand "I want a human" would cost money and time for no gain, so a simple phrase list does the job for free.

That became my main rule: use an LLM when you need judgment, and plain code when the rule is clear.

One Conversation, Three States

Before answering anything, the HITL gate checks what state the conversation is in:

AI Active: normal RAG flow, the bot answers.
Human Pending: an escalation was sent but no human has replied yet. If nobody answers within 5 minutes, the bot takes the chat back so the customer isn't left hanging.
Human Active: a human is handling it, so the bot stays quiet and forwards the customer's messages to the agent. The customer never gets two answers.

When the case is resolved, the customer goes straight back to chatting with the AI.

The Slack Side

Sending the alert was the easy part. When either trigger fires, the bot posts to an #escalations channel in Slack with the customer, the channel, the reason, the sentiment score, and the conversation ID:

A human replies in the thread, and no special format is needed. The customer's new messages also show up in that thread, so the whole conversation lives in one place. To hand the chat back to the bot, the human types resolved.

The Hard Half: Getting the Reply Back to the Customer

Reading a Slack reply and delivering it to someone on Discord was the tricky part. I solved it with an n8n workflow:

Slack Trigger fires on a new message in the thread.
Is real user reply checks it's a person and not the bot's own message.
Extract conversation_id and message pulls out the two things the API needs.
Is resolved branches:
- Yes: an API call resolves the conversation in PostgreSQL.
- No: an API call (/human_reply) stores the reply.

On the Discord side, a small bot checks the API every 5 seconds and DMs any new human reply to the customer. Here's what the customer sees:

![Discord DM: agent replies, case resolved, bot resumes](

The customer's message is confirmed as sent to a support agent, the agent's replies appear in the chat, and once the case is resolved the bot says so and starts answering again.

Watching It Work

A Grafana dashboard has six panels: total messages, total escalations, cache hit rate, average RAG response time, active conversations, and cache hits versus misses.

LangSmith traces each request, so I can open any answer and see exactly what happened inside the pipeline:

In my test run (23 traces), answers took about 1.4 to 2.4 seconds end to end. Those numbers come from my own test conversations, not real customers.

What I Learned

Not everything needs an LLM. Use it for judgment, like reading sentiment. Use plain code for clear rules, like phrase matching.
Tracing saves hours. LangSmith showed me what each request actually did, so I stopped guessing where things broke.
Connecting tools is most of the work. Slack, Discord, n8n, and the database each have their own quirks, and the real effort goes into making them talk to each other.
Use n8n when you need a fast integration. It makes tricky steps easy and keeps the workflow readable for other people. Skip it when you need deep customization, low latency, or custom logic. For those, write code.

Try It or Hire Me

Code: github.com/Shahzaib30/ai-support-agent
Upwork: My Upwork profile
Linkedin: My Linkedin profile

I'm a Full Stack AI Engineer who builds AI agents, RAG systems, and automation. If you're building something similar, send me a message.

What's the worst support bot you've ever dealt with? Tell me in the comments.

Why Your OpenAI Wrapper Is Costing Too Much (And How LangGraph Fixes It)

Shahzaib S — Thu, 28 May 2026 08:36:57 +0000

Many businesses rush into artificial intelligence by building a basic OpenAI wrapper. They connect a simple user interface to an API endpoint, upload a few documents, and call it an enterprise solution.

Initially, the tool looks impressive. However, as user traffic grows, the monthly cloud bill spikes dramatically. Even worse, the chatbot starts repeating itself, hallucinating, or failing to complete multi-step workflows.

If your company experiences soaring token usage and unpredictable chatbot behavior, you have a structural problem. A simple linear wrapper cannot handle complex enterprise operations efficiently.

The Costly Reality of Basic AI Wrappers

Standard OpenAI wrappers rely on a single, continuous prompt chain. Every single time a user asks a question, the entire chat history and every relevant document chunk must be sent back to the language model.

This architecture causes major financial and operational inefficiencies.

Runaway Loop Costs: When a linear chatbot encounters an ambiguous user query, it frequently gets stuck in a loop. It repeatedly queries the LLM for clarification, burning through thousands of tokens in seconds.
Irrelevant Context Loading: Poorly designed Retrieval-Augmented Generation systems pull massive blocks of data from the vector database. Sending unoptimized context to the API forces you to pay premium prices for processing useless background text.
Lack of Native Memory: Without a robust system to track state, wrappers either pass massive text files to preserve memory or forget user details entirely. Both outcomes cost you money and lower client satisfaction.

To achieve reliable business automation without going bankrupt, you must replace linear code with a dynamic, self-correcting state machine.

How LangGraph Optimizes API Budgets

LangGraph redefines agentic workflows by introducing cycles and strict state preservation. Instead of letting an LLM wander freely through a massive prompt, LangGraph breaks your business logic down into specific graph nodes and edges.

An advanced LangGraph AI agent architecture optimizes your API budget through structural intelligence.

Controlled Routing
Your application does not need to use a costly model like GPT-4o for every trivial user interaction. A FastAPI backend powered by LangGraph evaluates incoming traffic immediately. Simple greetings or basic filtering tasks are handled by lightweight, low-cost models or hardcoded scripts. The system routes complex requests to premium models only when absolutely necessary.

Cyclic Self-Correction
If a tool output contains an error or missing data, the agent detects the anomaly before responding to the user. The system passes the incorrect output back to a validation node, allowing the model to correct its own work locally. This prevents the user from receiving broken data and eliminates the need for entirely new chat sessions.

Smart State Persistence
LangGraph utilizes database checkpointers, saving the precise conversational state into a secure database like PostgreSQL. The system loads only the exact data required for the current step, keeping prompt context windows incredibly tight and token costs exceptionally low.

Moving to Production-Grade AI Automation

Deploying a professional AI agent requires moving past basic templates. By migrating to a robust FastAPI backend combined with LangGraph state tracking, you secure full control over your data workflows and your operational expenses. You gain a scalable system that captures leads, protects customer privacy, and executes complex tasks flawlessly.

Stop paying for inefficient API loops that harm your business reputation. Invest in structured, token-conscious intelligence that scales alongside your company.

Need an enterprise-ready AI Agent built with a cost-optimized architecture? Let's design your custom system workflows and state schemas. Click here to launch your advanced LangGraph AI Agent project today.

How to Build a Stateful AI Agent with FastAPI, LangGraph, and PostgreSQL.

Shahzaib S — Tue, 19 May 2026 09:47:38 +0000

Your AI demo worked perfectly in development.

You opened a local notebook, wrote a clean prompt wrapper, and watched the model respond beautifully to your test queries. It felt like magic.

Then production traffic hit.

User sessions started losing memory. API latency exploded under concurrent requests. Long-running inference calls blocked your backend workers, and server restarts wiped active conversations entirely.

This is why most enterprise AI systems fail after deployment. The problem is not the LLM. The problem is the architecture.

In this article, I’ll show how to build a production-ready AI agent backend using FastAPI, LangGraph, and PostgreSQL to guarantee scale, memory, and reliability.

The Core Problem: Why Stateless APIs Break AI Systems

Standard web development relies on stateless APIs. A client sends a request, the server processes it, returns a response, and completely forgets the transaction ever happened.

When you apply this stateless model to AI orchestration, everything breaks. Real humans do not talk to AI in linear paths. They ask a question, change their mind, trigger a tool, provide partial data, and expect the AI to maintain perfect context over hours or days.

If you try to pass an ever-growing array of raw chat logs back and forth over the network on every click, you crush your server performance and waste thousands of dollars in token overhead.

(Note: When I audit failing enterprise AI infrastructure for my clients, this stateless bottleneck is the #1 issue I have to fix.)

To achieve production-grade stability, your AI infrastructure needs a cyclic graph state machine.

The Solution: Stateful AI Architecture with LangGraph

To solve the state preservation problem, we need to abandon linear chains and adopt LangGraph.

Unlike traditional frameworks that force data one way, LangGraph introduces a persistent state graph. This architecture allows us to define specific code execution steps as nodes and use conditional edges to evaluate what the agent should do next — including self-correction loops.

Here is a look under the hood at a standard LangGraph workflow:

LangGraph stateful workflow diagram showing router nodes, conditional edges, retrieval flow, and AI agent orchestration

The Code Implementation

Instead of relying on a single massive prompt, we isolate logic into focused nodes. Here is a simplified snippet of how you compile a stateful graph:

Python code example demonstrating LangGraph state graph compilation for a production-ready AI agent

Scaling with an Async FastAPI AI Backend

Even the best LangGraph agent will fail if your web server blocks threads. If you are using traditional synchronous frameworks (like standard Flask or Django), a single LLM API call taking 5 seconds will freeze your server for all other users.

By wrapping our graph in a FastAPI AI backend, we utilize native asynchronous event loops.

Async FastAPI webhook endpoint handling concurrent AI agent requests using background task processing
This guarantees that when a client’s system experiences a sudden traffic spike of 10,000 concurrent sessions, the server processes the network handshakes effortlessly without dropping webhooks.

Locking Down Persistent Conversational Memory (PostgreSQL)

A stateful agent is only as stable as its underlying memory layer. If your server restarts mid-session, active memory vanishes.

To prevent data loss, the LangGraph backend must be paired with persistent conversational memory. Every node transition, updated state parameter, and user token extraction is routed asynchronously into a PostgreSQL database.

PostgreSQL checkpoint persistence setup for stateful conversational memory in a LangGraph AI backend
If a connection drops, the system instantly looks up the thread_id in PostgreSQL, pulls the chronological chat history, and restores the exact operational state of the agent in milliseconds.

(This specific PostgreSQL checkpointing setup recently allowed me to reduce response latency by over 40% for a multi-session customer support workflow).

Local Deployment vs. Cloud APIs

For enterprise teams with strict data privacy mandates, this architecture is completely decoupled.

You can run this exact LangGraph and FastAPI setup using global cloud APIs (OpenAI GPT-4o, Anthropic Claude), or you can deploy it 100% locally and offline using open-source models via Ollama (Llama 3, Mistral) on private Linux droplets. The architecture stays the same; only the LLM endpoint changes.

Common Production Failures in AI Systems

Most AI prototypes fail in production not because of poor models, but because of weak backend architecture.

Here are the most common scaling failures I encounter when auditing enterprise AI systems:

1. Context Window Explosion
Many AI applications continuously append raw chat history into prompts. Over time, token usage becomes extremely expensive and response latency increases dramatically.

2. Stateless Memory Resets
Without persistent conversational memory, server restarts or failed sessions wipe active user context entirely.

3. Blocking LLM Calls
Synchronous backend frameworks freeze under long-running inference requests, causing webhook failures and severe concurrency bottlenecks.

4. Race Conditions in Multi-User Sessions
When multiple requests hit the same workflow simultaneously, poorly designed agent systems can corrupt memory state or overwrite session variables.

5. Unstructured Tool Orchestration
Linear chains struggle with retries, self-correction loops, and dynamic routing. This creates brittle AI behavior that breaks under real-world user interactions.

6. Token Cost Escalation
Passing massive conversational payloads between the client and backend creates unnecessary token overhead and infrastructure costs.

Production-ready AI systems require stateful orchestration, persistent memory, asynchronous execution, and reliable workflow routing from the beginning.

Final Thoughts:Don’t Build Wrappers, Build Systems
Brittle prompts and basic wrappers do not belong in production software. To deploy enterprise AI, you must treat your agents as robust, self-correcting software systems.

By combining the asynchronous speed of FastAPI, the state-machine orchestration of LangGraph, and the persistent memory of PostgreSQL, you can build AI applications that actually scale.\

FAQ
Why is LangGraph better for production AI systems?
LangGraph supports cyclic workflows, persistent state management, and conditional routing logic. This makes it significantly more reliable for enterprise AI systems than traditional linear chains.

Why use FastAPI for AI backends?
FastAPI provides asynchronous request handling, making it ideal for high-concurrency AI systems that process long-running LLM inference calls and webhook traffic.

Why use PostgreSQL for conversational memory?
PostgreSQL provides durable, scalable, and recoverable state persistence for AI agents. It allows conversations to resume instantly even after crashes or server restarts.

Can this architecture run locally without cloud APIs?
Yes. The exact same architecture can run entirely offline using local LLMs through Ollama with models such as Llama 3 or Mistral.

What types of AI systems benefit from this architecture?
This setup is ideal for:

AI customer support systems
enterprise copilots
AI sales agents
RAG pipelines
workflow automation tools
multi-session conversational AI systems

Is LangGraph better than standard LangChain for agents?
For complex stateful AI agents, LangGraph is generally more suitable because it supports cyclic execution, self-correction loops, and persistent workflow orchestration.

Need help building production-ready AI infrastructure?
If your team is struggling with AI latency, context loss, or scaling issues, I help startups and enterprises deploy scalable LangGraph agent systems. I specialize in:

Persistent conversational memory schemas (PostgreSQL / Supabase)
Async FastAPI backends optimized for high-traffic webhooks
Custom RAG pipelines (ChromaDB / Pinecone)
Local and cloud LLM orchestration (OpenAI, Claude, Ollama)
Let’s build a reliable system:

👉 View LangGraph deployment packages

👉 Hire me for custom AI engineering projects