Series: Building DevDocAI — A Production Multi-Agent LangGraph System
Part 6 — The Full Loop Works. Now: Deployment.
The Milestone
Something clicked this week that hasn't clicked before in this project.
I connected a real GitHub repo through the dashboard, watched the pipeline parse it, generate docs with an LLM, enrich them with external context, pause for my review, got the "Docs approved" screen after clicking approve, and then asked the onboarding chatbot a question about that exact repo — and got a real, grounded answer back.
Not a demo. Not a curl command against an endpoint. A real browser, a real GitHub account, five agents, two databases, and a vector store, all talking to each other correctly.
That's the milestone. Here's everything that had to get fixed to get there — because it wasn't one clean run.
The Debugging Gauntlet
I'm not going to pretend this was smooth. Getting from "the pipeline runs once" to "the pipeline runs reliably, every time, from a cold start" surfaced a string of production-grade bugs, each one small but each one a real lesson.
1. The checkpointer connection kept dying
The first version of the LangGraph PostgreSQL checkpointer opened a single raw connection and held onto it. Neon — being serverless — closes idle connections after a while. The second time I ran the pipeline, everything blew up with the connection is closed.
The fix was switching to a proper connection pool:
_pool = AsyncConnectionPool(
conninfo=psycopg_url,
max_size=10,
kwargs={"autocommit": True},
open=False,
)
await _pool.open()
_checkpointer = AsyncPostgresSaver(_pool)
A pool self-heals. A single connection doesn't. That's the whole lesson, but it cost a few hours to land on.
2. aupdate_state, not update_state
The HITL resume endpoint calls doc_graph.update_state(...) to inject the human's decision back into a paused graph. Except with an async checkpointer, that call has to be awaited and it has to be the async variant:
# silently broken
doc_graph.update_state(config, {...}, as_node="human_review")
# correct
await doc_graph.aupdate_state(config, {...}, as_node="human_review")
The sync version doesn't error loudly with an async backend — it just doesn't behave. Easy to miss, annoying to trace.
3. Cohere → HuggingFace, mid-project
Embeddings were originally wired to Cohere. When the API key turned out to be invalid and I didn't want another external dependency with rate limits, I swapped to local sentence-transformers embeddings — completely free, runs on-device, no network call:
embeddings = HuggingFaceEmbeddings(
model_name="sentence-transformers/all-MiniLM-L6-v2"
)
The catch: Cohere's model outputs 1024-dimension vectors, MiniLM outputs 384. Qdrant collections are locked to a vector size at creation time. Every "vector dimension error: expected 1024, got 384" was the collection remembering the old model. The collection had to be deleted and recreated once the embedding model changed — not something obvious until you hit it.
4. Qdrant Cloud wants an explicit index
Filtering search results by repo_id worked fine on paper, then Qdrant Cloud rejected it:
Index required but not found for "repo_id" of type: [keyword]
Local Qdrant setups can be more lenient; the managed cloud version requires you to explicitly create a payload index before you can filter on a field:
await client.create_payload_index(
collection_name=settings.QDRANT_COLLECTION_NAME,
field_name="repo_id",
field_schema=PayloadSchemaType.KEYWORD,
)
Now baked into the same startup routine that creates the collection, so it never has to be a manual step again.
5. Brave Search killed its free tier mid-build
Genuinely — while building this, Brave Search API dropped its card-free 2,000 queries/month plan in favor of a credit-card-required $5-monthly-credit model. Rather than add a billing dependency to a side project, I swapped the research agent over to Tavily, which still has a genuinely free, card-free tier and is built specifically for agent/RAG use cases. Same node, same interface, different provider underneath — this is exactly why the agent is isolated behind one function boundary in the graph.
Why None of This Broke the Architecture
Every one of these was a plumbing fix — a connection strategy, an await keyword, a vector dimension, an index, a provider swap. Not one of them touched the graph structure, the agent responsibilities, or the API contracts the frontend depends on.
That's not an accident. It's the payoff of:
- One node per responsibility — swapping Brave for Tavily meant editing one file, not touching the graph definition
- A repository layer — the DB never leaked SQL into the pipeline logic
-
A single embeddings module — changing providers meant editing one file,
vectorstore/embeddings.py, and nothing that calls it
The infrastructure decisions from Part 1 kept paying rent, four months later.
Also Fixed This Round
-
RAG counting questions — the chatbot was confidently answering "how many modules are in this repo?" using only its top-4 retrieved chunks, not the whole repo. Fixed with a stricter system prompt that forces the model to say "here's what I can confirm from what I retrieved" instead of presenting a partial view as a total, plus raising
top_kfrom 4 to 15 for broader coverage. -
Markdown rendering in chat — responses were coming back with literal
**bold**and table pipes instead of rendered formatting. Addedreact-markdown+remark-gfmto the chat bubble. - A polished HITL review UI — proper loading/error/empty/done states, a full-screen pipeline loader with staged progress instead of a blank screen during the multi-minute run, hover states and skeleton loaders across the dashboard.
Where Things Stand
| Piece | Status |
|---|---|
| Backend (13 endpoints) | ✅ Complete |
| Multi-agent pipeline, end-to-end | ✅ Confirmed working live |
| GitHub OAuth | ✅ Working |
| Dashboard, review, chat — wired to real data | ✅ Complete |
| RAG chatbot honesty/accuracy | ✅ Improved |
| Phase 7 — Docker, ECR, ECS Fargate, CI/CD | 🔨 Starting now |
This is genuinely the last major checkpoint before deployment. Everything from here is Dockerfiles, a docker-compose setup for local full-stack testing, GitHub Actions for build-and-push, and getting this running on AWS instead of my laptop.
What's Next — Part 7
Phase 7. Containerizing the backend and frontend, pushing images to ECR, standing up ECS Fargate services, and wiring a CI/CD pipeline so a merge to main ships to production without me touching a terminal.
This is also where the "dev environment held together with local Docker and Neon" setup meets real infrastructure decisions — VPCs, security groups, environment secrets, the works.
GitHub
Nevin100
/
DevdocxAI
DevDocxAI is a production-grade multi-agent AI system that automatically generates, maintains, and updates engineering documentation by deeply understanding you…DevDocAI is a production-grade multi-agent AI system that automatically generates, maintains, and updates doc. by understanding deeply.
DevDocxAI 🤖📄
DevDocxAi is a production-grade multi-agent LangGraph system that automatically generates and updates engineering documentation from your GitHub codebase.
🚨 The Problem:
Every engineering team has the same dirty secret — the docs are lying.
Not intentionally. Code moves fast, documentation doesn't.
- New dev joins → 2 weeks reading outdated wikis
- Senior engineers constantly interrupted with "what does this do?"
- PR gets merged → docs never updated
- Generic RAG chatbots don't understand code structure
DevDocAI fixes this.
✨ What It Does
- 🔍 Connects to your GitHub repo via OAuth
- 🌳 Parses your codebase at the AST level — understands functions, classes, modules
- 📝 Auto-generates structured documentation per module and function
- 🔄 Updates docs on every PR merge via GitHub webhooks
- 👀 Human-in-the-Loop review — you approve before anything goes live
- 💬 Onboarding chatbot — new devs ask questions, get answers from live code
🤖 Agent Pipeline
START
↓
codebase_parserBuilding in public — from "it ran once" to "it's about to run in production."
Tags: python langgraph fastapi qdrant rag docker aws opensource
Top comments (0)