Building DevDocAI — A Production Multi-Agent LangGraph System | Part 7 — From Laptop to Production: The Deployment Gauntlet

Series: Building DevDocAI — A Production Multi-Agent LangGraph System

Part 7 — From Laptop to Production: The Deployment Gauntlet

---

The Milestone

Part 6 ended with "everything works on my laptop." This part is about what it took to make it work on the internet.

DevDocAI is now live — real domain, real GitHub OAuth, real users connecting real repos. A 20+ file repository goes from "Trigger Pipeline" to approved docs in under a minute. The onboarding chatbot answers from live code. The whole thing runs on AWS, deploys itself on every merge to main, and survived everything I'm about to describe.

Because "it works locally" turned out to be roughly 40% of the job. The other 60% was a gauntlet of deployment bugs, rate limits, stale Docker images, and security hardening — each one small, each one a real lesson.

---

The Debugging Gauntlet

1. The Docker image that never actually deployed

This one hurt the most because it was invisible. CI/CD was green. GitHub Actions built the image, pushed :latest to ECR, ran force-new-deployment on the ECS service. Everything looked deployed. But production behavior never changed — none of my performance fixes, none of the new code, was running.

The cause: the ECS task definition (revision 5) pinned the image by SHA digest:

156397422443.dkr.ecr.ap-south-1.amazonaws.com/devdocai-backend@sha256:97f3bf94...

force-new-deployment restarts tasks — it does not re-resolve :latest. It faithfully restarted the old digest over and over. I was shipping code into the void.

The fix was a new task definition revision pointing at :latest instead of a pinned digest, and manually selecting that revision for the deployment. The lesson: a green pipeline is not a deployed pipeline. After every deploy, verify the running revision, not the build status. I now check CloudWatch startup logs (🔑 Groq key pool: 6 key(s) loaded) as my deploy confirmation — if I don't see the new log line, it didn't deploy.

2. The rename that broke production on first boot

The moment the real new image finally ran, ECS killed it three times and rolled back. CloudWatch showed:

ImportError: cannot import name 'AsyncSessionLocal' from 'db.database'
Did you mean: 'async_session_local'?

I'd renamed the session factory to lowercase async_session_local everywhere — except main.py, the one file that boots the app. Locally everything passed because my local checkout had the fix; the prod image was the first thing to actually exercise the new code path.

The fix was a one-line backward-compat alias in db/database.py:

AsyncSessionLocal = async_session_local

Ugly? Slightly. But it decoupled the deploy from a full-file audit, and the old name dies whenever I feel like it. Lesson: renames are the most dangerous refactors — they're invisible to every test that doesn't import the old name, and the failure only shows up at boot time in the environment you test least.

3. The column that didn't exist

With the backend finally healthy, the first real pipeline run died with:

asyncpg.exceptions.UndefinedColumnError:
column repositories.last_processed_commit does not exist

I'd added last_processed_commit to the SQLAlchemy model (it tracks the diff base so re-runs only re-document changed files), and create_all() happily created it on fresh databases. But create_all() never alters existing tables. Production's repositories table was created weeks ago — the column simply wasn't there.

The fix was a one-off migration run through the app's own async engine:

async with engine.begin() as conn:
    await conn.execute(
        text("ALTER TABLE repositories "
             "ADD COLUMN IF NOT EXISTS last_processed_commit VARCHAR(100)")
    )

No redeploy needed, ran in seconds. The real lesson: this project needs a proper migration story. create_all() is a development convenience, not a schema management strategy — the next model change gets Alembic, not another one-off script.

4. Groq rate limits and the multi-key pool

One Groq API key, a pipeline that fans out dozens of LLM calls, and Groq's rate limiter — you can guess how that went. 429s mid-pipeline, retries burning time, the whole run stretching past Cloudflare's 100-second timeout.

The fix: backend/utils/groq_pool.py — a round-robin pool across up to 10 keys, read straight from the environment with zero config changes:

# GROQ_API_KEY, GROQ_API_KEY_2, ... GROQ_API_KEY_10
keys = [os.getenv("GROQ_API_KEY")] + [
    os.getenv(f"GROQ_API_KEY_{i}") for i in range(2, 11)
]
keys = [k for k in keys if k]  # pool = whatever you configured

And doc_generator.py was rewired so the old module-level ChatGroq singleton is gone — chains are rebuilt per attempt via make_llm(), so every call and every 429 retry rotates to a fresh key. Six keys live in AWS Secrets Manager, wired into the ECS task definition.

Combined with 5-module batching and batch Qdrant upserts, the result: a ~20-file repo completes in under a minute. The standing target is sub-100 seconds on big repos, and we're inside it.

5. The 40-file gate: fail fast instead of timing out

Some repos are just too big for v1. Before the gate, triggering a pipeline on a large repo meant watching a loader spin until something timed out — the worst possible UX, and wasted compute.

Now there's an upfront repo-size check in the trigger path: count Python files via the GitHub recursive tree API, and if it's over MAX_REPO_FILES (default 40), return 413 immediately with a clear message instead of starting a doomed pipeline:

if file_count > MAX_REPO_FILES:
    raise HTTPException(
        status_code=413,
        detail="This repository exceeds the 40-file medium-project limit. "
               "Large-repository support is under development.",
    )

The frontend got a matching upgrade: a proper ApiError class carrying the HTTP status (instead of a bare Error that swallowed it), the pipeline loader closes on a definitive rejection instead of spinning forever, and the message shows in a styled dialog — not a window.alert. Small detail, big difference in how the product feels.

6. Security hardening: CSP, httpOnly cookies, and the front door

Phase 6's auth worked, but "works" isn't "hardened." This round:

• Content Security Policy headers — the app only loads resources it explicitly trusts
• httpOnly + Secure cookies for the JWT — JavaScript can't touch the token, so XSS can't steal the session
• Cloudflare in front of everything — TLS, DDoS absorption, and caching at the edge
• ALB → ECS Fargate — the load balancer terminates traffic and routes to the Fargate tasks; the containers themselves never face the internet directly

None of this is glamorous. All of it is the difference between a demo and a product.

7. The infrastructure itself: Docker → ECR → ECS → CI/CD

The actual Phase 7 checklist, now complete:

• Docker — multi-stage builds for backend and frontend, docker-compose for local full-stack testing
• ECR — private image registry; CI pushes :latest on every merge to main
• ECS Fargate — serverless containers, no EC2 instances to babysit; web process and background pipeline workers run in the same task
• GitHub Actions — build, push, deploy, done. Merging to main ships to production without touching a terminal
• Secrets Manager — Groq keys, JWT secrets, Fernet keys, all out of env files and into managed secrets referenced by the task definition
• VPC + security groups — the containers live in private subnets; only the ALB is public

One sharp edge worth knowing: background pipeline tasks run inside the web container, so a deploy mid-pipeline kills the run and leaves its status stuck at running. Don't trigger big pipelines right before a deploy. (A proper worker queue is a v2 problem.)

---

Why None of This Broke the Architecture

Same story as Part 6, one level up. Every item above was plumbing — a task definition field, an import name, a missing column, a key pool, a gate, headers. Not one touched the LangGraph structure, the agent responsibilities, or the API contracts the frontend depends on.

The decisions still paying rent:

• One node per responsibility — the Groq pool slotted under doc_generator without the graph knowing keys exist
• Layered backend — the 413 gate lives in the service layer; routes, repos, and models didn't move
• Typed API client — adding ApiError with a status code was a 10-line change in one file, and every error path in the dashboard got smarter

---

Also Shipped This Round

• 🎨 34 stable DaisyUI themes — a full theme system with a dedicated picker page and provider; the whole app re-skins cleanly, light to dark to everything in between
• 🚫 Explicit 404 page — a proper not-found route in the Next.js app instead of the framework default, on-brand ("This page has no documentation")
• 💬 Dialogs, not alerts — every window.alert in the dashboard replaced with a styled modal
• 📝 Landing page rewrite — honest v1 positioning: Python-only, right in the product name and hero, with the JS/TS/Go/Java/C++ roadmap visible instead of hidden
• 📖 README overhaul — accurate project structure (the old one was two refactors out of date), live demo link, real env var table including the multi-key pool and the size gate
• ⚡ Pipeline loader that never lies — closes on definitive failures, streams real progress on real runs

---

Where Things Stand

Piece	Status
Backend + 13 endpoints	✅ Complete
Multi-agent pipeline, end-to-end	✅ Confirmed working live
Docker → ECR → ECS Fargate + CI/CD	✅ Complete — auto-deploys on merge
Custom domain + Cloudflare + ALB	✅ Live at devdocai.nevinbali.me
Security: CSP, httpOnly cookies, Secrets Manager	✅ Hardened
Multi-key Groq pool + batching (sub-60s on 20+ files)	✅ Live
40-file repo gate + honest error UX	✅ Live
34 DaisyUI themes, 404 page, landing page	✅ Complete
v2 — multi-language parsers	🔜 Planned
v2 — large-repo support (40+ files)	🔜 Planned
v2 — dedicated worker queue (Celery)	🔜 Planned

---

What's Next — Part 8

v2. The two big unlocks: parsers for JavaScript, TypeScript, Go, Java, and C++, and large-repository support so the 40-file gate can retire. Plus a proper worker queue so deploys stop being able to kill running pipelines, and Alembic so I never hand-write another ADD COLUMN IF NOT EXISTS.

The pattern so far: every part ends with "it's basically done" and the next part starts with the list of things production taught me. I expect Part 8 to be no different.

---

GitHub

Nevin100 / DevdocAI

DevDocAI is a production-grade multi-agent AI system that automatically generates, maintains, and updates engineering documentation by deeply understanding you…DevDocAI is a production-grade multi-agent AI system that automatically generates, maintains, and updates doc. by understanding deeply.

DevDocAI 📄

DevDocxAi is a production-grade multi-agent LangGraph system that automatically generates and updates engineering documentation from your GitHub codebase.

---

---

🌐 Live: devdocai.nevinbali.me

🚨 The Problem:

Every engineering team has the same dirty secret — the docs are lying.

Not intentionally. Code moves fast, documentation doesn't.

• New dev joins → 2 weeks reading outdated wikis
• Senior engineers constantly interrupted with "what does this do?"
• PR gets merged → docs never updated
• Generic RAG chatbots don't understand code structure

DevDocAI fixes this.

---

✨ What It Does

• 🔍 Connects to your GitHub repo via OAuth
• 🌳 Parses your codebase at the AST level — understands functions, classes, modules
• 📝 Auto-generates structured documentation per module and function
• 🔄 Updates docs on every PR merge via GitHub webhooks (diff-aware — only changed files re-documented)
• 👀 Human-in-the-Loop review — you approve before anything goes live
• 💬 Onboarding chatbot — new devs ask questions, get answers…

View on GitHub

---

Building in public — from "it works on my laptop" to "it's live on the internet."

Tags: python langgraph fastapi aws ecs docker cicd qdrant rag opensource