I just shipped the backend for YouTube RAG Chat: paste a YouTube link, chat with the video, and get AI-generated summaries, study notes, and quizzes — every answer grounded with clickable timestamp citations.
It runs in production. It streams responses token-by-token. It costs me exactly ₹0 per month.
Getting there was not smooth. My container OOMed on every startup, Google deprecated my embedding model mid-build, and YouTube blocked my server's IP the moment I deployed. This article is the story of the system, the three walls I hit, and what each one taught me about building AI backends under real constraints.
The problem: video is unsearchable
We've all done it: scrubbing back and forth through a 40-minute tutorial trying to find the 90 seconds where the presenter actually explains the thing. Video transcripts exist, but reading a raw transcript is worse than watching the video. What you actually want is to ask the video a question and jump straight to the answer.
That's a textbook Retrieval-Augmented Generation problem — and RAG remains the backbone of production AI in 2026, powering the majority of deployed AI applications. So instead of another chatbot-over-PDFs demo, I built RAG over the medium people actually consume: YouTube.
The requirements I set for myself:
- Grounded answers. Every claim traceable to a moment in the video, via inline timestamps.
- Streaming. First token in 1–2 seconds, not a 15-second spinner.
- Cached everything. Same video, second user → sub-second responses.
- ₹0 infrastructure. Not "cheap." Zero. As a forcing function for good architecture.
That last constraint turned out to be the most educational decision of the project.
The stack
| Layer | Choice |
|---|---|
| Language | Python 3.11 |
| Web framework | FastAPI (async) |
| ORM + migrations | SQLAlchemy 2 (async) + Alembic |
| Vector DB | ChromaDB (local, persistent) |
| Metadata DB | SQLite |
| LLM | Groq — llama-3.3-70b-versatile, streamed via SSE |
| Embeddings | Google Gemini — gemini-embedding-001 (768 dims) |
| Transcripts | Supadata API |
| Hosting | Render free tier (Docker) |
| Keep-alive | UptimeRobot, 10-minute pings |
Note that half of these choices weren't my first choices. They're what survived contact with production. More on that shortly.
Architecture
Clean architecture, four layers, dependencies pointing strictly inward:
Routers → Services → Repositories → Infrastructure
Diagram description (for Excalidraw): Four horizontal bands stacked top to bottom. Band 1 — Routers: boxes for /videos, /chat, /summary, /notes, /quiz, /health, each receiving arrows from a "Client" box above. Band 2 — Services: boxes for IngestionService, ChatService, ContentService (summary/notes/quiz). Arrows from each router down to its service. Band 3 — Repositories: VideoRepository, ChunkRepository. Band 4 — Infrastructure: boxes for TranscriptFetcher (Supadata), EmbeddingClient (Gemini), LLMClient (Groq), ChromaDB, SQLite. Arrows from repositories/services down into infrastructure. One dashed arrow labeled "SSE stream" going from ChatService back up and out to the Client, annotated with the event sequence sources → tokens → done.
Three design decisions worth calling out:
1. No user identity on the backend
The backend is stateless per user. There are no accounts, no sessions, no per-user tables. Each user's personal video library and chat history will live in their browser's IndexedDB — a frontend concern I'll cover in the next post. The backend only knows about videos, not people.
This sounds like a limitation. It's actually the unlock for decision #2.
2. A shared, deduplicated cache
Videos are keyed by YouTube video ID. If User B ingests a video User A already processed, the backend detects the cache hit and returns metadata in under a second — no transcript fetch, no re-chunking, no re-embedding. One popular video costs the system exactly one ingestion, ever.
Look at the cached: true flag in the ingestion response below — that request returned in well under a second because someone (me, in this case) had already processed the video.
3. Timestamps as first-class citizens
The LLM is prompted to emit [ts:MM:SS] markers inline in its answers, sourced from the retrieved chunks' time ranges. The upcoming frontend will render these as tappable chips that seek the video player. The backend contract is simple: every answer carries its receipts.
The API, live
All examples use {{live}} as a placeholder for the base URL. Response bodies below are real, pasted from production.
Ingest a video
curl -X POST {{live}}/api/v1/videos \
-H "Content-Type: application/json" \
-d '{"url": "https://www.youtube.com/watch?v=xpDnVSmNFX0"}'
{
"video_id": "xpDnVSmNFX0",
"title": "System Design BASICS: Horizontal vs. Vertical Scaling",
"channel": "Gaurav Sen",
"thumbnail_url": "https://i.ytimg.com/vi/xpDnVSmNFX0/hqdefault.jpg",
"duration_seconds": 476,
"transcript_lang": "en",
"chunk_count": 18,
"created_at": "2026-08-09T09:20:07.152949Z",
"cached": true
}
The pipeline behind this: validate the URL → fetch the transcript → chunk it (this 8-minute video became 18 chunks) → embed each chunk via Gemini → store vectors in ChromaDB and metadata in SQLite. On a cache hit, all of that is skipped.
Chat, streamed over SSE
curl -N -X POST {{live}}/api/v1/chat \
-H "Content-Type: application/json" \
-d '{"video_id": "xpDnVSmNFX0", "question": "What is this video about?", "history": []}'
The stream arrives as three event types, in order:
1. sources — the retrieved chunks, sent first so a client can render citations before the answer starts:
{
"chunks": [
{
"start_time": 0.04,
"end_time": 26.829,
"preview": "This video is on the basics of system design. If you have never designed a system before, this is probably the place..."
},
{
"start_time": 457.42,
"end_time": 475.52,
"preview": "We design a system which is going to meet the requirements, and the requirements are such that it's going to be Compu..."
}
]
}
2. token — the answer, one token at a time:
token → {"content":"This"}
token → {"content":" video"}
token → {"content":" is"}
token → {"content":" about"}
token → {"content":" the"}
...
3. done — stream complete.
Sending sources before tokens is a small ordering decision with a big UX payoff: the user sees where the answer will come from while the answer is still being generated. First-token latency in production is 1–2 seconds, courtesy of Groq's inference speed.
Generated content: summary, notes, quiz
Three more endpoints turn any ingested video into study material, each cached after first generation:
curl -X POST {{live}}/api/v1/videos/xpDnVSmNFX0/summary
{
"video_id": "xpDnVSmNFX0",
"short": "This video covers the basics of system design, starting with a simple algorithm running on a computer and exposing it to others through an API...",
"detailed": [
"The video starts by introducing the concept of system design and how it begins with a simple algorithm running on a computer.",
"As the algorithm becomes useful to others, it needs to be exposed through an API, allowing others to send requests and receive responses.",
"..."
],
"cached": false,
"generated_at": "2026-08-09T09:45:48.459477Z"
}
The quiz endpoint is the most interesting of the three. The LLM must return strict, structured JSON — question, four options, correct index, explanation with a timestamp — and the backend validates the structure before accepting it (agent-style output validation, retry on malformed responses):
curl -X POST {{live}}/api/v1/videos/xpDnVSmNFX0/quiz
{
"video_id": "xpDnVSmNFX0",
"question_count": 5,
"questions": [
{
"question": "What is the purpose of exposing code using an API?",
"options": [
"To store output in a file",
"To connect to a database",
"To allow others to use the code over the internet",
"To configure endpoints"
],
"correct_index": 2,
"explanation": "The purpose of exposing code using an API is to allow others to use the code over the internet, as stated at [ts:02:45]."
},
{
"question": "What is the difference between a desktop and a cloud?",
"options": [
"A desktop is a set of computers, while a cloud is a single computer",
"A desktop is a single computer, while a cloud is a set of computers",
"A desktop is used for personal use, while a cloud is used for business",
"A desktop is faster than a cloud"
],
"correct_index": 1,
"explanation": "A desktop is a single computer, while a cloud is a set of computers that can be used to run a service, as stated at [ts:06:30]."
}
]
}
The full endpoint list: POST /videos, GET /videos/{id}, POST /chat, POST /videos/{id}/summary, POST /videos/{id}/notes, POST /videos/{id}/quiz, GET /health.
Now for the part that actually earned this article: nothing above worked on the first deployment.
Wall 1: The 512 MB ceiling
This is the wall that reshaped the whole architecture, so it gets the deepest treatment.
My original embedding plan was the obvious one: run sentence-transformers locally with all-MiniLM-L6-v2. It's the default in every RAG tutorial, it's free forever, and it works beautifully on a dev machine.
Then I deployed to Render's free tier and the container OOMed on startup. Every time.
The math, once I actually did it, was brutal:
- PyTorch runtime: several hundred MB of RAM before doing anything useful
- The MiniLM model loaded into memory
- ChromaDB's own footprint
- FastAPI + the async stack
Total: ~800 MB. Render free tier: 512 MB. The container never even reached the health check. There was no clever flag or lazy-loading trick that closes a 300 MB gap — the architecture itself was wrong for the box.
So I stopped trying to squeeze a local model into 512 MB and asked a better question: why is my 0.1 vCPU web server doing ML inference at all? Embedding is a compute-heavy, stateless operation — exactly the kind of thing to push to a managed API. I migrated embeddings to Google Gemini's embedding API.
The results:
- Backend RAM: ~800 MB → ~200 MB. Comfortable headroom on a 512 MB box.
- Docker image: shrank by ~500 MB. No PyTorch, no model weights. Final compressed image is under 500 MB — small for an AI backend.
- Embedding quality went up. 768 dimensions vs MiniLM's 384, with roughly a 10-point improvement on MTEB benchmarks. The constraint didn't just save the deploy — it produced better retrieval.
Lesson: free-tier constraints are an architecture review you didn't ask for. Local models are lovely on a beefy dev machine and terrible on a 0.1 vCPU box. If a component is stateless and compute-heavy, it probably doesn't belong in your web server's process.
Wall 2: My embedding model no longer existed
Fresh off the Wall 1 fix, I wired up Gemini's text-embedding-004 — the model most Gemini RAG tutorials reference — and the very first call returned a 404.
Not a rate limit. Not an auth error. The model was gone. Google had shut down text-embedding-004 on January 14, 2026, and I'd picked my model from tutorials instead of release notes.
The fix took minutes, which is the actual point of the story:
- Switched to
gemini-embedding-001— stable, generally available. - It uses Matryoshka Representation Learning, meaning the model is trained so that truncated prefixes of its embedding vector are themselves valid embeddings. The default output is 3072 dimensions; I explicitly request 768, which is plenty for transcript chunks and keeps ChromaDB's storage and query costs down.
- Because the model name lived in an environment variable rather than being hardcoded, the swap was a one-line config change. No code touched, no redeploy logic, no migration script.
Lesson: when you pick an API model, read the release notes and the deprecation timeline, not just the tutorial. And treat model names as configuration, not code — the day your provider kills a model (and that day will come), you'll fix it in one line instead of one afternoon.
Wall 3: "Works on my machine" — literally
With memory fixed and embeddings live, ingestion worked flawlessly on my laptop. I deployed. Transcript fetching failed instantly.
The culprit: YouTube blocks datacenter IPs. My laptop's residential IP fetched transcripts happily via youtube-transcript-api; Render's cloud IP got stonewalled. This isn't a Render problem — AWS, GCP, Azure all hit the same wall, because YouTube treats datacenter IP ranges as scraper traffic.
The realistic options were: run my own residential proxy layer (cost, complexity, ToS gray zones), or use a managed transcript API that handles that layer for me. I swapped to Supadata's API, which does exactly that, transparently, with a free tier generous enough for an MVP.
Two design decisions made this migration painless:
-
The abstraction held. All transcript fetching went through a single
TranscriptFetcherclass. Swappingyoutube-transcript-apifor Supadata was a single-file change — routers, services, and the chunking pipeline never knew anything happened. - Deduplication multiplies the free tier. Because videos are cached by ID, one popular video costs one transcript fetch total — not one per user. The cache isn't just a latency feature; it's what makes the ₹0 economics hold as usage grows.
Lesson: "it works on my machine" is a real infrastructure signal, not just a meme. Your laptop and your server can live in genuinely different networks with different rules. And the boring advice about wrapping external dependencies in your own abstraction pays for itself the first time an external dependency betrays you — which, per Walls 2 and 3, was twice in one project.
The ₹0 bill
| Service | Purpose | Monthly cost |
|---|---|---|
| Render | Backend hosting | ₹0 |
| Groq | LLM inference | ₹0 (free tier) |
| Google Gemini | Embeddings | ₹0 (free tier) |
| Supadata | YouTube transcripts | ₹0 (free tier) |
| ChromaDB | Vector storage | ₹0 (local, open source) |
| SQLite | Metadata storage | ₹0 (local, open source) |
| UptimeRobot | Keep-alive pings | ₹0 (free tier) |
| Total | ₹0 |
The UptimeRobot row deserves a sentence: Render's free tier spins containers down when idle, which means multi-second cold starts for the first unlucky user. A ping every 10 minutes keeps the container warm. Cold starts still exist in theory but are rare in practice (~10 seconds when they happen).
Where it landed
The numbers, in production:
- Backend RAM: ~200 MB (down from ~800 MB in the original design)
- Docker image: under 500 MB compressed
- First-token latency (chat): ~1–2 seconds
- Cache-hit responses: under 1 second
- Cold starts: ~10 seconds, rare thanks to keep-alive pings
Not bad for a stack whose total infrastructure line item is a hard zero.
What's next
The backend streams grounded, timestamped answers — but right now the only client is cURL. Next milestone: a Flutter Web app that turns those [ts:MM:SS] markers into tappable chips that seek the actual video player, with each user's library and chat history living entirely in their browser. That gets its own post.
If there's one thing to take from this build: constraints are a feature. The 512 MB ceiling forced a cleaner architecture. The deprecated model enforced config discipline. The IP blocking validated the abstraction layer. A bigger budget would have let me ship all three mistakes to production and never learn from them.
About the Author
I'm Pankaj Batra, a Software Engineer focused on Flutter, automation, and enterprise integrations.
I write about practical engineering: mobile architecture, workflow automation, APIs, event-driven systems, and lessons from production systems.
Connect
- Portfolio: https://pankajbatra.vercel.app
- LinkedIn: https://linkedin.com/in/pankaj-batra-0a294a205
- GitHub: https://github.com/Pankaj0405
If this was useful, follow for more engineering notes.
Top comments (2)
Hey, thanks for mentioning us!
We're actually doing a spotlight series, where we highlight members of our community: if you built something in the past and used UptimeRobot, we would love to have a short written interview with you.
I can DM you with more info. Let us know if you're interested!
Great writeup. I am deploying a FastAPI + Gemini 2.5 Flash MVP (NL-to-pandas code generator) on the same Render free tier and hit a different set of walls.
Wall 1: The deploy was impossible and I did not know for 35 sessions. My project folder lived inside a workspace whose root .gitignore had projects/ on line 80. The MVP verified locally every time — green health check, real LLM output, 22 few-shot examples passing. But Render deploys from GitHub, and GitHub had 0 tracked files. Local green does not equal deployable. Verify the deploy path itself, not just the app.
Wall 2: Free-tier cold starts compound. You mentioned UptimeRobot pings — same here. Render free tier sleeps after ~15 min idle; first request takes 30-50s. For an API a frontend calls on button-click, that is dead UX. 10-min pings fix it but feel like a hack. Curious if you considered Render cron trick or just accepted the ping tax.
The Gemini embedding deprecation mid-build is brutal. I am on google.generativeai which is now deprecated in favor of google.genai, and I am deliberately not migrating pre-deploy because it works and migrating risks breaking the verified MVP.
One thing I would add: rate-limit at the IP level from day 1. Gemini free tier is 250 RPD. Without IP limiting, one user exhausts the daily quota for everyone. I added a 5/day/IP in-memory limiter (Redis-swappable) — costs nothing, prevents the tragedy of the commons.
What is your ChromaDB persistence story on Render free tier? The free disk is ephemeral on redeploys — do you re-embed on every cold start, or persist to an external store?