100+ Users, Free API Keys and ₹0 Cloud Budget: How I Built Kairos

Pawan Eswaran — Sun, 13 Sep 2026 21:54:29 +0000

I wanted to build a simple RAG system.

That sentence aged badly.

The original problem was actually pretty simple.

We were organizing a departmental event called Kairos, where participants had to study company information and create pitch decks.

The organizers had a lot of company reports.

The participants had to read them.

And the obvious solution was:

Print everything.

Except there was one tiny problem.

There were a lot of pages.

And there were a lot of participants.

Printing enough copies for everyone wasn't exactly friendly to the budget.

So I had a stupidly simple thought:

Why don't we just put all the documents into an AI system and let participants ask questions?

And that's how I accidentally created a much bigger problem for myself.

Problem #1: "Just build a RAG"

If you've built RAG systems before, you know how innocent that sentence sounds.

Upload PDFs → chunk them → embed them → retrieve relevant chunks → send them to an LLM.

Easy.

Until you actually try to make people use it.

I needed:

PDF ingestion
embeddings
vector search
retrieval
reranking
LLM generation
citations
a frontend
a backend
authentication
rate limiting
logging
an admin dashboard
and eventually, somehow, 100+ people using it at the same time

And I had one major constraint:

I didn't have a budget.

Not "small startup budget."

I mean ₹0 infrastructure budget.

Problem #2: The LLM was going to bankrupt my imaginary company

The next question was obvious:

Which LLM should I use?

Paid APIs?

No.

I didn't have money for that.

So I started looking at free API tiers.

I ended up using Gemini as the primary model and OpenRouter/Nemotron as a fallback.

Problem solved?

Absolutely not.

Free APIs have limits.

And when you have one user asking a question, rate limits are someone else's problem.

When you have 100+ users asking questions...

Congratulations. You have invented a distributed systems problem.

Problem #3: "What if we just use multiple API keys?"

This was probably my favourite hack.

Instead of relying on one API key, I created multiple API lanes.

Something roughly like:

         Incoming Request
                |
                v
         LLM Gateway
                |
    +-----------+-----------+
    |                       |
Gemini lanes          OpenRouter lanes
 G01...G10               N01...N10
    |                       |
    +-----------+-----------+
                |
                v
             Response

I ended up with 20 lanes:

10 Gemini lanes
10 OpenRouter/Nemotron fallback lanes

Each lane had its own quota and concurrency control. The backend could distribute requests across them instead of hammering a single API key until it died. And because multiple users could hit the system simultaneously, I used PostgreSQL-based locking to coordinate access.

Was this the most elegant infrastructure ever created? No. Was it considerably cheaper than paying for a giant API bill? Yes.

Problem #4: Now I needed a server

At this point I had an AI system. Now I needed somewhere to run it.

I looked at cloud hosting. Then I looked at the prices. Then I looked at my budget. Then I looked back at my laptop.

We reached an agreement. My laptop was now the cloud.

I ran the application locally and exposed it through ngrok

At some point I stopped calling it a laptop and started calling it production infrastructure. It made me feel better.

Problem #5: What if everyone asks questions at once?

The system wasn't being built for me. It wasn't: "Look, here's my RAG demo." People were actually going to use it. Potentially 100+ people. And I didn't want the first question on event day to be: "Why is the website down?"

So I added:

Per-team question quotas
Request logging & rate limiting
API key routing & fallback handling
Concurrency controls & event timers
Monitoring/admin views

I limited each team to 10 questions. This wasn't only about saving API calls—it also made the event more interesting because we could later look at the prompts participants were asking and evaluate how they were using the system.

Problem #6: RAG started lying to me

This was where the project became more interesting technically. I didn't just want: "Here are some chunks that look relevant." I wanted the system to actually reason about what kind of information it needed to retrieve.

Two-Stage Retrieval: The first stage interpreted the question and created a retrieval plan. The second stage actually searched the company knowledge base.
Permission Boundaries: I kept the internal instruction knowledge separate from the company documents that participants were allowed to access.
Page-Level Citations: Instead of "According to the document...", the system pointed participants toward the exact page containing the information.

Event Day

The system was running on my laptop. The laptop was connected to the internet. ngrok was exposing it. The APIs were running through free quotas. PostgreSQL was handling state. Qdrant was handling vector search. The RAG pipeline was running.

And then... people started using it. Questions came in. Answers came back. People were actually using the system to search through the company information.

And it worked.

No paid cloud infrastructure. No expensive LLM subscription. No giant GPU server. No fancy production cluster. Just a lot of engineering, questionable financial decisions, and one laptop that was working much harder than it deserved to.

What I Actually Learned

The biggest lesson wasn't about RAG. It was about constraints.

When you have unlimited resources, the solution is often:

"Just use a bigger server."
"Just use the paid API."
"Just scale horizontally."

When you have ₹0, you can't say that. You have to ask: "Okay. What can I change instead?"

Free API limits became an API-routing problem.
No cloud budget became a deployment problem.
Too many users became a concurrency problem.
Large documents became a retrieval problem.

And eventually, all those little problems became an actual system. That's probably the part of the project I'm most proud of.

The Final Stack

Layer	Technologies
Frontend	React, Vite, TypeScript, Tailwind, PDF.js
Backend	FastAPI, Uvicorn, Async REST APIs, SSE streaming
AI & Retrieval	Gemini, OpenRouter / Nemotron, SentenceTransformers (`BAAI/bge-small-en-v1.5`), Two-stage retrieval, Reranking, Adaptive response depth
Data & Storage	PostgreSQL, SQLAlchemy, Qdrant
Infrastructure	Docker, Docker Compose, ngrok (and my laptop)

🔗 GitHub Repository

The complete project is open source:

https://github.com/Pawan-19012006/Kairos-RAG-System

One Thing I'd Do Differently

If I rebuilt this today, I'd spend much more time designing the evaluation and observability layer from the beginning.

Getting a RAG system to answer is relatively easy. Getting it to answer correctly, consistently, with the right evidence, under real usage constraints is where the interesting engineering begins.

And that's where this project stopped being "I built a chatbot" and became "I built a system that had to survive actual users."

That's a much more fun problem.

P.S. If you're building something similar and you're worried because you don't have a huge cloud budget: Don't let the lack of money decide whether the project is possible. Sometimes the most interesting engineering happens when you have no other option.

And sometimes... your laptop becomes the cloud. 😅

#DevWatch — Turning GNOME into a Developer-Aware OS

Pawan Eswaran — Mon, 23 Mar 2026 05:55:36 +0000

Most developer tools on Linux are process-centric.

But we don’t think in processes.

We think in:

projects
services
ports
builds

So i built DevWatch — a GNOME Shell extension that makes your desktop understand your development workflow.

🧠 What is DevWatch?

DevWatch transforms GNOME from a generic desktop into a developer-aware operating layer.

Instead of showing random PIDs and CPU usage, it tells you:

👉 Which project is running what
👉 Which port belongs to which service
👉 What your dev environment is actually doing

The DevWatch Panel

When you click the DevWatch button in your GNOME panel, the dropdown gives you a complete, real-time view of your development environment.

🧾 Header Bar

Live stats showing:
- number of active projects
- number of open ports
- total RAM usage
Stop All Projects — instantly kills all running dev processes
Settings — opens the preferences window

🔍 Running Projects

Each project is shown as an expandable card:

Service-oriented view 👉 shows "Python Server" instead of raw python3.12
Stop Project — kills all processes in that project
Open Terminal — opens gnome-terminal at the project’s git root

🌐 Open Ports

Highlights dev ports (3000, 5173, 8080, etc.) with colored dots
Shows project name linked to each port
Stop button kills the process using that port
Displays runtime (how long the port has been active)

📷 Sessions

Save — name and store your current workspace
Last Workspace — auto-saved every refresh (persists across reboots)
Resume — restores terminals, services, and editors
Delete — remove saved sessions

⚡ Build Activity

Shows live builds with:
- elapsed time
- CPU usage
Recent Builds — collapsible history with:
- duration
- peak CPU usage
Detects builds from:
- npm, cargo, make, go build, gradle, webpack, vite, tsc, gcc, and more
Build history persists across reboots

⚠ Problems (Conditional Section)

Appears only when issues are detected:

⚠ trackitte using 3.2 GB RAM
⚠ backend CPU at 92%

🧠 Why I Built This

As a developer, I kept doing things like:

lsof -i :3000 ps aux | grep node kill -9 <pid>

And I realized:

Why is my OS not aware of my projects?

DevWatch is my attempt to fix that.

What I Need From You

I’d love honest feedback from developers:

Specifically:
• Does this solve a real problem for you?
• What feature would you actually use daily?
• What feels unnecessary?
• What would make you install this instantly?

Try It Out !!!

👉 GitHub:
https://github.com/Adithya-Balan/DevWatch

(Installation steps are in the README)

🙌 Feedback Welcome

This is still evolving — and I want to shape it around real developer workflows, not assumptions.

👉 Drop your thoughts, critiques, or feature ideas below in the comments.

Even harsh feedback is welcome — that’s how this gets better 🚀

DEV Community: Pawan Eswaran