DEV Community: Danish Raza Bangash

Redis, Caching, and Queues: The Boring Stuff That Makes Apps Feel Fast

Danish Raza Bangash — Tue, 04 Aug 2026 10:56:22 +0000

When I started building Footalyzer, I didn't think much about Redis. I had Next.js on the frontend, Express on the backend, MongoDB for data — that felt like enough. Then I started hitting real problems: slow API calls, repeated requests hammering an external football API, and AI-generated content that took too long to feel "live." That's when Redis, caching, and job queues stopped being buzzwords and became things I actually needed.

Here's the simple version of what I learned.

What even is Redis?

Redis is basically a super-fast storage box that lives in memory (RAM) instead of on disk. Because RAM is way faster than a database like MongoDB, reading and writing to Redis takes microseconds instead of milliseconds.

The catch: it's not meant to be your main database. It's meant to hold stuff temporarily — things you'll ask for again and again, or things you need to process in the background.

That's really the two big uses: caching and queues. Different problems, same tool.

Caching: stop doing the same work twice

Say a user opens Footalyzer and asks for a match briefing. Behind the scenes, that might mean:

Calling an external football API for match data
Calling an AI model to generate the analysis
Formatting everything nicely

That's slow and, if it's hitting a paid API, expensive too. Now imagine 50 people ask about the same match in the next hour. Do you really want to repeat all that work 50 times?

This is where caching comes in. The first time someone asks, you do the work and save the result in Redis with an expiry time:

const cached = await redis.get(`match:${matchId}:briefing`);
if (cached) return JSON.parse(cached);

const briefing = await generateBriefing(matchId); // slow, expensive
await redis.set(`match:${matchId}:briefing`, JSON.stringify(briefing), "EX", 300); // cache for 5 min

return briefing;

Now the next 49 people get an instant response, and you saved 49 API calls. That "EX 300" part just means "forget this after 5 minutes" — useful for live football data that changes during a match.

The mental model that helped me: cache anything that's expensive to compute but doesn't change every second. Match briefings, standings, player stats — all good candidates. Live scores that update every few seconds? Not so much, unless your expiry is really short.

The other problem: things that take too long to do "live"

Caching solves repeated work. But some work is just plain slow the first time too — like generating an AI briefing, sending emails, or processing a webhook from a payment provider. If you make the user wait for all of that inside a single request, your app feels frozen, and on a slow connection the request might just time out.

The fix is to not do it "live" at all. Instead, you hand the task off to a queue, respond to the user immediately ("we're on it!"), and process the actual work in the background.

Enter BullMQ

BullMQ is a job queue library built on top of Redis. It's what actually makes "background jobs" possible in a Node.js app. The idea is simple:

Something happens (a user requests a briefing)
You add a "job" to a queue instead of doing the work right there
A separate "worker" picks up jobs from the queue and processes them, one at a time or several in parallel

// adding a job
await briefingQueue.add("generate-briefing", { matchId });

// a worker processing it, elsewhere in the codebase
new Worker("briefing-queue", async (job) => {
  const { matchId } = job.data;
  const result = await generateBriefing(matchId);
  await saveBriefing(matchId, result);
});

Redis is quietly doing the heavy lifting underneath — storing the queue, tracking which jobs are pending, done, or failed, and letting BullMQ retry jobs automatically if something breaks.

This is where things clicked for me: caching makes reads fast, queues make writes/processing not block the user. They solve different halves of the same "make the app feel instant" problem.

How this plays out in Footalyzer

In practice, a lot of it comes down to:

Cache the AI-generated briefings and stats pulls from the football API, since regenerating them for every request is wasteful and slow
Queue the actual AI generation and any heavier background work, so a request that triggers it doesn't just hang
Use Socket.io to notify the frontend once a queued job finishes, instead of making the user refresh or poll

None of this needed to be fancy from day one. I started with just caching a few slow endpoints, and added queues once I noticed requests were timing out during AI generation. That's probably the right order for most people — cache first, queue when you actually feel the pain.

A few honest lessons

Set expiry times on everything you cache. Stale football data is worse than no cache at all.
Don't cache user-specific or fast-changing data unless you really think it through — it's an easy way to serve someone the wrong info.
Queues need monitoring. A silently failing worker is worse than no queue, because now things are "processing" forever and nobody notices.
Redis is cheap to add, easy to misuse. It's tempting to cache everything. Start with the slowest, most repeated calls first.

That's really it. Redis isn't magic — it's just a very fast, temporary shelf. What you put on that shelf (cached results) and how you use it to hand off slow work (queues via BullMQ) is where the actual speed-up comes from.

SQL vs NoSQL: A Simple Guide to Picking the Right Database

Danish Raza Bangash — Sat, 01 Aug 2026 11:31:21 +0000

If you're building your first real app, you'll hit this question pretty fast: "Should I use SQL or NoSQL?"

It sounds intimidating, but once you get the core idea, it's actually pretty simple. Let's break it down.

What's a database anyway?

A database is just a place to store your app's data so it doesn't vanish the moment you close the browser. Think of it as a very organized filing cabinet. How you organize that cabinet is where SQL and NoSQL split off.

SQL databases: structured and strict

SQL databases (like PostgreSQL, MySQL, or SQLite) store data in tables — rows and columns, like a spreadsheet. Every row follows the same structure, and tables can be linked to each other.

Example: a simple users and orders setup.

CREATE TABLE users (
  id SERIAL PRIMARY KEY,
  name VARCHAR(100),
  email VARCHAR(100)
);

CREATE TABLE orders (
  id SERIAL PRIMARY KEY,
  user_id INT REFERENCES users(id),
  item VARCHAR(100),
  price DECIMAL
);

Want all orders for a specific user? One clean query:

SELECT orders.item, orders.price
FROM orders
JOIN users ON orders.user_id = users.id
WHERE users.email = 'jane@example.com';

That JOIN is SQL's superpower. Relationships between data are enforced by the database itself, and querying across them is straightforward.

NoSQL databases: flexible and fast

NoSQL databases (like MongoDB, Firebase, or Redis) don't force your data into rigid tables. The most common type — document databases — store data as JSON-like objects, and each document can look different from the next.

Example: the same "user with orders" idea in MongoDB.

{
  "_id": "0001",
  "name": "Jane",
  "email": "jane@example.com",
  "orders": [
    { "item": "Keyboard", "price": 45 },
    { "item": "Mouse", "price": 20 }
  ]
}

Everything about Jane lives in one document. No joins needed — you just fetch it.

The real differences

	SQL	NoSQL
Structure	Fixed tables, rows & columns	Flexible documents/objects
Schema	Defined upfront	Can change anytime
Relationships	Strong, via joins	Usually nested or duplicated
Scaling	Mostly vertical (bigger server)	Built for horizontal scaling
Good for	Transactions, complex queries	Fast iteration, huge write volume

So which one should you use?

Honestly, it depends on what you're building.

Building something involving money, inventory, or anything that needs strict consistency (banking, e-commerce checkout)? Go SQL.
Building something where the data shape changes often, or that needs to handle a ton of reads/writes fast (chat apps, activity feeds, content platforms)? NoSQL fits better.

Here's the part most tutorials skip: you don't have to pick just one. A lot of real production apps use both — SQL for core transactional data, and something like Redis for caching or MongoDB for logs and analytics. This mix is sometimes called polyglot persistence, and it's more common than people think.

Wrapping up

SQL vs NoSQL isn't really a battle — it's a tradeoff between structure and flexibility. Once you understand what each one is actually good at, the choice usually makes itself based on what you're building.

Docker for MERN Developers: How I Actually Learned It (and Why It Clicked)

Danish Raza Bangash — Sun, 12 Jul 2026 11:27:00 +0000

For the longest time, Docker was that thing on my resume-worthy-tools list that I hadn't really earned. I knew the buzzwords — containers, images, docker-compose — but I didn't know Docker the way you know a tool you've actually shipped something with.

That changed while building BotForge, my multi-tenant no-code AI chatbot builder. Here's how the pieces fell into place, and how Docker now fits naturally into my MERN workflow.

The "it works on my machine" wall

BotForge isn't a single service. It's a MERN app with a Node/Express API, MongoDB, a four-tier RAG pipeline hitting Gemini embeddings, and integrations with WhatsApp and Telegram. Running all of that locally meant juggling environment variables, Node versions, and Mongo instances — and every time I handed the project to my teammate or tried deploying to Render, something that worked perfectly on my laptop broke somewhere else.

That's the exact pain Docker exists to solve, and once I framed it that way, the concept stopped being abstract.

The mental model that made it click

Forget the textbook definitions for a second. Here's the analogy that actually worked for me:

Image = a recipe. It lists every ingredient and step needed to make your app run — Node version, dependencies, source code, start command.
Container = the dish made from that recipe. You can make the same dish a hundred times, on a hundred different stoves, and it comes out identical every time.
Dockerfile = the recipe card itself, written in a language Docker understands.

Once containers stopped feeling like "mini virtual machines" and started feeling like "a guaranteed-identical environment," everything else — layers, volumes, networking — became details I could look up as needed instead of concepts I had to memorize upfront.

How I actually learned it (step by step)

Dockerized one thing first — not the whole BotForge stack, just the Express API. A minimal Dockerfile, one EXPOSE, one CMD. Got it running with docker run.
Understood images vs. containers hands-on by running the same image twice and watching two independent containers spin up.
Introduced docker-compose once I needed Mongo alongside the API — this is where Docker started feeling useful instead of academic, because one docker-compose up replaced a page of setup instructions.
Broke things on purpose — killed containers mid-run, misconfigured ports, forgot .dockerignore and watched node_modules bloat my image. Debugging is where the concepts actually stick.
Recently, I turned this whole progression into an interactive Docker learning page for other MERN devs walking the same path — writing it out taught me as much as building the thing did.

Where Docker actually fits in a MERN stack

This is the part a lot of tutorials skip: Docker isn't one container, it's usually several, talking to each other.

A typical MERN docker-compose.yml looks roughly like this:

services:
  api:
    build: ./server
    ports:
      - "5000:5000"
    environment:
      - MONGO_URI=mongodb://mongo:27017/botforge
    depends_on:
      - mongo

  client:
    build: ./client
    ports:
      - "3000:3000"

  mongo:
    image: mongo:7
    volumes:
      - mongo-data:/data/db

volumes:
  mongo-data:

Each service — React frontend, Express API, MongoDB — gets its own container, its own isolated environment, and they talk to each other over Docker's internal network using service names instead of localhost. For BotForge specifically, this meant my RAG pipeline, the API layer, and the database could all be versioned and deployed together, and the exact same setup that ran on my machine ran identically on Render.

The payoff

The real win wasn't learning Docker commands — it was what it removed: no more "install Node 18.x, then Mongo, then set these seven env vars" onboarding docs. Just docker-compose up. For a multi-tenant app like BotForge, where consistency across environments actually matters, that's not a nice-to-have, it's the difference between a demo and something deployable.

If you're a MERN dev starting today

Don't start with docker-compose. Don't start with Kubernetes (please don't). Start by Dockerizing one Express route. Watch it run identically twice. Then add Mongo. Then add your frontend. The stack builds itself once the first container clicks — and it will click faster than the docs make it seem.

Building for Bursty Traffic: Queues and Real-Time Design for a World Cup App July 11, 2026

Danish Raza Bangash — Sat, 11 Jul 2026 10:33:45 +0000

Sports traffic is a brutal test case for system design. Nothing happens for 89 minutes, then a goal fires and every connected client wants an update in the same second. I hit this directly building a World Cup 2026 companion app — live scores, match analysis, notifications — using Next.js, Socket.io, and Redis-backed job queues (BullMQ).

This isn't a system handling millions of concurrent users. But the design questions that traffic pattern forces on you are the same ones you'd face at much larger scale — you just get more room to experiment before it actually breaks. Here's what I learned about async processing and real-time delivery.

1. Async-first beats synchronous-and-hope

Handling bursty traffic synchronously — API request comes in, app does the work, app responds — falls over exactly when it matters most, because your spikes are correlated with the thing your users actually care about. A missed goal notification during the one minute it mattered is worse than no feature at all.

So anything that wasn't required to happen in the request-response cycle got pushed into a queue: fetching and processing match events, fanning out notifications, running post-match analysis. The web server's job became "accept the request fast and hand off the work," not "do the work." Redis-backed queues gave me retry logic and backpressure almost for free — if a burst of events comes in faster than workers can process them, they queue instead of timing out or crashing the server.

The tradeoff is real: async systems are harder to reason about, harder to debug (where did this job go?), and you now have eventual consistency to think about instead of "it just happened." But for anything bursty, that tradeoff is almost always worth it over a synchronous system that's simpler right up until the one moment it isn't.

2. Real-time doesn't mean "broadcast to everyone"

Socket.io makes it dangerously easy to broadcast every update to every connected client. That works fine in a demo. It does not work once you have thousands of people watching different matches, because now every client's browser is filtering out most of the messages it receives just to find the one it cares about.

The fix was scoping connections into rooms — one per match — so a client only receives events for what it's actually watching. This is an obvious idea in hindsight, but it's the kind of thing that's easy to skip when you're focused on "does real-time work at all," and only becomes a problem once your Socket.io server is doing meaningfully more work than it needs to per connected client.

The general principle: real-time infrastructure scales by what you don't send, not just by how fast you send it. Scoping and targeting your broadcasts is usually a bigger scalability lever than optimizing the transport itself.

3. Your real bottleneck is probably someone else's API

The app leans on third-party services for live data, AI-powered analysis (Gemini), and billing (Stripe) — each with its own rate limits and failure modes that don't show up until traffic actually spikes. A burst of match events can mean a burst of downstream calls to generate analysis or trigger notifications, and none of those upstream services promise to absorb that burst gracefully just because your own queue can.

That pushed rate-limiting and backoff logic down into the worker layer itself, not just the request layer — workers pulling jobs off the queue needed to respect external rate limits independently of how fast jobs were arriving. The queue smooths out your bursts; it doesn't automatically smooth out the burst you're about to hand to someone else's API.

The lesson: when you sketch your architecture diagram, the boxes representing external APIs deserve as much design attention as the boxes you control. Queueing your own work is necessary but not sufficient — you have to think about what your workers do when they hit someone else's limits too.

4. Cheap infrastructure, disciplined architecture

This system runs on free/low-tier hosting, not a fleet of load-balanced servers — because that's the reality of building as a student. That constraint was useful. It forced decisions that scale regardless of the infrastructure underneath: stateless services that don't care which instance handles a request, background work that lives in a queue instead of memory, and real-time connections scoped tightly enough that they don't waste bandwidth they don't need.

The uncomfortable truth is that most "scalability" problems people reach for infrastructure to solve are actually architecture problems. A tightly-coupled, synchronous system doesn't get more scalable by throwing more servers at it — it just gets more expensive in the same shape. Fixing the shape first, on cheap infrastructure, means that if I do need to scale the infrastructure later, I'm scaling something already designed to be scaled.

Takeaways

Push bursty, non-critical-path work into queues — synchronous systems fail exactly when traffic spikes matter most.
Scope real-time broadcasts — sending less is usually a bigger lever than sending faster.
Rate-limit at the worker layer, not just the request layer — your queue smooths your own bursts, not the ones you hand downstream.
Fix the architecture before reaching for infrastructure — cheap hosting with disciplined design scales further than you'd expect.

None of this required expensive infrastructure or massive traffic to learn — it just required paying attention to why a design choice mattered, not just whether it worked in the demo. That's the part that transfers, regardless of what you're building next.

I'm a final-year CS student building AI-integrated MERN stack applications. Part one of this series covers the multi-tenant RAG side of things, built for a very different kind of load — drop a comment if you've hit similar queueing or real-time tradeoffs.

Designing a Multi-Tenant RAG Pipeline: Lessons From Building an AI Chatbot Platform

Danish Raza Bangash — Fri, 10 Jul 2026 10:26:28 +0000

Most "scalable system design" articles describe theoretical systems handling millions of users that neither the writer nor the reader will ever actually build. This isn't that. This is what I actually learned architecting a multi-tenant AI chatbot builder — the kind of system where dozens of businesses share one backend, each with their own bots, knowledge bases, and messaging integrations — on a student budget, with free-tier infrastructure.

The system doesn't serve millions of requests a day. But the design questions you hit at 100 requests a day are the same ones you hit at 100,000 — you just get more room to be wrong and fix it. Here's what I learned about retrieval-augmented generation (RAG) and multi-tenancy before I actually had scale to worry about.

1. Design for tenants before you design for traffic

The chatbot builder needed to support many businesses (tenants), each with their own bots, knowledge bases, WhatsApp/Telegram connections, and usage limits — all on one shared backend.

The tempting shortcut is to bolt tenancy on later: one database, a userId field sprinkled into every collection, and hope nothing leaks. I went the other way and treated tenant isolation as a first-class part of the schema from day one — every document that mattered (bots, conversations, knowledge base chunks, subscription state) carried a tenant identifier that every query had to pass through, no exceptions.

The tradeoff: it's slower to build features early on, because you can't cut corners on "whose data is this." The payoff: when you add a new feature later, you're not doing an emergency audit of every collection to figure out where tenant data might be leaking into another tenant's view. Isolation-by-default is one of those decisions that's nearly free at the start and extremely expensive to retrofit.

2. Not every request needs the same amount of work

The chatbot's core feature is retrieval-augmented generation: a user asks a question, the system searches a knowledge base, and an LLM answers using the retrieved context. The naive version of this does one thing — embed the query, run a vector search, stuff results into a prompt — for every single request, regardless of how the question is phrased or what's actually being asked.

That's wasteful. A one-word follow-up question and a complex multi-part query don't need the same retrieval effort, but they'll cost the same if you treat them identically.

I ended up building the retrieval pipeline in tiers — cheaper, faster paths handle requests that don't need deep retrieval, and progressively heavier tiers only kick in when the query actually warrants it. In production this pipeline consistently retrieves relevant context with around 90% accuracy at roughly 780ms average latency — good enough to feel responsive in a chat interface, without paying full retrieval cost on every message.

The general lesson isn't "build a four-tier pipeline." It's: look for the cases where you're doing expensive work uniformly across requests that don't uniformly need it. That gap is usually where your easiest scalability wins live, and it's almost always cheaper than reaching for more infrastructure.

3. Your real bottleneck is probably someone else's API

The platform leans heavily on third-party APIs — Gemini for embeddings and generation, WhatsApp's Cloud API and Telegram's Bot API for messaging. These have their own rate limits, latency, and failure modes that you don't control, and at small scale it's easy to forget they're there until a burst of traffic hits one at once.

Multi-tenant WhatsApp integration made this concrete: each business brings its own WhatsApp number via Meta's Embedded Signup flow, but the app still has to be a good citizen of Meta's platform-wide rate limits across every tenant combined, not just per-tenant. That means rate limiting and backoff logic had to be designed at the application layer, not assumed away because "each tenant has their own number."

The lesson: when you sketch your architecture diagram, the boxes representing external APIs deserve as much design attention as the boxes you control. They will rate-limit you, they will have outages, and they will not always fail gracefully. Plan for it before you need to.

4. Cheap infrastructure, disciplined architecture

This system runs on free/low-tier hosting (Render), not a fleet of load-balanced servers — because that's the reality of building as a student. That constraint was useful. It forced decisions that scale regardless of the infrastructure underneath them: strict tenant isolation, tiered retrieval instead of brute-force retrieval, and rate-limit-aware integrations instead of naive ones.

The uncomfortable truth is that most "scalability" problems people reach for infrastructure to solve are actually architecture problems. Fixing the shape of the system first, on cheap infrastructure, means that if I do need to scale the infrastructure later, I'm scaling something already designed to be scaled — not retrofitting scalability onto something that was never built for it.

Takeaways

Isolate tenants at the schema level from day one — retrofitting isolation later means auditing everything you've already built.
Don't do uniform work for non-uniform requests — tiering retrieval effort to what a query actually needs is often a bigger win than adding infrastructure.
Design for the external APIs you don't control — they're often the actual bottleneck, not your own code.
Fix the architecture before reaching for infrastructure — cheap hosting with disciplined design scales further than you'd expect.

I'm a final-year CS student building AI-integrated MERN stack applications. Part two of this series covers the async and real-time side of things, built for a very different kind of load — drop a comment if you've hit similar RAG or multi-tenancy tradeoffs.

Building a Four-Tier Parallel RAG Pipeline with Gemini

Danish Raza Bangash — Tue, 07 Jul 2026 12:39:27 +0000

The Problem

When building BotForge, our AI no-code chatbot platform, we needed a retrieval system that could handle messy, real-world user queries — typos, partial phrases, semantically similar-but-differently-worded questions.

A naive vector search alone wasn't good enough. It's powerful but brittle to out-of-vocabulary terms and exact keyword lookups.

The Solution: Four-Tier Parallel Retrieval

We ran four retrieval strategies simultaneously using Promise.all\, then merged results with a weighted scoring function.

\javascript const [semanticResults, textResults, regexResults, fuzzyResults] = await Promise.all([ semanticSearch(query, embeddings), // weight 1.8x mongoFullTextSearch(query), // weight 1.5x regexKeywordSearch(query), // weight 1.0x fuzzyPerWordMatch(query), // weight 0.6x ]) \\

Tier 1: Semantic Search (1.8× weight)

Using Gemini gemini-embedding-2\ to produce 3072-dimensional vectors, we compute cosine similarity against stored document embeddings. This catches meaning — "how do I reset my login?" matches "account recovery options" even with no shared words.

Tier 2: MongoDB Full-Text Search (1.5× weight)

A native MongoDB Atlas text index for fast, exact keyword hits. Great for technical terms, product names, and precise phrases.

Tier 3: Regex Keyword Matching (1.0× weight)

Each significant word in the query is compiled to a case-insensitive regex. Catches partial matches and hyphenated variants.

Tier 4: Fuzzy Per-Word Matching (0.6× weight)

Levenshtein distance matching per query word — handles typos and misspellings like "configuraton" → "configuration".

Weighted Score Merging

Each result carries a base score from its retrieval strategy. We deduplicate by chunk ID, sum scores across strategies, and sort descending:

\javascript function mergeResults(tiers, weights) { const scoreMap = new Map() tiers.forEach((results, i) => { results.forEach(({ id, score, chunk }) => { const weighted = score * weights[i] scoreMap.set(id, { chunk, total: (scoreMap.get(id)?.total || 0) + weighted, }) }) }) return [...scoreMap.values()].sort((a, b) => b.total - a.total).slice(0, 5) } \\

Results

~90% retrieval accuracy on held-out test queries
~780ms median latency end-to-end (including embedding generation)
Robust to typos, paraphrasing, and domain-specific terminology

The parallel architecture was the key insight — running all four strategies concurrently keeps latency close to the slowest individual strategy (semantic search) rather than multiplying it.

What I'd Do Differently

Cache embeddings for frequently asked questions
Add a re-ranking step (cross-encoder) for the top 10 candidates
Explore ColBERT-style late interaction for finer-grained scoring