API Throttling 🏎️

Throttle in a car

The throttle (aka accelerator pedal) controls how much fuel enters the engine.
More fuel → more power → more speed.

Push the pedal, the car zooms. Release it, the car eases up.

API

or Application Programming Interface
is the middleman between user and application. Every tap, swipe, or click you make sends a request through an API.

Now imagine millions of users doing that all at once.
Boom 💥 — a flood of requests. Servers can get overwhelmed.

Throttling is how we control that flood. It sets limits on how many requests can be sent over a period — just like how a car throttle limits speed.

Why throttling

If an app has large user-base, users with blazing fast connection could hog all the server bandwidth.

Slower users might be refrained from usage (talking legally haha).

Heard of DDos attack right? The one where person D sends tsunami of requests to crash the app.

How-to throttle

Leaky bucket

Requests filling a bucket at random speeds, but leaking (processing) at a fixed rate.

if queue (bucket) is full
  "429 Too Many Requests";
else 
  queue.push(request);

Issue: starvation; newer requests get dumped if current processes are long-running tasks.

Fixed window

Requests are counted in fixed intervals (like per second/minute).

while (each interval) {
  process min (incoming_requests, requests_per_interval);
}

Issue: bursts; too many users at the start of the window can overwhelm the system before the second is up.

Sliding window

Smarter cousin of fixed window. It smooths out request bursts by calculating usage over a rolling time window.

On each request:
  if user_requests_in_last_x_seconds < limit
    process user_requests_in_last_x_seconds;
  else "429";

Like checking speed over the last interval instead of just the current one.

Misc.

1. Retry strategy - don't keep honking, wait a little longer after each response failure
2. Many APIs tell a user's closeness to rate limit, use them
3. 429 = server saying Have a Break, Have a KitKat