"Add rate limiting" sounds like one task. It's actually three separate layers, each defending against a different failure mode, with different trade-offs and different amounts of trust you can place in them. I only understood the distinction properly once I had to pick a strategy for two systems solving different problems: LogicVisor (a public AI tool anyone can hit anonymously) and Titan (a payments platform where the cost of a bad actor is different from the cost of someone burning through free AI credits).
1. Client-side: UX, not security
Debouncing a search input, greying out a submit button after the first click, backing off exponentially after a 429. All of this makes an app feel considerate. None of it stops anyone. A malicious actor skips your JavaScript entirely and hits the endpoint directly with curl. Client-side limiting is worth doing (it saves you real traffic and prevents accidental double-submits), but it is not a security control. If it's the only thing standing between your API and abuse, you don't have rate limiting, you have a polite suggestion.
2. Server-side: the layer that actually protects you
This is where LogicVisor and Titan diverge, because they're not defending against the same thing.
LogicVisor: several checks before a single AI token gets spent
LogicVisor is public. Anyone gets 3 free code reviews with no signup, which means the abuse surface is wide open by design. The submission route runs a stack of checks before it ever calls Gemini or Groq, because every AI call costs real money:
// 1. Check if this exact code has already been reviewed by this model
const cachedReview = await getCachedAIReview(preferred_model.id + "-" + canonicalHash);
if (cachedReview) {
return NextResponse.json({ success: true, data: cachedReview }, { status: 201 });
}
// 2. Enforce the actual rate limit (IP + session based for anon users)
const rateLimitResult = await enforceAIRateLimit(user.id, request);
// 3. Slow down premium users intelligently instead of hard-blocking them
const throttleDelay = await getThrottleDelay(user.id);
if (throttleDelay > 0) {
await new Promise((resolve) => setTimeout(resolve, throttleDelay));
}
// 4. Under heavy load, degrade gracefully instead of rejecting outright
const shouldDegrade = await shouldGracefullyDegrade(user.id, "ai_request");
if (shouldDegrade) {
// return a basic, non-AI response instead of a 429
}
A few things worth naming separately, because they get lumped together under "rate limiting" but aren't the same mechanism:
- Abuse prevention: anonymous sessions are capped at 3 reviews, tracked by a UUID session (7-day expiry) plus a browser fingerprint (User-Agent + Accept-Language + Accept-Encoding). This is the classic server-side layer, IP and fingerprint based, closest to the "token bucket per identity" pattern.
- Cost control via deduplication: before rate limiting even runs, the code is canonicalized (AST-based) and hashed. If someone (or the same person twice) submits identical logic, they get the cached review instead of triggering a new paid AI call. This isn't rate limiting in the strict sense, it's closer to database-level duplicate detection, but it does the same job of protecting a scarce resource.
- Graceful degradation over hard rejection: instead of a flat 429 when usage spikes, authenticated users can get a stripped-down, non-AI response. It costs nothing and still gives the user something, rather than a wall.
None of this is a single algorithm out of a textbook. It's several cheap, layered checks, ordered so the most expensive resource (the AI call) is the last thing hit, not the first.
Titan: NestJS Throttler backed by Redis
Titan's rate limiting is intentionally boring by comparison, and that's the correct choice for what it is. It's @nestjs/throttler wired up as a global guard, with Redis swapped in as the storage backend instead of the default in-memory store:
ThrottlerModule.forRootAsync({
imports: [ConfigModule],
inject: [ConfigService],
useFactory: (configService: ConfigService) => ({
throttlers: [
{
ttl: 60000,
limit: 10,
},
],
storage: new ThrottlerStorageRedisService(
configService.get<string>('REDIS_URL'),
),
}),
}),
// registered alongside the JWT guard as a global APP_GUARD
{
provide: APP_GUARD,
useClass: ThrottlerGuard,
},
10 requests per 60 seconds, globally, enforced before a request even reaches a controller. No per-route tuning yet, no per-user tiers, just a flat ceiling applied everywhere via APP_GUARD.
The reason Redis matters here isn't exotic: an in-memory counter only knows about requests hitting that one process. The moment you run more than one instance behind a load balancer, in-memory rate limiting stops being true rate limiting, each instance is independently under-counting. Swapping the storage to ThrottlerStorageRedisService gives every instance a shared, consistent view of who's made how many requests, which is the actual requirement once you're not running a single box.
LogicVisor's layered, identity-aware approach makes sense for a system where the main threat is "someone is farming free AI reviews." Titan's flat, Redis-backed throttle makes sense for a system where the main threat is "someone is hammering an endpoint," and the priority is consistency across instances over nuance per user.
3. Database-level: the layer I'm deliberately not using yet
Postgres and Supabase both support rate limiting closer to the data layer (unlogged tables with triggers, PL/pgSQL token bucket functions, TTL-based counters in Mongo). I don't use this in either project, and I don't think I should yet. It's the right call for protecting specific high-value operations (financial writes, heavy analytical queries) where you need the database itself to be the source of truth and can't tolerate a race between the app layer and the DB. LogicVisor's canonical-hash cache lookup is adjacent to this idea (checking the data layer before doing expensive work), but it's a caching pattern, not a rate limiter, and I'd rather keep that distinction honest than dress it up as something it isn't.
What actually mattered, in order
- Client-side limiting is UX. Treat it as UX. It's not a line item in your security review.
- Server-side is where the real decision lives, and the right shape of it depends on what you're protecting: identity-based abuse (LogicVisor) versus flat request volume across a distributed system (Titan). They are not the same problem and don't need the same solution.
- The most expensive resource should be the last thing your code touches. LogicVisor checks a cache, then a rate limit, then a throttle delay, before it ever calls an LLM. Ordering is a rate limiting decision too, even when it's not framed as one.
- Database-level limiting is a tool for a specific job, not a default. I'm not reaching for it until there's an operation that actually needs that guarantee.
Cover photo by HsinKai Tai on Unsplash
Top comments (1)
This separation is useful, especially the point that ordering protects the expensive resource. I’d add a fourth control that often gets hidden inside “rate limiting”: concurrency.
A shared Redis counter can correctly enforce 10 requests/minute and still allow all 10 long-running AI or payment requests to execute at once. That protects request volume, but not connection pools, worker slots, provider concurrency, or worst-case spend. For expensive paths I’d combine the arrival-rate limit with an in-flight semaphore and a per-request token/cost reservation, then reconcile the reservation to actual usage.
Those controls also need different overload behavior: queue briefly when latency is tolerable, degrade when a cheaper path is safe, and reject before reserving scarce downstream capacity otherwise. The operational dashboard should show cache-hit rate, arrivals, admitted requests, in-flight work, queued work, and reserved versus actual cost separately—otherwise a healthy 429 rate can mask a saturated dependency.