Scaling Shadow Bans: From Naive SQL to Bloom Filters

#systemdesign #softwareengineering #database #redis

TL;DR

Implementing a shadow ban (like YouTube's "hide user from channel") creates a unique, per-user view of data. While a simple SQL query works at first, scaling this feature requires moving away from heavy read-time exclusions toward write-time flagging, asynchronous event queues, or memory-efficient probabilistic data structures like Bloom filters.

I've been looking into YouTube's "hide user from channel" feature recently, and it's a fascinating piece of product design. The banned user can still see all of their own comments, but to everyone else, they've vanished. They're essentially shouting into an empty room. While this is great for dealing with trolls without tipping them off, it's an absolute headache to build at scale. Let's look at why it breaks traditional database patterns and how I would actually go about building it.

Why is shadow banning hard to scale in SQL databases?

Naive SQL approaches rely on checking every comment against a massive list of banned users during read-time. As a channel's ban list grows into the thousands, these heavy exclusion filters degrade query performance, turning a simple comment fetch into a database bottleneck.

If I try to implement this using a naive SQL query, I'm going to run into performance issues fast. In a standard setup, you might try to query comments while filtering out banned authors on the fly:

SELECT * FROM comments 
WHERE channel_id = :channel_id 
  AND (author_id = :viewer_id 
       OR author_id NOT IN (
           SELECT banned_user_id 
           FROM channel_bans 
           WHERE channel_id = :channel_id
       ));

This works fine when a channel has three trolls. But imagine a major creator with 50,000 banned accounts. Running a subquery or a massive NOT IN join on every single page load of a highly active comment section is going to melt your database CPU. You are trying to generate a completely customized view of the data for every single visitor, which doesn't scale.

How can we optimize shadow ban checks at write time?

Instead of evaluating bans during expensive read-time queries, you can stamp a visibility flag directly onto the comment when it is written. The database then only needs to check a simple boolean flag at read time, keeping queries incredibly fast.

By checking the ban status at write time (when the comment is created), we shift the computational weight from the high-traffic read path to the lower-traffic write path.

Strategy	Read-Time Filtering (`NOT IN`)	Write-Time Flagging
Query Performance	Slows down exponentially as ban lists grow	`O(1)` read overhead (simple boolean check)
Write Path Complexity	Extremely low	Medium (requires a ban check on insert)
State Reversal (Unbanning)	Instantaneous	High overhead (requires historical updates)

What happens when you unban a user?

Unbanning a user requires you to flip the visibility flag on all of their historical comments. Doing this in a single, synchronous batch write will lock your database tables, meaning you must process these state changes asynchronously using event-driven background workers.

If a channel owner decides to forgive a user, you can't just run a massive, synchronous UPDATE query on your live comments table. That's a great way to lock tables, trigger API timeouts, and set off your team's pager alerts.

Instead, you have to handle this in the background. My approach would be to emit an unban_user event to a message broker like RabbitMQ or SQS. A pool of background workers then picks up this event and updates the comments in small, throttled batches, protecting your primary database from falling over.

Can probabilistic data structures solve the scaling issue?

Yes, probabilistic data structures like Bloom filters can verify if a user is banned in constant time with near-zero database overhead. Because Bloom filters have zero false negatives, you can immediately trust them if they say a user is not banned.

If you want to bypass the database entirely on the read path, a Bloom filter in Redis is probably your best bet. When fetching comments, you quickly query the Bloom filter. If it returns "not in set," you instantly know the user is not banned and display the comment. If it returns "is in set" (allowing for a tiny, configurable margin of false positives), only then do you query the primary database to confirm the exact ban status.

FAQ

How does a Bloom filter handle unbanning users?

Standard Bloom filters don't support deleting items because clearing bits could accidentally unban other users. If your platform permits frequent unbanning, you'll need to use a Cuckoo filter instead, or periodically rebuild your Bloom filter from the source-of-truth database in the background.

Why not just delete a banned user's comments?

If you delete a troll's comments, they will immediately realize they have been blocked because their posts will disappear from their own screen. The goal of a shadow ban is to keep them screaming into the void so they don't simply log out and create a new account.

Is Redis a good tool for managing shadow ban states?

Yes, Redis is excellent for this. I recommend using Redis sets or the native Bloom filter module to handle high-throughput read paths with sub-millisecond latency, keeping the load off your primary database.

DEV Community