A query that runs in a few milliseconds on your laptop against a thousand rows can quietly become the thing that takes down production against ten million. This is one of the most common ways systems fail, and it's insidious precisely because everything looks fine right up until it doesn't. The code was tested. It worked. It just never met a table that had grown up.
The trap is that performance problems are usually invisible at small scale. A missing index doesn't matter when the database can scan the whole table in an instant. A query that touches more data than it needs is fine when there's barely any data. So the code ships, works beautifully for months, and the table steadily grows in the background. Then one day it crosses a threshold, the query that was always fine starts taking seconds, and everything that depends on it begins to back up.
What makes this hard is that nothing changed at the moment it broke. No deploy, no config change, no obvious cause. The query is the same one that's been running happily all along. All that happened is the data caught up with a weakness that was there from the first day. You're not debugging a new bug. You're meeting an old one that finally had enough rows to reveal itself.
The habit that saves you is thinking about scale while you write the query, not after it hurts. Ask what this does when the table is a hundred times bigger. Does it use an index, or scan everything? Does it pull back more rows than it needs? Does it grow linearly with the data, or worse? You don't have to prematurely optimize everything, but you do have to know which of your queries will fall apart as the data grows, because those are your future outages hiding in plain sight.
The systems that scale gracefully aren't the ones that got lucky. They're the ones built by people who assumed the data would grow, because it always does. Test against realistic volumes, watch how your slow queries trend over time, and remember that "fast enough" today is a statement about today's row count, not a permanent property of your code.
– Serguey Shinder
Top comments (0)