Have You Ever Thrown Threads at a Problem?
Well, I did! Five threads later, my C# file search and indexing code was crumbling. Way worse than the non-threaded version. Nearly broke me. Like a classic noob, I blamed C#.
It took wrestling an FFmpeg worker node to teach me the difference between a text file and an algorithm.
FFmpeg is weird because it lives in the valley between CPU and I/O:
video -> [READ: I/O][PROCESS: CPU][WRITE: I/O] -> output
See the trick? One phase is I/O‑bound, the other CPU‑bound, now that distinction is step one in mastering scale.
CPU‑bound: All the heavy lifting happens inside your processor. Every extra cycle competes for the same CPU time.
I/O‑bound: Your code is waiting on external entities (files, networks, that one slow AWS API).
Why the Distinction Matters
Let’s use one of my new favorite analogies from Data-Oriented Design:
the restaurant
Your Computer is a Kitchen
- Main thread = Head chef
- Cores = Workstations
- Programming language = Knives/utensils
- I/O = Delivery trucks outside
SCALING LAWS OF THE KITCHEN:
- More trucks ≠ faster chopping (adding I/O won’t fix CPU limits)
- Better knives ≠ faster deliveries (optimizing code won’t fix slow disks, networks, databases)
You need to know the difference between a txt file and an algorithm.
Scaling an Algorithm (CPU‑bound)
Algorithms live in the CPU. In a single‑threaded program, nothing else runs until the algorithm finishes:
function fib(n) {
if (n < 2) return n;
return fib(n - 1) + fib(n - 2);
}
fib(50)
// nothing here will run
You need sous‑chefs (threads) so the head chef can delegate and juggle other tasks. more sous‑chefs means you must scale your kitchen too. Add more RAM, more cores. This is vertical scaling.
But as the kitchen grows, you’ll need more trucks to supply ingredients! So now you spin up clusters of processes. Instead of one Node.js process, run multiple isolated ones. Distribute the weight.
Here’s the fun part: adding a process creates a fresh event loop. Now you’ve got multiple trucks fetching ingredients on the same machine.
This is mini horizontal scaling. Still one machine, but you’re thinking like a distributed system.
Browsers do this brilliantly with tabs. If one tab crashes, no others die—that’s a cluster of processes.
But eventually, you’ll need to go beyond 1 machine
True Horizontal Scaling
- Build new restaurants (servers)
- Replicate kitchen blueprints (containers)
- Deploy a global delivery network (CDNs/cloud)
This is the rabbit hole of distributed systems.
But it all loops back to one question: What’s the difference between a txt file and an algorithm?
You can find me on x.
More Content
Free:
payhip.com
Top comments (4)
Pretty cool, Ive smashed my head on this stuff too and it always gets messy fast
oo facts. very messy! it's so easy to "shoot your leg off". years in I still feel like a noob. every problem feels unique for some reason
been there way too many times tbh, scaling stuff always messes with my head at first - you ever hit that point where it feels like no matter what you tweak, something else gets slow?
LOL i know it too well. I haven't touched my C# project in over a year 😔. but one advice I got since starting a C++ journey, is using a GUI profiler. it's so easy to see bottlenecks.