I have worked on a few "high traffic" applications in my time. Of course what is considered high traffic for some would be low for others.
In my case I was working on systems that needed to cope with around 1,000 requests per second (RPS), while staying resilient and performant. Depending on the application, 1,000 RPS isn't actually that high. As I demonstrated last year, even my underpowered Raspberry Pi 2 was capable of 1,000 RPS with a simple CPU and Memory bound API, although it started to struggle with 2,000 RPS.
In every system there will be a bottleneck, which is usually caused by one of the following:
- CPU
- Memory
- Disk I/O
- Network Bandwidth
Like a physical bottleneck, if one of these gets maxed out, then it doesn't matter how much data is coming in, it isn't going to flow through the system any faster. (Side-note: I wonder what the technical equivalent of a strawpedo would be?)
As your system gets more complex, and you start adding in databases and other components, the chances of hitting a bottleneck somewhere increases.
There are generally 2 ways to cope with bottlenecks, vertical scaling and horizontal scaling.
Vertical Scaling #
In the case of my Raspberry Pi 2 with 900 MHz CPU and 1GB RAM, it is pretty clear that a more powerful system would have been able to cope with a higher load.
Vertical scaling is essentially throwing money at the problem and buying a more powerful machine. Eventually though you will likely hit another bottleneck, either with the system or your bank balance.
Horizontal Scaling #
The alternative is to scale horizontally. If a Raspberry Pi 2 can cope with 1,000 RPS then 2 Raspberry Pi 2s can cope with 2,000 RPS. It is generally cheaper to buy multiple smaller machines and stick a load balancer in front, than it is to buy one more powerful machine.
If your load requirements increase you can always just buy additional cheap servers to cope with the load instead of having to upgrade to a more powerful and expensive machine.
Synchronous vs Asynchronous Systems #
With a synchronous system you need to provide the answer to the users query in real time. Just like my age API that I used for load testing, the user is waiting for an answer. In these cases, you can use both vertical and horizontal scaling to increase the load capacity of your system.
Not all systems however require an immediate response. The classic example is sending an email. When you send an email, the server immediately gets back to you with a response to say that the email has been sent. However, in reality your message has been added to a queue to be sent at some point in the near future.
If the user doesn't need an immediate response, you can put your requests on a queue and then deal with them later.
This is particularly helpful if you are finding that your database is becoming a bottleneck. Requests can be queued up and dealt with as quick as they can be processed. Provided that the increased load isn't constant the database will eventually catch up.
It also adds some resiliency to your system. Even if the database goes down you haven't lost any of the messages that have come in.
Of course, you then need to deal with the problem of eventual consistency. What do you do if a request comes in that needs data that is still sitting on the queue? This generally involves some form of caching, so the data is still available when it needs it.
Is all this overkill? #
Software developers have a tendency to over-engineer systems. If you have a system that is considered "high traffic" then chances are you will have to worry about scalability at some point.
For many though, we have a tendency to put these scalability practices in place even when we are only getting a few requests per second.
Every year computers are getting more and more powerful. I recently upgraded my 8-year-old Intel based MacBook Pro for one of the Apple Silicon versions, and wow the difference is night and day.
At what point will these scalability techniques such as queues and horizontal scaling become obsolete?
When your personal laptop can comfortably cope with 100,000 RPS will there still be a need for all this extra work?
❤️ Picks of the Week #
📝 Article - AI Poetry Camera? Seriously? - AI is being put into everything recently and in most cases I am not a fan. It is fun however seeing it be used for ridiculous things like this.
⚽ Ball - GitHub - nate-parrott/ball - Speaking of ridiculous things. It is this kind of silliness that I think everyone needs at some point in their day.
🛠️ Tool - Figma Slides - I have yet had a job where I haven't had to make or sit through a slideshow presentation. Most of the tools are pretty boring and produce the same boring slides. If anyone can make a less boring slides program it is probably Figma.
📝 Article - Justin Jaffray on approximating domain knowledge - Understanding how to program is only half the challenge of being a developer. Domain knowledge is equally important. The domain knowledge differentiator is often why some find it easier to get jobs than others.
🦠 Security - Polyfill supply chain attack hits 100K+ sites - If you have ever used pollyfill.js before you should read this.
📝 Article - dropofahat.zone - Using AI to drop hats on people from a window in New York, because why not?
🛠️ Tool - GitHub - Lissy93/web-check - This looks like an interesting tool to analyse your website for vulnerabilities.
📝 Article - My Windows Computer Just Doesn't Feel Like Mine Anymore - I have been a Mac user for 8 years now and even before that I tended to use Linux more than Windows. I really don't understand companies who think is okay to make you pay for something and show you adverts anyway.
💬 Quote of the Week #
None of the 2,000 books picking apart Buffett’s success are titled This Guy Has Been Investing Consistently for Three-Quarters of a Century. But we know that’s the key to the majority of his success. It’s just hard to wrap your head around that math because it’s not intuitive.
From "The Psychology of Money" by Morgan Housel.
Top comments (0)