When we start building backend applications, we usually focus on writing APIs, connecting databases, and making everything work.
And honestly, that's enough for many small applications.
But things start getting interesting when more users come in.
Suddenly, APIs become slow. The database gets overloaded. One service going down starts affecting other services. Sometimes the same request gets processed twice.
That's when we start thinking about system design.I've been exploring different architecture concepts, and I wanted to share some of the things I think every backend developer should understand.
Not with complicated definitions, but with examples we can relate to.
1. Load Balancer — What Happens When One Server Is Not Enough?
Imagine you open an online shopping website during a big sale.
Thousands of people are trying to use it at the same time.
If all requests go to one server, that server might struggle to handle the traffic.
One solution is to run multiple instances of the application and distribute requests between them.
That's basically what a load balancer does.
Users
|
v
Load Balancer
/ | \
v v v
API 1 API 2 API 3
Instead of depending on one server, we have multiple instances handling requests.
If one instance becomes unhealthy, the load balancer can stop sending traffic to it, assuming health checks and routing are configured correctly.
Of course, running three servers isn't automatically enough. We also need to think about the database, shared state, and other dependencies.
But the basic idea is simple: don't make one server do all the work when it doesn't have to.
2. API Gateway — One Door for Your Application
Let's say you're building an application with separate services for users, orders, payments, and notifications.
Without a common entry point, the frontend might need to know where each service lives and how to communicate with it.
An API gateway can simplify that.
Frontend
|
v
API Gateway
/ | \
v v v
Users Orders Payments
Service Service Service
The frontend sends a request to the gateway, and the gateway routes it to the appropriate service.
It can also help with authentication integration, rate limits, and other shared API concerns.
Think of it like the reception desk in an office. You don't need to know where everyone sits; you explain what you need, and the receptionist directs you to the right person.
One thing I've learned is that not every application needs an API gateway. For a small application, it might just add unnecessary complexity.
3. Message Queues — Why Make Users Wait for Everything?
Imagine a customer places an order.
Your application needs to save the order, send an email, update analytics, and notify another service.
Should the customer wait until every single task finishes?
Probably not.
Some work can happen in the background.
Order Service
|
v
Save Order
|
v
Message Queue
/ \
v v
Email Worker Analytics Worker
The order service can publish a message, and other components can process it separately.
Tools like Amazon SQS, RabbitMQ, and Kafka help with different kinds of messaging requirements.
The benefit is that the user doesn't need to wait for every background task to finish.
But there's a catch: messages can sometimes be delivered more than once, consumers can fail, and processing might be delayed.
That's why retries, error handling, and idempotency matter.
4. Caching — Do We Really Need to Query the Database Every Time?
Imagine your application displays a list of product categories.
Thousands of users request the same information every day, but the categories rarely change.
Does it make sense to query the database every time?
Sometimes, no.
We can keep frequently requested data in a cache, such as Redis, and return it without making another database query.
API Request
|
v
Redis Cache
/ \
Found Not Found
| |
v v
Return Database
|
v
Store in Cache
This can reduce database load and improve response times.
But caching isn't always straightforward.
What happens when the price of a product changes? What if the cache still contains the old price?
Now we have another problem to solve: keeping cached data reasonably up to date.
Caching can be very useful, but adding it without understanding the data and its update patterns can create confusing bugs.
5. Circuit Breaker — What If Another Service Stops Responding?
Imagine your order service depends on a payment service.
The payment service suddenly becomes slow.
Your order service keeps calling it, waiting for responses, and consuming resources.
Soon, your own service might start struggling too.
A circuit breaker helps limit this situation.
Think of an electrical circuit breaker at home. When something goes wrong, it interrupts the connection to prevent further problems.
In software, a circuit breaker generally has three states:
- Closed: Requests are allowed through.
- Open: Calls are blocked temporarily because the dependency has been failing.
- Half-open: A few calls are allowed to check whether the dependency has recovered.
This doesn't fix the payment service. It helps stop that failure from causing even more trouble elsewhere.
We still need sensible timeouts, retry policies, monitoring, and proper handling of failed payments.
6. Database Replication vs Sharding — They're Not the Same Thing
As an application grows, the database can become a bottleneck.
Two terms that often come up in system design discussions are replication and sharding.
Replication means keeping copies of data on multiple database instances.
For example, a primary database might handle writes while replicas handle some read traffic.
Sharding means splitting data across multiple database partitions or servers.
Imagine a shopping application with millions of customers. Instead of storing every customer record on one database server, records can be distributed across multiple shards.
Here's the simple difference:
- Replication gives you copies of data.
- Sharding divides data across different places.
Both can help with particular scaling problems, but both add complexity.
Replicas can have delays before receiving new changes, and sharding introduces questions about how data should be distributed and queried.
So before reaching for either solution, I'd first want to understand what is actually causing the database problem.
7. Monitoring and Logs — How Do We Know What's Going Wrong?
Your application works perfectly on your laptop.
You deploy it to production, and suddenly a customer reports that placing an order takes 15 seconds.
Now what?
Is the API slow? Is the database struggling? Is another service timing out?
Without useful logs and metrics, debugging becomes guesswork.
This is why observability matters.
The three things I try to distinguish are:
- Logs: What happened?
- Metrics: How is the system performing?
- Traces: Where did a request spend its time across different services?
Tools such as CloudWatch, Grafana, and OpenTelemetry can help collect and investigate this information.
The goal isn't to collect as much data as possible. It's to collect information that helps us understand what went wrong and where to look.
One Last Thing: Don't Overcomplicate Your Architecture
Something worth remembering is that we don't need all these components from day one.
A small application with a single backend and a database might be perfectly fine.
Adding Kafka, Redis, multiple microservices, and an API gateway just because they are popular can make the application harder to maintain.
I'd rather start with a simple design, understand where the real problems are, and introduce additional components when they solve a specific problem.
For me, that's the interesting part of system design: understanding the trade-offs, not just knowing the names of different technologies.
I'm curious to hear from other backend developers:
What's one system design problem you've faced in a real project, and how did you solve it?
Top comments (0)