An API can work perfectly when I'm testing it alone.
Send one request.
It responds quickly.
Everything looks fine.
But what happens when 10 users make requests at the same time?
What about 100?
What about 1,000?
That's where load testing becomes important.
As a backend developer working with Node.js and Express, I want to know how an API behaves before real traffic exposes its limitations.
In this post, I'll walk through how I think about load testing a Node.js API running on AWS EC2.
Why load test?
Functional testing answers:
Does the API work?
Load testing asks a different question:
How does the API behave under pressure?
An API might return the correct response for a single request but struggle when concurrency increases.
For example:
1 request
↓
Everything looks good
100 concurrent requests
↓
Latency increases
500 concurrent requests
↓
Database becomes a bottleneck
1000 concurrent requests
↓
Errors / timeouts
Without load testing, I may discover these limitations only after production traffic arrives.
The setup
For a typical Node.js backend, my architecture looks like:
Client
↓
Nginx
↓
AWS EC2
↓
Node.js + Express
↓
MongoDB
The API may contain endpoints such as:
GET /api/users
GET /api/products
POST /api/orders
GET /api/orders/:id
For load testing, I don't want to test every endpoint equally.
I focus on the endpoints that are:
- Frequently used
- Business-critical
- Database-heavy
- Computationally expensive
- Likely to experience high concurrency
Tooling
There are several tools I can use for API load testing.
Some popular options include:
- k6
- Artillery
- Locust
- Apache JMeter
For a Node.js/backend workflow, k6 is particularly convenient because the test scenarios are easy to define and version-control.
The important thing isn't which tool I choose.
It's being able to reproduce the same test consistently.
A simple k6 test
A basic test could look like:
import http from "k6/http";
import { check } from "k6";
export const options = {
stages: [
{ duration: "30s", target: 10 },
{ duration: "1m", target: 50 },
{ duration: "1m", target: 100 },
{ duration: "30s", target: 0 }
]
};
export default function () {
const response = http.get(
"https://api.example.com/api/products"
);
check(response, {
"status is 200": (r) => r.status === 200
});
}
The idea is to gradually increase traffic rather than immediately sending a huge number of requests.
This gives me a better understanding of how the system behaves as load increases.
The test stages
A useful load test isn't just:
Send lots of requests
I prefer thinking in stages.
Users
│
100 │ ╭──────
│ ╱
50 │ ╭───╯
│ ╱
10 │──────╯
│
└──────────────────── Time
For example:
Stage 1 — Baseline
Start with a small number of users.
The purpose is to understand normal performance.
Stage 2 — Ramp up
Gradually increase concurrent users.
Now I'm looking for:
- Increasing latency
- CPU changes
- Memory growth
- Database pressure
- Error rates
Stage 3 — Peak load
Push the API toward the expected production workload.
This is where bottlenecks often become visible.
Stage 4 — Recovery
Reduce traffic again.
I want to see whether the system returns to its normal state.
This last step is easy to overlook.
A system that survives peak traffic but never properly recovers can still have a serious problem.
What I measure
There are several metrics I care about.
P50 latency
P50 represents the median request latency.
It tells me what a typical request looks like.
P95 latency
P95 is more interesting for production behavior.
It tells me that approximately 95% of requests are faster than that value.
If P50 looks good but P95 becomes very high, some users are experiencing significantly worse performance.
P99 latency
P99 helps expose the tail of the latency distribution.
This is especially useful when investigating occasional slow requests.
Error rate
Latency isn't the only concern.
I also want to know:
How many requests failed?
A fast API returning lots of 500 errors isn't a healthy API.
Throughput
I also want to understand how many requests the system can handle.
Conceptually:
Requests/sec
│
│ ───────
│ ───
│ ───
│ ───
│────
└──────────────────── Load
At some point, adding more traffic may no longer increase useful throughput.
That's an important signal.
Results
For a real test, I'd record the results in a table like this:
| Metric | Result |
|---|---|
| P50 latency | Measure |
| P95 latency | Measure |
| P99 latency | Measure |
| Error rate | Measure |
| Max sustained RPS | Measure |
| Peak concurrent users | Measure |
I prefer recording these numbers rather than simply saying:
"The API was fast."
Numbers make performance discussions much more useful.
Watching the EC2 instance
While the load test is running, I also want to know what is happening on the server.
For example:
top
or:
free -h
For a Node.js application running with PM2:
pm2 monit
This helps correlate API performance with system resources.
For example:
Traffic increases
↓
CPU increases
↓
Latency increases
or:
Traffic increases
↓
CPU remains reasonable
↓
Database connections increase
↓
Latency increases
Those two scenarios require completely different solutions.
The surprise: the API isn't always the bottleneck
One of the biggest things load testing can reveal is that the application server isn't necessarily the problem.
Consider this architecture:
Load
↓
EC2
↓
Node.js
↓
MongoDB
You might expect the EC2 server to reach its limit first.
But the actual bottleneck could be:
MongoDB queries
Database connections
Slow indexes
Network latency
External APIs
Large responses
Memory usage
This is why I don't look at only one metric.
I want to correlate:
API latency
+
Error rate
+
CPU
+
Memory
+
Database behavior
+
Network
A common database problem
Suppose an endpoint performs:
const users = await User.find({
status: "active"
});
With a small dataset, this might appear completely fine.
But under load, if the query returns a large number of documents, the application may spend significantly more time processing the result.
That can increase:
- Response latency
- Memory usage
- CPU usage
- Database workload
Pagination can help:
const users = await User.find({
status: "active"
})
.limit(50)
.lean();
Indexes can also be important.
For example, if the application frequently searches by:
status
email
userId
createdAt
I want to make sure the database query patterns are properly supported.
What happens when the test gets too aggressive?
A load test can eventually push the system beyond its intended capacity.
That's useful information—but it should be done carefully.
If the system reaches:
High CPU
High memory
Database saturation
High error rate
I don't immediately conclude that the server needs to be bigger.
First I ask:
What became the bottleneck?
Only then do I decide whether the solution is:
- Better application code
- Database optimization
- Caching
- Connection-pool tuning
- Horizontal scaling
- Vertical scaling
- Architecture changes
Changes I would make after testing
Depending on the results, the next step could be one or more of these.
1. Optimize database queries
Reduce unnecessary data and make frequently used queries efficient.
2. Add pagination
Avoid returning huge datasets in a single request.
3. Improve connection handling
Make sure database connections are managed efficiently under concurrency.
4. Add caching
For frequently requested data that doesn't change often, caching can reduce unnecessary database work.
5. Scale the application
If the application is genuinely CPU or memory constrained, scaling the EC2 infrastructure may be appropriate.
6. Improve observability
Load testing should ideally be combined with monitoring so that performance problems can be investigated instead of guessed.
Load testing isn't about finding one magic number
One mistake I try to avoid is thinking:
"Our API supports 500 requests per second."
That's rarely the whole story.
The real question is:
500 requests/sec
under what conditions?
For example:
- Which endpoint?
- What payload size?
- How many concurrent users?
- What database size?
- What response size?
- What error rate?
- What latency target?
A meaningful performance number always needs context.
A simple performance checklist
Before considering an API ready for significant traffic, I'd check:
[ ] Baseline performance measured
[ ] Peak expected traffic tested
[ ] P50 latency recorded
[ ] P95 latency recorded
[ ] P99 latency recorded
[ ] Error rate recorded
[ ] Maximum sustainable throughput measured
[ ] CPU monitored
[ ] Memory monitored
[ ] Database performance monitored
[ ] Slow queries investigated
[ ] Recovery after peak load tested
What I learned
Load testing changed the way I think about backend performance.
Before testing, I might ask:
"Is the API fast?"
After testing, I ask:
"How does the API behave as concurrency increases?"
That's a much better question.
A backend isn't truly production-ready just because it works with a few requests.
I want to understand how it behaves when:
Traffic increases
↓
Resources increase
↓
Latency changes
↓
Bottlenecks appear
↓
System reaches its limits
↓
System recovers
Final takeaway
Load testing isn't something I want to do only after an application starts having production problems.
It should be part of understanding the system.
For a Node.js + Express backend running on AWS EC2, I want to know:
How much traffic can it handle?
Where does it start slowing down?
What becomes the bottleneck?
Does it recover correctly?
Those answers are much more valuable than simply knowing that the API works.
Don't wait for production traffic to discover your application's limits. Test those limits before your users do.
Top comments (1)
The ramp-down stage at the end of the k6 script is a handy touch for spotting leaks.