Most of us have heard the term performance testing, whether before releasing a big new feature, launching a new application or simply checking whether an application can handle a specific load.
Performance tests help us understand how our application behaves under load and whether it can handle the expected pressure. However, they need to be designed properly. Otherwise, the results can be misleading and lead us to the wrong conclusions.
Properly designed performance tests, on the other hand, can help us identify bottlenecks, understand the limits of our application and give us more confidence before a release.
In this article, we will first look at the basic concepts and different types of performance tests. Then we will move to the more practical part: how to design realistic test scenarios, how to determine the load our application should handle, how to deal with external services and how closely our test environment should match production.
The goal is not just to generate a large number of requests, but to create performance tests that actually tell us something useful about how our application will behave in the real world.
Table of Contents
- Basic Definition
- How to Properly Design Performance Tests
- Performance Testing and External Services
- Environment Configuration
- Metrics and Results
- Summary
So, let's start with the basic definition and different types of performance tests.
Basic Definition
We can summarize performance testing as a non-functional software testing method that evaluates an application's speed, stability, scalability and responsiveness under a specific workload.
To simplify it, let's use an API as an example. We prepare tests that call the endpoints we want to test under load, usually in a specific order that represents how the application is actually used. The main idea behind performance testing is to avoid releasing an application that is not prepared for the expected load.
Poor performance can lead to production outages, slow response times, or situations where the application is unable to process a job within the required time. All of these problems can eventually lead to lost customers, lost revenue, or damage to the product's reputation.
We can divide performance tests into different types depending on their configuration, purpose, and the metrics we want to observe.
Types of Tests
I think the picture says it all, but let's briefly describe each type:
Load tests verify how the application behaves under an expected or normal level of traffic. The goal is usually to confirm that the system can handle the required number of users or requests while keeping acceptable response times, error rates, and resource usage.
Stress tests push the application beyond its expected limits to find out where it starts to degrade or fail. They help us identify the maximum capacity of the system and observe how it behaves when resources such as CPU, memory, database connections, or threads become exhausted.
Endurance (Soak) tests run the application under sustained load for a longer period of time. Their purpose is to uncover problems that may not appear during shorter tests, such as memory leaks, connection leaks, resource exhaustion, or performance degradation over time.
Spike (Peak) tests simulate a sudden and significant increase in traffic. They help us verify how the application reacts to rapid changes in load and whether it can recover once the traffic returns to normal.
Volume tests focus on how the application behaves when it needs to process or work with a large amount of data. For example, we may test how database queries, imports, exports, or batch operations behave when the data volume is much larger than usual.
Scalability tests verify how the application's performance changes when we increase the workload and add more resources. The goal is to understand whether the system can scale efficiently, for example by adding more application instances, CPU, memory, or database capacity.
You don't necessarily need to implement a completely different test for each type. In many cases, you can reuse the same performance test scenario and change its configuration depending on what you want to measure, for example, the number of concurrent users, duration, request rate or workload pattern. You then observe different metrics depending on the goal of the test.
Not every application needs every type of performance test either. Some applications may mainly need load and stress tests, while others may benefit more from load and spike tests. It really depends on the application's requirements and, more importantly, on how users actually use it.
How to Properly Design Performance Tests
Properly designed performance tests are very important because, most of the time, we don't want to simply fire requests at random endpoints. That usually doesn't tell us much about the real behavior of the application or where its bottlenecks are.
What has worked well for me over time is identifying typical user behavior and simulating it in performance tests. For example, imagine an e-commerce application with an ordering system.
A typical user might:
- Search for a few products.
- Add products to the shopping cart.
- Go through the checkout process.
- Complete the payment.
Instead of testing each endpoint in isolation, I would design a performance test that simulates this whole flow and calls the same endpoints that the frontend normally calls.
Another example could be a business application with multiple types of users, where each user type has different permissions and uses the application differently. In that case, we can design a typical workflow for each user type and call the API endpoints in the same order that the frontend does. The benefit of this approach is that we simulate realistic user behavior. We can then change the test configuration: for example, by increasing the number of concurrent users, while keeping the same realistic workflows.
Another example could be API-to-API communication. Imagine that we have a POST endpoint followed by a GET endpoint. In that case, we can simulate different input parameters and request patterns under a specific load and observe how the system behaves.
Let's say we have designed our user paths. There are still a few important things we should consider when building the actual test flow. The image below summarizes some of the key ideas:
- Real users don't click as fast as a performance test can send requests, so we should introduce realistic think time between operations.
- Not every user follows the same path. In an e-commerce application, some users complete an order, while others only browse products or add items to the cart and return later. We should therefore simulate multiple user paths with different behavior.
- Users usually don't all connect at exactly the same moment, so for a normal load test we should ramp up the load gradually.
A realistic user journey tells us what to test. The next question is how much load the system needs to handle.
Defining the Expected Load
Designing a realistic workload is only one part of the job. Before running the test, we also need to define what a successful result actually looks like. Not every application needs to handle 1,000 requests per second. Every system has its own expected workload and performance requirements.
For example:
Expected load: 150 RPS
p95 < 400 ms
p99 < 1 s
Error rate < 0.5%
Required throughput maintained
No continuously growing queues/connections
So how can we determine those numbers?
Let's return to our e-commerce example. Many applications have periods when traffic is significantly higher than usual. For an e-commerce application, this could be Christmas, Black Friday, or another major sales event. If the company already has good observability, historical production metrics can give us a useful starting point. We can check tools such as Grafana and determine the peak request rate, number of concurrent users, order volume, CPU usage, memory consumption, and other relevant metrics. We can then estimate future load. For example, if we expect traffic to grow by 10% next year, we may decide to test the system with an additional safety margin above that expected load.
Another way to estimate the required load is to start from a business requirement. For example, let's say the business expects our e-commerce application to process 10,000 orders during a two-hour peak period. We can look at a typical order flow and estimate how many HTTP requests are generated while a user searches for products, adds or removes items from the cart, goes through checkout and completes the order. If one completed order generates, for example 20 requests on average, then 10,000 orders mean roughly 200,000 requests during those two hours.
From there, we can calculate the average request rate:
200,000 requests / 7,200 seconds ≈ 28 requests per second
This gives us an average of roughly 28 RPS. It does not mean that 28 RPS should automatically become our test target. This calculation only estimates the traffic generated by completed order flows. Real application traffic will usually be higher because browsing, abandoned carts, background requests and other user journeys also contribute to the total workload. Real traffic is also rarely distributed evenly, so we should account for shorter traffic peaks and add an appropriate safety margin.
This approach gives us another way to estimate the required load when reliable production metrics are not available, showing that we can derive performance targets from both technical data and business requirements.
Performance Testing and External Services
External services require special attention during performance testing.
In some cases, we can mock an external service because we simply don't need or want to test it for several reasons. A good example is a paid API behind our endpoint, such as an LLM API or another paid third-party service. Cost is a valid concern because during performance testing our application may generate a large number of requests in a relatively short period of time.
Another scenario is when we simply don't need to call the external API at all. For example, it could be a simple reference-data API or a payment provider whose performance is outside the scope of our test. Instead, we can replace it with a controlled dependency using a tool such as WireMock and return the responses we expect during the test. This allows us to focus on the performance of our own application without having the external service's performance, rate limits or availability influence the results and make it harder to identify the actual bottleneck.
On the other hand, if communication with the external service is an important part of the system's real-world performance, we may want to test it separately or include it in the performance test.
So, whether we mock an external service should depend on the behavior we want to simulate and what exactly we want to measure.
Environment Configuration
Environment configuration is another important part of performance testing because, ideally, we want our performance test environment to be as close to production as possible. If the environment differs significantly from production, the results can become misleading, especially when we want to estimate how the application will behave under real production load.
So, what should we configure?
First, the application itself should use the same or very similar configuration as production. The server or cluster where the API is running should also have comparable CPU, memory, scaling rules and other resource limits.
The same applies to the database. It is not only about using similar database resources, though. We should also avoid testing against an almost empty database. The amount and distribution of data can have a significant impact on performance. Under high load, CRUD operations, joins, filtering, and sorting can behave very differently when tables contain millions of rows compared to just a few test records. Indexes, query execution plans, statistics and caching can all behave differently depending on the size and structure of the data.
For that reason, the test database should contain a realistic amount of representative data whenever possible.
We should also pay attention to caching configuration, for example Redis or in-memory caching. A different cache configuration, or testing with a permanently warm cache, can produce results that do not represent real production behavior.
Network conditions are another important factor. For example, if an external service is mocked locally, requests may complete almost instantly, while the real service in production could add tens or hundreds of milliseconds of network latency. Depending on what we want to measure, we may need to simulate this latency to get more realistic results.
And finally, there is the load generator itself. This one is easy to forget. The machine generating the load must have enough CPU, memory, network capacity and available connections to generate the required workload. Otherwise, the load generator can become the bottleneck instead of the application we are actually testing.
Metrics and Results
Once we run our performance tests, we need to evaluate the results and usually create some form of report. The metrics we focus on depend on the type of test, because different test types answer different questions.
Let's return to our e-commerce application and say that we decided to run both load and stress tests.
For a load test, we already know the expected workload, for example a specific number of requests per second or concurrent users. Now we want to verify that the application can handle this load while maintaining acceptable performance.
Some of the most important metrics are:
- Response time, especially percentiles such as p95 or p99.
- Error rate — what percentage of requests failed.
- Throughput — whether the application actually processed the expected number of requests.
- Resource usage — CPU, memory, database connections, connection pools and other relevant resources.
For example, if the p95 response time of a particular endpoint is much higher than expected, we can start investigating where the bottleneck is. Similarly, if the error rate exceeds our acceptable limit, we need to find out which requests are failing and why.
For a stress test, our goal is slightly different. We intentionally increase the load beyond the expected level and try to find the point where the application starts to degrade. Here, we watch how response times and error rates change as the load increases, together with CPU, memory, database connections, queues and other limited resources. We are looking for the point where the system becomes saturated, starts producing too many errors or can no longer maintain the required throughput.
It is also useful to observe how the application behaves after the load decreases. A system that slows down under extreme load but recovers afterwards behaves very differently from one that remains stuck or requires a restart.
The image below shows the metrics we monitored in our examples and highlights why we need to look at multiple graphs together to identify a specific bottleneck or saturation point.
The final report should therefore not contain only a single number such as average response time. It should connect the generated workload with latency, throughput, errors and resource utilization so that we can understand not only whether the application failed to meet our expectations, but also why.
Summary
In this article, we covered some performance testing basics and then focused on how to design tests for real-world scenarios.
The key is to understand how the application is actually used, prepare a realistic test environment, generate a representative workload and monitor the right metrics. When these parts are designed properly, performance tests can give us useful information about bottlenecks, system limits and overall application behavior under load.
There are already many great articles explaining performance testing theory and individual test types. My goal here was not to repeat all of that, but to look at performance testing from a more practical point of view and show how I approach designing realistic tests.
In the end, a performance test is only useful when its workload, environment and metrics are realistic enough to make the results meaningful.
Top comments (74)
I usually follow the 80/20 rule: 80% of traffic arrives within 20% of the time window. I use this to design my load test scenarios. Writing the test code is the easiest part. The most annoying work is resource coordination before the load test and root-cause analysis afterward — that stuff is brutal 😂
Interesting, I usually approach it more from the business side, so I calculate the expected traffic and time window based on things like the number of orders in e-commerce or the expected number of operations within a specific period.
And yes, I agree, writing the tests is usually the easiest part. 😄
Btw, what do you use for creating and executing the tests? Most of the time, I’ve been pretty satisfied with JMeter.
Back in the day I used LoadRunner — it was so heavy and costly. Then I moved to JMeter. Once AI took off, I started having it write Python load test scripts with Locust, and it works surprisingly well.
Hmm, sounds interesting. I’ll check it out.
One benefit of JMeter is that testers don’t really need to know how to program, since most of it is GUI-based. But you’re right in the AI age it can feel a bit heavy.
Last time I had to edit something, I used Copilot on our ~35k-line JMeter XML file and it ate through a lot of tokens. I actually had to change my Copilot settings so it wouldn’t burn all my monthly credits. 😅
JMeter’s XML is its biggest flaw, lol 😂
Yeah, fair point. 😄 To be honest, I never really needed to touch the raw JMeter XML before AI, so it never bothered me much.
It’s not completely unreadable, it just gets insanely verbose once the test grows, which becomes a bit of a bottleneck in the AI age 🤣
That verbosity is the real killer. Way too many redundant tags, and my tokens cry 😂
And the wallet too 🤣🤣🤣
Thanks, this was genuinely interesting! I especially liked the point about making tests reflect real user behavior instead of just throwing a huge number of requests at the API.
At work, we've generated so many tests with AI across different projects that it's honestly getting a little scary 😄 I wouldn't be surprised if somewhere there's already a test checking whether the tests themselves can actually run. 😂
Thanks, glad you liked it!
Yeah, realistic user behavior vs. just throwing requests at the API is really important and in my opinion a lot of performance-testing articles don’t highlight it enough.
That sounds funny and scary at the same time. 😂 But now I’m curious with so many AI-generated tests, who is actually responsible for them? Testers or developers? I’m asking because I’m also curious about the code quality and long-term maintainability. From my experience, a lot of testers can vibe-code tests that work initially, but maintaining them later can become a real problem.
I guess the responsible person is whoever still remembers where the tests are 😄
The good news is that every company will probably need at least one person who remembers that. So AI won't replace all of us after all 😂
Now it’s even scarier. 🤣
😂😂😂
Great post about API performance testing! 🤗 First, explaining the basics and later explaining an example test using a user simulation helped me understand it better. Also, the images helped me understand it too. Every developer using APIs should read this post! 💯
Oh, thanks! I really appreciate that! 😊
I was trying to show how to think when designing performance tests, because some of the things I describe in the article I had to learn the hard way. 😅
Ah! Because you experienced it the hard way, now you can avoid the bug beforehand. Nice tackle! 💪
The load generator bottleneck is the one that catches everyone sooner or later. We hit this in a monitoring stack where the "test" ran as a sidecar on the same box as the thing being measured — the generator and the target starved each other for CPU, and the p95 that came out told us more about the scheduler than the service.
The part I'd underline is defining the acceptable result before running the test. We have a watchdog that asserts against concrete SLOs (thresholds, error rate, max consecutive misses with backoff) rather than p95-vs-vibes. When the numbers disagree with the SLO you wrote down beforehand, you actually investigate. When you only have "it seems slow," everything looks slow.
Yeah, running the generator on the same box as the target is a great example of how easily the test itself can influence the result.
And I agree, defining the acceptable result before the test is probably one of the most important parts. Otherwise, you can easily focus on numbers that don’t really matter. It’s a bit like shooting in the dark. 😄
"Shooting in the dark" is exactly the right phrase — the acceptance value is what forces you to say before the run what an acceptable result is. We now keep it in the task contract rather than in the summary, so the number you show at the end has to match the promise you made at the start. And for the generator: we treat "the test box itself got slower" as a valid finding of its own, not an excuse to rerun. If the tooling is drifting, that's a fact about the loop, and it's cheaper to know it early than to discover it in a baseline that suddenly no longer matches.
Exactly. We define what success looks like before the test runs and then compare the actual results against those criteria.
When the load generator itself starts degrading, for example when it can no longer maintain the requested RPS, it becomes a bit more situational. Depending on the circumstances, we may stop and rerun with more resources, or let the test finish and use the run for diagnostics. But once the generator becomes the bottleneck, I wouldn’t treat that run as a reliable performance result for the application itself.
That distinction is the one I'd codify: the load generator gets its own health gate. If achieved RPS sits below ~95% of target for more than a few seconds, the run is void for pass/fail and only useful as a diagnostic — otherwise you're quietly measuring the harness's saturation point instead of the app. Predefined acceptance criteria plus that abort rule together kill most of the shooting-in-the-dark problem.
Great breakdown on designing realistic user journeys rather than just hammering a single
/pingendpoint!One critical addition to the Metrics section: Always avoid reporting Mean (Average) Latency.
Averages hide tail latency outliers completely. If 95 out of 100 requests return in 10ms, but 5 requests hang for 10 seconds (due to GC pauses or DB thread pool exhaustion), your average latency will sit at a misleading ~500ms.
In production microservices:
Also, when calculating throughput vs. response time, watch out for the Coordinated Omission problem (a term coined by Gil Tene). If your test harness blocks on a slow response before sending the next request, your load generator is accidentally hiding the queueing delay that real-world concurrent users would actually experience.
Thanks, I appreciate it! And yes, relying on average latency alone can definitely hide important problems, that’s why I prefer looking at p95/p99 together with the other metrics.
Coordinated Omission is actually a new concept to me. That’s a really interesting problem and something I hadn’t considered before. Thanks for bringing it up! I’ll definitely dig into it more and I might update the article itself.
Daniel, the external services section is where I would add one layer. LLM calls do not have fixed latency like a normal REST call.
A prompt can return in 200ms or take 8 seconds, depending on output length and provider load. A single mock delay hides that variance completely.
I test with a latency distribution instead of one fixed number, so the p99 still means something once an LLM sits behind the endpoint.
That’s a really good point. With LLM calls, simulating one fixed latency would definitely hide a big part of the real behavior. Using a latency distribution makes much more sense, especially when we care about p95/p99. Thanks for adding this.
LLM endpoints would probably deserve their own performance-testing section. I’ve worked with AI endpoints and their unpredictable latency, but I don’t have enough hands-on experience with performance testing them in real production yet, so anything deeper from me would mostly be theory or observations from hobby projects.
Great article — especially the focus on realistic user flows instead of isolated endpoints helped me identify a few gaps in my own approach.
My existing suite already measures p95/p99, throughput, error rates, warm-up, and stepped load. What I’m taking from this article is the need for role-based journey profiles, realistic think time, a mix of different user paths, and load targets derived more clearly from business requirements or production telemetry. I also want to correlate results more closely with resources such as database connections, queues, and worker capacity.
Thank you for this practical perspective — connecting test design, environment, and metrics is what makes performance testing genuinely useful.
Thanks, I really appreciate this!
I’m genuinely happy that the article gave you a few concrete ideas to improve your existing suite. Your current setup already sounds solid, so adding more realistic user journeys should make the results even more useful.
Really like the focus on realistic workflows rather than testing endpoints in isolation. I’m exploring something related with Kaktoos, an open-source tool for API verification and engineering change impact. The workflow-based approach is very much aligned with what I’m building
Thanks! Yeah, the workflow-based approach is really important if you want the results to reflect actual usage.
Really enjoyed this perspective on API performance testing. One thing that stands out is how easy it is to focus on sending more requests and forget that real users don’t interact with APIs as isolated endpoints. They follow journeys, create different workloads, and experience the system as a whole.
The idea of designing tests around realistic user behavior is something more teams should adopt. A test that only measures numbers but ignores actual usage patterns can give a false sense of confidence.
Performance testing is not just about finding the maximum load an API can handle, it’s about understanding how the system behaves under real-world conditions and where improvements will have the biggest impact.
Great article and a good reminder that quality testing starts with understanding users, not just tools and metrics.
Thanks, glad you liked it!
Exactly, the tools and metrics are important, but if the workload doesn’t reflect real user behavior, the test results won’t tell us much about how the system will behave in production.
Excellent guide on realistic API testing! The point about simulating real user behavior instead of just hammering endpoints is crucial. I've found that adding think time between requests and using realistic payload sizes makes a huge difference in getting actionable results. What tools do you recommend for load testing in a CI/CD pipeline?
To be honest, I mostly used JMETER, usually run from a script in the CI/CD pipeline. It works and was sufficient in most cases, so from my own experience that’s still the tool I can recommend and it also integrates well with BlazeMeter.
That said, JMETER does feel a bit heavy nowadays, especially with its verbose XML when AI is involved. 😄
@xulingfeng suggested Python with Locust, which I think could be a cleaner option and should also be easy to run in a pipeline. But I haven’t used Locust yet, so I can’t really recommend it.
Some comments may only be visible to logged-in visitors. Sign in to view all comments.