Performance Test: Basics

I have been working as an engineer for a cloud-based SaaS system. As the amount of data increases, I have encountered problems in the system. For example, the web servers are not down, but it's response time becomes extremely long. I come up with learning about performance testing because I wanted to anticipate these future problems in advance and prepare trump cards. In this article, I would like to summarize performance testing.

Purpose of Performance Testing

The goal of performance testing is to measure the performance of the system and plan to increase system's availability. We look at this following point of views.

1. To estimate the response performance of the system in each of the various use cases.
2. Improve system performance under high load
3. Ensure that the system is scalable.
4. Check the scale characteristics of the system.

In a cloud environment, web severs can be provisioned quickly and can be scaled out after actual user access occurs. However, depending on the application and the configuration of the middleware, scaling out in a hurry after the load has increased will not well handle traffics because of lacking of scalability.

System Performance Metrics

As performance metrics, throughput and latency should be considered.

Throughput

For a web application, this often indicates the number of HTTP requests to be processed per second. The throughput of the overall system is constrained by the throughput of the bottleneck portion of the system. We investigate where the bottleneck is located and improve it's throughput. It is important to correctly identify where the bottleneck is. If you get it wrong, the solution will not only be ineffective, but it can cause severe congestion at worse.

Latency

It is a processing time. There are two aspects:

1. Processing time from the user's point of view. The time between when the user sends a request and when the response is received.
2. Processing time in the system. This is the time between when the system sends the request and when it returns the response.

In a performance test, we will focus on the processing time in the system (2.) because it is difficult to simulate actual network conditions from the user to the system.

Latency is different from throughput improvement; reducing the latency at one location reduces the latency of the entire system.

Characteristics of scalability

As Characteristics of scalability, we consider What parts of the system need to be augmented to increase performance. For example, we think, can we increase the throughput by increasing the number of web servers? Do we need to increase the memory size of the database at the same time?

We want to mainly know following perspectives:

1. Optimal infrastructure configuration to handle several throughput levels (100rps, 500rps,,,)
2. The actual performance limit of the web system

It is difficult to know exactly what the marginal performance is. Because we don't know what our future application code or system configuration or users will look like.　However, it is meaningful to investigate the system performance. Because in the process of investigating the critical performance, it is possible to discover some improvements or unexpected bottlenecks.

Conclusion

At this post, I wrote about the purpose of performance testing, system metrics, and the system scalability these for the basis of performance testing. I will write about how to plan a performance test in the future post.