All code is guilty, until proven innocent. — Anonymous
The purpose of this article is to help you design and run load test 👍
As developers we already have the foundational knowledge and with a little effort we could expand our skillset.
- Your company cannot afford to hire performance engineers
- Not enough testers compared to developers
- The skill & knowledge could help you write better and scalable code
- Less dependant on other’s expertise
While unit & integration tests ensure code is functionally correct, load testing measures its performance which is equally important.
Only a Load Test can shed light into concurrency issues, whether the database queries make good use of indexes instead of a full table scans, where are the bottlenecks, does the application scale efficiently, what is the application’s response time and throughput and so on.
In this section we will design and run Apache JMeter load tests.
For environment, either find a suitable online resource (not recommended), come up with your own simple service (node, python, whatever) or simply use the web service provided in this article.
We will be using a simple java-based spring boot web service that exposes four (4) endpoints. The requirement are Java 1.8 and Apache Maven.
git clone https://github.com/rhamedy/tinyservice4loadtest.git cd tinyservice4loadtest mvn spring-boot:run
Please go ahead and install Apache JMeter from download site, unzip and execute the following command
apache-jmeter/bin/jmeter.sh // Linux & MacOS apache-jmeter/bin/jmeter.bat // Windows
Let’s load test the following apis
http://localhost:8080/students/list - [GET] List students http://localhost:8080/students/id - [GET] Get student by id http://localhost:8080/students - [POST] Create student http://localhost:8080/students/id - [DELETE] Delete student by id
All the sample tests for above endpoints are available in the GitHub repository.
Right click on test plan and pick
Add > Threads (Users) > Thread Group. A test plan must have at least 1 Thread Group.
Number of Threads (users)
Ramp-Up period (in seconds). How long to reach the max users?
- How many times or for how long to run the test
Let’s specify what to test. Right Click on
ThreadGroup and select
Add > Config Elements > HTTP Request Defaults option.
Config Elements are useful if you wish to share configuration among one or more requests for example i.e. server address, port number, token, etc.
Let’s fill out the HTTP Request Defaults Config element
Also let’s add an
HTTP Header Manager via
Add > Config Elements > HTTP Header Manager for
Let’s configure the http requests, Right Click on the
ThreadGroup and select
Add > Sampler > HTTP Request
- What is the method type i.e.
- What is the api path i.e.
Right click to
duplicate a request and update it.
Listeners are used to collect results. Right Click on
ThreadGroup and select
Add > Listener > Summary Report option.
The test plans in words, we create an
Apache JMeter Test Plan to
Load Test two apis with
50 Users with a ramp-up for a duration of
If you have the
tinyservice4loadtest (or your own) running, then let’s hit the play button and see the results in
The GUI is not recommended for running complex tests. Let’s open a complex sample test from here.
The above test plan has more elements
Random Variablegenerates a value between x and y
Loop Controllerexecutes the content of loop x times
To generate a meaningful report we need
user.properties (Source) in the directory where
.jmx test is.
jmeter.reportgenerator.report_title=Apache JMeter Dashboard jmeter.reportgenerator.overall_granularity=60000 jmeter.reportgenerator.graph.responseTimeDistribution.property.set_granularity=100 jmeter.reportgenerator.apdex_satisfied_threshold=1500 jmeter.reportgenerator.apdex_tolerated_threshold=3000 jmeter.reportgenerator.exporter.html.property.output_dir=/tmp/test-jmeter.reportgenerator.exporter.html.filters_only_sample_series=true
Run the test script using command (output-directory should be empty)
jmeter.sh -n -t loadtest.jmx -l log.jtl -e -o output-directory
In the above command the
-n stands for
no gui the
-t indicates the scripts
-l is for
-o are for reporting.
output-directory will contain a bunch of files including an
index.html that opens the graphical results of the test as shown below.
In this graph the left side is APDEX and the right side is Request summary. The red indicates all our 404 errors and the green is 200 successful requests.
Some numbers relating to
Number of samples, Response time, Throughput
Most importantly the
Response Time and
Lastly, it’s worth-mentioning that
Apache JMeter can be configured to listen to a browser’s activity and capture the network request. For details see here.
Now that we know how to use
Apache JMeter to run a basic load test, let’s explore open-source framework Taurus. In short, one of the reasons for birth of Taurus was because
Apache JMeter has a steep learning curve and
Taurus make things a lot simpler.
Taurus is an abstraction layer (or a wrapper) on top of Apache JMeter and that means you can run an Apache JMeter script using Taurus. So go ahead and install Taurus using the easy to install instruction
Taurus scripts can be written in
JSON using follow blocks
Scenarios is basically where one or more requests are defined. For each
execution is defined with props such as
no of users,
ramp-up period and so on. The
modules allow us to configure
executor that could be
Selenium and so on. Likewise, the
reporting allows configuring how the report should be generated i.e. csv, live reporting in console, or push the result to the blazemeter website.
scenarios: StudentList: requests: - url: http://localhost:8080/students/list label: Student List execution: - scenario: StudentList concurrency: 15 ramp-up: 2s hold-for: 10s reporting: - module: console - module: final-stats summary: true percentiles: true test-duration: true dump-csv: single_scenario_single_req.csv
/students/list api reaching
15 users within
2s (ramp-up) for a duration of
10s and display
live result in the console as well as
To run the Taurus test (sample provided here), simply run the command bzt
Taurus test you can also configure a scenario to point to an
Apache JMeter script and override execution and other parameters.
scenarios: JMeterExample: script: student_crud_jmeter.jmx
Taurus seems to be a very interesting framework and it is worth checking it out. It is very well-documented here.
Read some of my other articles on Medium
Note: I am thinking of moving away from Medium because of their Paywall. I see a huge difference in stats of my stories before and after introduction of the Paywall. I prefer my articles to be accessible at no cost.
Feel free to connect and follow me on your favourite social media for future articles.