Scale out your Gatling tests with Jenkins

Table Of Contents

• Quickstart
• Requirements
• Scaling out our testing
• SBT Setup
• Gatling Simulation Setup
  • Gatling Runner class
  • Load Simulation class
• Jenkins Pipeline Setup
  • Jenkinsfile
  • Jenkins Job Setup

Gatling is a load testing tool that allows developers to programmatically define flexible and powerful testing scenarios, via its' inbuilt DSL. Through the use of Akka Actors, it is able to push large amounts of load from even a single host.

Having said that, Gatling is capable of overwhelming the resources on the single host it is run from, given enough incentive i.e., very large amounts of connections, complex scenarios, large data feeders, etc. Once you've reached that point one of your options is to scale out your Gatling instances, thereby splitting the load between them.

Although this can be done manually, running the tests and collecting and collating the data can be tedious and error-prone. There is a basic scaling out procedure defined on the Gatling website but we can do better.
We will look at automating this procedure via Jenkins Pipelines to a high degree, allowing you to spend more time testing and less time performing boilerplate manoeuvers to get to your test results.

Quickstart

• Fork http://github.com/imavroukakis/gatling-scale-out/
• Edit Jenkinsfile to match your configuration and requirements
• Create a new Jenkins MultiBranch Pipeline on the forked repo
• Approve staticMethod java.lang.Math round double in Jenkins
• Load test!

Requirements

• Jenkins v2.219 or higher on Linux, Pipeline capable and with the following plugins
  • SBT plugin
  • Github Pipeline plugin
  • AdoptOpenJDK plugin (or other JDK provider)
  • Build Name and Description Setter
  • Workspace Cleanup Plugin
• Linux Jenkins Workers
• A smattering of Scala
• Some familiarity with Gatling

Scaling out our testing

The approach we follow is this:

• Create a packaged version of our load tests via the SBT Pack plugin
• Run our packaged app in a Jenkins Pipeline, supplying the required command-line parameters for users-per-second and test duration
• Parallel out tests to Jenkins nodes (if the users-per-second rate is higher than some limit)
• Collect, process and archive results from the Jenkins nodes

Let's start by taking a look at the code. We begin with our SBT build script.

SBT setup

We are using the SBT Pack plugin, to create our packaged application.
Create a file called plugins.sbt inside the project directory and add the following line:

addSbtPlugin("org.xerial.sbt" % "sbt-pack" % "0.12")

This will pull down the Pack plugin and make its' tasks available to SBT.

Our SBT build file is as follows

At line 22, are the JVM options the application will run with, taken from the defaults recommended by Gatling.

packJvmOpts := Map("load-test" -> 
Seq("-Xms2G -Xmx2G -XX:+HeapDumpOnOutOfMemoryError 
-XX:+UseG1GC -XX:MaxGCPauseMillis=30 
-XX:G1HeapRegionSize=16m 
-XX:InitiatingHeapOccupancyPercent=75 
-XX:+ParallelRefProcEnabled 
-XX:+PerfDisableSharedMem 
-XX:+OptimizeStringConcat 
-Djava.net.preferIPv4Stack=true 
-Djava.net.preferIPv6Addresses=false"))

You may have to tweak the heap memory settings (-Xms -Xmx), depending on your test requirements.

Gatling Simulation setup

GatlingRunner

We have to have some way of passing in command-line options, so that we can easily tweak our users-per-second and our test duration. There are several ways to do this, but a good option is Scallop. This gives us (amongst others) GNU-style long option names e.g.

--users-per-second=40 --test-duration 20_seconds

We default users-per-second to 5 at line 11 and the test duration to 60 seconds at line 12.

val usersPerSecond: ScallopOption[Int] = opt[Int](default = Some(5))
val testDuration: ScallopOption[String] = opt[String](default = Some("60_seconds"))

At line 29, we check if we're running in --report-only mode. This is our stats collection mode, which happens at the end of our run.

if (conf.reportOnly.isDefined) {
   props.reportsOnly(conf.reportOnly())
}

If that option is not present, we prepare our Simulation by asking Gatling to store our results in a folder with the system's current date/time.

else {
  val now = Calendar.getInstance().getTime
  val dateFormat = new SimpleDateFormat("yyyy-MM-dd'T'HH_mm_ss")          
  props.resultsDirectory(s"results/${dateFormat.format(now)}")
}

Finally, at line 36, we kick off our simulation.

LoadSimulation

This is the basic Gatling Simulation. Through the command line parameters supplied, at line 27 we set up our users-per-second and duration and execute our scenario. Our test scenario will perform a GET against jsonplaceholder.

Jenkins Pipeline setup

Jenkinsfile

Lines 1-6 should be changed to match your setup so change the following to:

your GitHub repo

def gitUrl = 'https://github.com/your_repo/gatling-scale-out'

your stored Jenkins Github credentials ID

def gitCredentials = 'Github'

the number of parallel tests you require

def numberOfTestNodes = 5

the high-watermark users-per-second above which, your tests will be run in parallel

def splitTestsAbove = 50.0

your Jenkins JDK installation ID

def jdkTool = 'openjdk-11'

your SBT installation ID

def sbtTool = '1.3.8'

A brief explanation of what the pipeline blocks do follows:

environment

Sets up SBT to be on the common path for all nodes

parameters

Provides drop-down menus for users-per-second and duration counts.

stage('Checkout')

Checks out the project from GitHub - works with both Multibranch Pipeline jobs and ad-hoc Pipeline Jobs

stage('Build')

Builds the projects, packages it and stashes the application for further use in the pipeline

stage('Load Test')

Looks at the value of usersPerSecond and splits the test if the number is equal to, or above the value of splitTestsAbove

stage('Collect results')

Collects all instances of simulation.log across the nodes, collates them by running the application in reporting mode and archives the results for download.

Jenkins Job setup

Once you've set up the Jenkinsfile to your satisfaction, and committed it to the repo create a new Jenkins MultiBranch Pipeline and configure it as follows:

Save the job and Jenkins will scan your repo, and execute an initial run of the job.

Jenkins may fail the first run of the pipeline, if Math.round is not sandboxed. If that is the case, you should permit this by going to Manage Jenkins -> In-Process Script Approval and approving
staticMethod java.lang.Math round double

After this is done, the Build with Parameters option will be available on the left-hand side menu. Clicking on it should bring you to this screen:

Choose 60 users, click on the Build button and wait for the job to finish and then, look at the Build Result.

Click on the tar archive under Build Artifacts to download the Gatling bundle. Decompress the bundle and open the index.html file.

In the above example, 3600 requests were executed across 5 nodes for 1 minute. This translates to around 12 users per node. You can confirm this by looking at the build logs.

Conclusion

Congratulations, you now have a basic scaled-out Gatling test. I hope this tutorial has been helpful, but if there's anything that is unclear or puzzling, feel free to drop a comment and I'll do my best to help out.