Table Of Contents
- Quickstart
- Requirements
- Scaling out our testing
- SBT Setup
- Gatling Simulation Setup
- Jenkins Pipeline Setup
Gatling is a load testing tool that allows developers to programmatically define flexible and powerful testing scenarios, via its' inbuilt DSL. Through the use of Akka Actors, it is able to push large amounts of load from even a single host.
Having said that, Gatling is capable of overwhelming the resources on the single host it is run from, given enough incentive i.e., very large amounts of connections, complex scenarios, large data feeders, etc. Once you've reached that point one of your options is to scale out your Gatling instances, thereby splitting the load between them.
Although this can be done manually, running the tests and collecting and collating the data can be tedious and error-prone. There is a basic scaling out procedure defined on the Gatling website but we can do better.
We will look at automating this procedure via Jenkins Pipelines to a high degree, allowing you to spend more time testing and less time performing boilerplate manoeuvers to get to your test results.
Quickstart
- Fork http://github.com/imavroukakis/gatling-scale-out/
- Edit
Jenkinsfileto match your configuration and requirements - Create a new Jenkins MultiBranch Pipeline on the forked repo
-
Approve
staticMethod java.lang.Math round doublein Jenkins - Load test!
Requirements
- Jenkins v2.219 or higher on Linux, Pipeline capable and with the following plugins
- SBT plugin
- Github Pipeline plugin
- AdoptOpenJDK plugin (or other JDK provider)
- Build Name and Description Setter
- Workspace Cleanup Plugin
- Linux Jenkins Workers
- A smattering of Scala
- Some familiarity with Gatling
Scaling out our testing
The approach we follow is this:
- Create a packaged version of our load tests via the SBT Pack plugin
- Run our packaged app in a Jenkins Pipeline, supplying the required command-line parameters for users-per-second and test duration
- Parallel out tests to Jenkins nodes (if the users-per-second rate is higher than some limit)
- Collect, process and archive results from the Jenkins nodes
Let's start by taking a look at the code. We begin with our SBT build script.
SBT setup
We are using the SBT Pack plugin, to create our packaged application.
Create a file called plugins.sbt inside the project directory and add the following line:
addSbtPlugin("org.xerial.sbt" % "sbt-pack" % "0.12")
This will pull down the Pack plugin and make its' tasks available to SBT.
Our SBT build file is as follows
| enablePlugins(PackPlugin) | |
| organization := "dev.to" | |
| name := "gatling-scale-out" | |
| version := "1.0" | |
| scalaVersion := "2.12.10" | |
| val gatlingVersion = "3.3.1" | |
| resolvers += Resolver.sonatypeRepo("releases") | |
| resolvers += Resolver.jcenterRepo | |
| libraryDependencies ++= Seq( | |
| "io.gatling" % "gatling-app" % gatlingVersion, | |
| "io.gatling.highcharts" % "gatling-charts-highcharts" % gatlingVersion exclude("io.gatling", "gatling-recorder"), | |
| "org.rogach" %% "scallop" % "3.4.0", | |
| ) | |
| packMain := Map("load-test" -> "dev.to.gatling.GatlingRunner") | |
| packJvmOpts := Map("load-test" -> Seq("-Xms2G -Xmx2G -XX:+HeapDumpOnOutOfMemoryError -XX:+UseG1GC -XX:MaxGCPauseMillis=30 -XX:G1HeapRegionSize=16m -XX:InitiatingHeapOccupancyPercent=75 -XX:+ParallelRefProcEnabled -XX:+PerfDisableSharedMem -XX:+OptimizeStringConcat -Djava.net.preferIPv4Stack=true -Djava.net.preferIPv6Addresses=false")) | |
| Compile / run / fork := true | |
| javaOptions ++= { | |
| val props = sys.props.toList | |
| props.map { | |
| case (key, value) => s"-D$key=$value" | |
| } | |
| } |
At line 22, are the JVM options the application will run with, taken from the defaults recommended by Gatling.
packJvmOpts := Map("load-test" ->
Seq("-Xms2G -Xmx2G -XX:+HeapDumpOnOutOfMemoryError
-XX:+UseG1GC -XX:MaxGCPauseMillis=30
-XX:G1HeapRegionSize=16m
-XX:InitiatingHeapOccupancyPercent=75
-XX:+ParallelRefProcEnabled
-XX:+PerfDisableSharedMem
-XX:+OptimizeStringConcat
-Djava.net.preferIPv4Stack=true
-Djava.net.preferIPv6Addresses=false"))
You may have to tweak the heap memory settings (-Xms -Xmx), depending on your test requirements.
Gatling Simulation setup
GatlingRunner
| package dev.to.gatling | |
| import java.text.SimpleDateFormat | |
| import java.util.Calendar | |
| import io.gatling.app.Gatling | |
| import io.gatling.core.config.GatlingPropertiesBuilder | |
| import org.rogach.scallop.{ScallopConf, ScallopOption} | |
| class Conf(arguments: Seq[String]) extends ScallopConf(arguments) { | |
| val usersPerSecond: ScallopOption[Int] = opt[Int](default = Some(5)) | |
| val reportOnly: ScallopOption[String] = opt[String]() | |
| val testDuration: ScallopOption[String] = opt[String](default = Some("60_seconds")) | |
| verify() | |
| } | |
| object GatlingRunner { | |
| var conf: Option[Conf] = None | |
| def main(args: Array[String]) { | |
| conf = Some(new Conf(args)) | |
| conf match { | |
| case Some(conf) => { | |
| val simClass = classOf[LoadSimulation].getName | |
| val props = new GatlingPropertiesBuilder | |
| props.simulationClass(simClass) | |
| props.runDescription("Gatling Load Test") | |
| if (conf.reportOnly.isDefined) { | |
| props.reportsOnly(conf.reportOnly()) | |
| } else { | |
| val now = Calendar.getInstance().getTime | |
| val dateFormat = new SimpleDateFormat("yyyy-MM-dd'T'HH_mm_ss") | |
| props.resultsDirectory(s"results/${dateFormat.format(now)}") | |
| } | |
| Gatling.fromMap(props.build) | |
| } | |
| case None => throw new IllegalArgumentException | |
| } | |
| } | |
| } |
We have to have some way of passing in command-line options, so that we can easily tweak our users-per-second and our test duration. There are several ways to do this, but a good option is Scallop. This gives us (amongst others) GNU-style long option names e.g.
--users-per-second=40 --test-duration 20_seconds
We default users-per-second to 5 at line 11 and the test duration to 60 seconds at line 12.
val usersPerSecond: ScallopOption[Int] = opt[Int](default = Some(5))
val testDuration: ScallopOption[String] = opt[String](default = Some("60_seconds"))
At line 29, we check if we're running in --report-only mode. This is our stats collection mode, which happens at the end of our run.
if (conf.reportOnly.isDefined) {
props.reportsOnly(conf.reportOnly())
}
If that option is not present, we prepare our Simulation by asking Gatling to store our results in a folder with the system's current date/time.
else {
val now = Calendar.getInstance().getTime
val dateFormat = new SimpleDateFormat("yyyy-MM-dd'T'HH_mm_ss")
props.resultsDirectory(s"results/${dateFormat.format(now)}")
}
Finally, at line 36, we kick off our simulation.
LoadSimulation
| package dev.to.gatling | |
| import com.typesafe.scalalogging.StrictLogging | |
| import io.gatling.core.Predef._ | |
| import io.gatling.core.scenario.Simulation | |
| import io.gatling.http.Predef._ | |
| import scala.concurrent.duration._ | |
| import scala.language.postfixOps | |
| import scala.util.{Failure, Success, Try} | |
| class LoadSimulation extends Simulation with StrictLogging { | |
| GatlingRunner.conf match { | |
| case Some(conf) => { | |
| val duration: FiniteDuration = Try(Duration(conf.testDuration().replace('_', ' '))) match { | |
| case Success(duration) => duration.asInstanceOf[FiniteDuration] | |
| case Failure(exception) => throw exception | |
| } | |
| val usersPerSecond = conf.usersPerSecond().toDouble | |
| val postsScenario = | |
| scenario("Posts") | |
| .exec(http("Get 100 posts").get("http://jsonplaceholder.typicode.com/todos")) | |
| setUp( | |
| postsScenario.inject( | |
| constantUsersPerSec(usersPerSecond) during duration | |
| ).protocols(http) | |
| ) | |
| } | |
| case None => throw new IllegalStateException | |
| } | |
| } |
This is the basic Gatling Simulation. Through the command line parameters supplied, at line 27 we set up our users-per-second and duration and execute our scenario. Our test scenario will perform a GET against jsonplaceholder.
Jenkins Pipeline setup
Jenkinsfile
| def gitUrl = 'https://github.com/your_repo/gatling-scale-out' | |
| def gitCredentials = 'Github' | |
| def testGroups = [:] | |
| def numberOfTestNodes = 5 | |
| def splitTestsAbove = 50.0 | |
| def jdkTool = 'openjdk-11' | |
| def sbtTool = '1.3.8' | |
| pipeline { | |
| agent any | |
| tools { | |
| jdk jdkTool | |
| } | |
| environment { | |
| SBT_HOME = tool name: sbtTool, type: 'org.jvnet.hudson.plugins.SbtPluginBuilder$SbtInstallation' | |
| PATH = "${env.SBT_HOME}/bin:${env.PATH}" | |
| } | |
| parameters { | |
| choice(choices: ['5', '10', '15', '20', '30', '40', '50', '60', '70', '80', '90', '100'], description: 'The amount of users per second to generate', name: 'usersPerSecond') | |
| choice(choices: ['1_minute', '2_minutes', '5_minutes', '10_minutes', '15_minutes', '20_minutes'], description: 'The amount of time to run the simulation for', name: 'duration') | |
| } | |
| stages { | |
| stage('Checkout') { | |
| steps { | |
| deleteDir() | |
| git branch: 'main', credentialsId: "$gitCredentials", poll: false, url: "$gitUrl" | |
| } | |
| } | |
| stage('Build') { | |
| steps { | |
| sh "sbt clean compile packArchiveTgz" | |
| stash name: 'load-test', includes: 'target/gatling-scale-out-1.0.tar.gz' | |
| } | |
| } | |
| stage('Load Test') { | |
| steps { | |
| script { | |
| currentBuild.description = "Users/sec:${params.usersPerSecond}/Duration:${params.duration}" | |
| def userPerSecond = "${params.usersPerSecond}" as Double | |
| int usersPerNodeCount | |
| if (userPerSecond >= splitTestsAbove) { | |
| usersPerNodeCount = Math.round(userPerSecond / numberOfTestNodes) | |
| } else { | |
| usersPerNodeCount = userPerSecond | |
| numberOfTestNodes = 1 | |
| } | |
| for (int i = 0; i < numberOfTestNodes; i++) { | |
| def num = i | |
| testGroups["node $num"] = { | |
| node { | |
| def javaHome = tool name: jdkTool | |
| deleteDir() | |
| unstash 'load-test' | |
| sh 'mv target/gatling-scale-out-1.0.tar.gz ./' | |
| sh 'tar xf gatling-scale-out-1.0.tar.gz' | |
| sh "JAVA_HOME=$javaHome gatling-scale-out-1.0/bin/load-test --users-per-second=$usersPerNodeCount --test-duration=${params.duration}" | |
| stash name: "node $num", includes: '**/simulation.log' | |
| } | |
| } | |
| } | |
| parallel testGroups | |
| } | |
| } | |
| } | |
| stage('Collect results') { | |
| steps { | |
| script { | |
| for (int i = 0; i < numberOfTestNodes; i++) { | |
| def num = i | |
| unstash "node $i" | |
| } | |
| } | |
| sh 'mv target/gatling-scale-out-1.0.tar.gz ./' | |
| sh 'tar xf gatling-scale-out-1.0.tar.gz' | |
| sh "gatling-scale-out-1.0/bin/load-test --report-only \"${env.WORKSPACE}/results\"" | |
| sh "mv results results-test-${env.BUILD_NUMBER}" | |
| sh "tar zcf results-test-${env.BUILD_NUMBER}.tar.gz results-test-${env.BUILD_NUMBER}" | |
| archiveArtifacts artifacts: "results-test-${env.BUILD_NUMBER}.tar.gz", caseSensitive: false, onlyIfSuccessful: true | |
| } | |
| } | |
| } | |
| } |
Lines 1-6 should be changed to match your setup so change the following to:
your GitHub repo
def gitUrl = 'https://github.com/your_repo/gatling-scale-out'
your stored Jenkins Github credentials ID
def gitCredentials = 'Github'
the number of parallel tests you require
def numberOfTestNodes = 5
the high-watermark users-per-second above which, your tests will be run in parallel
def splitTestsAbove = 50.0
your Jenkins JDK installation ID
def jdkTool = 'openjdk-11'
your SBT installation ID
def sbtTool = '1.3.8'
A brief explanation of what the pipeline blocks do follows:
environment
Sets up SBT to be on the common path for all nodes
parameters
Provides drop-down menus for users-per-second and duration counts.
stage('Checkout')
Checks out the project from GitHub - works with both Multibranch Pipeline jobs and ad-hoc Pipeline Jobs
stage('Build')
Builds the projects, packages it and stashes the application for further use in the pipeline
stage('Load Test')
Looks at the value of usersPerSecond and splits the test if the number is equal to, or above the value of splitTestsAbove
stage('Collect results')
Collects all instances of simulation.log across the nodes, collates them by running the application in reporting mode and archives the results for download.
Jenkins Job setup
Once you've set up the Jenkinsfile to your satisfaction, and committed it to the repo create a new Jenkins MultiBranch Pipeline and configure it as follows:
Save the job and Jenkins will scan your repo, and execute an initial run of the job.
Jenkins may fail the first run of the pipeline, if Math.round is not sandboxed. If that is the case, you should permit this by going to Manage Jenkins -> In-Process Script Approval and approving
staticMethod java.lang.Math round double
After this is done, the Build with Parameters option will be available on the left-hand side menu. Clicking on it should bring you to this screen:
Choose 60 users, click on the Build button and wait for the job to finish and then, look at the Build Result.
Click on the tar archive under Build Artifacts to download the Gatling bundle. Decompress the bundle and open the index.html file.
In the above example, 3600 requests were executed across 5 nodes for 1 minute. This translates to around 12 users per node. You can confirm this by looking at the build logs.
Conclusion
Congratulations, you now have a basic scaled-out Gatling test. I hope this tutorial has been helpful, but if there's anything that is unclear or puzzling, feel free to drop a comment and I'll do my best to help out.
Top comments (0)