Artillery Load Testing: A Practical Guide for APIs

Artillery is an open-source Node.js load-testing toolkit for driving high-concurrency traffic against APIs from a YAML script. You define load phases, model virtual-user flows, run artillery run script.yml, and inspect latency percentiles, request rates, and errors. This guide shows how to install Artillery v2, write a practical test, run it locally, export results the current v2 way, and add it to CI.

Try Apidog today

What Artillery Is and When to Use It

Artillery creates virtual users (VUs) that execute scenarios against your API. Each VU behaves like a client: it starts, runs a sequence of requests, and exits.

Use Artillery when you need to answer questions like:

• What happens to p95 latency at 50 requests per second?
• At what arrival rate do errors start appearing?
• Can the API stay stable for five minutes of sustained load?
• Does a new release introduce a performance regression?

Artillery works well because tests are declarative. Instead of writing your own concurrency loop, you describe the load shape and request flow in YAML. The same script can run on a laptop, in CI, or in a scheduled performance pipeline.

If you are comparing load-testing tools, these references cover trade-offs across k6, JMeter, Gatling, and others:

• Top load testing tools roundup
• Load testing software comparison

Install Artillery v2

Install the latest Artillery CLI from npm:

npm install -g artillery@latest
artillery version

Artillery runs on Windows, macOS, and Linux. Use a recent LTS version of Node.js.

If you do not want a global install, run Artillery with npx:

npx artillery@latest run script.yml

Write an Artillery Test Script

An Artillery test file has two main sections:

• config: target host, load profile, variables, payloads, and environment settings
• scenarios: the actions each virtual user performs

Create script.yml:

config:
  target: "https://api.example.com"
  phases:
    - name: "Warm up"
      duration: 60
      arrivalRate: 5
    - name: "Ramp to peak"
      duration: 120
      arrivalRate: 5
      rampTo: 50
    - name: "Sustained load"
      duration: 300
      arrivalRate: 50
      maxVusers: 500
  variables:
    productId:
      - "1001"
      - "1002"

scenarios:
  - name: "Browse and create order"
    flow:
      - get:
          url: "/v1/products/{{ productId }}"
      - post:
          url: "/v1/orders"
          json:
            productId: "{{ productId }}"
            quantity: 2

This script does three things:

1. Starts at 5 new virtual users per second.
2. Ramps from 5 to 50 new virtual users per second.
3. Holds 50 new virtual users per second for 5 minutes.

Each VU gets a productId, fetches a product, and creates an order.

Understand the config Section

config.target is the base URL for all requests:

config:
  target: "https://api.example.com"

A request like this:

- get:
    url: "/v1/products/{{ productId }}"

runs against:

https://api.example.com/v1/products/1001

Common phases fields

phases:
  - name: "Sustained load"
    duration: 300
    arrivalRate: 50
    maxVusers: 500

Use these fields most often:

• duration: how long the phase spawns new VUs, in seconds or strings like "5m"
• arrivalRate: how many new VUs start per second
• rampTo: linearly ramps from arrivalRate to this value
• arrivalCount: starts a fixed number of VUs across the phase
• maxVusers: caps concurrent VUs
• name: labels the phase in output

Important detail: duration controls how long Artillery starts new virtual users. It does not guarantee the whole test ends exactly when the phase ends. If a VU starts near the end of a phase and still has requests to run, Artillery waits for that VU to finish.

Understand the scenarios Section

scenarios is an array of user journeys:

scenarios:
  - name: "Browse and create order"
    flow:
      - get:
          url: "/v1/products/{{ productId }}"
      - post:
          url: "/v1/orders"
          json:
            productId: "{{ productId }}"
            quantity: 2

A scenario can include:

• name: readable label
• weight: relative probability that Artillery selects this scenario
• flow: ordered list of HTTP steps

Supported HTTP step keys include:

• get
• post
• put
• patch
• delete

Request bodies go under json:

- post:
    url: "/v1/orders"
    json:
      productId: "{{ productId }}"
      quantity: 2

Variables use double-curly syntax:

"{{ productId }}"

Drive Test Data from a CSV File

Inline variables are useful for small tests. For more realistic traffic, load data from a CSV file with config.payload.

Example users.csv:

[e-mail],password123
[e-mail],password456

Example Artillery script:

config:
  target: "https://api.example.com"
  payload:
    path: "./users.csv"
    fields:
      - "email"
      - "password"
  phases:
    - duration: 120
      arrivalRate: 20

scenarios:
  - name: "Login"
    flow:
      - post:
          url: "/login"
          json:
            email: "{{ email }}"
            password: "{{ password }}"

Each virtual user receives values from the CSV, and the column names become variables in the scenario.

Run the Test

Run the script:

artillery run script.yml

Override the target without editing YAML:

artillery run --target https://staging.example.com script.yml

Pass variables as JSON:

artillery run -v '{ "productId": ["1001","1002"] }' script.yml

Useful flags:

# Override config.target
artillery run --target https://staging.example.com script.yml

# Short form
artillery run -t https://staging.example.com script.yml

# Select an environment from config.environments
artillery run --environment staging script.yml

# Short form
artillery run -e staging script.yml

# Load config from a separate file
artillery run --config config.yml script.yml

# Short form
artillery run -c config.yml script.yml

# Skip TLS verification for test environments with self-signed certificates
artillery run --insecure script.yml

# Short form
artillery run -k script.yml

Read the Results

During execution, Artillery prints aggregate metrics roughly every 10 seconds. At the end, it prints a summary.

Focus on these metrics:

• Request rate: requests per second actually achieved
• Latency percentiles: p50, p95, and p99 response times
• Error counts: failed requests, timeouts, and non-2xx responses

Tail latency matters more than averages. An average can look fine while p95 or p99 climbs into multi-second territory.

Use this rule of thumb:

• If p95 rises during the ramp, the system may be nearing saturation.
• If errors appear only during sustained load, investigate resource exhaustion.
• If p99 spikes while p50 stays stable, look for outliers such as slow database queries, queue contention, or downstream service latency.

For more on performance metrics, see this API performance testing guide.

Generate a Report in Artillery v2

Artillery reporting changed in v2.

Older tutorials often show:

artillery run --output report.json script.yml
artillery report report.json

The first command still works. The second command does not.

The --output flag still writes a machine-readable JSON results file:

artillery run --output report.json script.yml

But the old artillery report HTML generator has been removed from the Artillery CLI. Do not rely on:

artillery report report.json

Use one of these current options instead.

Option 1: Parse JSON with jq

For CI, parse the JSON and enforce thresholds.

Get aggregate p95 response time:

jq '.aggregate.summaries["http.response_time"].p95' report.json

Example CI-friendly threshold check:

P95=$(jq '.aggregate.summaries["http.response_time"].p95' report.json)

if (( $(echo "$P95 > 500" | bc -l) )); then
  echo "p95 latency too high: ${P95}ms"
  exit 1
fi

Option 2: Record to Artillery Cloud

Use Artillery Cloud for hosted dashboards:

artillery run --record --key $ARTILLERY_CLOUD_API_KEY script.yml

Option 3: Publish Metrics to Your Observability Stack

Use the publish-metrics plugin or OpenTelemetry to send load-test metrics to the dashboards you already use for production monitoring.

This is useful when you want to correlate load-test results with:

• CPU usage
• memory usage
• database latency
• queue depth
• downstream service errors
• application logs

Run Artillery in CI

Because Artillery is a Node.js CLI, it fits naturally into CI pipelines.

Here is a GitHub Actions workflow that installs Artillery, runs the test, and uploads the JSON report as an artifact:

name: Load test

on: [workflow_dispatch]

jobs:
  artillery:
    runs-on: ubuntu-latest

    steps:
      - uses: actions/checkout@v4

      - uses: actions/setup-node@v4
        with:
          node-version: "lts/*"

      - run: npm install -g artillery@latest

      - run: artillery run --output report.json script.yml

      - uses: actions/upload-artifact@v4
        with:
          name: artillery-report
          path: report.json

This workflow runs manually with workflow_dispatch.

For heavier tests, prefer manual or scheduled runs instead of running on every commit. Load tests can take minutes, consume real bandwidth, and hit shared environments hard.

You can also fail the workflow when p95 exceeds a threshold:

      - name: Check p95 latency
        run: |
          P95=$(jq '.aggregate.summaries["http.response_time"].p95' report.json)
          echo "p95 latency: ${P95}ms"

          if (( $(echo "$P95 > 500" | bc -l) )); then
            echo "p95 latency exceeded 500ms"
            exit 1
          fi

Where Apidog Fits: Functional Testing and CI Gating

Artillery answers:

Can this API survive this traffic level?

That is load and performance testing.

A separate question is:

Does this API still return correct responses after a code change?

That is functional and regression testing, and it is where Apidog fits.

Apidog is an all-in-one API platform for design, debugging, mocking, documentation, and automated testing. Its test scenarios group endpoints into logical steps with conditions like if, for, and foreach, so you can validate response bodies, status codes, and contracts.

Use the boundary clearly:

• Use Artillery for high-concurrency load testing.
• Use Apidog for functional, regression, and contract checks in CI.
• Use Apidog’s performance-testing feature for smaller checks up to 100 virtual users, not as a replacement for Artillery at high concurrency.

This same framing appears in this guide to load-testing APIs without Python, and Apidog’s 100-VU feature is covered in API performance testing in Apidog.

A practical setup is:

1. Run Artillery to validate scalability.
2. Run Apidog CLI in CI to catch broken behavior before merge or deploy.

Install the Apidog CLI from npm:

npm install -g apidog-cli

Run a test scenario:

apidog run \
  --access-token $APIDOG_ACCESS_TOKEN \
  -t <TEST_SCENARIO_ID> \
  -e <ENVIRONMENT_ID> \
  -r cli,junit \
  --out-dir ./apidog-reports

Flag reference:

• --access-token: Apidog access token
• -t: test scenario ID
• -e: required environment ID
• -r cli,junit: output console results and JUnit XML
• --out-dir: report output directory

For a full walkthrough, see the Apidog CLI tutorial. For pipeline design patterns, see these CI/CD best practices for API testing.

Want functional and contract tests gating your CI alongside your Artillery load runs? Download Apidog for free and build your first test scenario.

Frequently Asked Questions

What is Artillery load testing?

Artillery load testing is the practice of using the open-source Artillery toolkit to simulate many concurrent virtual users hitting your API. You describe the load profile and request flow in YAML, run the script, and measure latency percentiles, request rates, and errors.

Is Artillery free and open source?

Yes. The core Artillery CLI is free and open source, distributed on npm as the artillery package. Artillery Cloud is a paid hosted offering for dashboards and result tracking, but you can run load tests locally and in CI without it.

How do you run an Artillery load test?

Install Artillery:

npm install -g artillery@latest

Create a YAML script with:

• config: target and load phases
• scenarios: request flow for virtual users

Then run:

artillery run script.yml

Artillery prints live metrics during the run and a summary at the end.

How do you generate an Artillery report?

Use --output to write JSON results:

artillery run --output report.json script.yml

The old artillery report HTML command has been removed from the Artillery CLI. Instead, parse the JSON with tools like jq, use Artillery Cloud with --record --key, or publish metrics with the publish-metrics plugin or OpenTelemetry.

Artillery vs k6 or JMeter: which should you use?

All three can support high-scale load testing.

• Artillery uses declarative YAML and runs in the Node.js ecosystem.
• k6 uses JavaScript with a code-first scripting model.
• JMeter is Java-based, GUI-driven, and has a long plugin history.

The Gatling vs JMeter comparison covers related trade-offs in more depth.

Pick the tool whose scripting model fits your team, then pair load tests with functional CI checks for better coverage.