DEV Community

Cover image for How to Run API Tests in CI With apidog run
Hassann
Hassann

Posted on • Originally published at apidog.com

How to Run API Tests in CI With apidog run

#api #cicd #cli #testing

You copied a command from a CI/CD tab, pasted it into a pipeline, and it worked. Then a teammate asks why the build still passes when a test failed, how to run the same scenario against staging, or how to generate a report your CI dashboard can parse. At that point, the one-liner is not enough. You need to know what each flag does.

Try Apidog today

apidog run is the core command in the Apidog command-line runner. It executes API test scenarios you built in Apidog headlessly from a terminal: no GUI, no manual clicks. You use flags to choose the scenario, target environment, iteration behavior, reports, TLS settings, and failure handling.

What apidog run does

The Apidog CLI ships as an npm package named apidog-cli. Install it once and you get the apidog binary. Its main subcommand is run.

Install globally:

npm install -g apidog-cli
Enter fullscreen mode Exit fullscreen mode

Verify the install:

apidog -v
Enter fullscreen mode Exit fullscreen mode

apidog run does not define a separate test format. It runs the test scenarios you already authored in Apidog: chained requests, assertions, extracted values, scripts, and optional data-driven iterations.

A minimal command looks like this:

apidog run --access-token $APIDOG_ACCESS_TOKEN -t 605067 -e 1629989 -n 1 -r html,cli
Enter fullscreen mode Exit fullscreen mode

This command:

  • Authenticates with $APIDOG_ACCESS_TOKEN
  • Runs test scenario 605067
  • Uses environment 1629989
  • Runs once
  • Produces both HTML and CLI output

Option reference

Use this table as a quick lookup. Run apidog run --help on your installed version for the authoritative list.

Group Flag Argument What it controls
Select -t, --test-scenario <testScenarioId> Run one scenario by ID
Select -f, --test-scenario-folder <folderId> Run every scenario in a folder
Select --test-suite <testSuiteId> Run a test suite by ID
Select --project <projectId> The project the scenario belongs to
Select --branch <branchName> Run against a branch; defaults to main
Auth --access-token <accessToken> Authenticate the run; required online
Env -e, --environment <environmentId> Which environment to target
Iterate -n, --iteration-count <n> Run the scenario n times
Iterate -d, --iteration-data <path> Drive iterations from a JSON or CSV file
Iterate --variables <path> Load variables from a local file
Iterate --global-var <value> Set a global variable inline, key=value
Iterate --env-var <value> Override an environment variable inline, key=value
Report -r, --reporters [reporters] Report formats: cli, html, json, junit
Report --out-dir <outDir> Where reports get written
Report --out-file <outFile> Report filename, with name placeholders
Report --out-json-failures-separated <value> Write failures to a separate JSON file
Report --upload-report [value] Upload a report overview to the Apidog cloud
Errors --on-error <behavior> ignore, continue, or end on failure
Errors --ignore-redirects Do not follow HTTP redirects
Errors --delay-request [n] Wait n ms between requests
Errors --timeout-request [n] Per-request timeout in ms
Errors --timeout-script [n] Script-execution timeout in ms
TLS -k, --insecure Disable SSL certificate verification
TLS --ssl-client-cert <path> Client certificate PEM file
TLS --ssl-client-key <path> Private key for the client cert
TLS --ssl-client-passphrase <passphrase> Passphrase for the client cert
TLS --ssl-client-cert-list <path> Config mapping certs to hosts
TLS --ssl-extra-ca-certs <path> Extra trusted CA certificates
Output --verbose Print full request and response detail
Output --silent Suppress console output
Output --color <value> Force colored output on or off
Output --external-program-path <path> External-programs file for custom logic
Output --database-connection <path> Database config for steps that query a DB
Output --preferred-http-version <version> Preferred HTTP protocol version
Output -b, --bigint Enable BigInt for large numeric values
Output --lang <language> CLI language
Output -h, --help Print help

Choose what to run

Pick one selector per command.

Use -t, --test-scenario <id> to run one scenario:

apidog run --access-token $APIDOG_ACCESS_TOKEN -t 605067 -e 1629989 -r cli
Enter fullscreen mode Exit fullscreen mode

Use -f, --test-scenario-folder <id> to run every scenario in a folder:

apidog run --access-token $APIDOG_ACCESS_TOKEN -f 88012 -r html,junit
Enter fullscreen mode Exit fullscreen mode

Use --test-suite <id> to run a curated test suite:

apidog run --access-token $APIDOG_ACCESS_TOKEN --test-suite 40231 -r junit
Enter fullscreen mode Exit fullscreen mode

Add --project <id> when your token can access multiple projects.

Add --branch <name> when you want to validate the scenario as it exists on a non-main branch:

apidog run --access-token $APIDOG_ACCESS_TOKEN -t 605067 --branch feature-payments -e 1629989 -r cli
Enter fullscreen mode Exit fullscreen mode

Authenticate safely

Every online run needs:

--access-token <token>
Enter fullscreen mode Exit fullscreen mode

Generate the token from the scenario’s CI/CD tab in Apidog. That tab also gives you a prefilled apidog run command with the scenario ID and environment ID.

Do not commit the token. Store it as a CI secret and inject it as an environment variable:

apidog run --access-token $APIDOG_ACCESS_TOKEN -t 605067 -e 1629989 -r cli
Enter fullscreen mode Exit fullscreen mode

If the token leaks, regenerate it from the same CI/CD tab.

Target environments and run iterations

Use -e, --environment <id> to point the same scenario at different environments:

# staging
apidog run --access-token $APIDOG_ACCESS_TOKEN -t 605067 -e $STAGING_ENV_ID -r cli

# production
apidog run --access-token $APIDOG_ACCESS_TOKEN -t 605067 -e $PROD_ENV_ID -r cli
Enter fullscreen mode Exit fullscreen mode

This lets you reuse one scenario for pull-request checks, staging validation, and production smoke tests. For more on environment values, see environment and secrets management for API clients.

Run the same scenario multiple times with -n:

apidog run --access-token $APIDOG_ACCESS_TOKEN -t 605067 -e 1629989 -n 10 -r cli
Enter fullscreen mode Exit fullscreen mode

Drive iterations from a CSV or JSON file with -d:

apidog run --access-token $APIDOG_ACCESS_TOKEN -t 605067 -d ./accounts.csv -r junit
Enter fullscreen mode Exit fullscreen mode

Use this for cases like:

  • Running one login scenario across many accounts
  • Running one order flow across multiple payloads
  • Checking multiple tenants with the same assertions

For the deeper pattern, see data-driven API testing with CSV and JSON.

You can also inject values at runtime:

# Load variables from a local file
apidog run --access-token $APIDOG_ACCESS_TOKEN -t 605067 --variables ./vars.json -r cli

# Set a global variable
apidog run --access-token $APIDOG_ACCESS_TOKEN -t 605067 --global-var buildId=$GITHUB_RUN_ID -r cli

# Override an environment variable for this run only
apidog run --access-token $APIDOG_ACCESS_TOKEN -t 605067 -e 1629989 --env-var baseUrl=https://staging.internal -r cli
Enter fullscreen mode Exit fullscreen mode

If you need a refresher on scopes, see mastering variables in Apidog.

Generate reports your CI can use

Select reporters with -r, --reporters. Pass multiple formats as a comma-separated list:

apidog run --access-token $APIDOG_ACCESS_TOKEN -t 605067 -r cli,html,junit,json
Enter fullscreen mode Exit fullscreen mode

Available reporters:

  • cli: readable terminal output
  • html: self-contained HTML report for humans
  • junit: XML format most CI dashboards can parse
  • json: structured raw output for custom processing

A common CI setup is HTML for artifacts plus JUnit for the dashboard:

apidog run \
  --access-token $APIDOG_ACCESS_TOKEN \
  -t 605067 \
  -e 1629989 \
  -r html,junit \
  --out-dir ./apidog-reports
Enter fullscreen mode Exit fullscreen mode

Report output flags:

# Write reports to a specific directory
--out-dir ./apidog-reports

# Set a filename pattern
--out-file "{SCENARIO_NAME}-{GENERATE_TIME}"

# Split failures into a separate JSON file
--out-json-failures-separated true

# Upload a report overview to Apidog cloud
--upload-report
Enter fullscreen mode Exit fullscreen mode

Example with a timestamped report filename:

apidog run \
  --access-token $APIDOG_ACCESS_TOKEN \
  -t 605067 \
  -e 1629989 \
  -r html,junit \
  --out-dir ./reports \
  --out-file "{SCENARIO_NAME}-{GENERATE_TIME}"
Enter fullscreen mode Exit fullscreen mode

Make failures fail the pipeline

apidog run exits with:

  • 0 when all assertions pass
  • Non-zero when anything fails

That exit code is your CI gate. Do not hide it with shell patterns like this:

apidog run --access-token $APIDOG_ACCESS_TOKEN -t 605067 -e 1629989 || true
Enter fullscreen mode Exit fullscreen mode

Use --on-error to control what the runner does after a failure:

# Stop at the first failed step
apidog run \
  --access-token $APIDOG_ACCESS_TOKEN \
  -t 605067 \
  -e 1629989 \
  -r cli \
  --on-error end
Enter fullscreen mode Exit fullscreen mode 
# Continue and collect all failures
apidog run \
  --access-token $APIDOG_ACCESS_TOKEN \
  -f 88012 \
  -r html,junit \
  --on-error continue \
  --out-dir ./nightly-reports
Enter fullscreen mode Exit fullscreen mode

Supported values:

  • end: stop at the first failed step
  • continue: keep running and collect all failures
  • ignore: continue past a known issue without treating that step as fatal

Use ignore carefully. It can hide regressions if overused.

Request-behavior flags:

# Assert on redirect responses instead of following them
--ignore-redirects

# Wait between requests, in milliseconds
--delay-request 500

# Per-request timeout, in milliseconds
--timeout-request 10000

# Script timeout, in milliseconds
--timeout-script 5000
Enter fullscreen mode Exit fullscreen mode

Configure TLS and certificates

Use these flags when testing endpoints behind internal CAs or mutual TLS.

Disable certificate verification:

apidog run \
  --access-token $APIDOG_ACCESS_TOKEN \
  -t 605067 \
  -e 1629989 \
  -r cli \
  -k
Enter fullscreen mode Exit fullscreen mode

Use -k, --insecure only for trusted internal hosts. It disables the certificate verification that protects the connection.

For mutual TLS, provide a client certificate and key:

apidog run \
  --access-token $APIDOG_ACCESS_TOKEN \
  -t 605067 \
  -e 1629989 \
  -r cli \
  --ssl-client-cert ./client.pem \
  --ssl-client-key ./client-key.pem
Enter fullscreen mode Exit fullscreen mode

If the private key is encrypted:

--ssl-client-passphrase "$CLIENT_CERT_PASSPHRASE"
Enter fullscreen mode Exit fullscreen mode

For internal certificate authorities, prefer adding the CA certificate instead of disabling verification:

apidog run \
  --access-token $APIDOG_ACCESS_TOKEN \
  -t 605067 \
  -e 1629989 \
  -r cli \
  --ssl-extra-ca-certs ./internal-ca.pem
Enter fullscreen mode Exit fullscreen mode

When different hosts need different certificates, use:

--ssl-client-cert-list ./cert-list.json
Enter fullscreen mode Exit fullscreen mode

Debug output and runtime details

Use --verbose when a scenario passes locally but fails in CI:

apidog run \
  --access-token $APIDOG_ACCESS_TOKEN \
  -t 605067 \
  -e 1629989 \
  -r cli \
  --verbose
Enter fullscreen mode Exit fullscreen mode

It prints full request and response details for each step.

Use --silent for scheduled jobs where you only care about the exit code and saved reports:

apidog run \
  --access-token $APIDOG_ACCESS_TOKEN \
  -f 88012 \
  -r junit \
  --silent
Enter fullscreen mode Exit fullscreen mode

Force color output on or off:

--color on
--color off
Enter fullscreen mode Exit fullscreen mode

Other execution-specific flags:

# External programs file for scenario custom logic
--external-program-path ./external-programs.json

# Database config for DB-querying steps
--database-connection ./db.json

# Preferred HTTP version
--preferred-http-version 2

# Preserve precision for large numeric values
--bigint

# Set CLI display language
--lang en

# Print help
apidog run --help
Enter fullscreen mode Exit fullscreen mode

CI examples you can copy

Production smoke test after deploy

Fail fast and print readable logs:

apidog run \
  --access-token $APIDOG_ACCESS_TOKEN \
  -t 605067 \
  -e $PROD_ENV_ID \
  -r cli \
  --on-error end
Enter fullscreen mode Exit fullscreen mode

Nightly regression over a folder

Collect all failures and publish HTML plus JUnit:

apidog run \
  --access-token $APIDOG_ACCESS_TOKEN \
  -f 88012 \
  -r html,junit \
  --on-error continue \
  --out-dir ./nightly-reports
Enter fullscreen mode Exit fullscreen mode

Data-driven CI run

Run one scenario against a CSV and emit JUnit:

apidog run \
  --access-token $APIDOG_ACCESS_TOKEN \
  -t 605067 \
  -d ./test-cases.csv \
  -r junit
Enter fullscreen mode Exit fullscreen mode

Feature-branch validation

Run the scenario version from a branch:

apidog run \
  --access-token $APIDOG_ACCESS_TOKEN \
  -t 605067 \
  --branch feature-payments \
  -e 1629989 \
  -r cli
Enter fullscreen mode Exit fullscreen mode

Debug a CI-only failure

Dump request and response details:

apidog run \
  --access-token $APIDOG_ACCESS_TOKEN \
  -t 605067 \
  -e 1629989 \
  -r cli \
  --verbose
Enter fullscreen mode Exit fullscreen mode

Use npx instead of a global install

For ephemeral CI runners, avoid global installation:

npx apidog-cli run \
  --access-token $APIDOG_ACCESS_TOKEN \
  -t 605067 \
  -e 1629989 \
  -r junit
Enter fullscreen mode Exit fullscreen mode

The command surface is the same. Choose global install or npx based on how your CI runner is provisioned.

To create the scenario that the CLI runs, Download Apidog, build a scenario in the app, copy the generated command from its CI/CD tab, and replace the literal token with a CI secret.

Top comments (0)