DEV Community

Cover image for Apidog CLI Test Reports: Generate CLI, HTML, and JSON Output
Hassann
Hassann

Posted on • Originally published at apidog.com

Apidog CLI Test Reports: Generate CLI, HTML, and JSON Output

A test run that prints nothing useful is a test run nobody trusts. Your pipeline goes red, someone opens the build log, and all they find is a wall of timestamps and a non-zero exit code. Which assertion broke? Against which environment? On which row of the data file? The run knew. It just never wrote it down anywhere you could read it later.

Try Apidog today

That is the gap a reporter fills. When you run API tests from the command line, the report is the artifact you actually use: the file your CI dashboard parses, the HTML report you archive, and the output your teammate reads when they were not watching the pipeline fail at 2 a.m.

The Apidog command-line runner handles this with one flag: -r, --reporters. You pass a comma-separated list of report formats, and the runner writes each one from the same test execution.

Apidog ships a CLI that runs the test scenarios you built visually in the app. This guide focuses on the reporter flag, the files it produces, where they land, and how to wire them into CI. For the full CLI surface, see the apidog run command reference.

Why reports matter in CI

When you run a scenario locally, you can watch each request, assertion, and summary in real time.

In CI, that feedback loop disappears. The run happens on a remote machine, and the only useful record is what the runner wrote before exiting.

A useful API test report should tell you:

  • Which scenario ran
  • Which environment it used
  • How many iterations ran
  • Which assertions passed or failed
  • Which request and response caused the failure
  • Which data row or iteration failed in data-driven tests

Without that, a failing pipeline becomes a reproduction exercise. With it, debugging usually starts from the failed assertion and response body.

Install the Apidog CLI

The Apidog CLI is available as the npm package apidog-cli.

Install it globally:

npm install -g apidog-cli
Enter fullscreen mode Exit fullscreen mode

Then run test scenarios with:

apidog run
Enter fullscreen mode Exit fullscreen mode

You can also use npx on ephemeral CI runners:

npx apidog-cli run
Enter fullscreen mode Exit fullscreen mode

The reporter behavior is the same either way.

The reporter flag

All report formats are controlled by one flag:

-r, --reporters
Enter fullscreen mode Exit fullscreen mode

It accepts a comma-separated list of reporters:

Reporter Use it for
cli Human-readable terminal output
html Shareable browser report
json Custom scripts and post-processing
junit CI dashboards and test result widgets

If you do not pass -r, the default is:

-r cli
Enter fullscreen mode Exit fullscreen mode

A typical run with multiple reporters looks like this:

apidog run \
  --access-token "$APIDOG_ACCESS_TOKEN" \
  -t 605067 \
  -e 1629989 \
  -n 1 \
  -r html,cli
Enter fullscreen mode Exit fullscreen mode

This command:

  • Authenticates with APIDOG_ACCESS_TOKEN
  • Runs test scenario 605067
  • Uses environment 1629989
  • Runs once with -n 1
  • Produces both HTML and CLI output

You can copy the scenario ID, environment ID, access token, and generated command from the scenario’s CI/CD tab in Apidog. If you need the full setup flow, see the Apidog CLI complete guide.

The important part: one execution can produce multiple reports. You do not need to rerun the same test to generate HTML, JUnit, and CLI output.

cli: the report in your build log

The cli reporter prints a readable summary to the terminal.

apidog run \
  --access-token "$APIDOG_ACCESS_TOKEN" \
  -t 605067 \
  -e 1629989 \
  -r cli
Enter fullscreen mode Exit fullscreen mode

Use it for the first layer of debugging in a CI job log. It helps you quickly see:

  • How many requests ran
  • How many assertions passed
  • How many assertions failed
  • Which assertion failed
  • Which endpoint or step was involved

Keep cli enabled even when you also emit machine-readable formats:

apidog run \
  --access-token "$APIDOG_ACCESS_TOKEN" \
  -t 605067 \
  -e 1629989 \
  -r cli,junit \
  --out-dir ./apidog-reports
Enter fullscreen mode Exit fullscreen mode

A pipeline that emits only JUnit XML may produce a useful dashboard but an unhelpful raw log. Adding cli keeps the job output readable.

Two flags affect console output:

--verbose
Enter fullscreen mode Exit fullscreen mode

Use --verbose when you need full request and response detail in the terminal:

apidog run \
  --access-token "$APIDOG_ACCESS_TOKEN" \
  -t 605067 \
  -e 1629989 \
  -r cli \
  --verbose
Enter fullscreen mode Exit fullscreen mode

Use this when a scenario passes locally but fails in CI and you need to inspect what the runner actually sent and received.

--silent
Enter fullscreen mode Exit fullscreen mode

Use --silent when you want to suppress console output and rely on files and exit codes:

apidog run \
  --access-token "$APIDOG_ACCESS_TOKEN" \
  -t 605067 \
  -e 1629989 \
  -r junit \
  --silent \
  --out-dir ./apidog-reports
Enter fullscreen mode Exit fullscreen mode

html: the report you share with humans

The html reporter writes a self-contained HTML file.

apidog run \
  --access-token "$APIDOG_ACCESS_TOKEN" \
  -t 605067 \
  -e 1629989 \
  -r html \
  --out-dir ./apidog-reports
Enter fullscreen mode Exit fullscreen mode

Open the file in a browser to inspect:

  • Each request
  • Assertions for each step
  • Pass/fail status
  • Request details
  • Response details
  • Iteration results for data-driven runs

Apidog HTML report

This is the format to archive as a CI artifact. It is useful when someone asks “what failed?” and you want to send them one file instead of asking them to install the CLI or rerun the scenario.

HTML is especially useful for data-driven API tests. If one scenario loops over a CSV with 50 rows, the HTML report can show which iteration failed. That is much faster than debugging from a single pass/fail count.

For more on that workflow, see data-driven API testing with CSV and JSON.

Use HTML for people. Do not scrape it in scripts. Use JSON or JUnit for automation.

junit: the report your CI dashboard parses

The junit reporter emits XML in standard JUnit format.

apidog run \
  --access-token "$APIDOG_ACCESS_TOKEN" \
  -t 605067 \
  -e 1629989 \
  -r junit \
  --out-dir ./apidog-reports
Enter fullscreen mode Exit fullscreen mode

Use JUnit when you want your CI platform to render API test results in its native test UI.

Most CI systems understand JUnit XML, including:

  • GitHub Actions
  • GitLab CI
  • Jenkins
  • CircleCI
  • Azure Pipelines

If you choose one machine-readable format for CI dashboards, choose junit.

GitLab CI example 

api-tests:
  stage: test
  image: node:20
  script:
    - npm install -g apidog-cli
    - apidog run --access-token "$APIDOG_ACCESS_TOKEN" -t 605067 -e 1629989 -r junit,cli --out-dir ./apidog-reports
  artifacts:
    when: always
    paths:
      - apidog-reports/
    reports:
      junit: apidog-reports/*.xml
Enter fullscreen mode Exit fullscreen mode

This job:

  1. Installs the CLI.
  2. Runs the Apidog scenario.
  3. Writes CLI output and JUnit XML.
  4. Always uploads the report directory.
  5. Tells GitLab where to find the JUnit XML.

Jenkins example

In Jenkins, point the junit step at the generated XML files:

pipeline {
  agent any

  stages {
    stage('API tests') {
      steps {
        sh 'npm install -g apidog-cli'
        sh 'apidog run --access-token "$APIDOG_ACCESS_TOKEN" -t 605067 -e 1629989 -r junit,cli --out-dir ./apidog-reports'
      }
    }
  }

  post {
    always {
      junit 'apidog-reports/*.xml'
      archiveArtifacts artifacts: 'apidog-reports/**', allowEmptyArchive: true
    }
  }
}
Enter fullscreen mode Exit fullscreen mode

GitHub Actions

In GitHub Actions, generate the report, upload it as an artifact, and optionally use a JUnit-aware action to render the results. For a complete workflow, see running Apidog CLI tests in GitHub Actions.

json: the report your scripts process

The json reporter writes structured test results for custom automation.

apidog run \
  --access-token "$APIDOG_ACCESS_TOKEN" \
  -t 605067 \
  -e 1629989 \
  -r json \
  --out-dir ./apidog-reports
Enter fullscreen mode Exit fullscreen mode

Use JSON when you want to build your own workflow around test results, for example:

  • Send a custom Slack summary
  • Push pass-rate metrics to monitoring
  • Trigger rollback logic
  • Compare today’s result with yesterday’s
  • Extract only failed endpoints
  • Store results in your own database

For large runs, generate a failures-only JSON file:

apidog run \
  --access-token "$APIDOG_ACCESS_TOKEN" \
  -t 605067 \
  -e 1629989 \
  -r json \
  --out-json-failures-separated true \
  --out-dir ./apidog-reports
Enter fullscreen mode Exit fullscreen mode

The relevant flag is:

--out-json-failures-separated <value>
Enter fullscreen mode Exit fullscreen mode

This is useful for regression suites where most steps pass and you only want to diff or inspect failures.

Control output paths with --out-dir

By default, reports are written to:

./apidog-reports
Enter fullscreen mode Exit fullscreen mode

Set a custom directory with:

--out-dir <dir>
Enter fullscreen mode Exit fullscreen mode

Example:

apidog run \
  --access-token "$APIDOG_ACCESS_TOKEN" \
  -t 605067 \
  -e 1629989 \
  -r html,junit \
  --out-dir ./apidog-reports
Enter fullscreen mode Exit fullscreen mode

In CI, keep this path stable so your artifact upload configuration does not need to change between jobs.

Control filenames with --out-file

Use --out-file when you want predictable, non-overwriting report names.

--out-file <name>
Enter fullscreen mode Exit fullscreen mode

The runner supports placeholders:

Placeholder Meaning
{SCENARIO_NAME} Name of the scenario that ran
{FOLDER_NAME} Folder name when running a folder
{GENERATE_TIME} Timestamp generated at write time

Example:

apidog run \
  --access-token "$APIDOG_ACCESS_TOKEN" \
  -t 605067 \
  -e 1629989 \
  -r html \
  --out-file "{SCENARIO_NAME}-{GENERATE_TIME}"
Enter fullscreen mode Exit fullscreen mode

This prevents one run from overwriting the previous report and makes report directories easier to scan.

For folder runs:

apidog run \
  --access-token "$APIDOG_ACCESS_TOKEN" \
  -f 88012 \
  -r html,junit \
  --out-dir ./nightly-reports \
  --out-file "{FOLDER_NAME}-{GENERATE_TIME}"
Enter fullscreen mode Exit fullscreen mode

Upload a report overview to Apidog

Use:

--upload-report [value]
Enter fullscreen mode Exit fullscreen mode

This uploads a report overview to Apidog cloud so the run appears in your project history in addition to any local files written on the CI runner.

Use this when you want results visible inside Apidog itself, not only as CI artifacts.

Recommended reporter combinations

Map reporters to their readers:

Reader Reporter
Developer scanning CI logs cli
Teammate opening a saved report html
CI dashboard junit
Custom script json

For most CI pipelines, use:

-r cli,html,junit
Enter fullscreen mode Exit fullscreen mode

Full command:

apidog run \
  --access-token "$APIDOG_ACCESS_TOKEN" \
  -t 605067 \
  -e 1629989 \
  -r cli,html,junit \
  --out-dir ./apidog-reports
Enter fullscreen mode Exit fullscreen mode

This produces:

  • Readable terminal output
  • A browser-friendly HTML artifact
  • JUnit XML for your CI dashboard

One run, three outputs.

Do not swallow the exit code

Reports explain what happened. The process exit code controls whether the pipeline fails.

The Apidog CLI exits non-zero when an assertion fails. That non-zero exit code is what blocks the build or merge.

Avoid this:

apidog run --access-token "$APIDOG_ACCESS_TOKEN" -t 605067 -e 1629989 -r cli,html,junit || true
Enter fullscreen mode Exit fullscreen mode

That can produce a failed report while still allowing the CI job to pass.

For more on quality gates, see automating API tests in CI/CD.

Putting it together

Run a scenario in CI and produce reports for logs, humans, and the CI dashboard:

apidog run \
  --access-token "$APIDOG_ACCESS_TOKEN" \
  -t 605067 \
  -e 1629989 \
  -n 1 \
  -r cli,html,junit \
  --out-dir ./apidog-reports
Enter fullscreen mode Exit fullscreen mode

Run a nightly folder sweep and keep going after failures:

apidog run \
  --access-token "$APIDOG_ACCESS_TOKEN" \
  -f 88012 \
  -r html,junit \
  --on-error continue \
  --out-dir ./nightly-reports \
  --out-file "{FOLDER_NAME}-{GENERATE_TIME}"
Enter fullscreen mode Exit fullscreen mode

Run a data-driven scenario and keep a failures-only JSON file:

apidog run \
  --access-token "$APIDOG_ACCESS_TOKEN" \
  -t 605067 \
  -d ./accounts.csv \
  -r json \
  --out-json-failures-separated true \
  --out-dir ./apidog-reports
Enter fullscreen mode Exit fullscreen mode

Use npx instead of a global install on temporary runners:

npx apidog-cli run \
  --access-token "$APIDOG_ACCESS_TOKEN" \
  -t 605067 \
  -e 1629989 \
  -r html,junit \
  --out-dir ./apidog-reports
Enter fullscreen mode Exit fullscreen mode

Final checklist

Before adding Apidog CLI reports to CI, decide:

  • Do developers need readable logs? Add cli.
  • Do you need a shareable artifact? Add html.
  • Should the CI dashboard show test results? Add junit.
  • Will custom scripts process results? Add json.
  • Should reports be archived? Set --out-dir.
  • Do you need unique filenames? Set --out-file.
  • Do you need failures-only JSON? Use --out-json-failures-separated.
  • Do you want results in Apidog cloud? Use --upload-report.

Flag names, defaults, and reporters can change between CLI releases, so the installed runner is the source of truth:

apidog run --help
Enter fullscreen mode Exit fullscreen mode

To create the scenario the CLI runs, Download Apidog, build a scenario in the app, copy the generated CI/CD command, and add the reporters your pipeline needs.

Top comments (0)