DEV Community

Cover image for How to Run API Tests Without Tavern's YAML Boilerplate
Hassann
Hassann

Posted on • Originally published at apidog.com

How to Run API Tests Without Tavern's YAML Boilerplate

#python #testing #tutorial #api

Tavern is useful when your API tests already live in a Python/pytest workflow. You write a test_*.tavern.yaml file, pytest discovers it, and you keep pytest features like fixtures, plugins, pytest-xdist, coverage, and the same CI command your team already uses.

Try Apidog today

The tradeoff appears as your API flows grow. A request that sends a body, stores a token, and validates several response fields becomes a deeply nested YAML block with request, response, save, and verify_response_with sections. Multi-step flows such as login → create → read → delete can become long files where indentation errors break parsing, while the actual API contract lives somewhere else: OpenAPI, Swagger, a wiki, or an old Postman collection.

What Tavern does well

Tavern is a solid tool if your team already works in pytest.

It runs inside pytest

Install Tavern:

pip install tavern
Enter fullscreen mode Exit fullscreen mode

Add a Tavern test file to your test directory, then run:

pytest
Enter fullscreen mode Exit fullscreen mode

Because Tavern is a pytest plugin, you keep familiar pytest behavior:

pytest -k orders
pytest -x
pytest -n auto
Enter fullscreen mode Exit fullscreen mode

You can also use pytest fixtures, reporting plugins, and CI configuration you already have.

Simple tests are readable

A basic Tavern test is easy to scan:

test_name: Get a single order

stages:
  - name: Fetch order 1042
    request:
      url: https://api.shop.test/orders/1042
      method: GET
    response:
      status_code: 200
      json:
        id: 1042
        status: shipped
Enter fullscreen mode Exit fullscreen mode

For a status-code check and a few response fields, this is cleaner than writing Python with requests and assert.

It supports value saving and chaining

Tavern can extract data from one response and reuse it in another request:

stages:
  - name: Log in
    request:
      url: https://api.shop.test/auth/login
      method: POST
      json:
        username: tester
        password: hunter2
    response:
      status_code: 200
      save:
        json:
          token: access_token

  - name: Read the profile with the saved token
    request:
      url: https://api.shop.test/me
      method: GET
      headers:
        Authorization: "Bearer {token}"
    response:
      status_code: 200
Enter fullscreen mode Exit fullscreen mode

That makes common flows like login → authenticated request possible without custom Python code.

It supports more than HTTP

Tavern also supports MQTT, which can matter for IoT or message-driven systems. Since pytest is still underneath, you can run Tavern YAML tests and Python tests in the same test suite.

Where Tavern gets harder to maintain

Tavern’s main cost is not the runner. It is the authoring model.

YAML becomes brittle in large flows

The simple example above is readable. A production test often includes:

  • Headers
  • Query parameters
  • Request bodies
  • Response-body assertions
  • Type checks
  • Saved variables
  • External validation functions with verify_response_with

That quickly becomes four or five indentation levels deep. One misplaced space can fail parsing before the test even reaches your API.

You also do not get the same autocomplete and refactoring support you would get from an IDE working with typed code. You need to remember whether a key is status_code or status, json or body, and where each block belongs.

The test and API contract are separate

A Tavern file hardcodes the endpoint, method, request data, expected fields, and expected response shape.

Your actual API definition may live in:

  • An OpenAPI file
  • Swagger docs
  • Framework route decorators
  • A separate API documentation tool
  • A manually maintained wiki

When the API changes, Tavern does not automatically know. The YAML keeps asserting the old contract until CI fails, or worse, until stale assertions continue to pass while the documented contract drifts.

It assumes Python knowledge

Tavern works best for Python-heavy teams. If QA, frontend engineers, or product-adjacent team members need to read or add API tests, they must deal with:

  • Python installation
  • pip
  • Virtual environments
  • pytest conventions
  • Tavern’s YAML schema

That can be too much overhead for someone who only wants to send an HTTP request and check the response.

A spec-first alternative: build tests from the API definition

Instead of writing API tests as separate YAML files, you can build them from the API definition itself.

In Apidog, the API spec, request, and test scenario are connected. You can import or design an API from OpenAPI, Swagger, or a Postman collection, then build scenarios by chaining real endpoints.

A typical flow looks like this:

  1. Import or create your API definition.
  2. Add the login endpoint to a scenario.
  3. Save the returned token.
  4. Add an authenticated endpoint.
  5. Pass the token into the next request.
  6. Add assertions against the expected response.
  7. Run the scenario locally or in CI.

The key difference is that the test is built against the spec. The spec becomes the source of truth instead of something separate from the test file.

When you need CI automation, the Apidog CLI runs the same scenarios from the command line.

Install and run the Apidog CLI

Install the CLI globally:

npm install -g apidog-cli
Enter fullscreen mode Exit fullscreen mode

Run a test scenario:

apidog run --access-token $APIDOG_ACCESS_TOKEN -t 605067 -e 1629989 -r cli
Enter fullscreen mode Exit fullscreen mode

What each option does:

  • --access-token: authenticates the run against your Apidog account. Store it as a CI secret.
  • -t: test scenario ID.
  • -e: environment ID, such as dev, staging, or prod.
  • -r: reporter type. cli prints output in the terminal.

You do not need to manually find every ID. In Apidog:

  1. Open the test scenario.
  2. Go to the CI/CD tab.
  3. Choose the command-line option.
  4. Generate a token.
  5. Copy the generated command.
  6. Move the token into your CI secret store.

If you do not want a global install, use npx:

npx apidog-cli run --access-token $APIDOG_ACCESS_TOKEN -t 605067 -e 1629989 -r cli
Enter fullscreen mode Exit fullscreen mode

For CI, generate machine-readable and human-readable reports:

apidog run \
  --access-token $APIDOG_ACCESS_TOKEN \
  -t 605067 \
  -e 1629989 \
  -r html,junit \
  --out-dir ./apidog-reports
Enter fullscreen mode Exit fullscreen mode

Useful reporters include:

  • cli: terminal output
  • html: browsable report
  • json: structured output
  • junit: XML output for CI dashboards

To inspect the flags supported by your installed version:

apidog run --help
Enter fullscreen mode Exit fullscreen mode

Tavern vs Apidog

Tavern Apidog
Test format YAML files (*.tavern.yaml) Visual scenarios built against the spec
Runtime Python + pytest Headless apidog run npm package
Authoring Hand-written YAML Chained endpoints and assertions
API contract Separate artifact, manually synchronized Spec is the test source of truth
Variable chaining save: blocks and {var} substitution Pass values between scenario steps
Reporters pytest-html, JUnit through pytest plugins cli, html, json, junit built in
Who can write tests Python-comfortable testers Developers, QA, and non-Python contributors
Protocols HTTP and MQTT HTTP, SOAP, WebSocket, gRPC, and more

Use Tavern if your team is fully invested in Python and wants API tests to live beside unit tests in the same pytest run.

Use Apidog if you want to reduce YAML maintenance, keep the test aligned with the API contract, and let more team members contribute to API testing.

Run Apidog tests in GitHub Actions

Here is a minimal GitHub Actions workflow:

name: API tests
on: [push, pull_request]

jobs:
  api-tests:
    runs-on: ubuntu-latest
    steps:
      - name: Install Apidog CLI
        run: npm install -g apidog-cli

      - name: Run API test scenario
        run: |
          apidog run \
            --access-token "$APIDOG_ACCESS_TOKEN" \
            -t 605067 \
            -e 1629989 \
            -r html,junit \
            --out-dir apidog-reports
        env:
          APIDOG_ACCESS_TOKEN: ${{ secrets.APIDOG_ACCESS_TOKEN }}

      - name: Upload report
        if: always()
        uses: actions/upload-artifact@v4
        with:
          name: apidog-report
          path: apidog-reports
Enter fullscreen mode Exit fullscreen mode

Important details:

  • Store APIDOG_ACCESS_TOKEN as a repository secret.
  • Use if: always() so the report uploads even when tests fail.
  • Use junit so your CI system can parse test results.
  • Use html so developers can inspect a readable report.
  • Let the command exit non-zero when assertions fail.

That exit code is the CI quality gate. If a scenario fails, apidog run exits non-zero, the job fails, and the pull request shows a failing check.

For more setup examples, see Apidog CLI in your CI/CD pipeline and the GitHub Actions walkthrough.

Keep pytest where it fits

Moving API-contract scenarios out of Tavern does not mean removing pytest.

A practical split is:

  • Use pytest for Python unit tests and lower-level integration tests.
  • Use Apidog for API scenarios that should track the contract and be readable by non-Python contributors.

The tools are not mutually exclusive. Tavern’s strength is bringing API tests into pytest. Apidog’s strength is keeping API tests tied to the API definition as the suite grows.

If you are comparing other API runners, see Postman CLI vs Newman and API testing without Postman.

FAQ

Is the Apidog CLI free?

Yes. The apidog-cli npm package is free to install and run:

npm install -g apidog-cli
Enter fullscreen mode Exit fullscreen mode

It executes test scenarios from your Apidog project, so what you can run depends on your Apidog plan. The command-line runner itself is not a separate paid product.

Can I still use pytest alongside Apidog?

Yes. Keep Python unit and integration tests in pytest, and use Apidog for the API-contract layer. Many teams run both.

How does Apidog block a bad merge in CI?

Through the process exit code. If any assertion fails, apidog run exits non-zero. CI marks the step as failed and blocks the merge or deploy according to your branch protection rules.

Which reporter should I use in CI?

Use junit for CI dashboards and html for a browsable artifact:

apidog run \
  --access-token $APIDOG_ACCESS_TOKEN \
  -t 605067 \
  -e 1629989 \
  -r html,junit \
  --out-dir ./apidog-reports
Enter fullscreen mode Exit fullscreen mode

Does Apidog only test HTTP?

No. Apidog covers HTTP plus SOAP, WebSocket, Server-Sent Events, and gRPC. Tavern supports HTTP and MQTT, so keep Tavern in mind if MQTT testing is central to your stack.

Takeaway

Tavern is a good fit when your team already works in pytest and wants API tests in YAML next to Python tests. Its main cost is long-term maintenance: deeply nested YAML grows brittle, and the test file is separate from the API contract it verifies.

If you want headless CI execution without hand-maintaining YAML and a separate copy of your API spec, build tests against the contract and run them with apidog run.

Download Apidog and import an existing API to try the spec-first workflow.

Top comments (0)