I added CI/CD support to my natural language API testing CLI

#ai #opensource #programming #api

Today I shipped v0.4.0 of Octrafic. Here's what changed and why.

If you haven't seen it before - Octrafic is a CLI tool that lets you test APIs using plain English instead of writing test scripts. First post here.

The problem with the previous version

Octrafic worked well interactively. You'd open the TUI, describe what you want tested, get results. Fine for day-to-day use.

But if you wanted to wire it into a CI/CD pipeline, it just didn't work. The TUI would block, waiting for input that was never coming.

Headless mode

The main addition in v0.4.0 is the new octrafic test subcommand. It runs non-interactively - no TUI, no prompts.

You can either run an existing test file:

octrafic test \
  --url https://api.example.com \
  --path tests/api-tests.json

Or generate and run tests on the fly from a prompt:

octrafic test \
  --url https://api.example.com \
  --spec openapi.json \
  --prompt "test all user endpoints"

Exits with 0 if all tests pass, 1 if anything fails. Easy to plug into GitHub Actions or any other pipeline.

Export to Postman, curl, pytest

Once you've run your tests interactively, you can ask the AI to export them:

export these tests to postman
export tests as a shell script
export to pytest and name it test_users_api.py

The exported files include auth configuration too, so they work out of the box. Useful if you want to run them outside of Octrafic or commit them to your repo.

/save command and temp project indicator

Small thing, but it was causing real confusion - you'd close the session and lose everything because you didn't realize the project wasn't saved. There's now a clear indicator in the UI and a /save command to make it explicit.

Still a solo project, still early days. But it's getting more useful with every release.