probatum — your tests pass. does it actually boot?

probatum is a test-oriented check runner I build (in Rust) — one config file, embedded checks (curl, grep and process supervision built in), and only the failures that matter surfaced. This post stands on its own: it's about why I wrote it next to cidx, my CI/CD runner.

Here's the itch. Your test suite is green. 142 tests pass. You tag, you deploy. And the app doesn't boot — because the tests mocked the store, and the real boot replays the WAL, and one segment is missing. No test ever touched that path.

That gap — between the tests pass and it actually stands up — is the whole reason probatum exists.

What probatum checks (that tests don't look at)

One file, probatum.yaml. Flat checks, no logic. The curl, the grep, the process supervision are embedded — you just declare the rules that make a check pass or fail.

# start the real service, wait until it answers, keep it alive for what follows
- name: api boots
  run: ./target/debug/myapp --port 8080
  ready: http://127.0.0.1:8080/healthz
  timeout: 15

# embedded curl
- get: http://127.0.0.1:8080/api/version
  expect: 200
  contains: ['"version"']

# embedded grep — only the lines written DURING this run
- name: app log is clean
  log: /var/log/myapp/app.log
  absent: ["ERROR", "panic"]

probatum starts your real system, interrogates it, reads its logs — then kills the whole process tree on every exit path: normal end, probatum's own panic, Ctrl-C, SIGTERM. If probatum dies, nothing it started survives. No zombie port between runs, no orphan server hanging around.

And the contract that every hand-rolled bash script is missing:

failed (exit 1) ≠ couldn't observe (exit 2).

A missing binary, an unreachable URL, a dirty environment — that's not a bug in your code, that's "I couldn't look." Conflating the two makes you chase ghosts. probatum refuses an already-dirty environment (the port answers before the service starts) instead of destroying it: it never purges what it doesn't own.

The rest is the noise filter: panic, traceback, FATAL caught by default; and humble when unsure — it shows the tail of the output and says "exit 1", it never invents a cause. A false cause is worse than no cause.

Why not just a cidx stage?

The real question. I already maintain cidx, a declarative CI/CD runner: it runs tools against my code (trivy, gitleaks, go-test), identically local and in CI, grouped into phases and pipelines. So why a second tool?

The honest admission first: both can run an arbitrary command. A cidx stage could script the same checks. But they don't verify the same object:

  cidx  ──▶  the CODE, statically
            (which scanners/linters/tests run, local == CI)

  probatum ──▶ the RUNNING SYSTEM, dynamically
            (boot, readiness, HTTP behavior, log window, teardown)
            ← cidx has none of that machinery

The sharing rule is one line: run a tool against my sources → cidx stage. Start my system and observe its behavior → probatum.

So no merge. probatum joins the roster of tools cidx orchestrates — like trivy, like go-test — linked by a preset. One source of truth, two launchers:

  probatum.yaml  ← one file
      │
      ├── inner loop (a dev or an agent, in seconds) : probatum run
      └── outer loop (CI, pipeline)                  : cidx run test

And the guardrail — the actual design decision: if probatum grows phases and pipelines, it's becoming cidx → refuse. If cidx grows readiness and process ownership, it's becoming probatum → same. The boundary isn't an accident, it's held on purpose. Two small tools that each do one thing and keep their edge.

The proof engine with nothing to prove

Confession: probatum didn't start like this. It was born a "proof engine" — promises, oracles, an evidence registry, the whole vocabulary of formal verification. Impressive on paper.

I cut it. That vocabulary was overhead, not value — grandiosity around a simple need: one line, one file, run my checks, show me only what matters. A proof engine that, in the end, had nothing to prove — just something to make useful. (Yes, the pun is cheap. It's also exactly what happened.)

Dropping the word "proof" is the same discipline as everywhere in these projects: name the limit, throw out the vocabulary that inflates. A tool earns its words; it doesn't borrow them from theory to look impressive.

The real first reader is the agent

What drove the cut: probatum's first reader isn't a human scanning a terminal. It's an AI agent (and the human over its shoulder).

An agent chaining fifty ad-hoc bash runs drowns in output and piles up orphan servers on port 8080. probatum hands it one verdict (probatum run --json) and owns everything it starts — so you can loop it tight without cleaning up behind it. That's the differentiator versus "just run bash": process ownership, the collapsed verdict, and the evidence trail to replay. A misspelled YAML key is an error, never a silently skipped check — because an agent believing it tested something it didn't is the worst lie of all.

Where it actually stands

probatum is twelve days old. v0.1.0, 21 commits, a ~1 MB static binary, a 14 MB image, zero external contributors, zero users besides me. It's not on crates.io — I haven't decided to claim the name yet.

What's solid today: the contract is frozen (sources × flat rules, exit 0/1/2, process ownership on every exit path), it tests itself — the root probatum.yaml builds, lints, runs the demo end-to-end and asserts the negative scenarios are actually caught — and it runs inside cidx 2.1.0 as a test-phase preset. The two dogfood each other: cidx's pipeline runs inside probatum, probatum ships as a cidx preset. Each has already found bugs in the other.

What's not done: everything else. This is a v0.1.0, and this post is about its boundary, not its maturity.

The ecosystem I'm aiming at

The map, then: cidx orchestrates the outer loop, probatum verifies the running system in the inner loop. They dogfood each other — cidx's pipeline runs inside probatum, probatum ships as a cidx preset — both are built for the agent, and each refuses to become the other.

It's not a platform. It's a toolchain where every link does one thing, stays replaceable, and is honest about its edge. The bet isn't in the size of each tool — it's in the boundaries between them, held on purpose. probatum is one more link: the one that answers, once the tests are green, the only question left — sure, but does it actually stand up?

French original on arcker.org.