TL;DR: Unit tests missed nine CLI journey defects; run the tool from zero state, using only its published instructions. The parent slice log is a reminder to test the artifact people actually touch.
Your command works on your machine, your unit tests are green, and a new operator still cannot get through the first run. Can Ranex gate Ranex for real?
In this note
- The CLI is where your system becomes real
- Nine defects appeared only in the journey
- Drive these shapes like a new operator
- Approval reduces hidden change. It cannot make code truthful.
- Run your own first-day test
Your tool lives at its CLI surface. That is where paths, locks, machines, empty state, instructions, and human assumptions collide.
SLICE-006 found nine defects by driving the Ranex CLI as a person does. None came from a unit test. That is not an argument against unit tests. It is an argument against asking them to prove a journey they never take.
The CLI is where your system becomes real
Your tool’s real failure modes live where a person invokes it. A unit test can prove a component; it cannot automatically prove that setup, documentation, state, and the command path join into a usable and truthful run.
Ranex had a specific problem. It materialises committed blobs to observe the subject tree. That is the right subject, but ignored directories such as .venv and node_modules are not in that tree. The bound test command needed dependencies and a resolver that the observed party could not choose from an ambient writable path.
So the question became practical: can Ranex gate Ranex for real?
The answer built in SLICE-006 has four deliberate steps. deps fetch derives the lock clean under pinned inputs and byte-compares it with the committed lock. Only SHA-256-addressed wheels enter the store. deps approve records the named package delta an approver accepted. Before anything runs, a fresh environment is assembled from the verified store entries and made read-only. Then run executes sealed and offline.
The gate command remains the catalog-bound uv run pytest -q. The environment is assembled from verified store entries, made read-only before spawn, and set so the command cannot sync, build, or fetch.
That is a dependency process, not a truth machine. A hash tells you which bytes arrived. Approval tells you which package change a person accepted. Neither tells you that dependency code will behave honestly once it runs.
Nine defects appeared only in the journey
Nine defects were found by driving the CLI as a person does, and none by a unit test. They appeared at the joins: real locks, cold-start state, operator instructions, CI, and the fresh repository used for observation.
The first defect is a familiar one. Plain uv run pytest -q re-locked and rewrote uv.lock. In this system, that lock is a trust root. After a clean re-lock, the rewrite silently dropped the resolution epoch block. Then deps fetch refused the committed lock against its own clean derivation.
The fix had two parts. The gated run sets UV_FROZEN=1, while the repository’s own commands use uv run --frozen. The difference is not cosmetic. The gated argv must stay exactly as the catalog binds it.
Other journey failures had the same practical flavor. A real lock held one package at several versions, and the parser rejected it as corruption. A check for a fabricated hash was passing for the wrong reason because the resolution epoch was omitted. CI was also invoking uv unfrozen, silently mutating the same trust root where it would be hardest to notice.
Then the cold-start journey caught product-facing defects. keygen told a first-time operator to create an invalid keyring. The README walkthrough had rotted: it omitted the fetch and approval steps, failed to name installation of the pinned resolver, and described a requirement that had been removed. Another journey re-entered itself inside the materialised sample because its recursion guard lived in an environment that the sample deliberately built from empty.
None of those defects is a tiny detail to the person blocked by it. The tool either works from zero state or it does not.
Drive these shapes like a new operator
Test the command surface as a stranger with no warm cache, no hidden setup, and no memory of why the tool works. The checklist is not “add more end-to-end tests.” It is a list of states your unit suite is built to avoid.
- Start from zero state: no generated keyring, no provisioned store, no prior approval, no warm cache.
- Follow the README command by command. Confirm every command it requires is present, ordered, and still names current behavior.
- Run with the real committed lock and manifest, then deliberately alter a package, graph edge, URL, or hash and require byte comparison to refuse.
- Use the pinned resolver path and prove a user-writable resolver is rejected.
- Observe the dependency run offline. A network attempt must deny and produce no evidence.
- Exercise the tool in CI, where a separate command spelling can rewrite a trust root without a developer noticing.
- Run inside every clean-room or materialised repository shape the tool creates for itself.
- Force an ordinary success path too. A refusal-only test can pass because the command never ran.
The last item is non-negotiable. SLICE-006 records an actual dependency-bearing execution that provisions once, reuses the store without downloads, runs offline, signs evidence, and evaluates PASS. A system that only proves its refusals is an outage wearing responsible clothes.
You do not need a large product to borrow this method. Take one onboarding command. Use a fresh environment. Follow your own docs without filling gaps from memory. Every place you have to “just know” something is a candidate defect.
Approval reduces hidden change. It cannot make code truthful.
An approved, hash-correct wheel can still choose its own exit code. SLICE-006 demonstrates this with tests/security/test_slice006_approved_wheel_can_lie.py, where an approved wheel forces a passing verdict.
That test is labelled not caught. It should be. Calling integrity proof would be a lie.
Python makes the route concrete through installed pytest11 entry points, but the boundary is larger than one plugin mechanism. Direct imports also execute dependency code. The dependency is part of the trusted computing base for that run.
This is why an accountability apparatus needs its limits stated plainly. You can reduce hidden change with a clean derivation, pinned inputs, SHA-256-addressed wheels, a reviewable package delta, and sealed offline execution. You cannot derive truthful behavior from a package hash.
Ranex is pre-release. This slice closes a runnable self-gate for its Python dependency path. Other ecosystems need their own manifest, lock, and artifact policies. No generic abstraction is claimed here.
Also, the lock story is worth keeping in your operating memory. A lockfile is not boring generated clutter when it decides what code enters a measured run. Plain uv run rewrote uv.lock and silently dropped the resolution epoch once. The later refusal was the control doing its job, not the control being inconvenient.
Questions people actually ask
These answers cover CLI testing from zero state and the limits of unit tests.
What does Ranex gates Ranex mean?
Ranex provisions and runs its own committed suite sealed and offline, then evaluates the signed evidence against the gate.
What does end-to-end CLI testing catch that unit tests miss?
End-to-end CLI testing catches defects in real operator journeys across command setup, lock handling, documentation, cold-start state, and the materialised repository boundary.
Can an approved hash-correct dependency force a passing verdict?
Yes. The SLICE-006 security test demonstrates an approved, hash-correct wheel forcing a passing verdict. Approval reduces hidden change; it cannot make third-party code truthful.
Run your own first-day test
Pick the most consequential command your users run. Clear the state it is meant to create. Open the docs. Execute only what they say. Watch every file it reads, every tool it resolves, every network call it attempts, and every trust root it can rewrite.
Then preserve the journey. Put the defect reproduction beside the code, not only in a ticket. The source records for this work are in the Ranex repository, under docs/slices/done/.
Try it. Break it. Tell me what broke. If the journey catches something your unit suite missed, star the repository and send an honest critique. That is the result worth keeping.
Disclosure: this post was drafted with AI assistance. Every factual claim traces to the repository’s README or slice records — the same fact gate the product enforces on code. It ships only after Anthony’s own review.
Top comments (0)