"Source of the Unknown" proposed three metrics for measuring how much fog is in an AI-consumed spec — and closed with a deferral: "implementation details are the subject of a separate, purely engineering piece."
This is that piece. Over 25 days (July 3–27, 2026), the methodology became Tumanomir — a Go CLI, v0.1.0-dev, 66 commits, zero reverts, and no dependency besides gopkg.in/yaml.v3. It deterministically gates its own specification in CI.
Five commands, real terminal output
check runs the deterministic layer instantly, no network:
$ tumanomir check docs/requirements.md
K_drift: 0.00 [ok] (threshold 0.20, 0/33 requirements untraced)
D_const: 0.03 [warn] (threshold 0.35, 101 markers / 3905 prose tokens)
D_pair: — (stochastic layer: run `tumanomir measure` with an instrument)
That's Tumanomir dogfooding — measuring its own spec. $D_{const}$ landing in [warn] isn't a bug: the metric is architecturally incapable of blocking the gate (REQ-CHK-06). It's a lexical proxy, not ground truth.
measure runs the stochastic layer against a live model:
$ tumanomir measure --instrument ollama:qwen3-coder:30b \
-n 3 --temp 1.0 --sim-threshold 0.95 \
docs/investigation/_sanity/specs/sharp.md
D_pair: 0.21 [ok] (95% CI [-0.00, 0.21]; threshold 0.30, mean sim 0.79, N=3 valid, 0 discarded)
H: 1.58 bits (ordinal signal only, not gated)
We ran the same instrument, same parameters, three times in a row against the same spec — not hunting for a result, just gathering material for the article. The result wasn't one number:
| Run | Command | D_pair | verdict |
|---|---|---|---|
| 1 | measure |
0.21 | ok |
| 2 | gate --format json |
0.31 | block |
| 3 | gate --explain |
0.42 | block |
At temp=1.0, the same spec and the same instrument swing from "ok" to "block" depending on exactly when you hit Enter. That's instrument noise, not a measurement bug — exactly what the first article warned about, now visible live.
gate combines both layers for CI, and refuses to silently degrade:
gate: --temp was passed but no instrument resolved (no --instrument and
no .tumanomir.yaml instrument: section) — refusing to silently downgrade
to deterministic-only (REQ-GATE-02)
Architecture that earns trust, not just correctness
The four deterministic packages (internal/metrics, internal/spec, internal/config, internal/calibrate) have no right to touch the network — enforced by a test, not a comment:
$ go test ./internal/ -run TestNoNetworkImports -v
--- PASS: TestNoNetworkImports (0.05s)
The first $K_{drift}$ implementation used regexp and scaled allocations 1:1 with requirement count (3260 allocs/op on a 1MB corpus). A rewrite as a hand-written byte scanner (PR #68) brought that to 14 allocs/op — independent of requirement count, full check on 1MB in 16.7ms.
What we found that we didn't expect
A test isolating naming noise from structural divergence found that on three fixtures varying only identifier text over one fixed structure, $D_{pair} = 0.6667$. Two-thirds of the similarity space was consumed by name choice alone. On the empty-body Go type skeletons Tumanomir's prompt generates, this means a meaningful share of $D_{pair}$'s signal near the 0.30 blocking threshold can be naming-style noise rather than structural ambiguity — an open Phase-2 question, not a fixed conclusion. Two fixes were explicitly rejected and documented in the test: embedding-based synonym matching (reintroduces model non-determinism) and Levenshtein distance (catches typos, not synonyms).
What's still unresolved
No real outcome-labeled calibration corpus exists yet — the threshold values (0.20 / 0.35 / 0.30) remain hypotheses from the first article, not calibrated numbers. $D_{pair}$ is instrument-relative, not document-relative: v0.1 gates on an absolute value at a stress-test temperature (1.0), not a delta against a calibrated baseline at working temperature — a declared, not hidden, divergence (REQ-CFG-01). $D_{const}$ is still a lexical proxy; the RFLP graph from the first article doesn't exist. $K_{drift}$ is strict-mode only. Only one instrument (Ollama) has been tested, and only one projection (Go type definitions).
Full write-up, all five command outputs, and the complete ladder of claims separating "implemented" from "hypothesis": tumanomir on my blog. Code: github.com/valpere/tumanomir.
Valentyn Solomko — Ukrainian software engineer
Top comments (0)