DEV Community

bigkijimon
bigkijimon

Posted on

Approve the hash, not the button: sealing what an AI coding CLI is about to send

#ai #javascript #opensource #security

Can you say, precisely, what Claude Code or Codex carried off your machine on its last run?

I couldn't. The logs name the files it read. They do not record which lines of those files went out, to which model, or what preprocessing they went through on the way. That gap costs nothing while everything works. It costs everything exactly once.

I built a desktop app around that problem and have just made it public under Apache-2.0:

https://github.com/bigkijimon/bigkiji-universe

This post isn't a pitch. It's about one mechanism inside it — the disclosure manifest — which I think is reusable outside this project, so here it is with the code.

The gap

Handing work to an external AI CLI usually looks like this:

  1. you assemble a prompt
  2. the CLI reads files
  3. something goes to an API
  4. a result comes back

There is no seam between 2 and 3 that a human can inspect. "The thing about to be sent" does not exist after it has been sent.

You might think logging solves it. Logging is a record of what already left. To have any say, you need to stop before the send, look at the contents, and then permit it. A record you read afterwards and an object you approve beforehand are not the same thing.

Making the seam

The idea is small: seal the exact thing you're about to send behind a hash, and make the human approve that hash.

The whole file is 46 lines (src/domain/pi-core/security/disclosure-manifest.js); the sealing function is these twelve:

function createDisclosureManifest({ runId, provider, purpose, policy, slices,
                                    redactions, estimatedTokens, payload,
                                    externalTools, model }) {
  const files = slices.map((item) => {
    const absolute = path.resolve(policy.vaultRoot, item.path);
    return {
      path: item.path.replace(/\\/g, '/'),
      ranges: item.ranges || [],
      sha256: fileHash(absolute),
    };
  });
  const base = {
    version: 2, runId, provider,
    model: String(model || ''),
    purpose: String(purpose || '').slice(0, 240),
    files,
    redactions: redactions.map(({ type, count }) => ({ type, count })),
    externalTools: normalizeExternalTools(externalTools),
    estimatedTokens: Number(estimatedTokens || 0),
    payloadHash: sha(String(payload || '')),
    policyHash: policy.security.policyHash,
  };
  return { ...base, disclosureHash: sha(JSON.stringify(base)) };
}
Enter fullscreen mode Exit fullscreen mode

What each field is doing there:

Field Why it is in the seal
files[].sha256 + ranges A filename is not enough. This pins which lines went
model So "let Opus read these files" cannot become approval for a different model
payloadHash A fingerprint of the assembled body itself
policyHash Which sandbox rules it was built under
redactions What was masked and how much — types and counts only, never contents
externalTools For anything leaving via the broker, the verbatim query

All of it goes into one JSON object, and the SHA-256 of that object is the disclosureHash.

That hash is what the owner approves.

Why a hash rather than a button

This is the part I actually care about.

A normal confirmation is "Run this? yes / no". That is consent to the screen at the moment you pressed it. If anything changed between the press and the spawn, what did that consent cover?

So the spawn side re-verifies before it starts anything:

if (policy.security?.policyHash !== task.disclosure?.policyHash)  throw new Error('STALE_SECURITY_POLICY');
if (!verifyDisclosureManifest(task.disclosure, policy, task.preparedPrompt)) throw new Error('STALE_DISCLOSURE_MANIFEST');
if ((task.disclosure.model || '') !== (task.model || ''))         throw new Error('STALE_MODEL_SELECTION');
Enter fullscreen mode Exit fullscreen mode

Verification re-reads the files and re-hashes them:

function verifyDisclosureManifest(manifest, policy, payload = '') {
  if (!manifest || manifest.policyHash !== policy.security.policyHash) return false;
  if (!manifest.payloadHash || manifest.payloadHash !== sha(String(payload || ''))) return false;
  return manifest.files.every(
    (item) => fileHash(path.resolve(policy.vaultRoot, item.path)) === item.sha256
  );
}
Enter fullscreen mode Exit fullscreen mode

If a single byte moved between approval and launch, the run is refused rather than started. Nothing runs on "it's probably still the same". This is the one place I refused to be optimistic.

STALE_MODEL_SELECTION is separate on purpose, because a model swap is the change that happens most quietly. Falling back to a cheaper model is often correct operationally — but it is an unapproved execution. Refuse it and ask again.

Shrink the payload before sealing it

A manifest is only honest if the thing it seals is small. Hashing everything you sent proves only that you sent everything.

The pruner ahead of it defaults to (src/domain/pi-agent/context-pruner.js):

  • 10 files
  • 48,000 characters
  • 12,000 tokens

and takes ±24-line slices around relevant regions — line ranges, not whole files. The manifest's ranges come from there.

Being straight about this: there is no "we cut context by N%" number in this repository. There is no benchmark in it, so the README states no percentage and neither does this post. For a tool like this, only claiming what you measured seems like the minimum.

What the spawned process does not get

Once approved, the child gets a deliberately thin environment:

  • a private, throwaway 0700 HOME and TMPDIR
  • only that provider's key. No other vendor's credentials
  • Claude Code runs with --strict-mcp-config, an empty MCP config and --disallowed-tools WebSearch,WebFetch,mcp__.*
  • Codex runs --ephemeral --ignore-user-config with web search disabled
  • work happens in an isolated git worktree that cannot merge, commit or push

Tool calls pass a PreToolUse hook that denies every web tool and every mcp__*, and allows only an allowlisted shell subset — no pipes, no redirection, no networking binaries. One brokered path out. One path is a thing you can watch.

What building it taught me

A sandbox hides what you need as well as what you meant to hide.

After I gave every task a throwaway HOME, Claude Code started answering Not logged in · Please run /login. On macOS, security looks for the login keychain under $HOME/Library/Keychains — so replacing HOME had hidden the store the credential lives in, not just a file. Codex returned 401.

That was the whole explanation for 27 assignments and zero paid completions. The providers were never broken. They were never authenticated.

The fix was not to scatter secrets around. It was to lend, by absolute path, the one file that CLI cannot start without — read-only, dying with the task.

A bonus lesson from going public

Fixing CI for the launch showed me it had never passed anywhere but macOS. npm ci was refusing an out-of-sync lock file, and behind that first red were several portability bugs stacked up.

The best one: the sandbox check used a different resolver on each side of the comparison.

  • allowed roots: fs.realpathSync
  • candidate path: fs.realpathSync.native

Identical on macOS and Linux. Not on Windows, where the JS implementation leaves 8.3 short names alone and the native one expands them:

allowed:   C:\Users\RUNNER~1\AppData\Local\Temp\...\project
candidate: C:\Users\runneradmin\AppData\Local\Temp\...\project
Enter fullscreen mode Exit fullscreen mode

Judged as different places, so every read inside the sandbox was refused. It fails closed, so it was never a hole — but the app could not read its own working directory on Windows.

Generalised: in any code that handles paths, it is worth asking whether you are comparing two spellings of one place. And note that I did not find this. A CI job I had been ignoring found it — red for four days while still describing itself as a three-OS matrix. A check that does not pass is not a check, however briefly.

The regression guard is a source-level assertion rather than a behavioural one, because a condition that only fires on platforms with short names cannot be pinned by behaviour:

assert.doesNotMatch(sandboxSource, /fs\.realpathSync(?!\.native)/,
  'sandbox-policy must canonicalise through security-policy.canonical');
Enter fullscreen mode Exit fullscreen mode

(It failed on its first run by matching my own comment. It strips comment lines now.)

Takeaways

  • Build a seam you can approve beforehand, not a log you read afterwards. If that seam is one hash, it is small enough for a human
  • Have the executor re-verify that what was approved is what is about to run. Never "probably the same"
  • Put model selection inside the approval. It is the substitution that happens most quietly
  • Sandboxes hide what you need too. Assume credentials will vanish and plan for it
  • CI that does not pass is not CI. Fix it and a pile of hidden bugs surfaces at once

The code is all readable. Design history is in docs/architecture.md and docs/v3/; the checks that still fail are written down in docs/known-issues.md rather than papered over.

https://github.com/bigkijimon/bigkiji-universe

Environment: Apple Silicon Mac, Node 24, Electron 43. Apache-2.0, eight runtime dependencies, 61 selftests (Linux passes; Windows does not yet — see known-issues).

Every snippet above is the real thing, at commit a5b19be. Links go straight to it, so you can check the line counts too.

Written by Uma (@bigkijimon), running unattended local-plus-external AI pipelines off a single Apple Silicon Mac.

Top comments (0)