- Book: AI Agents Pocket Guide
- Also by me: Thinking in Go (2-book series) — Complete Guide to Go Programming + Hexagonal Architecture in Go
- My project: Hermes IDE | GitHub — an IDE for developers who ship with Claude Code and other AI coding tools
- Me: xgabriel.com | GitHub
"Reply with exactly: OK. Do not read or open any files."
That was the whole prompt. The researcher who publishes as cereblab sent it to Grok Build CLI 0.2.93, xAI's terminal coding agent, with mitmproxy sitting in the middle. The model answered. In the background, the CLI also posted a git bundle of the repository to xAI's storage endpoint: 4 commits, 47 files, full history. Cloning the captured bundle gave back a planted file the agent had been told not to open, marker string intact (cereblab's write-up).
A second run, on a repo of about 12 GB of random files the agent never read, put numbers on the gap. About 192 KB went to the model. About 5.10 GiB went to the storage channel, in 73 chunks of roughly 75 MB, every one answered with HTTP 200, before the capture was stopped mid-stream (gist, The Hacker News).
xAI switched the upload off on July 13, 2026 with a server-side flag. What's worth keeping is the method: a proxy and a canary repo, plus a short mitmproxy addon, will show you what any agent CLI sends, Grok or not.
What the capture showed
The setup was ordinary. cereblab used a consumer SuperGrok/X login on macOS, trusted the mitmproxy certificate, and pointed HTTPS_PROXY and SSL_CERT_FILE at the proxy. Grok did not pin its certificates, so every request was readable (gist).
Two channels came out of one binary. Model turns went to cli-chat-proxy.grok.com/v1/responses. Storage went to /v1/storage on the same host, plus a multipart init and a batch_exists check. In a 3 GB run cereblab also saw chunks go straight to a Google Cloud Storage bucket named grok-code-session-traces, though that log wasn't kept. Mixpanel and grok.com event calls rode along too.
The privacy toggle didn't change it. With "Improve the model" turned off, the git-bundle upload still went through with a 200, and the server's /v1/settings response still carried "trace_upload_enabled": true. cereblab's reading: that setting governs training, not upload.
Secrets showed up in both channels. On a real task where the agent did read files, a git-tracked .env holding fake canary values went into a model turn verbatim. The same values also landed in a session_state archive bound for storage (The Hacker News).
One CLI, two channels: a few kilobytes to the model, gigabytes to storage.
The researcher is careful about limits, and so should you be. The capture proves transmission, acceptance and storage. It does not prove xAI trained on anything. The 12 GB run shows multi-gigabyte upload with no failures, not a finished transfer, and only that run's log was kept. Everything was pinned to version 0.2.93, and every "secret" was a fake canary string.
July 13: the server flipped a flag
The same 0.2.93 binary stopped making storage requests on July 13. cereblab retested six times and saw zero /v1/storage uploads; the server now returned disable_codebase_upload: true and trace_upload_enabled: false (The Hacker News).
xAI answered on X. Per The Hacker News, it said enterprise zero-data-retention teams never had code or trace data stored, and that consumers can run /privacy in the CLI to turn off retention and delete synced data. Elon Musk said previously uploaded data would be "completely and utterly deleted." cereblab tested /privacy and concluded it controls retention, not what gets sent (gist).
On July 15 xAI published the Grok Build source, Apache 2.0 for first-party code. Tech Times reported the Cloud Storage upload code still in the published source. Simon Willison found "remnants of the code" there too: upload_session_state() now returns a hard-coded session_state_upload_unavailable error. The upload path is switched off, and it is still in the code.
A flag delivered by the server can be flipped back by the server, with no new binary on your machine and nothing in a changelog. As of The Hacker News report, xAI had not said whether the change covers every account or is permanent, how long uploads were kept, or how many users were affected.
Deny rules stop the agent. The client uploads anyway
cereblab's repro repo has the sharpest detail: a deny rule like Read(secret.txt) did stop the agent from reading the file, and the tracked file was still in the uploaded bundle.
There are two layers in every agent CLI. One is the model's tool calls: read this file, run that command. Your approval prompts and deny rules live there. The other is the client's own network code, which runs whatever the model decides. Nothing in the first layer sees the second.
cereblab's comparison notes also pin down what a bundle carries: tracked files and their history. Gitignored files that were never committed stayed out. Files committed and later deleted came along. If a key ever touched a commit, every bundle of that repo carries it, so rotate the key; deleting the file doesn't take it out of history.
The same notes ran the canary repo through three other CLIs on July 13, one machine and one account each. Claude Code 2.1.204 sent only POST api.anthropic.com/v1/messages. Codex sent its model turn over a WebSocket to chatgpt.com, plus telemetry. Gemini 0.38.2, in API-key mode, showed no bundle in the idle run; its real-task run hit a 429 quota error. None uploaded a repo bundle in these captures. That is four CLIs on one day, and next week's release could behave differently.
Audit any agent CLI with a proxy and a canary
You don't need to take any of this on trust, mine included. Use a throwaway account where you can, as cereblab recommends.
Plant something the agent must never touch, then watch the wire for it.
First, a repo with three canaries: one file the prompt forbids, one secret that lives only in history, and one gitignored secret that was never committed.
set -eu
mkdir canary && cd canary && git init -q
git config user.name canary
git config user.email canary@example.com
mkdir -p src/_probe
echo "CANARY-NEVERREAD-7Q2Z9" > src/_probe/never_read.txt
echo "TOKEN=CANARY-HISTORY-4K8M1" > old.env
git add -A && git commit -qm "init"
git rm -q old.env && git commit -qm "remove secret"
echo ".env" > .gitignore
echo "API_KEY=CANARY-IGNORED-9X3V5" > .env
git add .gitignore && git commit -qm "ignore .env"
Next, an addon that totals outbound bytes per host, flags any canary string in a request body, and saves anything that starts like a git bundle. Bundles begin with # v2 git bundle or # v3 git bundle (git docs), and the hooks are standard mitmproxy events.
# count_egress.py
import collections
import logging
MARKERS = [
b"CANARY-NEVERREAD-7Q2Z9",
b"CANARY-HISTORY-4K8M1",
b"CANARY-IGNORED-9X3V5",
]
BUNDLE_SIGS = [b"# v2 git bundle", b"# v3 git bundle"]
sent = collections.Counter()
saved = 0
def request(flow):
global saved
body = flow.request.get_content(strict=False) or b""
host = flow.request.pretty_host
path = flow.request.path
sent[host] += len(body)
for m in MARKERS:
if m in body:
logging.warning("canary %s -> %s%s",
m.decode(), host, path)
for sig in BUNDLE_SIGS:
at = body.find(sig)
if at >= 0:
saved += 1
name = f"bundle-{saved}.bundle"
with open(name, "wb") as f:
f.write(body[at:])
logging.warning("git bundle -> %s from %s%s",
name, host, path)
break
def done():
for host, size in sent.most_common():
logging.warning("%12d bytes %s", size, host)
Start the proxy with mitmdump -p 8080 -s count_egress.py. The first start writes its CA to ~/.mitmproxy/mitmproxy-ca-cert.pem (mitmproxy docs). In a second terminal, route the CLI through it. Claude Code reads NODE_EXTRA_CA_CERTS for an extra CA (Anthropic docs); SSL_CERT_FILE is the variable cereblab set for Grok. Check your CLI's docs for its own.
export HTTPS_PROXY=http://127.0.0.1:8080
CA="$HOME/.mitmproxy/mitmproxy-ca-cert.pem"
export NODE_EXTRA_CA_CERTS="$CA"
export SSL_CERT_FILE="$CA"
cd canary
claude -p "Reply with exactly: OK. Do not read or open any files."
claude -p "What does this project do?"
Swap claude for whichever CLI you're auditing. Run both prompts: the idle one tells you what leaves when the agent does nothing, the real task tells you what leaves when it works. Stop mitmdump with Ctrl-C and the done hook prints the per-host totals. If a bundle file was saved, open it:
git clone -q bundle-1.bundle recovered
cat recovered/src/_probe/never_read.txt
git -C recovered log --all --oneline -- old.env
How to read the results:
- The model host must appear. If it doesn't, the CLI ignored the proxy, and a quiet capture proves nothing.
- Compare hosts with the vendor's own list. Anthropic, for one, publishes each host Claude Code contacts and what it's for. A host that isn't on the list is a question for the vendor.
-
NEVERREADin the idle run means file contents left without a read.HISTORYanywhere means history is travelling. -
IGNOREDon the model host in the real-task run means the agent read your.env.IGNOREDon any other host means something beyond git history is leaving. - A saved bundle that clones is the strongest evidence you can get.
This setup has blind spots. The marker check only sees uncompressed bodies, and git compresses the objects inside a bundle, so for uploads like Grok's the saved bundle and the per-host byte totals are your evidence. A missing canary warning proves nothing on its own. A CLI that pins certificates will refuse the proxy, which is a finding in itself. The request hook counts HTTP bodies only, so for a CLI that talks over WebSocket, as Codex does for model turns, add a websocket_message hook. A large upload can arrive in chunks, with the bundle header only in the first, and a multipart body needs its trailing boundary trimmed before git clone accepts it. A CLI may ignore these variables and read only the OS trust store; trust the CA there on a throwaway user account, and remove it afterwards.
Where Hermes sits in this
I build Hermes IDE, so weigh this section with that in mind. Hermes runs each vendor's own CLI in a real terminal, signed in with that CLI's own login, and it never asks for your tokens. The CLI signs in and connects by itself, so whatever it uploads, it uploads on its own connection. The thing to audit is the CLI.
Each task gets its own branch and worktree in Hermes, which keeps agents from overwriting each other's files. Worktrees still share one repository (git docs), so a bundle made from a task's worktree can carry the same shared history as your main checkout.
Because any command-line agent can be added, the audit fits in: put the proxy exports in a small script that ends with exec grok "$@" (or exec claude "$@"), add that script as an agent, and give it the canary task on its own branch. I haven't run cereblab's harness myself; the numbers above are theirs.
Hermes makes network calls of its own, and its release notes describe two of them. Since 2.3.0, anonymous usage statistics are on by default, never including code, prompts or file names, and you can turn them off in Settings > Privacy. Hermes also checks hermes-ide.com for news at most once a day, which stops when you switch off "Show notices from the Hermes team." Those are claims, and the setup above is how you test claims like them. Hermes is free, source-available under BSL 1.1, runs on macOS, Windows and Linux, and is not affiliated with xAI, Anthropic, OpenAI, Google or Nous Research.
If this was useful
Run the canary against the agents you use this week, and keep the addon around for the next release. If you want one place to run Claude Code, Codex and the rest side by side, each on its own branch, download Hermes IDE, and if it helps, star it on GitHub. For the patterns behind agents, their tools and the guardrails around them, the AI Agents Pocket Guide is the short read.
Top comments (0)