The agent installed a tool last turn. It exported a dummy flag. It changed directory into src.
Then you sent the next prompt. The tool was gone. Why?
I treat that gap as a lab, not a vibe. The chat looks like one long shell. It is not.
Here are five claims I still hear. Each claim has a probe. Run the probe. Do not trust the narrator.
The three stores nobody names
Agents mix three stores. People talk like they are one.
- Model context — tokens sitting in the thread.
- Process state — cwd, env, and running jobs.
- Filesystem — files, venvs, caches, objects.
Which store did the last action touch? Name it first.
If you cannot name the store, you cannot debug the miss.
Myth 1: "The next prompt keeps my shell"
Does it? Only if the same process stayed alive.
A new turn may spawn a new shell. Your cd dies with it. Your export dies with it.
Want proof? Do not argue with the model. Probe the process.
# turn A — process canaries, dummy values only
pwd
echo "PROBE_CWD=$(pwd)"
export PROBE_TURN=alpha
echo "PROBE_TURN=$PROBE_TURN"
python -c "import os; print(os.environ.get('PROBE_TURN', 'MISSING'))"
Now open a new prompt. Add no story. Run this.
pwd
echo "PROBE_TURN=${PROBE_TURN:-MISSING}"
python -c "import os; print(os.environ.get('PROBE_TURN', 'MISSING'))"
Did pwd match turn A? Did PROBE_TURN survive?
If both died, you never had a login session. You had typed commands.
Corrected model
Each turn is a fresh process until a probe says otherwise. cd is memory. Memory is not a contract.
Myth 2: "pip install means the tool is mine"
Whose machine? Whose user? Which prefix?
pip install black on a remote box is not your laptop. It may also miss $HOME after a new image.
Check the prefix. Check the shebang. Then start a new turn and check again.
python -m pip -V
python -m pip show black | sed -n '1,8p'
command -v black || true
type black || true
python -c "import black,sys; print(black.__file__); print(sys.executable)"
hash -r
Still importable after the new turn? You wrote disk. Missing now? You installed into a disposable layer.
I run that second-machine check on MonkeyCode when I need a box that is not my laptop. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Free model access and a free server option exist there. I still only trust the probe output.
Corrected model
An install is a filesystem write. It is not a promise across turns.
Myth 3: "I set the env var, so the app saw it"
Who parsed it? The model? A subshell? The app process?
Typing export DATABASE_URL=... in chat is theatre. A child process must read it. Pasting a real secret into the prompt copies it into tokens. That is leakage, not config.
Do this instead. Write a dummy file. Load it in-process. Never print the full value.
# turn A — dummy DSN only, never a live token
mkdir -p "$HOME/.config/probe"
printf 'PROBE_DSN=sqlite:////tmp/probe.db\n' > "$HOME/.config/probe/env"
chmod 600 "$HOME/.config/probe/env"
# probe_env.py — labeled example, not production code
from pathlib import Path
import os
p = Path.home() / ".config" / "probe" / "env"
for line in p.read_text().splitlines():
if not line or line.startswith("#") or "=" not in line:
continue
k, v = line.split("=", 1)
os.environ.setdefault(k, v)
print("has_dsn", "PROBE_DSN" in os.environ)
print("dsn_prefix", os.environ.get("PROBE_DSN", "")[:7])
New turn: python probe_env.py.
If has_dsn is True, the file store worked. The export theatre was optional.
Corrected model
Env is process memory. Files are how you recross a turn. Chat is a leaky copy.
Myth 4: "node_modules on the box is my node_modules"
Is it? Different npm prefix. Different Node. Different libc.
A green npm test on the box describes that tree. Copying the folder home is a new experiment. Do you compare fingerprints, or feelings?
node -v
npm -v
npm prefix
npm root
ls -ld node_modules || true
find node_modules -name package.json 2>/dev/null | wc -l
Save that log. Run the same block on your laptop. Compare lines, not adjectives.
Numbers match? You got lucky. Numbers differ? You compared two worlds.
Corrected model
Build artifacts belong to the filesystem that built them. Chat cannot transplant a tree.
Myth 5: "The model remembers the file, so skip disk"
Context is not a disk. Tokens drop. Summaries lie. Attachments get cut.
If the file matters, write the file. Then read it back with a hash.
printf 'hello probe\n' > /tmp/probe.txt
sha256sum /tmp/probe.txt
# later turn, no preamble
sha256sum /tmp/probe.txt
ls -l /tmp/probe.txt
Mismatch or missing path? The story in chat was the only copy. That copy is gone.
Corrected model
If you did not hash it, you do not have it. Memory is not a backup.
Artifact: a two-turn persistence lab
Do not debate the product page. Run a lab. Treat the script as unexecuted until you run it.
Create probe_state.sh:
#!/usr/bin/env bash
# probe_state.sh — labeled lab script, run it yourself
set -euo pipefail
MODE="${1:-write}"
ROOT="${HOME}/.cache/agent-probe"
STAMP_FILE="$ROOT/stamp"
ENV_FILE="$ROOT/env"
HASH_FILE="$ROOT/hash"
mkdir -p "$ROOT"
write_state() {
date -u +%Y-%m-%dT%H:%M:%SZ > "$STAMP_FILE"
printf 'PROBE_MARK=alive\n' > "$ENV_FILE"
printf 'lab-body\n' > "$ROOT/body.txt"
sha256sum "$ROOT/body.txt" | awk '{print $1}' > "$HASH_FILE"
if python -m pip install --user --quiet packaging; then
echo "pip_user=ok"
else
echo "pip_user=SKIPPED"
fi
echo "wrote $ROOT"
}
read_state() {
echo "cwd=$(pwd)"
echo "home=$HOME"
echo "probe_mark=${PROBE_MARK:-MISSING}"
echo "stamp_file=$(cat "$STAMP_FILE" 2>/dev/null || echo MISSING)"
echo "env_file=$(cat "$ENV_FILE" 2>/dev/null || echo MISSING)"
echo "hash_file=$(cat "$HASH_FILE" 2>/dev/null || echo MISSING)"
echo "body_hash=$(sha256sum "$ROOT/body.txt" 2>/dev/null | awk '{print $1}' || true)"
python -c "import packaging,sys; print('packaging', packaging.__version__, sys.executable)" 2>/dev/null \
|| echo "packaging=MISSING"
}
case "$MODE" in
write) write_state; read_state ;;
read) read_state ;;
*) echo "usage: $0 write|read" >&2; exit 2 ;;
esac
Turn A:
chmod +x probe_state.sh
./probe_state.sh write
Turn B, new prompt, no preamble:
./probe_state.sh read
echo "PROBE_MARK=${PROBE_MARK:-MISSING}"
Fill this table. That is the artifact. Not a screenshot.
| Store | Signal | Survived turn B? | Meaning |
|---|---|---|---|
| Process env |
PROBE_MARK in the shell |
yes / no | Same process or not |
| Filesystem |
stamp_file is not MISSING
|
yes / no | Disk kept the lab dir |
| Content |
body_hash equals hash_file
|
yes / no | Bytes were not rewritten |
| User site |
packaging import works |
yes / no / skip | Install landed on disk |
How I read the rows
- Three disk yes, env no? You have files, not a shell.
- All no? You are on a fresh machine. Stop assuming.
- Env yes too? You got a persistent process. Do not generalize.
-
pip_user=SKIPPED? Network or pip policy blocked you. Do not fake the row.
I keep that table next to the change. I do not paste "it worked in chat."
If disk survived and env died, config goes in files. If disk died, the box is not a workstation.
Failure patterns I actually use
These are cheap tells. None of them need a dashboard.
-
pwdflips between turns, but the model still says "we are insrc." -
echo $PROBE_TURNprints empty, while the recap claims the export stuck. -
python -c "import black"fails after a proud install paragraph. -
/tmp/probe.txtis missing, yet the chat quotes its contents. -
npm rooton the box does not matchnpm rooton your laptop.
See a tell? Name the store. Rerun the smallest probe. Then stop negotiating with the recap.
Limitations
This lab does not measure model quality. It does not measure latency. It does not prove an SLA.
Clocks can be wrong. date -u is a stamp, not NTP. $HOME can be a tmpfs. pip --user can land on a mount that vanishes.
I am not naming models, quotas, or hardware here. Those change. Your probe output is the source.
Do not put real tokens in ENV_FILE. Use dummy values. Rotate anything you already leaked into chat.
Who should skip this
Do not use a shared agent box for production secrets. Do not deploy from it.
Do not treat a free server as CI. CI has pinned images and logs you own.
If you cannot run two turns and save the table, skip the demo. You are collecting screenshots, not evidence.
If your team needs a long-lived VM, rent one. Label it. Back it up.
Corrected mental model
Chat is a narrator. The process is a bag of memory. The disk is the only object you can hash.
Ask one question after every agent claim: which store?
Then run the smallest probe that can falsify it.
If you run the two-turn lab, keep the table. Leave the recap in the thread.
Top comments (0)