When the Model, the Shell, and the Disk Disagree

Did your agent really change the remote machine today?
The transcript looked confident about every claimed write.
The directory on that host did not agree.

I keep seeing the same mix-up this year.
People treat one chat turn as three different systems.

First there is the model context window itself.
Second, there is the live shell process.
Third, there is the actual filesystem state.

Those three stores do not share a clock.
They also do not share a memory model.
When they drift, the chat still sounds finished.

Why this FAQ exists

Agent write-ups keep celebrating successful tool use.
A JSON blob is not a mutation event on disk.

Have you hashed the path after the victory line?
Have you printed $? from a shell you control?

If you skipped both checks, you trusted a narrator.
Narrators compress failures. Filesystems do not.

Three stores, not one workspace

I use a throwaway remote box for dirty experiments.
MonkeyCode offers free model access and a free server option.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

I do not treat that box as production capacity.
I treat it as scratch disk beside a chat window.

The checks below still work on any SSH host.
Strip the product name and the method remains useful.

The myth FAQ

Myth 1: A tool result means the write landed

The model returned {"ok": true, "path": "app.py"}.
Did you stat that path on the server yourself?

Tool payloads can be summarized, truncated, or invented.
A pretty result is still only a claim.

Corrected model: Treat every tool result as testimony.
Require an independent filesystem witness after the turn.

# proposed check — not a recorded benchmark
stat -c '%n %s %Y %i' app.py
sha256sum app.py

No stat output? Then no write happened.
Do not argue with the JSON. Ask the inode.

Myth 2: Model memory is server memory

The agent "remembered" the venv from last turn.
Was that a token, a process, or a directory?

Context windows forget on a brand new session.
Processes die when the SSH session drops.
Disk may survive both events. Or neither.

Corrected model: Name the store before you trust it.
Say "tokens," "pid," or "inode" out loud.

echo "pid=$$ shell=$0"
ls -ld .venv 2>/dev/null || echo 'NO_VENV_ON_DISK'

If the folder is missing, the memory was prose.
Tokens cannot hold a virtualenv in place.

Myth 3: "I ran the tests" includes the exit code

The chat said the whole suite passed.
Where is the integer from your own $??

Summaries drop non-zero exits all the time.
Agents also paraphrase red logs as almost green.

Corrected model: Capture the shell, not the recap.

# proposed harness — run this in your tty
python -m pytest -q
echo "EXIT:$?"

If EXIT: never appears, you have a story.
You do not have a test result yet.

Myth 4: A file listing in prose is ls

The model printed a tidy project tree.
Was that stdout, or a guess from training data?

Hallucinated paths look extra complete and symmetrical.
Real ls looks boring, dated, and slightly messy.

Corrected model: You run the listing command.
Pipe it. Save it. Diff it after the turn.

ls -la --time-style=full-iso > /tmp/listing.before
# ... one agent turn, then:
ls -la --time-style=full-iso > /tmp/listing.after
diff -u /tmp/listing.before /tmp/listing.after

Empty diff means the tree did not move.
A chat tree is not a directory listing.

Myth 5: A printed patch is an applied patch

The agent dumped a unified diff inside the thread.
That is a proposal, not a write to disk.

Did any process open the file for write?
Printing hunks is cheap. Applying hunks is not.

Corrected model: Witness the bytes with tools you own.

# proposed: snapshot, then compare
sha256sum app.py > /tmp/app.py.before
# ... agent claims it patched app.py ...
sha256sum app.py > /tmp/app.py.after
cmp /tmp/app.py.before /tmp/app.py.after && echo UNCHANGED || echo CHANGED

Label this as a check you still must run.
I am not claiming a timed lab result here.

Myth 6: A new chat restores the same machine

You opened a fresh thread on the same host.
Did the workspace reset? Did the context reset?

People assume both reset together every time.
They almost never reset on the same boundary.

You can get old files with empty model memory.
You can get leftover exports inside a new shell.
You can get a wiped home and a confident recap.

Corrected model: Session start is two separate questions.
What tokens survived? What inodes survived?

pwd; hostname; id
echo "PID=$$"
ls -lid .
test -f .env && echo DOTENV_PRESENT || echo DOTENV_ABSENT
env | grep -E '^(VIRTUAL_ENV|PATH)=' 

Read those lines before you prompt again.
A blank chat is not a blank machine.

Artifact: a prove-it harness

I do not trust a write until this script moves.
It is a proposed workflow, not a published lab report.

Save it as prove_write.sh on the remote host.
Make it executable. Pass one path argument.

#!/usr/bin/env bash
set -euo pipefail
# Proposed witness log. Run this yourself.
TARGET="${1:?path}"
STAMP="$(date -u +%Y%m%dT%H%M%SZ)"
LOG="${PROVE_LOG:-$HOME/prove-write.log}"

{
  echo "=== $STAMP ==="
  echo "host=$(hostname) pwd=$(pwd) user=$(id -un) pid=$$"
  echo "target=$TARGET"
  if [[ -e "$TARGET" ]]; then
    stat -c 'exists=1 size=%s mtime=%Y inode=%i mode=%a' "$TARGET"
    sha256sum "$TARGET"
  else
    echo "exists=0"
  fi
} | tee -a "$LOG"

Run it before the agent turn starts.
Run it after the agent turn ends.
Then compare the two blocks in the log.

Need a second target? Run the script twice.
Do not ask the model to summarize both snapshots.

Decision table

Keep this table beside the log file.

• Chat claimed file created. You must see exists=0 then exists=1. If missing, the create did not land.
• Chat claimed file edited. You must see size, hash, or mtime move. If missing, the edit was talk.
• Chat claimed tests passed. You must see EXIT:0 from your shell. If missing, the recap is unverified.
• Chat claimed a package install. You must see new files under the site-packages path. If missing, the install was narrated.
• Chat claimed a secret removed. You must see the path gone and the env unset. If missing, the secret may still sit.
• Chat claimed a workspace reset. You must see a new pid and a tree you measured. If missing, you are on leftover state.

Do not argue with the assistant after a mismatch.
Argue with the rows. Then rerun the witness.

A tight loop I actually follow

This is a method, not a performance claim.
I keep it short so failures stay visible.

1. Snapshot the target with prove_write.sh.
2. Ask the agent for one change only.
3. Refuse extra "while we are here" files.
4. Snapshot again into the same log.
5. Keep the change only if the table agrees.

Why one change per turn?
Bundled turns hide which write failed.

Why refuse extra files in the same prompt?
Extra files inflate hallucinated trees fast.

Why keep the log on the host?
Because the chat will not store inodes for you.

What this does not prove

A matching hash is not a security review.
A green local pytest run is not CI.

A free remote box is not your laptop.
It is also not your production cluster.

I will not quote latency, quotas, or model names.
Those numbers rot fast, and I did not measure them here.

Do not paste customer data onto a shared scratch host.
Do not treat chat transcripts as audit logs.
Do not skip review because a tool result looked busy.

A witness script proves mutation, not correctness.
Wrong bytes can still change a hash.

Who should skip this approach

Skip it if you cannot SSH into the box.
Skip it if policy forbids third-party model hosts.
Skip it if you need signed build provenance.

Skip it if the work is live incident response.
You want a controlled jumphost, not scratch storage.

This FAQ is for developers debugging agent confidence.
It is not a platform migration plan.
It is not a substitute for your real pipeline.

Closing

So, did the agent write the file this time?
Ask the inode. Ask the hash. Ask $?.

If those three stay quiet, the chat performed.
The machine did not move with it.

If you already have a throwaway remote shell, start there.
Run the witness script twice and keep the log.