DEV Community

Jordan Huang
Jordan Huang

Posted on

FAQ: Five Myths About the Agent's Remote Shell

The agent said the remote shell was fine. Can you prove that claim with a file?

It pasted a traceback, then a patch, then a shrug. I still needed a receipt from the box itself.

This FAQ kills five claims developers repeat. They sound like caution. They are not.

This is an identity problem, not a prompt problem. Which Python. Which clock. Which tree. Which exit.

What this FAQ is not

This is not a model bake-off. I will not name models.

This is not a quota thread. I will not invent limits.

This is not a speed story. I have no honest benchmark here.

That JSON file is the only bar I trust.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

A free model plus a free server is one convenient place to collect the remote side of this receipt. The checklist still works on any throwaway shell you already own.

Myth 1: Remote Python is "the same Python"

The claim

"It is Python 3. The box is fine." Why does that sentence feel finished to reviewers?

Why it spreads

The prompt said python3. The traceback looked familiar. One import worked.

What the claim skips

python3 is a name, not an identity. Distros ship different ABIs. SoABI can drift. The stdlib path can drift too.

Evidence I actually want

Run this on the remote shell. Run it on your laptop. Then diff the files.

# proposal: interpreter fingerprint, not a vibe
import hashlib, json, platform, sys, sysconfig
from pathlib import Path

exe = Path(sys.executable).resolve()
payload = {
    "executable": str(exe),
    "version": sys.version,
    "platform": platform.platform(),
    "implementation": platform.python_implementation(),
    "soabi": sysconfig.get_config_var("SOABI"),
    "json_module": str(Path(__import__("json").__file__).resolve()),
}
print(json.dumps(payload, indent=2))
print("exe_sha256_16", hashlib.sha256(exe.read_bytes()).hexdigest()[:16])
Enter fullscreen mode Exit fullscreen mode

Did the executable path match? Did SOABI match? Did the stdlib path match?

If any of those drift, your "same traceback" is a coincidence.

Corrected mental model

Treat python3 as an alias. Demand a fingerprint before you compare failures.

Myth 2: which python3 proves the interpreter that ran

The claim

"which returned /usr/bin/python3. We are good." Are you sure that path was the live process?

Why it spreads

PATH feels like a contract. It is only a search order. Agents mix python, python3, uv run, and venv hooks.

What the claim skips

which cannot see a process that already exited. It cannot see argv either.

Evidence I actually want

# proposal: inspect the live shell, not a memory of PATH
ps -o pid,ppid,user,lstart,cmd -p $$
readlink -f /proc/$$/exe || true
tr '\0' '\n' < /proc/$$/environ | egrep '^(PATH|VIRTUAL_ENV|PYTHON|PWD)=' || true
type python python3
command -v python3
bin=$(command -v python3)
ls -l "$bin"
sha256sum "$bin" || shasum -a 256 "$bin"
file "$bin"
Enter fullscreen mode Exit fullscreen mode

Ask for the exact argv next. Compare it to history or script output. If you cannot, you do not know.

Corrected mental model

PATH is a rumor. Argv plus a hash is evidence.

Myth 3: The prompt said UTC, so the box used UTC

The claim

"We logged timestamps in UTC. The server is UTC." Who actually configured that clock on the box?

Why it spreads

Cloud screenshots look UTC. Dockerfiles set TZ. People assume the next box follows.

What the claim skips

Locale, TZ, and /etc/localtime are three knobs. Agents rarely print all three. A "passed at 16:00" line can be local, UTC, or leftover state.

Evidence I actually want

date -u +%Y-%m-%dT%H:%M:%SZ
date +%Y-%m-%dT%H:%M:%S%z
python3 - <<'PY'
import datetime, os, time
print("TZ", os.environ.get("TZ"))
print("tzname", time.tzname)
print("offset", datetime.datetime.now().astimezone().utcoffset())
print("aware_now", datetime.datetime.now().astimezone().isoformat())
PY
timedatectl 2>/dev/null || echo "timedatectl missing"
ls -l /etc/localtime
Enter fullscreen mode Exit fullscreen mode

Compare laptop versus remote. If offsets differ, your merged log is fiction.

Labeled example, not a measured incident: both sides print 16:04 passed. One clock is PDT. One clock is UTC. That is a two-hour lie with matching strings.

Corrected mental model

Time is configuration. Print it on every run, both sides.

Myth 4: Exit code 0 belongs to the files you edited

The claim

"The remote command returned 0. Ship it." Returned zero against which git tree, exactly?

Why it spreads

Exit codes feel atomic. Agents narrate them with confidence. Chat wraps them in a bow.

What the claim skips

You may not know cwd. You may not know which clone. You may not know if the files match your editor. A green run against a stale tree is a green lie.

This is not a test-quality FAQ. I already wrote that one. This is a tree-identity FAQ.

Evidence I actually want

# proposal: bind the later exit code to a tree fingerprint
git rev-parse HEAD
git status --porcelain
git diff --stat
pwd
python3 - <<'PY'
import hashlib, json, pathlib
root = pathlib.Path(".").resolve()
files = sorted(
    p for p in root.rglob("*")
    if p.is_file()
    and ".git" not in p.parts
    and "node_modules" not in p.parts
    and p.stat().st_size <= 1_000_000
)
h = hashlib.sha256()
for p in files:
    h.update(p.relative_to(root).as_posix().encode() + b"\0")
    h.update(p.read_bytes())
print(json.dumps({
    "cwd": str(root),
    "file_count": len(files),
    "tree_sha256": h.hexdigest(),
}, indent=2))
PY
echo "now run your command"
echo "EXIT:$?"
Enter fullscreen mode Exit fullscreen mode

Keep HEAD, the hash, and the exit in one artifact. If HEAD moved, the exit is about another world. If the tree hash moved, stop quoting it.

Corrected mental model

An exit code has no meaning without a tree hash.

Myth 5: The free box is your audit log

The claim

"The server kept the session. We can replay it." Can you still replay it after the tab closes?

Why it spreads

SSH feels permanent. Cloud shells feel like machines. Chat transcripts feel like journals.

What the claim skips

A free server is a convenience. It is not an evidence locker. I will not claim retention. I will not claim disk size. I will not claim uptime.

If you did not copy the receipt off the box, you do not have it.

Evidence I actually want

Write one JSON file. Leave with that file. Paste it into git if the run matters.

./receipt.sh /tmp/remote-receipt.json
# then copy that file off the box before you close anything
Enter fullscreen mode Exit fullscreen mode

Corrected mental model

Logs you do not export did not happen. Chat is not storage.

The artifact: receipt.sh

This is the whole method. Remote once. Local once. Diff the JSON.

Treat the script as a proposal. I am not attaching a public dataset. I am not inventing a pass rate.

#!/usr/bin/env bash
# receipt.sh — proposal, not a product claim
set -euo pipefail
out="${1:-./shell-receipt.json}"

python3 - "$out" <<'PY'
import hashlib, json, os, platform, socket, subprocess, sys, sysconfig, time
from datetime import datetime, timezone
from pathlib import Path

def sh(cmd):
    try:
        p = subprocess.run(
            cmd, shell=True, capture_output=True, text=True, timeout=20
        )
        return {
            "cmd": cmd,
            "code": p.returncode,
            "stdout": p.stdout.strip(),
            "stderr": p.stderr.strip()[:400],
        }
    except Exception as e:
        return {"cmd": cmd, "error": str(e)}

root = Path.cwd().resolve()
exe = Path(sys.executable).resolve()
tree = hashlib.sha256()
count = 0
for p in sorted(root.rglob("*")):
    if not p.is_file():
        continue
    if ".git" in p.parts or "node_modules" in p.parts:
        continue
    if p.stat().st_size > 1_000_000:
        continue
    tree.update(p.relative_to(root).as_posix().encode() + b"\0")
    tree.update(p.read_bytes())
    count += 1

receipt = {
    "schema": "shell-receipt/v1",
    "utc": datetime.now(timezone.utc).isoformat(),
    "host": socket.gethostname(),
    "platform": platform.platform(),
    "uname": sh("uname -a"),
    "cwd": str(root),
    "uid": getattr(os, "getuid", lambda: None)(),
    "tz": os.environ.get("TZ"),
    "tzname": time.tzname,
    "python": {
        "executable": str(exe),
        "version": sys.version,
        "implementation": platform.python_implementation(),
        "soabi": sysconfig.get_config_var("SOABI"),
    },
    "path_python3": sh("command -v python3"),
    "git": {
        "head": sh("git rev-parse HEAD"),
        "status": sh("git status --porcelain"),
        "remote": sh("git remote -v"),
    },
    "tree": {"file_count": count, "sha256": tree.hexdigest()},
}

Path(sys.argv[1]).write_text(json.dumps(receipt, indent=2) + "\n")
print("wrote", sys.argv[1])
PY
Enter fullscreen mode Exit fullscreen mode

How I compare the two files

chmod +x receipt.sh
./receipt.sh /tmp/remote.json
./receipt.sh /tmp/local.json
python3 - <<'PY'
import json
from pathlib import Path

a = json.loads(Path("/tmp/remote.json").read_text())
b = json.loads(Path("/tmp/local.json").read_text())
for k in ("platform", "python", "cwd", "tree", "tzname"):
    print(f"{k:10} match={a.get(k)==b.get(k)}")
if a["git"]["head"] != b["git"]["head"]:
    print("HEAD mismatch: stop sharing exit codes")
if a["tree"]["sha256"] != b["tree"]["sha256"]:
    print("tree mismatch: the remote exit is another world")
PY
Enter fullscreen mode Exit fullscreen mode

Decision table

Check If it matches If it does not
Python fingerprint Tracebacks are comparable Do not paste remote stacks onto local code
git HEAD You may discuss the exit Fetch or re-clone first
Tree sha256 The command saw the same files The green bar is another tree
TZ offset You may merge logs Split the timelines
JSON copied off-box You have a receipt You have a story

Print that table in the PR. Or refuse the PR. Those are the two options I respect.

Limitations

The script skips files over 1MB. That is a bias.

It skips node_modules on purpose. That is another bias.

It does not freeze the network. A remote install can still race.

It does not prove the agent used this interpreter. It proves the interpreter you invoked while collecting the receipt.

timedatectl may be missing. That absence is data too.

/proc may be missing. Then record that gap in the JSON.

A free server can disappear. Export that file before the session dies.

Do not dump full environ into git. Secrets hide in PATH-adjacent variables.

This habit does not make a model "safe." It makes your claims checkable.

Who should not use this approach

Do not use a shared free server for customer data.

Do not put private keys on that box.

Do not treat it as CI if you need retention, IAM, or an artifact store.

Do not use it when policy forbids unknown hosts.

If you need those controls, use your real CI. Pay for isolation in your real pipeline instead. This FAQ is not that design.

The only question I ask now

Where is the JSON for that remote claim?

No JSON means I reject the remote claim. Run receipt.sh on the throwaway box. Run it on the laptop. Diff the files.

The agent can still be wrong. At least the box cannot hide behind a shrug.

Top comments (0)