FAQ: Five Myths About 'The Agent Installed the Deps'

Did your agent actually install that package?
Or did it only narrate a successful install?

This mix-up shows up in agent loops constantly.
The chat text often looks exactly like pip.
The server often disagrees a few steps later.

Why do we keep trusting the narrator?
This piece is not a model-quality rant.
At root it is a bookkeeping problem.

Free models can still emit very confident logs.
Free servers still keep ordinary real filesystems.
Those two layers do not share a brain.

The failure mode that actually burns time

You ask an agent to add one library.
It prints a cheerful successfully-installed line.
The next command tries a clean import.
Then ImportError shows up anyway.

Sound familiar yet?
The import line was never the real bug.
The bug was treating chat text as dpkg.

Here is the boring rule I follow.
Install claims stay untrusted until a probe runs.
The probe must ignore the model completely.
It should read one interpreter, not a story.

Where cheap model access actually helps

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

I mention MonkeyCode only for a practical split.
Free model access can draft the install plan.
A free server option can run the probe instead.
The model does not get to grade itself.

Keep that probe inside your own repo.
Do not paste secrets into the prompt.
The probe should print paths, never tokens.
Need a scratch host for the probe?
Use a host you control, including that free server.
Rehearsal is the job. Production truth is not.

Myth 1: A pretty pip log means site-packages changed

Can a model quote pip's happy path? Yes.
Can it paste a fake looking exit code? Yes.
Did anyone hash the files afterward?

Ask the interpreter, not the transcript.

python -c "import sys; print(sys.executable); print(sys.version); print(sys.prefix)"
python -m pip --version
python -m pip show requests || true

Then ask distribution metadata without any chat.

python -c "import importlib.metadata as m; print(m.version('requests'))"

If that throw happens, the package is missing.
A chat log cannot override ImportError.
Do you still believe the success line?

Corrected mental model

The corrected mental model fits in one line.
Chat is advertising copy; metadata is the receipt.
No receipt, no install, no next refactor step.

Myth 2: "Already satisfied" means your pin is present

Why do agents love that phrase so much?
It sounds like a green check mark.
It often means some other version sat nearby.

Which interpreter answered the install command?
Which virtualenv was actually active then?
Was PYTHONPATH pointing at a leftover tree?

python -c "import sys,os; print(sys.executable); print(os.environ.get('VIRTUAL_ENV')); print(os.environ.get('PYTHONPATH'))"
python -c "import requests,inspect,sys; print(requests.__version__); print(inspect.getfile(requests)); print('\n'.join(sys.path[:6]))"

Now compare that version with your lockfile.
If they drift, the agent satisfied the wrong world.
Already satisfied is not the same as pinned.

Want a second witness from freeze output?

python -m pip freeze | rg '^requests=='
git grep -n 'requests' requirements.txt requirements.lock pyproject.toml 2>/dev/null || true

Corrected mental model

Satisfaction is always per interpreter, not per chat.
Your pins live only in the lockfile.
Those two documents are not interchangeable.

Myth 3: One working import proves later steps share that env

Did the agent export a venv in step four?
Did step seven start a brand new shell?
Guess which state just disappeared on you.

Environment is not a novel the model remembers.
It is process state on that host.
A free server will not keep exports forever.

I force later commands to be painfully explicit.
This snippet is a labeled proposal, not magic.

#!/usr/bin/env bash
set -euo pipefail
ROOT="$(cd "$(dirname "$0")/.." && pwd)"
VENV="$ROOT/.venv"
# proposal: never assume the agent's exported PATH survived
if [[ ! -x "$VENV/bin/python" ]]; then
  echo "missing venv python: $VENV/bin/python" >&2
  exit 2
fi
# shellcheck disable=SC1091
source "$VENV/bin/activate"
command -v python
python -c "import sys; assert sys.prefix == r'$VENV', sys.prefix"

If the assert fires, stop the whole loop.
Do not let step eight rewrite application code.
Fix the environment before touching features.
Would you ship a binary with unknown rpath?

Corrected mental model

Each later command still needs an environment contract.
Chat memory is not that environment contract.
If the prefix drifts, application diffs are noise.

Myth 4: The lockfile is honest because the agent edited it

Do agents edit poetry.lock all the time? Yes.
Do they always run the real resolver? No.
Have you read the diff as data?

I treat every lock edit as a claim.
Then I re-resolve on the server itself.
Prose in the chat does not count here.

# proposal for a pip-style repo; swap in your real resolver
git diff --stat -- requirements.txt requirements.lock pyproject.toml
python -m pip install -r requirements.txt --dry-run
git hash-object requirements.txt

For npm, keep the same shape, different binary.

git diff --stat -- package.json package-lock.json
npm ci --ignore-scripts --dry-run

If dry-run cannot see the graph, stop.
That lockfile is theater, not a resolution.
A model cannot wish a graph into existence.

Corrected mental model

The resolver is the author of the lock.
The agent is only a suspect editor.
Restore first, resolve second, commit third.

Myth 5: A free-server install is production dependency truth

This myth is tempting, right?
The remote box already ran some install command.
The chat said the site-packages looked green.
Why keep a pipeline around then?

Because CI is a second independent witness.
It uses a known image on purpose.
It refuses the agent's dirty prefix.
It also drops leftover working-tree souvenirs.

Use a free server as scratch rehearsal.
Do not promote that host to release truth.
Promote hashes, metadata, and lockfiles instead.
Please do not promote vibes as evidence.

Corrected mental model

Those cheap remote runs are only rehearsal.
Your CI still remains the recorded performance.
Skip CI only if you enjoy silent drift.
Install truth is an artifact, not a host nickname.

Artifact: an install-claim probe

This script is a labeled proposal.
I am not tying it to any vendor SLA.
Drop it at scripts/verify_install_claim.py.
Run it after every agent install step.
Pass the distribution name you actually care about.

#!/usr/bin/env python3
"""Probe whether a distribution exists in THIS interpreter."""
from __future__ import annotations

import argparse
import hashlib
import importlib
import importlib.metadata as metadata
import json
import sys
from pathlib import Path


def file_digest(path: Path) -> str | None:
    if not path.is_file():
        return None
    h = hashlib.sha256()
    with path.open("rb") as fh:
        for chunk in iter(lambda: fh.read(65536), b""):
            h.update(chunk)
    return h.hexdigest()


def probe(dist_name: str, import_name: str | None) -> dict:
    report = {
        "executable": sys.executable,
        "version": sys.version,
        "prefix": sys.prefix,
        "base_prefix": sys.base_prefix,
        "in_venv": sys.prefix != sys.base_prefix,
        "dist": dist_name,
        "dist_found": False,
        "dist_version": None,
        "dist_files_head": [],
        "import_name": import_name,
        "import_ok": False,
        "import_file": None,
        "import_digest": None,
        "error": None,
    }
    try:
        dist = metadata.distribution(dist_name)
        report["dist_found"] = True
        report["dist_version"] = dist.version
        files = list(dist.files or [])[:8]
        report["dist_files_head"] = [str(f) for f in files]
    except metadata.PackageNotFoundError:
        report["error"] = f"distribution {dist_name!r} not in this interpreter"
        return report

    if import_name:
        try:
            mod = importlib.import_module(import_name)
            report["import_ok"] = True
            import_file = getattr(mod, "__file__", None)
            report["import_file"] = import_file
            if import_file:
                report["import_digest"] = file_digest(Path(import_file))
        except Exception as exc:  # probe must never crash the job
            report["error"] = f"import failed: {exc}"
    return report


def main() -> int:
    parser = argparse.ArgumentParser()
    parser.add_argument("dist_name")
    parser.add_argument("--import-name")
    args = parser.parse_args()
    report = probe(args.dist_name, args.import_name)
    print(json.dumps(report, indent=2))
    if not report["dist_found"] or (args.import_name and not report["import_ok"]):
        return 2
    return 0


if __name__ == "__main__":
    raise SystemExit(main())

Wire it so the agent cannot self-certify green.

python scripts/verify_install_claim.py requests --import-name requests
echo $?
python -m pip freeze > /tmp/freeze.after.txt
diff -u /tmp/freeze.before.txt /tmp/freeze.after.txt || true

An exit of two means the chat drifted.
You should stop the loop either way.
Do not debug application code on a missing dist.

Decision table I keep next to the PR

Chat claim	Probe result	Next action
"Installed X"	dist missing	Re-run install with an explicit venv
"Already satisfied"	version does not match the lock	Recreate venv, then re-resolve
"Import works"	import_file is outside prefix	Print sys.path; drop extra paths
"Lock updated"	resolver dry-run fails	Restore the lock; run the real resolver
"Ready to ship"	probe fails	Ignore feature diffs; fix env first

Print this table in the pull request.
It beats a screenshot of the chat window.
Can a reviewer hash a screenshot? No.
Can a reviewer hash this JSON? Yes.

A one-hour drill, labeled as a proposal

Treat this drill as an unexecuted proposal.
It is not a published benchmark.
Do not turn it into fake capacity numbers.

1. Snapshot sys.executable and pip freeze before the agent runs.
2. Ask the agent to add one pinned dependency only.
3. Refuse to read the install prose in the chat log.
4. Run the probe on the same host, same interpreter.
5. Diff freeze output against the lockfile and stop on drift.
6. Only then run a single unit test, not a whole suite.

If step four fails, you caught the myth.
If step four passes, you earned one test.
Please notice that order on purpose here.
The tests come last for a reason.

Limitations

This probe does not prove the library is safe.
It does not prove wheels match production glibc.
It does not replace pip-audit or npm audit.
It does not freeze any free server into gold.

Native extensions can still fail at runtime.
Optional extras can still be missing entirely.
Editable installs can still point at deleted paths.
JSON output can still land in the wrong ticket.

Who should skip this whole approach?
Anyone shipping from the agent's working tree.
Anyone who needs a certified toolchain today.
Anyone who cannot pin their interpreter.
Anyone treating a free server as production.

Also skip it if CI images are already strict.
The probe only covers the gap before CI.
This probe is not a new religion.

The mental model I want stuck in your head

The model proposes the change.
The server mutates real files.
The probe reports measured state.
CI records the witness statement.

Those remain four separate jobs.
Do not let one chat sentence collapse them.
If a log cannot be hashed, skip it.
If a module path leaves sys.prefix, distrust it.

Would you accept an invoice without a receipt?
Then stop accepting install narration as proof.
Keep the install probe sitting in git.
Always keep those secrets out of prompts.
Please keep CI as the second witness.
The cheap remote run stays rehearsal only then.