DEV Community

Jordan Huang
Jordan Huang

Posted on

The Agent Installed It. Did Your Lockfile Move?

Did the agent install the package, or only narrate it?

I keep seeing both claims treated as one fact. They are not the same event at all.

An agent runs on a free remote box. It prints a cheerful pip success line anyway. You merge, then CI explodes on a missing extra.

Sound familiar to anyone on a mixed team?

This is a myth FAQ

I am not ranking models in this post. I am not selling a runtime in disguise.

I want a corrected mental model up front. Then I want a fingerprint you can run.

I sometimes park an agent on a free remote server. I use that box for a scratch install only.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode offers free model access and a free server option. I still refuse to treat that box as my lockfile.

Why take that stance in such blunt terms? Because install theater stays cheap for chatty agents. Real version pins are not cheap at all.

Myth 1: A printed install is a recorded install

The claim. The agent printed Successfully installed as the last line. So the dependency now exists in the project.

The evidence I want.

  • Did pip freeze change after that command?
  • Did a lockfile change on disk afterward?
  • Did git show a real diff in the manifest?

Printed success is only stdout from a tool. A pin is a file with a stable name.

Ask the agent for the exact command first. Then run that same command yourself next. Compare hashes before the run and after.

Corrected model. Stdout is a story the model told you. The lockfile is the ledger you actually ship.

Myth 2: The interpreter the agent called is yours

The claim. It ran python -m pip install for me. That binary is my project Python.

The evidence I want.

command -v python
command -v python3
python3 -c "import sys; print(sys.executable); print(sys.version)"
Enter fullscreen mode Exit fullscreen mode

Agents love the first python on PATH. That binary may be system Python. That binary may be a login-shell wrapper. That binary may be nothing like your CI image.

Corrected model. Name the interpreter every single time. Never assume python means the venv.

Myth 3: The session remembered, so the env persisted

The claim. Yesterday the agent installed a library. Today the import still works on the box.

The evidence I want.

  • Is this the same machine image?
  • Is this the same volume under /home?
  • Is this the same venv path on disk?

Free remote sessions can vanish without a warning. Chat memory is not a filesystem. A model can recall an install that the disk already wiped.

Corrected model. Memory is text in a transcript. Persistence is a path you can ls.

Myth 4: Import worked, so the extra is declared

The claim. import foo succeeded in the remote shell. The optional extra lives in pyproject.toml.

The evidence I want. The import, then the manifest, then the lock. Check them in that order every time.

A transitive dependency can satisfy a naked import. An agent can install a package into a user site. Your app still ships without the extra declared.

Corrected model. Import proves presence in that process. It does not prove declaration in the repo.

Myth 5: The free box is allowed to own the pins

The claim. The remote install looks green enough. Copy those versions home tonight.

The evidence I want. OS family, libc, wheel tags, and the Python minor version.

python3 -c "import platform,sys; print(sys.version); print(platform.platform())"
Enter fullscreen mode Exit fullscreen mode

A wheel that imports on the free box can be the wrong tag for CI. A compiled extra can bind to a library you do not ship.

Corrected model. The free box is a scratchpad only. CI is the pin authority. Your laptop is a third vote, not the law.

Freeze is not a lock

People paste pip freeze into a PR. They call that dump a lockfile.

It is not a lockfile. It is a dump of that interpreter.

A lockfile records extras and markers your tool supports. A freeze dump records whatever that process can see.

Did the agent freeze the user site? Did it freeze the venv? Did it freeze system packages too?

If you cannot answer those three, you do not have a pin. You have a paste from a chatty shell.

Labeled example, not a measured run:

# pyproject may list flask
# a freeze dump may also list blinker, click, jinja2, werkzeug
# only the lockfile (or a hashed installer) is the contract
Enter fullscreen mode Exit fullscreen mode

Which lockfile? Use whatever the repo already committed. uv.lock, poetry.lock, requirements.txt, or Pipfile.lock all count. The agent does not get to invent a fourth format.

The artifact: an env fingerprint

Do not argue with the chat log. Capture the runtime instead.

Save this as agent_env_fingerprint.py. Run it locally first. Run it in the agent session next. Diff the two files in your terminal.

This script is an example. Run it in a throwaway directory.

#!/usr/bin/env python3
"""agent_env_fingerprint.py — example only. Not a security scanner."""
from __future__ import annotations

import hashlib
import os
import platform
import shutil
import subprocess
import sys
from datetime import datetime, timezone
from pathlib import Path


def add(lines: list[str], key: str, value: str) -> None:
    lines.append(f"{key}={value}")


def git(args: list[str]) -> str:
    try:
        r = subprocess.run(
            ["git", *args],
            capture_output=True,
            text=True,
            check=False,
        )
    except OSError:
        return ""
    return r.stdout.strip() if r.returncode == 0 else ""


def main() -> None:
    out = Path(sys.argv[1] if len(sys.argv) > 1 else "agent-env.fingerprint")
    lines: list[str] = []
    add(lines, "utc", datetime.now(timezone.utc).strftime("%Y-%m-%dT%H:%M:%SZ"))
    add(lines, "host", f"{platform.system()}-{platform.machine()}")
    add(lines, "pwd", os.getcwd())
    add(lines, "user", os.environ.get("USER", "unknown"))
    add(lines, "shell", os.environ.get("SHELL", "unknown"))
    add(lines, "path_sha256", hashlib.sha256(os.environ.get("PATH", "").encode()).hexdigest())
    add(lines, "which_python", shutil.which("python") or "")
    add(lines, "which_python3", shutil.which("python3") or "")
    add(lines, "sys_executable", sys.executable)
    add(lines, "sys_version", " ".join(sys.version.split()))
    add(lines, "platform", platform.platform())

    try:
        import importlib.metadata as md

        pkgs = sorted(
            f"{d.metadata['Name']}=={d.version}" for d in md.distributions()
        )
    except Exception as exc:  # labeled fallback, not a benchmark
        pkgs = [f"metadata_error={exc}"]

    blob = "\n".join(pkgs).encode()
    add(lines, "dist_count", str(len(pkgs)))
    add(lines, "dist_sha256", hashlib.sha256(blob).hexdigest())

    inside = git(["rev-parse", "--is-inside-work-tree"])
    if inside == "true":
        add(lines, "git_head", git(["rev-parse", "HEAD"]))
        status = git(["status", "--porcelain"])
        dirty = [ln for ln in status.splitlines() if ln]
        add(lines, "git_status_short", str(len(dirty)))
        manifests = git(
            ["ls-files", "*lock*", "requirements*.txt", "pyproject.toml"]
        )
        add(lines, "git_manifests", ",".join(manifests.splitlines()))
    else:
        add(lines, "git_head", "")
        add(lines, "git_status_short", "")
        add(lines, "git_manifests", "")

    out.write_text("\n".join(lines) + "\n")
    print(f"wrote {out}")


if __name__ == "__main__":
    main()
Enter fullscreen mode Exit fullscreen mode

Then compare the two snapshots:

python3 agent_env_fingerprint.py local.fingerprint
# run the same file on the remote box, copy remote.fingerprint back
python3 - <<'PY'
from pathlib import Path

def load(path):
    d = {}
    for line in Path(path).read_text().splitlines():
        if "=" not in line:
            continue
        k, v = line.split("=", 1)
        d[k] = v
    return d

a = load("local.fingerprint")
b = load("remote.fingerprint")
for k in sorted(set(a) | set(b)):
    if a.get(k) != b.get(k):
        print(f"MISMATCH {k}")
        print(f"  local ={a.get(k, '')}")
        print(f"  remote={b.get(k, '')}")
PY
Enter fullscreen mode Exit fullscreen mode

I do not need a pretty dashboard here. I need the mismatch in plain text.

Watch dist_sha256 and sys_executable first. Those two lines catch most install theater.

Decision table

  • If the agent printed pip success, do not conclude the pin updated. Conclude stdout happened. Check freeze and git.
  • If python ran a command, do not conclude it used your venv. Conclude PATH picked an interpreter. Fingerprint it.
  • If chat recalls an install, do not conclude the disk still has it. Ask ls and dist_sha256.
  • If import foo works, do not conclude foo is declared. Open pyproject.toml next.
  • If remote freeze looks green, do not ship those versions. Compare wheel tags and Python minor.

A workflow I actually use

This is a procedure. It is not a benchmark. It is not a published latency number.

  1. Snapshot local with the script above.
  2. Ask the agent to run the same script on the box.
  3. Diff fingerprints before any install starts.
  4. Allow one install command. One. Named interpreter only.
  5. Snapshot again. Diff dist_sha256 and git porcelain.
  6. If a pin should change, I edit the lockfile. I do not edit the chat.
  7. Run tests in CI. Never treat the free box as CI.

If step 3 already mismatches, I stop cold. The agent is not in my repo. It is sitting in some other tree.

One install command means one. Not a chain of "and also upgrade pip". Not a hidden curl | sh. Not a second interpreter because the first import failed.

What this does not prove

A matching fingerprint is not a security review. It is not a license audit. It is not a performance test.

I am not claiming the free server is fast. I am not claiming it is permanent. I am not claiming a quota either.

The script hashes installed distributions on that interpreter. It does not verify hashes from a package index. It can miss conda, nix, or container layers.

Clock lines can lie if the box has no NTP. That is fine for this job. I want a snapshot, not a court record.

Who should not use this

Skip this if you already have a sealed CI image. Skip this if policy forbids unknown remote shells. Skip this if you cannot run python3 at all.

Do not use a free remote box to hold secrets. Do not paste tokens into the agent chat. Do not let the agent pip install from an unknown index as a "fix".

If you ship wheels for several platforms, this FAQ is not enough. You still need your existing matrix.

Keep the three roles separate

The agent is a narrator with tools. The free server is a scratch disk. Your lockfile is the contract you merge.

Three roles. Three artifacts. Do not collapse them into one green sentence.

Did the agent install it? Maybe it did.

Did your lockfile move? That is the only question that ships.

If you try the fingerprint, keep the diff in the PR. That single file beats another "it worked for the agent" comment.

Top comments (0)