Myth FAQ: Your Lockfile Is Not a Reproduction

Your CI is green. Your lockfile is committed. So your build reproduces, right?

That is the claim I hear most often. It also breaks loudest at 2 a.m., on a machine you have never touched.

This is a myth-busting FAQ. Claims first, evidence second, corrected mental model last.

Myth 1: "The lockfile pins everything."

A lockfile pins what your resolver selected. It does not pin what your environment will install.

Watch the gaps:

• Extras and optional dependencies. requests[socks] and requests resolve differently.
• Environment markers. PEP 508 markers like sys_platform == "win32" evaluate at install time.
• Optional platform binaries. Node packages ship per-arch optional deps for things like bundlers.
• Features and build tags. Cargo.lock pins versions, not the feature set or target triple.

Corrected model: a lockfile describes a set of constraints. Your platform, extras, and flags pick the actual subset.

Myth 2: "The same lockfile gives the same tree."

Same inputs, same outputs. Sounds obvious. It is not true across environments.

Two boxes, one lockfile, different results:

• A wheel exists for linux/amd64 but not linux/arm64.
• One falls back to a source build, the other does not.
• A post-install script fetches a prebuilt binary from a CDN.

The resolved versions match. The installed artifacts do not.

Corrected model: reproducibility is a property of an environment pair, not of a file.

Myth 3: "Hashes prove the dependency set."

Hashes prove artifact identity. They do not prove set completeness.

pip install --require-hashes will refuse to install anything unpinned. That is strong. It still says nothing about a package fetched inside a build hook.

npm records integrity hashes in the lockfile. Great for the tarball. Useless for what a lifecycle script downloads afterward.

So ask the sharper question: can anything enter this environment that the lockfile never named?

Myth 4: "Green CI proves reproducibility."

CI is a warm room. That is the whole problem.

• Docker layer caches keep an old wheel alive.
• Runners preinstall toolchains, compilers, and system libs.
• Reused virtualenvs carry leftover packages.

Your pipeline may be deterministic and misleading. Both at once.

Corrected model: reproducibility evidence comes from a cold environment. Caches are a performance feature, not proof.

Myth 5: "A pinned version is immutable code."

Version strings are labels. Registries vary in whether labels are permanent.

• Some registries allow re-upload or yank-and-republish behaviors.
• Git dependencies without a full commit SHA can move under you.
• Mirrors and proxies may serve different bytes than upstream.

Check your registry's policy instead of assuming. Where the option exists, prefer commit SHAs and verified hashes over friendly version numbers.

The artifact: a two-environment fingerprint

The fix is cheap. Install on two machines that share nothing, then compare a normalized fingerprint.

Here is a template. I have not executed it against your repository, so adapt paths and read it before you run it.

#!/usr/bin/env bash
# fingerprint.sh — write a sorted, normalized view of an installed Python env
set -euo pipefail

OUT="${1:-env-fingerprint.txt}"

python - <<'PY' > "$OUT"
import importlib.metadata as md
import platform, sys

pkgs = []
for dist in md.distributions():
    name = dist.metadata["Name"]
    if not name:
        continue
    pkgs.append(f"{name.lower().replace('_', '-')}=={dist.version}")

pkgs.sort()
print("# interpreter:", platform.python_version(), platform.python_implementation())
print("# platform:", sys.platform, platform.machine())
for line in pkgs:
    print(line)
PY

sha256sum "$OUT" | awk '{print "fingerprint_sha256=" $1}'

Step 1 — cold install, box A

python -m venv /tmp/venv-a
/tmp/venv-a/bin/pip install --require-hashes -r requirements.lock
./fingerprint.sh fp-a.txt

Step 2 — cold install, box B

Box B must never have seen this project. Different architecture if you can get one.

python -m venv /tmp/venv-b
/tmp/venv-b/bin/pip install --require-hashes -r requirements.lock
./fingerprint.sh fp-b.txt

Step 3 — diff, then explain every line

diff <(grep -v '^#' fp-a.txt) <(grep -v '^#' fp-b.txt) || true

Three outcomes, three meanings:

1. Identical package lines. Your dependency set is stable across platforms. Good.
2. Different versions of the same package. A marker or resolver branch fired. Investigate.
3. Packages present on one side only. Something installed outside the lockfile. This is the failure you wanted to find.

Record the interpreter version and platform in the fingerprint. Otherwise your diff is noise.

Node and Rust equivalents

# Node: install strictly from the lockfile
npm ci --ignore-scripts   # then compare `npm ls --all --json` output

# Rust: lockfile plus target triple plus features
cargo metadata --format-version 1 > meta-a.json

--ignore-scripts is diagnostic, not a default. It tells you whether lifecycle scripts are part of the picture.

Decision table: when is the lockfile enough?

Scenario	Lockfile alone enough?	Extra evidence to collect
Pure-Python deps, one platform, no extras	Often yes	Fingerprint from one cold box
Native extensions or platform wheels	No	Fingerprint per OS/arch pair
Optional extras installed	No	Record which extras were enabled
Any post-install or lifecycle script	No	Mirror or vendor what the script fetches
Container base image drift	No	Pin and log the base image digest
Git dependency without full SHA	No	Replace refs with commit SHAs
Build tags or feature flags	No	Log the exact flags beside the fingerprint

Print this table's answers into your build log. A fingerprint nobody recorded is a fingerprint nobody can compare.

Limitations, and who should skip this

Be honest about the cost before you adopt it.

• Two cold installs are slow. Do them on release branches, not every commit.
• Fingerprints are environment-specific. They compare like with like, not across everything.
• This catches drift, not correctness. A stable wrong tree is still wrong.
• Solo prototypes do not need this. If you rebuild weekly from scratch, you already have the signal.
• Regulated builds need more. Signed artifacts, provenance attestation, audit trails. This is a floor, not a ceiling.

Skip the whole exercise if your deployment target is a single long-lived machine. You have one environment, so you have one truth.

Where an ephemeral box fits

The hard part is box B. It must be cold, and it must not be your laptop.

That is the honest use case for a throwaway server. Spin one up, run step 2, copy the fingerprint, destroy it. MonkeyCode's free server option is one place to do this, and its free model access is handy for turning a raw diff into a short explanation list. Disclosure: This article was prepared as part of MonkeyCode's product outreach.

Do not trust a vendor for the result, though. Your fingerprint is your evidence.

What to remember

A lockfile is a constraint set, not a receipt. Hashes prove identity, not completeness. Green CI proves your cache is warm.

So here is the question I keep coming back to: when your build last passed, which machine had actually never seen it before?

If you cannot answer that, you do not have reproducibility yet. You have a habit that usually works.