Pinning a Llama Checkpoint by Commit Hash Instead of a Moving Tag

#ai #machinelearning #llm #tutorial

from_pretrained("meta-llama/Llama-3.1-8B-Instruct") does not name a version. It names a Git branch, and a branch is whatever it was last pushed to. The fix is one argument, and knowing which files it protects you from is what makes it worth doing.

main is a branch, and branches move

A model repository on the Hugging Face Hub is a Git repository with large files. When you pass a repo id and nothing else, every library in the ecosystem resolves it to the tip of main at the moment of download. That is a moving target maintained by the model’s publisher, and for a popular Llama repo it does move — for licence text, for README edits, for added quantisations, and, more to the point, for configuration fixes that change what the model does.

The cache makes it worse rather than better, because it hides the change from you locally. A machine with a warm cache keeps serving the old revision; a machine built from scratch fetches the new one. Two identical container images, built a month apart from the same Dockerfile, then run different models. That is the failure this page exists to prevent, and it presents as “staging behaves differently from production” rather than as an error.

What actually changes when it moves

The weight files themselves rarely change after release. The small files around them change more often, and they are the ones that decide behaviour:

generation_config.json — default sampling parameters and, most importantly, eos_token_id. Llama 3’s launch is the canonical example: the Instruct models needed generation to stop on <|eot_id|> as well as on the base end-of-text token, and a config listing only one of them produced models that ran on past the end of their answer. Repositories were updated after release. Which side of that edit you are on is decided entirely by your revision.
tokenizer_config.json — including chat_template. This is a Jinja template that renders your message list into the raw prompt string. An edit to it changes every prompt your application sends, with no change in your code and nothing in your diff.
config.json — max_position_embeddings, rope_scaling. Llama 3.1’s RoPE scaling fields were revised shortly after release as library support landed.
special_tokens_map.json — the mapping the tokenizer uses for pad, bos and eos.

Every one of those changes output without changing a single weight, which is why “the weights are open, so it cannot change” is the wrong intuition. See what open weights do and do not guarantee about determinism.

Resolving the commit hash

A revision can be a branch name, a tag, or a full commit sha. Only the sha is immutable — a tag can be moved, and on the Hub it usually is not, but “usually” is not the property you are buying.

Read the current sha for the repo. The Hub API returns it directly:
```
from huggingface_hub import HfApi

info = HfApi().model_info("meta-llama/Llama-3.1-8B-Instruct")
print(info.sha)
```
Llama repositories are gated, so accept the licence on the model page and export a token first (HF_TOKEN in the environment, or huggingface-cli login). Without one this fails with a 401 rather than telling you the repo exists.
Or read it from what you already downloaded, which is the better move if a deployment is currently working and you want to pin that. The cache directory encodes the resolved commit in its path under snapshots/, and huggingface_hub.snapshot_download returns that path.
Record it where a human will see it — next to your dependency pins, not in a comment in one Dockerfile. It is a dependency version and it deserves the same treatment as one.

Pinning it everywhere it is loaded

The argument is called revision and it is accepted by every entry point. The trap is that a model and its tokenizer are loaded separately, so pinning one and not the other leaves you half-pinned:

from transformers import AutoModelForCausalLM, AutoTokenizer

REPO = "meta-llama/Llama-3.1-8B-Instruct"
REV  = "0e9e39f249a16976918f6564b8830bc894c89659"  # replace with your sha

tok = AutoTokenizer.from_pretrained(REPO, revision=REV)
model = AutoModelForCausalLM.from_pretrained(
    REPO, revision=REV, torch_dtype="bfloat16", device_map="auto"
)

For a pre-baked image, download at the pinned revision at build time and run offline, so that a network failure or a repo change cannot alter what starts:

huggingface-cli download meta-llama/Llama-3.1-8B-Instruct \
  --revision 0e9e39f249a16976918f6564b8830bc894c89659 \
  --local-dir /models/llama-3.1-8b-instruct

# then, at run time
export HF_HUB_OFFLINE=1

And for vLLM, the flag is passed on the server command line — with a separate one for the tokenizer, for the same reason as above:

vllm serve meta-llama/Llama-3.1-8B-Instruct \
  --revision 0e9e39f249a16976918f6564b8830bc894c89659 \
  --tokenizer-revision 0e9e39f249a16976918f6564b8830bc894c89659 \
  --max-model-len 32768

The sha above is a placeholder to show the shape of the argument. Resolve the real one for your repository with the snippet in the previous section — do not copy a hash out of an article, including this one.

Quantised and third-party repositories

Everything above applies more strongly, not less, to the repository you are actually most likely to be using: somebody else’s quantised build. A GGUF or AWQ repository is a derivative work whose maintainer re-uploads when the quantisation toolchain improves, when a bug in the conversion is found, or when the upstream metadata changes. Those re-uploads land on main under the same file names.

Three consequences that do not arise with Meta’s own repositories:

The same filename can be different weights. A file named for a particular quantisation level is a description of a method, not an identity. Re-quantising with a newer converter produces a different file with the same name.
Metadata baked into the file matters. A GGUF carries its own chat template and special-token IDs inside the file, so the equivalent of a tokenizer_config.json change arrives as a new binary rather than as a visible diff. This is the mechanism behind most “the same model behaves differently in Ollama than in llama.cpp” reports.
Ollama tags are mutable too. A tag like llama3.1:8b points at a manifest that its publisher can update. Pin by digest where the tooling allows it, and at minimum record the digest you tested against so that a later divergence is detectable.

The general rule is that a checksum you recorded is worth more than a name somebody else controls. Hash the file you validated, store the hash next to your configuration, and compare on startup. That is the same discipline as pinning a revision, applied where revisions are not on offer.

Verifying and upgrading deliberately

Assert the pin at startup rather than trusting it. Log the resolved snapshot path, or hash the tokenizer’s rendered output for a fixed message list and compare it against a recorded value. A changed chat template shows up immediately in that hash, which is exactly the failure that is otherwise invisible.
Keep the raw prompt observable. If you can print what your stack actually sent — special tokens included — a template change becomes a one-minute diagnosis instead of a week of “quality regressed”.
Upgrade as a change, not as a side effect. Fetch the new sha, diff the small files between revisions, run your evaluation set, then move the pin in one commit. The Hub exposes per-revision file views, so the diff of a chat_template is readable before you download 16 GB.

The same discipline applies to hosted models, where you cannot pin a hash but can usually pin a dated snapshot name — the open-weights case just gives you a stronger guarantee than the hosted one does.

The awkward part of a mixed estate is that the pin has a different shape everywhere: a commit sha for a self-hosted Llama, a dated snapshot string for one hosted API, a version suffix for another. If you route through a gateway such as Multigrid, the useful habit is to name a pinned model explicitly on every route and record the served model id on every response, so that “which weights answered this” is always a lookup rather than an investigation.