- Book: AI Agents Pocket Guide
- Also by me: Thinking in Go (2-book series) — Complete Guide to Go Programming + Hexagonal Architecture in Go
- My project: Hermes IDE | GitHub — an IDE for developers who ship with Claude Code and other AI coding tools
- Me: xgabriel.com | GitHub
A Hugging Face repository called burner22/numbat says it holds "Anthropic's next Mythos-class training run": about 14 trillion parameters, 7.72 TB of weights across 772 shards. A post on X sent developers to it on November 1, and by 20:03 UTC that day it had 5,846 downloads, according to CellCog's timeline.
Within hours, Hugging Face users showed that the tensors they checked can be rebuilt bit for bit from nothing but each tensor's name and a seeded random number generator (discussion #13). Anthropic had not commented as of 20:20 UTC on November 1, CellCog reported.
The 7.72 TB of weights turned out to be the harmless part. The loading instructions are what deserve your attention, on this repo and on the next one like it.
What the repo claims
The model card lists "~14T param, 114 layers, hybrid GDN/MLA, MoE (1,224 experts, top-48 + 1 shared), NVFP4." It ships two Python files, configuration_numbat.py and modeling_numbat.py, and says to load the model with AutoModelForCausalLM.from_pretrained(..., trust_remote_code=True). It also says: "Mirrors welcome. Do not DM me asking for the source."
The hook was plausible. TokenPost reports that Anthropic limited Mythos 5.1 to trusted-access programs, so a leaked checkpoint would be real news. The repository was created on October 23 and last changed on October 24. The X post that spread it went up on November 1 at 14:53 UTC (CellCog).
How the weights were shown to be generated
Nobody needed a GPU cluster. The audit in discussion #13 used small HTTP Range reads to pull single tensors out of the safetensors files, and its author says no Python code from the repo was run. The recipe:
- Hash the tensor's name with SHA-256.
- Use the first 8 bytes to seed NumPy's PCG64 generator.
- Draw standard-normal values in the tensor's shape, scale them (
exp(0.06 * z)for norm weights,z / 6for conv kernels) and round to bfloat16.
That reproduced "43 complete tensors — 730,368 BF16 values" exactly. The author then predicted 21 tensors they had not read yet, 423,168 values, and every one matched (discussion #13).
A second user extended the check to the large 4-bit expert matrices and reported sign matches on 162,246,496 of 162,246,496 values across 770 of the 772 shards (discussion #19). CellCog reran the first recipe on shard 1: all 9 tensors matched, and one of them rerun with a wrong seed (12345) did not match (CellCog).
The audit read a few kilobytes per tensor and never ran the repo's code.
A separate Reddit analysis pointed the same way using statistics: output-layer rows with a similarity of about 0.89 to their word embeddings, against 0.0144 for Qwen2.5-7B, and statistics across three types of weight matrices that differed by only 0.4% (TokenPost).
Safetensors files versus trust_remote_code
A safetensors file holds tensors and a JSON header. The Transformers docs call safetensors "more secure" than pickle-based PyTorch weights. Downloading 7.72 TB of noise wastes your disk and bandwidth. It doesn't run anything.
trust_remote_code=True does. With it, Transformers loads the modeling code from the repo instead of its own library, and that Python runs in your process, as your user. The same docs say: "Take extra precaution when loading a custom model," and warn you to "avoid inadvertently executing malicious code" even though the Hub runs malware scanning on repository files under 2 GB.
To be fair to the uploader, nobody has shown numbat's code is malicious. The review in discussion #11 calls it "a careful fake" built so that load_state_dict(strict=True) succeeds, and lists bugs such as a missing KV cache. Another commenter in the same thread disputes several of those points. On Reddit, one user warned against running the bundled Python and another raised a concern that the Reddit post could be bait to get people to download a malicious model (TokenPost). I haven't read the modeling files myself.
There is a second problem that has nothing to do with today's code. A repo that asks for mirrors and refuses DMs can push a new modeling_numbat.py at any time, and trust_remote_code=True without a revision loads whatever is on main. The Transformers docs recommend loading custom models "from a specific revision to avoid loading model code that may have changed."
trust_remote_code=True runs the repo's .py files in your process.
Check a custom-code model before you load it
These habits work for any repo that asks for trust_remote_code, not just this one.
First, see what you are about to download. hf download has a dry-run mode that lists files and sizes without fetching them (huggingface_hub docs):
hf download some-org/some-model --dry-run
Then pull only the code and config, pinned to one commit, and read them. snapshot_download takes allow_patterns and revision for this, and a commit hash "must be the full-length hash" (huggingface_hub docs):
from huggingface_hub import snapshot_download
path = snapshot_download(
repo_id="some-org/some-model",
revision="<full-40-char-commit-sha>",
allow_patterns=["*.py", "*.json", "*.md"],
)
print(path)
If the code passes your review, load from that same commit, so what runs is what you read (Transformers docs):
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained(
"some-org/some-model",
trust_remote_code=True,
revision="<full-40-char-commit-sha>",
)
When you read the .py files, search for subprocess, os.system, eval(, exec(, socket, requests, urllib, long base64 strings, and anything that touches ~/.ssh or reads credentials from environment variables. Modeling code builds layers and does math. It rarely has a reason to open a network connection. Run the first load in a throwaway container or VM with no cloud credentials mounted.
If a coding agent does the loading for you
This is where the checks above get skipped. Ask a coding agent to "try the leaked model" and it will read the README to find out how. If the README says trust_remote_code=True, that is what goes into the script it writes. The agent did what the README said, and nobody vetted the README.
A few guardrails that fit any agent:
- Treat a model card like any other untrusted web page. Anything it says is input to the agent. Only you give instructions.
- Keep commands that run downloaded code behind an approval step. Claude Code, for example, documents permission modes that decide which actions need your confirmation.
- Give model experiments their own throwaway environment, without the API keys and SSH config your day-to-day work uses.
What's still open
The audits leave two gaps, by their own authors' account. The discussion #13 author writes that the matches "cannot logically exclude a training setup that deliberately kept all these parameters frozen." The discussion #19 rebuild depends on one fitted scale constant and still has a small number of mismatched 4-bit codes (discussion #19). Neither TokenPost nor CellCog reported any response from Hugging Face, and the repo page was still up when this post was written.
The next repo like this one will come with a better README. The check before the first load stays the same: a pinned commit and a read of its modeling_*.py.
If this was useful
Agents now download models, write the loading scripts and follow READMEs for you, so the guardrails around what they may run matter more than the prompt. The AI Agents Pocket Guide covers the patterns behind autonomous agents, including the guardrails that keep them inside the lines you set.
I also build Hermes IDE, so weigh this with that in mind: it's a free, source-available (BSL 1.1) app for macOS, Windows and Linux that runs Claude Code, Codex and other agents side by side, each task on its own branch and worktree. It's on GitHub, and it isn't affiliated with Nous Research, Anthropic or any agent vendor.
Top comments (0)