A widely downloaded guardrail-stripped version of Google's Gemma 4 was shipped with 54 of its attention tensors missing, and the people running it had no reliable way to tell. The publisher, OBLITERATUS, documents the defect on its own model card: version 2 of the release "deleted 54 K/V projection tensors from layers 24-41 due to Gemma 4's shared KV architecture," leaving packaged files with 666 tensors where there should have been 720. The card says this "caused hallucinations and degraded quality in the quantized GGUFs."
Key facts
- 54 key and value projection tensors were silently dropped from the released files, 666 total instead of 720.
- The publisher rates the broken version's output quality at 3.1 out of 10 on its own judged scale.
- The card states the model was "built by an AI agent with less than 10 human prompts."
- Primary source: the OBLITERATUS Gemma 4 E4B model card on Hugging Face.
Some background for anyone who has not been near this corner of the open-weights world. When a lab releases model weights, people modify them. One popular modification is abliteration: rather than retraining, you find the internal direction in the model's activations that corresponds to refusing a request and mathematically project it out. The result answers everything. These edited models are published, quantized into small files that run on a laptop or a phone, and downloaded by the tens of thousands. The pipeline from a lab's original release to the file someone actually runs can involve three or four intermediaries, none of whom the original lab knows.
That is a software supply chain, and this incident is a textbook supply-chain failure. Here is the mechanism, in the publisher's own account. Gemma 4 shares key and value projection weights across layers - layers 24 through 41 all point at the same tensors owned by layer 24. The editing tool did not know that. It applied its refusal-removal projection once per borrowing layer, meaning it hit the same shared tensor eighteen times in a row. The result was numerically corrupted, and PyTorch's save routine then dropped the corrupted entries entirely rather than raising an error. The fix, per the card, is to "project from shared K/V weights exactly ONCE (on the owning layer), then skip them on all borrowing layers."
It is worth appreciating the failure mode. Nothing crashed. No checksum failed. The files loaded, the model generated fluent English, and roughly half of an attention stack was simply gone. If you were running the model to write fiction or answer casual questions, the symptom was that it seemed a bit worse than expected - which is precisely how a four-billion-parameter model is supposed to seem. The publisher's own before-and-after table scores the broken version at 3.1 out of 10 on judged quality. Users were consuming a materially damaged model for as long as it took someone to look at a tensor count.
Two details make this more than a hobbyist mishap. First, the card states the whole thing was "built by an AI agent with less than 10 human prompts." An automated pipeline edited a frontier lab's weights, packaged them, wrote the documentation, and published to a public registry with no human reviewing the tensor manifest. Second, popularity is not a quality signal here. Hugging Face's own documentation on download counts is explicit that the platform counts server-side requests to designated files, including HEAD requests, and that repository clones can double-count. Downloads measure distribution. They say nothing about whether the file you pulled has all its parts.
The community does have tools for this. The abliterlitics methodology published by DreamFast is a reasonable template: diff every tensor against the base model, measure how far the edited model's behaviour on harmless prompts has drifted using KL divergence, run a multi-domain capability suite, and separately measure harmful-completion rates. A tensor diff against the base weights would have caught this defect in seconds. Nobody ran one before publication.
The honest caveat is that the corrected version may well be fine. The card says version 3 restores all 720 tensors and preserves the shared projections properly, and that claim is plausible and specific. But it is a publisher claiming its own repair, and no independent evaluator has pinned the exact revision, quantization files, and chat template used and re-measured. The card's own numbers table is also worth reading against its headline: alongside a claimed zero percent hard-refusal rate, it reports roughly 51 percent of outputs coherent and on topic, about 28 percent soft deflection, about 20 percent degenerate repetition loops, and about 4 percent answering in the wrong language.
The general lesson generalises past jailbroken models. As agents increasingly install, quantize, and serve weights automatically, the number of hands a checkpoint passes through before it reaches production is growing, and almost none of those hands verify integrity. Downloads tell you what spread. Reproducible artifact checks tell you what to trust.
Originally published on Ground Truth, where every claim is checked against the primary source.
Top comments (0)