There are thousands of abliterated models on Hugging Face now. If you are evaluating one, the thing that degrades is probably not what you are testing for.
What abliteration does
Briefly: you identify a refusal direction in the model's activation space and project it out of the weights. No gradient steps, no training data. It is a weight edit, not a finetune.
The assumption most people carry into evaluation is that this is a capability tradeoff - you get compliance, you lose some general intelligence, and you test for that by checking whether the model still knows things and still writes well.
That is not where the damage shows up.
The first thing to go is obedience, not knowledge
Across variants and across model families, what degrades first is instruction-following and output-format adherence.
The model still knows the material. What gets measurably worse:
- respecting the instruct template and prefill
- honouring stop sequences
- staying inside a structured-output contract (JSON schema, tool-call syntax)
- holding a system-prompt constraint across a long context
So the model that scores within noise of its base on MMLU will fail structured output at a materially higher rate. It knows what to do. It has become worse at doing what it was told.
Why your evaluation misses it
The standard way people assess an abliterated variant is: chat with it, check it does not refuse, check the prose is good. That test cannot detect this. The model looks fine. Often better than fine, because the refusal behaviour that used to interrupt you is gone.
Then you wire it into something that parses its output and the failure rate climbs.
Perplexity does not help either. On a generic corpus it barely moves while format compliance falls off a cliff. The metric says the model is fine and the application says it is not.
What to measure instead
Test format compliance separately from quality, and score it independently of whether the answer is correct.
The shape that works:
- issue N requests that specify an exact output contract
- score binary compliance with the contract, not correctness of the content
- report a compliance rate you can compare across variants and across quants
The key property is independence from correctness. A model returning a well-formed wrong answer scores 1.0. A model returning a correct answer wrapped in prose that breaks the schema scores 0. That is the number you actually care about when something downstream is parsing.
This will separate two variants that look identical in a chat window.
The quantization interaction
A hunch I have not measured properly, offered as a hunch: the damage appears to compound with quantization damage rather than sit alongside it. An abliterated model seems to degrade faster down a quant ladder than its base does, measured on format adherence.
If that is right, the mechanism would be that abliteration has already flattened some weight structure that quantization then rounds away. Which would mean staying at higher quants is worth more on an ablated model than on a clean one - I have had better luck at q6_K and above on smaller models, and a q8_0 on a 4B is only around 4.3GB, so the saving from going lower is small next to what it seems to cost.
If you want to test it properly: quantize a base model and its abliterated counterpart to the same bpw with the same calibration set, then measure compliance rate rather than perplexity.
Practical implications if you are building on these
Your tool-call parser needs to be more forgiving than it does against a frontier model. A strict parser turns a recoverable formatting wobble into a failed run.
Grammar-constrained decoding is doing more load-bearing work here than it does on clean models, because it forecloses malformed paths at the token level rather than relying on the model to want to follow the schema.
Track compliance rate as a first-class metric, not something you infer from task success. When a run fails you want to know whether the model was wrong or merely malformed. Those have different fixes.
Founder disclosure: I build Grunz, which serves these models, so I learned this from production rather than from a paper. If anyone has measured the quant interaction properly I would rather cite a real result than keep repeating a hunch.
Top comments (0)