How LLM Evaluation Produces Comparable Numbers: Inside the LFORLA Capability Retention Benchmark

TL;DR: LLM evaluation only becomes useful when every model faces the same prompts, the same fixed judge, per-axis rubrics, and a public verbatim trail. The LFORLA Capability Retention benchmark does exactly that across 20 deterministic tasks, and the resulting numbers show what uncensoring costs in capability.

How the evaluation works

The LFORLA Capability Retention benchmark is built to answer one question: what does uncensoring cost in raw capability? To make that answer comparable across models, the mechanism is deliberately rigid.

1. Same prompts for every model. The benchmark uses 20 deterministic math, logic, code, instruction, and reading tasks. Every model receives the identical prompt text, with no per-model tuning or prompt adaptation.
2. Same fixed judge. A single judge configuration scores every response. Because the judge does not change between models, differences in scores reflect differences in model output, not differences in grading.
3. Per-axis rubrics. Each response is scored against explicit rubrics for the axes: reasoning, code, math, instruction, and uncensored. The metric axes include Code, Math, Logic, Reading, Accuracy, and Instruction, each expressed as a percentage from 0 to 100 where higher is better.
4. Public verbatim trail. Every answer is stored verbatim so anyone can re-read what a model actually answered. This is the audit layer: you can inspect the exact text behind any score.

This combination is what makes LLM evaluation comparable rather than anecdotal. The benchmark slug is capability-retention, and the categories are reasoning, code, math, instruction, and uncensored.

The real numbers

Model	Provider	Score
Qwen2.5 7B Instruct	local	90
Llama 3.1 8B Abliterated	local	85
Qwen2.5 7B Abliterated	local	80
Llama 3.1 8B Instruct	local	70
Mistral 7B Instruct	local	60
Dolphin Mistral 7B	local	35

For context, we submitted our own model to this benchmark: GLM 5.2 scored 78.0 overall.

Why the ranking looks this way

The top of the table is not simply the most uncensored model. Qwen2.5 7B Instruct leads at 90, while its abliterated variant, Qwen2.5 7B Abliterated, sits at 80. That 10-point gap is the capability retention cost of uncensoring in this benchmark. Llama 3.1 8B Abliterated at 85 actually outperforms Llama 3.1 8B Instruct at 70, which shows the cost is not uniform across model families. Dolphin Mistral 7B at 35 is the clearest signal that aggressive uncensoring can come with a steep capability penalty on deterministic tasks.

In practice, these scores are percentages on the benchmark axes. A score of 90 means the model handled the fixed task set with high accuracy under the shared rubric; a score of 35 means it frequently missed. Because the judge and prompts are fixed, the ranking is a controlled comparison, not a vibe check.

What this means for practitioners choosing a model

If you need deterministic reliability in math, logic, code, instruction following, and reading, the instruct variants in this table are the safer baseline. If you need uncensored behavior, the abliterated variants show you the trade-off explicitly: Qwen2.5 7B Abliterated retains more capability than Dolphin Mistral 7B in this evaluation. Our own GLM 5.2 submission at 78.0 sits between the abliterated Qwen and the instruct Llama, which is useful context for where a general-purpose model lands on this specific task set.

How to read a leaderboard

• Check that every model faced the same prompts and the same judge before comparing scores.
• Look for per-axis rubrics, not just a single overall number.
• Demand a public verbatim trail so you can re-read the actual answers.
• Treat small gaps as noise until you inspect the underlying responses.
• Match the benchmark categories to your own use case before adopting a ranking.

Honest limitations

This benchmark uses 20 deterministic tasks, so it measures a narrow slice of capability. The scores are percentages on those tasks, not a universal intelligence rating. The judge is fixed, which improves comparability but also fixes any judge bias into every score. All listed models are local providers, so the table does not represent hosted or larger models. Finally, a single overall score can hide axis-level differences that matter for your application.

Conclusion

Comparable LLM evaluation comes from discipline: same prompts, same fixed judge, per-axis rubrics, and a public verbatim trail. The LFORLA Capability Retention benchmark applies that discipline to show what uncensoring costs, and the numbers are concrete. Explore the full results and methodology at lforla.org.