I ran a 110B LLM on 16GB of RAM. Here's the equation that predicts any model's speed on your machine

My 2016 desktop — 16 GB RAM, SATA SSD — ran GLM-4.5-Air, a 110B-parameter model, streamed from disk.

One equation predicted the speed before I pressed enter: 0.2-0.3 tok/s.

It measured 0.19.
That equation (tok/s = eta(tier) x bandwidth / active-bytes-per-token) is what this post hands you, plus the 30-minute probe that finds where your model breaks.

But first, the control experiment that proves placement is the whole game. I have two Q2_K GGUF files of Gemma 4 12B. Same model, same quantizer, same bit budget, same 5.22 GB on disk. One scores 12.27 perplexity on WikiText-2. The other scores 10.02.

Recipe	PPL (WikiText-2)	File
Uniform Q2_K FFN	14.41	—
Protect first 12 layers	12.27	5.22 GB
Protect last 12 layers	10.02	5.22 GB

The bottom two rows are byte-identical in size. Same number of protected blocks, same types — I only moved them to the other end of the stack. That's a −2.25 PPL swing from placement alone, worth roughly 2x the byte budget. The "spend more bits everywhere" instinct is leaving quality on the table.

And the mechanism is stock llama.cpp — no custom runtime, one flag:

llama-quantize \
  --tensor-type "blk\.([0-9]|[12][0-9]|3[0-5])\.ffn_.*=q2_k" \
  --tensor-type "blk\.(3[6-9]|4[0-7])\.ffn_.*=q4_k" \
  --tensor-type "attn_.*=q4_k" --token-embedding-type q4_k \
  gemma-4-12B-f16.gguf out-depthaware.gguf Q2_K 8

You can run that tonight. The catch: those layer ranges are right for Gemma. For your model they're probably wrong — and guessing can cost you a 25x fragility differential. That's the problem quantprobe solves, and I'll get to the commands in a minute.

Why this works: four laws

I'm Federico, a solo researcher (AI-supported). Everything here was measured on one 2016 desktop: i5-7600K (4c/4t), GTX 1060 6 GB (Pascal, ~$150 used, 192 GB/s), 16 GB DDR4, SATA SSD at 0.45 GB/s. Months of falsification experiments on that box distilled into four laws:

1. Rotation is rank-conditional. Incoherence rotation (QuIP#/QTIP/QuaRot) costs +0.006 PPL on a full-rank MLP and +1623 PPL on a low-rank KV-latent — a ~270,000x swing on effective rank alone. It is not a universal good.
2. Trained networks are dense everywhere. Experts sit at the Gaussian rate-distortion floor; routing is domain-flat (prose vs code use identical expert sets, Jaccard 1.00); 1-bit collapses (+253 PPL) under every codec. ~2-bit is the data-free floor — there is no free sparsity.
3. Fragility is measurable, not predictable. Gemma is late-fragile (~4x), Qwen late (~2–3x), Mistral — Qwen's architectural near-twin — is early-fragile at ~25x. Weight statistics mislead. Only a ~30-minute functional probe decides.
4. The tiered decode law. tok/s = η(tier) × bandwidth ÷ active-bytes-per-token, with η collapsing per memory tier (VRAM 0.56, RAM 0.29–0.68, disk 0.88–1.0). One equation, validated from 7B to 744B.

The depth-aware recipe isn't a trick; it falls out of Law 3. It took Gemma 4 12B's 2-bit quality cost from 1.91x to 1.45x at ~4.5 GB resident, and — data-free — edged an imatrix-calibrated community Q2_K at 30B scale (11.08 vs 11.27) with zero calibration data.

How I know the laws are real

I pre-register: write the number down, then run the hardware.

Prediction (written first)	Measured (after)
110B (GLM-4.5-Air) streamed from SATA, 16 GB RAM: 0.2–0.3 tok/s	0.19
RAM overclock (XMP 2133→3000) scales dense decode ×1.41+	×1.52
Qwen3-30B-A3B hybrid on the 2016 box: ~19 tok/s	19.30 ± 0.88
Laguna S 2.1 on DGX Spark, from the config alone: ~47 tok/s	47.5 published

That last row is the fun one. The day after Poolside shipped Laguna S 2.1 (117.6B MoE, 8B active, NVFP4, single DGX Spark) — before any benchmark was out — I predicted its decode from the config alone: 8B × NVFP4 = 4.6 GB/token, 273 GB/s, η≈0.79 → ~47 tok/s. MiaAI-Lab's published ×2 per-stream number: 47.5. Within 1%, and I never touched the hardware. The law even caught that their ×1 number (76.7 tok/s) implies η = 1.29 — impossible in a single memory pass — mathematically detecting that speculative decoding was switched on.

Hands-on: do this today

quantprobe packages the loop — 8 commands driving stock llama.cpp (validated on build b9596+, which has --tensor-type regex support).

pip install git+https://github.com/FedericoTs/quantprobe   # PyPI release pending

Predict before you download. plan needs nothing installed — it computes from Law 4. 16 machine presets ship in (GTX 1060 → RTX 5090, Apple M-series, DGX Spark, Epyc), or pass raw specs:

quantprobe plan --model qwen3-30b --machine 2016-xmp
# or: quantprobe plan --model qwen3-30b --vram 8 --vram-bw 300 --ram 32 --ram-bw 50 --disk-bw 2

For my box it predicts 18.9 tok/s for the config I measured at 19.30 ± 0.88.

Build a depth-aware ~2-bit GGUF directly:

quantprobe quantize --gguf model-f16.gguf --out model-2bit.gguf

It protects the fragile band, drops the rest to ~2-bit, and verifies the output loads and generates. This is the recipe class that put a 16B MoE (DeepSeek-V2-Lite) fully resident on my 6 GB card at PPL 6.31 → 6.96 — a 1.10x gap that beats calibrated SOTA MxMoE's 1.18x without any calibration data.

Probe YOUR model's fragile band (~30 min on a 12B):

quantprobe probe --gguf your-model-f16.gguf --eval wiki.test.raw

Expected output — the spike is the fragile band, and the recipe follows automatically:

quantprobe probe: gemma-4-12B-f16.gguf | 48 layers -> 4 bands
[2/3] band probe (one band's FFNs -> Q2_K at a time)
  layers 0-11 : PPL 9.51  (delta +2.14)
  layers 12-23: PPL 10.59 (delta +3.22)
  layers 24-35: PPL 10.53 (delta +3.16)
  layers 36-47: PPL 15.35 (delta +7.98)   <- fragile band
[3/3] recipe: protect layers 36-47 at Q4_K
  llama-quantize --tensor-type "blk\.(3[6-9]|4[0-7])\.ffn_.*=q4_k" ...

Add --apply and it builds the recommended GGUF for you.

Run with the law-optimal placement:

quantprobe run --gguf model-2bit.gguf --model qwen3-30b --machine 2016-xmp

This plans placement, prints the prediction, and drops you into llama.cpp chat with the right flags already set. Those flags matter: on my Pascal card, serving MoE experts from CPU RAM instead of VRAM (-ot "exps=CPU") was +54% (11.81 vs 7.67 tok/s), and hybrid placement beat llama.cpp's naive best by +53% (19.3 vs 12.6 tok/s) on the same GGUF. That's a 30B MoE at 19.3 tok/s on a 2016 desktop.

Test the law on your own box:

quantprobe bench --gguf model-2bit.gguf --model qwen3-30b --machine 2016-xmp

bench runs a real llama-bench and prints predicted-vs-measured — a 7B smoke test landed within 7% of its file-calibrated prediction. Run it and you've personally tested the law.

No install at all? The live calculator does plan/target in the browser: https://federicots.github.io/quantprobe/

Help the law grow

quantprobe bench --contribute

This prints a pre-filled, opt-in data point you can submit to the η dataset. You review it before anything is sent — zero telemetry, nothing auto-sent, fully reviewed. Every new machine tier tightens the bands, and the most valuable contributions are the ones that break a prediction. The Mac and modern-GPU presets are exactly that: falsifiable predictions waiting for someone with the hardware.

Limits, stated plainly

• Primary metric is WikiText-2 perplexity; no task-level evals (MMLU/HellaSwag) yet.
• The fragility atlas covers four model families — enough to disprove universality, not to chart every architecture.
• The 110B-from-SATA 0.19 tok/s is a capacity demo, not usable inference.
• Every number of mine comes from one box; speed varies ±25% across environments, and the η values are fitted, not derived.
• Some documented dead ends, because a law you only confirm is a law you haven't tested: dynamic top-k routing — dead; semantic expert paging — dead (domains use identical expert sets); self-speculation on bandwidth-bound MoE — measured 2.3x slower.

Repo (MIT, v1.0.0, CI green): https://github.com/FedericoTs/quantprobe · Calculator: https://federicots.github.io/quantprobe/

Probe first. Placement beats budget.