PCST: A Systematic Study of Extreme Low-Bit LLaMA-7B Compression Without Retraining

Ilia — Fri, 04 Sep 2026 14:52:00 +0000

What Works, What Fails, and Why Local Weight Error Poorly Predicts Model Quality

Project: PCST — Product Code Structured Transform

This article deliberately reports both positive and negative results. It does not claim that PCST outperforms modern standard quantization. Its purpose is to document a reproducible search for the limits of LLaMA-7B compression without retraining and to show which locally promising methods fail end-to-end validation.

Publication links

Formal preprint and DOI: Zenodo record 22304197, DOI: 10.5281/zenodo.22304197.
Russian-language article: published on Habr — public URL to be added.
Source code and reproducibility package: public GitHub release is in preparation.

Abstract

We study extreme low-bit compression of LLaMA-7B weights without retraining the original model, distillation, pruning, or architectural changes. The goal of PCST—Product Code Structured Transform—is to find a compact representation of capabilities the model has already acquired, using post-training mathematical transformations and a small calibration corpus.

The project evaluated more than 60 documented methods and variants, and substantially more individual configurations, including Product Quantization, SVD and low-rank residuals, Hessian- and activation-aware optimization, orthogonal gauge transformations, additive and residual quantization, sparse corrections, joint optimization of coupled matrices, techniques inspired by image and signal compression, and output-head calibration. Early project results were invalidated after a tensor-orientation bug was discovered. All central claims in this article use only the corrected pipeline.

The final reproducible PCST v5 artifact occupies 2.05 GiB. On 32 independent WikiText-2 validation documents with 6,688 prediction positions, it achieves perplexity 15.56, Top-10 overlap with Q8 of 59.59%, and next-token accuracy of 43.56%. By comparison, a 3.07 GiB Q3_K_M model reaches perplexity 7.36, Top-10 overlap 86.93%, and next-token accuracy 56.25%. PCST v5 is therefore 1.02 GiB smaller than Q3_K_M, but remains substantially behind it in quality and runtime performance.

The strongest positive result is a low-rank bridge between PCST hidden states and the LM Head. Selected on a development split and evaluated once on a sealed test split, the bridge improves Top-10 overlap by 1.706 percentage points at unchanged Q5 head payload, reduces KL from 0.5478 to 0.4942, and increases logit cosine similarity from 0.8538 to 0.8823. The central negative finding is that reducing weight MSE or local block error frequently fails to improve—and can even damage—final token ranking.

This work does not establish a fundamental impossibility of compressing a 7B model to 1–2 GiB. It identifies a practical limit of the tested independent-PQ-plus-local-corrections design and motivates a different direction: network-wide joint representation and error shaping rather than an accumulation of isolated matrix-level improvements.

1. Motivation

A language model stores billions of numbers, but useful behavior may not require every number to be preserved exactly. The weights are one particular encoding of a computation, not necessarily its shortest possible description.

The practical motivation is direct. A model that runs locally on a small device does not depend on connectivity, an external API, or cloud infrastructure. Such models matter for autonomous systems, field devices, personal assistants, education, and regions where reliable connectivity is unavailable or expensive.

The project began with an intentionally ambitious target: approach 1 GiB while retaining quality close to Q8. That target has not been reached. Instead of hiding the gap, we treat it as a measurable research problem.

Model intelligence != exact storage of the original weights
Model intelligence = preservation of the relevant computation

2. Scope and Constraints

The main PCST track permits:

activation statistics from a calibration corpus;
SVD, least squares, eigendecomposition, and orthogonal transforms;
post-training selection of codes, codebooks, scales, and bit budgets;
function-preserving reparameterizations;
folding corrections into existing matrices;
storage-format and inference-kernel optimization.

It excludes:

gradient fine-tuning of the original Transformer blocks;
distillation into a different model;
parameter, channel, or neuron pruning;
changing depth, hidden size, head count, or FFN width;
training the original model on task labels.

Calibration that uses teacher hidden states or logits is explicitly described as teacher-aware post-training calibration, not as an exact transformation.

3. A Critical Bug and Experimental Restart

On August 21, 2026, we found an orientation error in the NumPy loader. A GGUF tensor already decoded as (out, in) was reshaped rather than transposed for an x @ W implementation. This permuted matrix elements and invalidated the early quality measurements.

After the fix, the NumPy Q8 forward pass was validated against llama.cpp on identical tokens:

logit cosine similarity: 0.999942;
relative error: 0.01140;
Top-1 agreement: 100%;
Top-10 overlap: 97.5%.

Pre-fix experiments are retained as a record of infrastructure and hypotheses, but they are not used as evidence for method quality. The incident provides a broader methodological lesson: a custom decoder must first reproduce a trusted runtime on an effectively uncompressed model.

4. PCST Representation

PCST is an experimental heterogeneous weight representation. Different layers and matrices receive different encoders and bit budgets. Its primary building block is Product Quantization (PQ).

A matrix is partitioned into short vectors of length B. Each vector is represented by the index of its nearest centroid in a codebook of size K:

w_block ≈ C[index]

The real footprint includes indices, codebooks, scales, low-rank factors, normalizations, embeddings, the LM Head, and metadata. We therefore distinguish logical payload, artifact bytes, peak process RSS, and steady-state working memory.

PCST does not impose one configuration on the full model. Sensitivity changes by matrix type and network depth. Compression is therefore treated as a rate–distortion allocation problem: where should the next megabyte be spent to produce the largest improvement in final model behavior?

5. Experimental Protocol

5.1. Model and reference

model: LLaMA-7B from the Llama 1 family;
source checkpoint: compatible Q8_0 GGUF;
primary teacher-logit reference: Q8;
standard GGUF runtime: llama.cpp;
PCST v5 evaluation loads the materialized .pcstc artifact.

5.2. Calibration data

The final calibration corpus contains 24,480 tokens across 360 reset contexts drawn from WikiText, BoolQ, HellaSwag, and PIQA:

train: 16,320 tokens;
development: 4,080 tokens;
sealed test: 4,080 tokens.

Splits are document-separated. The sealed test split is not used for greedy policy selection.

5.3. External WikiText evaluation

The final comparison uses 32 independent WikiText-2 raw validation documents, up to 256 tokens per document, with BOS and state reset at document boundaries. It evaluates 6,688 prediction positions using identical tokenization and positions for every format.

5.4. Metrics

No single metric is treated as “intelligence.” We report:

ground-truth NLL and perplexity;
next-token target accuracy;
Top-1, Top-10, and Top-50 agreement with Q8;
KL(Q8 || PCST);
logit cosine similarity and relative Frobenius error;
teacher-probability MassRecall@10;
logit-margin error;
local activation error for diagnostics;
artifact size, throughput, and peak RSS.

6. Methods with Reproducible Positive Signals

6.1. Non-uniform bit allocation

Early layers and individual matrix types have very different effects on final logits. Spending additional bytes on V/O in the early network produced substantially more value than the same allocation in upper layers. This motivated a selective layer-by-matrix policy rather than uniform precision.

6.2. Column-RMS for attention O

After fixing the orientation bug, Row-RMS normalization harmed V and FFN matrices, while Column-RMS was critical for O. On corrected L0, local O error fell from 0.6791 to 0.1619, while cosine increased from 0.9283 to 0.9920.

The result cautions against universal normalization recipes: the same transform can help O while damaging V, Gate, or Up.

6.3. Selective SVD residuals

For a small subset of matrices, the residual left by coarse PQ admits a useful low-rank approximation:

W ≈ W_PQ + U_r S_r V_rᵀ

A rank-32 correction for FFN Down was worthwhile only in layers {0, 1, 2, 4, 30, 31}. Sequential validation retained local error reductions between 42.5% and 91.4%. V/O residuals were retained only for layers 24, 27, and 31. Residuals are therefore best treated as rare corrections for anomalous matrices, not a universal quality layer.

6.4. Function-preserving Hadamard gauges

Coupled orthogonal transforms can change weight geometry without changing the exact model function, provided related matrices are transformed consistently. This creates an opportunity to search for coordinates that quantize more efficiently.

The effect was strongly layer-specific. Seed 4 was retained for V/O in L5 and L8 after corrected validation, while broader multi-seed or global application did not generalize.

6.5. Sequential calibration

A compressed layer does not receive clean Q8 activations; it receives the already distorted outputs of all preceding compressed layers. Later layers were therefore refined on actual PCST-prefix states. The improvement was modest but reproducible, and the method establishes an important principle: local compression must be calibrated on the distribution induced by the compressed network itself.

6.6. Static components

Embeddings and the LM Head require separate policies:

Q4 group-32 embeddings provided substantial savings with limited damage in corrected short-context tests;
a Q5 group-32 LM Head reduced its footprint from roughly 132.8 to 85.9 MiB while retaining local logit cosine 0.9989;
aggressive PQ of the LM Head reduced MSE but severely damaged ranking.

The materialized quality artifact uses Q8 embeddings and a Q5 LM Head with a baked bridge.

6.7. Low-rank head bridge

After 32 compressed layers, PCST hidden states occupy a slightly shifted geometry, while the original LM Head still decodes them in the Q8 coordinate system. We estimated a low-rank mapping from PCST to Q8 hidden states and folded it into the Q5 LM Head. This required no additional runtime operation and did not change head payload.

The rank=32, alpha=0.7 candidate was selected on development data and evaluated once on the sealed test split:

Metric	Q5 baseline	Q5 + bridge	Change
Top-1 agreement	65.025%	65.809%	+0.784 pp
Top-10 overlap	62.917%	64.623%	+1.706 pp
Top-50 overlap	63.349%	65.456%	+2.108 pp
MassRecall@10	71.813%	72.256%	+0.443 pp
KL	0.54776	0.49419	−0.05358
Logit cosine	0.85381	0.88232	+0.02850

This is the strongest validated quality improvement in the project. It suggests that part of the network-wide distortion is systematic and can be corrected by a compact output mapping.

7. Methods That Failed—and What They Teach Us

“Failed” below refers to a tested recipe, bitrate, objective, and protocol. It does not invalidate an entire research family.

7.1. Image geometry does not transfer directly to raw weights

Haar wavelets, quadtree coding, Gaussian splatting, fractal approximation, and coordinate polynomials rely on local spatial smoothness or repeated neighboring structure. Raw LLaMA matrices largely lack this geometry. Neighboring V coordinates at L0 had near-zero correlation, and adjacent differences had approximately 1.414× the RMS of the original weights.

7.2. Naive cross-layer sharing is weak

Direct base-plus-delta coding, shared FFN codebooks, shared V/O subspaces, and neighboring-layer prediction yielded little useful compression. Layers may serve similar roles while using unrelated coordinate systems. This leaves open shared error dictionaries or alignment after a suitable gauge, but not naive subtraction of raw weights.

7.3. Better reconstruction does not imply better language behavior

Residual VQ, additive quantization, Hessian-aware centroid fitting, joint V/O, and joint FFN frequently improved local objectives without improving final token ranking.

The clearest example is alternating joint V/O at L16. Updating all codes and codebooks for a local A·V·O objective at constant payload damaged independent development performance:

Top-10: 62.42% → 56.61%;
cosine: 0.8620 → 0.8130;
KL: 0.6174 → 0.8970.

A local objective can redirect error into dimensions that are amplified by the remaining network.

7.4. Ranking-aware objectives overfit easily

Top-K is discrete and high variance. Improvements found on 68–272 positions often vanished on independent documents. A margin-subspace head refinement trained on 16,320 positions and evaluated on 4,080 development positions also failed:

Top-10: 62.42% → 60.76%;
target Top-1: 38.16% → 37.39%;
NLL: 3.2337 → 3.2637.

Direct ranking optimization therefore needs cross-validation, strong regularization, and guardrails on NLL, KL, and probability mass recall.

7.5. Lossless coding is nearly exhausted

The entropy of current PQ indices is close to their stored bit width. The remaining ideal entropy gain was estimated at only 17.5 MiB, far short of the 2.05-to-1.5 GiB gap. A general custom bit-packed container was larger than the current representation once metadata was included.

7.6. Other tested recipes

The following were not accepted at their tested budgets: Chebyshev/polynomial SVD, universal per-head SVD, Tensor Train, sparse coding, sparse FP16 outliers, hierarchical/residual/additive VQ, raw-weight shared codebooks, LM Head–embedding tying, B-frame and inter-layer delta coding, graph ordering, Block-KLT, binary layers, hidden-state anchors, entropy-constrained PQ, tile-wise dense/sparse schemes, bounded QTIP-like and VPTQ-style pilots, fixed-codebook joint V/O and FFN, and aggressive margin-aware head correction.

These experiments do not constitute full replications of AQLM, QTIP, or VPTQ; they close only the compatible bounded recipes tested inside PCST.

8. PCST v5 Artifact

The materialized llama-7b-pcst-v5-quality.pcstc contains 32 compressed Transformer layers, 224 PQ streams, selective SVD residuals, selected gauges, sequential refinement, calibrated output RMSNorm, Q8 embeddings, and a Q5 LM Head with a baked rank-32 bridge.

Validation results:

artifact size: 2,204,308,700 bytes = 2,102.19 MiB = 2.05 GiB;
lossless PQ stream round-trip: passed;
container-to-source-policy numerical mismatches: 0;
deterministic hidden-state delta: 0.

9. Main External Results

Format	Size	PPL ↓	Top-1 vs Q8 ↑	Top-10 vs Q8 ↑	Target Top-1 ↑	Throughput*	Peak RSS*
Q8_0	6.67 GiB	6.922	100.00%	100.00%	57.13%	20.33 tok/s	13.80 GiB
PCST v5	2.05 GiB	15.560	60.47%	59.59%	43.56%	2.43 tok/s	4.26 GiB
Q3_K_M	3.07 GiB	7.356	89.89%	86.93%	56.25%	18.64 tok/s	6.12 GiB
Q4_K_M	3.80 GiB	7.009	95.01%	92.65%	56.89%	30.36 tok/s	7.13 GiB

* PCST uses a research Python decoder, while GGUF runs through optimized llama.cpp. The values describe the current implementations rather than the intrinsic performance limits of the formats.

PCST is approximately 3.25× smaller than Q8 and 1.02 GiB smaller than Q3_K_M. However, its perplexity is 2.12× that of Q3_K_M. Matching Q3 NLL would require removing approximately 92.5% of PCST's current excess NLL over Q8. The remaining gap is therefore not an incremental tuning problem.

10. Interpretation

10.1. What the study establishes

A technically functional 2.05 GiB LLaMA-7B representation can retain a substantial portion of Q8 behavior without retraining.
Sensitivity is highly non-uniform across layers and matrices.
A small number of anomalous matrices benefit from selective low-rank residuals.
Calibration should use activations produced by the compressed prefix.
Part of the final error is systematic and correctable by a compact head bridge.
Local MSE is useful for diagnosis but insufficient for model selection.

10.2. What the study does not establish

PCST v5 does not beat standard quantization on the quality–size–speed frontier.
The result cannot be described as Q6-, Q4-, or Q3-equivalent quality.
The current Python runtime does not demonstrate Raspberry Pi readiness.
One LLaMA-7B experiment does not establish transfer to other models.
The study does not prove a fundamental limit for no-training compression.

11. Threats to Validity

Single model. The policy may exploit Llama 1 7B-specific behavior.
Limited external corpus. 6,688 positions provide an honest checkpoint, not a complete capability evaluation.
Incomplete downstream evaluation. The current HellaSwag fast check is too small for a paper claim.
Missing equal-size IQ baselines. Strong Pareto claims require IQ2_XXS, IQ2_XS, IQ2_S, TQ2_0, and Q2_K comparisons.
Different runtime maturity. Python PCST and llama.cpp do not provide an apples-to-apples systems comparison.
Multiple hypothesis testing. Many experiments increase the chance of accidental development-set wins; central results require sealed tests and confidence intervals.

12. Next Research Directions

12.1. Equal-size baselines

The immediate priority is to evaluate IQ2_XXS, IQ2_XS, IQ2_S, TQ2_0, and Q2_K produced from the same source checkpoint, with the same importance matrix where applicable, using the identical benchmark runner.

12.2. Hybrid IQ–PCST

If an existing i-quant is a stronger bulk encoder than independent PQ, PCST can become a global optimizer around it:

i-quant for most weights;
PCST sensitivity mapping for bit allocation;
selective residuals for rare critical matrices;
a head bridge for systematic output error;
mixed precision under a fixed global byte budget.

12.3. Joint `Q + LR` decomposition

Instead of computing PQ(W) and then SVD(W − PQ(W)), jointly solve:

W ≈ Q + LR

under an activation-weighted objective, while quantizing both the base and low-rank factors. This CALDERA-like direction is materially different from the sequential residual SVD already tested.

12.4. Network-wide causal allocation

Each matrix should be valued by final benefit per stored byte:

value(matrix) = improvement in final NLL / additional artifact bytes

A counterfactual audit can temporarily restore one matrix to Q8 inside the complete PCST model. This may reveal a small set of causal bottlenecks that local reconstruction metrics miss.

12.5. Native runtime

Once a competitive representation is found, the systems path includes mmap storage, fused lookup-and-accumulate kernels, direct multiplication without FP32 matrix reconstruction, AVX2/AVX-512 and ARM NEON implementations, layer streaming, and a controlled memory arena.

13. Conclusion

PCST v5 does not meet the original goal of 1–1.5 GiB at near-Q8 quality, and it currently trails Q3_K_M. It nevertheless provides a useful map of the search space.

Extreme Transformer compression cannot reliably be built as a sum of local victories. Error travels through the residual network, shifts downstream activation distributions, and can improve in MSE while token ranking becomes worse. The strongest positive result—the head bridge—works precisely because it corrects a systematic network-level mismatch.

The next question is therefore not “which compressor should be applied to another matrix?” It is:

How can we encode the computation of the complete network so that layer-wise errors cancel on the real activation manifold?

We do not yet have the answer. We do have a reproducible formulation, an honest baseline, and dozens of dead ends that no longer need to be rediscovered.

References

Frantar, E. et al. GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers, 2022.
Chee, J. et al. QuIP: 2-Bit Quantization of Large Language Models With Guarantees, 2023.
Egiazarian, V. et al. Extreme Compression of Large Language Models via Additive Quantization, 2024.
Liu, Y. et al. VPTQ: Extreme Low-bit Vector Post-Training Quantization for Large Language Models, 2024.
Saha, R. et al. Compressing Large Language Models using Low Rank and Low Precision Decomposition, 2024.
ggml-org. llama.cpp Tensor Encoding Schemes.

DEV Community: Ilia