QUASAR: How Saliency-Weighted Reconstruction Closes the Loss Floor Gap in LLM Quantization-Aware Training

Quantization is one of the most practical tools in the LLM deployment toolkit. Shrinking a model from 16-bit to 4-bit or even 2-bit precision can cut memory requirements by 4–8×, making it possible to run large models on consumer hardware, edge devices, or cost-constrained cloud instances. But quantization is not free — and the further you push it, the more accuracy you sacrifice.

A new paper from August 2026, QUASAR: Lowering the Loss Floor of Quantization-Aware Training with Loss-Aware Reconstruction, identifies a specific, fixable cause of that accuracy loss in Quantization-Aware Training (QAT) and proposes a lightweight solution that reduces held-out KL divergence by up to 29% at 2-bit precision — with only a 1.4% increase in training step time.

The Two Paths to Quantization — and Why QAT Is Harder

Before getting into QUASAR, it helps to understand the landscape. There are two main approaches to quantizing LLMs:

Post-Training Quantization (PTQ) — methods like GPTQ and AWQ — quantize a fully trained model without any further gradient updates. PTQ is fast and requires no training infrastructure, but it struggles at very low bit-widths (2-bit, 3-bit) where the rounding errors compound and accuracy drops sharply.

Quantization-Aware Training (QAT) runs the quantization simulation during training. The model sees fake-quantized weights in the forward pass and learns to compensate for the precision loss. QAT consistently outperforms PTQ at low bit-widths, but it introduces its own problem: a persistent gap between the loss achievable with full-precision weights and the loss floor that QAT converges to. This gap is what QUASAR targets.

The Structural Mismatch at the Heart of QAT

Standard QAT has a subtle but consequential design flaw. During training, the forward pass uses reconstructed (quantized-dequantized) weights r, but the optimizer updates the latent full-precision weights w. These are two different things.

The optimizer computes gradients with respect to r (via the straight-through estimator), but applies those updates to w. Because w and r are related by a lossy quantization function, the gradient signal is not the optimal descent direction for w. The model converges, but to a higher loss than it would if the reconstruction were better aligned with the loss landscape. The authors call this the loss floor gap.

QUASAR's theoretical contribution is showing that the reconstruction error — specifically, the saliency-weighted distance between r and the ideal reconstruction — is the sole reconstruction-dependent term in the QAT convergence bound. Fix the reconstruction, and you fix the floor.

How QUASAR Works: Online Saliency-Weighted Reconstruction

QUASAR adds a lightweight reconstruction step inside the training loop. At each step, before the forward pass, it finds the best possible dequantization parameters (scale s and zero-point z) for each weight tensor, weighted by how much each parameter matters to the loss.

The three-part mechanism:

1. Online Saliency Estimation. Rather than computing the full Hessian (which would be prohibitively expensive), QUASAR approximates per-parameter importance using the exponential moving average (EMA) of squared gradients. This is a well-known Hessian proxy — the same idea used in second-order optimizers like Adam — and it runs essentially for free since the gradients are already computed.

2. Saliency-Weighted Least Squares. Given the saliency estimates, QUASAR fits the affine dequantizer (s, z) in closed form by minimizing the saliency-weighted reconstruction error. Parameters that matter more to the loss get higher weight in this fit, so the reconstruction prioritizes accuracy where it counts most.

3. Clipping Range Search. The code assignment (which integer each weight maps to) depends on the clipping range. QUASAR searches over a small set of candidate ranges, picks the one that minimizes the weighted reconstruction error, and then fits the dequantizer for that assignment. This search is cheap because the candidate set is small and the fitting step is closed-form.

The result is that at every training step, the reconstructed weights r are as close as possible to the ideal reconstruction given the current quantization grid — which tightens the convergence bound and lowers the loss floor.

Results: What the Numbers Show

The paper evaluates QUASAR on Qwen3 and Llama-3.1 model families across 2-bit, 3-bit, and 4-bit quantization, comparing against strong QAT and PTQ baselines.

At 2-bit quantization — the regime where standard methods struggle most — QUASAR:

• Reduces held-out KL divergence by 29% relative to standard QAT
• Improves average accuracy across eight downstream tasks by 3.5–4.3 percentage points
• Outperforms PTQ baselines (GPTQ, AWQ) by at least 10.9 percentage points on supervised fine-tuning for mathematical reasoning

At 3-bit and 4-bit, the gains are smaller but consistent — QUASAR achieves the lowest held-out KL divergence among all competitive QAT methods tested.

The method also extends to floating-point quantization formats. On NVFP4 (NVIDIA's 4-bit float format used in Blackwell-generation hardware), QUASAR reduces held-out KL by roughly 30% relative to standard QAT, with no changes to the inference path.

Training overhead is minimal: the saliency EMA is computed from already-available gradients, and the closed-form dequantizer fitting adds approximately 1.4% to total training step time. There is zero inference-time overhead — the quantized model is deployed exactly as it would be without QUASAR.

Why This Matters for Practitioners

The practical implication is straightforward: if you are running QAT to prepare a model for low-bit deployment, QUASAR is a drop-in improvement to the training loop that costs almost nothing and consistently lowers the loss floor.

This is particularly relevant for teams targeting:

• Edge and mobile deployment, where 2-bit or 3-bit models are necessary to fit within memory budgets of devices running on ARM chips or NPUs
• NVFP4 inference on Blackwell GPUs, where the format is natively supported but QAT quality has been a limiting factor
• Reasoning-intensive tasks (math, code), where the accuracy gap between full-precision and quantized models is largest and most consequential

As recent work on reasoning-QAT has shown, 2-bit quantization of reasoning models is particularly hard — quantization noise disrupts the reasoning trajectories these models depend on. QUASAR's improvement at 2-bit is therefore most valuable precisely where the problem is hardest.

What QUASAR Does Not Solve

QUASAR improves the reconstruction step within QAT; it does not address the higher data requirements of QAT versus PTQ, the training infrastructure needed to run QAT at scale, or the challenge of applying QAT to models not designed with quantization in mind. It also does not replace knowledge distillation or RL-based recovery techniques — those address different aspects of the accuracy gap. QUASAR is best understood as a targeted fix for a specific, well-defined failure mode in the QAT training loop.

The Takeaway

QUASAR is a good example of what careful theoretical analysis can produce: a precise diagnosis of why a widely-used technique underperforms, followed by a minimal, principled fix that is cheap to implement and consistently effective. The loss floor gap in QAT has been a known annoyance for practitioners working at low bit-widths; QUASAR gives them a concrete tool to close it.

The paper includes full experimental details, ablations on the saliency estimation and clipping range search components, and comparisons against a broad set of baselines. For teams running QAT pipelines today, it is worth reading.