SGLang outputs endless repetition on NVFP4 models: the FP8 lm_head bug

Your NVFP4 model serves fine on vLLM but outputs an endlessly repeated phrase on SGLang, from the very first token, even on a trivial prompt. The response content comes back empty, every request ends with finish_reason: length, and reasoning traces look like this:

need analysis there need analysis there need analysis there ...

If that matches, check the server load log for one line:

Parameter lm_head.weight_scale not found in params_dict

That warning plus degenerate repetition is a specific, known failure with a specific fix. This post is the short version of an A/B verification I ran on a single RTX PRO 6000 Blackwell (SM120, 96 GB).

What is actually broken

Mixed-precision compressed-tensors checkpoints such as unsloth/Qwen3.8-27B-NVFP4 quantize most of the network to NVFP4 but keep a few parts, including lm_head, in FP8 W8A8 with per-channel weight scales. The quant config names the head explicitly with a re:.*lm_head target.

SGLang's CompressedTensorsConfig.get_quant_method() used to dispatch only LinearBase and FusedMoE. ParallelLMHead fell through, so the head loaded via the unquantized embedding path and the FP8 weight_scale tensor was never applied. Raw FP8 weights (values up to around 448) were consumed as if they were bf16. With per-channel scales averaging 1.55e-4 and a 12x row-to-row spread, the logit ranking is corrupted badly enough that greedy decoding locks into a repetition loop immediately.

The same checkpoint works on vLLM because vLLM fixed the identical gap earlier (vllm-project/vllm#37291).

The fix, and where it is not

SGLang fixed this on main on 2026-08-19: PR #35228 added a dedicated get_lm_head_scheme resolver, so a head named by the quant config now routes through the compressed-tensors linear method and its scale loads.

Two things worth knowing as of late August 2026:

• The fix is in no released version. v0.5.18, the latest release, was cut from a branch point that predates the fix, so pip-installed SGLang still has the bug. You need main at or past commit 5375babb.
• Separately, sglang[all]==0.5.18 currently does not resolve from PyPI at all (a cuda-tile==1.6.0rc5 pin with no matching distribution), so you could not stay on the release even if you wanted to.

The A/B, one variable

Same machine, same venv, same launch flags, TP=1, --attention-backend flashinfer, temperature 0. The only difference is the git checkout.

	pre-fix (parent of #35228)	current main
weight_scale not found warnings	1	0
"Which is larger, 9.11 or 9.9?"	repetition, empty content	9.9
"Capital of Australia?"	repetition, empty content	Canberra, plus a correct fact
finish_reason	length, both prompts	stop, both prompts

Full table and logs are in the issue thread: sgl-project/sglang#34895. The before and after rows also live in a public Blackwell serving matrix I maintain of verified model x engine x GPU cells.

Practical guidance

• On SGLang releases up to v0.5.18 with an FP8-lm_head checkpoint: you will hit this. Either build from main past 5375babb or serve that checkpoint on vLLM until an SGLang release ships the fix.
• The failure is silent at startup. The server boots cleanly and the health endpoint is green; only generation is garbage. If you automate deployments, grep load logs for weight_scale not found and fail the deploy on it.
• To identify your GPU and stack cell quickly, blackwell-doctor prints a stable environment key you can match against the serving matrix: github.com/jahnclawdmonet/blackwell-doctor.

I keep an index of exact Blackwell serving error signatures mapped to reproductions here.