My RTX PRO 4000 Blackwell SFF went from 35.04 to 58.53 tokens per second on Qwen3.8 27B with DFlash2. The best observed gain was 67.05%, after raising the runtime power cap and tuning memory and core clocks.
The measurement problem was more interesting than the headline. A second Qwen configuration gained much less from power alone, benefited more from memory tuning, and preferred a different profile for maximum generation throughput. The profile I selected for long prompts was not its decode winner.
This is a technical adaptation of my experiment on one card. The 70 W baseline used the same modified VBIOS as the 120 W runs, with zero clock offsets. It is not a factory-firmware-versus-modified-firmware comparison. The card also has no auxiliary power connector: 120 W exceeds the standard 75 W PCIe slot budget. The results below are observations, not recommended power settings.
Define the workload before comparing tokens per second
A local LLM request has two important phases:
- Prefill: processing the input tokens.
- Decode: generating the output tokens.
Both can be reported in tok/s, but they do different work. A long source file followed by a short answer can be dominated by prefill. A short prompt followed by a long answer gives decode more influence. Prompt caching changes the balance again.
I tested two existing inference configurations:
| Configuration | Quantization / decoding | Output per request |
|---|---|---|
| Qwen3.8 27B | NVFP4 + DFlash2 speculative decoding | 512 tokens |
| Qwen3.6 35B A3B | Q4_K_S, no speculative drafter in this series | 640 tokens |
The full identifiers in the CSV are Qwen3.8-27B-iMatrix-NVFP4-256K-DFlash2 and Qwen3.6-35B-A3B-UD-Q4_K_S-256K.
These are different models, quantizations and decoding paths. Their absolute rates are not a quality ranking or a controlled dense-versus-MoE comparison. The useful question is how each configuration responds relative to its own baseline.
The main comparisons use exactly 8,000 uncached input tokens, ECC OFF. Selected tests use 32,000, not 32,768. The grid contains 158 requests total: 79 ECC OFF and 79 ECC ON, across 102 aggregate cells, with only 1–3 requests per cell. That is a broad exploration, not 158 repetitions of the headline result.
The platform was an ASRock B650M Pro RS with a Ryzen 7 7700, 32 GB DDR5-6000 CL30 and a Lexar NM790 2 TB SSD. Software was Debian 13.7, kernel 6.12.38+deb13-amd64, NVIDIA Open 615.71.10 and an existing custom inference build. The Blackwell used its stock blower with automatic fan control. A second RTX 2000 Ada, 16 GB, was present without OC.
Change one factor before combining them
At zero clock offsets, changing the runtime power cap produced these generation rates:
| Power cap | Qwen3.8 27B + DFlash2 | Qwen3.6 35B A3B |
|---|---|---|
| 70 W | 35.04 tok/s | 84.97 tok/s |
| 90 W | 47.05 tok/s | 93.63 tok/s |
| 106 W | 52.05 tok/s | 96.21 tok/s |
| 120 W | 54.18 tok/s | 96.43 tok/s |
| 70 → 120 W change | +54.65% | +13.48% |
27B continued gaining. In this zero-offset grid, 35B was almost flat between 106 and 120 W. That observation does not isolate whether architecture, quantization, speculation or their interaction caused the difference.
Combining power and clock changes gave higher best observed results:
| Configuration | Best generation | Settings |
|---|---|---|
| Qwen3.8 27B + DFlash2 | 58.53 tok/s, +67.05% vs 70 W | 120 W, memory clock +20%, core +180 MHz |
| Qwen3.6 35B A3B | 115.56 tok/s, +36.01% vs 70 W | 120 W, memory clock +33.3%, core offset 0 |
The 67% is a combined-tuning result. It is not the gain from memory OC alone, and it is not a universal multiplier for other inference workloads.
There was also a useful instrumentation trap: initially, the driver showed a 106 W limit while the card still drew about 70 W under load. The draw changed only after additional VBIOS policy fields were raised. A requested setting was not enough to establish that the intervention had taken effect.
The modified VBIOS was loaded into memory through patched NVIDIA open kernel modules using Panchovix's Linux VBIOS Vix Loader. There was no EEPROM flash, soldering, shunt mod or motherboard BIOS modification. Fields decoded as board_fixed_rated_w and slot/input-current entries mattered alongside the visible power ceiling. Those are parser interpretations of GPU policies, not measured physical protection thresholds.
Isolate memory tuning from the power-cap change
For the memory sweep, I held the cap at 120 W and the core offset at zero:
Each curve starts at 120 W with no memory OC. These are not gains relative to 70 W. Lines connect measured points, not predictions; n = 1–3 per point.
On 35B, memory clock +33.3% gave +19.8% decode versus no memory OC. On 27B, the same increase gave +5.0%; +20% memory was slightly better at +6.5%. Prefill barely changed in either configuration.
The 27B reversal is small and needs more repetitions. The higher memory setting also came with a lower actual SM clock during generation. That is consistent with memory and compute competing within a shared power budget, but memory power was not measured separately and the mechanism was not isolated.
Clock offsets also need careful interpretation. The helper used nvmlDeviceGetClockOffsets() and nvmlDeviceSetClockOffsets() for P0 and P2, with separate graphics and memory domains: NVML_CLOCK_GRAPHICS = 0 and NVML_CLOCK_MEM = 2. An accepted offset is not a guarantee of the clock reached during a workload.
The CSV memory offsets 1800, 3600 and 6000 correspond to the approximate +10%, +20% and +33.3% settings discussed here. They are recorded NVML API values, not portable slider instructions for other tools.
Turn phase rates into a workload-specific estimate
The profile I selected was 120 W, memory clock +20%, core +360 MHz. At 8,000 input tokens, ECC OFF, it delivered:
| Configuration / phase | 70 W baseline | Selected profile |
|---|---|---|
| Qwen3.8 27B prefill | 981.08 tok/s | 1,492.34 tok/s |
| Qwen3.8 27B decode | 35.04 tok/s | 58.49 tok/s |
| Qwen3.6 35B A3B prefill | 1,441.07 tok/s | 2,336.75 tok/s |
| Qwen3.6 35B A3B decode | 84.97 tok/s | 112.57 tok/s |
27B's 58.49 and its 58.53 best result are effectively a tie at this experiment's precision. For 35B, I gave up some decode throughput to get faster prefill.
A simple estimate makes that trade-off visible:
response_time ≈ input_tokens / prefill_tps + output_tokens / decode_tps
At 32,000 input tokens, the two 35B profiles measured:
- Balanced (+20% memory clock, +360 MHz core): 2,121.56 tok/s prefill, 82.07 tok/s decode.
- Memory-focused (+33.3% memory clock, zero core offset): 1,972.45 tok/s prefill, 82.28 tok/s decode.
For 500 output tokens, the estimate is 21.18 s versus 22.30 s, about 5% less time for the balanced profile despite its slightly lower decode rate.
That is calculated from measured rates, extrapolating output length from the 640-token test. It is not a measured 500-token end-to-end response. It excludes loading, network overhead, tool calls, scheduling and cache effects. It helps choose a candidate; real application latency still needs measurement.
The headline needs reciprocal arithmetic too:
throughput gain = tuned_rate / baseline_rate - 1
fixed-output generation time reduction = 1 - baseline_rate / tuned_rate
67% more tok/s means approximately 40% less generation time for a fixed token count. It does not mean a 67% shorter complete request.
Compare the last watts inside one loaded instance
The broad grid found candidates. To compare 106 and 120 W more closely, I used ABBA: 106 → 120 → 120 → 106 W, with memory clock +20%, core offset 0, inside one loaded instance of each model. Each power/context combination had two requests.
Putting 106 W at both ends helps balance roughly linear drift; it does not remove every source of variance.
In ECC OFF runs, 27B gained 8.55–8.63% decode from the last 14 W of allowed power. 35B gained 2.16–2.98%. Decode throughput per measured GPU watt fell 2.47–7.30%, depending on model and context.
The benchmark computes decode_tps_per_w per request as decode_tps / decode_power["mean_w"], then averages the request-level ratios. Phase power comes from integrating sampled GPU telemetry with linear interpolation and splitting phases at the first streamed content.
Sampling was approximately 1 Hz. Short phases therefore have few samples. This is GPU telemetry, not wall power, and dividing tok/s by the configured 120 would answer a different question. Neither phase-level efficiency nor throughput alone establishes energy per completed application task.
Audit the headline arithmetic without changing GPU settings
Download model-grid-summary.csv. This Python standard-library example selects explicit cells rather than searching for a maximum across incompatible conditions:
import csv
import sys
with open(sys.argv[1], newline="", encoding="utf-8") as file:
rows = list(csv.DictReader(file))
def cell(model, config):
matches = [r for r in rows if r["model"] == model
and r["config"] == config and r["ecc"] == "OFF"
and r["prompt_n"] == "8000"]
if len(matches) != 1:
raise ValueError(f"Expected one cell: {model}, {config}")
return matches[0]
cases = [
("Qwen3.8-27B-iMatrix-NVFP4-256K-DFlash2", "p120-m20-c180"),
("Qwen3.6-35B-A3B-UD-Q4_K_S-256K", "p120-m33"),
]
for model, config in cases:
baseline, tuned = cell(model, "p70-stock"), cell(model, config)
base, rate = float(baseline["decode_tps"]), float(tuned["decode_tps"])
gain = 100 * (rate / base - 1)
time_saved = 100 * (1 - base / rate)
print(f"{model}: {base:.2f} -> {rate:.2f} tok/s; "
f"+{gain:.2f}% throughput; {time_saved:.2f}% less generation time; "
f'n={baseline["n"]}/{tuned["n"]}')
Save it as audit_llm.py and run:
python3 audit_llm.py model-grid-summary.csv
It reports +67.05% and +36.01% throughput, with approximately 40.14% and 26.47% less fixed-output generation time, respectively. It also prints the baseline/tuned sample counts. This audits the published aggregation; it does not independently reproduce the GPU measurements.
What the experiment does not establish
Selecting the best cells from an exploratory grid can favor noise. The 27B 70 W baseline has one request; its headline tuned cell has two. The 35B baseline and headline tuned cell have two each. No confidence interval or universal optimum is claimed; the 35B memory result is at the highest setting tested.
Response hashes matched within a session/context, but differed across ECC sessions. With speculative decoding, different generated sequences can change the work per delivered token. I therefore do not subtract the two ECC sessions' rates and label that difference pure ECC overhead. Response-hash agreement is also not a broad answer-quality evaluation.
The highest recorded GPU temperature in the grid was 92°C. VRAM, VRM and PCIe contact temperatures were not measured directly. Raising a policy does not improve a connector or increase the motherboard's physical current-handling capability. The standard 75 W PCIe slot budget remains relevant even when the VBIOS modification is loaded into memory. Completing these runs does not establish long-term electrical safety or stability.
The measurements were collected on 24–25 October 2026. They describe that experiment, not a fresh check of the server's current profile or a completed long-term reliability test.
The next measurements I would prioritize are more repeats of shortlisted profiles, a factory-firmware baseline, wall power, direct thermal observations of the power path, and actual application latency with realistic prompt caching.
The result worth carrying into the next benchmark
The 67% result belongs to one configuration under stated conditions. The more transferable lesson is how to choose what to measure: verify the intervention took effect, keep prefill and decode separate, compare each model with its own baseline, and test the input/output mix your application actually uses.
The useful profile is the one that reduces your workload's elapsed time. The largest offset and the highest output tok/s are only candidates.
The original experiment on piszczek.pl contains the full account and six CSV downloads: request-level measurements, the complete grid, selected 8K results, ABBA pairs, control drift and response-time estimates. The exports support arithmetic inspection; they do not include all original logs, prompts, model artifacts or a complete environment for independently reproducing the hardware experiment.
Top comments (0)