📝 Originally published (in Japanese) at forge.workstyle.tech.
TL;DR gpu-unlock changes FP16/BF16 instruction throughput on some RTX 3000-series GeForce cards. An RTX 3090 report measured 33% faster vLLM prefill and 24% less time to the first token. Decode changed by only 1.2%, because it is limited by memory bandwidth.
What Was Being Held Back
According to the README, the affected cards are RTX 3000 series GPUs based on GA102, GA103, GA104, and GA106. It says the unlock doubles the instruction issue rate for FP16/BF16 multiply-accumulate operations using FMA, where the results are accumulated in FP32. In the README’s words, this makes them “as fast as their professional equivalents.” It does not speed up the whole GPU; only this type of operation gets faster. README
According to the explanation of how it works, the signed firmware that runs during Ampere startup—the Booter running on SEC2—has a buffer overflow vulnerability. This can be used to execute code via ROP. Through that path, the module changes write permissions for registers that drivers normally cannot write to, then changes the value that controls the instruction issue rate. The rate is selected using a 4-bit value, with options such as full speed and half speed. How it works
The module is loaded before the NVIDIA driver connects to the card. It changes the setting, performs an FLR (a per-card reset), and then hands control to the unmodified driver. It is neither a UEFI app nor a modified driver. For that reason, the README says there is no need to reapply patches every time a new driver comes out. The module only runs when the target card is specified on the kernel command line; removing that setting restores the original behavior. README
What Gets Faster—and What Doesn’t
The top of the README lists numbers for an RTX 3090 Ti. They are the median of five runs of a dense 8192³ GEMM at 450 W, with results accumulated in FP32.
| Numbers in the README (reported in issue #1) | Before | After |
|---|---|---|
| FP16 | 87.46 TFLOPS | 118.39 TFLOPS |
| BF16 | 87.50 TFLOPS | 157.41 TFLOPS |
| TF32 | 42.58 TFLOPS | 82.61 TFLOPS |
These numbers weren’t measured by the repository’s author. They came from someone who posted in issue #1 and tested an experimental fork with RTX 3090 Ti support. The setup included a “modified NVIDIA 610.43.02 driver” and a custom startup script. The same issue also reports tests with the unmodified package and a standard driver. In those tests, BF16 performance rose from 63.0 to 105.9 TFLOPS. However, the power limit also changed from 300 W to 380 W, so the conditions weren’t the same before and after.
A GEMM speedup doesn’t translate directly into an application speedup. The README says workloads with a lot of matrix computation get faster. It lists LLM prefill, image generation (diffusion), and FlashAttention. LLM decode is limited by memory bandwidth, so it barely changes. INT8 computation is also unchanged. One report comparing outputs found them identical down to the bit. README
Reports from users show a more nuanced picture:
- A vLLM report showed prefill getting 33–36% faster, with similar results at 8K, 32K, and 128K context lengths. issue #15
- On the same card, a GGUF engine showed no change at 8K and a 12–13% speedup at 32K and 128K. The reporter noted that gains depend on the engine and kernels, and that short prompts may see no change at all. issue #15
- Two reports on an RTX 3090 showed 14–16% faster prefill at 16K, with no change at 4K. One MoE model showed no change with a very long prompt of around 32K, because loading experts from RAM took longer. issue #9 issue #33
- Decode was within the margin of error in every report. INT8 GEMM also showed no change in reports that measured it for comparison. issue #15 issue #33
Another factor affecting the results is the power limit. During GEMM, the card stays at its power limit, so the clock drops when it does more work per clock cycle. In one report with the limit set to 220 W, the SM clock fell from about 1,720 MHz to about 1,330–1,365 MHz. Even with roughly 1.9× more work per clock cycle, the real-world speedup doesn’t reach that level. The README also notes that its smallest gain, +25%, came from a card capped at 225 W. issue #15 README
Which Cards Have Been Reported
The compatibility table in the README summarizes reports from users. The reported BF16 GEMM gains come from different benchmarks, so they can’t be compared side by side.
| Card | BF16 GEMM gain (reported) | Source |
|---|---|---|
| RTX 3090 Ti | +68–80% | #1 |
| RTX 3090 | +25–67% | #9 #25 #33 and others |
| RTX 3080 Ti 20 GB (OEM) | Some reports say it remains at half speed; others report about 2× | #28 |
| RTX 3080 10 GB | +70–73% | #14 |
| RTX 3080 20 GB (memory-modified) | +56–59% | #10 #15 |
| RTX 3070 | +73–81% | #10 #12 |
| RTX 3060 12 GB | About +80% | #13 #31 |
The RTX 3070 Ti has not been tested with this module. There are no reports yet for GA103 cards.
The results are mixed for the OEM RTX 3080 Ti 20 GB. In the first report, the unlock appeared to succeed and the setting’s value remained changed, but BF16 was still running at half speed. Downgrading the driver to 610.43.03 raised performance to about 65% of full speed, but it still didn’t reach full speed. Another user with two cards of the same model reported about 2× performance with driver 610.43.03. The other card, which was not unlocked, remained at its original speed. The two users’ cards have different board IDs. As of October 26, the cause is still unknown. issue #28
Things to Consider Before Installing
This article doesn’t cover the installation steps. Please read the README. Instead, here are the points that stood out to me after reading the primary sources.
The primary sources say nothing about warranty coverage. Check with your card’s manufacturer or retailer.
The boot process also changes. In the reports I read, everyone had Secure Boot disabled. Because this loads an unsigned kernel module, the kernel is marked as tainted. issue #9 issue #25 Whether you can unlock the card again after a warm reboot depends on the board. Some users report success, while others only use it after a cold boot. issue #1 issue #25 You can also use it with a card that’s driving a display. However, the screen goes blank between the reset and when nvidia-drm takes over. You’ll need to unlock the card again after resuming from suspend, and hibernation isn’t supported. README
Its relationship to NVIDIA’s terms is unclear. The firmware used by the module is extracted from the NVIDIA driver package and is subject to the NVIDIA driver license agreement. The NOTICE states that permission to redistribute it publicly has not been obtained. The primary sources I read contain no record of NVIDIA commenting on this module. The mechanism itself exploits a vulnerability in NVIDIA-signed firmware. firmware/NOTICE.md
Power consumption and stability are also worth considering. As noted above, gains are small when the power limit is low. Reported figures with a higher limit vary widely in their testing conditions. The longest stability report describes more than eight hours of mixed workloads without errors. issue #25 Another user wrote, “a few hours, not weeks.” issue #33 We still don’t know what happens with long-term use.
Key takeaways
- Measure whether prefill or decode is the source of your wait time.
- Compare reports with similar cards, power limits, drivers, and engines.
- Do not expect decode token/s to improve from this module.
- The README lists image generation as a workload, but the issues reviewed here provide no before and after performance numbers.
- Check Secure Boot requirements, kernel tainting, and the limited long-term stability reports.
Top comments (0)