I kept seeing claims about single-GPU throughput that sounded too good, so I decided to just show the meter running. The new short is up: https://www.youtube.com/shorts/WoTq58fIag0
The concrete detail: I'm running a 4-bit quantized model through vLLM with continuous batching enabled, and the 776 tok/s figure comes from averaging generation speed over a 30-second curl loop against the /v1/completions endpoint — not a synthetic benchmark, but real streaming output you can watch tick by.
The cost nobody mentions in these threads is that the GPU is pegged at 100% utilization the entire time, pulling roughly 350W from the wall. That one minute of footage cost me about 6 watt-hours and a few cents in electricity, but if you're renting the card by the hour, that minute is a line item. The walkthrough exists because "trust me, it's fast" doesn't scale; a 60-second screen recording does.
Top comments (0)