Ready is not warm: one SGLang cold start on a T4

I measure what GPU time actually buys. One number surprised me this month, so here it is with the raw timeline.

The setup. One T4 on Colab. SGLang 0.5.21. Qwen2.5-0.5B-Instruct in fp16. I started a clock just before launching the server, the way a scale-to-zero platform starts billing when a worker is created. Then 8 short chat requests, then 60 seconds of health checks, then stop. One run, 5 October 2026.

The timeline.

• 0 s: server launched (billing starts)
• ~371 s: prefill CUDA graph capture finished (it took 287.7 s)
• 386.7 s: /health returns 200, log says "ready to roll"
• ~563 s: first chat request finishes (it took about 176 s)
• ~579 s: second request finishes (about 16 s), the other six within the same second
• 639.6 s: billing stops

After that, 8 more requests on the same server took 0.08 to 0.22 s each.

What this means for the bill. If you split the bill at the health check, boot looks like 60% of it (386.7 of 639.6 s). But the first user got an answer about 563 s in. That is 88% of the bill gone before anyone was served.

So "ready" was not the end of the wait. It was about two thirds of it.

What I can and cannot say.

• The server's own counters matched my client exactly: 8 requests, 264 prompt tokens, 80 output tokens.
• The log does not say what the first 176 s were spent on, so I am not claiming a cause.
• Each health check in the tail shows up in the SGLang log as a one-token prefill on the GPU, but the request counters do not move. So "the GPU did work" and "a request was served" are different things.
• For comparison, on vLLM 0.27.1 on the same card, the 8 requests after ready took 5.8 s in total. Boot there was about 165 of 171 billed seconds.
• This is one run on one small card. A bigger GPU or another model can look different. That is the point: measure your own.

If you run scale-to-zero, record three times, not one:

1. billing start to health check OK
2. billing start to the first finished request
3. the idle tail after the last request

Then price the whole billed time, not just the part where requests were running.

The method and the open-source tool I used are here, if useful: github.com/qaisermehdi3-coder/qvunex

If you have measured this on other cards or servers, I would like to see your numbers.