DEV Community

Qaiser Mehdi profile picture

Qaiser Mehdi

Qvunex. I measure what GPU bills actually serve: cold starts, idle time, cost per request. qvunex.com

Joined Joined on  github website
RTX 3090 vs A100 for a 7B model on vLLM: the cheaper GPU depends on your answer time

RTX 3090 vs A100 for a 7B model on vLLM: the cheaper GPU depends on your answer time

Comments 1
2 min read

Want to connect with Qaiser Mehdi?

Create an account to connect with Qaiser Mehdi. You can also sign in below to proceed if you already have an account.

Already have an account? Sign in
Same RTX 3090, 22x cheaper per token: what concurrency does to vLLM cost

Same RTX 3090, 22x cheaper per token: what concurrency does to vLLM cost

Comments
2 min read
Two vLLM servers on one GPU: the startup check that bites, and the fix

Two vLLM servers on one GPU: the startup check that bites, and the fix

Comments 8
2 min read
A 3090 vs an L40S: 1.09x at batch 1, 0.46x at batch 128

A 3090 vs an L40S: 1.09x at batch 1, 0.46x at batch 128

Comments
2 min read
The same H100 hour cost $1.26 or $0.22 per million tokens

The same H100 hour cost $1.26 or $0.22 per million tokens

Comments
2 min read
Ready is not warm: one SGLang cold start on a T4

Ready is not warm: one SGLang cold start on a T4

Comments
2 min read
loading...