RTX 3090 vs A100 for a 7B model on vLLM: the cheaper GPU depends on your answer time

The setup

Same model (Qwen2.5-7B-Instruct), same vLLM (0.27.1, default flags), same load: prompts of about 512 tokens, 256-token answers, 1 to 256 requests at once, 60 seconds per level. Two rented cards: an RTX 3090 at $0.259/hour and an A100 80GB PCIe at $1.017/hour.

Same load: the cheap card wins

• 1 at once: 3090 $1.81, A100 $3.16 per 1M output tokens
• 4 at once: 3090 $0.49, A100 $0.80
• 16 at once: 3090 $0.16, A100 $0.24
• 64 at once: 3090 $0.08, A100 $0.11

At every load we ran on both, the 3090 cost less per token. The A100 does 2.2x to 3x more tokens per second at the same load, but it costs 3.9x more per hour.

Same answer time: the A100 wins

Answers don't take the same time on both cards. At 64 at once the A100 answered in 6.0 s (p50), the 3090 in 17.9 s. So compare at the answer time your users accept:

• About 3 s: only the A100 can do it. The 3090 already takes 6.3 s with one request at a time.
• About 6 to 7 s: 3090 at 4 at once (p50 6.8 s, p95 7.2 s) costs $0.490 per 1M. A100 at 64 at once (p50 6.0 s, p95 7.1 s) costs $0.106. The A100 is about 4.6x cheaper.
• About 18 s: 3090 at 64 at once and A100 at 256 at once both cost $0.082.

What it means

"Which GPU is cheapest" has no answer until you say how fast answers must be and how many requests you can keep in flight. Price per hour alone can point the wrong way when answers must be fast.

How it was counted

Cost = GPU price x the measured window, which runs until the last in-flight request finishes (that time is paid for too). Requests and tokens came from vLLM's own /metrics before and after each level, and a separate client-side meter counted the same things. They matched exactly at every level on both cards. 0 errors.

Limits

These costs assume the GPU is busy the whole time; idle hours in a real day raise them, and only real traffic shows by how much. One run per level, two hosts on different days, synthetic prompts, two vast.ai prices. Other prices change the answer.

Both reports, raw /metrics and server logs: https://github.com/qaisermehdi3-coder/qvunex/tree/main/benchmarks/data/planb-a100-2026-10-29

If you want this for your own model and settings: send the model, your vLLM flags and typical prompt and answer length, and I'll run a short mini-report on a rented GPU for free. The full report is paid work: [e-mail]