TL;DR
Short version: no. I dropped a much older GPU (Quadro P2000, 5GB, Pascal, 2016) next to an RTX 3090 (24GB, Ampere) on the same box, ran the same context-length ladder (8K→128K) through Ollama and vLLM on qwen3-coder:30B-A3B, and got zero extra usable context in either engine — and a 74% decode-speed hit for the trouble. Ollama hits the identical Chunk too big wall at ctx=65536 whether the P2000 is there or not. vLLM refuses tensor-parallel across the two cards entirely — not a VRAM problem, a flat compute-capability rejection (Minimum capability: 75. Current capability: 61.) that fails in 40 seconds, before any memory profiling. And the one real, measured effect of adding the P2000 to Ollama: decode speed goes from 76 → 19.5 tok/s at ctx=49152 once the P2000 gets pulled in as an actual compute device.
Full narrative version — the two-stage collapse, the prompt-cache validation bug caught mid-sweep, the CUDA13-silently-drops-Pascal finding — is on Medium.## The setup
ardi (dual Xeon E5-2680 v4, 128GB RAM, openSUSE Leap) has a Quadro P2000 sitting in a second slot next to the RTX 3090 this whole series has run on so far. Same model as phase 1 (qwen3-coder:30B-A3B), same box, four legs: {Ollama, vLLM} × {3090 only, 3090+P2000 tandem}, priced through HomeLab Monitor against real GPU power draw.
Ollama: same wall, extra tax
| ctx | 3090 only decode tok/s | tandem decode tok/s | P2000 VRAM (tandem) |
|---|---|---|---|
| 8,192 | 124.3 | 122.0 | 6 MB / 0% |
| 24,576 | 108.2 | 70.0 | 62 MB / 0% |
| 32,768 | 99.4 | 61.0 | 62 MB / 0% |
| 49,152 | 75.7 | 19.5 | 3,580 MB / 55% |
| 65,536 | fatal: Chunk too big
|
fatal: identical Chunk too big
|
— |
Two separate costs, not one: decode already falls behind at ctx=24576 while the P2000 is still basically idle (62MB, 0% util) — some scheduling overhead just from having a second visible device. Then the real collapse hits at ctx=49152, when the P2000 actually gets pulled into the compute path (3.58GB, 55% util) and decode craters to 19.5 tok/s. Same context ceiling either way, worse speed the whole way there.
vLLM: doesn't even get to try
Expected failure mode going in: tensor-parallel splits the ~17GB AWQ checkpoint roughly in half, and the P2000's 5GB doesn't hold its ~8.5GB share. Actual failure, at ctx=8192, in 40 seconds, before any memory profiling:
ValueError: The quantization method auto_awq is not supported for the current GPU.
Minimum capability: 75. Current capability: 61.
AWQ's Marlin kernel needs compute capability 7.5+ (Turing and later). The P2000 is 6.1 (Pascal). Not a close VRAM call — a flat architectural exclusion, decided before capacity is even checked.
Bonus finding: Ollama's own CUDA13 build almost drops the P2000
Boot log, before any of the above:
skipping CUDA device — compute capability not in compiled architectures
device="Quadro P2000" cc=610
archs="[750 800 860 870 890 900 1000 1030 1100 1200 1210]"
Falls back to a legacy cuda_v12 runtime that does support Pascal — so it works, just via a path most people wouldn't notice without reading boot logs. This 2016 card is now old enough that modern quantized-inference stacks are starting to architecturally step around it, not just outrun it.
What wasn't the point of this one
Not claiming a second GPU is never worth it — a matched pair, or a smaller-but-newer card, is a different setup entirely. This was specifically: does this 5GB Pascal card, next to this 3090, on these two engines, buy anything. Check compute capability against your quantization scheme before you do the VRAM math — it can end the conversation first.
Every number above priced through HomeLab Monitor — open source, MIT licensed — against ardi's real GPU power draw. Full write-up with all four charts and the mid-sweep debugging on Medium.What's the oldest card you've tried to tandem into a rig — did it actually pull weight, or did you just assume it was?
Top comments (0)