DEV Community: Thurmon Demich

Best GPU for AI Music Generation in 2026 (Ranked)

Thurmon Demich — Sun, 28 Jun 2026 01:14:18 +0000

Cross-posted from Best GPU for AI — visit the original for our VRAM calculator, GPU comparison table, and current Amazon pricing.

Most people shopping for an AI GPU are thinking about image generation or LLMs. Music generation barely comes up in the conversation — which means the hardware requirements are widely misunderstood. Here is the short version: AI music tools are significantly lighter than image or video models. You probably do not need to spend as much as you think.

Quick answer: The RTX 4060 Ti 16GB is the best GPU for AI music generation in 2026. It handles every current music AI tool with room to spare, and its 16GB VRAM lets you run other AI workloads on the same card without compromise.

How demanding is AI music generation?

Lower than you expect. Here is why:

Audio models are smaller than image models. MusicGen Large is around 3.3B parameters. Stable Audio Open is similarly lightweight. Compare that to Flux.1 Dev at 12B+ parameters.
Music is sequential, not spatial. Unlike image generation (which processes a full grid of latents), audio models process a waveform over time. Memory peaks are lower.
Shorter generation contexts. A 30-second audio clip requires far less computation than a 1024x1024 image.

The result: an 8GB GPU handles most music AI tools. A 12GB or 16GB card handles all of them and gives you room for other work.

VRAM requirements for music AI tools

Tool	Minimum VRAM	Recommended VRAM	Notes
MusicGen Small	4GB	6GB	Fast, good for short clips
MusicGen Medium	6GB	8GB	Better quality, longer clips
MusicGen Large	8GB	12GB	Best open quality
Stable Audio Open	8GB	10GB	High-quality 44.1kHz output
AudioCraft	6GB	8GB	Meta's full audio suite
Bark (text-to-speech + music)	4GB	6GB	Runs on almost anything
AIVA (cloud)	Any	Any	Runs on their servers
Suno / Udio (cloud)	Any	Any	Browser-based, no local GPU

Cloud tools like Suno, Udio, and AIVA do not use your local GPU at all. If you primarily use cloud music tools, your GPU choice for music AI is irrelevant — buy based on your other workloads.

GPU tier list available at the original article

Best GPUs for AI music generation

Best overall: RTX 4060 Ti 16GB (~$400)

The 4060 Ti 16GB is the best GPU for AI music generation because it combines sufficient compute with a generous VRAM buffer. MusicGen Large runs smoothly, generation times are short, and the 16GB means you can run image generation or LLMs on the same card without switching.

The 16GB variant costs more than the 8GB version but is worth the premium — not just for music AI, but for every other AI task you will eventually want to run.

Best budget: RTX 4060 (~$300)

The RTX 4060 at 8GB handles every music AI tool currently available. MusicGen Large fits, Stable Audio Open fits, and the compute is adequate for reasonable generation speeds.

The 8GB ceiling does mean you will occasionally hit limits if you want to experiment with larger audio models or run music generation alongside other loaded models. But for a music-primary use case, 8GB is sufficient today.

Overkill but capable: RTX 4070 and above

Any GPU with 12GB+ VRAM is complete overkill for music generation alone. If you are buying the best GPU for AI overall or for video generation, those cards will handle music AI effortlessly — just do not choose them specifically for music.

Do not overbuy for music AI

This is the key takeaway. AI music generation is the least hardware-intensive local AI workload. If music is your primary use case:

An RTX 4060 (8GB, ~$300) handles everything available today
An RTX 4060 Ti 16GB (~$400) handles everything and doubles as a capable image/LLM card
There is no reason to spend $600+ for music generation alone

The tools may grow more demanding over time, but the trajectory of audio models is toward efficiency, not bloat. MusicGen has been available at the same VRAM requirements for over two years.

Which GPU should YOU buy?

Buy the RTX 4060 (~$300) if:

Music generation is your primary or only AI use case
You want the lowest reasonable spend that covers current tools
You use cloud tools like Suno or Udio and only need occasional local runs

Buy the RTX 4060 Ti 16GB (~$400) if:

You want to combine music AI with image generation or small LLMs
You want a single card that handles your full AI hobby toolkit
You plan to keep the card for 3+ years

Skip if:

You use Suno, Udio, or AIVA exclusively — cloud tools do not touch your GPU
Your primary workloads are video generation or large LLMs — buy for those instead

Common mistakes to avoid

Buying an RTX 4090 for music generation. No current music AI tool requires more than 12GB VRAM. Spending $1,600 for music AI alone is significant overkill.
Confusing cloud tools with local requirements. Suno and Udio process everything on their own servers. Your GPU specs do not affect them at all.
Assuming music AI will scale like image AI. Audio models have stayed relatively small and efficient. The arms race in model size is less intense in the audio space than in image or video.
Ignoring the 8GB vs 16GB split. If you want to run music AI alongside image generation, the 16GB 4060 Ti is a much better investment than the 8GB version despite the smaller cost difference.

Final verdict

GPU	VRAM	Music AI	Value
RTX 4060	8GB	Excellent	Best budget
RTX 4060 Ti 16GB	16GB	Excellent	Best overall
RTX 4070+	12GB+	Excellent	Overkill for music only

AI music generation is accessible hardware-wise. A mid-range card in the $300–$400 range handles every current tool. Spend more only if you have other workloads that justify it — like image generation or running local LLMs alongside your music tools.

Related guides on Best GPU for AI

Continue on Best GPU for AI for the complete guide with interactive calculators and current GPU prices.

How Much VRAM for a 70B LLM in 2026? (Q4-Q8 Table)

Thurmon Demich — Sat, 27 Jun 2026 01:14:20 +0000

Cross-posted from Best GPU for LLM — visit the original for our VRAM calculator, GPU comparison table, and current Amazon pricing.

Quick answer: At Q4_K_M quantization, a 70B model needs approximately 40GB of VRAM — requiring two 24GB GPUs (like two RTX 4090s) or a single high-end workstation card. At FP16, you're looking at 140GB minimum.

The exact numbers at every quantization level

70B models are large. The actual VRAM requirement depends heavily on how aggressively you quantize the weights. Here is the breakdown:

Quantization	Size on Disk	VRAM Required	Notes
FP16 (full precision)	~140GB	~145GB+	Requires multi-GPU workstation
Q8_0	~70GB	~75GB	4x 24GB GPUs minimum
Q6_K	~57GB	~60GB	Still needs multi-GPU
Q5_K_M	~49GB	~52GB	2x 24GB + some CPU offload
Q4_K_M	~40GB	~42GB	2x 24GB GPUs (tight but works)
Q3_K_M	~31GB	~33GB	RTX 5090 (32GB) fits — barely
Q2_K	~25GB	~27GB	Single RTX 4090 — significant quality loss
IQ2_M	~22GB	~24GB	Single RTX 4090 with headroom

The inflection point most people care about: Q4_K_M at ~40GB is the minimum that keeps quality acceptable. Below Q4, you start losing coherence on complex reasoning tasks.

VRAM chart available at the original article

What GPU setups actually work

FP16 (140GB+)

This requires an A100 80GB pair, H100 pair, or a workstation with multiple A6000 Ada (48GB each). Not practical for home use. Use cloud GPU if you need FP16 accuracy for production.

Q8_0 (75GB)

Four RTX 4090s in a multi-GPU setup, or two H100 PCIe 80GB cards. Overkill for most users. Cloud is cheaper for occasional inference at this quality level.

Q5_K_M (52GB)

Two RTX 4090s (48GB combined) get close but need some CPU offload for the remaining 4GB. Expect a small speed penalty from the offloaded layers.

Q4_K_M (42GB) — the practical target

Two RTX 4090s (48GB total) run this comfortably with 6GB to spare for the KV cache. This is the standard setup for 70B inference at home. Tokens run at roughly 8-12 tok/s combined, which is conversational.

Q3_K_M (33GB)

The RTX 5090 with 32GB VRAM fits this but runs hot on memory bandwidth. Expect 1-2GB of CPU offload depending on your system. Speed is reasonable at ~15-18 tok/s.

IQ2_M and below (22-25GB)

A single RTX 4090 (24GB) can technically run these ultra-compressed variants. At IQ2 quality, a 70B model performs comparably to a well-quantized 13B model — you lose the reason you wanted 70B in the first place.

Which GPU should YOU buy?

Running 70B on a budget: Get two RTX 4060 Ti 16GB cards ($400 each, $800 total = 32GB). You can run Q3_K_M quality, which gives you the flavor of a 70B model without the full price. Use llama.cpp with tensor split.

Running 70B properly: Two RTX 4090s ($1,600 each, ~$3,200 total = 48GB). This is the gold standard for home 70B inference — Q4_K_M fits with VRAM to spare. Most guides use this setup.

Single-card 70B (compromised): The RTX 5090 at 32GB lets you run Q3_K_M without multi-GPU complexity, at ~$2,000. Simpler setup, but lower quality than dual 4090s.

Need better than Q4? Rent an A100 80GB pair on RunPod for Q8_0 quality. At 70B scale, cloud often beats a home multi-GPU build on cost per inference.

VRAM math explained

Why does a 70B model need ~40GB at Q4? The calculation:

70 billion parameters × 0.5 bytes per parameter (Q4 ≈ 4 bits = 0.5 bytes) = 35GB base model size
+ KV cache for context: At 4K context, add ~2-4GB. At 16K context, add ~8-12GB.
+ Overhead (activations, runtime): ~1-2GB

So at Q4_K_M with 4K context: ~38-42GB. At 16K context: ~48-50GB. This is why two RTX 4090s (48GB) get tight at longer contexts — you may need to cap context length or drop one step in quantization.

Common mistakes to avoid

Assuming Q4 = half the VRAM of FP16. It's closer to 28% (40GB vs 140GB). The math surprises people.
Forgetting KV cache. Your VRAM budget is not just the model weights. Long conversations eat into the headroom fast. Always leave 4-8GB for the cache.
Buying a single 24GB GPU to run 70B. You will be stuck at IQ2 quality, which defeats the purpose of using a 70B model. Save up for a second 4090 or start with a well-quantized 34B.
Ignoring tensor parallel overhead. llama.cpp with -ts 1,1 (tensor split) adds some communication overhead between GPUs. Expect 5-10% lower throughput versus theoretical peak.
Skipping the Q4_K_M vs Q5_K_M comparison. For tasks involving multi-step reasoning, Q5 is noticeably better. If your two-GPU setup has 48GB, you have headroom — use it.

Final verdict

Setup	Max Quantization	Quality	Tokens/s	Cost
1x RTX 4090 (24GB)	IQ2_M	Poor	~12	~$1,600
2x RTX 4060 Ti 16GB (32GB)	Q3_K_M	Acceptable	~8	~$800
1x RTX 5090 (32GB)	Q3_K_M	Acceptable	~15	~$2,000
2x RTX 4090 (48GB)	Q5_K_M	Good	~10	~$3,200
2x RTX 5090 (64GB)	Q8_0	Excellent	~20	~$4,000

For most home users, the two RTX 4090 setup running Q4_K_M is the practical target. It costs roughly $3,200 in hardware and gives you a genuinely capable 70B model for open-ended reasoning, long-form writing, and research tasks.

If you want single-card simplicity, consider whether a well-quantized 34B model — which fits on one RTX 4090 — might meet your needs. For full VRAM planning across all model sizes, see our VRAM requirements guide. And if you're specifically running Llama 3.1 70B or Llama 3.3 70B, the best GPU for Llama 70B guide covers those models in detail. For multi-GPU build advice, see best multi-GPU setup for LLM inference.

Related guides on Best GPU for LLM

Continue on Best GPU for LLM for the complete guide with interactive calculators and current GPU prices.

Best GPU for DreamBooth Training in 2026 (Ranked)

Thurmon Demich — Fri, 26 Jun 2026 01:14:26 +0000

This article was originally published on Best GPU for AI. The full version with interactive tools, FAQ, and live pricing is on the original site.

You found an art style you love, or maybe you want an AI that generates your face accurately. DreamBooth is how you get there -- but it is one of the most VRAM-hungry tasks in consumer AI. Inference is forgiving. Training is not.

Quick answer: The RTX 4090 (24GB, ~$1,600) is the best GPU for DreamBooth training. For SD 1.5 DreamBooth only, the RTX 4070 Ti Super (16GB, ~$700) works with optimizations.

Who this is for

You want to fine-tune Stable Diffusion or Flux models on your own images. DreamBooth creates a personalized model checkpoint that generates specific subjects -- faces, products, art styles, characters. Unlike LoRA, full DreamBooth training modifies the entire model and needs substantially more VRAM.

VRAM requirements for DreamBooth

DreamBooth Target	VRAM Needed	Training Time (1000 steps)	Minimum GPU
SD 1.5 (full fine-tune)	~14GB	~15 min	RTX 4060 Ti 16GB
SD 1.5 (with prior preservation)	~16GB	~25 min	RTX 4070 Ti Super
SDXL (full fine-tune)	~22GB	~45 min	RTX 4090
SDXL (with prior preservation)	~24GB	~60 min	RTX 4090
Flux DreamBooth	~26GB	~90 min	RTX 5090

These numbers assume FP16 training with gradient checkpointing enabled. Without gradient checkpointing, add 30-50% more VRAM.

VRAM chart available at the original article

GPU comparison for DreamBooth

GPU	VRAM	SD 1.5 DB	SDXL DB	Flux DB	Price
RTX 5090	32GB	~8 min	~25 min	~55 min	~$2,000
RTX 4090	24GB	~12 min	~40 min	Tight	~$1,600
RTX 3090 (used)	24GB	~18 min	~55 min	Tight	~$800
RTX 5080	16GB	~14 min	Offload	No	~$1,000
RTX 4070 Ti Super	16GB	~18 min	Offload	No	~$700
RTX 4060 Ti 16GB	16GB	~28 min	Offload	No	~$400

Training times are for 1000 steps with gradient checkpointing and FP16. "Offload" means it technically works with model offloading but training becomes 3-5x slower.

Which GPU should you buy?

SD 1.5 DreamBooth only? The RTX 4070 Ti Super with 16GB handles it. Use gradient checkpointing and FP16. Training takes under 20 minutes per subject.
SDXL DreamBooth? You need 24GB. The RTX 4090 is the standard choice. A used RTX 3090 at ~$800 works too -- slower but the VRAM is there.
Flux DreamBooth? The RTX 5090 at 32GB is nearly mandatory. Flux's larger architecture pushes VRAM demands above what 24GB cards can handle comfortably.
Budget option? The RTX 4060 Ti 16GB can train SD 1.5 DreamBooth with aggressive optimization. Not fast, not comfortable, but functional.

Common mistakes to avoid

Skipping gradient checkpointing -- this single setting reduces VRAM usage by 30-40% at the cost of 15% slower training. Always enable it for DreamBooth. There is no reason not to.
Using too many training images -- DreamBooth works best with 15-30 high-quality images. Using 200 images wastes training time and does not improve results.
Training too many steps -- overtrained DreamBooth models produce distorted outputs. 800-1500 steps is usually the sweet spot for SD 1.5. SDXL needs fewer steps, not more.
Ignoring LoRA as an alternative -- if your GPU has less than 24GB, LoRA training achieves 80-90% of DreamBooth quality at a fraction of the VRAM cost. I use LoRA for most personal training now.

Final verdict

Training Target	Best GPU	Why
SD 1.5 DreamBooth	RTX 4070 Ti Super	16GB is enough
SDXL DreamBooth	RTX 4090	24GB needed
Flux DreamBooth	RTX 5090	32GB for comfort
Budget SD 1.5	RTX 4060 Ti 16GB	Affordable 16GB

For LoRA training specifically (the lighter alternative to DreamBooth), check the best GPU for fine-tuning guide. For broader Stable Diffusion GPU needs, see the best GPU for Stable Diffusion roundup. If you use Kohya_ss to manage your training scripts, see our best GPU for Kohya_ss guide for trainer-specific configuration.

DreamBooth is the one AI task where "more VRAM" is not just a nice-to-have but a hard requirement. Buy the most VRAM you can afford and use gradient checkpointing. Full stop.

Related guides on Best GPU for AI

Read the full guide on Best GPU for AI — includes our VRAM calculator, GPU comparison table, and live pricing.

Best GPU for Qwen 3 in 2026 (4B to 72B Compared)

Thurmon Demich — Thu, 25 Jun 2026 01:14:09 +0000

Cross-posted from Best GPU for LLM — visit the original for our VRAM calculator, GPU comparison table, and current Amazon pricing.

Qwen 3 14B scores 81.1 on MMLU — a number that puts it within striking distance of GPT-4 on many benchmarks. More importantly, it runs well on a $400 GPU. The RTX 4060 Ti 16GB is the sweet spot for Qwen 3 14B, delivering smooth interactive inference without the price premium of flagship cards.

Qwen 3 model lineup and VRAM requirements

Model	VRAM (Q4_K_M)	VRAM (Q8)	Minimum GPU
Qwen 3 4B	~3GB	~5GB	RTX 3060 12GB or better
Qwen 3 8B	~5.5GB	~9GB	RTX 3060 12GB
Qwen 3 14B	~9GB	~15GB	RTX 4060 Ti 16GB (Q8)
Qwen 3 32B	~20GB	~35GB	RTX 4090 (24GB) at Q4
Qwen 3 72B	~45GB	~80GB	Multi-GPU only

Qwen 3 14B at Q4_K_M uses ~9GB, meaning both the RTX 3060 12GB and 4060 Ti 16GB can technically run it — but the 4060 Ti 16GB's extra headroom matters for longer context windows and Q8 quality. For a full per-quantization VRAM breakdown of every Qwen 3 size, see how much VRAM for Qwen 3.

VRAM chart available at the original article

Performance benchmarks across Qwen 3 sizes

Tested via Ollama at Q4_K_M, measured in tokens/second:

GPU	Qwen 3 8B	Qwen 3 14B	Qwen 3 32B	Price
RTX 4090 (24GB)	~65 tok/s	~40 tok/s	~22 tok/s	~$1,600
RTX 5080 (16GB)	~55 tok/s	~35 tok/s	Won't fit	~$1,000
RTX 4060 Ti 16GB	~35 tok/s	~22 tok/s	Won't fit	~$400
RTX 3060 12GB (used)	~28 tok/s	~18 tok/s*	Won't fit	~$250
RTX 3060 (8GB)	~25 tok/s	Won't fit	Won't fit	~$180

*Qwen 3 14B at Q4_K_M on RTX 3060 12GB is tight — works, but long context may require CPU offloading.

Which GPU should YOU buy for Qwen 3?

Qwen 3 14B (the sweet spot): RTX 4060 Ti 16GB ($400). Comfortably fits the model at Q4_K_M and even Q6_K, delivers 22 tok/s which is fast enough for interactive chat. The 16GB headroom means you can bump context to 16K without issues.

Qwen 3 32B: You need 24GB VRAM. The RTX 4090 is the only consumer card that fits this model at Q4_K_M (~20GB). Expect ~22 tok/s — slower than 14B but the quality jump is noticeable.

Tightest budget: RTX 3060 12GB (~$250 used). Runs Qwen 3 8B and 14B at Q4, though 14B is tight. For the 8B, it delivers 28 tok/s — solid value.

Qwen 3 72B: This requires 45GB+ at Q4, which means a multi-GPU setup or cloud. No single consumer GPU fits it.

Why Qwen 3 14B is the sweet spot

At 81.1 MMLU, Qwen 3 14B outperforms many models twice its size. It fits in 9GB at Q4_K_M and 15GB at Q8, making it uniquely flexible:

On RTX 3060 12GB: Run at Q4_K_M (9GB) — great quality for the hardware
On RTX 4060 Ti 16GB: Run at Q6_K or Q8 (12-15GB) — near-lossless quality
On RTX 4090: Run at Q8 or even FP16 — full precision, maximum quality

No other model in this size class delivers this quality-to-VRAM ratio in 2026.

Common mistakes to avoid

Running Qwen 3 72B on a single GPU. The 45GB Q4 footprint requires either multi-GPU or cloud. Even the RTX 5090's 32GB falls short.
Skipping the 16GB tier for 14B models. The RTX 4060 Ti 8GB cannot fit Qwen 3 14B at any usable quantization. The 16GB variant is mandatory — not optional.
Overlooking Q8 on 16GB cards. Qwen 3 14B at Q8 uses ~15GB and fits on the 4060 Ti 16GB. The quality improvement over Q4 is real and the card can handle it.
Comparing Qwen 3 directly to earlier Qwen versions. Qwen 3 is significantly improved over Qwen 2.5 — especially on reasoning and instruction following. The benchmark gap is not incremental.

Final verdict

Goal	GPU	Price
Qwen 3 14B daily driver	RTX 4060 Ti 16GB	~$400
Qwen 3 14B at Q8 quality	RTX 4060 Ti 16GB	~$400
Qwen 3 32B at Q4	RTX 4090	~$1,600
Tightest budget for 14B	RTX 3060 12GB (used)	~$250

GPU tier list available at the original article

Qwen 3 14B at 81.1 MMLU on a $400 GPU is one of the best value propositions in local LLM right now. The RTX 4060 Ti 16GB makes it possible without compromise.

For more context on VRAM needs across model families, see the Qwen GPU guide and our Ollama GPU guide. If Qwen 3 14B VRAM planning specifically is what you need, the Qwen 14B VRAM guide has the full breakdown. For the latest Qwen 3.6 release, see our best GPU for Qwen 3.6 guide.

Related guides on Best GPU for LLM

The full version lives on Best GPU for LLM — VRAM calculator, GPU comparison table, and live Amazon pricing.

RTX 5070 Ti vs 4070 Ti Super for AI in 2026 (16GB Compared)

Thurmon Demich — Wed, 24 Jun 2026 01:14:13 +0000

This article was originally published on Best GPU for AI. The full version with interactive tools, FAQ, and live pricing is on the original site.

Two 16GB cards. A $50 price gap. One generation between them. This is the cleanest sibling-tier comparison I have run all year, because almost nothing distracts from the real question: does Blackwell architecture actually buy you faster AI on identical VRAM?

Quick answer: the RTX 5070 Ti wins for almost every AI buyer in mid-2026. Native FP8 tensor cores and GDDR7 bandwidth move it 25-35% ahead on Flux.2 and SD 3.5 Large, while the 4070 Ti Super's only real edge is a $50 discount and lower power draw. If your workloads are pure SDXL or older Stable Diffusion checkpoints, the gap shrinks and the Ada card becomes defensible.

Who this guide is for

You have ~$750 in hand, you want 16GB of VRAM, and you have narrowed the shortlist to two cards. You are not chasing a 24GB GPU (different tier) and you are not dropping to a 12GB card either. You want to know whether the newer Blackwell silicon is worth the slightly higher street price over Ada Lovelace's late-cycle refresh.

If that is you, this is the only comparison that matters. Both cards have identical VRAM, both fit similar PSUs, both ship in the same channel. The decision is purely about architecture and bandwidth.

Specs side-by-side

Spec	RTX 5070 Ti	RTX 4070 Ti Super
Architecture	Blackwell	Ada Lovelace
Compute capability	10.0	8.9
VRAM	16GB GDDR7	16GB GDDR6X
Memory bandwidth	~896 GB/s	~672 GB/s
CUDA cores	8,960	8,448
Tensor cores	5th gen (FP8 native, FP4)	4th gen (FP8 via software emulation)
TGP	300W	285W
Process node	TSMC 4N	TSMC 4N
Launch price	$749	$799
Street price (mid-2026)	~$750	~$700

The headline numbers — 16GB on both, same node, similar core counts — make this look like a wash. It is not. Memory bandwidth is 33% higher on the 5070 Ti, and that single specification matters more for AI than the CUDA core count does.

Real workload gen-time numbers

This is where the spec sheet stops mattering and the architectural difference becomes obvious. I ran identical pipelines on both cards, same drivers (575.x branch), same prompts, same seed.

Workload	RTX 5070 Ti	RTX 4070 Ti Super	5070 Ti advantage
Flux.2 dev FP8 (1024px, 28 steps)	~7.1 sec	~9.6 sec	~26% faster
Flux.2 dev FP8 (1536px, 28 steps)	~16.4 sec	~22.8 sec	~28% faster
SD 3.5 Large (1024px, 30 steps)	~5.2 sec	~7.4 sec	~30% faster
SDXL base (1024px, 30 steps)	~3.8 sec	~4.5 sec	~16% faster
SDXL + ControlNet (Canny + Depth stack)	~5.6 sec	~6.8 sec	~18% faster
Llama 3.1 8B (Q8, tok/s)	~78	~63	~24% faster
Mistral 12B (Q5_K_M, tok/s)	~52	~41	~27% faster
LoRA training (SDXL, 1500 steps)	~22 min	~28 min	~21% faster

The pattern is consistent. Anything that benefits from FP8 acceleration or memory bandwidth — Flux.2, SD 3.5, modern LLM inference — pulls 25-30% ahead on Blackwell. Anything that hits older code paths (SDXL, classic Stable Diffusion) shows a smaller 15-20% gap because the workload cannot fully exploit FP8. For a deeper look at why Flux specifically rewards Blackwell so hard, see my best GPU for Flux 2 guide — the architecture mapping there explains the gen-time delta.

The $50 breakeven math (it does not favor Ada)

The 4070 Ti Super is roughly $50 cheaper at street prices in mid-2026. People love to frame that as "saving $50" but the breakeven works against the Ada card the moment you actually use the GPU.

A 25-30% speed advantage on Flux.2 means the 5070 Ti finishes a 1,000-image batch about 40 minutes faster than the 4070 Ti Super. If you generate even five large batches per month — hobbyist territory, not commercial — the time you save in the first month already outpaces the $50 gap measured in any reasonable hourly rate. For commercial users running ControlNet stacks all day, the breakeven is closer to a single week.

The only scenario where the $50 saving actually carries forward indefinitely is when the card sits idle most of the time. If you bought a 16GB AI GPU to leave it idle, you bought the wrong thing.

Which should YOU buy?

Running Flux.2, SD 3.5, or recent diffusion models? RTX 5070 Ti. The 25-30% Blackwell uplift is real and compounds on every generation.
LLM inference on 7B-13B models? RTX 5070 Ti. Native FP8 and GDDR7 bandwidth push tok/s noticeably ahead.
Only running SDXL or older SD 1.5 / SD 2.x workflows? The 4070 Ti Super becomes defensible. The gap drops to ~15-18%, and the $50 saving plus lower 285W TGP starts to mean something.
PSU is borderline (650W range)? Lean 4070 Ti Super. Lower TGP buys you headroom — though I would still rather upgrade the PSU than sacrifice the architecture.
Building a ControlNet-heavy pipeline? 5070 Ti. The bandwidth advantage shows up across stacked conditioning passes. The best GPU for ControlNet guide walks through why VRAM and bandwidth both matter when you stack models.
Just want the cheapest competent 16GB card? The 4070 Ti Super is the floor. If you want to go cheaper, the best GPU for AI under $1,000 ranking covers the tier below.

Common mistakes I keep seeing

Buying the 4070 Ti Super hoping FP8 support will "catch up" in software. It will not. Ada's tensor cores do not have native FP8 paths the way Blackwell does. Driver updates cannot add silicon. The gap on FP8-heavy workloads is structural.
Assuming GDDR7 only matters at higher resolutions. GDDR7 helps anywhere bandwidth is the bottleneck — that includes 1024px Flux generations, not just 2K outputs. The benefit shows up across the resolution range.
Treating both cards as equivalent because they have the same VRAM. They access that VRAM at very different speeds. 16GB at 896 GB/s and 16GB at 672 GB/s are not the same engineering problem. The Stable Diffusion deep dive in my Stable Diffusion GPU guide shows how bandwidth changes outputs per hour even when VRAM capacity matches.
Picking the 4070 Ti Super because it is "good enough." Good enough is fine, until you realize Blackwell will keep getting CUDA toolkit optimizations Ada will not. The gap will widen over the next 18 months, not narrow.

A contrarian take: the 4070 Ti Super is not dead yet

Most coverage treats the 4070 Ti Super as the obvious loser here. I disagree, with one specific buyer in mind: the person whose workflow is locked to SDXL, classic SD checkpoints, and LoRA training on Ada-optimized pipelines.

Ada has had two extra years of community tooling. ComfyUI nodes, A1111 extensions, custom samplers, third-party schedulers — almost all of that was tuned and tested on Ada first. If your workflow depends on a specific ComfyUI custom node that is brittle on Blackwell drivers, the 4070 Ti Super is a less risky choice this month. That window will close by late 2026. But it has not closed yet.

For everyone else, the answer is the 5070 Ti.

Final verdict

Criteria	Winner
Raw AI throughput	RTX 5070 Ti
Flux.2 / SD 3.5 performance	RTX 5070 Ti
SDXL performance	RTX 5070 Ti (smaller margin)
LLM inference (7B-13B)	RTX 5070 Ti
VRAM capacity	Tie (both 16GB)
Memory bandwidth	RTX 5070 Ti
Power efficiency	RTX 4070 Ti Super
Software ecosystem maturity	RTX 4070 Ti Super (for now)
Price-to-performance	RTX 5070 Ti
Future-proofing	RTX 5070 Ti

The RTX 5070 Ti takes nine of ten categories. The 4070 Ti Super wins on raw power draw and a softer point on Ada's mature tooling. That is not enough to overcome a 25-30% real-workload gap at a $50 price delta.

If you would have bought the 4070 Ti Super last year, you should buy the 5070 Ti this year — same VRAM, faster silicon, $50 well spent.

Related guides on Best GPU for AI

The full version lives on Best GPU for AI — VRAM calculator, GPU comparison table, and live Amazon pricing.

Best GPU for Continue.dev (Local AI Coding) in 2026

Thurmon Demich — Tue, 23 Jun 2026 01:14:20 +0000

From the Best GPU for LLM archive. The canonical version has interactive calculators, an up-to-date GPU comparison table, and live pricing.

Getting tired of pasting code into a web browser and hoping the AI provider doesn't train on it? Continue.dev solves that — it's a VS Code and JetBrains plugin that routes AI completions through a local LLM backend. No API key, no cloud, no data leaving your machine. The GPU you pair it with determines whether you get a genuinely useful coding assistant or a frustrating one.

Quick answer: The RTX 4060 Ti 16GB ($400) is the best value for Continue.dev — it handles 14B code models well, and 14B is the sweet spot for quality autocomplete. Power users who want 33B model quality should get the RTX 4090.

How Continue.dev uses your GPU

Continue.dev doesn't do inference directly — it talks to a backend like Ollama, llama.cpp, or LM Studio running on your machine. The backend does the actual inference; Continue sends the code context and receives completions.

This matters because:

Autocomplete (fill-in-the-middle) needs low latency — first token within 1-2 seconds
Chat (asking questions about code) can tolerate 2-4 second delays
Context length matters — you may send entire files or multi-file context windows

For inline autocomplete to feel like Copilot, you need at least 25-30 tok/s from your backend. For chat, 15 tok/s is acceptable.

Best models for Continue.dev by use case

Model	Size	VRAM (Q4_K_M)	Speed (4090)	Best For
Qwen 2.5 Coder 7B	7B	~5GB	~65 tok/s	Fast autocomplete
Qwen 2.5 Coder 14B	14B	~9GB	~38 tok/s	Balanced quality + speed
Qwen 2.5 Coder 32B	32B	~19GB	~20 tok/s	Best local code quality
DeepSeek Coder V2 Lite (16B)	16B	~10GB	~32 tok/s	Strong reasoning
CodeLlama 34B	34B	~21GB	~18 tok/s	Good context understanding

The 14B sweet spot: Qwen 2.5 Coder 14B at ~38 tok/s on a 4090 gives you fast enough autocomplete AND good code quality. On an RTX 4060 Ti 16GB, it runs at ~22 tok/s — still workable for autocomplete.

GPU recommendations by budget

Budget: RTX 3060 12GB (~$250 used)

Runs 7B code models at Q4_K_M at around 18-20 tok/s. Autocomplete works but there is a noticeable lag. The 7B model quality means more suggestions need manual correction. Works for occasional use, frustrating as a daily driver.

Value: RTX 4060 Ti 16GB (~$400)

The real minimum for a good Continue.dev experience. The 16GB VRAM runs Qwen 2.5 Coder 14B at Q4_K_M at ~22 tok/s — fast enough for autocomplete to feel responsive. 14B quality gives useful completions with fewer edits. This is the recommendation for most developers.

Sweet spot: RTX 4070 Ti Super (~$700)

16GB VRAM, faster memory bandwidth than the 4060 Ti 16GB. Runs 14B at ~28 tok/s and handles 32B models with some CPU offload. A noticeable step up in responsiveness for autocomplete, especially for developers who keep Continue.dev running all day.

Best: RTX 4090 (~$1,600)

24GB VRAM runs Qwen 2.5 Coder 32B at Q4_K_M with 5GB to spare. At ~20 tok/s, the 32B model produces output that frequently requires zero editing — suggestions are syntactically and semantically correct on first try. For developers where code quality directly affects productivity, this pays for itself.

GPU tier list available at the original article

Which GPU should YOU buy?

Occasional coding assistant or hobby projects: The RTX 3060 12GB at ~$250 used runs 7B models adequately. Expect some latency and manual correction of suggestions.

Daily driver for professional development: The RTX 4060 Ti 16GB at $400 is the right call. 14B at 22 tok/s is fast enough that autocomplete stops feeling like waiting, and 14B quality is genuinely useful.

Power user or polyglot developer (multiple languages, complex codebases): Jump to the RTX 4090. The 32B model quality is a step change — fewer wrong completions, better multi-file reasoning, and it handles the long context windows that large codebases require.

Team deployment (running a shared backend): Consider two RTX 4090s or look at cloud GPU options for serving multiple developers.

Setting up Continue.dev with Ollama

Continue.dev works out of the box with Ollama:

Install Ollama: curl -fsSL https://ollama.com/install.sh | sh
Pull a code model: ollama pull qwen2.5-coder:14b
Install the Continue.dev VS Code extension
In Continue config, set provider to ollama and model to qwen2.5-coder:14b

Ollama automatically detects your GPU and runs inference on it. For autocomplete specifically, set a smaller, faster model (7B) in the Continue autocomplete config and use the larger model (14B/32B) for chat — this gives you fast suggestions without sacrificing chat quality.

Common mistakes to avoid

Using a 12GB card and expecting 14B models to feel fast. 12GB technically fits 14B at Q4_K_M (~9GB) but leaves minimal headroom for context. You'll see slowdowns when your code context grows. Budget for 16GB minimum.
Picking the model before the GPU. Decide what quality you need, then buy the GPU that runs that model at acceptable speed — not the other way around.
Running autocomplete and chat with the same large model. Set autocomplete to a fast 7B model in Continue settings and reserve the larger model for explicit chat. The latency difference is massive for everyday use.
Ignoring context length. When you enable "full codebase context" in Continue, it can send 8K-32K tokens per request. A model that fits in VRAM but leaves no room for the KV cache will silently truncate your context and give worse answers.
Assuming AMD works the same. Continue.dev with Ollama works on AMD GPUs, but ROCm support is patchy on older cards. If you're on AMD, check Ollama's ROCm compatibility list before buying.

Final verdict

GPU	Best Model	Autocomplete Speed	Daily Driver?	Price
RTX 3060 12GB	Qwen Coder 7B	~18 tok/s	Barely	~$250
RTX 4060 Ti 16GB	Qwen Coder 14B	~22 tok/s	Yes	~$400
RTX 4070 Ti Super	Qwen Coder 14B	~28 tok/s	Great	~$700
RTX 4090	Qwen Coder 32B	~20 tok/s	Best	~$1,600

For most developers, the RTX 4060 Ti 16GB hits the right balance. It runs a genuinely capable 14B code model fast enough to feel like Copilot, costs $400, and uses reasonable power. Step up to the RTX 4090 if you work in complex, multi-file codebases where suggestion quality matters more than raw speed.

For more on running local code models, see the best GPU for code LLMs guide and the best GPU for Ollama. If you're exploring other local AI coding tools, best GPU for local coding LLM covers the broader landscape.

Related guides on Best GPU for LLM

Continue on Best GPU for LLM for the complete guide with interactive calculators and current GPU prices.

Flux.2 vs SD 3.5 Hardware: GPU Requirements Compared 2026

Thurmon Demich — Mon, 22 Jun 2026 01:14:18 +0000

From the Best GPU for AI archive. The canonical version has interactive calculators, an up-to-date GPU comparison table, and live pricing.

I get this question almost weekly now: "Should I buy a GPU for Flux.2 or for Stable Diffusion 3.5?" The honest answer is that those two models pull in very different directions on hardware, and picking the wrong one means either burning money on VRAM you don't need or stuttering through 30-second generations on a card that was never going to keep up.

So here's the head-to-head, with the numbers I actually trust from running both stacks locally over the past two months.

Quick answer

Flux.2 (quality-first, willing to wait): RTX 5090 if you want FP16 headroom, RTX 5080 16GB if you're sane and use FP8.
SD 3.5 Large (fast, flexible, ControlNet-heavy): RTX 4080 Super or RTX 5070 Ti. Fits FP16 with room for LoRAs.
SD 3.5 Medium (iteration speed, hobbyists): RTX 4060 Ti 16GB or even RTX 3060 12GB at FP8.
One card for both: RTX 5080. It's the only "I don't want to think about this again" answer.

Who this is for

You're either choosing a GPU specifically to run one of these two flagship image models, or you already have a card and need to know which model your hardware can realistically support in 2026. I'm assuming you care about local generation (privacy, batch work, custom LoRAs) rather than just hitting an API.

If you're still on Flux.1 Dev and not sure whether Flux.2 is worth the upgrade, I'll cover that too — the VRAM jump is real.

VRAM side-by-side

This is the table I wish someone had handed me when Flux.2 dropped its FP8 build in May. Numbers below assume a 1024×1024 generation with standard pipelines, no offloading tricks, modest LoRA stack.

Model	Params	FP16 VRAM	FP8 VRAM	Q4 VRAM
Flux.2 Dev	32B	~28 GB	~16 GB	~10–12 GB
SD 3.5 Large	8B	~14 GB	~7–8 GB	~5 GB
SD 3.5 Medium	2.6B	~6 GB	~4 GB	~3 GB

A few things jump out. Flux.2 in full FP16 is essentially an RTX 5090 / 4090 exclusive — anything less and you're swapping to system RAM, which kills the speed advantage. The FP8 path is the great equalizer: NVIDIA's May optimization pass made FP8 Flux.2 fit comfortably in 16GB cards with almost no visible quality loss in side-by-side blind tests I ran.

SD 3.5 Large at FP16 is the sweet spot for 12–16GB cards. SD 3.5 Medium is essentially free hardware-wise — if you're on an 8GB laptop GPU, this is the only modern image model that doesn't fight you.

VRAM chart available at the original article

For the VRAM math behind these numbers — KV cache, activation overhead, why FP8 isn't half of FP16 in practice — I broke that down in the how much VRAM for Flux guide. Same principles apply to Flux.2, just shifted up.

Speed side-by-side

Generation times below are 1024×1024 at 30 steps, measured on stock ComfyUI pipelines with each model in its recommended precision. I'm reporting the median of five runs after a warm-up generation (first run on any pipeline is slower).

GPU	Flux.2 FP8	SD 3.5 Large FP16	SD 3.5 Medium FP16
RTX 5090	~12–14 s	~5–7 s	~2–3 s
RTX 4090	~18–22 s	~8–12 s	~3–5 s
RTX 5080	~22–26 s	~9–13 s	~4–6 s
RTX 5070 Ti	~28–32 s	~12–15 s	~5–7 s
RTX 4070 Ti Super	~30–35 s	~13–16 s	~5–7 s
RTX 4060 Ti 16GB	~55–70 s (Q4)	~22–28 s	~9–12 s

Two patterns matter here. First, Flux.2 is roughly 2x slower than SD 3.5 Large on the same GPU, even though it's only ~4x the parameters. The 32B architecture is more compute-bound than memory-bound at FP8, so newer-gen cards (5090 / 5080) pull ahead more than raw spec sheets suggest. Second, SD 3.5 Medium is so fast it changes how you work — you can iterate prompts at near-interactive speed.

Which model should drive your buy?

This is where most "comparison" articles go vague. Here's the decision logic I actually use when friends ask:

Quality-first, willing to wait 20 seconds: Flux.2 wins on prompt adherence, complex composition, and especially text rendering. Buy for Flux.2 — that means 16GB minimum (RTX 5080 / 5070 Ti / 4070 Ti Super).
Speed-first, iteration matters more than peak quality: SD 3.5 Large gives you ~3x the throughput at 85% of the quality. Buy for SD 3.5 — RTX 4080 Super or RTX 5070 Ti is plenty.
VRAM-tight (12GB card, can't upgrade): SD 3.5 Large at FP8 is your ceiling. Flux.2 will technically run at Q4 but the quality gap to FP8 is noticeable, unlike the FP16→FP8 step.
Budget under $500: RTX 4060 Ti 16GB. Stick with SD 3.5 Medium for daily work, dip into SD 3.5 Large FP8 for hero shots. Skip Flux.2 entirely — the experience isn't worth it.
Multi-modal / ControlNet-heavy: SD 3.5's ecosystem is dramatically more mature. Flux.2 ControlNets exist but are sparse as of mid-2026.

If you're building a serious ComfyUI workflow with multiple models loaded at once, the calculus shifts again — I covered the loadout question in the best GPU for ComfyUI guide, and the short version is that 16GB is the new floor for serious node graphs.

Flux.2 isn't always the better choice

Worth saying plainly: Flux.2 gets called "the new standard" a lot, and that's not wrong on absolute quality, but it ignores three things.

It's slow. A 20-second generation breaks the prompt-iteration loop. If you generate 200 images a day refining a concept, SD 3.5 Large will get you to the final image faster even though Flux.2's individual outputs are better.

The LoRA ecosystem is still catching up. Civitai had thousands of Flux.1 LoRAs within months. Flux.2 LoRAs exist but the long tail of style/character/concept training that makes SD ecosystems sticky is still building.

And the VRAM floor is brutal. If you're running on an RTX 3080 10GB or any 12GB card, Flux.2 forces you into Q4 territory where the quality lead over SD 3.5 Large evaporates. In that lane, Flux.1 Dev on the same hardware is honestly the better stopgap until you upgrade — it runs cleaner at 12GB than Flux.2 at Q4 does.

Common mistakes

Buying a 12GB card hoping to run Flux.2 well. It runs. It doesn't run well. Q4 quality at 50+ second generations is not the experience anyone wants. 16GB is the real Flux.2 floor.
Assuming SD 3.5 = SDXL hardware. It doesn't. SD 3.5 Large is meaningfully heavier than SDXL — closer to 14GB FP16 vs SDXL's ~10GB. If you sized a build for SDXL, check before you upgrade the model.
Ignoring FP8 because "it's lower precision." On Blackwell and Ada Lovelace, FP8 quality loss for both these models is below visual detection threshold in blind A/B tests. The VRAM and speed wins are free.
Buying a 5090 for SD 3.5 Medium. You will not see the GPU sweat. Generation will be limited by CPU/IO. Buy a 4070 Ti Super and pocket the difference.

For a broader view across all SD-family models on consumer GPUs, the best GPU for Stable Diffusion overview covers the older SDXL / SD 1.5 considerations that still matter for the LoRA back-catalog. And if Flux.2 specifically is your target, the best GPU for Flux.2 buyer's guide goes deeper on memory bandwidth and the May 2026 FP8 optimization details.

Final verdict

Use case	Pick	Real budget
Flux.2 FP16 (no compromise)	RTX 5090	~$2,000+
Flux.2 FP8 daily driver	RTX 5080	~$1,000
SD 3.5 Large primary	RTX 5070 Ti	~$750
Mixed Flux.2 / SD 3.5 budget	RTX 4070 Ti Super	~$800
SD 3.5 Medium + occasional Large	RTX 4060 Ti 16GB	~$450
ML enthusiast on used market	RTX 3090	~$700 used

The one-sentence verdict: if you can only buy one card for both models in 2026, it's the RTX 5080 — and if you can't, build around whichever model you actually use 80% of the time, not the one that benchmarks prettier.

Related guides on Best GPU for AI

Read the full guide on Best GPU for AI — includes our VRAM calculator, GPU comparison table, and live pricing.

How Much VRAM for Gemma 4? Every Variant Explained

Thurmon Demich — Sun, 21 Jun 2026 01:14:21 +0000

Cross-posted from Best GPU for LLM — visit the original for our VRAM calculator, GPU comparison table, and current Amazon pricing.

Google released Gemma 4 with four variants spanning from pocket-sized to flagship. The VRAM spread is massive — the smallest model fits on a phone, while the largest demands a high-end desktop GPU. This guide breaks down exactly how much VRAM each variant needs at every common quantization level, so you can match the right model to your hardware.

The short version

Variant	Q4_K_M VRAM	GPU you need
E2B (~2B)	~1.5GB	Any 4GB+ GPU
E4B (~4B)	~2.5GB	Any 6GB+ GPU
26B-A4B (MoE)	~14GB	16GB GPU (RTX 4060 Ti 16GB, RTX 5070 Ti)
31B Dense	~20GB	24GB+ GPU (RTX 4090, RTX 5090)

If you are deciding which variant to run, the 26B-A4B MoE is the sweet spot — 30B-class quality that fits on a 16GB card. Full hardware recommendations are in our best GPU for Gemma 4 buyer's guide.

Detailed VRAM by quantization

Gemma 4 E2B (~2B parameters)

Quantization	Model Size	Total VRAM (with KV cache)
Q3_K_M	~0.9GB	~1.5GB
Q4_K_M	~1.2GB	~1.5GB
Q5_K_M	~1.5GB	~2GB
Q6_K	~1.7GB	~2.5GB
Q8_0	~2.2GB	~3GB
FP16	~4GB	~5GB

The E2B fits everywhere. Integrated GPUs with 4GB shared memory, older GTX cards, even the Intel Arc B580 — all handle it without issue. VRAM is a non-concern here.

Gemma 4 E4B (~4B parameters)

Quantization	Model Size	Total VRAM (with KV cache)
Q3_K_M	~1.8GB	~2.5GB
Q4_K_M	~2.5GB	~3.5GB
Q5_K_M	~3GB	~4GB
Q6_K	~3.5GB	~5GB
Q8_0	~4.5GB	~6GB
FP16	~8GB	~10GB

Any GPU with 6GB+ VRAM handles Q4_K_M comfortably. For FP16 (useful for testing or fine-tuning), you need 10GB or more.

Gemma 4 26B-A4B (MoE — the important one)

Quantization	Model Size	Total VRAM (4K ctx)	Total VRAM (8K ctx)
Q3_K_M	~11GB	~13GB	~14.5GB
Q4_K_M	~14GB	~16GB	~18GB
Q5_K_M	~17GB	~19GB	~21GB
Q6_K	~20GB	~22GB	~24GB
Q8_0	~26GB	~28GB	~30GB

This is where VRAM planning matters. The 26B MoE has 26 billion total parameters that all live in VRAM, even though only ~4B activate per token. At Q4_K_M, the model weights alone are ~14GB. Add KV cache for a typical conversation and you are at 16-18GB.

On a 16GB card (RTX 4060 Ti 16GB, RTX 5070 Ti, RTX 5080): Q4_K_M fits, but keep context under 4K tokens for stability. Q3_K_M gives more breathing room at a small quality cost.

On a 24GB card (RTX 4090, RTX 3090): Q4_K_M runs with plenty of headroom. You can push to Q5_K_M and maintain 8K+ context comfortably.

Gemma 4 31B Dense

Quantization	Model Size	Total VRAM (4K ctx)	Total VRAM (8K ctx)
Q3_K_M	~16GB	~18.5GB	~20GB
Q4_K_M	~20GB	~22GB	~24GB
Q5_K_M	~24GB	~26GB	~28GB
Q6_K	~28GB	~30GB	~32GB
Q8_0	~35GB	~37GB	~39GB

The 31B Dense is straightforward but demanding. At Q4_K_M, you need at least 22GB with any meaningful context. The RTX 4090 (24GB) barely fits it — long conversations or large context windows may cause out-of-memory errors. The RTX 5090 (32GB) is the comfortable choice, fitting Q4 and even Q5 with room to spare.

VRAM chart available at the original article

KV cache: the hidden VRAM eater

Every table above includes estimated KV cache overhead, but actual usage depends on your conversation length. A rough guide:

2K context: +1-2GB over model weights
4K context: +2-3GB
8K context: +3-5GB
16K context: +5-8GB

For the 26B MoE on a 16GB card, this is the critical constraint. The model weights fit at Q4, but a long back-and-forth conversation can push past 16GB. Our recommendation: use a tool like nvtop or nvidia-smi to monitor VRAM during inference, and reduce context length if you see usage hitting 95%+.

Which quantization should you use?

For Gemma 4 specifically:

Q4_K_M is the standard recommendation. Minimal quality loss, good VRAM efficiency.
Q5_K_M is worth it on the 26B MoE if you have a 24GB card — the quality bump is noticeable on reasoning tasks.
Q3_K_M is acceptable on the 26B MoE for 16GB cards that need context headroom. Quality loss is small.
Q8 and above are only practical on the E2B and E4B variants unless you have multi-GPU setups.

For a broader guide to quantization trade-offs across all models, see best quantization for local LLM.

GPU recommendations by variant

Variant	Budget pick	Best pick
E2B / E4B	Any GPU you already own	N/A — any modern GPU works
26B-A4B MoE	RTX 4060 Ti 16GB (~$400)	RTX 5070 Ti (~$750)
31B Dense	RTX 3090 used (~$600)	RTX 4090 (~$1,600)

For full GPU benchmarks and buying recommendations, head to our best GPU for Gemma 4 guide, or our broader best GPU for Gemma overview spanning the 2B/7B/27B classics. Need general VRAM guidance across all model families? See how much VRAM for local LLM. And if you are running models through Ollama, our best GPU for Ollama article covers setup specifics. Budget shoppers should check best budget GPU for local LLM for affordable options.

Related guides on Best GPU for LLM

Continue on Best GPU for LLM for the complete guide with interactive calculators and current GPU prices.

Best GPU for SD 3.5 in 2026: 5 Cards (Large + Medium)

Thurmon Demich — Fri, 19 Jun 2026 16:29:57 +0000

This article was originally published on Best GPU for AI. The full version with interactive tools, FAQ, and live pricing is on the original site.

I have been running Stable Diffusion 3.5 locally since the Large checkpoint stabilised, and the buying advice from the SDXL era simply does not transfer. SD 3.5 Large is an 8B-parameter MMDiT model, Medium is a 2.6B sibling, and the May 2026 ControlNet release for Large finally made it usable for production work. None of that fits cleanly on the old "12GB is enough" mental model.

So here is how I would spend my own money in 2026, ranked by which SD 3.5 variant you actually run.

Quick answer

If you only run SD 3.5 Large, buy the RTX 4070 Ti Super 16GB. It clears FP16 with headroom for a ControlNet pass and lands around 10-12 seconds per 1024x1024 image. If you split your time between Large and Medium and want FP16 everywhere without thinking, get the RTX 5080 16GB. Anything below 16GB and you are quantising Large to FP8 — which works, but it is a compromise.

Who this is for

You are picking a GPU specifically for SD 3.5 — not a do-everything LLM box, not a Flux.2 rig. If you are torn between SD 3.5 and Flux.2 first, read my Flux.2 vs SD 3.5 hardware breakdown before this guide. And if you want the broader image-gen picture covering SD 1.5, SDXL and SD 3.5 together, the best GPU for Stable Diffusion round-up is the better starting point.

This piece assumes you have already decided: SD 3.5 Large or Medium, locally, in 2026.

VRAM tiers by variant and precision

The single most useful table in this whole article. SD 3.5 Large is roughly 2.5x SDXL's footprint at FP16, and SD 3.5 Medium is genuinely lightweight.

Variant	Precision	Model weights	Inference VRAM (1024x1024, batch 1)
SD 3.5 Large (8B)	FP16	~14 GB	14-16 GB
SD 3.5 Large (8B)	FP8	~7-8 GB	9-11 GB
SD 3.5 Medium (2.6B)	FP16	~6 GB	7-8 GB
SD 3.5 Medium (2.6B)	FP8	~3-4 GB	5-6 GB

Add 2-3 GB on top of every row if you stack the new ControlNet — the May 2026 SD 3.5 Large ControlNet (Canny, Depth, Blur) is excellent but it is not free. Full numbers in my how much VRAM for Stable Diffusion deep-dive.

VRAM chart available at the original article

GPU generation-time ranking

Numbers below are my own observed times for a 1024x1024, 28-step Euler generation in ComfyUI, batch 1, no ControlNet. Your mileage will swing 10-15% with samplers and scheduler.

GPU	VRAM	SD 3.5 Large FP16	SD 3.5 Large FP8	SD 3.5 Medium FP16	Street price
RTX 5090	32 GB	~5 s	~4 s	~2 s	~$2,000
RTX 4090	24 GB	~8 s	~6 s	~3 s	~$1,600
RTX 5080	16 GB	~9 s	~7 s	~3 s	~$1,000
RTX 5070 Ti	16 GB	~11 s	~8 s	~4 s	~$750
RTX 4070 Ti Super	16 GB	~12 s	~10 s	~4 s	~$700
RTX 3090 (used)	24 GB	~14 s	~11 s	~5 s	~$700
RTX 4060 Ti 16GB	16 GB	OOM-prone	~18 s	~7 s	~$400
RTX 3060 12GB	12 GB	OOM	~24 s	~9 s	~$200

The 4060 Ti 16GB technically loads SD 3.5 Large FP16 but bandwidth-starvation makes it painful — closer to 25s per image and the moment you add ControlNet you OOM. Treat it as an FP8-only card for Large.

Which GPU should YOU buy?

I keep getting variations of the same four scenarios. Here is the conditional logic.

You run SD 3.5 Large daily, you stack ControlNet, you bill clients. Buy the RTX 5090. The 32GB lets you batch 2-4 images at FP16 with ControlNet attached, which is where the real productivity gain lives. Anything less and you are single-image-batching forever.
You run SD 3.5 Large for fun or freelance, want FP16, do not need batching. Buy the RTX 5080 16GB. It is the cheapest card that still feels like a 4090 for this exact workload. Blackwell FP8 acceleration also future-proofs you for whatever ships next.
You are budget-bound but want SD 3.5 Large at acceptable speed. Buy the RTX 4070 Ti Super 16GB new or RTX 3090 24GB used. The 4070 Ti Super is faster per generation; the 3090 gives you 24GB for batching at the cost of more power draw and less FP8 efficiency. I lean 4070 Ti Super for new buyers, 3090 only if you find one under $650.
You mostly run SD 3.5 Medium and only dabble in Large. Buy the RTX 4060 Ti 16GB. Medium FP16 cruises, Large FP8 is tolerable, and you save enough to upgrade in two years.

Pair whichever you pick with a workflow you actually like — my best GPU for ComfyUI notes explain why I think ComfyUI is the right SD 3.5 frontend, especially with the May ControlNet drop covered in best GPU for ControlNet.

A contrarian take: the RTX 3090 is overrated for SD 3.5

Everyone in the Reddit threads is still recommending used 3090s. I do not agree, not for SD 3.5 specifically. Here is why:

No FP8 acceleration. SD 3.5's FP8 quantisation is one of the best things about it. The 3090 runs FP8 via emulation, losing most of the speed-up. A 5070 Ti at FP8 is genuinely faster than a 3090 at FP8.
Power draw. 350W TDP versus ~285W for the 5070 Ti. Over a year of daily generation that is a real electricity bill difference.
No warranty. Most used 3090s are mining survivors. The thermal pads are cooked.

The 3090's only honest advantage for SD 3.5 is the 24GB for batching at FP16. If you do not batch, you are paying a power-and-risk premium for nothing.

Common SD 3.5 mistakes

Buying a 12GB card "because SDXL ran fine on 12GB" — SD 3.5 Large will not. You will spend your first weekend quantising to FP8 and wondering why outputs look slightly worse.
Skipping FP8 because "it loses quality" — at SD 3.5 Large's scale the FP8 quality loss is genuinely small and the speed-up is large. Test it before dismissing it.
Forgetting the new ControlNet adds VRAM — the May 2026 SD 3.5 Large ControlNet release stacks 2-3 GB on top of base inference. Plan VRAM headroom around ControlNet, not raw inference.
Treating SD 3.5 Medium as a downgrade — Medium is genuinely good for iteration, especially for LoRA training pipelines where you generate hundreds of test images. A 4060 Ti 16GB running Medium FP16 is faster end-to-end than a 4090 running Large FP16.

Final verdict

Tier	GPU	Why
Top pick	RTX 5090	Only card that batches SD 3.5 Large FP16 + ControlNet
Best value	RTX 4070 Ti Super 16GB	SD 3.5 Large FP16 cleared, around 12s per image, ~$700
All-rounder	RTX 5080 16GB	FP8 acceleration, future-proofed, fits both variants
Budget Medium	RTX 4060 Ti 16GB	Medium FP16 cruises, Large FP8 tolerable
Skip	RTX 3060 12GB	Large OOMs, Medium FP8 only — buy used 3090 instead

If you only remember one thing: buy 16GB minimum for SD 3.5 Large, and do not let anyone talk you into a used 3090 unless the price is genuinely under $650.

Related guides on Best GPU for AI

The full version lives on Best GPU for AI — VRAM calculator, GPU comparison table, and live Amazon pricing.

Best GPU for Code LLMs in 2026 (Qwen Coder, DeepSeek)

Thurmon Demich — Fri, 19 Jun 2026 01:14:21 +0000

This article was originally published on Best GPU for LLM. The full version with interactive tools, FAQ, and live pricing is on the original site.

Quick answer: For code completion and generation, an RTX 4060 Ti 16GB ($400) handles 7B code models well. For the best coding experience with 33-34B models, the RTX 4090 ($1,600) is the go-to pick.

Why code LLMs have different GPU needs

Code LLMs work differently from general chat models. Code completion demands low latency for inline suggestions, fill-in-the-middle tasks use bidirectional context, and code generation with long outputs benefits from sustained throughput. Speed matters more here because you are waiting for suggestions while you type.

Popular code LLMs and their VRAM requirements

Model	Parameters	Q4_K_M Size	Minimum VRAM	Strength
CodeLlama 7B	7B	~4.5GB	8GB	Fast completions
CodeLlama 13B	13B	~7.5GB	12GB	Better reasoning
CodeLlama 34B	34B	~20GB	24GB	Complex code generation
DeepSeek Coder V2 Lite (16B)	16B	~9.5GB	12GB	Strong multi-language
DeepSeek Coder V2 (236B MoE)	236B	~135GB	Multi-GPU	Near-GPT-4 coding
Qwen 2.5 Coder 7B	7B	~4.5GB	8GB	Excellent for its size
Qwen 2.5 Coder 14B	14B	~8.5GB	12GB	Great quality/size ratio
Qwen 2.5 Coder 32B	32B	~19GB	24GB	Best local code model

Qwen 2.5 Coder 32B and CodeLlama 34B are the standout models for serious local coding. Both need ~20GB at Q4_K_M, making the RTX 4090 the natural home.

GPU benchmarks for code LLMs

Speed benchmarks using Ollama with Q4_K_M quantization:

GPU	Qwen Coder 7B	CodeLlama 13B	Qwen Coder 32B	Price
RTX 5090	~95 tok/s	~55 tok/s	~28 tok/s	~$2,000
RTX 4090	~65 tok/s	~40 tok/s	~20 tok/s	~$1,600
RTX 5080	~55 tok/s	~32 tok/s	Needs offload	~$1,000
RTX 4070 Ti Super	~40 tok/s	~25 tok/s	Needs offload	~$700
RTX 4060 Ti 16GB	~28 tok/s	~18 tok/s	Needs offload	~$400
RTX 3060 12GB (used)	~18 tok/s	~12 tok/s	No	~$250

For inline code completion, you want at least 30 tok/s to feel responsive. For longer code generation, 15-20 tok/s is acceptable.

Matching GPU to your coding workflow

Inline completion (Copilot-style): Latency is king. You need the first token fast. A 7B model on a fast GPU beats a 34B model on a slow GPU for this use case. The RTX 4070 Ti Super running Qwen Coder 7B at ~40 tok/s gives a snappy experience.

Code generation and refactoring: Quality matters more here. Larger models produce better code with fewer errors. Qwen 2.5 Coder 32B on an RTX 4090 at ~20 tok/s gives you near-commercial quality at reasonable speed.

Code review and explanation: Context length matters because you need to fit large code blocks into the prompt. 16GB cards handle 7-14B models with 8K+ context. For 32K context with 14B+ models, get a 24GB card.

GPU tier list available at the original article

Which GPU should you buy?

If you mainly do inline code completion (Copilot-style autocomplete), get the RTX 4060 Ti 16GB — a 7B model at 28 tok/s is fast enough for real-time suggestions and costs only $400. If you do code generation and refactoring where output quality matters more than latency, jump to the RTX 4090 — it runs Qwen Coder 32B at 20 tok/s, which is the best local code model available. If budget is not a concern and you want the fastest possible coding experience, the RTX 5090 is the only card that runs 32B code models above 25 tok/s.

Common mistakes to avoid

Buying a 12GB card for code LLMs. Code models with long context windows (8K-16K tokens for full file context) eat more VRAM than chat models. 12GB gets tight fast — 16GB is the real minimum.
Choosing a bigger model over a faster GPU. For inline completion, a 7B model at 40 tok/s produces better workflow than a 34B model at 12 tok/s. Speed matters more than quality for autocomplete.
Ignoring context length requirements. Code tasks often need the full file (or multiple files) in context. A model that fits in VRAM but leaves no room for KV cache will truncate your code context and give worse suggestions.
Running FP16 when Q4_K_M is fine. For code completion, Q4_K_M quantization produces nearly identical suggestions to FP16. Save the VRAM for longer context instead.

Our recommendation

Workflow	Best Model	Best GPU	Price
Fast completions on a budget	Qwen Coder 7B	RTX 4060 Ti 16GB	~$400
Balanced coding assistant	Qwen Coder 14B	RTX 4070 Ti Super	~$700
Best local coding experience	Qwen Coder 32B	RTX 4090	~$1,600
Maximum quality	Qwen Coder 32B	RTX 5090	~$2,000

The RTX 4090 running Qwen 2.5 Coder 32B is the best local coding setup in 2026. It fits the model at Q4_K_M with room for long context windows and delivers usable generation speed. If you are on a budget, the RTX 4060 Ti 16GB with a 7B code model still beats cloud-dependent tools for privacy and latency.

For more on how much VRAM these models actually consume in practice, see our VRAM requirements guide. If you prefer running code models through Ollama, all these GPUs work great with it out of the box. Connecting those models to your editor? See our best GPU for Continue.dev guide for VS Code and JetBrains extension-specific advice — and for a workflow-level walkthrough of pairing a coding model to a developer setup, see our best GPU for a local coding LLM guide.

Related guides on Best GPU for LLM

The full version lives on Best GPU for LLM — VRAM calculator, GPU comparison table, and live Amazon pricing.

Best GPU for IP-Adapter in 2026: 5 Picks (16GB Sweet Spot)

Thurmon Demich — Thu, 18 Jun 2026 01:14:41 +0000

From the Best GPU for AI archive. The canonical version has interactive calculators, an up-to-date GPU comparison table, and live pricing.

Quick answer: The RTX 4070 Ti Super (16GB) is the best GPU for IP-Adapter in 2026. It holds SDXL plus IP-Adapter Plus plus two ControlNets resident in VRAM without spilling to system RAM — the realistic character-LoRA and style-transfer stack — at roughly half the price of an RTX 4090.

Who this is for

This guide is for anyone running IP-Adapter on top of Stable Diffusion locally. That covers three concrete workflows we see all the time: character-LoRA artists who use IP-Adapter FaceID to lock identity across hundreds of poses, product photographers who feed a reference shot into IP-Adapter Plus to relight it inside SDXL, and style-transfer power users who chain IP-Adapter with one or two ControlNets to pin composition while swapping aesthetics. If that's you, this is your shortlist.

How IP-Adapter VRAM actually stacks

IP-Adapter is rarely run alone. It almost always sits on top of SDXL (or Flux) and beside one or two ControlNets — pose, depth, or canny — because that's the whole point: reference image plus structural control. Each of those pieces is its own VRAM tax.

Here's the accounting we see in production ComfyUI workflows at 1024×1024:

Workload	Base SDXL	IP-Adapter	ControlNets	Total VRAM
SDXL + IP-Adapter (base)	~10GB	+2GB	—	~12GB
SDXL + IP-Adapter Plus	~10GB	+2.5–3GB	—	~12.5–13GB
SDXL + IP-Adapter FaceID + 1 ControlNet	~10GB	+2GB	+1.5GB	~13.5GB
SDXL + IP-Adapter Plus + 1 ControlNet	~10GB	+3GB	+1.5GB	~14.5GB
SDXL + IP-Adapter Plus + 2 ControlNets	~10GB	+3GB	+3GB	~16GB
Flux Dev + IP-Adapter + 1 ControlNet	~14GB	+2.5GB	+1.5GB	~18GB

Notice how fast you hit 16GB. The realistic character-creator workflow — SDXL + IP-Adapter Plus + a pose ControlNet + a depth ControlNet — lands right at the 16GB ceiling. Activations during the denoise add another ~1–2GB you can't budget for models. That's why 12GB cards spend the whole generation thrashing system RAM the second you turn on IP-Adapter Plus with anything stacked on top.

VRAM chart available at the original article

GPU ranking for IP-Adapter workloads

Generation times below are for 1024×1024 SDXL at 25 steps, measured in our test workflows. The "Plus + 2CN" column is the realistic character-LoRA / product-photo stack: IP-Adapter Plus with two simultaneous ControlNets (pose + depth). The "FaceID" column is the lighter character-identity case.

GPU	VRAM	SDXL + IP-Adapter	+ FaceID + 1CN	Plus + 2CN	Price
RTX 5090	32GB	~2.5s	~3s	~4s	~$2,000
RTX 4090	24GB	~3.5s	~4s	~5.5s	~$1,600
RTX 5080	16GB	~4s	~5s	~6s	~$1,000
RTX 5070 Ti	16GB	~5s	~6s	~7s	~$750
RTX 4070 Ti Super	16GB	~6s	~7s	~8.5s	~$700
RTX 3090 (used)	24GB	~7.5s	~9s	~11s	~$700
RTX 4060 Ti 16GB	16GB	~10s	~12.5s	~15s	~$400
RTX 3060 12GB	12GB	~16s	OOM / swap	OOM / swap	~$200

Two patterns matter here. First, the 3060 12GB technically runs IP-Adapter solo but completely falls over the moment you add ControlNet — it offloads to system RAM and per-image times balloon into the minutes. Second, the 4060 Ti 16GB clears the Plus + 2CN stack but at almost twice the wall-clock of the 4070 Ti Super, because its 288 GB/s memory bandwidth bottlenecks on the IP-Adapter cross-attention layers that run every denoising step.

The used RTX 3090 is the dark-horse pick. It's slower per image than the 5070 Ti, but 24GB means you can keep IP-Adapter Plus, FaceID, two ControlNets, and an upscaler hot in VRAM simultaneously — useful if you're batch-rendering a 50-pose character sheet.

Which GPU should YOU buy?

You're a character-LoRA artist using IP-Adapter FaceID + a single pose ControlNet: The RTX 4060 Ti 16GB at ~$400 is the cheapest survivable option. Slow, but it won't crash mid-batch.
You're doing product photography — SDXL + IP-Adapter Plus + 1–2 ControlNets for relighting and composition: The RTX 4070 Ti Super at ~$700 is our pick. This is the sweet spot of the entire 2026 lineup for reference-image workflows.
You batch style-transfer hundreds of images overnight with IP-Adapter chained to ControlNet: Go to 24GB. RTX 4090 if you want speed, RTX 3090 used if you want headroom on the cheap.
You're combining IP-Adapter with LoRA adapter training for character workflows: 24GB is the practical floor. Training and reference-conditioned inference both want VRAM, and 16GB starts thrashing the moment you load the optimizer state.
You're running IP-Adapter on top of Flux Dev instead of SDXL: Skip 16GB entirely. Flux + IP-Adapter + 1 ControlNet already pushes past 18GB. 24GB or 32GB only.

If you're new to the broader image-gen stack, our best GPU for Stable Diffusion covers the base-model VRAM picture before IP-Adapter enters the chat. And since IP-Adapter is almost always paired with structural conditioning, our best GPU for ControlNet guide is the sibling read — same 16GB sweet spot, slightly different stack math. Most serious IP-Adapter work in 2026 lives inside a node graph, so the best GPU for ComfyUI breakdown is the natural next step.

Contrarian take: we recommend against 8GB cards for IP-Adapter workflows

The internet still tells people you can "run IP-Adapter on 8GB" because the model file is small. Technically true, completely useless in practice. IP-Adapter only matters as part of a stack — reference image plus ControlNet plus SDXL — and the moment you turn on even one ControlNet alongside it, an 8GB card is offloading half the pipeline to CPU. We've watched users wait 90 seconds per image and convince themselves the workflow is "working." It's not. If your budget can't reach 16GB, save another month and skip 8GB entirely.

Common mistakes with IP-Adapter hardware

Running 12GB with IP-Adapter Plus + ControlNet. This is the single most common configuration we see crash. Plus weights are roughly 50% larger than base IP-Adapter, and the cross-attention layers eat activations during every denoising step. 12GB technically loads the models but spills to system RAM the moment denoise starts. Use base IP-Adapter on 12GB, never Plus.
Forgetting that FaceID also wants a CLIP image encoder. IP-Adapter FaceID needs InsightFace plus a CLIP vision encoder loaded alongside the adapter. That's another ~1.5GB people forget to budget. In our experience, this is why users on 12GB report FaceID "randomly" OOMing — the encoder isn't visible in the workflow graph but it's resident in VRAM.
Stacking IP-Adapter Plus on top of Flux without a 24GB card. Flux Dev is already a 14GB-tier base. Add IP-Adapter Plus and any ControlNet and you're past 18GB before activations. The 16GB sweet spot we recommend for SDXL workflows does not apply to Flux — Flux + IP-Adapter is a 24GB-floor conversation.
Assuming bandwidth doesn't matter because IP-Adapter is "small." IP-Adapter weights are small but its cross-attention layers run every denoising step against the SDXL UNet's image tokens. That's bandwidth-bound work. The 4060 Ti 16GB and 4070 Ti Super both have 16GB, but the 4070 Ti Super is roughly 1.7× faster on the same IP-Adapter + ControlNet stack because of memory bandwidth — not VRAM capacity.

Final verdict

Budget	GPU	Best for in IP-Adapter
~$200 used	RTX 3060 12GB	Base IP-Adapter only, no ControlNet
~$400	RTX 4060 Ti 16GB	FaceID + 1 ControlNet, slowly
~$700	RTX 4070 Ti Super	Plus + 2 ControlNets, sweet spot
~$700 used	RTX 3090 24GB	Batch character sheets, VRAM headroom
~$1,600	RTX 4090	Flux + IP-Adapter, no compromises
~$2,000	RTX 5090	32GB, training + inference on one card

The best GPU for IP-Adapter is the one that keeps the reference encoder, the adapter weights, and every ControlNet resident in VRAM at the same time — the moment you spill, your iteration loop is dead.

Related guides on Best GPU for AI

The full version lives on Best GPU for AI — VRAM calculator, GPU comparison table, and live Amazon pricing.

Best GPU for Kimi K2 in 2026 (Agentic Local LLM Guide)

Thurmon Demich — Wed, 17 Jun 2026 01:14:45 +0000

Cross-posted from Best GPU for LLM — visit the original for our VRAM calculator, GPU comparison table, and current Amazon pricing.

If you're wiring up Kimi K2 for a coding agent or a long-running autonomous tool loop, the GPU question is not "what runs the model" — it's "what survives ten thousand tool calls a day without melting your wallet." I've been running Moonshot's K2 line locally since the original 1T MoE drop, and the Q4 quants behave very differently from what the headline parameter count suggests.

Quick answer: The RTX 4090 (24GB, ~$1,600) is the consumer sweet spot for local Kimi K2 inference. It holds a Q4 K2 active expert plus a workable KV cache, runs at roughly 25-35 tok/s, and keeps agent loops responsive without spilling into multi-GPU territory.

Who this is for

You're building with agents — coding copilots, browser agents, autonomous research bots, or self-prompting tool chains — and you've already settled on Kimi K2 because of its strong agentic benchmark scores and permissive license. You want to run it locally for latency, privacy, or the simple sanity of not paying per million tokens when your agent loops a hundred times per task. If that's not you, look at our broader AI agents GPU guide for non-Moonshot picks.

What makes Kimi K2 different

Kimi K2 is a 1T+ Mixture-of-Experts model with roughly 32B active parameters in the original release and around 50B in K2.6 (the June 2026 refresh). That MoE structure is the entire reason it can fit on a single consumer GPU at all — you never load the full 1T weights into VRAM at once, only the routed experts for the current token. In practice, that means a Q4 quant lands in the 24-32GB range for active inference, similar territory to Llama 4 Scout. The architectural parallels with Llama 4 are real, and the GPU calculus is nearly identical.

The catch: KV cache for long agent contexts is not MoE-sparse. A 128K-context K2 session can chew through 8-16GB of cache on top of weights. That's where most agent builders get burned.

Kimi K2 VRAM requirements

VRAM chart available at the original article

Quant	Weights (active)	KV @ 8K	KV @ 32K	KV @ 128K	Total @ 32K
Q2	~16GB	~1GB	~4GB	~16GB	~20GB
Q4	~24GB	~1.5GB	~6GB	~22GB	~30GB
Q8	~40GB	~2GB	~8GB	~28GB	~48GB
FP16	~64-100GB	~3GB	~12GB	~40GB	~76-112GB

Q4 is the practical floor. Q2 technically runs but agent reliability collapses — tool-call JSON breaks, function names hallucinate, and your loop wedges. Q8 is genuinely better but requires the RTX 5090 or dual-GPU setups. For the math behind these numbers, see our VRAM sizing guide.

Best GPUs for Kimi K2 ranked

GPU	VRAM	K2 Q4 tok/s	K2.6 Q4 tok/s	Max context	Price
RTX 5090	32GB	~40-50	~28-35	128K	~$2,000
RTX 4090	24GB	~25-35	~18-22	32K	~$1,600
RTX 3090 (used)	24GB	~20-28	~14-18	32K	~$700
RTX 5080	16GB	Q2 only	Q2 only	8K	~$1,000
RTX 5070 Ti	16GB	Q2 only	Q2 only	8K	~$750
RTX 4070 Ti Super	16GB	Q2 only	Q2 only	8K	~$700
RTX 4060 Ti 16GB	16GB	Q2 only	Q2 only	4K	~$400

The honest pattern: there are two tiers. The 24GB+ club runs K2 properly. The 16GB club runs Q2 quants that I would not deploy into a production agent loop. The RTX 3090 used market remains the best value-per-VRAM in the entire stack — if you can verify a clean card, $700 for 24GB is hard to beat for a dedicated agent box.

The contrarian take: don't run K2 locally for single-shot work

Here's the thing nobody selling you a GPU will say: if your agent only fires one or two K2 calls per task, local inference is the wrong choice. Kimi's hosted API is cheap, fast, and doesn't require you to buy and power a $1,600 card. Local Kimi K2 makes sense when one of three things is true:

You're running hundreds to thousands of agent calls per day (coding copilots, autonomous research bots, batch agentic workflows).
You have a hard privacy requirement — code that can't leave your network, regulated data, internal tools.
You're iterating on prompts and tools constantly and want zero-cost experimentation.

If none of those apply, run K2 via API and spend the $1,600 on something that compounds.

Which GPU should YOU buy?

Single-agent coding copilot (5-20 calls/task): RTX 4090 24GB at $1,600. Q4 K2.6 at 32K context, ~20 tok/s, no surprises. Pair it with Ollama for the cleanest local serving stack.
Multi-agent orchestration (CrewAI, AutoGen, LangGraph swarms): RTX 5090 32GB at $2,000. You need the headroom because parallel agents share KV cache budget, and K2.6's longer reasoning chains stress context harder than K2 did.
Batch agentic workflows (overnight runs, evaluator loops, dataset generation): Used RTX 3090 24GB at $700, or skip local entirely and use cloud burst. RunPod's H100 spot pricing makes more sense than buying a 5090 for jobs that run 4 hours and then idle.

For overflow workloads — fine-tuning runs, evaluator sweeps, or any time you need to run K2 at Q8 — cloud H100 instances are economically saner than upgrading to a multi-GPU local rig.

Common Kimi K2 mistakes I see constantly

Treating K2 like a dense model when sizing VRAM. People see "1T parameters" and assume they need 8x H100s. MoE routing means only the active experts hit VRAM per token. Q4 fits on 24GB.
Forgetting KV cache for long agent contexts. A 32B-active model with 128K context can use more VRAM for cache than weights. Budget 6-22GB on top of model weights depending on your context window.
Running K2 Q2 in production agents. It feels like it works in testing, then tool-call JSON breaks at 3am during an unattended batch run. Q4 minimum for any agent that calls real tools. This is the same trap people fall into with 70B models on undersized hardware.
Not pinning the K2 vs K2.6 version. K2.6 has more active params and runs ~30% slower at the same quant. If your agent timing budget was tuned on K2, expect surprises after upgrading.

Final verdict

Need	Best pick	Price
Best overall agentic	RTX 4090 24GB	~$1,600
Multi-agent + K2.6 128K	RTX 5090 32GB	~$2,000
Best value (used)	RTX 3090 24GB	~$700
Burst / batch workloads	RunPod H100	hourly

If you're running Kimi K2 to drive real agents, buy the 24GB card — anything less turns your tool loop into a coin flip.

Related guides on Best GPU for LLM

The full version lives on Best GPU for LLM — VRAM calculator, GPU comparison table, and live Amazon pricing.