We’ve all been there. You see a trending new model on Hugging Face, you git clone the repo, wait 20 minutes for the weights to download, run the inference script, and then...
torch.cuda.OutOfMemoryError: CUDA out of memory. 😭
Calculating whether a model will fit on your GPU isn't as simple as looking at the file size. You have to factor in quantization, context window overhead, and system headroom.
To make life easier for myself and other devs, I built a free utility to do the math for you.
🛠️ The Tool: LLM Hardware Compatibility Checker
I wanted something lightweight and fast. No sign-ups, no "enter your email to see results"—just a straightforward calculator to see if your rig can handle a specific model.
Why use this?
When you’re running models locally (using Ollama, LM Studio, or vLLM), VRAM is your most precious resource. This tool helps you figure out:
Quantization Strategy: Can you run the full FP16 model, or do you need to drop to 4-bit (GGUF/EXL2) to make it fit?
_
_Hardware Planning: If you're looking to upgrade your GPU, you can simulate different VRAM capacities (12GB vs 16GB vs 24GB) to see what models they unlock.
_
_Avoid the OOM: Save time by knowing it won't work before you start the download.
How it works
The calculator looks at the parameter count and the bits-per-weight to estimate the base memory footprint, then adds a buffer for the KV cache. It’s a great "sanity check" before you commit to a new local setup.
**💬 I need your feedback!
**This is a work in progress. I’m planning to add more specific model presets and perhaps a "recommended GPU" feature based on the model you want to run.
Check it out here: https://id8.co.in/tools/can-i-run-llm
What features should I add next? Better support for MoE (Mixture of Experts) models? Multi-GPU spanning calculations? Let me know in the comments!
Top comments (0)