When someone asks which model your team used, what data it touched, and why the compute bill jumped, "we don't track it that closely" stops being an acceptable answer.
That's usually when self-hosting gets a real look. Not because it's always cheaper, but because it gives you control over the questions that matter: where requests went, which model served them, what changed, and what drove the bill.
TL;DR: There are three practical ways to run an LLM: subscribe to an API, buy and operate GPUs, or rent dedicated infrastructure and run the model yourself. The third option gets skipped most often. It's worth a look when usage is sustained, data handling matters, or you need control over the model itself.
Three ways to run it
Subscribe. Pay per token. No procurement, no serving stack, no capacity planning beyond budget and rate limits. The right call for low-volume or highly variable demand. Trade-off: the provider controls pricing, rate limits, and when the model version changes under you.
Buy. Purchase and rack the GPUs, own the stack end to end. No per-token bill, but you inherit procurement, cooling, power, hardware failures, and on-call. Rational for stable, long-term demand with the ops capacity to match it. For a small team, it can mean running a miniature data center alongside the actual product.
Rent and build. Rent dedicated GPU infrastructure and run the model yourself. You control the model, the serving runtime, the request path, and the logging design. The provider owns the physical hardware lifecycle. This is the option that gets skipped, and it sits in the middle of the other two.
The build, compressed
- VRAM is the ceiling. Check available VRAM, GPU count and interconnect, context length, concurrency, and latency target before picking a model.
- Check the license, not just the weights. "Open weight" doesn't mean unrestricted commercial use, and some repos gate access behind a request. Hugging Face's gated models docs cover this.
- Precision is a lever, not a default. FP16/BF16 is the quality baseline. 8-bit cuts memory. 4-bit cuts it further. Test against your actual task before trusting a generic benchmark.
- The KV cache is the part people forget. It grows with context length and concurrent users, and it's usually why a config that worked in testing falls over in production.
- Use vLLM for anything with concurrent traffic. Ollama and llama.cpp are fine for a local test. For production, and for models that don't fit on one GPU:
vllm serve <model-id> \
--tensor-parallel-size 4
That one flag shards the model across GPUs. vLLM's distributed inference docs cover the full pattern for tensor and pipeline parallelism.
- Don't expose the raw endpoint. Auth, rate limits, TLS, network segmentation, logging with a real retention policy. Treat every user message and retrieved document as untrusted input. OWASP's prompt injection cheat sheet is worth reading before you launch.
- Monitor hardware and serving separately. GPU utilization alone won't tell you if someone's request is stuck in a queue. ## Where it doesn't fit
Occasional or spiky traffic, a team that can't own security and serving ops, or a model that only exists under a license you can't use: an API is still the better call. A model that fits on one GPU isn't a frontier model either, but that's a limit of that configuration, not of self-hosting itself. Multi-GPU servers scale further with the same tensor-parallelism pattern above.
The real decision
Buying and subscribing are both defensible. The option most teams miss is the middle one: dedicated hardware you rent, with a stack you control. It's a different boundary of responsibility. You own the model-serving decisions, and the provider owns the physical hardware lifecycle.
That's the option worth naming when the question shifts from "can we add AI" to: which model touched this data, where did the request go, and what drove this month's bill.
We wrote up the full version, license nuances, VRAM math with real budget thresholds, and the complete security checklist, on the Leaseweb blog: Self-Hosting an LLM: How to Choose the Right Approach for Your Team
Disclosure: I work in infrastructure marketing at Leaseweb. This is general engineering guidance, not a universal recommendation. The right approach depends on your traffic shape, security requirements, model license, and team capability.
Top comments (0)