What Does a Local LLM Actually Cost per Month? I Read the Meters.
The Local LLM Lab — Part 5
One controlled experiment. One number. One verdict.
The question nobody answers in the local-LLM hype is the boring one: what does the electricity bill say?
Not "how many tokens per second." Not "how many GB of VRAM." The bill. The one that arrives on the first of the month and is the only number that actually matters for the person paying for the machine.
I've been running a local AI stack on an RTX 3090 for about three months now — Whisper transcription as a permanent service, an embedding model for a RAG pipeline, and a 27B chat model on a second machine. I instrumented the GPU with a power meter, set up a dual-rate tariff (day 0.30 BGN/kWh, night 0.18 BGN/kWh), and let the meter run.
This is what the last 30 days actually cost.
The setup
- GPU: NVIDIA RTX 3090, 24 GB, power limit 260 W, running WhisperX (a Whisper ASR webservice) as a permanent resident
- Tariff: dual-rate — 0.30 BGN/kWh day, 0.18 BGN/kWh night (22:00–06:00)
-
What's on the GPU 24/7:
- Whisper ASR webservice — avg 7.9 GB VRAM, peak 10.3 GB, resident
- nomic-embed-text (Ollama) — 308 MB VRAM, resident
- immich ML — negligible
- What's on the second machine (Vader): Qwen3.8-27B via vLLM, two RTX 3090s, on demand
The Whisper service is the interesting one. It's not a chatbot. It's a transcription endpoint that my own tools call whenever I record a voice memo, a meeting, or a podcast clip. It sits there all day, at rest, drawing power, waiting for audio.
That's the honest shape of a "local AI stack" — not a GPU that's 100% utilised 24/7, but a GPU that's 0% utilised 99% of the time and 100% for a few seconds when it's actually doing work.
The number
The last 30 days cost €2.00 in electricity.
That's the entire GPU. Not the whole machine. Just the GPU, measured at the card, over 30 days, with the Whisper service resident 24/7 and the embedding model resident 24/7.
Breakdown by service:
| Service | Avg W | Energy (kWh) | Cost (€) |
|---|---|---|---|
| Whisper ASR (3090) | 22 W | 13.24 | 1.76 |
| nomic-embed-text (Ollama) | 3 W | 1.75 | 0.23 |
| immich ML | ~0 W | 0.01 | 0.00 |
| Total | ~25 W | 15.03 | 2.00 |
The Whisper service is the big one, but even it is only 22 W on average. That's not a GPU under load. That's a GPU that's mostly idle with a model resident in VRAM, spiking to ~120 W for a few seconds when it transcribes something, and sitting at ~35 W the rest of the time.
The embedding model costs 23 cents a month. That's less than a coffee.
Why the number is so small (and why that's the point)
People who argue "local LLMs are expensive" usually mean one of two things:
- The hardware cost amortised over time. A 3090 costs €800–1,200 used. Amortised over 3 years, that's ~€25/month. That's real, and it's the dominant cost, not the electricity.
- A GPU that's pegged at 100% utilisation 24/7. That's a training rig, not an inference stack. My GPU is at 0% utilisation on average. The 30-day average power draw is ~25 W. A 3090 under sustained LLM inference at 260 W would burn ~187 kWh/month and cost ~€23/month. That's a different machine doing a different job.
The honest framing for a home inference stack is: the GPU is a mostly-idle appliance that costs a few euros a month to keep warm, and spikes when you actually use it.
The €2.00 number is the "keep it warm" cost. The spikes are the "actually use it" cost, and they're short enough that they barely move the monthly total.
What the power trace actually looks like
I pulled the 30-day power history at 2-hour resolution. The shape is:
- Baseline: ~35 W, 24/7 (Whisper resident, idle)
- Spikes: ~120 W, for a few seconds to a few minutes, when audio comes in
- Occasional higher spikes: ~50–54 W sustained for longer stretches when there's a queue of transcriptions
- One outlier: a single 2-hour bucket at 121 W (a batch of long audio files)
The 2-hour buckets that show 35 W are the honest baseline. The 2-hour buckets that show 121 W are the work. The average of all of them is 22 W for the Whisper service.
That's the shape of a local inference stack: a low baseline with short, sharp spikes. Not a sustained load.
The comparison that matters
Here's the comparison that actually answers "is local worth it":
| Scenario | Monthly electricity cost | Notes |
|---|---|---|
| This stack (Whisper + embed, 24/7 resident) | €2.00 | Measured, 30 days |
| 3090 at 100% util, 24/7 (training rig) | ~€23 | Hypothetical, sustained load |
| API transcription (Whisper API, ~10 hrs audio/month) | ~€14–29 | OpenAI Whisper API pricing |
| API LLM (GPT-4o, ~1M tokens/month) | ~€14 | List price |
The local stack costs less than a tenth of the equivalent API spend, even before you factor in that the API spend is per-use and the local spend is a flat "keep it warm" cost that doesn't scale with usage.
The electricity is not the cost. The hardware is the cost. But the electricity is small enough that it stops being the argument against.
What this means for the "is local worth it" question
The electricity bill is not the reason to choose cloud over local. It's not even close. A 3090 running a resident inference stack costs ~€2/month in electricity. The hardware amortisation is ~€25/month. The cloud API equivalent is ~€14–29/month for comparable usage, and it scales with usage while the local cost doesn't.
The real cost of local is the upfront hardware and the time you spend keeping it running. The electricity is a rounding error.
If you're already paying for a GPU for other reasons (gaming, rendering, a second machine), the marginal cost of adding a local inference stack is essentially the electricity: a few euros a month.
Verdict
A local LLM inference stack on a 3090 costs ~€2/month in electricity. The hardware is the real cost, not the power. The electricity bill is small enough that it should stop being an argument in the cloud-vs-local debate.
If your objection to local is "the electricity," the meter says otherwise. The meter says: a few euros a month, mostly idle, spikes when you use it.
That's not a cost problem. That's a hardware-cost problem, and it's a one-time problem, not a recurring one.
How I measured this
Every number in this article came out of my homelab monitor, not an estimate: a small self-hosted dashboard that reads the GPU's power draw at the card, tracks per-service VRAM, and prices the energy against my tariff. It's open source, and it exposes the same data through a read-only MCP server, so an AI agent can pull the numbers for you instead of you SSH-ing in to run nvidia-smi by hand.
If you want to run your own "what does my stack cost" experiment, that's the tooling I'd point you at.
The Local LLM Lab is a series of measured experiments on a home GPU stack. Every number in this article was pulled from a power meter on the card, not estimated. The next piece prices the same stack by € per 1,000 correct answers — the number that actually matters when the model is doing your work.
Top comments (0)