Everyone asks what hardware a local LLM needs. Almost nobody asks what it costs to leave one running. I did the math for a Raspberry Pi 5 serving a small model around the clock.
What the board actually draws
A Pi 5 idles around 2.7-3.5 W with a minimal OS. Sustained LLM inference on all four cores pushes it to roughly 7-9 W, more with an NVMe hat or USB SSD. Add an active cooler (0.5-1 W) and any accelerator HAT, and budget headroom on top.
The official 27 W (5V/5A) USB-C PSU exists for a reason: undervoltage during a long generation run shows up as silent errors and throttled clocks before it ever shows up as a reboot.
The monthly cost reality
At a sustained 8 W average, 24/7 operation is about 5.8 kWh per month. At typical residential rates that's roughly $1-2/month - cheap, but not zero, and it triples with a second board or an accelerator.
The number that actually matters: cost per query
Take monthly power + hardware amortization (a complete Pi 5 build over 3 years is ~$4-5/month) and divide by your monthly queries.
Below a few thousand queries a month, a cloud API at fractions of a cent per call usually wins. Above it, local wins and keeps winning forever. Your crossover point depends on your rates and query sizes - which is why this needs a calculator, not a blog table.
Do your own math
I packaged both sides of this: a Power Budget Planner that specs PSU, cooling draw and runtime budgets in one sheet, and a Total Cost Calculator that runs hardware + power + time against cloud API prices and gives you cost per month and per query.
More guides: Local AI on Raspberry Pi - hub
What's your setup costing you? Curious whether people actually meter their boards.
Top comments (0)