DEV Community

Cover image for Why I Run AI at Home, and LM Studio Tips for Beginners
GarySto
GarySto

Posted on Originally published at garystow.co.uk

Why I Run AI at Home, and LM Studio Tips for Beginners

Originally published on garystow.co.uk.

I grew up poor, homeless poor, and that still shapes how I think about money. What might be £20 to one person can be a week's worth of food for another, so when I started building with AI, I wanted to prove it could be done without a monthly bill that climbs every time you use it. Running AI on my own PC, with no token cost, is a big part of that.

Control, and finding out if I could

Cost was only part of it, because I also wanted control. I wanted to build my own things and understand what it actually takes to run a good AI system, rather than just renting one.

If I'm honest, I also wanted to know whether I could do it at all. I often feel like the idiot in the room when people start using big words, and I prefer the direct, simple approach. Building my own local AI, which I called Wade, was a way of finding out whether I was smart enough for this kind of thing.

What I'm running it on

My PC has a 16GB graphics card and 32GB of memory, which is enough to run mid-sized models comfortably. I'd love to run bigger ones, but my hardware sets the limit, and that's worth knowing before you start.

These are the models I have in LM Studio right now:

  • GPT-OSS 20B from OpenAI, the biggest one I run, about 11GB, usually loaded with a 32,768-token context window
  • Qwen2.5 Coder 7B Instruct, a smaller coding model at about 4.4GB
  • Qwen2.5 Coder 14B, a bigger coding model at about 8.4GB
  • Qwen2.5 14B Instruct 1M, a general model built for very long inputs
  • Gemma 4 E4B from Google, a small, quick model at about 5.9GB
  • Nomic Embed Text v1.5, a tiny model used for searching and matching text rather than chatting

LM Studio runs as a local server on my PC, so my scripts talk to it the same way they'd talk to a paid AI service, except nothing leaves the machine and nothing gets billed. My website's automation team uses GPT-OSS 20B for its writing and checking jobs.

What Wade does every day

Wade helps manage my automated trading project and my website. Day to day, it keeps me on track with my personal project tasks, things like submitting pages to Google for indexing, writing articles and staying on top of where the website is against its goals. I wrote up exactly how I built it in how to actually build your own local AI assistant.

My first mistake with LM Studio

I came into LM Studio feeling confident, because I'd already built agents with other AI tools. That confidence lasted about a minute, because I didn't have a clue where to click or what I was looking at. I loaded models that weren't right for my PC before I'd even started, and in the end I had to ask Claude which ones I should actually be downloading.

My advice for trying it for the first time

  1. Understand its limits. A local model won't behave exactly like the big paid ones, and knowing that up front saves a lot of frustration.
  2. Check your hardware first. Your graphics card's memory decides which models will run well, so match the model size to it.
  3. Ask your other AI for help. There's a lot in LM Studio, but in reality you only need a few things, and the AI tools you already use can point you to them quickly.
  4. Start with one job. Get one job working properly first, then build on it.
  5. Watch the context window. When my local model kept failing on bigger jobs, the fix was widening its context and trimming each request down to only what that job required.

The honest bit

I love that there's no token cost, and I should probably use Wade more than I do, but I like Claude too much. In practice I use both: Wade for the jobs it does well and cheaply at home, and Claude for heavier coding and problem solving. Knowing which tool fits which job is a skill in itself.

If cost is what's holding you back from AI, I wrote about exactly that in no, you don't need to spend thousands a month to actually use AI.


I write about practical AI for marketers and building with free tools at garystow.co.uk. This piece first appeared there.

Top comments (0)