DEV Community: Jonne Riepponen

Local LLMs as System Tooling: Running Sub-3B Models on Edge and Budget Hardware

Jonne Riepponen — Mon, 02 Nov 2026 18:34:38 +0000

You need an AI assistant that runs on your own machine. The cloud options are good but they are not yours. You want something that actually does things — reads your screen, controls your smart home, answers questions in Finnish or English, and never sends your data to a server you don't control.

The local LLM trend is about as practical as it sounds. It works on mid-range hardware if you pick the right model size. A sub-3B parameter model fits comfortably on consumer gear and handles everyday tasks without needing a dedicated GPU or a rack of servers. The trade-off versus cloud options is that the response times are slower, but your parents' conversations do not become training data for a future model owned by a third party.

Why "local" matters more than "smarter"

The most capable AI in 2026 is still a cloud-hosted model from OpenAI or Anthropic. But "most capable" is the wrong axis for elderly parents. They are not pushing the model with complex prompts or asking it to write production code. They are asking it to summarize a long email, explain a confusing medication instruction, draft a birthday reply to a grandchild, or settle a trivia dispute.

For all of those tasks, a smaller, more private model is more than sufficient — and the privacy trade-off is meaningfully better. The three options below all share one property: your parents' conversations do not become training data for a future model owned by a third party. That single property is worth more than a few IQ points of model quality.

You might think you need the biggest GPU to run this locally. You don't. A sub-3B model runs on an older laptop or a budget desktop. The key feature here is function calling — the model does not execute commands directly but returns a structured JSON decision that your script interprets. This is cleaner than giving the LLM raw bash access. The function list is a strict allowlist — defined in JSON Schema. The model can only call what is explicitly permitted. No rm -rf /, no matter how creatively phrased.

Option 1 — Run Ollama on an old machine

If your parents have an older Mac, a Windows PC from the last few years, or a Linux box gathering dust, Ollama is the strongest default choice. It is built into the system via a simple binary install, requires no separate app store login, and does not require a separate cloud account for the model itself. Your parents do not have to "use" Ollama — they just continue using their devices as they always have, and the AI shows up where it is useful: inside a chat window or a smart home dashboard.

The privacy architecture is genuinely best-in-class for consumer AI when you host it yourself:

On-device first. Most requests are processed locally on the device's CPU or integrated GPU. The data never leaves the machine unless you explicitly configure an API endpoint to send it elsewhere.
Private Cloud Compute (optional). For requests that need more compute than the phone can provide, you can route them to dedicated servers, but this is not required for sub-3B models. They run fine on a CPU.
No training on personal data. You are the model owner. Your parents' Ollama requests do not get sent to OpenAI or Anthropic. This is an architectural commitment, not just a setting.

The catch: you must manage the updates. If your machine does not have access to the internet, the model binaries will not update automatically. You need to check for new releases and pull them manually. If your parents are in a supported region for Apple Intelligence, you might still prefer that option over Ollama for simplicity. Check Settings → General → Apple Intelligence & Siri on their device; if the menu is present and the toggle works, they are eligible.

Setup time: 10 minutes on macOS or Linux, 20 minutes on Windows. Run the installer script or download the binary from the official site. Open a terminal or command prompt. Type ollama run qwen2.5:3b. Wait for the pull to finish. Talk them through three demo tasks: "summarize this notification stack," "rewrite this email to be shorter," and "create a fun image of a cat." That is enough for them to internalize what the AI can do without overwhelming them.

Option 2 — Use a browser-based local runner

If your parents are on Android, on a Windows PC, or simply do not have a recent enough machine for Apple Intelligence, the next best default is KoboldCPP or LM Studio, used through a browser at localhost:5001. These tools run locally but offer a web interface that looks familiar to anyone who has used a chatbot before.

KoboldCPP's privacy posture for local users is stronger than most cloud tools: you run the binary yourself, and it does not train on user conversations by default, full stop, on any tier. There is no setting to flip — the protection is built in because the data never leaves your machine. For elderly users who do not care which logo is on the page, KoboldCPP is currently the more privacy-respecting default choice for Windows machines with 8GB of RAM or more.

The trade-off: local runners on older hardware have slower response times than cloud services. You will see "processing" indicators for a few seconds longer. For someone using AI a few times a week, this is fine. For someone who has integrated it into daily routines, it is annoying but not dangerous. You can set the temperature and context window to match their needs. Lower temperature means more factual answers; higher temperature means more creative writing.

Tool	Platform	RAM needed	Best for
Ollama	macOS/Linux/Windows	6GB+	System integration
LM Studio	Windows/macOS/Linux	4GB+	Easy web UI
KoboldCPP	Windows/Linux	4GB+	High compatibility

All fit comfortably on mid-range hardware or run on CPU. Your Intel i5 desktop handles it fine if you pick the right model. The models I'm using are intentionally small: qwen2.5:3b for the main orchestrator, llama3.2:3b as a fallback, and phi3:mini for lightweight tasks.

The System Agent — Desktop Control

Once you have Ollama or LM Studio running locally, you can build a simple agent to control the desktop environment. Things it can do include setting volume with pamixer, adjusting brightness with xbacklight, launching applications with xdg-open, and getting active window info via wmctrl.

No raw bash. Every function is explicitly defined in a JSON schema file that the model reads before taking an action. The model sees a list of allowed functions and can only call what's on that list. This prevents the "I want to delete all my files" prompt from accidentally deleting anything, because the command to do so is not on the allowlist.

You might wonder if this setup is safe for your parents. If they are comfortable using their phones without fear of breaking something, they can use a local LLM agent with similar confidence. The only real risk is accidental deletion if the function list is too broad. Keep it strict. Define specific functions like set_volume(level) or open_app(name). Do not add delete_file() unless you really want to risk losing data.

The Vision Agent — "The Eyes"

If your parents have a tablet with a camera, you can take a screenshot or read a camera stream, then run the image through a lightweight VLM like Moondream2. It can answer: "What window is in the foreground?", "Did that build succeed?", "What's on my second monitor right now?".

The setup requires a tool to capture the screen image before sending it to the model. On Linux, use maim or grim. On Windows, use PowerShell commands with the built-in clipboard support. The model then analyzes the image and returns text. You can chain this with the system agent to read screen content aloud or highlight important windows.

The Home Assistant Agent

If your parents have smart home devices, you can talk to a Home Assistant instance over its WebSocket API. Lights, switches, climate, sensors — all accessible via the local network. I can say: "Turn off the office lights, but only if nobody's in there" and it checks the occupancy sensor before acting.

This setup requires the Home Assistant container to expose its API endpoint to the local LLM agent. The model sends a message like "dim the living room light" and the agent parses it into a JSON call to the Home Assistant API. The agent handles the HTTP request and returns success or failure status to the user interface.

Media & Data Agent

You might want the AI to pull in external data when needed: weather forecasts, electricity prices relevant in Finland, or web searches via a headless browser tool like curl or playwright. This is optional but useful if your parents need to plan their day around energy costs or check the weather for outdoor activities.

The agent runs in its own Docker container and does not know about the other agents. It just exposes a clean REST interface and waits for instructions from Ollama. Each agent is a FastAPI service inside a Docker container. They don't know about each other. They just expose a clean REST interface and wait for instructions from Ollama.

This isolation prevents one agent from breaking another. If the media agent crashes, the system agent keeps working. You can restart containers individually without affecting the whole stack. This is better than running everything as one monolithic Python script.

The tradeoff — Local vs Cloud

You might ask why not just use the cloud service. The answer is simple: privacy. When you run locally, your parents' conversations do not become training data for a future model owned by a third party. That single property is worth more than a few IQ points of model quality.

The local runner does not have unlimited context windows or infinite message quotas. You will hit hardware limits sooner than cloud services. A sub-3B model on a budget machine will choke on long documents that fit easily in the cloud. This is fine for most users who just want to summarize emails or draft messages. If you need more power, you can switch to a larger model or upgrade the hardware later.

Setup checklist

Here is what you should do next time you visit your parents:

Install Ollama or LM Studio on their machine.
Pull a sub-3B model like qwen2.5:3b for general tasks.
Configure a simple agent to control volume, brightness, and app launching.
Set up a web interface so they can chat without opening a terminal.
Test three demo tasks together to show what the AI can do.

If you are unsure about Docker or the command line, stick to the installer scripts. They handle most of the configuration for you. You just type ollama run qwen2.5:3b and wait for it to finish downloading. The model will start automatically after that.

Closing thought

You do not need a new computer to run local AI. An old laptop or a budget desktop works fine for sub-3B models. The setup takes ten minutes on Linux, twenty on Windows, and two on macOS. Your parents' conversations stay private because they never leave the machine.

Whatever you choose, take 30 minutes on a Saturday to set it up properly. The local LLM is the second-most-important thing; the setup is the first. A senior with a well-set-up local AI will be happier than a senior with a poorly-set-up flagship cloud service.

I Built a Password Strength Checker That Runs 100% in Your Browser (and What It Taught Me About Passwords)

Jonne Riepponen — Wed, 07 Oct 2026 15:38:39 +0000

The Phone Call That Started This

A client's junior admin called me last month. Their company had just rolled out a new password policy — 14 characters, three character classes, rotation every 90 days — and the helpdesk was drowning in reset tickets. The admin was proud of the policy. The users were writing the new passwords on sticky notes stuck to their monitors.

That is the entire password industry in one story: we keep making the rules harder without making the passwords better.

So I did what I always do when I'm annoyed: I built the thing I wish existed.

The Tool: 100% Client-Side, Nothing Leaves Your Browser

Try the Password Strength Checker — paste a password, get an honest strength rating: length, character classes, and common-pattern weaknesses. No account, no tracking, no server. Open the network tab in dev tools if you don't believe me; nothing is sent anywhere.

Why does that matter? Because a non-trivial number of online "password checkers" submit what you type to their backend for "analysis." Think about that for a second. You are sending your actual password to a stranger's server to find out if it's good. I built mine specifically so that can't happen — there is no backend. The HTML file doesn't even make a fetch request.

What I Actually Learned Testing 200 Passwords

I ran a pile of real-world-style passwords through the checker while building it — the kind of passwords people actually use, not the kind security training pretends they use. Three findings surprised me:

1. Length beats cleverness, every time. A 20-character passphrase of plain words (correct horse battery staple style) outscores an 8-character symbol-soup password on every honest metric. The math isn't close: each extra character multiplies the search space, while swapping e for 3 adds almost nothing because attackers already try that.

2. The complexity rules actively hurt. Forcing symbols and numbers pushes people toward predictable patterns: Password1!, Summer2025!, company-name-plus-year. Attackers' rule lists contain exactly these mutations. A policy that bans those patterns would be ten times more effective than a policy that demands a symbol — but "must contain !" is easier to audit than "must not be predictable," so here we are.

3. Rotation policies create the problem they claim to solve. Every 90-day rotation study I've read says the same thing: users increment a number (...Q3 → ...Q4) or rotate through three passwords. The new password is never stronger; it's just newer. If a password is strong and unique, leave it alone. Rotate on suspicion of compromise, not on a calendar.

What To Do Instead (The Pragmatic Version)

For yourself and your users:

Use a password manager. Bitwarden is free and open source. This is the single highest-leverage security change a normal person can make, and I wrote a 10-minute setup guide for elderly parents that works for anyone.
One long, unique password per service — generated, never reused. Reuse is what actually gets accounts taken over, not weak passwords.
Turn on 2FA everywhere that matters, especially email — email is the master key to every "forgot password" flow you have.
Test your own passwords with the checker. If your scheme produces anything the tool flags as a common pattern, change the scheme, not the password.

The Build Notes (For Fellow Tinkerers)

The whole tool is one self-contained HTML file: vanilla JS, no dependencies, no build step. The scoring is deliberately simple and explainable — entropy estimate from length and character classes, minus penalties for dictionary words, keyboard walks (qwerty, 1qaz), repeated characters, and date/sequence patterns. No machine learning, no API calls, no analytics.

Simplicity is the point. A security tool you can't audit is a security tool you shouldn't trust, and "trust me, it runs locally" is a claim anyone can verify in the network tab.

Try it: Password Strength Checker — free, private, no account. If it saves one person from sending their password to a stranger's server, it was worth the afternoon.

A Sandbox for the Linux Commands You're Afraid to Run

Jonne Riepponen — Mon, 05 Oct 2026 11:55:46 +0000

Almost nobody learns rm -rf the easy way. You read the man page, you use it carefully for a year, and then one day you watch your finger hit Enter a fraction of a second before your brain finishes checking the path.

The commands that destroy data don't punish typos. They punish the half second before the typo, when you were certain the path was right.

That half second of doubt isn't something documentation can give you. So I built a place where you can get it wrong on purpose, with nothing at stake but a score.

A fake shell that keeps score

The whole thing is one HTML file. No build step, no npm, no CDN — 2,340 lines, about 84 KB, and it runs from a local file if you want it to. You can try it here.

The prompt reads sysadmin@danger-zone:~$, and the ~ isn't decoration. Under the surface there's a small state object holding the entire world:

const state = {
    currentDir: '/home/sysadmin',
    dangerLevel: 'LOW',
    commandsUsed: 0,
    safetyScore: 100,
    history: [],
    currentFile: null
};

Seventeen commands are wired up. ls, cd, cat, find, df and ps handle the boring work, so you can navigate a fake filesystem that behaves the way you expect. The other five are the reason the tool exists.

Every dangerous command costs you twenty points

There is no confirmation prompt anywhere in it. That's on purpose. A real shell doesn't stop to ask, so a simulator that stops to ask teaches you the wrong reflex.

You pay in score instead:

if (result.dangerous) {
    triggerDangerEffects();
    state.safetyScore = Math.max(0, state.safetyScore - 20);
    updateStats();
}

Twenty points per mistake, starting from a clean 100. The danger meter recalculates on every keystroke: 80 and above is LOW, 60 is MEDIUM, 40 is HIGH, and anything under that is CRITICAL.

Five mistakes puts you at zero. That number isn't a punishment. It's a count of how many times you would have needed a restore.

The matcher reads tokens, not a real shell

Each dangerous handler inspects the argument array for the exact tokens that make the command lethal:

function rmCommand(args) {
    if (args.includes('-rf') && (args.includes('/') || args.includes('/home') || args.includes('/etc'))) {
        log('Files would be deleted in a real system.', 'error');
        log('In simulation: No actual files were harmed.', 'success');
        return { dangerous: true };
    }
    log('File deletion simulated safely.', 'info');
    return { dangerous: false };
}

dd if=/dev/zero of=/dev/sda triggers an explosion animation. chmod 777 /etc/passwd and iptables -F raise their own security warnings. mkfs /dev/sda1 formats nothing and tells you so.

It's a token check, not a parser. That distinction turns out to matter more than it sounds.

`rm -rf /*` walks straight through

Here's the hole, and I'd rather you hear it from me than find it yourself.

The check asks whether args contains the literal string /. The command that has actually ended careers is rm -rf /*, and after splitting on spaces its arguments are ['-rf', '/*'].

'/*' is not '/'. No match, no warning, no points deducted. The tool congratulates you: File deletion simulated safely.

Same story with rm -rf ~, rm -rf $HOME and rm -rf .. I matched the memorable version of the footgun and missed the common one. If you want to fix that, the honest version is to test each token with a prefix check, not an equality check:

const isRootish = args.some(a => !a.startsWith('-') && /^[/~.]/.test(a));

That's four lines and it catches the cases I shipped without. I'd rather show you the gap than let the tool quietly lie to you about your own reflexes.

Two handlers I wrote and could never reach

The dispatcher splits the line on spaces, then switches on the first token:

const parts = command.split(' ');
const cmd = parts[0];
switch (cmd) {
    case 'cat': catCommand(args[0]); break;
    case 'rm':  result = rmCommand(args); break;

Two cases, though, were written as whole commands with spaces still in them:

case 'cat /dev/random':                 result = randomCommand(); break;
case 'echo 1 > /proc/sys/kernel/panic': result = panicCommand(); break;

cmd is parts[0]. It can never contain a space, so neither case can ever fire. cat /dev/random falls through to the ordinary cat branch and gets quietly ignored.

I found that while going back through the file to write this post. It's the honest argument for writing build logs: the bug had been sitting there since the day I wrote it, and explaining the code out loud is what finally made me look.

What actually protects your data

A sandbox builds the reflex. It does not protect anything. Before you run anything recursive, work through this list:

Dry-run it. rsync -av --dry-run and find . -name '*.log' -print list what would be touched. Read the list properly instead of skimming it.
Print the variable. echo "$TARGET" immediately before you delete "$TARGET". Most rm -rf disasters are a variable that expanded to nothing, or to the wrong path.
Refuse a blank expansion. set -u turns an unset variable into an error instead of a silent empty string, which is the difference between a complaint and a wiped directory.
Check that the mount is mounted. A rm -rf on an unmounted /mnt/backup cheerfully deletes the real directory sitting underneath it.
Prove the backup, don't assume it. Restore one file from last night before you ever need to restore all of them.

The Friday backup audit is a 20-minute version of that last line, and it beats rebuilding a server on a Saturday. When something does go sideways, reading the logs properly is where the fix comes from — not from panic.

The transferable skill is the pause. Practising that pause in a simulator that scores you is cheap. Practising it on your only copy is how the stories start.

What's the command you'd never run without --dry-run? Tell me which one makes you double-check the path.

Related reads: