Building a Private AI Assistant in 2026 — No Cloud Required (Mostly)

#ai #selfhosted #privacy #homelab

I've been running a self-hosted AI assistant 24/7 for the past month. Here's what works, what doesn't, and what surprised me.

The Setup

Hardware: NVIDIA Jetson Orin Nano Super (67 TOPS, 8GB unified memory, 512GB NVMe)
Software: OpenClaw (open source) + Ubuntu
Power draw: 20W average — my desk lamp uses more.
Cost: €549 one-time (I use ClawBox, the pre-built version)

What Runs Locally (No Internet)

Voice Processing

Whisper — speech-to-text, 90+ languages, runs entirely on-device
Kokoro — text-to-speech, natural sounding, also fully local
My voice data never leaves the box. Period.

Local LLMs

Llama 3.1 8B: ~15 tok/s — good for quick tasks, conversations
CodeLlama 7B: decent for code snippets
LLaVA 7B: vision model, can describe images
Hermes 3 8B: good for structured/agentic tasks

The Reality Check

8GB unified memory = 7-8B parameter models max. For a daily assistant, this covers maybe 60% of what I need. The other 40%? Cloud APIs.

The Hybrid Approach (This Is the Real Insight)

The key insight: privacy where it matters most (voice, personal data processing) + capability where it matters most (complex reasoning).

The ChatGPT Plus Trick (v2.0.0-8)

OpenClaw now supports ChatGPT Plus/Pro OAuth. Instead of paying $20/month for Plus AND separately for API access, you just sign in with your existing OpenAI account. No API keys, no double billing.

My Daily Workflow

6:00 AM — ClawBox checks email, summarizes what's important, drafts replies
Throughout day — Voice commands for quick lookups, timers, smart home
Work hours — "Hey ClawBox, review this PR" / "Summarize this PDF"
Overnight — Monitors prices, checks for important emails

Total hands-on time per day: ~5 minutes of talking to it.

The Honest Numbers

Hardware cost (€549) ÷ monthly savings (~€25) = ~22 months break-even. But I'd pay for the privacy alone.

Who This IS For

Privacy-conscious professionals
Homelabbers who want a silent, low-power AI addition
Anyone tired of managing 4 different AI subscriptions
People who want AI that works FOR them 24/7

DEV Community