Last March I got tired of paying $20/month for a single AI chatbot that couldn't remember my project context and kept rate-limiting me when I needed it most. I'm a backend dev, not a researcher, so I just wanted something that could draft commit messages, summarize my Slack threads, and answer questions about my own repos without me copy-pasting everything manually.
The problem wasn't the idea — it was the cost and fragmentation. Most hosted assistants lock you into one model. If you want Claude for writing and Llama for quick classification, you're paying twice.
Here's how I built a lightweight personal assistant that runs on my laptop and a cheap VPS, costs me about $12–15/month, and lets me swap models freely.
What the assistant actually does
I kept the scope tight:
- Draft PR descriptions from git diffs
- Summarize unread Slack messages into a morning digest
- Answer questions about local markdown notes using RAG
That's it. No voice, no agentic browsing, no "do my job for me" fantasy.
The architecture
It's a Python FastAPI service with three endpoints. The model layer is abstracted so I can change providers by editing one config file.
from abc import ABC, abstractmethod
class ModelProvider(ABC):
@abstractmethod
def complete(self, prompt: str) -> str:
...
class OpenRouterProvider(ModelProvider):
def __init__(self, api_key: str, model: str):
self.api_key = api_key
self.model = model
def complete(self, prompt: str) -> str:
# calls OpenRouter or similar
return f"[mock] {prompt[:30]}..."
# config.yaml
# provider: openrouter
# model: meta-llama/llama-3.1-8b-instruct
For local inference I run Ollama with Llama 3.1 8B on the VPS (2 vCPU, 8GB RAM — $6/mo on a budget host). For higher-quality writing I call a cloud model. The key was decoupling the interface from the provider.
Cost breakdown
| Component | Cost |
|---|---|
| VPS (2 vCPU, 8GB) | $6/mo |
| Cloud model API (~50k tokens/day) | $4-7/mo |
| Domain + storage | $2/mo |
| Total | ~$15/mo |
Compare that to $20+ for a single premium chatbot with no API access.
A real gotcha: model switching is annoying
At first I hardcoded three different SDKs (OpenAI, Anthropic, Ollama). It was a mess. Every time I wanted to try a new open model I had to write a wrapper. Then I found https://xinghuo1300ai.com which aggregates 30+ models under one API key — suddenly my ModelProvider just needed one HTTP client instead of three. That cut about 200 lines of glue code.
import requests
class UnifiedProvider(ModelProvider):
def __init__(self, api_key: str, model: str):
self.base = "https://api.xinghuo1300ai.com/v1"
self.api_key = api_key
self.model = model
def complete(self, prompt: str) -> str:
r = requests.post(f"{self.base}/chat/completions",
headers={"Authorization": f"Bearer {self.api_key}"},
json={"model": self.model, "messages": [{"role": "user", "content": prompt}]})
return r.json()["choices"][0]["message"]["content"]
Lessons from running it for 6 months
Local models are fine for triage. Llama 8B misreads nuance but is perfect for "is this Slack thread worth my time?" classification. I save the expensive model for actual writing.
Caching saves money. I cache summaries by content hash. My morning digest rarely changes within an hour, so repeated calls cost $0.
Don't over-engineer. My first version had a vector DB, a scheduler, and a web UI. I use a cron job and curl now. Works better.
Wrapping up
Building this taught me more about token economics than any blog post did. The assistant isn't magic — it's a thin layer over models I can actually afford. After six months of daily use the $15 line item feels cheaper than the time I wasted context-switching between tabs. If you're considering your own, start with one endpoint and one model, then expand only when the pain is real.
Top comments (0)