Hanzzel Corp

Posted on May 1

I built a self-hosted PC automation system with local LLaMA — it verifies actions actually worked

#opensource #ai #python #node

The problem I wanted to solve

Every automation tool I tried had the same issue: fire and forget. It sends the command and hopes for the best. I wanted something that verifies the action actually worked.

So I built Blue Arrow — a modular PC automation system that runs entirely local, no cloud, no subscriptions.

How it works

You send a command via Telegram:

"open Chrome" / "search report.pdf" / "write a letter in LibreOffice"

The system processes it through a state machine:

IDLE → INTENT → PLANNING → EXECUTING → VERIFYING → COMPLETED

After every action, a Python Verifier Engine checks if it actually worked:

Detects the process PID
Detects the window ID via wmctrl/xdotool
Checks focus state
Returns a confidence score (0.0–1.0)

If confidence < 0.5, it retries or reports failure.

The AI layer

Local LLaMA via Ollama handles everything:

Intent parsing — interprets what you actually meant
Text generation — drafts letters, explains errors, generates code
Semantic memory — vector embeddings, remembers context across sessions
Adaptive learning — learns your patterns over time

No external API calls. Everything runs on your machine. Tested with LLaMA 3.1 8B and 3.2 3B.

Architecture

30 specialized modules connected via a JSON Lines message bus, orchestrated by a Node.js 20 runtime. Each module declares its ports in a manifest — no cross-imports allowed.

Modules are classified as Core or Satellite. Satellites fail gracefully without taking down the system.

Three execution profiles:

minimal — headless, core only
standard — daily use with Telegram UI
full — everything including AI, verifier, gamification

The part I didn't plan

The Telegram interface became an RPG. XP, levels, achievements, themed scenes. It started as a joke during development and now I can't bring myself to remove it.

╔══════════════════════════════════════╗
║ 🎮 JARVIS RPG v1.0 ║
╠══════════════════════════════════════╣
║ ⭐ Level 5 ║
║ XP: [██████░░░░] 60% ║
╚══════════════════════════════════════╝

Stack

Node.js 20 + Python 3.11
Ollama (any LLaMA-compatible model)
Linux (xdotool, wmctrl)
MIT License

Quick start

git clone https://github.com/Hanzzel-corp/blue-arrow.git
cd blue-arrow
npm install && pip install -r requirements.txt
npm start

GitHub: https://github.com/Hanzzel-corp/blue-arrow

v0.1.0 just released. Feedback and contributions welcome.

DEV Community