Amit Chandra

Posted on Jul 23 • Originally published at dev.to

I built a local-first personal AI gateway that connects 32 messaging platforms to 13 LLM providers — NeuralCleave

#python #opensource #ai #showdev

``Most AI assistants ask you to move to them. New app, new subscription, new interface. Your conversations live in their cloud, your integrations go through their store, and when you stop paying, it all disappears.

I wanted something different: one agent that lives on my machine, connects to every messaging platform I already use, remembers everything I've told it, routes requests to the right model automatically, and never sends my data anywhere I didn't choose.

That's what NeuralCleave is.

GitHub: TheAmitChandra/NeuralCleave

Install: pip install neuralcleave

Docs: docs.neuralcleave.com

Website: neuralcleave.com

What it is

NeuralCleave is a local-first AI assistant gateway written in Python. You run it on your machine (or a small VPS), point your Telegram/Discord/Slack/WhatsApp at it, and from that moment every message you send to any of those platforms reaches the same agent — with the same memory, the same tools, and the same personality.

Key numbers for v2.1.0:

Metric	Count
Channel adapters	32
LLM providers	13
REST endpoints	41
Tests passing	5,064
Memory tiers	3 (Redis → Qdrant → SQLite)

The problem: fragmented AI

If you use AI seriously today, you probably have:

ChatGPT open in a browser tab for general questions
Claude in another tab for writing and reasoning
A Telegram bot you set up once and forget about
A Discord bot your server uses
Ollama running locally when you care about privacy

None of these talk to each other. None of them remember what you told a different one yesterday. You're managing five different conversations with five different versions of "your" AI assistant.

NeuralCleave collapses all of that into a single agent you control.

How it works

The gateway pipeline

Every incoming message — regardless of which platform it came from — goes through the same pipeline:

`plaintext Channel Adapter ↓ AgentRuntime ↓ ModelRouter (picks the right LLM for this task type) ↓ 3-Tier Memory (reads context, writes back) ↓ ReflectionEngine (scores response, regenerates if quality < threshold) ↓ Channel Adapter (sends reply back to the originating platform) `

The gateway is a FastAPI application. Every integration is async, every LLM call is streamed, and the whole thing runs on a single Python process with optional Redis and Qdrant for the memory tiers.

Starting it

`bash pip install neuralcleave neuralcleave init # writes ~/.neuralcleave/config.toml neuralcleave start # gateway up on localhost:7432 `

That's it. The gateway starts with a WebSocket chat UI at http://localhost:7432/app and a full REST API at /api/v1/.

32 Channel Adapters

The point of a gateway is that every platform routes to the same agent. Here are all 32:

Mainstream
Telegram · Discord · WhatsApp · Slack · Microsoft Teams · Email (IMAP/SMTP)

Social & federated
Bluesky (AT Protocol) · Mastodon · Nostr (NIP-04 encrypted DMs) · Twitter/X (via webhook)

Asian platforms
WeChat Work · LINE · Zalo · QQ Bot · Feishu / Lark · Doubao

Workplace
Mattermost · Rocket.Chat · Google Chat · Nextcloud Talk · Synology Chat

Dev / infrastructure
IRC · XMPP · Matrix · Generic Webhook · WebSocket

Voice & telephony
Twilio Voice (multi-turn speech via TwiML) · SMS (Twilio)

Entertainment
Twitch (IRCv3) · Viber · iMessage (via BlueBubbles)

P2P
Tlon / Urbit

Every adapter handles authentication, webhook verification (HMAC-SHA256, Ed25519, JWT — whatever the platform requires), and maps the platform's native event format to NeuralCleave's internal Message model. You enable them in your config:

`toml
[channels.telegram]
enabled = true
bot_token = "ENV:TELEGRAM_BOT_TOKEN"

[channels.discord]
enabled = true
bot_token = "ENV:DISCORD_BOT_TOKEN"

[channels.slack]
enabled = true
bot_token = "ENV:SLACK_BOT_TOKEN"
signing_secret = "ENV:SLACK_SIGNING_SECRET"
`

All secret values support ENV:VAR_NAME resolution — nothing sensitive lives in the config file.

13 LLM Providers with Task-Aware Routing

NeuralCleave doesn't just pick one model and call it for everything. The ModelRouter classifies each incoming request into one of 10 task types and routes it to the optimal provider:

Task type	Default provider
`complex_reasoning`	Claude Opus / GPT-4o / Grok-3
`code_generation`	DeepSeek Coder / Qwen Max
`code_review`	DeepSeek / Qwen Max
`summarization`	Cohere Command-R+
`intent_extraction`	Zhipu GLM-4 Flash
`cheap_inference`	Ollama / Doubao / ERNIE Speed
`general`	Moonshot / GPT-4o-mini
`image_analysis`	Gemini Pro Vision
`voice_transcription`	Whisper (local)
`embedding`	Ollama / OpenAI

All 13 supported providers:

Global: Anthropic · OpenAI · Google Gemini · Mistral AI · xAI Grok · Cohere

Local/open: Ollama (any GGUF model)

Asia: DeepSeek · Moonshot / Kimi · Zhipu GLM · Alibaba Qwen · Baidu ERNIE · ByteDance Doubao

You can override routing per-request or set a privacy mode that forces everything to Ollama — no data ever leaves your machine.

`toml
[models]
default_provider = "anthropic"
privacy_mode = false

anthropic_api_key = "ENV:ANTHROPIC_API_KEY"
openai_api_key = "ENV:OPENAI_API_KEY"
deepseek_api_key = "ENV:DEEPSEEK_API_KEY"
ollama_base_url = "http://localhost:11434"
`

3-Tier Memory

This is one of the things I'm most proud of. Most AI assistants have no memory, or a flat vector store, or an expensive cloud sync. NeuralCleave uses a three-tier cascade:

`plaintext Hot tier → Redis (recent session, TTL-based, sub-millisecond) Vector tier → Qdrant (semantic ANN search, cosine similarity) Long-term → SQLite (importance-scored, permanent, offline-capable) `

When a new message arrives:

The hot tier is checked first — recent context loads in < 1 ms
A semantic search runs against Qdrant for relevant long-term memories
The combined context is prepended to the LLM prompt
After the response, important facts are extracted and written back to all three tiers

Redis and Qdrant are optional — the gateway runs with only SQLite if you don't have them, just with reduced recency/semantic capability.

Memory is namespaced per agent node (for multi-agent setups) and per channel, so your Telegram conversations don't pollute your Discord context unless you want them to.

ReflectionEngine: Automatic Quality Control

Every response is scored before it reaches you. The ReflectionEngine evaluates four dimensions on a 0–100 scale:

Relevance — did it actually answer the question?
Completeness — is anything obviously missing?
Accuracy — does it contradict known facts from memory?
Tone — does it match the requested persona?

If the weighted score falls below a configurable threshold, the response is discarded and regenerated once with an improved prompt that includes the failure reason. You see only the passing response.

`toml [reflection] enabled = true threshold = 72 # 0–100; responses below this score are regenerated `

This catches the lazy non-answers ("I don't have access to real-time data, but...") and the hallucinated confident wrong answers before they reach your inbox.

Voice Pipeline

The voice stack is fully local by default:

STT: OpenAI Whisper (tiny through large-v3), runs on-device
TTS: 3-tier cascade — ElevenLabs (highest quality) → Kokoro (offline, good quality) → pyttsx3 (offline, always available)
Wake word: OpenWakeWord — always-on detection, CPU-only
Voice cloning: ElevenLabs voice cloning API if you have a key

The ContinuousVoiceListener is an async process that runs a VAD (RMS energy) loop, detects utterances, transcribes them locally with Whisper, routes the text through the agent pipeline, and speaks the response. No cloud dependency required for any of this.

`bash neuralcleave voice listen # start always-on voice mode neuralcleave voice speak "Hello from NeuralCleave" `

Plugin SDK

NeuralCleave uses PEP 451 entry-points for plugin discovery — the same mechanism pip itself uses. You write a Python package, register it under the neuralcleave.plugins entry-point group, and the gateway discovers and loads it automatically.

`python

my_plugin/plugin.py

from neuralcleave_sdk import Plugin, Tool, ToolResult

class WeatherTool(Tool):
name = "get_weather"
description = "Get current weather for a city"
parameters = {
"city": {"type": "string", "description": "City name"}
}

async def run(self, city: str) -> ToolResult:
    # ... fetch weather ...
    return ToolResult(content=f"It's 22°C in {city}")

class WeatherPlugin(Plugin):
name = "weather"
version = "1.0.0"
tools = [WeatherTool]

async def on_load(self) -> None:
    print("Weather plugin loaded")

`toml

pyproject.toml

[project.entry-points."neuralcleave.plugins"]
weather = "my_plugin.plugin:WeatherPlugin"
`

`bash pip install . neuralcleave plugins list # → weather 1.0.0 ✓ `

Plugins support hot-reload — no gateway restart required:

`bash neuralcleave plugins reload weather `

Three official plugins ship with the project: neuralcleave-github, neuralcleave-notion, and neuralcleave-google-calendar. All are on PyPI.

Hub Marketplace

The Hub is NeuralCleave's built-in plugin marketplace. Every package goes through a dual-pass security scan before installation:

AST walk — blocks 13 dangerous imports (subprocess, ctypes, socket, etc. when used offensively)
Regex scan — 14 dangerous patterns (eval(, exec(, __import__, obfuscated base64 calls)

If the scan passes, the plugin is installed, verified by SHA-256 checksum, and hot-loaded:

`bash neuralcleave hub search weather neuralcleave hub install neuralcleave-weather neuralcleave hub scan neuralcleave-weather # audit without installing `

Multi-Agent Orchestrator

For more complex setups, you can define named agent nodes — each with its own model, routing rules, memory namespace, and concurrency limit:

`toml
[[orchestrator.nodes]]
name = "researcher"
model_override = "claude-opus-4-8"
task_types = ["complex_reasoning", "summarization"]
channel_patterns = ["telegram", "discord"]
priority = 10

[[orchestrator.nodes]]
name = "coder"
model_override = "deepseek-coder"
task_types = ["code_generation", "code_review"]
priority = 8

[[orchestrator.nodes]]
name = "fast"
model_override = "ollama/llama3"
task_types = ["general", "cheap_inference"]
priority = 5
`

The AgentOrchestrator runs a filter → priority → round-robin pipeline. Each node has its own LRU memory namespace, so the coder agent's context doesn't bleed into the researcher agent's memory.

Live Canvas

The Canvas is a real-time visual output stream. The agent can render structured blocks — text, markdown, code, tables, charts, HTML — to a live WebSocket feed that any connected viewer sees instantly:

`bash neuralcleave canvas open # opens http://localhost:7432/canvas in browser `

Block types: text · markdown · code · table · chart (bar/line/pie via Canvas API) · image · html

The canvas is particularly useful when asking the agent to reason through something complex — it can stream its thinking as structured blocks rather than a wall of text.

Prometheus Observability

13 built-in metrics exposed at GET /api/v1/metrics in Prometheus exposition format:

`prometheus neuralcleave_requests_total{channel,status} neuralcleave_llm_calls_total{provider,model,status} neuralcleave_llm_latency_seconds{provider,model} neuralcleave_memory_reads_total{tier} neuralcleave_memory_writes_total{tier} neuralcleave_reflection_scores{result} neuralcleave_plugin_calls_total{plugin,tool,status} neuralcleave_active_connections{channel} neuralcleave_websocket_messages_total neuralcleave_channel_errors_total{channel,error_type} neuralcleave_uptime_seconds neuralcleave_memory_entries_total{tier} neuralcleave_hub_installs_total{package,status} `

A pre-built Grafana dashboard JSON ships in the repo. Plug it into any Grafana instance and you get request rates, latency histograms, error breakdowns, and memory tier utilization — all from docker-compose up.

Desktop App + PWA

A Tauri v2 desktop app ships for all three platforms:

macOS — native .dmg, Apple Silicon + Intel
Windows — .msi and .exe NSIS installer
Linux — .AppImage and .deb

The desktop app bundles the Python backend via PyInstaller (no Python installation required), runs the gateway as a sidecar process, and surfaces the frontend in a native WebView. System tray integration, global hotkey (Ctrl+Shift+Space), close-to-tray, single-instance guard, and autostart are all included.

If you don't want the desktop app, the gateway also ships a PWA — installable from any browser, works on iOS and Android. No app store involved.

Deployment

Running it locally is neuralcleave start. Deploying it to a server or cloud is:

`bash neuralcleave cloud generate # writes Dockerfile, docker-compose.yml, railway.toml, render.yaml docker-compose up -d `

The cloud generate command auto-detects your config and writes manifests for Docker, Railway, Render, Fly.io, and generic VPS. Redis and Qdrant are wired into the compose file. Everything is included.

Install

`bash

Python 3.12+ required

pip install neuralcleave

Desktop app (macOS / Windows / Linux)

→ https://github.com/TheAmitChandra/NeuralCleave/releases/tag/app-v2.1.0

Docker

docker run -p 7432:7432 ghcr.io/theamitchandra/neuralcleave:latest
`

`bash neuralcleave init # interactive setup wizard (or -y for defaults) neuralcleave start # gateway on http://localhost:7432 `

Official plugins on PyPI

`bash pip install neuralcleave-sdk # Plugin/Tool/ChannelAdapter ABCs pip install neuralcleave-github # GitHub issues, PRs, repos pip install neuralcleave-notion # Notion databases and pages pip install neuralcleave-google-calendar # Google Calendar events `

Why open source?

I started this project because I was paying for four different AI subscriptions and none of them talked to each other. I wanted something that worked the way I actually work — across every app I use, with memory that persists, on hardware I own.

The code is under BUSL 1.1 (source-available, free for personal use). It converts automatically to Apache 2.0 on 2030-06-26.

DEV Community