🎲 A Tale of Inference Woes 🎲
Thy context window overflows with dread,
Thy API keys scattered 'cross thy thread.
Thou switchest providers mid-conversation,
And pray thy tokens find the right foundation.
No ward stands guard when tools go rogue,
No grimoire saves a session from the fog.
Thy agents wander, masterless and blind,
Thy prompts untested—leaving truth behind.
Thy wallet weeps. Thy latency doth creep.
Thy model's fine. Thy infrastructure? Not so deep.
Sound familiar?
I'm excited to share the public README for Arcanum — a .NET 10, single-binary, Native AOT, local-first AI inference hub that treats your infrastructure with the seriousness of a dungeon master and the organization of a well-kept grimoire.
Arcanum is one self-contained native executable. No runtime prerequisite. No "install the framework first." Just arcanum serve and you're running a full inference platform on loopback.
What's in the bag of holding:
🏰 Local-first & encrypted — SQLCipher-encrypted Grimoire persists every session, entry, and memory. Your data never leaves your machine unless you tell it to.
⚔️ Multi-provider native engine — Any OpenAI-compatible API (DeepSeek, Groq, Ollama via /v1, LM Studio, etc.) plus local GGUF models via a managed llama-server lifecycle. One hub, zero vendor lock-in.
🔮 OpenAI API compatible — POST /v1/chat/completions and GET /v1/models work with existing OpenAI clients out of the box. Drop-in replacement for your local stack.
🛡️ Wards & Sanctum — High-risk tools require operator approval before execution. Per-campaign sandboxes enforce path containment, network policy, and OS-level CPU/memory/FD limits via cgroups v2 and setrlimit.
📜 Spells, not prompts — Versioned markdown workflows with dependency resolution, tool allowlisting, and semantic routing. Dry-run cast previews before spending a single token.
🧙 Autonomous Apprentices — Goal-driven agents with plan generation, retry/backoff, autonomous plan revision, DM escalation, and parallel step execution.
🏰 The Conclave + A2A — Cross-apprentice delegation and full Agent-to-Agent protocol support. Your agents coordinate with each other and external A2A-compatible agents.
🌌 The Weave & Saga (RAG) — Five-phase embedding and semantic retrieval: session search, codebase retrieval, long-term associative memory, and embedding-based spell routing. All feature-flagged, all gracefully degrading.
⚖️ The Proving Grounds — Validate spells, prompts, and agent plans with Trials and Inquisitors before they hit production.
🔒 Native AOT — Ships as a self-contained native binary. Zero runtime prerequisite. Fast cold start. Minimal reflection surface.
The full architecture, every configuration key, every endpoint contract, every design decision — it's all documented. This is a project built for senior engineers and coding agents who want to understand why, not just how.
Built with .NET 10, ASP.NET Core Minimal API, EF Core 10 + SQLCipher, Microsoft.Extensions.AI, and ConsoleAppFramework.
📖 Check out the README on Google Drive
Feedback, questions, and war stories welcome. What's the one thing your inference stack is missing?
Top comments (0)