Single-Process Architecture in Rust: Why We Rejected Microservices for an Edge AI Platform

In 2026, "microservices" is practically a reflex answer for any new project. Split your system into independently deployable services, orchestrate with Kubernetes, glue them with a message bus. It works — until you're deploying to a $200 edge device in a factory with no reliable internet.

NeoMind is an open-source edge AI platform built in Rust that deliberately chose a single-process architecture. One binary. One process. Everything — the HTTP API, MQTT broker, storage engine, AI agent runtime, rule engine, and extension host — running in a single OS process.

This is the story of why, the trade-offs we accepted, and what we learned building it.

The Problem with Microservices at the Edge

Most AI platforms today follow the microservices playbook:

[API Gateway] → [Auth Service] → [Device Service] → [AI Service]
                     ↓                ↓                  ↓
               [PostgreSQL]      [TimescaleDB]      [Redis]
                     ↑                ↑                  ↑
              [Kafka/RabbitMQ] ← [Event Bus] ← [Notification Svc]

This works beautifully in the cloud. But at the edge, you hit a wall:

Resource overhead. Every microservice needs its own process, memory allocation, file descriptors, and network sockets. On a Raspberry Pi 5 with 8GB RAM, running 8 Docker containers with health checks, log aggregation, and service mesh proxies consumes 2-3GB before your application even starts.

Deployment complexity. docker-compose up sounds simple until you're SSH'ing into 50 factory floor devices to update certificates, debug a failing sidecar, or explain why the health-check probe is timing out on a flaky network.

Latency accumulation. A device telemetry reading that flows through Device Service → Event Bus → AI Service → Rule Engine → Notification Service touches 5 network hops. Each hop adds serialization, deserialization, and queue wait time. For real-time industrial monitoring, those milliseconds compound.

Operational burden. Twelve services means twelve log streams, twelve health endpoints, twelve restart policies, and twelve ways for a partial failure to cascade into a confusing debugging session at 3 AM.

The NeoMind Approach: One Process, Everything Inside

NeoMind's architecture looks like this:

┌─────────────────────────────────────────────────────┐
│                  Single Process                      │
│                                                      │
│  ┌─────────┐  ┌──────────┐  ┌───────────────────┐  │
│  │  Axum   │  │ Embedded │  │  AI Agent Runtime  │  │
│  │  HTTP   │  │  MQTT    │  │  (LLM + Memory +   │  │
│  │  Server │  │  Broker  │  │   Tools + Skills)  │  │
│  └────┬────┘  └────┬─────┘  └─────────┬─────────┘  │
│       │            │                   │             │
│  ┌────┴────────────┴───────────────────┴──────────┐  │
│  │              Event Bus (channels)               │  │
│  └────┬────────────┬───────────────────┬──────────┘  │
│       │            │                   │             │
│  ┌────┴────┐  ┌────┴─────┐  ┌────────┴──────────┐  │
│  │ Rule    │  │ Device   │  │  Extension Host    │  │
│  │ Engine  │  │ Manager  │  │  (Process-Isolated)│  │
│  └────┬────┘  └────┬─────┘  └─────────┬─────────┘  │
│       │            │                   │             │
│  ┌────┴────────────┴───────────────────┴──────────┐  │
│  │         Storage Layer (redb)                    │  │
│  │  Time-Series | State | LLM Memory | Logs       │  │
│  └────────────────────────────────────────────────┘  │
└─────────────────────────────────────────────────────┘

One cargo build --release produces a single binary. One ./neomind starts everything.

Why redb Instead of PostgreSQL

The storage layer uses redb, a pure-Rust embedded key-value store inspired by LMDB. This is a deliberate rejection of the "just use Postgres" default.

Zero external dependencies. No database server to install, no connection pooling to configure, no migration runner to orchestrate. The database is a file on disk, opened by the same process that serves HTTP requests.

B+ tree indexes with ACID transactions. redb provides serializable isolation, crash-safe writes, and efficient range scans — the same guarantees you'd expect from a traditional RDBMS, without the process overhead.

Time-series optimized. Telemetry data from IoT devices follows a write-heavy, append-mostly pattern. redb's B+ tree handles sequential inserts efficiently, and range queries over time windows are O(log n).

Memory-mapped reads. Hot data stays in the OS page cache without explicit caching logic. Cold data stays on disk. No Redis layer, no separate cache invalidation strategy.

The trade-off: no SQL. Queries are expressed in Rust with type-safe key ranges. For our access patterns (device state lookups, time-series scans, LLM memory retrieval), this is more efficient than an ORM layer over Postgres. For ad-hoc analytics, users export to their preferred tool.

The Event Bus: channels, Not Kafka

Inter-module communication uses Rust's tokio::sync::broadcast channels — in-process, zero-copy, backpressure-aware.

// Simplified: device telemetry event
pub struct DeviceEvent {
    pub device_id: String,
    pub metric: String,
    pub value: f64,
    pub timestamp: i64,
}

// Any module can subscribe
let mut rx = event_bus.subscribe::<DeviceEvent>();

// Any module can publish — no serialization, no network hop
event_bus.publish(DeviceEvent {
    device_id: "sensor-01".into(),
    metric: "temperature".into(),
    value: 28.5,
    timestamp: now(),
});

Compared to Kafka or RabbitMQ:

Aspect	Kafka/RabbitMQ	In-Process Channels
Latency	1-10ms per hop	< 1μs
Memory	Separate process + JVM/BEAM	Shared address space
Serialization	JSON/Protobuf required	Zero-copy struct passing
Durability	Built-in (disk-backed)	Optional (redb persistence)
Multi-node	Yes	No (single-node by design)

The durability gap is closed by persisting critical events to redb before publishing. If the process crashes, the event log survives on disk and replays on restart.

Extension Isolation: Processes, Not Containers

Here's where the single-process model gets nuanced. Extensions (YOLO object detection, Home Assistant bridge, Modbus adapter, etc.) run as separate OS processes, communicating with the core via FFI and a capability-based permission system.

┌─────────────────────────────────────────────┐
│  Core Process (Rust, single binary)          │
│  ┌─────────┐ ┌──────────┐ ┌──────────────┐  │
│  │ HTTP    │ │ MQTT     │ │ AI Agent     │  │
│  │ Server  │ │ Broker   │ │ Runtime      │  │
│  └─────────┘ └──────────┘ └──────────────┘  │
│                                              │
│  Extension Host                              │
│  ┌──────┐ ┌──────┐ ┌──────┐ ┌──────┐       │
│  │Spawn │ │Spawn │ │Spawn │ │Spawn │       │
│  └──┬───┘ └──┬───┘ └──┬───┘ └──┬───┘       │
└─────┼────────┼────────┼────────┼────────────┘
      │        │        │        │
  ┌───┴──┐ ┌──┴───┐ ┌──┴───┐ ┌──┴───┐
  │ YOLO │ │ HA   │ │Modbus│ │BACnet│
  │(Rust)│ │Bridge│ │(Rust)│ │(Rust)│
  └──────┘ └──────┘ └──────┘ └──────┘
  separate processes, FFI + capabilities

Why not run extensions in-process? Because extensions are where third-party code lives. A buggy YOLO model loader or a misbehaving MQTT bridge shouldn't crash the core. Process boundaries give us:

• Crash isolation — kill -9 the extension, core keeps running
• Memory isolation — an extension's memory leak doesn't starve the core
• Capability enforcement — extensions declare what they need (network, filesystem, device access) and get only those permissions
• Hot reload — restart an extension without restarting the whole system

This is Erlang's "let it crash" philosophy, adapted for Rust: the core almost never crashes (ownership + borrow checker), but extensions might — and that's fine.

Why not containers? Docker adds ~50MB overhead per container (runc, containerd-shim, cgroup setup). On edge devices, that's unacceptable. OS process spawning is ~1ms with negligible memory overhead beyond the extension's own footprint.

The AI Agent Runtime: In-Process LLM Orchestration

The AI agent is the most complex subsystem, and it runs entirely in-process:

• LLM backends (Ollama, OpenAI, Anthropic, Google, DeepSeek, etc.) are HTTP clients making external calls, but the orchestration logic, prompt construction, and response parsing stay in-process.
• Multi-tier memory (Profile, Knowledge, Task, Session) is stored in redb with automatic extraction and compression — no external vector database.
• Tool calling dispatches in-process via typed command structs, not string-based APIs. The AI says "query device temperature" and the runtime calls DeviceManager::get_metric("sensor-01", "temperature") directly.

// Type-safe tool dispatch — no eval(), no shell, no string parsing
enum AgentCommand {
    QueryDevice { device_id: String, metric: String },
    CreateRule { rule: RuleDefinition },
    ListDevices { filter: DeviceFilter },
    ControlDevice { device_id: String, action: DeviceAction },
}

// The AI's intent is deserialized into a typed enum, then matched
match command {
    AgentCommand::QueryDevice { device_id, metric } => {
        device_manager.get_metric(&device_id, &metric).await
    }
    // ...
}

This eliminates an entire class of injection vulnerabilities that plague string-based agent frameworks.

What We Gave Up

Honesty demands we acknowledge the trade-offs:

No horizontal scaling. A single process means a single node. If you need to process telemetry from 100,000 devices across 10 geographic sites, you run 10 instances — each managing its own fleet. There's no shared state between instances (by design). For multi-site coordination, you layer a separate aggregation tier.

No polyglot runtime. Everything in the core is Rust. If you want to contribute a core feature, you write Rust. Extensions can be any language that speaks the FFI protocol, but the core is monolingual.

No independent deployability. A bug in the rule engine requires redeploying the entire binary. There's no "just update the rule engine service." For edge deployments, this is actually simpler (one binary to distribute), but it means CI/CD is all-or-nothing.

No built-in multi-tenancy. Each instance serves one tenant. Multi-tenant setups run multiple instances behind a reverse proxy — simpler than building tenant isolation into every module.

Benchmarking: Single Process vs. Microservices

On a Raspberry Pi 5 (8GB RAM), we measured:

Metric	NeoMind (Single Process)	Equivalent Microservices (8 containers)
Idle memory	~120 MB	~1.8 GB
Cold start	< 2 seconds	~45 seconds (all containers healthy)
Telemetry latency (device → rule → action)	~3 ms	~85 ms
Binary size	~35 MB	~2.1 GB (all images)
Deployment	1 file copy + systemd	docker-compose + registry + certs

The memory difference alone makes single-process viable on hardware where microservices simply aren't.

When to Choose Single-Process

This isn't a "microservices are bad" take. Microservices are the right choice for large teams, high-scale cloud workloads, and systems that need independent scaling. But single-process is the right choice when:

1. Deployment target is constrained hardware — edge devices, IoT gateways, embedded systems
2. Operational simplicity matters more than architectural purity — no SRE team on the factory floor
3. Latency is critical — real-time device control, industrial automation
4. The team is small — one codebase, one language, one deployment artifact
5. Offline operation is required — no service mesh, no cloud dependency

NeoMind hits all five criteria. Your project might not — and that's fine.

Try It

NeoMind is open source (Apache 2.0). One command to get started:

curl -fsSL https://raw.githubusercontent.com/camthink-ai/NeoMind/main/scripts/install.sh | sh

• Source: github.com/camthink-ai/NeoMind
• Docs: wiki.camthink.ai
• Community: Discord

---

If you've built edge systems that rejected microservices, I'd love to hear about your trade-offs in the comments.