DEV Community: Ming

NeoMind: A Rust-Powered Edge AI Platform That Talks to Your IoT Devices

Ming — Fri, 07 Aug 2026 07:30:25 +0000

NeoMind: A Rust-Powered Edge AI Platform That Talks to Your IoT Devices

Talk to your devices in natural language. The AI understands your intent, queries device states, creates automation rules, and takes action — all on your own hardware, with zero cloud dependency.

The Problem We're Solving

Edge AI is stuck between two extremes. On one side, you have cloud-first platforms that ship every sensor reading to a remote server — introducing latency, bandwidth costs, and privacy concerns that make industrial customers cringe. On the other, you have bare-metal inference frameworks that run a single model well but offer no device management, no automation, no natural language interface.

What's missing is the middleware layer — something that sits on your edge hardware, connects to real devices, runs intelligent agents, and lets operators interact with the whole system conversationally. That's what NeoMind is.

What Is NeoMind?

NeoMind is an open-source, edge-deployed AI platform built in Rust. It brings together:

LLM-powered agents that run directly on your hardware (via Ollama or cloud backends)
Device connectivity through MQTT, BLE, and Webhook protocols
A JSON rule engine for automation with recursive conditions and debouncing
Real-time dashboards with drag-and-drop widgets
22 official extensions spanning vision AI (YOLO, face recognition, OCR), voice (TTS/ASR), and IoT bridges (Home Assistant, Modbus, BACnet, ONVIF, OPC-UA, LoRaWAN)

All packaged in a single binary. No Docker Compose. No external database. No message broker to install.

# One command. Everything runs.
curl -fsSL https://raw.githubusercontent.com/camthink-ai/NeoMind/main/scripts/install.sh | sh

Architecture: Single-Process, Self-Contained

NeoMind's most distinctive design choice is its single-process architecture. The API server (Axum), embedded MQTT broker, redb storage, rule engine, AI agent runtime, and extension runner all live in one process.

┌──────────────────────────────────────────────────────────┐
│               Desktop App / Web UI                       │
│                React 18 + TypeScript                     │
├──────────────────────────────────────────────────────────┤
│              Tauri 2.x / Browser                         │
└────────────────────────┬─────────────────────────────────┘
                         │ REST / WebSocket / SSE
                         ▼
┌──────────────────────────────────────────────────────────┐
│                   API Gateway (Axum)                      │
│  ┌────────┐ ┌────────┐ ┌────────┐ ┌────────┐            │
│  │  Auth  │ │Devices │ │Automate│ │Messages│            │
│  └────────┘ └────────┘ └────────┘ └────────┘            │
├──────────────────────────────────────────────────────────┤
│                     Event Bus                             │
│  ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌────────────┐ │
│  │ Devices  │ │Automation│ │ AI Agent │ │ Extensions │ │
│  │ MQTT/BLE │ │ Rules    │ │ Chat     │ │ Isolated   │ │
│  │ Webhook  │ │ Transform│ │ Tools    │ │ Native+WASM│ │
│  └──────────┘ └──────────┘ │ Memory   │ └────────────┘ │
│                            └──────────┘                  │
├──────────────────────────────────────────────────────────┤
│                  Storage Layer (redb)                     │
│  Time-Series │ State │ LLM Memory │ Push Logs            │
└──────────────────────────────────────────────────────────┘

Why this matters:

Deployment simplicity — cargo run and you're done. No orchestrating 12 microservices.
Resource efficiency — One process shares memory, file descriptors, and connection pools. Critical on constrained edge hardware.
Operational clarity — One log stream, one PID to monitor, one binary to update.

Three Design Philosophies

1. Edge-First, Cloud-Optional

NeoMind defaults to local LLMs via Ollama — your data never leaves your LAN. When you need frontier-model capabilities, switch to OpenAI, Anthropic, Google, or any OpenAI-compatible endpoint with a single config change. The agent runtime is backend-agnostic.

This isn't just a privacy play. For industrial IoT deployments on factory floors or remote campuses, reliable internet isn't guaranteed. NeoMind works fully offline.

2. Crash-Isolated Extensions

Extensions run in separate processes and communicate with the core via FFI with capability-based permissions. If your YOLO object detection extension crashes, the main service and all other extensions are completely unaffected.

This is modeled after Erlang's "let it crash" philosophy, adapted for a Rust codebase where the core itself rarely crashes — but third-party extensions might.

3. Type-Safe End-to-End

The Rust backend provides compile-time guarantees across the entire stack. Agent CLI commands dispatch in-process with structured data types — no fragile string parsing, no eval(), no shell injection vectors.

AI Agents: More Than a Chatbot

NeoMind's AI layer goes beyond simple Q&A:

Natural Language Chat — Conversational interface to query device states, create automation rules, and control hardware. "Turn off all cameras on Floor 3 after 10 PM" becomes a real command.
Autonomous Agents — Scheduled or event-driven agents that monitor telemetry, analyze patterns, and trigger actions independently. Think: "Watch the temperature sensors and alert me if any reading exceeds 30°C for more than 5 minutes."
Multi-tier Memory — Profile memory, knowledge base, task history, and session context with automatic extraction and compression. The AI remembers your facility layout and past incidents.
Skill System — YAML + Markdown skill files that guide agent behavior for specific scenarios. Write a skill for "factory floor safety check" and the agent follows it precisely.

Supported backends include Ollama, OpenAI, Anthropic, Google, xAI, Qwen, DeepSeek, GLM, MiniMax, and any OpenAI-compatible endpoint.

Device Management: MQTT, BLE, and Beyond

NeoMind treats IoT devices as first-class citizens:

Embedded MQTT Broker — with mTLS and CA certificate support. Your devices connect directly, no external broker needed.
BLE Provisioning — Zero-touch device setup via Bluetooth (Tauri native + Web Bluetooth). Walk up to a device, pair, configure.
Auto-Discovery — Plug in a new device and NeoMind detects it, registers its type, and offers AI-assisted onboarding.
Custom Device Types — Define metrics and commands via JSON type definitions. No code changes required.

The extension ecosystem adds bridges for Home Assistant (3000+ entity integrations), Modbus (TCP/RTU PLCs), BACnet (building automation), ONVIF (IP cameras), OPC-UA (industrial servers), and LoRaWAN (ChirpStack/TTN sensors).

Automation Without Code

The JSON rule engine supports recursive conditions with comparison, range, and logical operators:

{
  "condition": {
    "source": "device:sensor:temperature",
    "operator": "greater_than",
    "threshold": 30
  },
  "actions": [
    {"type": "notify", "channel": "slack", "message": "Temperature alert!"},
    {"type": "execute", "command": "device:fan:turn_on"}
  ],
  "cooldown": 300,
  "for_duration": 60
}

Combine rules with data transforms (JavaScript-based virtual metrics) and scheduled AI agents to build sophisticated automation pipelines — all without writing backend code.

22 Extensions and Growing

The extension marketplace ships with production-ready integrations:

Category	Extensions
Vision	YOLO object detection, face recognition, OCR, visual grounding
Voice	CosyVoice TTS, voice cloning, multilingual ASR, voice assistant pipeline
IoT Bridges	Home Assistant, Modbus, BACnet, ONVIF, OPC-UA, LoRaWAN
Utilities	Weather forecast, stream player, WASM demo

Each extension is process-isolated with capability-based permissions. Build your own following the Extension Development Guide.

Cross-Platform: Desktop, Server, Mobile

NeoMind ships native desktop apps via Tauri 2.x for macOS (Apple Silicon + Intel), Windows, and Linux. The responsive web UI is optimized for phone and tablet. Server deployments support Docker, systemd, and nginx reverse proxy.

# Docker — multi-arch, pulls official image
docker run -d --name neomind \
  -p 9375:9375 -p 1883:1883 \
  -v neomind-data:/app/data \
  camthink/neomind:latest

Get Started

Download: GitHub Releases (.dmg, .msi, .AppImage, .deb)
5-Minute Quick Start: wiki.camthink.ai
Source Code: github.com/camthink-ai/NeoMind (Apache 2.0)
Community: Discord · GitHub Issues

Contributing

NeoMind is open source and actively seeking contributors. Four ways to get involved:

Build an Extension — New AI models, data sources, or integrations
Add a Device Type — Define metrics and commands for your IoT hardware
Create a Dashboard Widget — Reusable React components for the community
Core Contributions — Rust backend, React frontend, documentation

NeoMind is developed by CamThink AI. Licensed under Apache 2.0.

Technical Implementation and Architectural Analysis of an ESP32-S3-Based Deep Sleep Outdoor Camera

Ming — Mon, 03 Nov 2025 15:16:38 +0000

In Internet of Things (IoT) applications, particularly in outdoor or edge deployment scenarios (such as smart agriculture, wildlife monitoring, and infrastructure inspection), providing a stable power supply for monitoring devices remains a core challenge. Traditional network cameras, which typically operate 24/7, have high power consumption and rely on external power sources, making deployment difficult in many situations.

This article will share the technical decisions and architectural design behind our development of the CamThink NeoEyes NE101. We will focus on how to leverage the features of the ESP32-S3 to implement an "on-demand" ultra-low-power camera, solving the battery life challenges inherent in battery-powered scenarios.

The Core Challenge: The "Impossible Triangle" of Power Consumption When developing a battery-powered outdoor camera, we face several key technical trade-offs:

Standby Power vs. Response Speed: The system needs to remain in an extremely low-power "deep sleep" state for long periods but must also respond and wake up quickly to external events (like PIR sensors or timers). This efficiency is, of course, related to chip performance.

Image Processing vs. Power Consumption: Image acquisition, Auto Exposure (AE) adjustment, encoding, and processing are compute-intensive tasks that cause instantaneous power spikes.

Data Transmission vs. Power Consumption: Wireless communication modules like Wi-Fi and Cat-1 (4G) are notorious "power hogs" when searching for networks and transmitting data.

Traditional solutions are either "always-on," leading to high power draw, or "completely dormant," resulting in slow wake-up times and missed events. Our goal was to achieve an "on-demand" working model.

Technical Selection: Why the ESP32-S3? We selected the ESP32-S3 as the main controller (MCU) primarily for its unique dual-core heterogeneous architecture:

High-Performance Main Cores: Two Xtensa® LX7 dual-core processors at up to 240MHz.

Ultra-Low-Power (ULP) Coprocessor: A RISC-V-based coprocessor.

This architecture is the technical cornerstone for achieving ultra-low power consumption. Our design philosophy is "task separation":

LX7 High-Performance Cores: Are only awakened when needed. They handle the "heavy lifting," including:

Data acquisition and processing from the image sensor (CPI/USB).

Auto Exposure (AE) and image parameter adjustments.

Running the network stack (Wi-Fi, TCP/IP, MQTT).

Executing complex logic like data uploads and OTA updates.

RISC-V ULP Coprocessor: Remains "on duty" 24/7. It continues to run while the main cores are in deep sleep, handling the "light work," such as:

Monitoring external GPIO triggers (e.g., connected PIR sensors, sound sensors, or buttons).

Managing the RTC (Real-Time Clock) for timed wake-ups.

Deciding whether to wake the main cores based on specific conditions.

Architectural Design: An Event-Driven Operational Flow Our firmware is not a "dumb" loop but a strict, event-driven state machine.

State 1: Deep Sleep This is the state the system is in 99.9% of the time. In this state, the LX7 main cores, most peripherals, and the Flash memory are powered down. Only the ULP coprocessor and the RTC are running on an extremely low current (around 22µA, according to our test data), awaiting a trigger signal.

State 2: Trigger & Wake-up When the ULP detects a predefined trigger condition (e.g., a level change on the PIR interface or an RTC timer expiration), it immediately sends a signal to the power management unit (PMU) to wake up the LX7 main cores.

State 3: Active Cycle This is the highest power consumption phase, and our core optimization goal is to minimize its duration.

System Initialization & Sensor Power-On: The main core wakes from Deep Sleep and boots up quickly.

Image Acquisition & AE Convergence: This is an often-overlooked time-consuming point. To capture clear, properly exposed photos in variable lighting conditions (dawn, dusk, high noon), the system requires a few seconds (the NE101 firmware defaults to 5 seconds) for Auto Exposure (AE) to converge. (Note: Developers can adjust this time based on scene requirements, balancing power consumption and image quality).

Capture & Processing: A single snapshot is taken.

Communications Module Power-On & Connection: Based on the configuration (Wi-Fi / Cat-1 / Wi-Fi HaLow), the communication module is activated and connects to the network.

Data Transmission: The image data is uploaded to the cloud via MQTT or another protocol.

Enter Sleep: Once the task is complete, the system immediately executes a command to clear its state and return to State 1 (Deep Sleep).

Based on new test data, the entire "Active Cycle" takes approximately 30 seconds for the Cat-1 version, depending on network conditions, while other versions can be controlled within 20 seconds.

Key Technical Points: Hardware Modularization and Interface Design To adapt this low-power platform to different developer needs, we adopted a modular and open-interface design for the hardware.

4.1. Pluggable Communication Solutions
The mainboard does not have the communication module soldered directly. Instead, it reserves standardized interfaces, allowing developers to choose based on the deployment scenario:

Wi-Fi: For near-field scenarios with existing Wi-Fi coverage.

LTE Cat-1: For wide-area scenarios without Wi-Fi but with cellular network access.

Wi-Fi HaLow (IEEE 802.11ah): For long-distance (up to 1km), low-power private network scenarios, requiring a gateway for connectivity.

4.2. Flexible Sensor and Camera Interfaces
Camera Support: The mainboard supports both CPI and USB interface cameras.

CPI (OV5640): Lower power consumption, simple interface, suitable for regular snapshots.

USB (SC200AI): Higher bandwidth, supports 1080P, but occupies additional I/O resources and the USB host function.

Trigger Source Expansion: We have opened up multiple hardware interfaces to be used as "triggers":

PIR Interface: A 4-pin Wafer connector dedicated to PIR sensor integration.

16-pin IO Interface: Reserves GPIO, DI, DO, etc., allowing developers to connect their own sensors (e.g., radar, sound, vibration, temperature/humidity) as wake-up sources.

Alarm Interface: A 2-pin Wafer connector for alarm input. By default, the PIR interface is used; this interface requires soldering an additional resistor for use.

Power Consumption Data and Battery Life Analysis The theoretical architecture must ultimately be validated by data. In a typical use scenario of 5 snapshots per day (with the fill light off), the NE101's battery life performance (based on test data) is:

Wi-Fi Version: Can support up to 2.73 years.

Wi-Fi HaLow Version: Up to 2.59 years.

Cat-1 Version: Can operate stably for 1.24 years.

This longevity is primarily due to the extreme optimization of both sleep and operating power.

5.1. Sleep Power
This is the power consumed when the device is not capturing images (deep sleep) and is the foundation for long battery life.

Technical Analysis:

The base sleep power consumption is nearly identical across all three communication modes, with annual consumption at around 8%.

This implies that in a pure standby (theoretical) scenario, the battery life could exceed 12 years (2500mAh / 195.79mAh ≈ 12.8 years).

This demonstrates the effectiveness of the ESP32-S3's ULP coprocessor architecture in achieving ultra-low base power consumption.

5.2. Operating Power
This refers to the total power required for the device to wake up, acquire an image, and complete the upload. This is the key factor determining the battery life differences between communication solutions.

Key Operating Parameters Test Data:

Technical Analysis:

Sleep Power: Again validates that the base sleep power for all versions is extremely low, in the 22-23.5µA range.

Fill Light Impact: Turning the fill light on or off has a noticeable impact on single-event power consumption (an increase of ~10-15%). It should be avoided when pursuing extreme battery life.

Operating Time: Cat-1 takes the longest "active cycle" (approx. 30 seconds) due to the need to search for and register on the cellular network, while Wi-Fi and HaLow are much faster (<20 seconds).

Cat-1 Power: The Cat-1 module's average current during operation (76.1mA) and its longer operating time (~30s) are both significantly higher than the Wi-Fi solutions. This is the primary reason its total battery life (1.24 years) is lower than the Wi-Fi version (2.73 years).

5.3. Power Estimation Model
Based on the data above, we can derive the technical model for battery life estimation:

Total Daily Power Consumption = (Base Sleep Power × 24h) + (Single Event Operating Power × Number of Daily Events)

It is precisely because the base sleep power (~22µA) has been pushed to an extremely low level that the first half of the equation (sleep consumption) becomes minimal. The system's total power consumption is therefore primarily determined by the second half (number of operating events). This is what enables us to achieve battery life measured in "years" in low-frequency scenarios like "5 snapshots per day."

Conclusion

Implementing a deep-sleep outdoor camera based on the ESP32-S3 is entirely feasible. The technical core lies in fully leveraging its heterogeneous architecture (LX7 high-performance cores + RISC-V ULP coprocessor).

By designing the system in an "event-driven" mode, allowing the main core to be in Deep Sleep (at ~22µA) 99.9% of the time, with only the ULP coprocessor handling low-power "sentry" tasks, we fundamentally solve the power bottleneck of traditional IPCs. The latest real-world test data strongly supports this, providing developers with a clear basis for making battery life trade-offs between different communication solutions (Wi-Fi, Cat-1, HaLow).

Known Drawbacks of Using the ESP32-S3 The ESP32-S3's image processing performance is relatively weak, making it difficult to support the processing of large images. Simultaneously, this lack of image processing power can lead to a longer time from wake-up to handling the image sensor, resulting in low real-time performance for image capture. However, for applications that do not require high-frequency image acquisition, this product architecture remains highly practical and balanced.

Firmware code：https://github.com/camthink-ai/lowpower_camera
Hardware Info：https://wiki.camthink.ai/docs/neoeyes-ne101-series/ne100-mb01-development-board/dev-guide

We are looking for developers in the community to work with us to explore low-power camera products. We look forward to your participation in the community construction. If you have any intention, you are welcome to join us!!

We have prepared a gift for the developers.
https://near.tl/sm/uCT0JnK9e