DEV Community: Ming

Hardening an Edge AI Platform: What a Real Security Release Looks Like (NeoMind v0.9.21)

Ming — Wed, 02 Sep 2026 11:10:22 +0000

Edge devices are a brutal attack surface. They bind to the LAN with nobody babysitting them, they hold credentials for cloud APIs, they run third-party extension code, and "patch it Tuesday" usually means someone drives to the site. When you ship an edge platform, the security work is not a blog post — it's a pile of specific, unglamorous fixes.

NeoMind is an open-source, Rust-powered edge AI platform for IoT automation (single binary: HTTP API + embedded MQTT broker + rule engine + AI agent runtime). Its just-shipped v0.9.21 is largely a security and data-safety release, and I think it's worth walking through in public — both what was found and how it was fixed. Everything below reflects the repository at commit 8a75c57ee09b and the official release notes.

1. Close the front door: self-registration was open

POST /api/auth/register was a public, unconditionally-open account-creation endpoint — on a server that binds 0.0.0.0. Any client on the LAN could mint a user account.

The fix in 0.9.21: registration now returns 403 unless an admin explicitly opens it (admin-only setting, persisted so the choice survives restarts). Nothing regresses for legitimate flows — the first admin comes from the setup wizard and additional users from the admin-only user-management endpoint. The open endpoint had no honest product caller; it was just a hole.

Related, from 0.9.20 and still the backbone: the public auth endpoints are brute-force throttled — login counts credential failures at 5 per 15 minutes keyed per username AND per client IP (either dimension over the cap blocks, stopping both targeted guessing and account-spraying; a successful login clears the counters), and registration/first-run setup count every attempt per IP.

2. Secrets at rest: the asymmetry fix

The platform's own API keys were encrypted with AES-256-GCM, but LLM provider API keys were stored as plaintext JSON in settings.redb. A copied data directory leaked every cloud key while looking superficially protected.

0.9.21 seals both stores with the shared crypto service (same data/encryption_key the auth store uses; an env override still wins). Legacy plaintext rows load unchanged and get sealed on next save — upgrades are transparent. Config-change history previously duplicated the plaintext key on every tracked save; it now records the sealed form.

3. The path-traversal family

If you take one lesson from this release, take this one: every path join is a vulnerability candidate. 0.9.20 + 0.9.21 together close four of them:

Share proxy dot-segments: /share/{token}/proxy/devices/../../auth/keys passed the first-segment allowlist and the loopback forwarder normalized the path away — an anonymous share viewer reached any authenticated route via the internal-proxy header. Dot-segments are rejected outright.
Absolute-path asset join: the extension asset server joined an attacker-controlled path onto the extension dir; an absolute path (/etc/passwd) replaced the base with no .. needed. Absolute paths are rejected and a canonicalized containment check backstops every join.
file_path request fields: extension register/upload/validate accepted any host path, making them a read-and-try-load primitive for anyone holding credentials. Paths now resolve against (and must stay inside) the data directory, after canonicalization.
Marketplace ID interpolation: the raw extension ID was interpolated into market URLs — ../.. turned the marketplace client into a limited arbitrary-GET against the market host. IDs are validated before URL building.

4. Supply chain: sandbox escapes and unverified packages

NeoMind extensions run in sandboxed processes; WASM tooling sits underneath. The 0.9.21 release-blocker repairs bump wasmtime 36.0.13 → 36.0.14 for RUSTSEC-2026-0269 — a HIGH severity filesystem sandbox escape when paths or symlinks contain trailing slashes, published 2026-08-31. It started failing cargo audit the day the advisory landed, and the lock update shipped in the same release. (We verified the advisory against the RustSec database; if you run wasmtime anywhere, check your lockfile.)

Two more supply-chain fixes:

Package integrity is now actually checked. The marketplace index carried no sha256, so the fail-closed verification branch never fired. The installer now falls back to release-level checksums.txt, warns loudly when no integrity data exists, and NEOMIND_STRICT_PACKAGE_SHA256=1 refuses unverified packages outright.
Zip-bomb defenses unified. The async package-extraction path had no size/count caps while the sync installer did — same crate, two implementations, asymmetric defenses. Both now share one set of caps and explicitly reject symlink entries. (A follow-up found a third extraction path had slipped the caps during unification — it got them too.)

5. Credential lifecycle

Deleting a user or changing a password revokes sessions immediately — both the in-memory whitelist and every persisted row. Previously a JWT minted earlier kept working for up to 7 days: a leaked token survived password rotation.
Remote-instance API keys are no longer handed back by the API. The instance list previously returned every instance's full key XOR-"encrypted" with a cipher hardcoded in the open-source repo — anyone who could list instances recovered every credential. Now only masked keys leave the server.
Interactive share links no longer mean "full write". An allow_interactive share token previously skipped the method check — an anonymous holder could POST/PUT/DELETE under the proxied prefixes (install extensions, delete agents). Both share modes now pass the same method gate; interactive adds exactly one write — device command actuation. Configuration editing stays blocked.

6. Upgrades across a privilege boundary (without sudoers)

The 0.9.21 headline for ops: the About page can upgrade a server deployment — check for updates, download with live progress, apply, reload. The security-relevant part is how the privilege boundary is handled:

The API runs as the sandboxed neomind user (ProtectSystem=full + NoNewPrivileges=true). It cannot write /usr/local/bin and cannot sudo.
So the API only stages: stream the release into data/upgrade/v<ver>/ (2GB cap, binary --version-verified before anything is touched), then write apply.trigger.
A root systemd .path unit watches that file with inotify and runs the apply step: back up → atomic install -m 755 swap → web-dir swap → restart. No sudoers rule, nothing relaxed in the main unit's sandbox.
The upgrade endpoints sit in the JWT-gated admin route group — API keys cannot trigger an upgrade.

7. Data safety is security too

Two additions that belong in any edge threat model:

Verified backups. neomind backup copies every database plus the two secret files (encryption_key — without it the sealed keys in the backup are undecryptable — and .jwt_secret) into a timestamped directory (0700; secrets 0600), verifies each copied database opens, writes a manifest, and only then renames the staging dir into place. A crashed backup never masquerades as a restorable one. A scheduler runs it on a configurable interval (6h–7d) with retention.

Rollback guard. The realistic corruption risk on edge boxes isn't old data meeting new code — it's the reverse: a rolled-back install opening newer data, silently dropping unknown fields, then destroying newer-format rows on the first save-back. Every storage database is now version-stamped, and a store refuses to open a database stamped by a newer build, with an explicit "upgrade instead of rolling back" error. About 100 lines; no framework needed.

8. Observe it or it didn't happen

A new public Prometheus endpoint (/api/metrics) exposes request/response counters, uptime, build info, and — my favorite — neomind_eventbus_dropped_total. The event bus's lagged-subscriber warning log has literally said "surface this, don't let the system fail quietly" since it was written. Now drops are measurable: non-zero and growing means an automation subscriber is missing events.

Checklist for anyone shipping edge software

Audit your registration and account-creation endpoints — open-by-default on 0.0.0.0 is commoner than you think.
Encrypt every secret at rest, then check for the asymmetry: one plaintext field next to ten encrypted ones.
Treat every path join as hostile: reject dot-segments and absolute paths, canonicalize, containment-check.
Verify third-party packages (checksums), and keep cargo audit (or equivalent) red-blocking your CI.
Revoke sessions on credential events, and never return secrets from list endpoints.
Design upgrades around privilege boundaries, not sudoers.
Back up with verification; guard against rollback as a data-integrity attack.

Full details are in the release notes, and the code is open — review it, run it, file issues: github.com/camthink-ai/NeoMind. The project wiki covers the platform itself.

All facts reflect the NeoMind repository at commit 8a75c57ee09b (release v0.9.21) and its official release notes; RUSTSEC-2026-0269 verified against the RustSec Advisory Database.

Making LLM Agents Reliable on Edge Hardware: Lessons from Shipping NeoMind 0.9.20

Ming — Tue, 01 Sep 2026 11:12:06 +0000

Everyone has a demo where the LLM agent works. Few people have an agent that keeps working when the MQTT broker times out mid-tool-call, when the local model's real context window is half what the registry claims, and when nobody is watching a dashboard in a server room because the "server" is a box in a factory cabinet.

NeoMind is an open-source, Rust-powered edge AI platform for IoT automation — single binary, embedded MQTT broker, rule engine, and an AI agent runtime that drives real devices. We just shipped v0.9.20, and the release's spine is agent execution reliability. This post is the engineering story: the failure modes we found in production-shaped testing, and the fixes that now sit in the loop.

Everything below reflects the repository at commit 8a75c57ee09b.

Failure mode 1: the dedup brake that punished retries

The agent loop runs up to 30 rounds of tool-calling. To stop infinite loops, we keep a dedup set of execution signatures — if the model proposes a call it already made, we end the loop.

The bug: signatures were inserted before execution. So when a tool call failed transiently — an MQTT timeout, an extension hiccup — and the model correctly retried, the retry looked like a "duplicate". The loop ended via AllDuplicate, with the original error still in hand and the task unfinished. The anti-loop mechanism was punishing the model for doing the right thing.

The fix, in the 0.9.20 release notes:

The cross-round dedup set now records only successful executions. Failed calls can retry — and a failed signature that keeps failing (budget: 3 consecutive failures) is blacklisted so the loop brakes instead of burning all 30 rounds.

We also deleted our StuckDetector entirely. It implemented five OpenHands-style stuck patterns — but they were mathematically unreachable behind the dedup logic. The docs described a brake that never fired; the dedup IS the brake. Sometimes reliability work is subtraction.

Failure mode 2: "context overflow" that was really a config mismatch

On local backends, context overflow is marked permanent per is_permanent(). But we found hosts where the running model's window was smaller than the registry default — e.g. a backend started with -c 16384 while the model entry claimed 128K. Result: every round overflowed, and small-model execution "inevitably failed".

The fix is a self-heal: on overflow, one hard-compaction retry with a halved effective window. That single retry converts "can never run" into "completes". Alongside it, custom backends now carry an explicit max_context on create/update, so a 16K backend never receives 128K-budgeted prompts, and the catalog caps imported models at 128K because a GGUF header claiming 1M would OOM the KV allocation.

Failure mode 3: the agent didn't know it was out of runway

Within the last 3 rounds, the loop now injects a [System] note telling the model how much runway is left. It sounds trivial; the effect is not. Without it, the model would start a multi-step chain at round 28 that the cap cuts off mid-flight — a half-finished task with no summary. With the countdown, it wraps up.

Related shutdown fixes: cancellation now exits the whole round loop (previously it only broke the tool-concurrency batch), and Phase-2 summarization is skipped during shutdown — no LLM calls while the process is going down.

Failure mode 4: the metrics lied

Our agent journal recorded a success_rate for every execution. It was pinned at 1.0, always. Reason: tool results returning Ok { success: false } were counted as successes — the transport succeeded, so the outcome flag was ignored.

Now the journal reflects real outcomes. This matters beyond dashboards: the journal is the learning signal the agent system trains and steers on. Honest metrics are a prerequisite for any self-improvement story.

Similarly, invoke used to await execution inline with a 60s timeout — which dropped the future. Long runs died mid-flight with no execution record and no journal entry: a ghost execution the agent could never learn from. The execution now runs in its own task; past the wait window the caller gets still_executing with poll pointers while the run completes and writes its record.

Failure mode 5: one bad schedule killed the whole scheduler

Found by our own design review: the frontend's "on-demand" option encodes manual-only as interval_seconds: 0. That value made the agent due on the very next tick, which hit a division by zero in update_next_execution — a panic inside the unguarded scheduler loop that stopped every agent on the platform. Process alive, no restart, one panic line in the logs.

Three-layer fix: 0 is now a first-class "manual-only" value (schedules to a never-due time), reschedule math guards against 0 for legacy rows, and the tick's reservation phase runs in its own spawned task so any future panic there is logged and skipped instead of unwinding the loop. We also promoted one-shot agents to a real type — ScheduleType::Manual with a Completed ready-state between runs — so "on-demand" no longer needs a numeric hack.

Per-model sampling: stop sharing one global temperature

The scheduled-agent loop hardcoded temperature 0.7 for everything. But 2026's small-model zoo does not share one optimal sampling point. 0.9.20 wires /api/settings/agent into both chat and scheduled paths, and each built-in model now carries its own best-known point, applied at llama-server startup and on the request side:

Qwen 3.5 (non-thinking): 0.7 / 0.8 / 20 — official values; the 4B tier (~3.4 GB quant, ~6 GB RAM, Apache 2.0) is currently one of the strongest laptop-class agent models
Gemma 4: 1.0 / 0.95 / 64 — Google's model card recommends temperature 1.0 for all modes; we were running 0.6
Ling-3.0-tiny: 1.0 / 0.95 / 20 — official
LFM: 0.6 / 0.85 / 20 — our measured best, which beat the official card values in a 154-case A/B

The catalog schema carries temperature/top_p/top_k; absent means legacy default. Registry tests lock all four values so a future "let's just tweak this" doesn't silently regress them.

The payoff: which small model actually drives agents?

We validate agent capability on a 30-case agent suite (tool selection, multi-step device control, recovery from failed calls). The 0.9.20 headline:

Ling-3.0-tiny Q4_K_M: 77% — ties Qwen 3.5 4B, while generating ~45% faster (~110–116 tok/s on M4-class hardware).

Ling-3.0-tiny ships through the open model catalog (4.8 GB Q4_K_M, 128K ctx, min 6 GB RAM), and our bundled runtime already carries the architecture support merged upstream, so the download runs out of the box. Tying the quality of a proven 4B agent model at roughly 1.5x the speed is exactly the trade an edge deployment wants.

What we'd tell anyone building agent loops

Dedup on outcomes, not intentions. Recording a signature before knowing if it succeeded turns transient errors into permanent dead-ends.
Make failure modes diagnosable. "Sorry, the model could not produce a response" now carries the actual reason instead of a bare "Please retry". Your future self debugging at 2am is the customer.
Budget the loop and tell the model about it. A countdown in the last rounds changes behavior more than any prompt tweak.
Measure what actually happened. If your success rate is 1.0, your metric is broken, not your agent.
Guard the scheduler from its own inputs. One malformed row should degrade one agent, not the platform.

If you're building or evaluating agent systems that must run where there is no cloud fallback, come look at the code or file an issue: github.com/camthink-ai/NeoMind. The project wiki covers the platform itself.

Facts in this article reflect the NeoMind repository at commit 8a75c57ee09b (release v0.9.20, 2026-08-26) and its official release notes; third-party model specs (Qwen 3.5 family, Gemma 4 sampling guidance) are from their respective 2026 publications.

From Prototype to Production: Deploying NeoMind on Edge Hardware

Ming — Thu, 20 Aug 2026 11:12:28 +0000

Getting an edge AI demo to run on your desk is the easy part. Keeping it alive on a $100 gateway in a factory cabinet, surviving power cuts, OTA updates, and the occasional kill -9 from a well-meaning technician — that's the job.

NeoMind is an open-source, Rust-powered edge AI platform for IoT automation: a single binary that bundles an HTTP API (Axum), an embedded MQTT broker, a rule engine, an AI agent runtime, and embedded storage (redb). One process, two ports, no external database. That design choice pays off exactly at the deployment stage.

This walkthrough covers the two production deployment paths available in the current release (v0.9.18, commit c0306de44d11), plus the operational details that separate a demo from a deployment: restarts, hardening, health checks, updates, and local LLM integration.

Two paths, one binary

NeoMind ships pre-built release binaries for linux/amd64, linux/arm64, and darwin/arm64, so you have two sane options:

Bare metal with systemd — smallest footprint, best for constrained gateways (2-4 GB RAM devices).
Docker / Docker Compose — multi-arch images on Docker Hub (camthink/neomind:latest), best when you already run containers or want the bundled local LLM stack.

Both paths expose the same surface: port 9375 for the HTTP API + Web UI + WebSocket, and port 1883 for the embedded MQTT broker that your devices connect to.

Path 1: one-line install + systemd

The repo ships an install script (scripts/install.sh) that detects your OS and architecture, resolves the latest release, and drops everything in place:

curl -fsSL https://raw.githubusercontent.com/camthink-ai/NeoMind/main/scripts/install.sh | sudo bash

What it does under the hood (and the knobs you can set as environment variables):

INSTALL_DIR=/usr/local/bin — the neomind binary
DATA_DIR=/var/lib/neomind — all persistent state (redb database, rules, config)
WEB_DIR=/var/www/neomind — the built-in web UI static files
PORT=9375, NO_WEB=1, NO_SERVICE=1, USE_NGINX=1 for non-standard setups

The script also installs a systemd unit (scripts/neomind.service) that is worth reading because it encodes a solid production baseline:

[Service]
Type=simple
User=neomind
WorkingDirectory=/var/lib/neomind
ExecStart=/usr/local/bin/neomind
Restart=always
RestartSec=5
Environment="RUST_LOG=info"
Environment="NEOMIND_DATA_DIR=/var/lib/neomind"
Environment="NEOMIND_BIND_ADDR=0.0.0.0:9375"
NoNewPrivileges=true
ProtectSystem=strict
MemoryMax=2G

Three details here are doing real work:

Restart=always + RestartSec=5 handles the reality of edge power and networking. The box will brown-out; the service comes back without anyone driving to the site.
ProtectSystem=strict plus a dedicated neomind user means the process can only write to explicitly allowed paths. If anything ever escapes into the process, the blast radius is tiny.
MemoryMax=2G is a cgroup hard cap. On a shared gateway, NeoMind cannot eat the memory that your other services need.

Check it with the usual tools:

sudo systemctl status neomind
sudo journalctl -u neomind -f

Path 2: Docker Compose (with optional local LLM)

If your fleet already runs containers, the compose path is two commands:

git clone https://github.com/camthink-ai/NeoMind.git
cd NeoMind && docker compose up -d

The docker-compose.yml pulls camthink/neomind:latest — a multi-arch image (amd64 + arm64), so the same compose file works on an x86 gateway and a Raspberry Pi 5 or Jetson-class board:

services:
  neomind:
    image: camthink/neomind:latest
    restart: unless-stopped
    ports:
      - "9375:9375"   # HTTP API + Web UI + WebSocket
      - "1883:1883"   # MQTT broker
    volumes:
      - neomind-data:/app/data

Two implementation details from the project's Dockerfile that I think are worth knowing, because they answer questions you'd otherwise hit the hard way:

Why Ubuntu 22.04 and not Alpine? NeoMind extensions ship native binaries built against glibc. A musl-based Alpine image cannot dlopen a glibc-linked shared library, so the extension marketplace would be unusable. The runtime stage pins glibc 2.35 — identical to the bare-metal release baseline — so Docker and bare metal load the exact same extension binaries.

Why does arm64 need a jemalloc flag? The build sets JEMALLOC_SYS_WITH_LG_PAGE=16 for ARM targets because hosts like Raspberry Pi 5 and Jetson can run 64 KB memory pages; without the flag the allocator crashes. If you build your own ARM container images for edge workloads, this is a classic trap.

The image also ships a built-in health check:

HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3 \
    CMD curl -f http://localhost:9375/api/health || exit 1

So docker ps and any container orchestrator can tell you whether the node is actually serving, not merely running.

Local AI without extra plumbing

Here's where it gets interesting for the AI-at-the-edge crowd. The repo includes a docker-compose.override.yml that is auto-loaded by docker compose and adds a complete local LLM stack:

llama-init: a one-shot container that downloads a Gemma 4 E2B GGUF (q4_0, ~4.3 GB, resumable) into a named volume
llama: a llama.cpp server with a 16K context window on port 8080
neomind: waits for the model server to become healthy, then auto-registers it as an AI backend

That's right: docker compose up -d gives you an IoT platform with a fully local, no-API-key LLM attached. On constrained hardware you'll want a small model — Gemma's E2B variant is built exactly for edge devices, where 7B+ models typically crawl at 1-2 tokens/second on CPU-only boards. First boot downloads the model; every restart after that reuses the volume.

To disable local AI, delete or rename the override file — the core platform runs fine without it.

The operations checklist

A few things that make day-2 operations painless:

Reverse proxy / TLS. Put nginx or Caddy in front of port 9375 for TLS. The install script has USE_NGINX=1 for this. Keep port 1883 (MQTT) firewalled to your device VLAN — device traffic should never traverse your public ingress.

Updates. Bare metal: re-run the install script (it resolves the latest release automatically) and sudo systemctl restart neomind. Docker: docker compose pull && docker compose up -d — the compose file sets pull_policy: always. Because all state lives in one place (/var/lib/neomind bare metal, neomind-data volume in Docker), updates are just "replace binary/image, keep data directory". Back up that one directory and you have everything.

Monitoring. The /api/health endpoint is your liveness probe; journalctl -u neomind (or docker compose logs -f neomind) is your log stream. MemoryMax=2G in the systemd unit keeps resource surprises out of your dashboards.

Secrets. In Docker, set NEOMIND_JWT_SECRET for stable auth across restarts and NEOMIND_ENCRYPTION_KEY for persistent data encryption — both documented in the compose file. On bare metal, drop them into an override env file rather than editing the unit.

Single binary, small ops team

The recurring theme in NeoMind's deployment story is fewer moving parts. No external broker, no external database, no separate web server. One process with a hard memory cap, one data directory to back up, two ports to firewall. When your "site" is a cabinet in a building you visit twice a year, boring is a feature.

If you run edge gateways — or you're evaluating what an on-prem AI + IoT stack looks like without cloud lock-in — give it a spin and tell the maintainers what breaks: github.com/camthink-ai/NeoMind.

Deployment details in this article reflect the NeoMind repository at commit c0306de44d11 (release v0.9.18).

Edge AI vs Cloud AI: What Running LLMs Locally Actually Costs in 2026

Ming — Wed, 19 Aug 2026 11:14:17 +0000

Edge AI vs Cloud AI: What Running LLMs Locally Actually Costs in 2026

(Based on NeoMind v0.9.18, commit c0306de44d11)

The open-weight LLM landscape has transformed in 2026. Models that once required datacenter GPUs now run on a Raspberry Pi. Qwen3 8B leads the 8B tier with dual-mode thinking and 128K context. Google's Gemma 4 12B brings multimodal understanding to a single consumer GPU. Microsoft's Phi-4-mini fits world-class reasoning into 3.8 billion parameters and 3GB of RAM.

For IoT deployments, this means a fundamental question has a new answer: do you still need the cloud?

This article breaks down the real economics of edge AI vs cloud AI for IoT in 2026, using NeoMind (an open-source Rust-based edge AI platform) as the reference implementation. We'll compare hardware costs, API bills, latency, privacy implications, and total cost of ownership across three deployment scales.

The 2026 Local LLM Landscape

Before comparing costs, here's what's actually available to run on your hardware today:

Model	Parameters	RAM (Q4)	Context	License	Best For
Phi-4-mini	3.8B	~3 GB	128K	MIT	Reasoning, math, constrained hardware
Qwen3	8B	~6-7 GB	128K	Apache 2.0	Multilingual, coding, daily driver
Gemma 4	12B	~7 GB	128K	Apache 2.0	Multimodal, general purpose
Qwen3.6	27B	~16 GB	262K	Apache 2.0	Coding (77.2 SWE-bench), complex analysis
Gemma 4	31B	~19 GB	128K	Apache 2.0	High-quality general + vision
Mistral Small 4	119B MoE	~71 GB	128K	Apache 2.0	Server-class, 22B active params

For edge IoT, the sweet spot is 3B–14B parameters: enough intelligence for device monitoring, anomaly detection, and natural language control, while fitting on hardware that costs $200–$500.

NeoMind v0.9.18 supports all of these through its llama.cpp backend (recommended for edge) plus Ollama, OpenAI, Anthropic, Google, DeepSeek, Qwen, GLM, MiniMax, and any OpenAI-compatible endpoint.

Scenario 1: Smart Building (50 Devices)

Setup: 50 sensors (temperature, humidity, occupancy, CO2), 10 smart switches, 5 cameras. One building, one edge server.

Cloud AI Approach

Cost Component	Monthly	Annual
Cloud LLM API (GPT-4o-mini at ~10K queries/day)	$45	$540
Cloud hosting (data ingestion + processing)	$80	$960
Bandwidth (sensor data upload, ~50GB/month)	$15	$180
Data storage (time-series, growing)	$20	$240
Total	$160	$1,920

Latency: 200-800ms per AI query (round-trip to cloud API).

Privacy: All sensor data transmitted to and stored on third-party servers.

Edge AI Approach (NeoMind + Local LLM)

Cost Component	One-Time	Annual
Hardware (Intel NUC i5, 16GB RAM)	$350	—
NeoMind software	$0 (Apache 2.0)	$0
Qwen3 8B model	$0 (Apache 2.0)	$0
Electricity (~15W continuous)	—	$13
Maintenance (estimated)	—	$50
Year 1 Total		$413
Year 2+ Total		$63/year

Latency: 15-50ms per AI query (local inference).

Privacy: All data stays on-premises. Zero external transmission.

Break-even: 3 months. After the initial hardware investment, edge AI costs 97% less annually.

Scenario 2: Factory Floor (500 Devices)

Setup: 500 sensors across 5 production lines, 50 actuators, 20 cameras with vision AI. Needs YOLO object detection + LLM-powered analysis.

Cloud AI Approach

Cost Component	Monthly	Annual
Cloud LLM API (50K queries/day, mixed models)	$380	$4,560
Vision API (camera analysis, ~100K images/month)	$200	$2,400
Cloud infrastructure (streaming + processing)	$300	$3,600
Bandwidth (industrial telemetry, ~500GB/month)	$100	$1,200
Total	$980	$11,760

Edge AI Approach (NeoMind + Local Models)

Cost Component	One-Time	Annual
Hardware (2× workstation, RTX 4090 each)	$6,000	—
NeoMind + Extensions	$0	$0
Qwen3.6 27B (analysis) + Phi-4-mini (monitoring)	$0	$0
YOLO Video extension (camera analysis)	$0	$0
Electricity (~400W combined)	—	$350
Year 1 Total		$6,350
Year 2+ Total		$350/year

Break-even: 7 months. And you get sub-50ms inference for safety-critical responses.

The factory scenario is where edge AI's advantage becomes decisive. Vision processing alone — 100K images/month through a cloud API — costs $2,400/year. With NeoMind's YOLO Video extension running locally on the RTX 4090, that cost drops to zero.

Scenario 3: Smart Campus (5000+ Devices, Multi-Site)

Setup: 5 buildings, 5000+ sensors, 200 cameras, industrial HVAC and access control. Multiple NeoMind instances (one per building).

Cloud AI Approach

Cost Component	Monthly	Annual
Cloud LLM API (500K queries/day)	$3,500	$42,000
Vision + analytics	$2,000	$24,000
Cloud infrastructure	$1,500	$18,000
Bandwidth (5TB/month)	$500	$6,000
Total	$7,500	$90,000

Edge AI Approach (NeoMind Multi-Instance)

Cost Component	One-Time	Annual
Hardware (5× workstations + networking)	$25,000	—
NeoMind (5 instances)	$0	$0
Electricity (~2kW combined)	—	$1,750
IT maintenance (estimated)	—	$5,000
Year 1 Total		$31,750
Year 2+ Total		$6,750/year

Break-even: 5 months. Annual savings of $83,250 from Year 2 onward.

The Hidden Costs of Cloud AI

The raw numbers above don't capture several costs that materialize over time:

1. Data Egress Fees
Cloud providers charge $0.05–$0.12 per GB for data leaving their platform. When you need to export historical telemetry for analysis or compliance, a 5TB archive costs $250–$600 to retrieve.

2. API Rate Limiting
At scale, cloud LLM APIs enforce rate limits. A factory running 50 concurrent device queries might hit 429 errors during peak hours, requiring retry logic and degraded user experience.

3. Compliance Overhead
GDPR, HIPAA, and industry-specific regulations often require data residency guarantees. Cloud deployments need additional legal review, data processing agreements, and potentially regional cloud instances — adding $10K–$50K in compliance costs.

4. Vendor Lock-in
Cloud AI APIs change pricing, deprecate models, and shift terms. When OpenAI retired GPT-3.5 in favor of GPT-4o-mini, thousands of deployments needed migration. With local models, you own the weights — no one can sunset your model.

5. Latency Tax on Automation
Cloud-based AI adds 200-800ms per inference. For a rule that chains 3 AI decisions (detect anomaly → classify severity → determine action), that's 600-2400ms of pure inference latency. On a factory floor where a conveyor belt moves at 2m/s, that delay translates to 1.2-4.8 meters of unmonitored production.

Where Cloud AI Still Wins

Edge AI isn't universally better. Cloud retains advantages in specific scenarios:

Frontier model access: GPT-5.4, Claude Sonnet, and other proprietary models offer capabilities that no open-weight model matches yet (though the gap narrows monthly)
Burst capacity: Seasonal spikes or one-off analysis tasks that would require over-provisioning local hardware
Training and fine-tuning: Large-scale model training still requires datacenter GPUs
Multi-language at scale: If you need 50+ languages at high quality, proprietary models still lead

NeoMind's architecture handles this gracefully: use local models for 95% of operations, and route complex or burst queries to cloud APIs when needed. The cloud-optional design means you're never locked into either approach.

Hardware Guide: What to Buy in 2026

Budget	Hardware	Model	Capability
$200	Raspberry Pi 5 (8GB)	Phi-4-mini (3.8B)	Basic monitoring, alerts, simple Q&A
$400	Intel NUC i5 (16GB)	Qwen3 8B	Full device management, automation, natural language
$800	Mini PC + RTX 4060 (24GB)	Qwen3.6 27B	Complex analysis, multi-camera vision, coding tasks
$2,500	Workstation + RTX 4090 (24GB)	Gemma 4 31B	Production-grade, multi-model, high-throughput

NeoMind's resource footprint adds ~120MB idle RAM and <5% CPU overhead on top of the LLM runtime. The single-process architecture means no microservice overhead — one binary handles everything from MQTT to dashboards to AI inference.

The Decision Framework

Ask these questions to determine your optimal approach:

Is internet connectivity guaranteed? If no → edge AI is mandatory, not optional
What's the latency requirement? If <100ms → edge AI
Are there data residency requirements? If yes → edge AI (or regional cloud, at higher cost)
What's the query volume? If >10K/day → edge AI breaks even within months
Do you need frontier-model capabilities? If yes → hybrid (local for routine, cloud for complex)

For most IoT deployments in 2026, the answer is edge-first, cloud-optional — exactly the architecture NeoMind was designed for.

Get Started

NeoMind v0.9.18: github.com/camthink-ai/NeoMind (Apache 2.0)
Wiki & Docs: wiki.camthink.ai
5-Minute Quick Start: wiki.camthink.ai/docs/neomind/quick-start
Community: Discord

Stop paying per-token for data that never needed to leave your building. Run AI where your devices live.

Running LLM Agents at the Edge: A Practical Guide with NeoMind + Ollama

Ming — Wed, 19 Aug 2026 10:16:52 +0000

Running LLM Agents at the Edge: A Practical Guide with NeoMind + Ollama

Everyone's building AI agents right now. Most of them live in the cloud — you send a request to OpenAI or Anthropic, get a response back, and hope the latency and cost stay reasonable. But what if your agent needs to control physical devices? Monitor factory sensors? Respond to security camera events in under 100ms?

Cloud-based agents add 200-800ms of round-trip latency per inference call. For a conversational chatbot, that's fine. For an autonomous agent monitoring a production line, it's the difference between catching a defect and shipping it.

This guide walks through building production-grade LLM agents that run entirely on edge hardware using NeoMind + Ollama. No cloud API keys. No data leaving your network. Sub-100ms inference on a $200 device.

What You'll Build

By the end of this guide, you'll have:

A local LLM agent that monitors IoT device telemetry in real-time
Natural language interaction with your agent via chat
Autonomous decision-making based on sensor data patterns
Multi-tier memory so the agent learns your environment over time
Zero cloud dependency — everything runs on your LAN

Prerequisites

Hardware: Any x86_64 or ARM64 machine with ≥8GB RAM (Raspberry Pi 5, Intel NUC, old laptop all work)
OS: Linux (Ubuntu 22.04+), macOS, or Windows
GPU: Optional but recommended. Ollama runs on CPU for smaller models (7B), but 13B+ benefits from GPU acceleration
NeoMind: The edge AI platform (handles device connectivity, automation, UI)
Ollama: The local LLM runtime (handles model serving and inference)

Step 1: Install NeoMind + Ollama

Install Ollama

curl -fsSL https://ollama.com/install.sh | sh

Pull a model suited to your hardware:

Hardware	Recommended Model	VRAM/RAM Needed	Inference Speed
Raspberry Pi 5 (8GB)	`llama3.2:3b`	~4 GB	~8 tokens/sec
Intel NUC (16GB)	`llama3.1:8b`	~6 GB	~15 tokens/sec
GPU (RTX 3060+)	`llama3.1:70b` (Q4)	~12 GB VRAM	~40 tokens/sec
Apple Silicon (M1+)	`llama3.1:8b`	~8 GB unified	~25 tokens/sec

ollama pull llama3.1:8b

Install NeoMind

curl -fsSL https://raw.githubusercontent.com/camthink-ai/NeoMind/main/scripts/install.sh | sh

Configure NeoMind to Use Ollama

Edit NeoMind's config (typically ~/.config/neomind/config.toml):

[ai]
backend = "ollama"
model = "llama3.1:8b"
api_url = "http://localhost:11434"
max_tokens = 2048
temperature = 0.7

Start NeoMind:

neomind start

Open the web UI at http://localhost:9375 and you'll see the AI chat panel ready to go.

Step 2: Connect Your First Device

Before the agent can monitor anything, it needs devices to talk to. NeoMind supports MQTT, BLE, and Webhook protocols out of the box.

Simulate a Device (for Testing)

If you don't have physical IoT devices yet, use NeoMind's device simulator:

neomind device simulate --type temperature-sensor --interval 5s

This creates a virtual temperature sensor that publishes readings to NeoMind's embedded MQTT broker every 5 seconds.

Connect a Real Device

For real devices, configure MQTT in NeoMind:

[mqtt]
enabled = true
port = 1883
# Devices connect to neomind-host:1883

Or use the auto-discovery feature — plug in a USB IoT gateway and NeoMind will detect and register it.

Step 3: Build Your First Autonomous Agent

Here's where it gets interesting. Instead of just chatting with the AI, we'll create an agent that autonomously monitors devices and takes action.

Define the Agent's Mission

In NeoMind's web UI, navigate to AI → Agents → New Agent:

name: "Temperature Guardian"
schedule: "every 5 minutes"
mission: |
  You are monitoring temperature sensors across a building.
  Your responsibilities:
  1. Check all temperature readings from the last 5 minutes
  2. Flag any reading above 28°C or below 16°C as anomalous
  3. If a sensor shows 3+ consecutive anomalous readings, alert the operator
  4. Log a summary of findings to the knowledge base

tools:
  - query_device_metrics
  - send_notification
  - update_knowledge_base

What Happens at Runtime

Every 5 minutes, NeoMind's agent runtime:

Wakes the agent and injects the current context (device states, recent history, knowledge base entries)
Runs inference via Ollama — the LLM analyzes the data and decides what actions to take
Executes tool calls — the agent queries specific device metrics, sends notifications if thresholds are breached, and updates its memory
Goes back to sleep until the next cycle

┌──────────────────────────────────────────────────┐
│            Agent Runtime Loop                     │
│                                                   │
│  ┌─────────┐    ┌──────────┐    ┌─────────────┐  │
│  │ Wake +  │───►│ LLM      │───►│ Execute     │  │
│  │ Inject  │    │ Inference│    │ Tool Calls  │  │
│  │ Context │    │ (Ollama) │    │ (typed)     │  │
│  └─────────┘    └──────────┘    └──────┬──────┘  │
│                                        │         │
│  ┌─────────────────────────────────────▼──────┐  │
│  │  Memory Update + Sleep until next cycle    │  │
│  └────────────────────────────────────────────┘  │
└──────────────────────────────────────────────────┘

Type-Safe Tool Dispatch

A critical design decision: NeoMind's agent tools are not string-based function calls. When the LLM decides to "query device temperature," it produces a structured command:

// The LLM output is deserialized into a typed enum
enum AgentToolCall {
    QueryDeviceMetrics { device_id: String, metric: String, since: Duration },
    SendNotification { channel: String, message: String, severity: Severity },
    UpdateKnowledgeBase { topic: String, content: String },
    ControlDevice { device_id: String, action: DeviceAction },
}

// Dispatch is a type-safe match — no eval(), no shell exec
match tool_call {
    AgentToolCall::QueryDeviceMetrics { device_id, metric, since } => {
        let readings = device_manager.query_range(&device_id, &metric, since).await?;
        Ok(ToolResult::MetricReadings(readings))
    }
    // ...
}

This eliminates injection attacks, hallucinated command strings, and the fragile JSON-to-shell pipelines that plague other agent frameworks.

Step 4: Multi-Tier Memory — Making the Agent Learn

A stateless agent is just a fancy cron job. NeoMind's memory system gives your agent persistent context across sessions:

Memory Tiers

Tier	Purpose	Example	Retention
Profile	User preferences and environment	"Operator prefers Celsius, works 8AM-6PM"	Permanent
Knowledge	Accumulated domain facts	"Server room #3 runs 2°C hotter than average"	Long-term
Task	Current mission state	"Monitoring anomaly on sensor-07 since 14:30"	Per-task
Session	Active conversation context	"User just asked about yesterday's temperature trends"	Per-session

Automatic Memory Extraction

The agent doesn't need explicit "save this" commands. NeoMind's memory engine automatically extracts and compresses important information from agent interactions:

Agent observes: "Sensor-07 has shown elevated readings (29.1°C, 29.4°C, 28.8°C) 
                 for the last 3 cycles. This is above the 28°C threshold."

→ Auto-extracted to Knowledge Base:
  "Sensor-07 recurring thermal anomaly, first detected 2026-08-19T14:30Z,
   readings consistently 28-30°C, may indicate HVAC issue in Zone 3"

Memory Compression

Long-running agents accumulate massive context. NeoMind compresses memory using:

Temporal deduplication — merge repeated observations into time ranges
Importance scoring — prioritize security events over routine readings
Hierarchical summarization — compress daily summaries into weekly overviews

This keeps the context window manageable even after months of operation.

Step 5: Natural Language Interaction

While the autonomous agent runs in the background, you can also chat with it conversationally:

You: "What's the situation with sensor-07?"

Agent: "Sensor-07 in Zone 3 has been running hot since yesterday at 14:30. Readings have been consistently 28-30°C, which is 4-6°C above the zone average of 24°C. I've logged 12 anomalous readings. Based on the pattern, this looks like an HVAC degradation rather than a transient spike. Want me to create a maintenance ticket?"

You: "Yes, and also set up hourly monitoring for the next 48 hours"

Agent: "Done. I've created a maintenance ticket for Zone 3 HVAC inspection and scheduled myself for hourly checks on sensor-07 through August 21st. I'll alert you immediately if readings exceed 32°C."

The agent's response is grounded in its actual memory and device data — not hallucinated. NeoMind enforces this by injecting only verified data into the LLM context.

Step 6: Benchmarking — Real Numbers

Here's what we measured on common edge hardware running llama3.1:8b via Ollama:

Metric	Raspberry Pi 5	Intel NUC i5	Apple M1	RTX 3060
First token latency	1,200ms	400ms	350ms	150ms
Tokens/sec (gen)	8	18	28	45
Full agent cycle (5 devices)	6s	2.5s	1.8s	0.9s
RAM usage (idle)	4.2 GB	3.8 GB	3.5 GB	3.2 GB
RAM usage (inference)	6.1 GB	5.4 GB	5.0 GB	4.8 GB

Agent cycle = wake + context injection + inference + tool execution + memory update.

For comparison, the same agent using OpenAI's gpt-4o-mini API (cloud) adds ~800ms of network latency per inference call, plus $0.15 per 1M input tokens.

Advanced: Skill-Based Agent Customization

NeoMind's Skill System lets you fine-tune agent behavior without retraining:

# skills/factory-safety.yaml
name: "Factory Floor Safety Monitor"
trigger: "scheduled:every 2 minutes"
context: |
  You are a safety monitor for an industrial facility.

  Critical rules (NEVER override):
  - If CO2 > 1000ppm in any zone, immediately alert + activate ventilation
  - If temperature > 45°C near equipment, trigger emergency shutdown
  - If unauthorized motion detected after 22:00, alert security

  Normal operations:
  - Log all readings to knowledge base
  - Flag anomalies (>2σ from 7-day rolling average)
  - Generate daily safety summary at 18:00

tools:
  - query_device_metrics
  - send_notification
  - control_device
  - update_knowledge_base

Skills are YAML + Markdown files that the agent runtime injects into the LLM context. They provide guardrails and domain knowledge without requiring model fine-tuning.

Production Tips

Start with a smaller model — llama3.2:3b is fast enough for monitoring tasks and leaves headroom for the OS and NeoMind services
Use GPU offloading selectively — if you have a GPU, use it for the agent inference while keeping NeoMind's core services on CPU
Set memory limits — configure max_knowledge_entries and max_session_history to prevent memory bloat on long-running deployments
Monitor agent cycles — NeoMind's dashboard shows agent execution time, token usage, and tool call success rates
Layer models — use a fast small model for routine monitoring and a larger model (or cloud API) for complex analysis tasks

The Edge AI Stack, Simplified

┌─────────────────────────────────────────────┐
│              Your Edge Device                │
│                                              │
│  ┌──────────┐  ┌──────────┐  ┌───────────┐  │
│  │ NeoMind  │  │  Ollama  │  │  IoT      │  │
│  │ Platform │──│  LLM     │──│  Devices  │  │
│  │          │  │  Runtime │  │  (MQTT)   │  │
│  └──────────┘  └──────────┘  └───────────┘  │
│                                              │
│  Everything local. Zero cloud dependency.    │
└─────────────────────────────────────────────┘

That's the entire stack. Two components. One machine. No Kubernetes, no service mesh, no cloud account.

Get Started

NeoMind: github.com/camthink-ai/NeoMind (Apache 2.0)
Ollama: ollama.com
Quick Start Guide: wiki.camthink.ai
Community: Discord

Run your AI agents where your data lives — at the edge. No cloud required.

Connecting 3000+ Smart Home Devices to Edge AI: Building a Home Assistant Bridge with NeoMind

Ming — Tue, 18 Aug 2026 09:50:51 +0000

Connecting 3000+ Smart Home Devices to Edge AI: Building a Home Assistant Bridge with NeoMind

Home Assistant is the undisputed king of open-source home automation. It supports over 3000 integrations — from Philips Hue bulbs to Tesla Powerwalls to Daikin heat pumps. But there's a gap that no integration can fill: intelligence.

Home Assistant can tell you "the living room temperature is 24°C." It can't tell you "based on the last 3 weeks of occupancy patterns, your HVAC is running 40% longer than needed, and here's a schedule adjustment that would save you $18/month."

That's where NeoMind comes in. Its Home Assistant bridge imports every entity from your HA instance, feeds them to an LLM-powered AI agent running on your local hardware, and lets you interact with your entire smart home through natural language — no cloud, no subscription, no data leaving your network.

Here's how it works, what you can build, and the technical details under the hood.

The Architecture: HA as the Device Layer, NeoMind as the Brain

┌─────────────────────────────────────────────────────────┐
│                    Your Local Network                     │
│                                                          │
│  ┌──────────────┐        ┌──────────────────────────┐   │
│  │ Home Assistant│◄──────►│  NeoMind (Edge AI)       │   │
│  │  (3000+      │  REST  │  ┌──────────────────┐    │   │
│  │  integrations)│  API   │  │ HA Bridge Ext.   │    │   │
│  └──────┬───────┘        │  │ Entity Import    │    │   │
│         │                │  │ State Sync       │    │   │
│  ┌──────┴───────┐        │  │ Command Dispatch │    │   │
│  │ Your Devices │        │  └────────┬─────────┘    │   │
│  │ Lights, HVAC,│        │           │               │   │
│  │ Sensors, ... │        │  ┌────────▼─────────┐    │   │
│  └──────────────┘        │  │ AI Agent Runtime │    │   │
│                          │  │ LLM + Memory     │    │   │
│                          │  │ + Automation     │    │   │
│                          │  └──────────────────┘    │   │
│                          └──────────────────────────┘   │
└─────────────────────────────────────────────────────────┘

The bridge is bidirectional:

HA → NeoMind: All entities (sensors, switches, lights, climate, cameras, etc.) are imported with their current states, attributes, and history. NeoMind sees your entire home as a structured data model.
NeoMind → HA: AI-generated commands flow back through the HA REST API. "Turn off everything on the ground floor after midnight" becomes a series of precise service calls.

Setup: 5 Minutes to a Smarter Home

1. Install NeoMind

curl -fsSL https://raw.githubusercontent.com/camthink-ai/NeoMind/main/scripts/install.sh | sh

Or if you already have NeoMind running, skip to step 2.

2. Enable the Home Assistant Extension

In NeoMind's web UI (http://localhost:9375), navigate to Extensions → Marketplace and enable "Home Assistant Bridge." Or via the config file:

[extensions.home_assistant]
enabled = true
url = "http://homeassistant.local:8123"
token = "your-long-lived-access-token"
sync_interval = 30  # seconds between state refreshes

Generate a long-lived access token in Home Assistant: Profile → Security → Long-Lived Access Tokens.

3. Configure Your LLM Backend

For fully local AI (recommended for privacy):

[ai]
backend = "ollama"
model = "llama3.1:8b"
api_url = "http://localhost:11434"

Or connect to a cloud model for more capable reasoning:

[ai]
backend = "openai"
model = "gpt-4o-mini"
api_key = "sk-..."

4. Start Talking to Your Home

Open the NeoMind chat interface and start asking questions:

You: "What's the energy consumption looking like today?"

NeoMind: "Your home has used 14.2 kWh so far today, which is 12% above your daily average. The main contributors are the HVAC (6.8 kWh, running since 6 AM due to the cold snap) and the EV charger (4.1 kWh, charging the Tesla from 32% to 78%)."

You: "Can you optimize the HVAC schedule for this week?"

NeoMind: "Based on your occupancy patterns over the last 14 days, nobody is home between 8:30 AM and 5:15 PM on weekdays. I can create an automation rule that sets the thermostat to eco mode (18°C) during those hours and pre-heats to 22°C at 4:45 PM. Estimated savings: ~$22/week. Shall I create this rule?"

Entity Import: How 3000+ Devices Become AI-Readable

When the bridge connects to Home Assistant, it imports the full entity registry:

// Simplified: what NeoMind sees for each HA entity
pub struct HomeAssistantEntity {
    pub entity_id: String,           // "light.living_room_main"
    pub domain: String,              // "light"
    pub friendly_name: String,       // "Living Room Main Light"
    pub state: String,               // "on" / "off" / "24.5"
    pub attributes: HashMap<String, Value>,  // brightness, color_temp, etc.
    pub area: Option<String>,        // "Living Room"
    pub device_class: Option<String>, // "temperature" / "humidity" / etc.
}

The AI agent uses this structured model to:

Answer questions — "Which lights are currently on?" queries the state of all light.* entities
Analyze patterns — "How has the bedroom humidity changed over the past week?" reads time-series data from sensor.bedroom_humidity
Create automations — "Turn off all lights when the last person leaves" combines person.* presence with light.* control
Suggest optimizations — "Your bathroom fan runs 6 hours daily but humidity rarely exceeds 55%" identifies waste

Natural Language → HA Service Calls

The most powerful feature is turning conversational intent into precise Home Assistant service calls. Here's what happens when you say "Dim the living room to 30% and turn on the reading lamp":

User Input: "Dim the living room to 30% and turn on the reading lamp"
    │
    ▼
AI Agent parses intent into structured commands:
    │
    ├── Command 1: light.turn_on
    │   entity_id: light.living_room_main
    │   brightness_pct: 30
    │
    └── Command 2: light.turn_on
        entity_id: light.reading_lamp
    │
    ▼
HA Bridge dispatches via REST API:
    POST http://ha:8123/api/services/light/turn_on
    {"entity_id": "light.living_room_main", "brightness_pct": 30}
    POST http://ha:8123/api/services/light/turn_on
    {"entity_id": "light.reading_lamp"}

This isn't regex matching or keyword spotting. The LLM understands context:

"Turn off the kitchen" → resolves to all entities in the "Kitchen" area
"Set the thermostat to what it was yesterday morning" → looks up historical state
"If the garage door is open after 10 PM, close it and notify me" → creates an automation rule with condition + action + notification

Automation Rules: Beyond HA's Native Automations

Home Assistant's YAML automations are powerful but verbose. NeoMind lets you create them conversationally:

You: "Every weekday at 7 AM, if it's raining, turn on the entryway light and send me a reminder to bring an umbrella"

NeoMind generates:

{
  "name": "Rainy Morning Routine",
  "condition": {
    "operator": "and",
    "conditions": [
      {"source": "time", "operator": "equals", "value": "07:00"},
      {"source": "time", "operator": "day_of_week", "value": ["mon","tue","wed","thu","fri"]},
      {"source": "device:weather.home:condition", "operator": "equals", "value": "rainy"}
    ]
  },
  "actions": [
    {"type": "ha_service", "service": "light.turn_on", "entity_id": "light.entryway"},
    {"type": "notify", "channel": "push", "message": "Rainy morning — don't forget your umbrella!"}
  ],
  "cooldown": 86400
}

This rule runs in NeoMind's engine — not in HA's automation system — which means it can incorporate AI analysis:

{
  "name": "Smart Energy Saver",
  "trigger": {"source": "schedule", "interval": "1h"},
  "actions": [
    {
      "type": "ai_agent",
      "prompt": "Analyze the last hour of energy consumption. If any device used more than 2x its typical hourly average, explain why and suggest whether it should be turned off or scheduled differently."
    }
  ]
}

Multi-Room Voice Control Without the Cloud

Combine NeoMind's Home Assistant bridge with the voice extensions (CosyVoice TTS + multilingual ASR) for a fully local voice assistant:

"Hey NeoMind, what's the air quality in the bedroom?"
    │
    ▼ (local ASR → text)
    │
    ▼ (AI Agent → query sensor.bedroom_air_quality)
    │
    ▼ (LLM generates natural response)
    │
    ▼ (local TTS → audio)
    │
"The bedroom AQI is 42 — good air quality. The PM2.5 level dropped significantly 
after you turned on the purifier at 3 PM."

No Alexa. No Google. No audio sent to the cloud. Everything runs on your hardware.

Scaling: From a Studio Apartment to a Smart Campus

The bridge handles anything Home Assistant supports:

Scale	Entities	NeoMind Performance
Studio	~20 entities	< 50ms response, ~80MB RAM
3-bedroom house	~150 entities	< 100ms response, ~150MB RAM
Large villa	~500 entities	< 200ms response, ~300MB RAM
Small office	~1500 entities	< 500ms response, ~600MB RAM
Smart campus (multi-HA)	~5000+ entities	Run multiple NeoMind instances, one per HA

For multi-site deployments, each NeoMind instance connects to its own Home Assistant, and you coordinate across instances through shared dashboards and cross-instance queries.

Privacy: Your Home Data Stays Home

This is the killer feature for privacy-conscious users:

No cloud processing — LLM runs locally via Ollama (Llama 3, Mistral, Qwen, etc.)
No telemetry — NeoMind doesn't phone home. Ever.
No third-party access — Your HA entities never leave your LAN
Offline capable — Works even when your internet is down

Compare this to cloud-based smart home AI (Google Home, Alexa, etc.), where every voice command and device state is transmitted to external servers.

Get Started

NeoMind: github.com/camthink-ai/NeoMind (Apache 2.0)
Home Assistant Bridge docs: wiki.camthink.ai
5-Minute Quick Start: wiki.camthink.ai/docs/neomind/quick-start
Community: Discord

Already running Home Assistant? Install NeoMind, enable the bridge, and ask your home a question. The AI is local, the integration is instant, and your data never leaves your network.

Single-Process Architecture in Rust: Why We Rejected Microservices for an Edge AI Platform

Ming — Mon, 17 Aug 2026 09:50:56 +0000

Single-Process Architecture in Rust: Why We Rejected Microservices for an Edge AI Platform

In 2026, "microservices" is practically a reflex answer for any new project. Split your system into independently deployable services, orchestrate with Kubernetes, glue them with a message bus. It works — until you're deploying to a $200 edge device in a factory with no reliable internet.

NeoMind is an open-source edge AI platform built in Rust that deliberately chose a single-process architecture. One binary. One process. Everything — the HTTP API, MQTT broker, storage engine, AI agent runtime, rule engine, and extension host — running in a single OS process.

This is the story of why, the trade-offs we accepted, and what we learned building it.

The Problem with Microservices at the Edge

Most AI platforms today follow the microservices playbook:

[API Gateway] → [Auth Service] → [Device Service] → [AI Service]
                     ↓                ↓                  ↓
               [PostgreSQL]      [TimescaleDB]      [Redis]
                     ↑                ↑                  ↑
              [Kafka/RabbitMQ] ← [Event Bus] ← [Notification Svc]

This works beautifully in the cloud. But at the edge, you hit a wall:

Resource overhead. Every microservice needs its own process, memory allocation, file descriptors, and network sockets. On a Raspberry Pi 5 with 8GB RAM, running 8 Docker containers with health checks, log aggregation, and service mesh proxies consumes 2-3GB before your application even starts.

Deployment complexity. docker-compose up sounds simple until you're SSH'ing into 50 factory floor devices to update certificates, debug a failing sidecar, or explain why the health-check probe is timing out on a flaky network.

Latency accumulation. A device telemetry reading that flows through Device Service → Event Bus → AI Service → Rule Engine → Notification Service touches 5 network hops. Each hop adds serialization, deserialization, and queue wait time. For real-time industrial monitoring, those milliseconds compound.

Operational burden. Twelve services means twelve log streams, twelve health endpoints, twelve restart policies, and twelve ways for a partial failure to cascade into a confusing debugging session at 3 AM.

The NeoMind Approach: One Process, Everything Inside

NeoMind's architecture looks like this:

┌─────────────────────────────────────────────────────┐
│                  Single Process                      │
│                                                      │
│  ┌─────────┐  ┌──────────┐  ┌───────────────────┐  │
│  │  Axum   │  │ Embedded │  │  AI Agent Runtime  │  │
│  │  HTTP   │  │  MQTT    │  │  (LLM + Memory +   │  │
│  │  Server │  │  Broker  │  │   Tools + Skills)  │  │
│  └────┬────┘  └────┬─────┘  └─────────┬─────────┘  │
│       │            │                   │             │
│  ┌────┴────────────┴───────────────────┴──────────┐  │
│  │              Event Bus (channels)               │  │
│  └────┬────────────┬───────────────────┬──────────┘  │
│       │            │                   │             │
│  ┌────┴────┐  ┌────┴─────┐  ┌────────┴──────────┐  │
│  │ Rule    │  │ Device   │  │  Extension Host    │  │
│  │ Engine  │  │ Manager  │  │  (Process-Isolated)│  │
│  └────┬────┘  └────┬─────┘  └─────────┬─────────┘  │
│       │            │                   │             │
│  ┌────┴────────────┴───────────────────┴──────────┐  │
│  │         Storage Layer (redb)                    │  │
│  │  Time-Series | State | LLM Memory | Logs       │  │
│  └────────────────────────────────────────────────┘  │
└─────────────────────────────────────────────────────┘

One cargo build --release produces a single binary. One ./neomind starts everything.

Why redb Instead of PostgreSQL

The storage layer uses redb, a pure-Rust embedded key-value store inspired by LMDB. This is a deliberate rejection of the "just use Postgres" default.

Zero external dependencies. No database server to install, no connection pooling to configure, no migration runner to orchestrate. The database is a file on disk, opened by the same process that serves HTTP requests.

B+ tree indexes with ACID transactions. redb provides serializable isolation, crash-safe writes, and efficient range scans — the same guarantees you'd expect from a traditional RDBMS, without the process overhead.

Time-series optimized. Telemetry data from IoT devices follows a write-heavy, append-mostly pattern. redb's B+ tree handles sequential inserts efficiently, and range queries over time windows are O(log n).

Memory-mapped reads. Hot data stays in the OS page cache without explicit caching logic. Cold data stays on disk. No Redis layer, no separate cache invalidation strategy.

The trade-off: no SQL. Queries are expressed in Rust with type-safe key ranges. For our access patterns (device state lookups, time-series scans, LLM memory retrieval), this is more efficient than an ORM layer over Postgres. For ad-hoc analytics, users export to their preferred tool.

The Event Bus: channels, Not Kafka

Inter-module communication uses Rust's tokio::sync::broadcast channels — in-process, zero-copy, backpressure-aware.

// Simplified: device telemetry event
pub struct DeviceEvent {
    pub device_id: String,
    pub metric: String,
    pub value: f64,
    pub timestamp: i64,
}

// Any module can subscribe
let mut rx = event_bus.subscribe::<DeviceEvent>();

// Any module can publish — no serialization, no network hop
event_bus.publish(DeviceEvent {
    device_id: "sensor-01".into(),
    metric: "temperature".into(),
    value: 28.5,
    timestamp: now(),
});

Compared to Kafka or RabbitMQ:

Aspect	Kafka/RabbitMQ	In-Process Channels
Latency	1-10ms per hop	< 1μs
Memory	Separate process + JVM/BEAM	Shared address space
Serialization	JSON/Protobuf required	Zero-copy struct passing
Durability	Built-in (disk-backed)	Optional (redb persistence)
Multi-node	Yes	No (single-node by design)

The durability gap is closed by persisting critical events to redb before publishing. If the process crashes, the event log survives on disk and replays on restart.

Extension Isolation: Processes, Not Containers

Here's where the single-process model gets nuanced. Extensions (YOLO object detection, Home Assistant bridge, Modbus adapter, etc.) run as separate OS processes, communicating with the core via FFI and a capability-based permission system.

┌─────────────────────────────────────────────┐
│  Core Process (Rust, single binary)          │
│  ┌─────────┐ ┌──────────┐ ┌──────────────┐  │
│  │ HTTP    │ │ MQTT     │ │ AI Agent     │  │
│  │ Server  │ │ Broker   │ │ Runtime      │  │
│  └─────────┘ └──────────┘ └──────────────┘  │
│                                              │
│  Extension Host                              │
│  ┌──────┐ ┌──────┐ ┌──────┐ ┌──────┐       │
│  │Spawn │ │Spawn │ │Spawn │ │Spawn │       │
│  └──┬───┘ └──┬───┘ └──┬───┘ └──┬───┘       │
└─────┼────────┼────────┼────────┼────────────┘
      │        │        │        │
  ┌───┴──┐ ┌──┴───┐ ┌──┴───┐ ┌──┴───┐
  │ YOLO │ │ HA   │ │Modbus│ │BACnet│
  │(Rust)│ │Bridge│ │(Rust)│ │(Rust)│
  └──────┘ └──────┘ └──────┘ └──────┘
  separate processes, FFI + capabilities

Why not run extensions in-process? Because extensions are where third-party code lives. A buggy YOLO model loader or a misbehaving MQTT bridge shouldn't crash the core. Process boundaries give us:

Crash isolation — kill -9 the extension, core keeps running
Memory isolation — an extension's memory leak doesn't starve the core
Capability enforcement — extensions declare what they need (network, filesystem, device access) and get only those permissions
Hot reload — restart an extension without restarting the whole system

This is Erlang's "let it crash" philosophy, adapted for Rust: the core almost never crashes (ownership + borrow checker), but extensions might — and that's fine.

Why not containers? Docker adds ~50MB overhead per container (runc, containerd-shim, cgroup setup). On edge devices, that's unacceptable. OS process spawning is ~1ms with negligible memory overhead beyond the extension's own footprint.

The AI Agent Runtime: In-Process LLM Orchestration

The AI agent is the most complex subsystem, and it runs entirely in-process:

LLM backends (Ollama, OpenAI, Anthropic, Google, DeepSeek, etc.) are HTTP clients making external calls, but the orchestration logic, prompt construction, and response parsing stay in-process.
Multi-tier memory (Profile, Knowledge, Task, Session) is stored in redb with automatic extraction and compression — no external vector database.
Tool calling dispatches in-process via typed command structs, not string-based APIs. The AI says "query device temperature" and the runtime calls DeviceManager::get_metric("sensor-01", "temperature") directly.

// Type-safe tool dispatch — no eval(), no shell, no string parsing
enum AgentCommand {
    QueryDevice { device_id: String, metric: String },
    CreateRule { rule: RuleDefinition },
    ListDevices { filter: DeviceFilter },
    ControlDevice { device_id: String, action: DeviceAction },
}

// The AI's intent is deserialized into a typed enum, then matched
match command {
    AgentCommand::QueryDevice { device_id, metric } => {
        device_manager.get_metric(&device_id, &metric).await
    }
    // ...
}

This eliminates an entire class of injection vulnerabilities that plague string-based agent frameworks.

What We Gave Up

Honesty demands we acknowledge the trade-offs:

No horizontal scaling. A single process means a single node. If you need to process telemetry from 100,000 devices across 10 geographic sites, you run 10 instances — each managing its own fleet. There's no shared state between instances (by design). For multi-site coordination, you layer a separate aggregation tier.

No polyglot runtime. Everything in the core is Rust. If you want to contribute a core feature, you write Rust. Extensions can be any language that speaks the FFI protocol, but the core is monolingual.

No independent deployability. A bug in the rule engine requires redeploying the entire binary. There's no "just update the rule engine service." For edge deployments, this is actually simpler (one binary to distribute), but it means CI/CD is all-or-nothing.

No built-in multi-tenancy. Each instance serves one tenant. Multi-tenant setups run multiple instances behind a reverse proxy — simpler than building tenant isolation into every module.

Benchmarking: Single Process vs. Microservices

On a Raspberry Pi 5 (8GB RAM), we measured:

Metric	NeoMind (Single Process)	Equivalent Microservices (8 containers)
Idle memory	~120 MB	~1.8 GB
Cold start	< 2 seconds	~45 seconds (all containers healthy)
Telemetry latency (device → rule → action)	~3 ms	~85 ms
Binary size	~35 MB	~2.1 GB (all images)
Deployment	1 file copy + systemd	docker-compose + registry + certs

The memory difference alone makes single-process viable on hardware where microservices simply aren't.

When to Choose Single-Process

This isn't a "microservices are bad" take. Microservices are the right choice for large teams, high-scale cloud workloads, and systems that need independent scaling. But single-process is the right choice when:

Deployment target is constrained hardware — edge devices, IoT gateways, embedded systems
Operational simplicity matters more than architectural purity — no SRE team on the factory floor
Latency is critical — real-time device control, industrial automation
The team is small — one codebase, one language, one deployment artifact
Offline operation is required — no service mesh, no cloud dependency

NeoMind hits all five criteria. Your project might not — and that's fine.

Try It

NeoMind is open source (Apache 2.0). One command to get started:

curl -fsSL https://raw.githubusercontent.com/camthink-ai/NeoMind/main/scripts/install.sh | sh

Source: github.com/camthink-ai/NeoMind
Docs: wiki.camthink.ai
Community: Discord

If you've built edge systems that rejected microservices, I'd love to hear about your trade-offs in the comments.

NeoMind: A Rust-Powered Edge AI Platform That Talks to Your IoT Devices

Ming — Fri, 07 Aug 2026 07:30:25 +0000

NeoMind: A Rust-Powered Edge AI Platform That Talks to Your IoT Devices

Talk to your devices in natural language. The AI understands your intent, queries device states, creates automation rules, and takes action — all on your own hardware, with zero cloud dependency.

The Problem We're Solving

Edge AI is stuck between two extremes. On one side, you have cloud-first platforms that ship every sensor reading to a remote server — introducing latency, bandwidth costs, and privacy concerns that make industrial customers cringe. On the other, you have bare-metal inference frameworks that run a single model well but offer no device management, no automation, no natural language interface.

What's missing is the middleware layer — something that sits on your edge hardware, connects to real devices, runs intelligent agents, and lets operators interact with the whole system conversationally. That's what NeoMind is.

What Is NeoMind?

NeoMind is an open-source, edge-deployed AI platform built in Rust. It brings together:

LLM-powered agents that run directly on your hardware (via Ollama or cloud backends)
Device connectivity through MQTT, BLE, and Webhook protocols
A JSON rule engine for automation with recursive conditions and debouncing
Real-time dashboards with drag-and-drop widgets
22 official extensions spanning vision AI (YOLO, face recognition, OCR), voice (TTS/ASR), and IoT bridges (Home Assistant, Modbus, BACnet, ONVIF, OPC-UA, LoRaWAN)

All packaged in a single binary. No Docker Compose. No external database. No message broker to install.

# One command. Everything runs.
curl -fsSL https://raw.githubusercontent.com/camthink-ai/NeoMind/main/scripts/install.sh | sh

Architecture: Single-Process, Self-Contained

NeoMind's most distinctive design choice is its single-process architecture. The API server (Axum), embedded MQTT broker, redb storage, rule engine, AI agent runtime, and extension runner all live in one process.

┌──────────────────────────────────────────────────────────┐
│               Desktop App / Web UI                       │
│                React 18 + TypeScript                     │
├──────────────────────────────────────────────────────────┤
│              Tauri 2.x / Browser                         │
└────────────────────────┬─────────────────────────────────┘
                         │ REST / WebSocket / SSE
                         ▼
┌──────────────────────────────────────────────────────────┐
│                   API Gateway (Axum)                      │
│  ┌────────┐ ┌────────┐ ┌────────┐ ┌────────┐            │
│  │  Auth  │ │Devices │ │Automate│ │Messages│            │
│  └────────┘ └────────┘ └────────┘ └────────┘            │
├──────────────────────────────────────────────────────────┤
│                     Event Bus                             │
│  ┌──────────┐ ┌──────────┐ ┌──────────┐ ┌────────────┐ │
│  │ Devices  │ │Automation│ │ AI Agent │ │ Extensions │ │
│  │ MQTT/BLE │ │ Rules    │ │ Chat     │ │ Isolated   │ │
│  │ Webhook  │ │ Transform│ │ Tools    │ │ Native+WASM│ │
│  └──────────┘ └──────────┘ │ Memory   │ └────────────┘ │
│                            └──────────┘                  │
├──────────────────────────────────────────────────────────┤
│                  Storage Layer (redb)                     │
│  Time-Series │ State │ LLM Memory │ Push Logs            │
└──────────────────────────────────────────────────────────┘

Why this matters:

Deployment simplicity — cargo run and you're done. No orchestrating 12 microservices.
Resource efficiency — One process shares memory, file descriptors, and connection pools. Critical on constrained edge hardware.
Operational clarity — One log stream, one PID to monitor, one binary to update.

Three Design Philosophies

1. Edge-First, Cloud-Optional

NeoMind defaults to local LLMs via Ollama — your data never leaves your LAN. When you need frontier-model capabilities, switch to OpenAI, Anthropic, Google, or any OpenAI-compatible endpoint with a single config change. The agent runtime is backend-agnostic.

This isn't just a privacy play. For industrial IoT deployments on factory floors or remote campuses, reliable internet isn't guaranteed. NeoMind works fully offline.

2. Crash-Isolated Extensions

Extensions run in separate processes and communicate with the core via FFI with capability-based permissions. If your YOLO object detection extension crashes, the main service and all other extensions are completely unaffected.

This is modeled after Erlang's "let it crash" philosophy, adapted for a Rust codebase where the core itself rarely crashes — but third-party extensions might.

3. Type-Safe End-to-End

The Rust backend provides compile-time guarantees across the entire stack. Agent CLI commands dispatch in-process with structured data types — no fragile string parsing, no eval(), no shell injection vectors.

AI Agents: More Than a Chatbot

NeoMind's AI layer goes beyond simple Q&A:

Natural Language Chat — Conversational interface to query device states, create automation rules, and control hardware. "Turn off all cameras on Floor 3 after 10 PM" becomes a real command.
Autonomous Agents — Scheduled or event-driven agents that monitor telemetry, analyze patterns, and trigger actions independently. Think: "Watch the temperature sensors and alert me if any reading exceeds 30°C for more than 5 minutes."
Multi-tier Memory — Profile memory, knowledge base, task history, and session context with automatic extraction and compression. The AI remembers your facility layout and past incidents.
Skill System — YAML + Markdown skill files that guide agent behavior for specific scenarios. Write a skill for "factory floor safety check" and the agent follows it precisely.

Supported backends include Ollama, OpenAI, Anthropic, Google, xAI, Qwen, DeepSeek, GLM, MiniMax, and any OpenAI-compatible endpoint.

Device Management: MQTT, BLE, and Beyond

NeoMind treats IoT devices as first-class citizens:

Embedded MQTT Broker — with mTLS and CA certificate support. Your devices connect directly, no external broker needed.
BLE Provisioning — Zero-touch device setup via Bluetooth (Tauri native + Web Bluetooth). Walk up to a device, pair, configure.
Auto-Discovery — Plug in a new device and NeoMind detects it, registers its type, and offers AI-assisted onboarding.
Custom Device Types — Define metrics and commands via JSON type definitions. No code changes required.

The extension ecosystem adds bridges for Home Assistant (3000+ entity integrations), Modbus (TCP/RTU PLCs), BACnet (building automation), ONVIF (IP cameras), OPC-UA (industrial servers), and LoRaWAN (ChirpStack/TTN sensors).

Automation Without Code

The JSON rule engine supports recursive conditions with comparison, range, and logical operators:

{
  "condition": {
    "source": "device:sensor:temperature",
    "operator": "greater_than",
    "threshold": 30
  },
  "actions": [
    {"type": "notify", "channel": "slack", "message": "Temperature alert!"},
    {"type": "execute", "command": "device:fan:turn_on"}
  ],
  "cooldown": 300,
  "for_duration": 60
}

Combine rules with data transforms (JavaScript-based virtual metrics) and scheduled AI agents to build sophisticated automation pipelines — all without writing backend code.

22 Extensions and Growing

The extension marketplace ships with production-ready integrations:

Category	Extensions
Vision	YOLO object detection, face recognition, OCR, visual grounding
Voice	CosyVoice TTS, voice cloning, multilingual ASR, voice assistant pipeline
IoT Bridges	Home Assistant, Modbus, BACnet, ONVIF, OPC-UA, LoRaWAN
Utilities	Weather forecast, stream player, WASM demo

Each extension is process-isolated with capability-based permissions. Build your own following the Extension Development Guide.

Cross-Platform: Desktop, Server, Mobile

NeoMind ships native desktop apps via Tauri 2.x for macOS (Apple Silicon + Intel), Windows, and Linux. The responsive web UI is optimized for phone and tablet. Server deployments support Docker, systemd, and nginx reverse proxy.

# Docker — multi-arch, pulls official image
docker run -d --name neomind \
  -p 9375:9375 -p 1883:1883 \
  -v neomind-data:/app/data \
  camthink/neomind:latest

Get Started

Download: GitHub Releases (.dmg, .msi, .AppImage, .deb)
5-Minute Quick Start: wiki.camthink.ai
Source Code: github.com/camthink-ai/NeoMind (Apache 2.0)
Community: Discord · GitHub Issues

Contributing

NeoMind is open source and actively seeking contributors. Four ways to get involved:

Build an Extension — New AI models, data sources, or integrations
Add a Device Type — Define metrics and commands for your IoT hardware
Create a Dashboard Widget — Reusable React components for the community
Core Contributions — Rust backend, React frontend, documentation

NeoMind is developed by CamThink AI. Licensed under Apache 2.0.

Technical Implementation and Architectural Analysis of an ESP32-S3-Based Deep Sleep Outdoor Camera

Ming — Mon, 03 Nov 2025 15:16:38 +0000

In Internet of Things (IoT) applications, particularly in outdoor or edge deployment scenarios (such as smart agriculture, wildlife monitoring, and infrastructure inspection), providing a stable power supply for monitoring devices remains a core challenge. Traditional network cameras, which typically operate 24/7, have high power consumption and rely on external power sources, making deployment difficult in many situations.

This article will share the technical decisions and architectural design behind our development of the CamThink NeoEyes NE101. We will focus on how to leverage the features of the ESP32-S3 to implement an "on-demand" ultra-low-power camera, solving the battery life challenges inherent in battery-powered scenarios.

The Core Challenge: The "Impossible Triangle" of Power Consumption When developing a battery-powered outdoor camera, we face several key technical trade-offs:

Standby Power vs. Response Speed: The system needs to remain in an extremely low-power "deep sleep" state for long periods but must also respond and wake up quickly to external events (like PIR sensors or timers). This efficiency is, of course, related to chip performance.

Image Processing vs. Power Consumption: Image acquisition, Auto Exposure (AE) adjustment, encoding, and processing are compute-intensive tasks that cause instantaneous power spikes.

Data Transmission vs. Power Consumption: Wireless communication modules like Wi-Fi and Cat-1 (4G) are notorious "power hogs" when searching for networks and transmitting data.

Traditional solutions are either "always-on," leading to high power draw, or "completely dormant," resulting in slow wake-up times and missed events. Our goal was to achieve an "on-demand" working model.

Technical Selection: Why the ESP32-S3? We selected the ESP32-S3 as the main controller (MCU) primarily for its unique dual-core heterogeneous architecture:

High-Performance Main Cores: Two Xtensa® LX7 dual-core processors at up to 240MHz.

Ultra-Low-Power (ULP) Coprocessor: A RISC-V-based coprocessor.

This architecture is the technical cornerstone for achieving ultra-low power consumption. Our design philosophy is "task separation":

LX7 High-Performance Cores: Are only awakened when needed. They handle the "heavy lifting," including:

Data acquisition and processing from the image sensor (CPI/USB).

Auto Exposure (AE) and image parameter adjustments.

Running the network stack (Wi-Fi, TCP/IP, MQTT).

Executing complex logic like data uploads and OTA updates.

RISC-V ULP Coprocessor: Remains "on duty" 24/7. It continues to run while the main cores are in deep sleep, handling the "light work," such as:

Monitoring external GPIO triggers (e.g., connected PIR sensors, sound sensors, or buttons).

Managing the RTC (Real-Time Clock) for timed wake-ups.

Deciding whether to wake the main cores based on specific conditions.

Architectural Design: An Event-Driven Operational Flow Our firmware is not a "dumb" loop but a strict, event-driven state machine.

State 1: Deep Sleep This is the state the system is in 99.9% of the time. In this state, the LX7 main cores, most peripherals, and the Flash memory are powered down. Only the ULP coprocessor and the RTC are running on an extremely low current (around 22µA, according to our test data), awaiting a trigger signal.

State 2: Trigger & Wake-up When the ULP detects a predefined trigger condition (e.g., a level change on the PIR interface or an RTC timer expiration), it immediately sends a signal to the power management unit (PMU) to wake up the LX7 main cores.

State 3: Active Cycle This is the highest power consumption phase, and our core optimization goal is to minimize its duration.

System Initialization & Sensor Power-On: The main core wakes from Deep Sleep and boots up quickly.

Image Acquisition & AE Convergence: This is an often-overlooked time-consuming point. To capture clear, properly exposed photos in variable lighting conditions (dawn, dusk, high noon), the system requires a few seconds (the NE101 firmware defaults to 5 seconds) for Auto Exposure (AE) to converge. (Note: Developers can adjust this time based on scene requirements, balancing power consumption and image quality).

Capture & Processing: A single snapshot is taken.

Communications Module Power-On & Connection: Based on the configuration (Wi-Fi / Cat-1 / Wi-Fi HaLow), the communication module is activated and connects to the network.

Data Transmission: The image data is uploaded to the cloud via MQTT or another protocol.

Enter Sleep: Once the task is complete, the system immediately executes a command to clear its state and return to State 1 (Deep Sleep).

Based on new test data, the entire "Active Cycle" takes approximately 30 seconds for the Cat-1 version, depending on network conditions, while other versions can be controlled within 20 seconds.

Key Technical Points: Hardware Modularization and Interface Design To adapt this low-power platform to different developer needs, we adopted a modular and open-interface design for the hardware.

4.1. Pluggable Communication Solutions
The mainboard does not have the communication module soldered directly. Instead, it reserves standardized interfaces, allowing developers to choose based on the deployment scenario:

Wi-Fi: For near-field scenarios with existing Wi-Fi coverage.

LTE Cat-1: For wide-area scenarios without Wi-Fi but with cellular network access.

Wi-Fi HaLow (IEEE 802.11ah): For long-distance (up to 1km), low-power private network scenarios, requiring a gateway for connectivity.

4.2. Flexible Sensor and Camera Interfaces
Camera Support: The mainboard supports both CPI and USB interface cameras.

CPI (OV5640): Lower power consumption, simple interface, suitable for regular snapshots.

USB (SC200AI): Higher bandwidth, supports 1080P, but occupies additional I/O resources and the USB host function.

Trigger Source Expansion: We have opened up multiple hardware interfaces to be used as "triggers":

PIR Interface: A 4-pin Wafer connector dedicated to PIR sensor integration.

16-pin IO Interface: Reserves GPIO, DI, DO, etc., allowing developers to connect their own sensors (e.g., radar, sound, vibration, temperature/humidity) as wake-up sources.

Alarm Interface: A 2-pin Wafer connector for alarm input. By default, the PIR interface is used; this interface requires soldering an additional resistor for use.

Power Consumption Data and Battery Life Analysis The theoretical architecture must ultimately be validated by data. In a typical use scenario of 5 snapshots per day (with the fill light off), the NE101's battery life performance (based on test data) is:

Wi-Fi Version: Can support up to 2.73 years.

Wi-Fi HaLow Version: Up to 2.59 years.

Cat-1 Version: Can operate stably for 1.24 years.

This longevity is primarily due to the extreme optimization of both sleep and operating power.

5.1. Sleep Power
This is the power consumed when the device is not capturing images (deep sleep) and is the foundation for long battery life.

Technical Analysis:

The base sleep power consumption is nearly identical across all three communication modes, with annual consumption at around 8%.

This implies that in a pure standby (theoretical) scenario, the battery life could exceed 12 years (2500mAh / 195.79mAh ≈ 12.8 years).

This demonstrates the effectiveness of the ESP32-S3's ULP coprocessor architecture in achieving ultra-low base power consumption.

5.2. Operating Power
This refers to the total power required for the device to wake up, acquire an image, and complete the upload. This is the key factor determining the battery life differences between communication solutions.

Key Operating Parameters Test Data:

Technical Analysis:

Sleep Power: Again validates that the base sleep power for all versions is extremely low, in the 22-23.5µA range.

Fill Light Impact: Turning the fill light on or off has a noticeable impact on single-event power consumption (an increase of ~10-15%). It should be avoided when pursuing extreme battery life.

Operating Time: Cat-1 takes the longest "active cycle" (approx. 30 seconds) due to the need to search for and register on the cellular network, while Wi-Fi and HaLow are much faster (<20 seconds).

Cat-1 Power: The Cat-1 module's average current during operation (76.1mA) and its longer operating time (~30s) are both significantly higher than the Wi-Fi solutions. This is the primary reason its total battery life (1.24 years) is lower than the Wi-Fi version (2.73 years).

5.3. Power Estimation Model
Based on the data above, we can derive the technical model for battery life estimation:

Total Daily Power Consumption = (Base Sleep Power × 24h) + (Single Event Operating Power × Number of Daily Events)

It is precisely because the base sleep power (~22µA) has been pushed to an extremely low level that the first half of the equation (sleep consumption) becomes minimal. The system's total power consumption is therefore primarily determined by the second half (number of operating events). This is what enables us to achieve battery life measured in "years" in low-frequency scenarios like "5 snapshots per day."

Conclusion

Implementing a deep-sleep outdoor camera based on the ESP32-S3 is entirely feasible. The technical core lies in fully leveraging its heterogeneous architecture (LX7 high-performance cores + RISC-V ULP coprocessor).

By designing the system in an "event-driven" mode, allowing the main core to be in Deep Sleep (at ~22µA) 99.9% of the time, with only the ULP coprocessor handling low-power "sentry" tasks, we fundamentally solve the power bottleneck of traditional IPCs. The latest real-world test data strongly supports this, providing developers with a clear basis for making battery life trade-offs between different communication solutions (Wi-Fi, Cat-1, HaLow).

Known Drawbacks of Using the ESP32-S3 The ESP32-S3's image processing performance is relatively weak, making it difficult to support the processing of large images. Simultaneously, this lack of image processing power can lead to a longer time from wake-up to handling the image sensor, resulting in low real-time performance for image capture. However, for applications that do not require high-frequency image acquisition, this product architecture remains highly practical and balanced.

Firmware code：https://github.com/camthink-ai/lowpower_camera
Hardware Info：https://wiki.camthink.ai/docs/neoeyes-ne101-series/ne100-mb01-development-board/dev-guide

We are looking for developers in the community to work with us to explore low-power camera products. We look forward to your participation in the community construction. If you have any intention, you are welcome to join us!!

We have prepared a gift for the developers.
https://near.tl/sm/uCT0JnK9e