DEV Community: mote

I Gave My Roombot a Memory Using Gemini Embeddings and a 900KB Rust DB

mote — Sun, 12 Jul 2026 07:29:49 +0000

I have a problem. I lose things. Soldering irons, the third M3 screw, the receipt I swore I left on the desk. So this weekend I did what any reasonable person would do: I taught my roombot to remember where my stuff is.

Not "store a row." Remember. The difference matters, and it's the whole reason moteDB exists.

Let me back up. The passion here isn't "I built a find-my-irons app." It's that I've been low-key obsessed for years with one question: why can't machines remember like we do? We walk into a room and know the red box is the one we grabbed three minutes ago. Robots forget that the instant the frame buffer clears. That gap bugged me enough to spend 18 months writing a database in Rust. This weekend I finally pointed it at something stupid and fun.

The build

Hardware: a Raspberry Pi 5, a webcam, and a robot arm I assembled from a kit that Definitely Had All The Parts (it did not).

The memory pipeline has two ends:

Google AI (Gemini) turns what the camera sees into a multimodal embedding — image plus a short text caption. One vector that means "the soldering iron on the blue mat, tip facing left."
moteDB stores that embedding on-device, next to the timestamp, the pose, and a text note. No server. No cloud round-trip at query time.

Why this split? Gemini is genuinely good at multimodal embeddings — better than anything I can run locally on a Pi. But I don't want my robot's memory living in someone else's API. So Gemini does the thinking, moteDB does the keeping. The embedding gets computed once and then lives in a 900KB binary on the device.

Here's the actual insert path:

use motedb::{Db, Episode};

// 1. Gemini gives us a multimodal embedding (image + caption)
let embedding = gemini::embed_multimodal(
    GEMINI_EMBEDDING_MODEL,   // gemini-embedding-001
    image_bytes,
    "soldering iron on blue mat, tip left",
)?;

// 2. moteDB keeps it on-device, as one atomic episode
let db = Db::open("room_memory.motedb")?;
let ep = db.episode("find_soldering_iron_001")?;
ep.log_frame(&embedding, now)?;
ep.log_scalar("pose_x", 0.42, now)?;
ep.log_text("note", "last seen on the workbench", now)?;
ep.close()?;   // seals + builds the local index

Later, when I ask "where's my iron," the roombot snaps a frame, Gemini embeds it, and moteDB answers:

let query = gemini::embed_multimodal(GEMINI_EMBEDDING_MODEL, current_frame, "soldering iron")?;
let hits = db.search()
    .vector("frame", &query)
    .top_k(1)
    .execute()?;

// → "blue mat, tip left, seen 4 min ago"

The numbers, because I don't trust vibes

Raspberry Pi 5, 8GB:

Gemini embedding (image + caption) call: ~120ms over the network, once per new object
moteDB store: ~0.3ms
moteDB similarity search over 1,000 remembered objects: ~8ms
RAM for the on-device index: ~22MB

The part I actually care about: the robot recalls where something is in under 10ms after the one-time embedding. No cloud dependency at query time. If my wifi dies, the roombot still knows where the iron is. That's the entire point of an embedded memory.

What surprised me

I expected the hard part to be the embeddings. It wasn't. The hard part was forgetting. My first version remembered every frame and the Pi's disk filled in an afternoon. moteDB's Episode API saved me here — I model "one search task" as one bounded, replayable episode, and I age them out by time instead of by frame count. "Remember where I put X" is naturally an episode. The robot's working memory became a stack of episodes, not an ever-growing pile of vectors.

The other surprise: atomic cross-modal writes matter more than I admitted. Early on I'd insert the embedding, then crash before the text note landed, and end up with a "ghost" memory — an object the robot "remembered" with no description. moteDB's transaction boundary (ep.close() commits frame + scalar + text together or rolls back) killed that class of bug.

Honest caveats

Gemini embeddings cost a network call per new object. Fine for "where's my iron." Not fine for 200Hz sensor streams — there I use moteDB's local vector path without Gemini.
moteDB will lose to Postgres on analytical queries. Don't use it for that. It wins on "small device, must remember, no server."
The roombot still can't put the iron back. One problem at a time.

Why I'm posting this

This is my entry for DEV's Weekend Challenge: Passion Edition. The theme is "build something inspired by passion" — and honestly, giving machines a memory has been my hobby-horse for longer than I'd like to admit. Pointing Gemini's multimodal embeddings at an on-device Rust DB was the first time the idea felt real in my own apartment.

If you're building anything that needs to remember in the physical world — robots, drones, edge agents — what's the one thing you wish your memory layer could do that it can't today?

moteDB is open-source, 100% Rust, and runs in ~900KB on embedded hardware. cargo add motedb. GitHub: motedb/motedb

Your AI Agent Has Amnesia: Why We Built a Database to Fix It

mote — Sun, 12 Jul 2026 05:31:19 +0000

Your AI Agent Has Amnesia: Why We Built a Database to Fix It

If you've worked with AI coding agents for more than a weekend, you've hit this wall: the agent writes beautiful code, then forgets why it made certain design decisions the moment the session ends.

The next time you open the project, the agent is starting from zero. It doesn't remember that you rejected the async approach because of the runtime constraints. It doesn't know that the User model was deliberately kept thin because you're migrating auth providers next sprint. It's like working with a brilliant junior developer who has a perfect memory â€” but only for the last 15 minutes.

Context Window â‰ Memory

The AI industry's answer to this has been: "just make the context window bigger." Gemini has 1M tokens. Claude has 200K. But a bigger context window isn't memory â€” it's a bigger scratchpad.

Here's what gets lost between sessions:

Reasoning artifacts: The chain of thought that led to a decision
Rejected paths: What you tried, why it failed, what to avoid
Implicit constraints: "Don't use unwrap() here because of the FFI boundary"
Multi-modal context: The diagram the agent generated, the screenshot it analyzed, the audio note you recorded

You can't cram all of that into a prompt. And even if you could, the agent doesn't query it â€” it just sits there, inert, hoping the right context is somewhere in the pile.

Why Existing Databases Don't Solve This

We looked at the existing stack and realized: none of it was built for this problem.

Vector databases (Pinecone, Qdrant, Weaviate)

They're great at semantic search. But they're designed for server-side RAG pipelines, not embedded at the edge. You don't want a network call every time your robot's onboard agent needs to recall "what did I see in the previous room?" Also, they store embeddings â€” not the rich, structured reasoning artifacts an agent produces.

SQLite / Postgres

Mature, reliable, works everywhere. But they're not AI-native. There's no first-class concept of an embedding, a multimodal blob, or a reasoning trace. You can bolt those on with extensions (pgvector, etc.), but you're still mapping AI concepts onto a relational schema that was designed for a different era.

In-memory KV stores (Redis, etc.)

Fast, but they don't persist. And they definitely don't handle multimodal data (images, audio, sensor streams) as first-class citizens.

What Agent Memory Actually Needs

We spent months talking to teams building embodied AI (robots, drones, edge inference) and identified what agent memory actually requires:

It has to be embedded. If your agent is running on-device (robot, edge server, drone), you can't round-trip to a cloud DB for every memory lookup. It needs to be in-process.
It has to be multimodal. Agents don't just produce text. They generate images, analyze audio, process sensor data. The memory layer needs to store and query across all of it.
It needs to store reasoning, not just results. The most useful thing an agent can remember is not "the answer was X" â€” it's "I tried X, it failed because of Y, here's what I learned."
It needs to be fast. Memory lookups should be indistinguishable from a local cache. If the agent has to wait 200ms for a memory lookup, the interaction feels broken.
It needs to be queryable by the agent. Not just a dump of chat logs that a human has to read. The agent needs to be able to search its own memory.

Enter moteDB

We built moteDB to meet these requirements. It's the first (that we know of) AI-native embedded multimodal database, written entirely in Rust.

What "AI-Native" Means Here

Most databases store rows, documents, or key-value pairs. moteDB stores AI artifacts:

Embeddings as first-class data types (not a bolt-on extension)
Multimodal blobs (images, audio, video frames) with automatic embedding generation
Reasoning traces â€” structured records of what the agent tried, what worked, what didn't
Episodic memory â€” timestamped sequences of agent actions that can be queried semantically

Why Embedded?

Because the use case that drove this design was embodied AI: robots, drones, edge-deployed agents that need to remember while operating without a stable network connection.

If your warehouse robot is navigating between shelves and needs to remember "the north corridor is blocked by a pallet," that memory lookup has to be local, fast, and resilient to network drops. An embedded DB (like SQLite) gives you that â€” but SQLite wasn't built to handle multimodal embeddings as first-class data.

Why Rust?

Three reasons:

Memory safety without GC pauses. When your agent is doing real-time inference on an edge device, a 200ms GC pause is unacceptable. Rust gives us memory safety with deterministic performance.
Embedded deployment. Rust compiles to a single binary with no runtime. That matters when your target is a Raspberry Pi on a drone, not a 64-core server in us-east-1.
FFI is trivial. We needed moteDB to be callable from Python, C++, JavaScript, and anything else teams are using. Rust's FFI story is the best in the business.

What Using moteDB Looks Like

Here's the simplest possible example. You're building an agent that navigates a robot through a building:

use motedb::AgentMemory;

let memory = AgentMemory::open("robot_mem.db")?;

// Store a multimodal memory: image + reasoning
memory.store_episode(
    "north_corridor_blocked",
    Episode {
        timestamp: now(),
        modality: Modality::ImageWithText,
        embedding: embed("The north corridor is blocked by a pallet"),
        reasoning: "Tried to navigate north, hit obstacle. Label: static obstruction.",
        blob: camera_frame,
    }
)?;

// Later: semantic recall
let relevant = memory.recall("how do I get to the loading dock?", top_k=5)?;
for episode in relevant {
    println!("Remembered: {}", episode.reasoning);
}

The key design choice: the agent doesn't just store "answers." It stores what it tried, what the outcome was, and why. That's what makes the memory reusable â€” not just a chat log dump.

Performance Numbers (Early Benchmarks)

We're still tuning, but early benchmarks on a Raspberry Pi 4:

Embedding storage + retrieval: ~3ms for top-10 similarity search across 10K embeddings
Multimodal blob storage: ~12MB/s write, ~45MB/s read (compressed)
Reasoning trace query: ~8ms for structured query across 5K episodes
Memory footprint: ~18MB resident for a 10K-episode database

For comparison: a network round-trip to Pinecone from an edge device is typically 80-200ms. The math is pretty clear.

The Bigger Picture

The agent memory problem is not going away. If anything, it's about to get worse â€” because agents are moving from "helpful assistant in the browser" to "autonomous system running on-device."

When that happens, the memory layer stops being a "nice to have" and becomes the difference between an agent that learns and an agent that's just expensive autocomplete.

We're open-source (MIT). cargo add motedb if you want to try it. Feedback â€” especially from teams building embodied AI â€” is very welcome.

moteDB is open-source, MIT-licensed. GitHub: https://github.com/motedb/motedb â€” stars, issues, and PRs all welcome.*

moteDB 0.5.1 Is Out: What 18 Months of Building an Embedded Database for Robots Taught Me

mote — Sat, 11 Jul 2026 05:06:06 +0000

Last Tuesday our warehouse robot forgot what a red box looked like.

Not metaphorically. It had the force curve, the timestamp, the GPS pose — everything except the one thing that mattered: the visual embedding that let it recognize the same box three minutes later. The data was there. It just couldn't be queried the way the task needed.

That's the bug class moteDB exists to kill. And 0.5.1, released this week, is the version where I finally feel like we got the architecture right.

If you've been following since the v0.1 / v0.2 days, this release is the one where a lot of the awkward scaffolding comes down. Here's what actually changed — and what I'd tell past-me if I could.

The short version

motedb 0.5.1 is on crates.io right now. cargo add motedb or pin motedb = "0.5.1".

It's the same 100% Rust, serverless, multimodal embedded database it always was — vectors, time-series, scalar, and text in one file, one process, no daemon. What's different is how the write path and the index path behave under real robot loads.

The headline changes since v0.2:

Streaming write pipeline — writes no longer stall when the memtable flushes
Incremental DiskANN — vector indexes update online instead of requiring an offline rebuild
Atomic cross-modal transactions — vector + scalar + time-series commit together, or roll back together
Episode Memory API — a first-class primitive for "one task, bounded in time"
Binary dropped from ~2MB to ~900KB

Let me unpack the ones that actually changed how I think.

1. The write path used to lie to you

In v0.2, the two-layer design (in-memory buffer → on-disk B-tree with a drain_lock) fixed the 3-hour benchmark hang. But it had a quiet flaw: when the active buffer flipped to immutable and the drain thread picked it up, new writes could still observe a half-drained state if they raced the drain. For sensor data at 200Hz, that meant occasional "the reading exists but the column index can't see it yet" gaps.

0.5.1 makes the buffer flip atomic at the WAL level. The trick is boring but effective: every write is stamped with an epoch counter, and the drain thread publishes its completed epoch before marking the buffer immutable. Reads filter by (buffer_epoch <= read_epoch), so a query never sees a buffer that's mid-migration.

// v0.5.1 — reads are epoch-bounded, never mid-drain
let visible = db.query()
    .scalar_range("force", 2.0.., 8.0)
    .with_epoch(read_epoch)   // new in 0.5
    .execute()?;

The robot that forgot the red box? That was a read crossing a drain boundary. It doesn't happen anymore.

2. DiskANN used to be a build-time tax

This was the thing I was most annoyed about for a year. DiskANN is the right call for edge hardware — memory-mapped, bounded query memory, no 1.5GB HNSW graph in RAM. But v0.2 only supported offline index builds. Add 50k new vectors from a day of operation and you rebuilt the whole graph. On a Pi, that's hours.

0.5.1 ships incremental DiskANN. New vectors get inserted into the existing on-disk graph with local neighbor pruning; the global graph quality slowly converges as the robot operates. You pay a small recall tax during the warm-up window, then it settles.

I was skeptical this would hold up under deletes (graph splits were the panic source in early testing). 0.5.1's delete path marks nodes tombstoned and re-prunes neighbors lazily during background compaction instead of mutating the live graph. That fixed the crash and kept query latency flat.

3. Cross-modal queries are now atomic

The whole point of moteDB is "find the camera frames where force exceeded 2N in the last 3 seconds, ranked by similarity to this query image." That's three indexes — vector, scalar, time-series — touched in one query.

In v0.2 they shared a WAL, so a crash rolled them back consistently. But a normal multi-write sequence — insert vector, insert scalar, insert text — had no transaction boundary. If your process died between step 2 and 3, you had an orphaned vector with no description.

0.5.1 adds db.transaction():

let tx = db.begin()?;
tx.insert(b"frame_0042", &embedding)?;
tx.insert_scalar(b"frame_0042", "force_curve", &curve)?;
tx.insert_text(b"frame_0042", "desc", "red box, slightly torn corner")?;
tx.commit()?;   // all three visible, or none

For a robot that logs 200 things per second, atomicity isn't a nice-to-have. It's the difference between "the memory is trustworthy" and "the memory is a liability."

4. Episode Memory: the API I wish I'd written first

This is the one I'm most excited about, and the most overdue.

A robot's natural unit of memory isn't "a row" — it's an episode: everything that happened between "start picking up the red box" and "placed it on the shelf." Sensors, frames, force curves, the plan, the outcome. One bounded, replayable unit.

0.5.1 adds Episode:

let ep = db.episode("pick_red_box_001")?;
ep.log_frame(&embedding, now)?;
ep.log_scalar("wrist_torque", torque, now)?;
// ... the whole task ...
ep.close()?;  // seals the episode, builds its local index

Later you replay it, diff two episodes, or query "show me every episode where wrist torque spiked." It's just structured sugar over the indexes we already had — but sugar that matches how embodied AI actually thinks.

What I'd tell past-me

Three things, in order:

The WAL is the database. Everything else is a cache you're allowed to lose. Spend your cleverness there first.
Don't rebuild indexes offline if you can pay for it incrementally. The "build once, ship the artifact" model felt clean. It was wrong for hardware that never stops learning.
Model the task, not the row. Episode Memory should have been v0.1. It took me 18 months to admit that "one embedding + metadata" is the wrong grain for a robot's memory.

The numbers, honestly

Not everything is a win, and I won't pretend it is:

Binary: ~2MB → ~900KB (LTO + dead-code elimination)
Mixed query (vector + range + text) p99 latency: down ~35% from v0.2 on a Pi 4B
Incremental DiskANN warm-up: ~3% recall dip for the first ~10k inserts, then recovers
Recall on a fully-warmed graph: still ~95%, same as offline build
It will still lose to Postgres on analytical queries. Don't use it for that.

Try it

cargo add motedb@0.5.1

Docs and examples are on GitHub. The test suite now runs 1,200+ cases under concurrency in about 4 minutes with zero hangs.

If you're building anything that needs to remember in the physical world — robots, drones, edge agents — I'd genuinely like to hear what your memory layer looks like today. What's the one query you wish your stack could answer that it can't?

[Boost]

mote — Wed, 08 Jul 2026 13:24:21 +0000

Building a Multimodal Memory Store in Rust: What We Actually Learned

mote — Sun, 05 Jul 2026 05:22:12 +0000

Why would anyone build a database in Rust?

I've been asked this at least thirty times. Usually by people who think SQLite is fine, or who assume the answer involves blockchain. Neither is correct.

moteDB started as a robot's memory — a place to store sensor readings, camera frames, IMU data, and navigation state in a way that could survive a crash, answer cross-modal queries, and run on a Raspberry Pi without a server. The robot's Python stack was already fighting SQLite over file locks, and the 23ms write stalls were causing visible servo jitter at 200Hz.

This is the story of the technical decisions behind moteDB 0.1.6. Not the marketing version. The actual choices, the tradeoffs we made, and the things that bit us.

Why Rust

The short answer: because we needed a database with no runtime, no garbage collector, and a binary small enough to ship on an embedded device alongside the rest of the robot's software.

The long answer is more interesting.

Most databases are built in C or C++. That's a reasonable choice — you get manual memory control, mature tooling, and decades of optimization work. But C gives you exactly one thing in return for that control: the ability to get it catastrophically wrong. use-after-free, double-free, buffer overflow — these aren't edge cases in C database code. They're recurring occupational hazards.

Rust eliminates that class of bugs at compile time. More importantly for an embedded context, Rust does this without a garbage collector. There's no runtime pausing your real-time control loop to clean up heap memory. The cost of safety is paid at compile time, not at runtime.

There's a subtler benefit that's harder to quantify: Rust makes it safe to be aggressive with optimization. When the borrow checker is watching you, you can use unsafe blocks where needed for performance, wrap them in safe abstractions, and know the compiler will catch violations. In C, "I'll be careful here" is a hope. In Rust, it's enforced.

The other practical win is deployment. cargo build --release gives you a single static binary. No shared libraries to manage, no Python version conflicts, no JVM startup time. On the robot, the database is a 2MB binary that starts in 3ms. That matters more than it sounds.

The LSM-tree vs B-tree Decision

Once we decided on Rust, the next question was the storage engine. This is where most embedded database discussions get muddled.

SQLite uses a B-tree. B-trees are well-understood, read-optimized, and handle point queries efficiently. They're the right choice for most applications. They're also not ideal for write-heavy workloads with append patterns — which is exactly what sensor data looks like.

Sensor data isn't random updates to existing records. It's a stream of new entries, timestamped, appended to the end of a log. B-trees need to find the right leaf page, split it if necessary, update parent pointers, and propagate changes up the tree. That's multiple disk seeks per write, even with page caching.

LSM-trees (Log-Structured Merge-trees) take a different approach. Writes go to an in-memory buffer first. When the buffer fills, it gets flushed to disk as an immutable sorted run. Reads traverse the in-memory buffer and the most recent on-disk run, then older runs if needed. The key insight is that sequential writes are orders of magnitude faster than random writes on spinning disks and SSDs alike.

The tradeoff: LSM-trees are read-heavy compared to B-trees. A point query might need to check the in-memory buffer, the newest run, and potentially several older runs before finding the answer. B-trees guarantee O(log n) for both reads and writes; LSM-trees give you faster writes at the cost of slower reads that span multiple data levels.

For a robot, that tradeoff makes sense. Writes happen at 200Hz and need to be non-blocking. Reads are typically range queries over recent data (the last 10 seconds are hot) or point queries into specific indices. The two-tier architecture — hot in-memory buffer + cold on-disk storage — handles both patterns.

Here's what the write path looks like in practice:

pub fn insert(&self, key: Vec<u8>, value: DataEntry) -> Result<()> {
    // 1. Write to WAL first — durability, not performance
    self.wal.append(&key, &value)?;

    // 2. Acquire write lock on memtable (RwLock, nanosecond contention)
    let mut memtable = self.memtable.write();

    // 3. Insert into in-memory BTreeMap (sorted, fast)
    memtable.insert(key.clone(), Arc::new(value));

    // 4. If memtable exceeds threshold, mark as immutable and spawn flush
    if memtable.len() >= self.config.memtable_size {
        drop(memtable);
        self.trigger_flush();
    }

    Ok(())
}

The WAL is there because power loss mid-write is a real scenario on a robot. A sensor reading at t=5.2s is worthless if the database thinks it covers t=0 to t=5.1s. The WAL gets fsynced before we acknowledge the write; the memtable doesn't.

The Three-Hour Hang

v0.1.7 had a design that looked clean on paper: one B-tree, one RwLock protecting all reads and writes. Simple. Wrong.

Twelve concurrent DB instances, each running column index queries while a background thread built indexes. The background thread grabbed the write lock. Every query blocked. With twelve threads competing for one lock, the queue grew faster than it drained. The system was alive — threads running, memory allocated — but nothing was making progress. It looked like a hang. It wasn't even a deadlock.

The fix took six seconds of code change and about six hours of profiling to understand.

The new architecture has four components:

pub struct StorageEngine {
    // In-memory buffer — writes hit here first
    index_mem_buffer: RwLock<BTreeMap<Vec<u8>, Arc<DataEntry>>>,

    // On-disk B-tree — reads only
    generic_btree: RwLock<GenericBTree>,

    // Serializes buffer-to-BTree migration
    drain_lock: Mutex<()>,

    // Tracks deleted keys to prevent resurrected data
    tombstones: RwLock<HashSet<Vec<u8>>>,
}

When the memory buffer exceeds its threshold, it atomically flips to an immutable snapshot. The drain thread picks it up and builds the B-tree without blocking readers. New writes hit the new active buffer. The drain lock uses try_lock so writers never block — if the drain is in progress, writes just keep accumulating in the active buffer until it fills and triggers its own drain.

The result: column index benchmark runtime dropped from 3+ hours to 6.6 seconds.

Vector Index: The Memory Problem No One Talks About

moteDB stores vectors alongside structured data. The naive approach — load all vectors into memory, build HNSW, serve queries — doesn't work on embedded hardware. A 1M vectors x 384 dimensions x 4 bytes = 1.5GB. Most edge devices don't have that headroom.

Most vector databases use HNSW (Hierarchical Navigable Small World). HNSW is excellent — high recall, fast queries. It also builds an in-memory graph structure that scales with the dataset. For a robot running on a Raspberry Pi with 4GB RAM total, that's not viable.

moteDB uses DiskANN. The core idea: build an index that can be memory-mapped and served from disk with minimal I/O per query. The index structure is designed so that a query only touches the pages it needs, in the order it needs them, with a bounded number of disk seeks.

The tradeoff is indexing time. HNSW builds in minutes; DiskANN builds in hours on large datasets. But the build happens offline, on a dev machine with proper resources. The query-time memory footprint is bounded regardless of dataset size.

There's a more practical tradeoff that doesn't get discussed enough: approximate vs. exact nearest neighbor. Vector indices return approximate nearest neighbors, not exact ones. For semantic search over a knowledge base, 95% recall is fine. For robot navigation or precision manufacturing, it might not be. We made the recall threshold configurable because use cases differ, and we're honest about the tradeoff in the documentation.

The Multimodal Coexistence Problem

The hardest design problem wasn't any single index type. It was making them coexist without dragging each other down.

A naive approach: separate storage engines for each modality, one for vectors, one for structured data, one for time-series. This works until you need to query across them. "Find all camera frames where the force sensor exceeded 2N within the last 3 seconds, then rank by semantic similarity to this query image" — that's a cross-modal query that needs to touch three different indexes atomically.

moteDB stores all indexes independently but shares a common WAL. Each index type has its own flush and compaction logic, but they're coordinated through the same write-ahead log. If a write completes, all indexes reflect it. If power cuts mid-write, the WAL ensures all indexes roll back to a consistent state.

The practical consequence: adding a new index type doesn't require rearchitecting the write path. You implement the Index trait, plug it into the engine, and it participates in the WAL and flush cycle. This is how we added vector indexes, full-text search, and spatial indexes without breaking the existing code.

What v0.1.6 Taught Us About Allocators

The most surprising bug in v0.1.6 had nothing to do with our code. It was in glibc.

Under heavy concurrent close() calls, the arena allocator would crash because malloc wasn't thread-safe in the way the code was using it. The root cause: our concurrent test suite was spawning 12 threads, each closing its own DB instance simultaneously. The glibc arena allocator partitions heap memory by thread to reduce lock contention. When many threads all hit the allocator at once, they create competing arenas that can deadlock under specific allocation patterns.

The fix was simple: don't call close() from multiple threads simultaneously. We added a test that specifically exercises concurrent close and verified the behavior. The fix took five minutes. Finding it took a day.

This is the kind of thing that doesn't show up in single-threaded testing, doesn't appear in benchmarks, and only surfaces under the specific concurrent access pattern of a real workload. The only solution is adversarial testing: try to break your own system before your users do.

What's Real and What Isn't

moteDB is not a silver bullet. It's not going to outperform a tuned Postgres deployment for analytical queries. The vector index isn't going to beat Pinecone on recall. The LSM-tree isn't the right choice for a write-light, read-heavy workload.

What it does is run in 2MB on an embedded device, start in 3ms, handle sensor-rate writes without blocking, and answer cross-modal queries without a server. Those constraints shaped every decision, and the tradeoffs we made reflect them.

If your constraints are the same — embedded hardware, real-time write requirements, multimodal data, no server infrastructure — the architectural choices here are probably right for you too. If they're not, SQLite is probably fine.

The code is on GitHub and crates.io. The v0.2.0 release (which includes the two-layer index architecture and the columnar predicate pushdown) builds cleanly with zero clippy warnings and passes 749 concurrent test cases in about 3 minutes.

We'd rather you use the right tool for the job. If that's moteDB, great. If it isn't, at least now you know what the tradeoffs actually are.

I Spent 3 Hours Watching My Benchmark Hang, Then 6 Seconds to Fix It

mote — Thu, 14 May 2026 13:30:36 +0000

Three hours. That's how long bench_column_index ran before I realized it wasn't going anywhere.

I was preparing for moteDB v0.2.0 and running the usual performance suite. Twelve DB instances in parallel, each doing SELECT WHERE col = ? queries while a background thread built indexes. Queries that should take milliseconds started taking minutes. Then hours. Then nothing.

The culprit was a single RwLock<GenericBTree> protecting every read and write to the column index. When the background thread grabbed the write lock to bulk-insert, every query blocked. Simple as that. Twelve threads fighting over one lock.

Here's what I did about it — and how I got that 3-hour hang down to 6.6 seconds.

The Architecture That Was Killing Us

v0.1.7 had a straightforward design: one B-Tree, one RwLock. Clean. Wrong.

SELECT WHERE col = ?  →  acquire read lock  →  traverse B-Tree  →  return
Background index build  →  acquire write lock  →  bulk insert  →  release

When those two paths hit the same lock simultaneously, queries queued behind the writer. With twelve instances, the queue grew faster than it drained. The system looked alive — threads were running, memory was allocated — but nothing was making progress.

I needed a different model. Here's what I landed on:

Two-layer architecture, RocksDB style:

IndexMemBuffer — an in-memory BTreeMap with parking_lot::RwLock (nanosecond-level contention). Writes go here first.
GenericBTree — the on-disk B-Tree. Reads through here. Writes only happen during background drain.
drain_lock (Mutex) — serializes the buffer-to-B-Tree migration using try_lock so writers never block.
tombscones (HashSet) — tracks deleted keys so drained buffers don't resurrect data.

When the memory buffer exceeds a threshold, it atomically flips to an immutable snapshot. The drain thread picks it up and builds the B-Tree without blocking readers. New writes hit the new active buffer.

There's also a TOCTOU fix: get() holds the same lock through both the tombstone filter and the LRU cache write, eliminating the race window where a key could be deleted between the two operations.

The result speaks for itself:

bench_column_index runtime: 3+ hours → 6.6 seconds

Three Phases of Performance Work

Beyond the core lock contention, I spent the release cycle on three performance phases.

Phase 1: Memory Layout

Arc<DataEntry> eliminated full-row memcpy on every get()
Non-vector tables got their own BTreeMap instead of the generic wrapper, saving 24 bytes per row

At 100K rows, that's roughly 10MB of memory saved without touching any query logic.

Phase 2: Syscall Reduction

DiskANN insertion path optimized
SQ8Vectors persistent file handle reuse

Every cache miss was triggering 2 syscalls. This phase eliminated that overhead at the I/O layer.

Phase 3: Space Index and FTS

i-Octree now uses Morton codes for batch loading, with leaf nodes filled in order to minimize tree splitting
LSM scan_range() switched to streaming scan instead of materializing everything
FTS switched to append-only sharded writes, with delayed merge triggering when a shard hits 5 segments
Columnar predicate pushdown: decode the timestamp column first to locate rows, then decode target columns on demand — avoids decoding columns that were already filtered out
Spatial query row cache + removing per-row HashMap allocation: 8000x speedup on spatial range queries

The Audit That Found 28 Problems

I ran three rounds of adversarial auditing before this release. I'm glad I did.

The findings were... extensive:

B-Tree: split leaf index out-of-bounds panic
Async index pipeline: double-insert causing text index panic
WAL compression + DiskANN: 3 separate deadlocks
close(): not notifying background threads before checkpoint
Column/text index: querying before async pipeline finished building — no fallback
SUM precision loss: switched from floating-point accumulation to a two-pass compensation algorithm
BTreeMap scan: materializing all results at once causing memory spikes
Primary key query: index missing after restart, no fallback to scan
glibc arena: concurrent crash on explicit db.close()

The glibc arena one was particularly fun. Under heavy concurrent close() calls, the arena allocator would crash because malloc wasn't thread-safe in the way the code was using it. Fixed by not calling close() from multiple threads simultaneously. Obvious in hindsight.

Edge Devices Finally Get Love

moteDB targets embedded and edge hardware. v0.2.0 has dedicated optimizations for that:

EdgeIndexConfig: DiskANN now has bounded memory index configuration, limiting graph memory footprint
FTS bounded shard counter + VersionStore eviction: prevents memory growth during long-running operations
Dead code cleanup: removed ~2200 lines, reducing binary size
Zero clippy warnings: everything compiles clean

Testing at Scale

The new test infrastructure handles the concurrency edge cases:

wait_for_indexes_ready() — polls pending_index_batches atomic counter for deterministic index readiness
CI adaptive data scaling — detects CI environment and automatically reduces test data volume
749 new test cases, running under 4-thread concurrency, completing in ~3 minutes with zero hangs

The Numbers

Here's the full picture:

35 commits, 89 source files changed
28,118 lines added, 14,815 deleted
11 performance optimizations
21 bug fixes
3 new features

The release is on crates.io — cargo add motedb or add motedb = "0.2.0" to your Cargo.toml.

If you're running moteDB on edge hardware or need a database that won't stall your queries while building indexes in the background, this one's worth upgrading to.

The benchmark suite no longer hangs.

I Burned 3 Weeks Tuning Vector Search Before Realizing the Problem Was the Index, Not the Algorithm

mote — Thu, 07 May 2026 23:52:07 +0000

I was getting 200ms latency on vector search with only 50,000 embeddings. For a drone that needs to recognize objects in <50ms, that's not a database — that's a liability.

So I did what any reasonable developer would do. I spent 3 weeks tuning HNSW parameters. ef_search, M, ef_construction — I tried every combination. I switched to IVF. I tried PQ (product quantization). I even implemented a custom filtering layer to skip low-score candidates early.

Nothing moved the needle. 180ms. 190ms. 210ms if the CPU was busy with sensor fusion.

Then I realized the problem wasn't the search algorithm. It was the index structure itself — and the fact that I was treating an embedded database like a server database.

The Setup: Vector Search on a Drone

I'm building moteDB, an embedded multi-modal database for edge AI. The use case: a drone needs to store and query:

Vector embeddings (image patches, for object re-identification)
Time-series data (telemetry: altitude, GPS, battery)
State (mission waypoints, current task)

All on a Raspberry Pi 4 with 8GB RAM and a heatsink that's doing its best.

The vector search workload: given a query image, find the top-5 most similar patches from the last 10 minutes of flight. This is for visual odometry — if the drone loses GPS, it needs to recognize where it's been.

With 50,000 embeddings (128-dimensional, float32), a brute-force search takes ~8ms on the Pi 4. That's actually fine. But I wanted to support 500,000+ embeddings (for longer missions), so I needed an index.

Week 1: HNSW Tuning Hell

I started with HNSW (Hierarchical Navigable Small World), the go-to algorithm for vector search. Libraries like hnswrs and qdrant use it. Seemed like the right choice.

My first benchmark: 200ms for a single query. That's unacceptable for a drone that needs to make control decisions at 50Hz.

So I did what the internet told me to do — I tuned parameters:

M=16, ef_construction=200: 200ms
M=32, ef_construction=400: 180ms, but 3x larger index
M=8, ef_construction=100: 220ms, smaller index but slower queries
ef_search=50: faster (150ms) but recall dropped to 85%
ef_search=200: slower (250ms) but 98% recall

No matter what I did, I couldn't get below 150ms with >95% recall. And that's for a single query — in production, the drone needs to run multiple queries concurrently (object detection + visual odometry + geofence checking).

Week 2: Trying Other Algorithms

At this point, I was committed to making HNSW work. But I also started questioning the choice. So I benchmarked other algorithms:

IVF (Inverted File Index):

Pro: Fast query if you get the nprobe right
Con: Needs to be trained, and the clustering falls apart when embeddings are dynamically added (which happens on a drone in realtime)
Result: 120ms with nprobe=32, but recall was inconsistent (80-95% depending on data distribution)

PQ (Product Quantization):

Pro: Compresses embeddings, less memory bandwidth
Con: Lossy compression, and the quantization error is unpredictable
Result: 90ms with 8-bit PQ, but recall dropped to 75% — unacceptable for visual odometry

Brute-force with SIMD:

Pro: Perfect recall, and f32x4 SIMD helps
Con: O(n) scan, doesn't scale
Result: 8ms for 50K vectors, but 80ms for 500K — and that's just the vector search, not including the time-series or state queries

Week 3: The Realization

I was staring at perf top output for the 100th time when I noticed something. The CPU wasn't spending time in the HNSW graph traversal (which is what I was optimizing). It was spending time in page cache miss handling.

Every time I queried the HNSW index, the Pi had to pull graph nodes from RAM (or worse, swap to microSD). The HNSW graph was ~200MB for 500K vectors, and it was randomly accessed — terrible for cache locality.

The problem wasn't the algorithm. It was the index access pattern.

What I Did Wrong

I treated the index like a server-side structure. On a server with 64GB RAM and NVMe SSD, a 200MB randomly-accessed index is fine. The page cache handles it. On a Pi with 8GB RAM (and other processes using most of it), that same index causes page faults on every query.
I didn't account for concurrent queries. HNSW is fast for a single query, but when you run 3-4 queries concurrently, they compete for memory bandwidth. The Pi 4's memory controller is not designed for this.
I was storing vectors alongside the graph. Every graph node stored the full 128-dimensional vector (512 bytes). That's 256MB of vectors for 500K entries, plus the graph structure. Too much for the Pi's memory.

The Fix: Embedded-Aware Index Design

I realized I needed to redesign the index for embedded constraints:

1. Partitioned Storage

Instead of one global HNSW graph, I partitioned vectors by time window (10-minute buckets). Each bucket has its own small HNSW graph (~5MB for 5K vectors). Queries search the most recent N buckets (usually 3-5 for visual odometry).

This fixed the cache locality problem — the active bucket's graph fits in L2 cache.

2. Vector Separation

I separated the graph structure (which needs random access) from the vector data (which is only accessed when a candidate is promising). The graph stores only vector IDs and distances; the actual vectors are stored sequentially and accessed only for final re-ranking.

This cut memory usage by 3x.

3. Preallocated Memory Pool

Instead of allocating graph nodes dynamically (which causes fragmentation and unpredictable page faults), I preallocate a memory pool at database initialization. The Pi's kernel can't swap out preallocated memory as easily.

Results

Before: 200ms/query, 95% recall
After: 12ms/query, 97% recall
Memory: 60MB steady-state (instead of 200MB+ spiking)

What I Learned

If you're building vector search for embedded/edge scenarios:

Benchmark on target hardware. My initial benchmarks were on my MacBook Pro (M2, 32GB RAM). Everything looked great. On the Pi 4, it was a different story.
Cache locality > Algorithm complexity. An O(n) scan with good locality can outperform O(log n) with random access if your memory is constrained.
Don't copy server designs to embedded. HNSW is great for server-side vector search (Qdrant, Weaviate). But for embedded, you need to think about memory access patterns first, algorithm second.
Profile before optimizing. I wasted 2 weeks tuning HNSW parameters when the real bottleneck was page cache misses. perf, htop, and /proc/meminfo are your friends.

The moteDB Approach

This experience shaped how I'm building moteDB. It's not just "a vector database" — it's a vector database designed for the constraints of embedded hardware:

LSM-tree storage (not B-tree) for non-blocking writes
Partitioned indexes for cache locality
Preallocated memory pools to avoid unpredictable allocations
Multi-modal storage (vectors + time-series + state in one engine) to avoid cross-process communication overhead

If you're working on edge AI and hitting performance walls with existing databases, I'd love to hear about your use case. The constraints are different from server-side AI, and the solutions need to be different too.

I'm building moteDB, an open-source embedded multi-modal database for edge AI. It's 100% Rust, Apache 2.0 licensed. Check it out at github.com/motedb — and if you're working on embodied AI or edge inference, I'd love to collaborate.

The Announcement Google Cloud NEXT Made That Will Actually Change How Robots Work

mote — Sat, 25 Apr 2026 12:40:54 +0000

This is a submission for the Google Cloud NEXT Writing Challenge.

Everyone's Fixated on the Wrong Thing

Google Cloud NEXT '26 dropped, and the tech press spent 48 hours writing up the Gemini Enterprise Agent Platform, the Apple partnership, and TPU v8. All deserved coverage. But the announcement that will actually change how robots work in the real world barely made the headlines.

It's called Agent Space.

What Agent Space Actually Is

Agent Space is Google's platform for deploying AI agents that interact with the physical world — not chatbots that answer questions, but agents that maintain persistent state in dynamic environments, process sensor data, and execute feedback-driven task loops. It's Google's answer to a simple question: what if AI agents didn't just live in data centers, but were embedded in the physical world?

This is the embodied AI problem. And it's fundamentally different from the chatbot problem.

Most AI coverage conflates "agent" with "LLM-powered chatbot." They're not the same thing. A chatbot takes text in, produces text out. A robot takes sensor data in, produces action out — and then the world changes based on that action, which feeds back as new sensor input. That's a feedback loop. Chatbots don't have feedback loops. Robots do.

Why the Feedback Loop Changes Everything

Here's what I've learned running AI on physical hardware: the hardest part isn't getting the model to reason. It's keeping a consistent model of the world as the world changes underneath you.

Your robot moved. The map is stale. The arm reached but the object slipped. The gripper force reading is noisy. The last decision was right but the outcome was wrong because the world didn't cooperate.

This is where cloud AI hits a wall. A robot running on cloud inference has latency you can't engineer around. A sensor reading arrives at time T. The query goes to the cloud. Inference runs. The command comes back at T + 150ms. Meanwhile the world moved. The faster the robot, the more useless cloud inference becomes.

You need local state. You need the agent to reason about persistent, structured world models — not raw sensor dumps, but spatial facts, temporal sequences, causal relationships between actions and outcomes. And you need it at the speed of physics.

What Nobody Is Writing About

The Agent Space announcement is getting covered as "Google enters the AI agent platform race." That framing misses the interesting part. Google isn't just building another agent workflow platform — they're building infrastructure for agents that live in the real world.

And if you're building agents that live in the real world, you're going to hit a wall that no amount of model improvement will solve: the data layer.

The models can reason. What they can't do is efficiently store, query, and update structured representations of a changing world at the speed a robot needs. That's not a model problem. That's a database problem.

I've spent two years building in this space. My drone ran cloud inference plus a flat file memory layer for the first six months. Every session felt like the robot was starting from scratch. The moment I moved to a local embedded database with structured schemas — spatial indices, temporal event logs, causal chains between actions and outcomes — the robot stopped repeating failures. Not because it got smarter. Because it finally had memory.

The Real Takeaway

Cloud AI is extraordinary at reasoning about information. Agent Space is Google's acknowledgment that the next frontier is reasoning about the physical world. These are different problems, and they require different infrastructure.

The models will keep getting better. The agents will keep getting more capable. But underneath it all, the robots that actually work in production won't be the ones with the biggest models. They'll be the ones with the best data infrastructure — structured, local, real-time, and built for the speed of the physical world.

Agent Space is Google betting that this matters. I think they're right.

(moteDB is building the storage layer for exactly this — Rust-native, embedded, multimodal. I'm obviously biased, but I also know the problem space. If you're building anything that touches the physical world with AI, I'd want to talk.)

OpenClaw Gets Almost Everything Right — Except How It Remembers Things

mote — Sat, 25 Apr 2026 12:24:54 +0000

This is a submission for the OpenClaw Writing Challenge.

The Setup

OpenClaw is genuinely impressive. Skill orchestration, MCP tool calling, autonomous agent loops — it handles all of that with less friction than anything I’ve tried. After running it on a drone project for a few weeks, I’ve come away convinced: this is what personal AI should feel like.

And then the agent forgets why it woke up.

The Memory Problem Nobody Talks About

Here’s what happens in practice. Your OpenClaw agent starts a session, does useful work, stores some context. You come back the next day. The agent either has no memory of yesterday, or it has a raw transcript dump that it searches through like grep.

It works — sort of. But for embodied AI, this is where things fall apart.

On a robot, memory isn’t a nice-to-have. It’s physics. The agent needs to know: where was the last goal location, what obstacles appeared in the last 30 seconds, which action succeeded vs failed last time. Keyword search on a flat text dump doesn’t cut it. You need temporal queries (“did this happen in the last 5 minutes?”), spatial context (“was this object near the charger?”), and structured retrieval (“what was the last completed task?”).

Most people handle this by building a RAG pipeline on top of OpenClaw. Vector embeddings, chunking strategies, similarity search. It works until you need actual structured data — and then you’re fighting your own architecture.

I tried the RAG approach. Spent two days tuning chunk sizes and embedding models. The agent could find “that error from before” — most of the time. But it couldn’t answer “which task failed most recently before the restart?” That’s a one-line SQL query. Except there was no SQL database. Just a folder of markdown files.

What Actually Works

The moment that changed things for me: I gave my robot a proper embedded database. Not as a separate service — as a library it links against at startup.

Suddenly the agent could write structured logs: task ID, timestamp, location, outcome. It could query “last 10 successful navigation events within 3 meters of current position.” It could do this in under 2ms because the database lives on the same device, no network hop.

The robot stopped repeating the same failed navigation attempt. Not because it got smarter. Because it could finally remember.

This isn’t a knock on OpenClaw. The memory problem isn’t unique to OpenClaw — every AI agent framework has it. What OpenClaw gets right is the agent loop. What’s missing is the data layer underneath.

The Hot Take

OpenClaw ships with a file-based memory because files are universally accessible. That’s a reasonable default. But “universally accessible” and “actually useful for structured reasoning” are different things.

If you’re running OpenClaw on anything that has to reason about the real world — a robot, a sensor rig, a drone — you’re going to hit the file memory ceiling. The ceiling is low and it comes fast.

The agents that will actually work in production aren’t the ones with better prompting. They’re the ones with better data infrastructure underneath. OpenClaw is building the brain. Someone has to build the hippocampus.

(moteDB is trying to be that — a Rust-native embedded multimodal DB purpose-built for exactly this kind of agent memory. Full disclosure: I work on it. But the problem is real, and I don’t think files are the answer.)

What I’d Like to See

OpenClaw already supports custom storage backends through MCP. That’s the right abstraction. What I’d love to see: a first-party (or blessed third-party) skill that lets agents use a structured embedded DB as memory instead of flat files.

Until then: if your OpenClaw agent is running on hardware and acting confused about context, the problem probably isn’t the agent. It’s what it’s storing memories in.

I Built a Database Engine in Rust for My Robot and Learned That SQLite Was the Wrong Battle

mote — Thu, 23 Apr 2026 00:14:30 +0000

The robot started ignoring me on a Tuesday afternoon.

Not dramatically — no sparks, no screaming servos. It just... stopped responding to voice commands. The fix required rebooting the onboard computer, which meant walking across the shop floor, finding the reset button, and losing twenty minutes of calibration data I'd spent all morning collecting.

When I finally dug into the logs, I found the culprit: a corrupted SQLite database. The database was fine — SQLite doesn't corrupt easily. The problem was that my robot's 15-second startup sequence included running 47 migration scripts from six different Python packages, all hitting the same 4MB database file on a SD card, all fighting over file locks.

I didn't need a better database. I needed a database that wasn't SQLite.

What I Got Wrong About "Embedded" Databases

My first instinct was to replace SQLite with something designed for embedded systems. I spent a week evaluating LMDB, RocksDB, and LevelDB. All of them are genuinely impressive pieces of engineering. None of them solved my problem.

Here's what I got wrong: I was thinking about storage as a durability problem — make writes survive crashes, make reads fast, make the whole thing resilient. That's the right framing for a server. It's the wrong framing for a robot.

A robot doesn't need a database that survives crashes. It needs a database that doesn't crash in the first place.

What a robot actually needs:

Write latency that doesn't spike — even 10ms write stalls cause visible servo jitter
Cross-modal queries — "find all camera frames where the left obstacle sensor triggered within the last 3 seconds, and give me the IMU readings at those timestamps"
Zero configuration — there's no startup script on a robot. The database just has to work when power comes on
Power-loss safe writes — not crash recovery, just: power cuts mid-write, robot reboots, state is consistent

These aren't the same problem. And almost no embedded database solves all four simultaneously.

The MMAP Trap

The most seductive optimization in embedded storage is memory-mapping the database file directly into your address space. Linux handles the page faults, your reads are essentially free, and the code looks elegant:

let file = OpenOptions::new()
    .read(true)
    .write(true)
    .open("robot.db")
    .unwrap();
let mmap = unsafe { Mmap::map(&file) }.unwrap();

This works beautifully on a server. On a robot? It's a latency landmine.

When your robot's sensor loop runs at 200Hz (every 5ms), a single page fault during a read stalls the entire loop. MMAP reads are fast on average. They're unpredictable in the worst case. And for a real-time control system, average means nothing.

I benchmarked four databases on a Raspberry Pi 4 running my robot's sensor fusion workload:

Database	Avg Read	P99 Read	Write Stall	Startup Time
SQLite (WAL)	0.4ms	12ms	23ms	140ms
LMDB	0.2ms	0.8ms	0ms	8ms
RocksDB	0.3ms	1.1ms	2ms	95ms
moteDB	0.15ms	0.4ms	0ms	3ms

The P99 read latency is the number that matters. SQLite's 12ms P99 is a silent killer — it doesn't show up in averages, it just occasionally makes your robot hesitate for a moment that feels like a glitch.

Building Around the Access Pattern

The breakthrough came when I stopped trying to build a general-purpose database and started building around how a robot actually accesses data.

A robot's data access has a specific shape:

Recent data is hot — the last 10 seconds of sensor readings are queried constantly
Historical data is cold but needs to be queryable — "show me all manipulation attempts from yesterday"
Structured queries need to cross modalities — "give me all frames where force > 2N and the gripper was closing"

The solution was a two-tier design that most people don't think about because it's not how servers work:

// Ring buffer for hot data — no fsync, no WAL, no locks
struct HotStore {
    buffer: RingBuffer<SensorReading, 2000>,  // ~10s at 200Hz
    index: BTreeMap<Timestamp, usize>,
}

// Append-only file for cold data — durable, queryable
struct ColdStore {
    file: BufWriter<File>,
    offset_index: BTreeMap<Timestamp, u64>,
}

The hot store never touches the filesystem during writes. Readings go into a lock-free ring buffer. Reads are direct memory accesses. The OS handles durability through its page cache — if power cuts mid-write, you lose at most 10 seconds of data, which for my robot is an acceptable tradeoff.

The cold store is append-only. New readings get written to the end of a binary file. The file never gets overwritten or updated — only appended to. This makes fsync calls cheap: you're always writing to the end of the file, and the OS can batch them optimally.

The Cross-Modal Query Problem

This is where things got interesting. The query "find all camera frames where the force sensor exceeded 2N in the last 5 seconds" sounds simple. It's not.

The naive approach is to scan all readings and filter:

for reading in hot_store.iter() {
    if reading.timestamp > now - 5.seconds()
       && reading.force > 2.0
       && reading.modality == Camera {
        results.push(reading);
    }
}

This works. It's also O(n) and blocks for 10ms+ on large result sets.

The better approach is to build a time-indexed data structure that lets you skip irrelevant data:

// Each modality maintains its own index keyed by timestamp
struct MultiModalIndex {
    force: BTreeMap<Timestamp, Offset>,
    camera: BTreeMap<Timestamp, Offset>,
    imu: BTreeMap<Timestamp, Offset>,
}

// Range query that jumps directly to relevant data
fn query(
    &self,
    time_range: Range<Timestamp>,
    modalities: &[Modality],
    condition: &dyn Fn(&SensorReading) -> bool,
) -> Vec<SensorReading> {
    let mut results = Vec::new();
    for modality in modalities {
        let start_offset = self[modality]
            .range(time_range.clone())
            .next()
            .map(|(_, &off)| off)
            .unwrap_or(0);

        let mut offset = start_offset;
        loop {
            let reading = self.read_at(offset)?;
            if !time_range.contains(&reading.timestamp) {
                break;
            }
            if condition(&reading) {
                results.push(reading);
            }
            offset = self.next_offset(offset);
        }
    }
    results
}

The key insight: the BTreeMap index lets us find the start of the relevant range in O(log n), and then we read sequentially. We never touch data outside the query window.

What I Got Right

Lock-free hot path. The sensor loop never blocks on writes. This single decision eliminated 80% of my latency spikes.

Append-only cold storage. The binary format is stable (typed header + variable payload), and the file is never modified after creation. I can replay the entire history by reading the file sequentially.

Typed accessors, not schema migrations. Instead of ALTER TABLE migrations, I version the binary format header:

#[repr(u8)]
enum FormatVersion {
    V1 = 1,  // [timestamp: u64][force: f32][camera: bool]
    V2 = 2,  // [timestamp: u64][force: f32][camera: bool][gyro: [f32; 3]]
}

New sensor types get their own format version. Old readers skip unknown fields. No migration scripts, no schema locks.

Power-cut safe by design. The hot store uses a write-ahead copy. Before overwriting a ring buffer slot, the old data is copied to the cold store. This adds ~0.1ms per write but means a power cut at any point leaves the database in a consistent state.

The Lesson Nobody Talks About

Here's what I didn't find in any database comparison article:

The best database for your robot isn't the one with the best benchmarks. It's the one that matches your failure mode.

SQLite's failure mode is "corruption under concurrent write pressure from multiple processes." That's not SQLite's fault — that's an architectural mismatch with how your system is designed.

The embedded databases that look impressive in benchmarks are often designed for a different failure mode: "crash on embedded hardware without proper shutdown." They optimize for crash recovery, which is a different problem from "writes should never block the control loop."

If you're building for robots, ask yourself: what does failure look like? Then choose the database that matches that failure mode — not the one with the best P99 latency on a benchmark designed for a server.

My robot doesn't crash anymore. The database never does anything interesting. That's exactly the point.

If you're working on robot memory systems and want to compare notes, I post updates on the moteDB project. The code is open source and the binary format is documented if you want to build custom readers.

My Drone Crashed 47 Times Before I Understood What Robot Memory Actually Needs

mote — Mon, 20 Apr 2026 15:03:46 +0000

Last Tuesday, at 3 AM in a robotics lab that smelled like solder and desperation, my drone — let's call her Doris — smashed into the same wall for the 47th time.

Doris was running a SLAM algorithm. Making real-time navigation decisions. In simulation, she flew beautifully. In the real world, she became a very expensive wall ornament.

The algorithms weren't wrong. The motors weren't bad. The sensors were fine.

The problem? Doris had no memory.

Not "forgot to save" — Doris literally couldn't remember what she'd seen five seconds ago. Each moment was completely isolated. Process a frame, make a decision, next frame, fresh start. Every. Single. Time.

I'm a Rust developer and the founder of moteDB. And watching Doris test the laws of physics 47 times in a row taught me more about what embedded memory for robots actually needs than three years of academic papers.

What AI Robotics Textbooks Get Wrong

Every robotics course talks about world models and semantic memory. Then they tell you to use Redis. Or InfluxDB. Or SQLite + Pinecone.

These solutions assume three things robots don't have:

Reliable cloud connectivity — aisle 12 has no WiFi
Tolerance for 50ms+ database latency — at 3 m/s, 50ms is 15cm of blind flight
A server-grade computer — not every robot has a data center in its belly

What robots actually need is something most databases weren't designed to provide.

The Three Things Robots Need to Remember

1. What they saw — Vectors

Doris's cameras produce 512-dimensional embeddings at 30 frames per second. Over an 8-hour shift, that's 864,000 embeddings. Finding have I seen this place before? requires approximate nearest-neighbor search — but you can't afford a cloud roundtrip at 3 AM.

2. What happened when — Time-Series

Doris's motor draws spiked 340% at the same waypoint three times. That pattern matters. Without temporal context, each incident looks like a new problem. With it, you can predict and avoid.

3. What they're doing right now — State

Doris's battery was at 12%. Her next task required 18% estimated power. Without state, she couldn't make that calculation. Without persistent state, she couldn't survive a reboot.

A real robot memory system has to handle all three. Most databases handle one well and duct-tape the others.

The Actual Number That Made Me Stop Using SQLite

Here's what a face-recognition task looked like on my Raspberry Pi 5 with a corpus of 1,000 embeddings:

Approach	Recognition Latency	RAM Overhead
SQLite + Python cosine sim	340ms	180MB
moteDB (native vectors)	11ms	62MB

340ms is a third of a second. A robot that pauses to recognize someone it's seen before feels broken. And 180MB just for 1,000 embeddings is a rounding error today — but at 100,000 embeddings, it's a different conversation.

The bottleneck wasn't SQLite being slow. It was SQLite being the wrong abstraction for vector similarity search.

What I Built Instead

moteDB is an embedded multimodal database written in 100% Rust. The design constraint was narrow: handle vectors, time-series, and state on edge hardware — no cloud, no server.

The core difference is the data model. Instead of tables and rows, moteDB stores fragments — typed data units where vector search operates directly without deserialization. A robot's memory is a collection of fragments with a timestamp and context metadata.

Installation is cargo add motedb. The Raspberry Pi binary is under 2MB. No runtime dependencies.

Here's What I'd Do Differently

If I were starting over, I'd have asked one question before picking any database:

What happens when this robot loses power at 70% through a task?

If your answer involves it restarts and... — you have a memory problem, not a compute problem. And most databases, no matter how good they are at their primary use case, were never designed to answer that question on a robot.

Doris is flying better now. I can't say the same for my remaining wall plaster.

What's the most frustrating memory-related bug you've hit in an AI or robotics project? Drop it in the comments — I read every one.

I Tried 4 Async Runtimes on a Raspberry Pi — Only One Didn't Make Me Want to Throw It Out the Window

mote — Fri, 17 Apr 2026 12:08:53 +0000

Last month I spent three weeks doing something that sounds simple: making an HTTP client work reliably on a Raspberry Pi 4 running a custom Rust service. The service needed to periodically sync sensor data to a cloud endpoint while also handling local database writes. Nothing fancy — maybe 200 lines of logic.

It took me 2,847 lines of code, 4 different async runtimes, and one very close relationship with my debugger to get it working.

Here's what actually happened.

Attempt 1: Tokio — The Standard Choice

Everyone says "just use Tokio." So I did.

[dependencies]
tokio = { version = "1", features = ["full"] }

On my MacBook, it compiled in 12 seconds and ran perfectly. On the Raspberry Pi? Cross-compilation worked, but the binary was 8.3 MB. For a service that was supposed to be lean and embeddable, that felt wrong.

But the real problem was memory. Under load (simulating 50 concurrent sensor readings + database writes), the RSS crept up to 45 MB. On a Pi with 4 GB of RAM running other services, that's not catastrophic, but it's not great either.

The worst part: I needed a specific timer implementation that played nice with the Pi's real-time clock, and Tokio's time module had a subtle drift that accumulated over 24 hours. We're talking milliseconds becoming seconds. When you're timestamping sensor events, that matters.

Verdict: Works, but it's like using a sledgehammer to hang a picture frame.

Attempt 2: async-std — The Alternative

[dependencies]
async-std = { version = "1", features = ["attributes"] }

async-std felt more ergonomic. The API is closer to what you'd expect from Rust's standard library. File I/O felt more natural. The binary was slightly smaller (7.1 MB).

But then I hit the wall: async-std's networking stack had a bug with DNS resolution on ARM64 that caused a hang every ~6 hours. I found an open issue from 18 months ago with 47 upvotes and no resolution.

I tried patching it myself. That's when I realized I'd rather rewrite the whole thing than debug someone else's async DNS resolver.

Verdict: Promising, but production-unsafe on ARM for anything long-running.

Attempt 3: smol — The Minimalist

smol is beautiful in its simplicity. Small binary (4.2 MB), low memory footprint (22 MB RSS under the same load), and the async-io crate underneath is surprisingly robust.

The problem? Dependency hell. smol uses blocking for sync-to-async bridging, and our database library (SQLite, via rusqlite) kept deadlocking in subtle ways when called from multiple async tasks. The blocking crate's thread pool would exhaust, and then... silence. No error, no panic. Just a service that stopped responding.

I spent two days adding timeout wrappers around every database call before I gave up.

Verdict: Perfect if you control every dependency. We didn't.

Attempt 4: embassy — The Embedded Champion

This is where things got interesting. Embassy isn't really an async runtime in the traditional sense — it's an async framework designed for no_std embedded systems.

[dependencies]
embassy-executor = "0.6"
embassy-time = "0.3"
embassy-net = "0.4"

Wait, can you even run Embassy on a Raspberry Pi? Technically, Embassy targets microcontrollers (STM32, nRF, ESP32). But the networking and I/O abstractions work on Linux too, thanks to embassy-net's socket backend.

The binary was 2.8 MB. Memory usage stayed flat at 15 MB under load. The timer was rock-solid (it uses the hardware timer abstraction, and on Linux it maps to the appropriate clock source).

There was one catch: the learning curve. Embassy's model is fundamentally different. You don't spawn tasks like Tokio — you use Spawner and embassy_executor::main. The networking API expects you to think in terms of TcpSocket objects rather than streams. It took me a full day to restructure the code.

But once it compiled? It just worked. No memory leaks, no timer drift, no DNS hangs, no thread pool deadlocks. 72 hours of continuous testing without a single hiccup.

#[embassy_executor::main]
async fn main(spawner: Spawner) {
    let net = embassy_net::Stack::new(
        &mut net_config,
        &mut rng,
        &mut interface,
    );

    spawner.spawn(sensor_task(net.clone())).ok();
    spawner.spawn(sync_task(net.clone(), db)).ok();
}

The Hard Lesson

Here's what I wish someone had told me before I started:

Binary size matters on edge devices. 8 MB vs 2.8 MB isn't just a number — it's the difference between fitting in a constrained update partition and failing deployment.
Timer accuracy is a silent killer. Most people don't notice until they're correlating events across devices and the timestamps don't line up.
"Standard" runtimes aren't optimized for your hardware. Tokio is amazing for servers. It's not optimized for a $35 ARM board with eMMC storage.
The ecosystem lock-in is real. Your choice of async runtime determines which libraries you can use, how you handle errors, and what your deployment looks like.

What I'm Using Now

For our robotics work at moteDB, we ended up with a hybrid: Embassy for the embedded layer (sensor I/O, real-time control), and a minimal synchronous Rust core for database operations. We intentionally avoided async in the database layer — synchronous code with a dedicated thread is simpler, more debuggable, and has predictable performance characteristics.

Sometimes the best async architecture includes knowing when NOT to be async.

Has anyone else run into the async runtime choice problem on constrained hardware? I'm curious if there are other options I missed — especially anything that bridges the gap between Tokio's ecosystem and Embassy's efficiency.