RocheDB v0.6 Planning: Safer Queries and Deeper Data Locality

This is not a release announcement.

It is a development note about the next areas I want to harden for RocheDB
v0.6.

RocheDB v0.5.x made the core data model more concrete. It added stellar
locality, coordinate locks, atomic embedded batch helpers, physical locality
reporting, a technical FAQ, and stronger documentation around the difference
between RocheDB and typical NoSQL systems.

The next step is not to add as many features as possible.

The next step is to make the existing direction easier to use, easier to test,
and harder to misunderstand.

Current state

RocheDB is a technical-preview NoSQL document/vector store written in Nim.

Its central idea is still:

Put data into meaningful coordinates, then use those coordinates at read time
to reduce unrelated work.

In RocheDB, a ring is not only a collection label. It is a locality
coordinate. Reads can start from that coordinate instead of scanning a broad
collection and filtering later.

v0.5.x added a second shape: stellar locality. A stellar lens lets related
rings be read together without copying payloads into one large document and
without pretending RocheDB is a relational join engine.

For example:

roche stellar attach --stellar=commerce/order/A-001 --ring=users/123
roche stellar attach --stellar=commerce/order/A-001 --ring=shops/1123
roche stellar attach --stellar=commerce/order/A-001 --ring=orders/A-001

roche get --stellar=commerce/order/A-001
roche get --stellar=commerce/order/A-001 --subring=shops

This made RocheDB's model more useful for ordinary application data, not only
AI/RAG experiments.

But v0.5.x also made the next weak spots easier to see.

Goal for v0.6

The theme I currently have in mind is:

safer query construction and deeper locality validation

That means four main areas:

• safer filter builders and typed query options;
• deeper physical locality tests and demos;
• better behavior around topology changes;
• clearer TLS and configuration UX.

Some of this may change as the implementation progresses. I would rather keep
the scope honest than pretend that a roadmap is a contract.

1. Safer filter builders

RocheDB already has prepared selections.

The selection syntax is a small GraphQL-like projection syntax:

let fields = prepareSelection("{ title author { name } }")
let first = db.query(firstId, fields)
let second = db.query(secondId, fields)

prepareSelection validates and parses the selection once. Embedded queries
reuse the parsed tree. Cluster nodes also keep a bounded cache of validated
selection trees.

That part exists.

The weaker part is filters.

Today, CLI and driver users commonly pass filters as JSON:

roche get --ring=docs/japan \
  --filter='{"status":"draft"}' \
  --selection='{ title }'

That is simple and explicit, but it is not always pleasant to use from
application code. It also pushes quoting and JSON construction details onto the
caller.

For v0.6, I want to improve this as a driver/API UX problem.

The target is not SQL prepared statements. RocheDB does not execute SQL.

The target is safer typed filter construction, such as:

filter.eq("status", "draft")
filter.eq("kind", "order")
filter.id(rawId)

The exact API will differ by language, but the goal should be the same:

• avoid string-concatenated JSON filters;
• make common equality filters easy;
• keep the wire/API representation explicit;
• keep RocheDB's read model centered on ring or stellar scope first.

This is especially important because RocheDB now has multiple language entry
points. The safer path should be the natural path in Rust, JavaScript /
TypeScript, PHP, Python, and future drivers.

v0.6 direction: prepared selection is already in the core. The next query
safety work is typed filter construction and driver-side ergonomics.

2. Deeper physical locality validation

RocheDB's first locality claim is logical:

Start from a meaningful ring so unrelated records do not enter the candidate
set.

But logical locality is not the whole story.

Storage layout matters. If a database says "locality", engineers will
reasonably ask:

• how records are grouped on disk;
• what happens after deletes;
• what happens after backfills;
• what happens when writes are interleaved;
• whether compaction preserves or improves locality;
• whether read latency changes before and after compaction.

v0.5.x added the first measurement surface:

roche locality --data=/var/lib/rochedb
roche locality --data=/var/lib/rochedb --metrics

The current embedded persistent store uses an append-only WAL. Before
compaction, physical record order mostly follows write order. If writes are
interleaved across many rings, the WAL can become physically interleaved too.

Compaction currently writes live particle records in stable (ringKey, seq)
order, grouping live records by ring in the compact snapshot.

That is a useful first step, but v0.6 should test it more seriously.

The test/demo cases I want to add are:

• random writes;
• delete-heavy workloads;
• backfill-heavy workloads;
• hot/cold ring skew;
• compact-before and compact-after locality metrics;
• compact-before and compact-after read latency;
• larger datasets where OS page cache and SSD behavior become visible.

The goal is not to create a perfect storage engine in one release.

The goal is to make locality drift observable and reproducible.

If RocheDB wants to make data locality a first-class concept, it should be able
to show when locality is good, when it is damaged, and when compaction improves
it.

3. Topology changes and remapping

There is another important design issue in the distributed side of RocheDB.

The current orbital ownership model can compute an owner from coordinate
metadata and topology. That is useful because placement can be deterministic
without a central directory for every lookup.

But the simplest owner segmentation has a known weakness: changing the number
of nodes can move too much data.

That is the same area where consistent hashing became important in other
systems. If membership changes cause unnecessary movement, the distributed
story is weaker than it should be.

For v0.6, I do not want to overpromise the final design.

The direction is:

improve topology changes so membership updates do not cause unnecessary data
movement.

Possible implementation directions include persisted arcs, virtual arcs, node
weights, or topology epochs.

The important constraint is that this should preserve RocheDB's coordinate
model. I do not want to throw away the orbital/locality idea and replace it
with a generic routing table. The better version should keep deterministic
coordinates while making membership changes less disruptive.

4. TLS and configuration UX

RocheDB already has TLS transport support.

But a database needs more than a feature existing somewhere in the code. It
needs a setup path that users can understand.

For v0.6, I want to improve configuration UX:

• clearer config file examples;
• local development config;
• production-like TLS config;
• Docker Compose examples where useful;
• easier explanation of what should be enabled by default;
• better documentation for server-style deployment.

The goal is not to claim enterprise production maturity too early. The goal is
to reduce unnecessary friction for people who want to try RocheDB as a real
service rather than only as an embedded library.

What I am not trying to do in v0.6

I do not want v0.6 to become a grab bag.

Some things are important but should not dominate this release:

• becoming a PostgreSQL replacement;
• becoming a MongoDB-compatible aggregation engine;
• adding arbitrary global secondary indexes;
• claiming financial-ledger correctness;
• adding every possible driver feature at once;
• hiding technical-preview boundaries.

RocheDB's strength is more specific:

Use meaningful placement to reduce retrieval working sets.

The v0.6 work should strengthen that idea instead of diluting it.

Why this matters

Many systems already know useful locality:

• tenant;
• user;
• region;
• product;
• topic;
• document domain;
• time window;
• application workflow;
• game world entity;
• prompt/context domain.

Usually that knowledge lives in application code, route structure, import
rules, or operational conventions.

RocheDB's bet is that the database should be able to use that information
directly.

That does not remove the need for indexes, caches, or good application design.
It gives the database a better starting point.

For AI/RAG systems, this can mean fewer unrelated chunks reaching reranking or
LLM context construction. For ordinary web systems, it can mean less
application-side filtering and clearer operational boundaries.

Feedback I would like

The areas where feedback would be most useful are:

• What should a safe filter builder look like in each language?
• How much filter expressiveness is enough before RocheDB starts becoming an ad-hoc query engine?
• Which physical locality benchmark cases would be most convincing?
• What topology-change behavior would make the distributed model more credible?
• What configuration examples would make RocheDB easier to try without overstating production readiness?

The current repository is here:

github.com/puffball1567/rochedb

The current technical FAQ is here:

github.com/puffball1567/rochedb/blob/main/docs/technical-faq.md

RocheDB is still early, but I think the direction is becoming clearer:

make locality explicit, measurable, and useful enough to become part of the
database design itself.