Modern applications, particularly AI-powered systems, often require more than simple data persistence.
They need searchable knowledge, durable state, background processing, and reliable recovery after interruptions.
These requirements frequently lead developers to integrate multiple storage technologies and services.
CedroDB is an experimental open-source project exploring another approach: combining several of these capabilities into a single local-first storage engine written in Rust.
The project provides native libraries for Python and Node.js/TypeScript, allowing developers to interact with the same underlying engine without deploying a separate database server.
This article explores the architecture, supported APIs, and practical examples.
1. Installing CedroDB
For Python 3.9+:
python -m pip install cedrodb==0.9.0
For Node.js 18+:
npm install cedrodb@0.9.0
Both SDKs interact with the underlying Rust engine through native bindings.
2. JSON Document Storage with Python
Let's start with a small orders collection.
from cedrodb import Database
with Database("./orders.cedro") as db:
orders = db.collection("orders")
orders.put_many({
"order:1": {
"customer": "Alice",
"status": "paid",
"total": 120
},
"order:2": {
"customer": "Bob",
"status": "pending",
"total": 35
},
"order:3": {
"customer": "Charlie",
"status": "paid",
"total": 80
}
})
print(orders.get("order:1"))
Documents are stored as JSON values identified by application-defined keys.
The put_many() method writes multiple documents atomically.
Querying with an index
from cedrodb import Database
with Database("./orders.cedro") as db:
orders = db.collection("orders")
orders.create_index("status")
result = orders.find_page(
[
{
"field": "status",
"op": "eq",
"value": "paid"
}
],
limit=20
)
for record in result["records"]:
print(record["key"], record["value"])
print("Index used:", result["used_index"])
CedroDB supports equality indexes, filtering, and keyset pagination.
Indexes are session-scoped and need to be recreated after reopening the database.
3. Atomic Transactions
An application may need to save a business document and an event together.
For example, an offline application could save a form and its corresponding outbox event in one transaction.
from cedrodb import Database
with Database("./offline.cedro") as db:
with db.transaction() as tx:
tx.put(
"forms",
"form:100",
{
"type": "inspection",
"status": "pending_upload"
}
)
tx.put(
"outbox",
"event:100",
{
"event_id": "event:100",
"type": "inspection.created",
"form_id": "form:100"
}
)
print(db.collection("forms").get("form:100"))
If the transaction fails before committing, its staged writes are not partially applied.
This is useful for offline-first workflows where application state and pending events must remain consistent.
However, committing an outbox event does not deliver it to an external service. The application still owns delivery, retries, and deduplication.
4. Monitoring Queries and Index Performance
CedroDB includes native diagnostics and query profiling APIs.
from cedrodb import Database
with Database("./monitor.cedro") as db:
memory = db.collection("memory")
memory.put_many({
"a": {
"tenant": "shop-1",
"text": "refund policy"
},
"b": {
"tenant": "shop-2",
"text": "shipping policy"
}
})
filters = [
{
"field": "tenant",
"op": "eq",
"value": "shop-1"
}
]
before = memory.profile_query(filters)
memory.create_index("tenant")
after = memory.profile_query(filters)
assert before["page"]["records"] == after["page"]["records"]
print("Before:", before["plan"])
print("After:", after["plan"])
retrieval = memory.profile_search(
text="refund",
filters=filters
)
print("Elapsed (µs):", retrieval["elapsed_micros"])
print("Plan:", retrieval["plan"])
print("Diagnostics:", db.diagnostics())
The profiling APIs expose query execution plans and native elapsed time.
Metrics describe the measured execution path, not end-to-end application latency or general superiority over another database.
CedroDB also provides a temporary benchmark utility:
python -m cedrodb benchmark --records 1000 --iterations 30
The benchmark uses an isolated temporary database rather than opening application data.
5. Full-Text and Vector Search
AI applications frequently need to retrieve relevant documents.
CedroDB provides three retrieval approaches:
- BM25 full-text ranking.
- Exact cosine similarity for supplied vectors.
- Hybrid retrieval using reciprocal rank fusion.
A search can look like this:
from cedrodb import Database
with Database("./knowledge.cedro") as db:
memory = db.collection("support-memory")
memory.put_many({
"doc:1": {
"tenant": "shop-1",
"model": "demo-v1",
"text": "Refund policy for online purchases",
"embedding": [1.0, 0.0]
},
"doc:2": {
"tenant": "shop-1",
"model": "demo-v1",
"text": "Shipping information and delivery",
"embedding": [0.0, 1.0]
}
})
memory.create_index("tenant")
results = memory.search(
text="refund policy",
vector=[1.0, 0.0],
vector_field="embedding",
text_fields=["text"],
filters=[
{
"field": "tenant",
"op": "eq",
"value": "shop-1"
},
{
"field": "model",
"op": "eq",
"value": "demo-v1"
}
],
limit=5,
max_scanned=10000
)
for hit in results["hits"]:
print(hit["key"], hit["score"])
The two-dimensional vectors above are synthetic and intended only to demonstrate the API.
CedroDB does not generate embeddings. Applications must obtain them from their chosen embedding model.
Vector retrieval is currently exact rather than ANN-based, so developers should evaluate candidate counts, scan budgets, latency, and memory usage before selecting it for a workload.
6. Durable Background Jobs
A common challenge in local applications is recovering background work after interruption.
CedroDB includes a persistent queue API with job leasing, retries, and acknowledgments.
from cedrodb import Database
with Database("./worker.cedro") as db:
queue = db.queue("reports")
queue.enqueue(
"report:1",
{"order_id": "order:1"},
max_attempts=3,
priority=10
)
job = queue.claim(
"worker-1",
lease_ms=30000
)
if job is not None:
completed = queue.ack(
job["id"],
job["lease_token"],
operations=[
{
"kind": "put",
"collection": "reports",
"key": job["id"],
"value": {"status": "ready"}
},
{
"kind": "put",
"collection": "outbox",
"key": job["id"] + ":ready",
"value": {
"event_id": job["id"] + ":ready",
"type": "report.ready"
}
}
]
)
print("Completed:", completed)
print(queue.stats())
The job acknowledgment, report result, and outbox record are committed together.
An expired or invalid lease token cannot acknowledge the job.
CedroDB does not automatically execute background workers, and external side effects may still require idempotency controls.
7. Where CedroDB Fits
Potential use cases include:
- Local AI assistants with persistent state.
- Offline applications with pending synchronization.
- Developer tools with embedded JSON storage.
- Searchable local knowledge bases.
- Desktop applications with durable work queues.
- Recoverable report-generation workflows.
The architecture is intentionally experimental.
CedroDB v0.9.0 currently has several important limitations:
- One owning process per database directory.
- Active documents and indexes remain in RAM.
- The append-only log has no compaction.
- No SQL, MVCC, or replication.
- No production-readiness guarantee.
- No built-in model execution or embedding generation.
SQLite and other established databases remain better choices for many workloads, especially those requiring SQL, larger-than-memory datasets, or mature production guarantees.
8. Source Code and Packages
Website: https://www.ltavarez.me/cedrodb
GitHub: https://github.com/uppy19d0/cedrodb
npm: https://www.npmjs.com/package/cedrodb
PyPI: https://pypi.org/project/cedrodb/
CedroDB is open source under the MIT license.
Final Thoughts
CedroDB explores a different approach to embedded storage: combining documents, search, agent checkpoints, and recoverable work through a Rust-powered engine.
It's still early, and practical evaluation is essential.
I'd particularly appreciate feedback about API ergonomics, storage architecture, failure recovery, performance testing, and use cases where this design might or might not make sense.
Would you prefer a unified embedded storage engine for AI agent state and local workflows, or separate specialized tools?
Contributions, suggestions, and technical discussion are welcome.
Top comments (0)