DEV Community: LTavarez

I Built container-memory-watch: Monitoring Memory Pressure in Node.js Containers 🦀

LTavarez — Mon, 19 Oct 2026 06:28:50 +0000

Memory management is one of the challenges of running Node.js applications in Docker, Kubernetes, and cloud environments.

A Node.js application can have relatively low V8 heap usage while its container is approaching its memory limit.

That's why I developed container-memory-watch, an experimental open-source tool that combines Node.js memory metrics with Linux cgroup v2 evidence using a Rust-based collector.

🔧 Key Features

Container memory pressure detection.
Node.js RSS, V8 heap, external memory, and ArrayBuffer monitoring.
Linux cgroup v2 memory measurements.
Configurable memory pressure thresholds.
OOM kill counter tracking.
Bounded diagnostic history.
Structured JSON logs.
Optional diagnostic report exports.
Native Node.js and Rust integration.

📦 Installation

npm install container-memory-watch

⚡ Quick Start

Start monitoring memory in your Node.js application:

import { startMemoryWatch } from 'container-memory-watch';

const memory = await startMemoryWatch();

console.log('Memory monitoring started');

// Call this during graceful shutdown.
await memory.stop();

By default, the monitor samples memory every five seconds, retains up to 120 samples, and uses an 80% pressure threshold.

Node.js metrics work without the Rust collector. Container-level evidence requires Linux cgroup v2 and the compiled collector.

📊 Generate Diagnostic Reports

import {
  startMemoryWatch,
  writeDiagnosticReport
} from 'container-memory-watch';

const memory = await startMemoryWatch();

const report = memory.report();

if (report) {
  console.log(JSON.stringify(report, null, 2));

  await writeDiagnosticReport(
    './memory-report.json',
    report
  );
}

await memory.stop();

Diagnostic reports can help investigate memory behavior over a bounded observation window.

🐳 Why It Matters

When operating containerized applications, memory pressure may originate from more than JavaScript objects.

Monitoring V8 heap usage alone may overlook other contributors to container memory consumption.

container-memory-watch brings these signals together to support more informed troubleshooting.

It does not automatically identify memory leaks, prevent OOM kills, or replace established observability platforms.

🔗 Source Code & Installation

GitHub Repository:

https://github.com/uppy19d0/container-memory-watch

Main Branch:

https://github.com/uppy19d0/container-memory-watch/tree/main

npm Package:

https://www.npmjs.com/package/container-memory-watch

README & Documentation:

https://github.com/uppy19d0/container-memory-watch/blob/main/README.md

💬 Feedback Welcome

The project is open source and currently experimental.

I'd appreciate feedback on memory monitoring strategies, Node.js observability, Rust integration, and container diagnostics.

How do you currently investigate memory pressure and OOMKilled containers in production?

Contributions and suggestions are welcome!

node #rust #opensource #kubernetes

Building CedroDB: A Rust-Powered Embedded Database for Python, Node.js, and AI Agents

LTavarez — Mon, 19 Oct 2026 06:24:00 +0000

Modern applications, particularly AI-powered systems, often require more than simple data persistence.

They need searchable knowledge, durable state, background processing, and reliable recovery after interruptions.

These requirements frequently lead developers to integrate multiple storage technologies and services.

CedroDB is an experimental open-source project exploring another approach: combining several of these capabilities into a single local-first storage engine written in Rust.

The project provides native libraries for Python and Node.js/TypeScript, allowing developers to interact with the same underlying engine without deploying a separate database server.

This article explores the architecture, supported APIs, and practical examples.

1. Installing CedroDB

For Python 3.9+:

python -m pip install cedrodb==0.9.0

For Node.js 18+:

npm install cedrodb@0.9.0

Both SDKs interact with the underlying Rust engine through native bindings.

2. JSON Document Storage with Python

Let's start with a small orders collection.

from cedrodb import Database

with Database("./orders.cedro") as db:
    orders = db.collection("orders")

    orders.put_many({
        "order:1": {
            "customer": "Alice",
            "status": "paid",
            "total": 120
        },
        "order:2": {
            "customer": "Bob",
            "status": "pending",
            "total": 35
        },
        "order:3": {
            "customer": "Charlie",
            "status": "paid",
            "total": 80
        }
    })

    print(orders.get("order:1"))

Documents are stored as JSON values identified by application-defined keys.

The put_many() method writes multiple documents atomically.

Querying with an index

from cedrodb import Database

with Database("./orders.cedro") as db:
    orders = db.collection("orders")

    orders.create_index("status")

    result = orders.find_page(
        [
            {
                "field": "status",
                "op": "eq",
                "value": "paid"
            }
        ],
        limit=20
    )

    for record in result["records"]:
        print(record["key"], record["value"])

    print("Index used:", result["used_index"])

CedroDB supports equality indexes, filtering, and keyset pagination.

Indexes are session-scoped and need to be recreated after reopening the database.

3. Atomic Transactions

An application may need to save a business document and an event together.

For example, an offline application could save a form and its corresponding outbox event in one transaction.

from cedrodb import Database

with Database("./offline.cedro") as db:
    with db.transaction() as tx:
        tx.put(
            "forms",
            "form:100",
            {
                "type": "inspection",
                "status": "pending_upload"
            }
        )

        tx.put(
            "outbox",
            "event:100",
            {
                "event_id": "event:100",
                "type": "inspection.created",
                "form_id": "form:100"
            }
        )

    print(db.collection("forms").get("form:100"))

If the transaction fails before committing, its staged writes are not partially applied.

This is useful for offline-first workflows where application state and pending events must remain consistent.

However, committing an outbox event does not deliver it to an external service. The application still owns delivery, retries, and deduplication.

4. Monitoring Queries and Index Performance

CedroDB includes native diagnostics and query profiling APIs.

from cedrodb import Database

with Database("./monitor.cedro") as db:
    memory = db.collection("memory")

    memory.put_many({
        "a": {
            "tenant": "shop-1",
            "text": "refund policy"
        },
        "b": {
            "tenant": "shop-2",
            "text": "shipping policy"
        }
    })

    filters = [
        {
            "field": "tenant",
            "op": "eq",
            "value": "shop-1"
        }
    ]

    before = memory.profile_query(filters)

    memory.create_index("tenant")

    after = memory.profile_query(filters)

    assert before["page"]["records"] == after["page"]["records"]

    print("Before:", before["plan"])
    print("After:", after["plan"])

    retrieval = memory.profile_search(
        text="refund",
        filters=filters
    )

    print("Elapsed (µs):", retrieval["elapsed_micros"])
    print("Plan:", retrieval["plan"])
    print("Diagnostics:", db.diagnostics())

The profiling APIs expose query execution plans and native elapsed time.

Metrics describe the measured execution path, not end-to-end application latency or general superiority over another database.

CedroDB also provides a temporary benchmark utility:

python -m cedrodb benchmark --records 1000 --iterations 30

The benchmark uses an isolated temporary database rather than opening application data.

5. Full-Text and Vector Search

AI applications frequently need to retrieve relevant documents.

CedroDB provides three retrieval approaches:

BM25 full-text ranking.
Exact cosine similarity for supplied vectors.
Hybrid retrieval using reciprocal rank fusion.

A search can look like this:

from cedrodb import Database

with Database("./knowledge.cedro") as db:
    memory = db.collection("support-memory")

    memory.put_many({
        "doc:1": {
            "tenant": "shop-1",
            "model": "demo-v1",
            "text": "Refund policy for online purchases",
            "embedding": [1.0, 0.0]
        },
        "doc:2": {
            "tenant": "shop-1",
            "model": "demo-v1",
            "text": "Shipping information and delivery",
            "embedding": [0.0, 1.0]
        }
    })

    memory.create_index("tenant")

    results = memory.search(
        text="refund policy",
        vector=[1.0, 0.0],
        vector_field="embedding",
        text_fields=["text"],
        filters=[
            {
                "field": "tenant",
                "op": "eq",
                "value": "shop-1"
            },
            {
                "field": "model",
                "op": "eq",
                "value": "demo-v1"
            }
        ],
        limit=5,
        max_scanned=10000
    )

    for hit in results["hits"]:
        print(hit["key"], hit["score"])

The two-dimensional vectors above are synthetic and intended only to demonstrate the API.

CedroDB does not generate embeddings. Applications must obtain them from their chosen embedding model.

Vector retrieval is currently exact rather than ANN-based, so developers should evaluate candidate counts, scan budgets, latency, and memory usage before selecting it for a workload.

6. Durable Background Jobs

A common challenge in local applications is recovering background work after interruption.

CedroDB includes a persistent queue API with job leasing, retries, and acknowledgments.

from cedrodb import Database

with Database("./worker.cedro") as db:
    queue = db.queue("reports")

    queue.enqueue(
        "report:1",
        {"order_id": "order:1"},
        max_attempts=3,
        priority=10
    )

    job = queue.claim(
        "worker-1",
        lease_ms=30000
    )

    if job is not None:
        completed = queue.ack(
            job["id"],
            job["lease_token"],
            operations=[
                {
                    "kind": "put",
                    "collection": "reports",
                    "key": job["id"],
                    "value": {"status": "ready"}
                },
                {
                    "kind": "put",
                    "collection": "outbox",
                    "key": job["id"] + ":ready",
                    "value": {
                        "event_id": job["id"] + ":ready",
                        "type": "report.ready"
                    }
                }
            ]
        )

        print("Completed:", completed)

    print(queue.stats())

The job acknowledgment, report result, and outbox record are committed together.

An expired or invalid lease token cannot acknowledge the job.

CedroDB does not automatically execute background workers, and external side effects may still require idempotency controls.

7. Where CedroDB Fits

Potential use cases include:

Local AI assistants with persistent state.
Offline applications with pending synchronization.
Developer tools with embedded JSON storage.
Searchable local knowledge bases.
Desktop applications with durable work queues.
Recoverable report-generation workflows.

The architecture is intentionally experimental.

CedroDB v0.9.0 currently has several important limitations:

One owning process per database directory.
Active documents and indexes remain in RAM.
The append-only log has no compaction.
No SQL, MVCC, or replication.
No production-readiness guarantee.
No built-in model execution or embedding generation.

SQLite and other established databases remain better choices for many workloads, especially those requiring SQL, larger-than-memory datasets, or mature production guarantees.

8. Source Code and Packages

Website: https://www.ltavarez.me/cedrodb

GitHub: https://github.com/uppy19d0/cedrodb

npm: https://www.npmjs.com/package/cedrodb

PyPI: https://pypi.org/project/cedrodb/

CedroDB is open source under the MIT license.

Final Thoughts

CedroDB explores a different approach to embedded storage: combining documents, search, agent checkpoints, and recoverable work through a Rust-powered engine.

It's still early, and practical evaluation is essential.

I'd particularly appreciate feedback about API ergonomics, storage architecture, failure recovery, performance testing, and use cases where this design might or might not make sense.

Would you prefer a unified embedded storage engine for AI agent state and local workflows, or separate specialized tools?

Contributions, suggestions, and technical discussion are welcome.

rust #database #opensource #python

Building CedroDB: Exploring Local-First Storage for AI Agents with Rust

LTavarez — Mon, 19 Oct 2026 06:21:15 +0000

Building CedroDB: Exploring Local-First Storage for AI Agents with Rust 🚀

Modern AI applications are becoming increasingly complex.

An AI agent may need to remember previous interactions, retrieve relevant information, maintain its execution state, and recover interrupted workflows.

These capabilities often require combining databases, search engines, job queues, and persistence mechanisms.

But what if we could explore bringing several of these capabilities into a single embedded storage engine?

That's the idea behind CedroDB.

🌱 What Is CedroDB?

CedroDB is an experimental, open-source, local-first database engine developed in Rust, with native libraries for Python and Node.js/TypeScript.

It explores combining document storage, search, durable workflows, and persistent agent state without requiring a separate database server.

The goal isn't to replace established databases. It's to experiment with a storage architecture designed around the needs of local applications and AI agents.

🧠 The Challenge: Persistent State for AI Agents

Imagine building an AI assistant that needs to:

Remember information between sessions.
Search stored documents and knowledge.
Save execution checkpoints.
Resume interrupted operations.
Process background tasks reliably.

Implementing these capabilities usually involves coordinating multiple components.

CedroDB explores an alternative: managing them through a unified local storage engine.

One particularly interesting capability is storing an agent checkpoint together with related document and outbox writes in a single atomic transaction.

This can help avoid inconsistent local state when a workflow is interrupted.

⚙️ What's Inside CedroDB?

1. JSON Document Storage

Store structured JSON documents and perform atomic operations across collections.

2. Full-Text and Vector Search

CedroDB supports:

BM25-based full-text search.
Exact vector similarity search.
Hybrid retrieval combining text and vector search.

This allows applications to retrieve local information using different search strategies.

The application remains responsible for generating embeddings.

3. Persistent Agent Checkpoints

Agent checkpoints allow applications to preserve execution state.

CedroDB also supports revision checks to prevent stale checkpoint updates.

4. Durable Background Jobs

Persistent job queues support retries, leases, and recoverable workflows.

Applications remain responsible for running workers and handling external operations.

5. Changefeeds and Backups

CedroDB includes durable changefeeds, backup capabilities, and integrity verification mechanisms.

These features help support local integrations and recovery workflows.

🦀 Why Rust?

Rust provides several characteristics that make it interesting for building storage engines:

Memory safety without garbage collection.
Explicit ownership and resource management.
Native performance.
Strong support for systems programming.
Cross-platform development.

CedroDB uses Rust for its core engine while exposing native bindings for other programming languages.

This allows Python and JavaScript developers to experiment with the same underlying storage system.

🐍 Using CedroDB with Python

Install the package:

pip install cedrodb

Example:

from cedrodb import Database

with Database("./app.cedro") as db:
    notes = db.collection("notes")

    notes.put_many({
        "note-1": {"state": "pending"},
        "note-2": {"state": "completed"}
    })

    notes.create_index("state")

    results = notes.find_page([
        {
            "field": "state",
            "op": "eq",
            "value": "pending"
        }
    ])

    print(results["records"])

No external database server is required.

📦 Using CedroDB with Node.js

Install CedroDB from npm:

npm install cedrodb

The native Node.js/TypeScript bindings provide access to the Rust storage engine.

You can explore the API and examples directly in the GitHub repository.

🔬 Experimental by Design

CedroDB is currently an experimental project (v0.9.0).

There are important limitations:

Active documents and indexes are stored in RAM.
One process owns a database directory at a time.
Vector retrieval currently uses exact search rather than ANN indexes.
SQL and MVCC are not supported.
Replication is not available.
Production readiness has not been established.

These trade-offs are part of the project's ongoing exploration.

SQLite and other mature databases remain better choices for many production workloads.

🚀 What's Next?

The roadmap toward CedroDB 1.0 focuses on validation, reliability, performance testing, and improvements to the storage engine.

Community feedback is particularly valuable in identifying architectural limitations and practical use cases.

🔗 Explore the Project

Official Website:

https://www.ltavarez.me/cedrodb

GitHub Repository:

https://github.com/uppy19d0/cedrodb

npm Package:

https://www.npmjs.com/package/cedrodb

Python Package:

https://pypi.org/project/cedrodb/

💬 Join the Discussion

Building an embedded database is an interesting engineering challenge, especially when combining storage, retrieval, and recoverable workflows.

I'd love to hear your thoughts:

Would you use a single embedded storage engine for AI agent memory, document search, and durable jobs, or would you prefer separate specialized components?

Feedback, technical discussions, and open-source contributions are welcome.

If you find CedroDB interesting, consider giving the repository a ⭐ on GitHub.

Built with Rust. Accessible from Python and Node.js. Open source.

rust #opensource #database #ai

🚀 Introducing CedroDB: An Open-Source Embedded Database Built in Rust for Python, Node.js & AI Agents

LTavarez — Mon, 19 Oct 2026 06:18:13 +0000

🚀 Introducing CedroDB — An Open-Source Embedded Database Built in Rust

I've been developing CedroDB, an experimental, local-first database engine written in Rust, with native bindings for Python, Node.js, and TypeScript.

The goal is to simplify how applications manage persistent data, searchable knowledge, AI agent memory, and recoverable background workflows — all without requiring an external database server.

🔧 Key Features

JSON Document Storage: Store and manage structured data with atomic transactions.
Full-Text Search: BM25-based search for efficient document retrieval.
Vector & Hybrid Search: Combine semantic similarity with text-based retrieval.
AI Agent Memory: Persistent checkpoints with revision checks to support recoverable agent workflows.
Durable Job Queues: Persistent jobs, retries, and recovery mechanisms.
Changefeeds: Track database changes for local integrations and event-driven applications.
Backups & Integrity: Backup and verification capabilities.
Native SDKs: Integrate directly with Python and Node.js/TypeScript.

⚡ Quick Installation

Node.js / TypeScript
npm install cedrodb

Python
pip install cedrodb

💡 Why CedroDB?

Modern AI applications frequently rely on multiple components to manage persistent state, document retrieval, vector search, and background processing.

CedroDB explores bringing these capabilities together within a single embedded database engine.

It is designed for experimentation with:

AI agents and local memory.
Offline-first applications.
Developer tools and automation.
Local knowledge retrieval.
Recoverable background workflows.

🧪 Current Status

CedroDB is currently experimental (v0.9.0) and under active development.

The engine currently operates with a single owning process, keeps active documents and indexes in memory, and does not support SQL, MVCC, or replication.

It is not intended to replace mature databases such as SQLite or PostgreSQL. Instead, it explores a different approach to combining persistence, retrieval, and agent-oriented workflows.

🔗 Explore CedroDB

🌐 Official Website
https://www.ltavarez.me/cedrodb

💻 GitHub — Source Code
https://github.com/uppy19d0/cedrodb

📦 npm — Node.js / TypeScript
https://www.npmjs.com/package/cedrodb

🐍 PyPI — Python
https://pypi.org/project/cedrodb/

💬 Feedback & Contributions

CedroDB is open source, and community feedback is highly appreciated.

I'm particularly interested in feedback on database architecture, API design, performance, reliability, and practical use cases.

What features would you expect from an embedded database designed for AI agents and local-first applications?

⭐ If you find the project interesting, consider starring the repository on GitHub.

Contributions, suggestions, and technical discussions are welcome!

Rust #Python #NodeJS #TypeScript #OpenSource #Database #AI #LocalFirst #SoftwareEngineering

DEV Community: LTavarez

I Built container-memory-watch: Monitoring Memory Pressure in Node.js Containers 🦀

🔧 Key Features

📦 Installation

⚡ Quick Start

📊 Generate Diagnostic Reports

🐳 Why It Matters

🔗 Source Code & Installation

💬 Feedback Welcome

node #rust #opensource #kubernetes

Building CedroDB: A Rust-Powered Embedded Database for Python, Node.js, and AI Agents

1. Installing CedroDB

2. JSON Document Storage with Python

Querying with an index

3. Atomic Transactions

4. Monitoring Queries and Index Performance

5. Full-Text and Vector Search

6. Durable Background Jobs

7. Where CedroDB Fits

8. Source Code and Packages

Final Thoughts

rust #database #opensource #python

Building CedroDB: Exploring Local-First Storage for AI Agents with Rust

Building CedroDB: Exploring Local-First Storage for AI Agents with Rust 🚀

🌱 What Is CedroDB?

🧠 The Challenge: Persistent State for AI Agents

⚙️ What's Inside CedroDB?

1. JSON Document Storage

2. Full-Text and Vector Search

3. Persistent Agent Checkpoints

4. Durable Background Jobs

5. Changefeeds and Backups

🦀 Why Rust?

🐍 Using CedroDB with Python

📦 Using CedroDB with Node.js

🔬 Experimental by Design

🚀 What's Next?

🔗 Explore the Project

💬 Join the Discussion

rust #opensource #database #ai

🚀 Introducing CedroDB: An Open-Source Embedded Database Built in Rust for Python, Node.js & AI Agents

🔧 Key Features

⚡ Quick Installation

💡 Why CedroDB?

🧪 Current Status

🔗 Explore CedroDB

💬 Feedback & Contributions

Rust #Python #NodeJS #TypeScript #OpenSource #Database #AI #LocalFirst #SoftwareEngineering

Placeholder Contributor

Intro

Highs and Lows

Growth