Building a Type-Safe In-Memory DB

#python #database #opensource #fastapi

Hi everyone! 👋 I’d like to share a backend engineering project that I’ve been working on:

YaraDB — a lightweight, document-oriented in-memory database built entirely with Python and FastAPI, designed to work natively with Pydantic models.

👉 Repo: Repo

Key Technical Features:

⚡ Hybrid Storage Engine: Organize data into Tables that support both Strict Mode (rigid schema enforcement) for critical data and Free Mode (schema-less) for rapid prototyping.
🔍 Advanced Indexing: Don't just scan—search. Includes built-in Hash Indexes for O(1) lookups and B-Tree Indexes for fast range queries and sorting (O(log n)).
🛡️ Optimistic Concurrency Control (OCC): Built-in protection against race conditions using document versioning (returns 409 Conflict on mismatches).
💾 Persistence: Implements WAL (Write-Ahead Logging) and Snapshots, so you don't lose data if the server restarts or crashes.
🔌 HTTP-Native: No drivers needed — it runs on FastAPI, so you can interact with it via standard REST calls or the built-in Swagger UI.

🌱 Why I Built It

I love Redis, but for many Python microservices, I found myself writing the same validation logic over and over again to ensure the JSON I got back wasn't broken.

My goals were:

To create a storage solution where Data Integrity is a first-class citizen (Schema-first approach).
To dive deep into database internals—implementing my own WAL, Indexing engine, and Concurrency Control logic in Python was a huge learning experience.
To make a tool perfect for AI Agents and LLMs that need structured short-term memory without the complexity of setting up a heavy SQL server.
It's currently in the Alpha/MVP stage. It’s not a "Redis killer" (C is faster, obviously), but it’s a robust, developer-friendly alternative for Python-heavy stacks.

🤝 Contributing

I am actively looking for feedback on the architecture, especially regarding the WAL implementation and performance optimizations.

If you are interested in backend engineering, database internals, or just want a lightweight DB for your next pet project, check it out!

Feedback and ⭐ stars are highly appreciated!