I Built a Crash-Safe Database From Scratch Because I Wasn't Allowed to pip install Anything

Lakshmi Venkatesan — Sat, 05 Sep 2026 19:44:53 +0000

When I saw the rule for Zero Dependency 2026 — "your dependency manifest must be empty" — my first reaction was honestly panic. For most small storage problems, my default choice is pip install tinydb and move on. Track D (Data & Storage) meant I had to actually build the thing TinyDB gives you for free, using nothing but the Python standard library.

So that's what I did. The result is ChronicleKV — an embedded key-value store with a real write-ahead log, crash recovery, and point-in-time history. Here's what it actually took.

The package I normally reach for

TinyDB. It's the obvious choice for small local document storage — dead simple API, no server, just works. But once I actually sat down to think about why it's simple, I realized it's simple because it just dumps a JSON file to disk. There's no write-ahead log, no checksum, nothing. If a write gets interrupted at the wrong moment — process killed, power loss — you can end up with a JSON file that no longer parses cleanly, which can put the whole database at risk depending on when the interruption hit. Not something that happens on every crash, but a real gap with no recovery path built in.

That's the exact problem I decided to solve instead of just reimplementing TinyDB's API for the sake of it.

What I actually built instead

The core is a write-ahead log (WAL). Every write gets appended as a binary record — I used struct.pack/struct.unpack for the encoding, with a fixed header (magic bytes, version, operation type, sequence number, timestamp, key length, value length) followed by the actual key/value bytes, followed by a CRC32 checksum computed with zlib.crc32.

The checksum is the whole point. On startup, ChronicleKV replays the log from the beginning. If a record's CRC doesn't match its bytes, that's the exact point where the process died mid-write — everything before it is valid, everything from that point on gets truncated and discarded. No corrupted file, no unreadable database. Just "recovery stopped at the last good write."

I didn't trust myself on this, so I built chronicle crash-demo-compare — it actually forks a writer process, kills it mid-flight at random points, and counts what survived (numbers below). Sync mode gets that guarantee because every write is fsync'd before it's acknowledged; async mode buffers writes and only flushes on a trigger, which is where the loss comes from.

What it actually took

Some numbers, since claims like "it works" mean more with something to check them against:

WAL + recovery logic: ~187 lines (wal.py)
Storage engine (indexing, compaction, durability modes): ~413 lines (store.py)
CLI: ~261 lines (cli.py)
Record overhead: the binary header is 30 bytes (struct.calcsize(">4sBBQQII") — magic, version, op, sequence, timestamp, key length, value length), plus a 4-byte CRC32 checksum. So every write costs 34 bytes of fixed overhead before the actual key/value bytes.
Tests: 51, all passing — covering WAL encode/decode, recovery, compaction, durability modes, and edge cases.
Crash-loss numbers: sync mode, 0 lost writes across every run I tried; async mode, averaging 37–50 lost writes per mid-flight kill, depending on buffer state at the moment of the crash.
Time: ~18 hours across the 72-hour window.

TinyDB gives you a simple JSON document store with fast reads but no durability guarantee beyond "the file happens to still be valid JSON." ChronicleKV trades a bit of write throughput and API simplicity for crash consistency and point-in-time history — that's the actual engineering trade-off, not just "TinyDB bad, ChronicleKV good."

Other packages I ended up not needing

Once I was deep into this, I noticed how many small dependencies I would have normally reached for without thinking:

click/typer → argparse. I always assumed argparse was clunky. Turns out subparsers handle a full CLI (chronicle get, chronicle diff, chronicle timeline, etc.) just fine once you actually sit with the docs for twenty minutes.
filelock → fcntl.flock. Needed this to enforce single-writer semantics on POSIX. Learned that Windows doesn't really have an equivalent that's as clean, so on Windows I fall back to trusting the process — documented that limitation instead of pretending it doesn't exist.
orjson → json. No surprises here, just used the standard module for the TinyDB-compatibility layer.
pytest → unittest. Honestly the hardest adjustment. I missed fixtures and parametrize immediately. But unittest.TestCase gets you 90% of the way there — see the test count above.
diskcache → a plain dict. The in-memory index is just {key: offset_in_file}. The WAL file is the actual source of truth; the dict is just so lookups aren't O(n) scans.

The feature I'm actually proud of

Because every write is a sequenced, appended record instead of an in-place mutation, I got something for free that I didn't originally plan for: time travel. db.get_at(key, seq=42) returns whatever the value was at that exact point in the log. chronicle diff <file> <key> --from 4 --to 6 shows you what changed between two points, and chronicle timeline prints every write across the entire store in order — basically git log for your data.

None of that was the assignment. It just fell out of designing the storage format around an append-only log instead of trying to bolt "history" on top of a mutable file later.

What I'd tell someone starting this track

Don't think of "zero dependency" as a restriction you're working around — it's forcing you to actually understand what the dependency was for. I always knew TinyDB was "simple" but I'd never actually thought about what durability guarantees it was quietly not giving me until I had to write the fsync calls myself.

Repo's here if you want to poke at the WAL format or the crash demo: github.com/lakshmiv3322/chroniclekv — there's a one-click Colab demo in the README if you want to run it without installing anything locally.