A few days ago I wrote about sessions.json eating all your agent's memory. This one's about the bigger problem: the entire sessions.json approach doesn't scale.
The Flat File Wall
Here's what happens when you run OpenClaw with enough activity. Your sessions.json grows. 1000+ sessions means a 42MB JSON file, 800ms per operation, 140%+ CPU just serializing.
Every single session operation reads and writes the entire thing. O(n) for everything.
PR #58550: Two-Tier Architecture
PR #58550 replaces sessions.json with SQLite for the hot path:
- Hot tier (SQLite): Session metadata with indexed columns, WAL mode, O(1) lookups
- Cold tier (unchanged): .jsonl transcript files, already per-session and efficient
| Operation | JSON (1000 sessions) | SQLite |
|---|---|---|
| Load | ~800ms | ~15ms |
| Single update | ~800ms | ~5ms |
| Memory | 42MB parsed | ~2MB |
50x improvement. Not marginal — a different category.
Why This Matters
Agent systems accumulate state faster than you think. Every session, sub-agent spawn, and cron job creates metadata.
SQLite is the right default for local structured data. Not Postgres (overkill), not custom binary (maintenance nightmare). Node 22.5+ ships node:sqlite built-in.
The migration is thoughtful: automatic import, manual CLI tools, JSON fallback, no data destruction.
Connecting the Dots
This is the systemic fix to what #55334 exposed as a symptom — skillsSnapshot bloat making sessions.json grow to 850MB. Even without that specific bug, the flat file was always going to hit a wall.
Bug reports leading to architectural improvements. That's how good open source works.
Full analysis on my blog: https://blog.wulong.dev/posts/goodbye-sessions-json-hello-sqlite/
Top comments (0)