How We Built an Atomic, Sharded Incremental Cache with 57ms Rebuild Times

When building a local-first repository intelligence engine, speed is everything. A cold build on a 100k-file benchmark can take 45 to 55 seconds. But during active coding, developers expect instantaneous updates when saving a file.

In TokenCap, we implemented an atomic sharded incremental cache in src/incremental/cacheManager.js.

Cache Architecture and Integrity Guarantees

1. Atomic Sharding: Instead of a monolithic cache file, intelligence graphs are sharded into modular JSON blocks keyed by file path hashes.
2. Content-Based Checksums: Cache validity is verified via SHA-256 checksums rather than volatile filesystem mtimes.
3. Version Guard: Any version mismatch between the running TokenCap binary and stored cache forces an automatic, safe cold rebuild instead of mis-reading stale schemas.
4. Scoped Invalidation: When src/auth.js changes, TokenCap only re-parses auth.js and updates direct incoming and outgoing graph edges.

Benchmark Numbers

On our benchmark repository of 1,200 files:

• Cold Build (Zero Cache): 1.14s
• Warm Full Rebuild: 290ms
• Scoped Incremental Build (1 File Edit): 57ms

# Watch mode with sub-60ms incremental rebuilds
tokencap make --watch

Output:

Watching for changes (rebuild after 300ms idle)...
✓ Change detected in src/auth/jwt.js
✓ Invalidation scope: 1 file, 4 affected callers
✓ Rebuilt incremental snapshot in 57ms (.tokencap/snapshot.md)

Because rebuilds take under 60ms, the VS Code extension auto-captures on save without interrupting typing cadence.

Explore the caching implementation at tokencap.vansharora.app