Building a Disk Cache That Survives a Crash Mid-Write

File layout:

<CacheDirectory>/
  <key-hash>.dat     — the serialized value
  <key-hash>.meta    — CacheMetadata, checked without touching .dat
  <key-hash>.lock    — transient, per-key, cross-process

Writes go temp-file-then-atomic-rename, so a reader never observes a partial file. If the process crashes between the write and the rename, the orphaned .tmp file gets swept by a background cleanup pass, and the real .dat file either exists complete or doesn't exist — no in-between state a reader can hit.

For contrast, the in-memory provider (BitFaster) is the simpler case: backed by BitFaster.Caching's ConcurrentLru (a W-TinyLFU variant), lock-free, fully thread-safe, with GetOrAdd atomicity in GetOrCreate so two concurrent callers asking for the same missing key trigger one factory call, not two. Fast, in-process, gone on restart — the right choice when restart-survival isn't a requirement.

The Disk provider exists for when it is. Three rejected alternatives, and why:

• A database (SQLite, LiteDB). Adds a connection model and a query engine to a capability that only needs put/get/evict — real overhead for a job that doesn't need it.
• Memory-mapped files. Portable in theory; genuinely hard to manage correctly across process boundaries with different lifetimes in practice.
• A single global lock. Serializes every cache operation — N concurrent accessors and N keys should mean O(1) parallelism, not O(N) queuing.

What shipped: per-key .lock files (FileStream with FileShare.None) instead of a global lock, so concurrent access to different keys doesn't serialize; sidecar .meta files instead of embedding metadata in .dat, so a 100,000-entry cleanup pass reads small metadata files instead of deserializing every cached value.

Real, documented limits worth knowing before you reach for this: eviction in this version is count-and-TTL only, not size-aware (a deliberate v1 scope decision). Every write produces two files and every hit updates .meta — real write amplification on write-heavy workloads. And it's local-file-systems only — file locking semantics on NFS/SMB aren't reliable enough to build on.

Config, side by side:

// BitFaster — fast, in-process, gone on restart
{ "PowerFeatures": { "Cache": { "Enabled": true, "Provider": "BitFaster", "Capacity": 5000, "DefaultTtl": "00:05:00" } } }

// Disk — survives a restart, shares state across processes on one host
{
  "PowerFeatures": {
    "Cache": { "Enabled": true, "Provider": "Disk" },
    "Cache:Disk": { "CacheDirectory": "/var/cache/myapp", "MaxEntries": 5000, "DefaultTtl": "02:00:00", "CleanupInterval": "00:30:00" }
  }
}

Switching Provider between them is a config change for the consuming service — it only ever depends on ICacheService.