Part 3 got me a clean rebuild from a recipe. It also got me a stupid situation. Every time I launch a Workshop, or every time a fresh Bench needs the same toolchain, the operating system fetches and compiles the exact same ffmpeg again. Ten Benches that all need ffmpeg means ten identical downloads and ten identical builds. The cleanliness is real and the repetition is absurd.
I wanted purity. I got inefficiency. I could not let that stand.
The instinct is to cache the whole assembled Bench. But that is the filesystem snapshot Part 3 already rejected, debris and secrets and all. So caching the environment as a lump is out. The thing worth caching is narrower. It is the derived bytes that come from following the recipe. Not the messy end state. The clean output of a declared step.
This distinction is everything. I had been treating the build artifact and the execution environment as the same thing. They are not. The artifact is the result. The environment is the stage. The stage changes every time. The artifact can be saved.
I need to separate these concerns. I need a mental model that forces me to stop conflating them. The model is simple. There are three different things here and I had been sloppily treating them as one.
The recipe is the source of truth. It is authoritative, declarative, and it is the only thing that persists as authority. The derived bytes are a cache. They are valuable but disposable, and they are never authority. The task state is disposable and per run.
recipe = truth, derived bytes = cache, task state = disposable.
I need to name the cache. I call it the Tool Shed.
The Tool Shed is where the derived bytes live. It is content-addressed. This means I do not name files by what they do. I name them by what they are. The key is a hash of the inputs the recipe approved. The key includes the sealed base it derives from. It includes the exact declared package set, with versions pinned. It includes the network profile the derivation was allowed to use.
Same inputs. Same key. Same bytes.
This changes the game entirely. ffmpeg gets built once. Every Bench or Workshop that declares that same derivation reuses the identical cached artifact. No duplicate copies. No repeated downloads.
The agent never touches any of this. At launch, the operating system computes the key from the recipe and checks the Tool Shed. If the bytes are there, it reuses them. If they are not, it re-derives from the recipe, stores the result, and hands over a ready environment. The agent is a client. It gets a working Bench either way, and it never knows which path ran.
This also solves the offline problem. Once a derivation is cached, a launch that would have needed the network to fetch packages can eventually run with no network at all. Ephemeral stops meaning download ffmpeg again. It goes back to meaning only the task state is thrown away.
This sounds like saving a snapshot. That is the danger. This matters, because saving derived bytes sounds a lot like saving a snapshot, which I spent all of Part 3 arguing against. The difference is direction and authority.
The snapshot was the source of truth. It was an opaque blob I had to trust. The Tool Shed is derived FROM the recipe. The recipe stays the source of truth. A cache miss is a non-event. The operating system just re-derives from the recipe.
I can delete the entire Tool Shed and lose nothing but time. It is a pure optimization. It is never allowed to become an independent thing the system trusts. It can never stand in as a base the way a saved image tried to.
The privileged supervisor ensures this boundary holds. The supervisor checks that every byte in the Tool Shed came from a verified recipe execution. It does not trust the Tool Shed to tell it what is safe. It trusts the Tool Shed to tell it what exists. The recipe tells it what should exist.
I am not claiming this is new. Content-addressed reuse is old and well understood. The Nix store keys built outputs by their inputs. Bazel caches build actions the same way. OCI layers are content-addressed and shared. Every package manager keeps a download cache.
I am not inventing any of this. I am borrowing a decades-old idea. Identify a build by its inputs. Reuse the output. I am just fitting it to the recipe lifecycle.
This brings me to the embarrassing question. I have to ask it because I see other people ask it. The question is: if the bytes match a hash I expected, doesn't that mean I can trust them?
No. It absolutely does not.
A hash is identity, not safety. A SHA-256 will faithfully and precisely identify a piece of malware. Content addressing tells me these bytes are the same bytes that came out of these inputs. It tells me nothing about whether those bytes are safe. It tells me nothing about what those bytes might have picked up while they were being produced.
The cache recognizes an artifact. Recognizing is not the same as trusting.
If I download apt packages during a derivation, the hash covers those packages. If the network is compromised, or if the package index is poisoned, the hash is still correct. It is correct for the poisoned data. The Tool Shed will happily serve that poisoned data to every subsequent Bench that needs it.
The Tool Shed is fast. The Tool Shed is consistent. The Tool Shed is not safe.
This is the hole in the logic. I have solved the waste problem. I have solved the consistency problem. I have not solved the trust problem.
The Tool Shed creates two new problems. One is privacy. If a derived artifact is reused across many Benches, and it was built somewhere that could see my files, the cache is now a path for something private to travel from where it was made to everywhere it gets reused. The hash does not know I was looking at sensitive data during the build. The hash only knows I built the thing.
Two is trust. "It hashes" cannot be allowed to stand in for "it is safe."
I have built a cache that is faster than scratch and consistent across launches. I have stopped wasting cycles on identical ffmpeg builds. I have stopped relying on fragile snapshots. But I have opened the door to leaking secrets and trusting poisoned artifacts.
The hash proves identity. It does not prove safety.
So how do I make derived bytes reusable without ever letting the cache learn my secrets, and without a hash quietly getting promoted into a trust decision?
Top comments (0)