DEV Community: faliye

The 21 Rules of the File System---Writing a COW Filesystem from Scratch, Part 4

faliye — Sun, 04 Oct 2026 17:12:15 +0000

In the previous article, I documented my collaboration with AI.

In this article, I’ll add a few details about the basic work specifications and use them as the starting point for the discussion that follows.

faliye

Oct 4

SOP and Gate Design under Claude---Writing a COW Filesystem from Scratch, Part 3

#claude #ai #rust #filesystem

8 min read

How do you draw an architectural design on a blank sheet of paper? With a grid.

How did the Renaissance masters draw so many towering, magnificent figures with such precision? With dividing lines.

Let me loosely imitate the 21 Articles of War and write something—drawing a few lines for this filesystem.

Contributors

Article 1. We don't distinguish between submitters; we review only the evidence. A patch that passes the gate deserves to be taken seriously. Show me the test.

Decisions, Design, and Experiments

Article 2. Every decision must be backed by experiments, unless explicitly exempted. Experiment files are committed to git.

Article 3. Every decision and design must leave a trace, written up using the standard template.

Article 4. No decision is true forever; a decision that is challenged must be verified.

Article 5. This project learns from the experience of excellent past projects, but its implementation must fit this project's own character.

Format

Article 6. The data format is self-contained and carries back-references. Data can prove its own identity and what it belongs to.

Article 7. Index trees and the like are derived structures and may be discarded. Data recovery never depends on the tree; the tree serves only as a lookup accelerator and aid.

Article 8. Following Article 7, derived structures stay pure: no small data may be stored in them.

Article 9. Stripe width is variable, and full-stripe writes never incur RMW (read-modify-write).

Article 10. No false ENOSPC: space reported as available must actually be usable.

Article 11. Encryption is a first-class capability: slots are reserved for it, but it is off by default.

Article 12. Slots are reserved for future extensions, with vendor interfaces provided as circumstances warrant.

Article 13. Data units are 32 KiB and index nodes are 16 KiB. 32 KiB is dictated by checksum granularity; 16 KiB is a trade-off made for the CPU cache. Neither is chosen to align with the SSD's write granularity.

Testing and Recovery

Article 14. Data recovery code and test infrastructure rank equal to the core code and are developed in lockstep with it.

Article 15. The verification implementation and the filesystem implementation share no code; each is implemented independently, sharing only the format.

Article 16. Crash testing must cover scenarios exhaustively, never by sampling, and must not be treated as a substitute for broad-spectrum testing or formal verification.

Priorities

Article 17. Keep the early stage pure. Early on, ignore the Linux branch and develop around transactions.

Article 18. The first line supports SSDs only; the disk type is held as a placeholder in incompat. Other disk types can be specialized later in separate branches.

Article 19. Until the format is frozen, don't optimize the filesystem code and don't chase performance.

Article 20. When designing garbage collection, consider the possibility of GPU-based collection algorithms.

Slogan

Article 21. The slogan is still under consideration.....

End of text.

A Few Words on These Articles

Contributors reflects the idea of community collaboration. One person's strength is, in the end, limited.

Article 1. This is a project with AI as its main workforce, carrying AI in its DNA from day one. We enjoy the conveniences of this era, and we should also rein in its shortcomings. An AI-driven project has unlimited throughput, but human energy is finite. Submission throughput can be unlimited; acceptance throughput is bounded by evidence.

Decisions, Design, and Experiments is the operating manual handed to the AI during the design phase. It is designed to minimize AI hallucination, raise the AI's autonomy, and make sure decisions, designs, and experiments don't lose their context.

Article 2. Decisions and designs are not daydreams; they must be validated by experiments, finding their footing in the code of individual experiments. A decision or design with no footing is a fantasy. A decision may be wrong, but it must be shown to have been argued through. Experiments, as part of the evidence, should be kept on record permanently.

Article 3. In an AI-driven project, the knowledge base is the most important part of the whole engineering effort. The archived knowledge will provide key grounds for future decisions and retrospectives. Write documents according to the standard template; the template may change, and when it does, have the AI update the documents one by one.

Article 4. Many decisions may become outdated during construction. Given the AI's ability to question history and context, we give it options and prompts through which it can challenge them. At the same time, this questioning must not be left unchecked, or the project will slow to a crawl.

Article 5. This filesystem's design differs from past filesystems. We stand on the shoulders of earlier filesystems and absorb their excellent algorithms and designs, but we must also stay distinctive. We must constantly remind the AI that this is a different project and that earlier implementations can't simply be copied.

The data-structure section is the core implementation of singlefs and gives a brief account of its data design.

Article 6. singlefs data comes in a fixed size of 32 KiB. Each unit contains a data-unit header and the payload, locked together and never stored separately. This way the data naturally carries its own identity, and its back-references state what it belongs to—which tree, which object. The data can thus explain its own situation, with no need for index-tree records to prove its identity and ownership.

Article 7. Following Article 6, if data can vouch for itself, the tree that records it is demoted. Since the data can state its own identity and ownership, in the worst case the index tree can be thrown away and rebuilt.

Article 8. True, under Article 6 a 32 KiB data size makes small files balloon in storage. But following Article 7, the index tree must remain pure. singlefs stores no small data or other structured information in the index tree. The current idea is to solve this with small-data packing: bundling multiple small pieces of data into a single 32 KiB unit.

Article 9. A stripe is a concept of storing data across disks: to keep data safe, you must write to several disks at once. In traditional RAID 5, for instance, three disks are required—two for data and one for parity—so every write is a fixed 64 KiB. But what if you have only 32 KiB of data? Then you must read the old data, recompute parity, and write it back. If power is lost in the middle of this, the data and parity no longer match (the write hole). singlefs instead uses variable-width stripes. In the same situation, storing 32 KiB of data writes to just two disks; storing 64 KiB writes to three. No read is needed, and every write exactly fills the stripe. The risk of data loss on power failure and the space overhead are both much smaller.

Article 10. In a traditional filesystem, the index tree is authoritative, so updating a file has to go through it. Once the index tree has no room left to write, files can no longer be written. And since deleting a file also requires modifying the index tree, deletion fails too, and the filesystem locks up. singlefs's self-containment can solve this problem to some extent, but we are still exploring.

Article 11. File encryption is a capability designed into the format from day one, but the first runnable version won't implement it; it only reserves the format bits (all zeros when disabled). Once the transaction layer and the checker are running, real encryption will be wired in. Whether it can be rolled back after being switched on is not yet decided.

Article 12. singlefs reserves a portion of space for future extension fields. We are also considering whether to offer an extend-like field that SSD vendors can open up. Our selling point is waste.

Article 13. The 32 KiB data unit is not actually designed to cater to SSDs. The checksum has to cover the whole unit, and the aim is to bind checksum granularity and extent granularity together for simpler logic, at the cost of paying roughly ten percent more on small random reads. 16 KiB is the size of an index node, a number settled by weighing CPU-cache considerations. A 16 KiB node and a 32 KiB data unit are two different kinds of unit; a data unit is not two index nodes joined together.

Testing and Recovery sets singlefs's own standard for testing and file recovery.

Article 14. Because singlefs writes a filesystem from scratch, and its underlying data format differs in places, it needs a test system and methods of its own design. Data recoverability is singlefs's top priority, so test code, recovery tools, and core code are implemented on the same day. Each milestone cross-verifies the others.

Article 15. If tests and core code shared the same logic, they would easily converge on the same bugs. So tests and core code are implemented separately.

Article 16. Beyond the routine adversarial cross-checking among three bodies of code, the core test of singlefs is exhaustively simulating power loss in QEMU and examining the state of the data on each stream, and whether it is recoverable. Broad-spectrum testing exists to catch bugs in the code and to keep the AI from producing results that merely look correct when it writes code. Formal verification is indispensable for concurrent features. singlefs aims to find bugs as early as possible through rigorous testing, while also improving observability.

Priorities were designed according to singlefs's own characteristics during development.

Article 17. Because singlefs is developed around transactions, work on upper-layer interfaces such as FUSE will be pushed back, which may delay compatibility with the Linux community. But it lowers the complexity of early development and lets us focus on the flow of data. I'm short on energy, and short on tokens too.

Article 18. singlefs defines incompat to distinguish disk media types (rotating disks, SSDs, ZNS, and so on), in the hope of optimizing each kind of media to the extreme on top of this setting. But for the same reasons—not enough energy, not enough tokens—the first line supports only generic SSDs.

Article 19. Across singlefs's milestones, we will compare against other filesystems horizontally, but only on the basis of per-node data records. Before the format is frozen, we will not optimize the core filesystem code for performance.

Article 20. singlefs's garbage collection will be heavy. Besides a standard GC running on the CPU, we'd like to bring in the GPU to do some of the work. And of course, we also hope to make use of features such as FTL and ZNS.

Article 21. [Big disks belong to the company; good sleep belongs to you?] [WAAAGH! Reckon it'll work!]

Well, it's finally written. The keyboard now belongs to you, Agent bros.

run Tokens, run!!!

SOP and Gate Design under Claude---Writing a COW Filesystem from Scratch, Part 3

faliye — Sun, 04 Oct 2026 02:53:06 +0000

What is an SOP?

SOP stands for Standard Operating Procedure. You split a process into a number of steps, and for each step you say what to do and how to verify it. The result is a set of concrete steps that can be repeated, checked and signed off.

In my boss's words, it's a bowl of ramen: how many noodles, how much water, how much salt, how much scallion. I'm pretty sure he meant Japanese ramen. When I cook it, it's more like: some noodles, a little oil, a reasonable amount of water, scallions at will.

What is a gate?

This one hardly needs explaining. When I was a kid, my family said be home before six. Of course I could choose to come home after eleven, but not coming home wasn't an option. The curfew was just there. It triggers when you walk in the door: men's singles, women's singles, or mixed doubles.

End of the main text.

My last article covered the three-way (Three-Body-style) adversarial setup for multiple agents, plus some code-style rules. But does three parties fighting it out solve the problem? Honestly, no.

faliye

Oct 4

Implementation That Favors the AI, Review That Favors the Human---Writing a COW Filesystem from Scratch, Part 2

#claude #rust #filesystem #ai

11 min read

Agents write the docs, run the experiments and give the reports. Without limits, an agent will freely write all sorts of things, just like my ramen: somewhere between delicious and awful, at random.

While working, I ran into a few fairly serious problems:

Even with prompt and skill constraints, agents often do things they were never authorized to do, or lose track because they forgot the context. How well it went, whether it's actually finished, all "more or less", just like my ramen. So quality depends entirely on how focused and responsible I am that day. My focus and sense of responsibility fluctuate, so of course the results differ between my highs and my lows.
Agents like to overthink. They fall into endless thinking, trying to find the perfect answer, especially on open-ended questions with no clear goal, where they deliberate at length all the time. It's like Go: I may be the weaker player, but I can always use the wear-down-the-old-man tactic. Unfortunately, next to a silicon life form, I'm the old man. A forty-minute think I can sit through. Three forty-minute thinks in a row, I can't.
Thanks to the breakneck progress of AI, a subagent can now send out minions of its own. The token explosion aside, the minions of the minions of the minions are a blind spot for me. I can go and read their thinking, but they're beyond what I can manage. My energy is limited, so I end up with minions of minions of minions who are not my minions.
The main agent does the scheduling, and I'd rather not hand it the writing of code or docs. The sub-agents that fight each other haven't settled on a conclusion until they're done, so they certainly shouldn't write into the main directory. That means designing more subagents for the main agent to call: some write code, some review it. And the ones who write code and the ones who review it can't sit together.

So there's nothing for it. To keep the project going, let's change a few things.

1. SOP and separation of duties (for problem 4)

Turn agent orchestration into a full pipeline: retrieval -> investigation -> implementation -> self-check -> knowledge rot -> gate.

Each stage has one subagent definition under .claude/agents/. Every description says "use only when the main agent dispatches you by name... never auto-dispatch", and what material the role needs goes in required-inputs. Here they are, excerpted (the originals are written in Chinese; I've translated the descriptions):

Retrieval: the retrieval agent may go online, carry material back and put it in the designated directory.

---
name: prior-art
description: Researcher. Looks up other filesystems' source and docs on demand, hands back only facts with citations, no arguments. Use only when the main agent dispatches you by name with the question to look up; never auto-dispatch.
tools: Read, Bash, WebFetch, WebSearch
model: sonnet
effort: high
omitClaudeMd: true
required-inputs: draft directory, report
---

Investigation: the three-way adversarial setup kicks in to find a suitable approach.

---
name: experiment-designer
description: Experiment designer. Takes an experiment number and writes the pre-run registration (arms, controls, criteria, failure clauses) before any code or artifact exists. Use only when the main agent dispatches you by name with the question to answer and the clause under test; never auto-dispatch.
tools: Read, Bash
model: opus
effort: high
omitClaudeMd: true
required-inputs: draft directory, clause under test, fork list|question list|rerun
---

---
name: three-way-attack
description: The cloud attacker leg of the three-way argument. Use only when the main agent dispatches you by name with this round's background material, attack surface and the verdicts of earlier rounds; never auto-dispatch.
tools: Read, Bash
model: opus
effort: high
omitClaudeMd: true
required-inputs: no-read list, draft directory, background material, attack surface
---

Implementation: start writing code.


---
name: implementation-writer
description: Implementer. Changes crates/ for one milestone step, one parallel line, or a cluster the main agent has bundled (may close several items), with tests, and proves the tests go red. Use only when the main agent dispatches you by name with the step number and the clauses in play; never auto-dispatch.
tools: Read, Edit, Write, Bash
model: opus
effort: high
omitClaudeMd: true
required-inputs: draft directory, report, clause, crates files to touch
---


---
name: tooling-writer
description: Tooling implementer. Changes gate stages, hooks, research scripts and watchdogs, or changes agent definitions, shared constraints and project rules according to a verdict. Every change first builds an input that should go red. Use only when the main agent dispatches you by name with which item is being closed and the exit; never auto-dispatch.
tools: Read, Edit, Write, Bash
model: sonnet
effort: high
omitClaudeMd: true
required-inputs: draft directory, report, exit, files to change
---

Self-check: a three-way siege to review code quality.


---
name: investigator
description: Investigator. When a test or gate result differs from expectation, reproduces the symptom, bisects down to file and line, and hands back a minimal repro plus the condition that would refute it. If the dispatch says "fix it once found" and the clause spells out the fix, carries on, fixes it and proves it red; if the clause doesn't, stops and hands back to the main agent. Use only when the main agent dispatches you by name with the verbatim symptom and the question to answer; never auto-dispatch.
tools: Read, Edit, Bash
model: opus
effort: high
omitClaudeMd: true
required-inputs: symptom, draft directory, report
---

Knowledge rot: audit the whole project to check that every change is consistent.


---
name: sweep
description: Sweeper. After a number is retracted, a format constant changes, a new criterion is established, or a stage of work ends, searches the whole repo for places that still cite the old value, should now be governed by the new criterion, or have been made false by this stage, and hands back a classified list. Use only when the main agent dispatches you by name with the old value and which quantity it is (or the stage's scope of change and what it achieved); never auto-dispatch.
tools: Read, Bash
model: sonnet
effort: high
omitClaudeMd: true
required-inputs: draft directory, report
---

Gate: at the end of a task, mechanically check the error ledger, then the main agent analyzes and fixes the problems.

Every node has its own specialist for its own job. The main agent only breaks down and dispatches tasks, and every subagent minds its own business. On top of that, there has to be a mechanism for "knowledge rot" that regularly clears out expired context, so the pipeline never runs overloaded.

2. Limiting abilities (for problems 2 and 3)


---
tools: Read, Bash
model: sonnet
effort: high
omitClaudeMd: true
---

Eliminate fuzziness and the gray zone. The main agent has to be clear what my task is, and then dispatch work to the subagents according to the task.

Depth=1. A subagent may not have minions. That roots out the "blind spot" problem. (This corresponds to the environment variable CLAUDE_CODE_MAX_SUBAGENT_SPAWN_DEPTH=1, and none of the roles has Agent in its tools anyway.)

omitClaudeMd=true. It mustn't know too much. Context it doesn't need to know is withheld without exception, so it doesn't start free-associating.

tools: tools are narrowed, strictly. Each role above gets a subset of Read, Edit, Write, Bash, WebFetch, WebSearch, and only the researcher can go online. Everyone has Bash, so the lock doesn't sit on tools; it sits on hooks: dangerous patterns are rejected before they execute, with no reliance on the agent's good manners.

effort: high. Don't use more power than needed to solve a simple problem.

3. The gate (Harimoto siblings, go!) (for problems 1 and 2)

Red-green dual proof: every piece of code must have a test. And a test that merely runs "green" (passes) isn't enough: you have to prove to me that the test can catch an error (red), and the red and green evidence must be submitted together. The red evidence doesn't rest on the agent's word either: the gate breaks the code under test in one spot and confirms that the test really goes red.

Strong provenance: implementation must be tied to decision and experiment records. "Ghost code" with no historical decision behind it is not accepted, and the gate checks hard whether the comments explain where the code came from.

Exhaustive probe points: plant lots of probes in the code, check the data state every time execution passes, weave a nerve net. For example, even if the code has run through two hundred thousand different points in a simulated environment, you still have to show that the data flow is complete and free of violations. Checks that aren't wired in yet are honestly marked unimplemented, and they're not allowed to pretend they passed.

Text and format gate: check the knowledge base and docs item by item. Is everything written strictly in the preset JSON/Markdown format? Any ad-libbing? Does the record pass? Even a date has to be written as a standard date.

Scheduled patrol (timeout mechanism): the main agent attaches a scheduled script that watches the subagents it has sent out at all times. If it sees the context grow too large or the runtime run over, it immediately checks task progress, and if things have drifted from expectation, pulls the agent back.

4. Dirty-write protection (for problem 4)

Most mature agent frameworks are already aware of this one. For example, they use a worktree and the workspace under the root for physical isolation, so I don't have to build it from scratch.

But I've added one extra lock: unless it has gone through the dedicated "implementation writer" agent node, no code is ever allowed to be written into crates.

The one who writes code and the one who reviews it must be physically isolated. Without a double signature, you don't get past the main repo's threshold.

With that, it should be able to run.

Tokenman, move out!

Implementation That Favors the AI, Review That Favors the Human---Writing a COW Filesystem from Scratch, Part 2

faliye — Sun, 04 Oct 2026 02:38:51 +0000

In the last post, I decided to hand the keyboard over to the AI. But how to use the keyboard, what the key layout is, which input method to use: all of that is still unclear. To get a filesystem built sooner rather than later, I still need to make a little more effort.

faliye

Oct 4

DAY1: Choosing to Be Wasteful---Writing a COW Filesystem from Scratch, Part 1

#ai #filesystem #rust #claude

5 min read

Part 1: Decisions and the Blueprint

The implementer is an AI Agent. An Agent can write code toward a goal, but we lack a blueprint. A filesystem blueprint takes far more than five decisions. What goes into the self-contained unit (how many rooms)? How is the data format defined (how big is each room)? Are stripes variable (can the partitions be moved)? How is the journal handled (do we install a gate)? What is the first goal (fix the foundation first, or the roof)? All of these need decisions. Otherwise you may end up with the roof finished and the foundation not yet dug. We all know that Agents are superb executors when the goal is clear, but they are a bit weaker at planning complex tasks, and without a blueprint a task easily goes off track.

But filesystem design is extremely complex and involves a mountain of prerequisite knowledge. Even if I started searching through references day and night without sleep, I might have a first design drawing five years from now. Fortunately, the Agent bro sitting next to me is exceptionally learned, so I can ask him to make the design decisions and draw the blueprint. His weakness is that he is prone to going off the deep end: overconfident, charging down one path to the bitter end. So I have to guard against an awkward situation: he may not be trying to deceive me (he is simply hallucinating), yet I have no way to tell whether the information he gives me is true or false. What to do?
As the saying goes, three monks have no water to drink. Well then, let's get three Agent bros to work together anyway. Here is the design:

The conversation where the task starts is called the Main Agent. It only asks questions according to the plan. For example, according to the plan, today we should decide how many rooms to design.

This question is handed to one strong Agent (Strong A), who is responsible for collecting information and finding an answer that looks correct. Once he brings back this seemingly correct answer, the Main Agent notifies two more Agents, B the supporter and C the attacker, to start work.

Supporter B looks for non-overlapping supporting evidence for the answer, that is, other arguments that also support it. If he finds any, he reports them; otherwise he reports that he could not find any.

Opponent C looks for arguments that show the answer does not hold at all. If he finds any, he reports them; otherwise he reports that he found none. But the opponent must find strong evidence; weak evidence is treated as failing to find any.

After collecting these reports, the Main Agent begins independent verification. The Main Agent does not judge who has the better argument; it judges and checks whether the results are valid. If they are valid, the Main Agent summarizes these points and has Strong A repeat the steps above and reason through a second round. Note that in the second round the Main Agent's materials have grown, now including both supporting and opposing views.

At the same time, four points need attention:

An Agent's conclusions are random in nature. So before Strong A starts work, he must set a target for himself. If the conclusion drifts away from that target, the conclusion is invalid. For example, suppose we want to build three rooms and the conclusion is that flooring is better. That conclusion cannot be accepted. The file header comment of an Agent's experiment program contains the following: the criterion (written in stone before the run, not to be changed after it), the failure clause (written in stone before the run), the reverse-acceptance clause, and the questions it cannot answer.
Agents have a natural tendency to agree with the user, and this must be eliminated completely. Even if a proposal comes from the user, the decision must still go through the three-party process; if it does not, it is not recognized as approved. The user can demand that it pass, but the AI will, at lightspeed, leave the evidence in a warning and then submit.
Agents from the same source tend to think alike. Although we bring in two AIs to participate, to avoid this tendency I also brought in a local model to join the thinking and the attacking. A local model is usually useless, but every now and then it has a flash of inspiration.
The Main Agent's data sources are not trustworthy. Each Agent must measure the data independently. This prevents the Main Agent from hallucinating some data into the context and thereby biasing the other Agents' judgment.

Of course, considering that AIs often have no stopping condition, if three rounds of three-party confrontation produce no result, reasoning stops and the question is summarized and handed to me to decide.

With this process, my job becomes much simpler. I generally no longer need to re-check whether the materials are real. My attention is not spent on reading materials and checking for hallucinations; I spend it only on four things a machine cannot judge:

Is this test measuring the right thing?
Is this invariant itself correct?
Does the basis of this decision hold up?
Do I accept this trade-off? This is the second half of our first heading: review that favors the human. Make sure the evidence is sufficient and the reasoning is valid, and let the human make the call. And the cost? What's your superpower, Tokenman?

Part 2: Implementation and Code

Before discussing this topic, two observations:

If, in this era, AI context has reached the million-token scale, the context window can even hold all the code and documentation of a medium-sized project at once.
The best practices of the past had human programmers as their subject.

Based on the second point, let's re-examine past best practices: function names should be short, for loop nesting should be shallow, functions should be reusable and abstracted and encapsulated, and so on. These practices actually address the problem of limited human brain bandwidth: short function names, shallow for nesting, and reuse, abstraction and encapsulation are all meant to reduce the load on the human brain. Today, for those of us programming toward Markdown (prompts), does the length of a variable name matter to us? Does shallow for nesting matter to us? Do reuse, abstraction and encapsulation matter to us?

But note that these matter a great deal to the Agent.

When an Agent modifies a function, it greps, and function names and variable names are always the first coordinates. If a name alone can tell you what something is and what it does, that lowers the Agent's hallucination rate, because comments may be forgotten, but the name anchor gets used again and again.

Ten levels of for loops with no if inside are just a traversal to an Agent. An exhaustive match with 12 arms is counted for you by the compiler; 12 independent layers of if/else are 2^12 paths, which you cannot finish testing.

Abstraction and encapsulation are the biggest source of bugs in code. Their main benefit is making code easier for humans to maintain, but for an AI, modifying one place costs about the same as modifying ten.

Today the main force writing and reading code is the model. Sorting out logic, maintaining consistency and exhaustive coverage are exactly what machines are good at. A floor set by the limits of the human brain is far too low for a machine.

So our default attitude toward "best practices" is skepticism, not compliance. This does not mean they are all wrong; it means the reason behind each one has to be re-examined.

Four steps to audit an old rule:

What problem was it originally meant to solve? If you can't say, drop it immediately.
Does that problem still exist today? If it was solving "humans can't remember," "humans can't read it all," or "reviewers are too busy," it most likely no longer exists.
Does it have a second reason, one that has nothing to do with humans? If so, keep it, and rewrite the reason to be that one.
Can the rewritten reason be turned into a gate? If it can't, downgrade it to a suggestion.

There is only one criterion: does this rule make the code easier to verify mechanically, or does it only make the code easier for human eyes to skim? Keep the former; the latter can be dropped.
Here is a brief summary of some past best practices.

Popular practice	Disposition	How we write it
Names should be short, and the closer to the declaration the shorter	Abolished	No length limit
Loop counter called i, temporary variable called tmp	Abolished	Name it for what it is: stripe_index
Functions under 20 lines	Kept, reason rewritten	Cut by "one thing that can be verified on its own," not by line count
Nesting no deeper than three levels	Relaxed	No limit on depth; what is limited is the number of paths
Single exit	Abolished	Early returns freely allowed
Replace conditional branches with polymorphism	Abolished for closed sets	enum plus exhaustive match
DRY, avoid repetition	Relaxed	Repetition must be generated, never copied by hand
default: fallback is safer	Abolished for closed sets	Leave missed cases for the compiler to catch
One assertion per test	Abolished	One scenario per test; multiple assertions allowed
No more than 80 columns per line	Left to tools	rustfmt decides; don't shorten names for line width
Avoid premature optimization	Downgraded to a suggestion	—

Part 3: Practice in Rust

1. No abbreviations, except those registered in the abbreviation table

Without reading comments, without looking at call sites, without looking at the implementation, from the name alone you should be able to say what it is, what it does, and what it does not do.

No abbreviations: write cnt, idx, buf, tmp out in full. Domain abbreviations (lba, crc) may be used only once registered, and the registry is their sole authoritative definition.

No single letters: write i as stripe_index, the generic parameter T as Key, the lifetime 'a as 'journal, and Err(e) as Err(error).

Write preconditions into the name: write_node says nothing; append_verified_node_to_journal says three things: it appends rather than overwrites, the input has been verified, and it lands in the journal region.

Test names state the scenario and the expectation: crash_between_data_write_and_commit_keeps_previous_generation, not test_commit.

One concept, one name across the whole repository: it must not be called generation here and epoch there.

2. Semantics go in types first: types > names > comments

Take the following example. The same thing written three ways; when the semantics live in the type, an AI that writes it wrong gets an error.

// 1. Semantics in comments: the worst
// dim0: nationality  dim1: gender  dim2: age band
pop[i][j][k]

// 2. Semantics in names: criticized as verbose in the past, machines love it
number_of_people_in_japan[foreigner][woman][teenager_18_to_24]

// 3. Semantics in types: the best, a wrong dimension simply won't compile
number_of_people_in_japan[Nationality::USA][Gender::Woman][AgeBand::Teenager18To24]

3. Branching: write every case out, never write _ =>

enum Medium {
    Rotational,
    SolidState,
    Zoned { zone_size_in_bytes: u32 },
}

fn node_size_in_bytes(medium: &Medium) -> u32 {
    match medium {  // the point is that there is no `_ =>` wildcard arm
        Medium::Rotational                   => 64 * 1024,
        Medium::SolidState                   => 16 * 1024,
        Medium::Zoned { zone_size_in_bytes } => (*zone_size_in_bytes).min(256 * 1024),
    }
}

The key is the _ => that was not written. Add it and the code looks shorter and more "general," but it switches off exactly the compiler's exhaustiveness check. Later, when a new kind of medium is added, the program will quietly walk into the wildcard arm; the behavior will be wrong and nobody will raise an alarm.

The criterion is not "how many branches there are" but "if a case is missed, who finds out first": the compiler, or production.

A few related practices:

For encodings read from disk that may gain new values later, make - "unknown" an explicit member (Unrecognized(raw_code)), and match remains exhaustive.
Use enum for closed sets, not trait objects. A trait's default method is in fact also a kind of wildcard arm.
Don't abstract a trait with only one implementation. Don't abstract "to make swapping easier later."
For interactions between two cases, write an exhaustive match on a tuple. In singlefs's code we enforce the following:

fn geometry(candidate: Candidate, map_scope: MapScope) -> Geometry {
    match (candidate, map_scope) {
        (Candidate::LogicalIdentityWriteOrder, MapScope::AllPointers) => Geometry { .. },
        (Candidate::LogicalIdentityWriteOrder, MapScope::Code1Only)   => Geometry { .. },
        (Candidate::MixedPath,                 MapScope::AllPointers) => Geometry { .. },
        (Candidate::MixedPath,                 MapScope::Code1Only)   => Geometry { .. },
        (Candidate::ClassReuseTagged,          MapScope::AllPointers) => Geometry { .. },
        (Candidate::ClassReuseTagged,          MapScope::Code1Only)   => Geometry { .. },
    }
}

Two of the rows actually have identical values, and most people would merge them. Here we do not merge them: the day a fourth candidate is added, the code will not compile until this spot is updated.

4. Types: make illegal states unwritable

#[derive(Clone, Copy, PartialEq, Eq)] pub struct LogicalAddress(pub u64);
#[derive(Clone, Copy, PartialEq, Eq)] pub struct PhysicalAddress(pub u64);
fn read_block(address: PhysicalAddress) -> Block;
// read_block(LogicalAddress(value)) simply won't compile

struct RawNode(Vec<u8>);       // just read from disk, unverified
struct VerifiedNode(Vec<u8>);  // checksum compared and passed
impl RawNode {
    fn verify(self, expected_checksum: Checksum) -> Result<VerifiedNode, CorruptBlock> { /* ... */ }
}
fn walk(node: &VerifiedNode) { /* ... */ }  // want to skip verification? the argument can't even be constructed

The first snippet turns "passing the wrong address" from a runtime bug into a compile-time error. The second is even harsher: it makes "not yet verified" a type too, so "forgetting to verify" simply cannot be written. Verify once at the boundary, then trust the types inside, instead of re-checking at every layer.

A few smaller points:

Replace boolean parameters with enums. write(node, true) says nothing at the call site; write(node, Durability::Synced) does.
When constructing a struct, write out every field; don't use ..Default::default(). It is a wildcard arm over fields: add a new field, and every place that didn't write it quietly gets the default value.
Don't use as for lossy numeric conversions; use try_from and handle the failure. as truncates, drops signs and wraps around, all without any error.

Branching and types are in fact two sides of the same thing: one lets the compiler catch "a missed case," the other makes "an illegal combination" unwritable. Both move the checking from humans to machines.

5. Errors: recoverable ones go into types, invariant violations get assertions

I/O errors, on-disk data corruption and running out of space are recoverable, and go through Result. The error enum is divided by the decision the caller must make (not found, data corrupt, out of space, other I/O error), not expanded one by one by underlying cause.

At the public boundary of a library, don't use a unified error type like anyhow. It lumps "retry, report corruption, report out of space" into one blob, and is a wildcard arm on errors.

A broken invariant is a bug: assert immediately, don't carry bad state forward, and above all don't write it to disk.

Don't write unwrap(); write expect, with a message stating which invariant it relies on.

An expect in experimental code looks like this:

.expect("leaf number is not in the position table: the position table did not keep up after the split")
u64::try_from(distinct_groups.len()).expect("the number of distinct nodes fits in u64")

Each expect is a named assertion. The day it blows up, the message says directly which assumption collapsed.

Longer names, more types, more complete branches. The compiler reports a few more errors. That is what being AI-friendly means on the implementation side.

Finally: What We Lost, and What We Gained

Wasted tokens. One question, three legs, three rounds, sometimes eight; one set of background material runs over two thousand lines.

Wasted time.The format design took a full three weeks, and not a single line of the filesystem itself has been written.

Wasted code. The experimental code may be very long, yet not a single line of implementation code has been written.

Wasted characters. Names written in full, branches written in full, every rejection accompanied by a next step.

What we gained:

A conclusion that has not been verified will never appear in the code.
The code may still be wrong, but when it is wrong, it will be caught.
What I decide is on the record, and when I say something wrong, someone comes to strike it down.

It doesn't matter that I don't have the knowledge: the Agents come to me with evidence and options, and multiple choice is something I can do. Reviewing makes me happy.

I don't write the code, but when it is written wrong, it fails loudly at any time. Simple, linear, multi-branch code should be fun for an Agent to write, I imagine.

DAY1: Choosing to Be Wasteful---Writing a COW Filesystem from Scratch, Part 1

faliye — Sun, 04 Oct 2026 02:07:33 +0000

I'm going to build a filesystem. The way I'm going about it might not be entirely proper. Still, I figured I should write something down, though I suspect this series won't have much actual technical content.

To open the series, allow me to explain the basic concepts of filesystems with my very limited knowledge.

A filesystem is the set of data structures and directory rules an operating system uses to organize, manage and retrieve data on disk. Whether or not you use a computer, you can't escape this storage system.

Traditional filesystems (like Ext4 and XFS) are update-in-place. The data unit (the block) has a fixed size, and changing one goes read → modify → write, back to the same spot. It's stable and solid, but since it's an overwrite, once you write, the old data is gone. Ext4 and XFS use a journal to protect metadata, so recovery after a power cut is fast. But the journal doesn't protect the data itself. And snapshots and rollback are hard to build in the filesystem alone, so you end up relying on something below it, like LVM, or on a separate backup system.

So if the traditional approach needs all this extra machinery, why not just put those features inside the filesystem? Don't overwrite data. Write it somewhere new, then point the pointer that used to reference the old data at the new data instead. When new data is written the old data isn't overwritten, and as long as some snapshot still references it, the old data stays. Snapshots come almost for free. Since old data doesn't vanish right away, rollback is easy too. By the same logic, if the power dies suddenly, as long as that final "pointer switch" hasn't completed, the disk still holds a complete old version. No long repair needed, and crash consistency is naturally stronger.

This is the COW (copy-on-write) filesystem, with Btrfs and ZFS as the famous examples. Of course, there's no free lunch. COW pays for it with write amplification, fragmentation, and the job of reclaiming space once nobody references the old data anymore.

That's the end of the main text. Everything below is not the main text.

So, why write a new COW filesystem?

One day in mid-August, while reading up on filesystems, two questions suddenly popped into my head:

Why is the unit size of filesystems almost always 4K?
Why do most systems go to such lengths to keep their data structures compact?

My answers:

First, 4K is the memory page size. The page cache and memory mapping are all page-aligned, so making the block the same size as a page is simply the easiest thing to do. It's also the physical sector size after disks moved from 512-byte to 4K sectors (Advanced Format). The bigger the block, the more space small files waste (internal fragmentation), so nobody dared make it larger.
Ten or twenty-odd years ago, storage was outrageously expensive. Every filesystem was desperately trimming itself down, saving as much disk space as possible for actual data, and leaving the organizing of structures to complicated, clever algorithms. It reminds me of a joke. The diameter of a rocket, the crystallization of human technology, is supposedly limited by the width of railway tracks, because of transport constraints. The track width came from the English, who set it to the width of two horses' backsides, because trains were originally pulled by horses. (It's a popular story, but whether it's true is doubtful. Take it as a joke.)

Looking at computers today, storage has become cheap and huge, even if prices have crept up a bit lately. An SSD has no head and no spinning platter. Data lives in NAND flash, is read and written in pages (roughly 16KiB), and inside there are multiple channels and dies working in parallel. So a big chunk of data can be pulled out at once.

So maybe a design like this

A data unit carries, besides its data, a description of who it is and what it belongs to. It holds the information it needs about itself (self-contained). Even if it sits in some far corner of the disk, a scan is enough to find its owner.
Since a unit can explain its own origin, the tree that used to describe it loses its main job. The tree becomes just an index for fast access to units. It has no other duties and doesn't carry small data. So it can be thrown away and rebuilt.
For future compatibility, leave some bytes empty as extension points. That doesn't save space. It actively wastes it.
Since we're choosing the future, old spinning hard disks stop being an optimization target. We aim entirely at new SSDs, newer FTLs and ZNS (Zoned Namespaces). That said, the SSD's own garbage collection and the filesystem's space reclamation are two different layers. The former handles erasing flash blocks. The latter handles which old versions nobody references anymore. We still have to do the latter ourselves, though it can cooperate with TRIM and ZNS.
Since SSDs are the target, we align data to the SSD's pages and parallel stripes. Metadata is 16KiB and data is 32KiB, so one read can take in a whole stripe.

Problems still to solve

How to implement it: who's going to build this?
Which language: how do we make it easy to verify?
Code style: how do we keep it simple and clear?
What to verify with: how do we tell right from wrong? Scratches head. Fine, let's just do this.

Choosing how to implement it

I know very little about filesystems and definitely can't write a line of it. Writing this myself isn't realistic.

The keyboard goes to the AI. Tokens aren't that expensive right now, so let's be a little wasteful.

Choosing the language

It must compile to something fast enough.
It must have strong verification. Content goes to Rust. Write more verification. Let's be a little wasteful.

Choosing the code style

Absolutely few branches. Too many if-else and the combinations to verify grow exponentially. AI makes mistakes easily.
Abstract as little as possible. Don't reuse a function if you can avoid it. Copy-paste if you can. Never couple things together or mix them up.
Variable names must be long. Seeing a name should tell you the function's past and present life. A bit bloated is fine. Write more code, write it longer. Let's be a little wasteful.

Choosing how to verify

I don't understand code, so every plan must be put to a vote by several parties. Three AIs debate for three rounds, and I'm the referee.
I can't tell right from wrong, so let's just simulate-verify every single point. As for power loss, QEMU, that's yours: cut at every possible crash point, replay, and see whether the filesystem still recognizes itself.
I don't understand timing, so let's enumerate every reordering of concurrent operations. herd7, you're up: given a memory model, it lists every outcome that's allowed to happen. Order out of chaos.

** Not understanding is fine. Verify more. Let's be a little wasteful. **

And with that, our core decisions are done.

Day 1, and the most critical decisions are made.

Looking back, I reckon it's not bad. WAAAGH!