Distillation Is a New Engineering Skill — Project Distillation in the Age of AI Agents

Imagine starting a new software project today.

You probably do not create a repository and immediately start coding. You first spend time with an AI: refining requirements, choosing an architecture, defining workflows, and creating files such as AGENTS.md, CLAUDE.md, specs, and ADRs.

Then you say:

Now implement it.

And code starts pouring out.

Work that once took weeks can now be produced in days or hours. But this also means bad decisions can scale just as quickly as good ones.

At some point, you realize that one of your foundational decisions was wrong. Maybe the domain model is flawed. Maybe the plugin architecture was unnecessary. Maybe the persistence model needs to change.

Small mistakes can be refactored. But some early decisions spread into everything that follows: code, tests, documentation, abstractions, naming conventions, and agent instructions.

When one of those decisions changes, I think we need something different from ordinary refactoring.

I call it Project Distillation.

By distillation, I mean extracting only the knowledge that deserves to survive into the next version of a project.

Refactoring Preserves Continuity. Distillation Breaks It.

Refactoring assumes continuity:

A → A' → A'' → A'''

You improve the existing system while preserving its underlying assumptions.

Distillation becomes useful when those assumptions themselves are wrong.

Suppose your project was designed around a plugin architecture. Interfaces, directories, tests, and documentation all reflect that decision. Later, you conclude that the plugin model was a mistake.

Deleting the plugin code is not enough.

The assumption has already spread across the repository.

This is why distillation is not:

Keep modifying the old project until it becomes the new one.

It is:

Treat the old project as input, extract what is still true, and use that to create a new starting point.

Dead Context Can Be More Dangerous Than Dead Code

Traditional projects accumulate dead code.

AI-assisted projects also accumulate dead context.

Consider this:

architecture-v1.md
architecture-v2.md
architecture-final.md
architecture-final2.md
new-architecture.md

A developer who lived through the project may know which document is current. A coding agent may encounter any of them through repository search, instructions, summaries, or conversation history.

This can cause a strange failure mode.

You replace architecture A with architecture B. Later, the agent implements a new feature and quietly brings back a pattern from A.

First an interface appears. Then a factory. Then a compatibility layer.

Eventually the architecture you removed starts growing back.

In traditional development, legacy architecture usually survived because legacy code survived.

With AI agents, the code may be gone while the assumptions that generated it remain available.

Legacy can be regenerated, not just copied.

That is why distillation is partly a context invalidation problem.

After a foundational decision changes, you must determine which parts of the existing project are still valid.

Distill Knowledge, Not Code

The most important question is not:

What should we move to the new project?

It is:

What should we deliberately leave behind?

A useful test for every artifact is:

Is this knowledge about the product and domain, or is it merely a consequence of an implementation decision we made earlier?

Requirements, domain rules, external contracts, edge cases, behavioral guarantees, and reproducible bugs usually deserve to survive.

Architecture-specific abstractions, historical workarounds, obsolete workflow documents, and implementation-coupled tests deserve much more scrutiny.

The result should not be a completely blank project.

It should be a Verified Seed.

Existing Project
      ↓
Requirements
Domain Knowledge
Contracts
Invariants
Tests
      ↓
Distillation
      ↓
Verified Seed
      ↓
New Project

The new project should be free from the old implementation, but not from everything you learned while building it.

A new Git repository is not always required. A clean branch, worktree, or isolated workspace may work.

The important boundary is not the repository boundary.

It is the context boundary.

Reset the Agent, Too

Cleaning the repository is not enough.

You also need to inspect the persistent context around the agent.

This does not mean the underlying LLM somehow learned your repository into its model weights. The problem is usually in the agent harness: conversation history, summaries, project instructions, memory files, or other persistent state.

An agent that worked on the old project may still carry assumptions such as:

This module has this responsibility.
This feature must use this abstraction.
We solved this problem using this pattern.

If those assumptions are precisely what you are trying to remove, continuing with the same context can undermine the entire distillation.

You deleted the code, but not the assumptions that generated the code.

I think of this as three different resets:

Artifact Reset
Remove obsolete code and documentation.

Context Reset
Break continuity with the previous working session.

Memory Reset
Remove or isolate stale persistent project state.

Old history does not need to be destroyed. Keep it as an archive.

The important thing is that it should no longer be part of the new agent's default context.

The new agent should initially see only:

This is the current system.
These are the current requirements.
These are the constraints.
These are the contracts that must remain true.

Distillation is therefore partly about deciding what the project is allowed to forget.

Do Not Let the Agent Write Immediately

After creating the new seed, it is tempting to start generating code immediately.

I think the safer sequence is:

Verified Seed
   ↓
Fresh Context
   ↓
Read
   ↓
Explain
   ↓
Verify
   ↓
Write

Let a fresh agent read the seed and explain the system back to you.

If assumptions you intentionally removed already appear in its explanation, you still have contamination somewhere.

This matters because the first few writes become strong anchors. Once a new abstraction exists, future work tends to build around it simply because it is already there.

After distillation, what the agent writes first matters more than how quickly it writes.

Tests Tell You What Deserves to Survive

Tests become especially valuable during distillation.

Their role is no longer just regression prevention. They help separate implementation from contract.

The old and new systems do not have to be internally similar. The classes, database schema, framework, and architecture may all change.

What needs to survive are the invariants you intentionally preserve:

The same payment request must not create two payments.

A user must not access another tenant's data.

External API contracts must remain compatible.

But tests themselves also need to be distilled.

This:

ServiceA must call RepositoryB exactly once.

may only preserve an old implementation.

This:

Repeating the same payment request
must still result in only one payment.

preserves behavior.

A good test suite becomes a specification of what deserves to survive the redesign.

More Context Is Not Always Better Context

AI coding agents encourage us to give them more context.

But more context is only useful when that context is relevant and consistent.

If 30 out of 100 documents describe an obsolete architecture, you have not given the agent 100 useful pieces of knowledge.

You have given it multiple incompatible versions of reality.

Over time, this creates what I think of as Epistemic Debt: the growing cost of figuring out what is actually true.

Is this document still valid?

Is this interface a real product requirement or an artifact of the old design?

Does this abstraction exist because the domain needs it, or because the previous architecture needed it?

Project Distillation is a way to pay down that debt and return the project to a high-signal, low-noise state.

AI Makes Deletion More Important

AI is reducing the cost of generating code.

That changes the economics of software maintenance.

In the past, throwing away 100,000 lines of working code could mean months of rebuilding. Increasingly, there may be situations where preserving a flawed architecture through compatibility layers, exceptions, and migrations costs more than extracting the verified knowledge and rebuilding around it.

At the same time, bad context becomes more expensive because it can generate new code, which then becomes context for the next generation of code.

Bad decisions can become generative too.

My current model of Project Distillation is simple:

1. A foundational decision changes.

2. Old and new context begin to mix.

3. Extract requirements, domain knowledge,
   contracts, and invariants that must survive.

4. Remove artifacts tied to the old architecture.

5. Preserve important behavior through tests.

6. Create a Verified Seed.

7. Establish a new context boundary.

8. Reset or isolate stale agent context and memory.

9. Let a fresh agent understand the seed.

10. Verify before allowing it to write.

None of these actions are individually new. We have always rewritten systems, removed legacy code, and started new repositories.

What changed is that coding agents consume more than code. They may use documentation, instructions, conversation history, summaries, and persistent memory to decide what to generate next.

That means outdated assumptions no longer just sit around.

They can produce new artifacts.

So AI-assisted development needs a new kind of project hygiene.

We need to manage not only what a project contains, but what it remembers—and what it is allowed to forget.

I call that practice Project Distillation.