Divide and Let the LLM Do the Rest: From Personal Experience to Architecture

Where This Started

A few years ago, I started thinking about a simple question:

How far can we actually push an LLM on a complex software project?

That's already well established. I mean something harder:

Can an LLM take a complex problem, break it into smaller pieces, solve those pieces, test the results, and eventually assemble everything into a working product?

At the time, the conversation seemed to be split between two extremes:

AI is basically useless, although admittedly pretty cool.

and:

AI is the future. Give it a few years and programmers will be gone.

I never really bought either position.
I've always thought that when you throw a genuinely difficult problem at a system, getting 100% right is unlikely. But getting 0% right is unlikely too.
And that's actually interesting. Because if we can reliably measure what worked and what didn't, we can start improving the system instead of arguing about whether AI "works."
So I spent almost a year trying to find that boundary. Almost a Year of Trial and Error

First: Instructions

I started with instructions. A lot of instructions.

The idea seemed straightforward: give the model enough rules and guidance, and let it dynamically select the instructions relevant to the current task.

It didn't work particularly well. The more instructions I added, the harder they became to manage. Eventually, I started getting confused by the instructions — and then the AI started getting confused by them too. So I needed a better way to organize knowledge.

Then Came RAG and "Atomic" Rules

The obvious next step was RAG.

What could go wrong?

Instead of having one giant set of instructions, we could break everything into small, atomic rules and let the AI retrieve the rules it needed.

On paper, this looked great. In practice, it opened up an entirely new set of problems:

• How do you actually split the knowledge?
• What does "atomic" mean?
• How many rules should the model retrieve?
• How do we know when it has enough?
• What happens when it doesn't have enough?
• Where should it search next?
• How do we know whether a source is trustworthy?
• How do we verify a rule?
• What happens when a rule becomes outdated?
• What happens when two rules contradict each other?

And that was just the beginning. The same fundamental problem kept coming back.
Once the project grew large enough — around 5,000 lines of code — things started getting difficult again.
The problem wasn't simply retrieving information. The problem was understanding the system as a whole. Maybe the Answer Was Refactoring
My next thought was that we needed some kind of automated refactoring mechanism. But eventually I realized that wasn't quite the right problem either. What I really wanted was a way for the LLM to answer a much more fundamental question:

"How well do I actually understand this project?"

That's when I noticed another threshold. At around 3,000 lines, at least with the models I was using at the time — including DeepSeek V4 Flash — the context would fill up extremely quickly.

Once that happened, every new task effectively became a fresh start. The model could no longer reliably maintain a mental model of the entire project. And that led me to an uncomfortable conclusion. Maybe we shouldn't try to make the software structure equally convenient for humans and LLMs. At least, I didn't know how to do that.

Something had to change.

The First Real Breakthrough

The breakthrough wasn't a better prompt.

It wasn't a better RAG system.
It wasn't a bigger context window.
It was artifacts.

I started using artifacts as a way to make decisions explicit and track what had actually happened during development.

At roughly the same time, I stopped using OpenCode and started building my own IDE. That turned out to be a much bigger decision than I expected.

The Principles

From that point on, I started designing around a few simple principles.

For the User

1. The user shouldn't need to understand software engineering to use the system.
2. They might have zero experience building software.

But they should always be able to answer:

• What happened?
• Why did it happen?
• What changed?
• What happens next?

For Software Development

The system should prioritize:

• Transparency
• Verifiability
• Falsifiability
• Reproducibility

And there was another principle I didn't want to lose:

Don't accidentally build SQL.

This might sound strange, but I've seen the pattern before.

Something starts as:

"Let's manage a database using natural language."

And eventually turns into a massive infrastructure project with layers upon layers of machinery. I didn't want to build an enormous system just to make LLMs slightly easier to use.

The system itself had to remain understandable.

A Real-World Example

Consider a clinic manager.
They probably know exactly how the clinic should operate.

They know:

• how the queues should work;
• which documents need to be processed;
• when those documents need to be processed;
• when notifications should be sent;
• what happens when something goes wrong.

They may understand the business process extremely well. But turning that knowledge into software normally requires a whole team of specialists.

So here's the question I'm interested in:

How much of that work can we safely move onto the shoulders of AI?

Notice the word safely.

I'm not arguing that we should "give everything to AI."

I'm asking:

Which parts can we delegate?

And I suspect the answer depends much less on the raw intelligence of the model than we currently think.

It may depend on the harness around the model. And on choosing the right artifacts.

Where Could This Lead?

There are several possible outcomes.

Maybe AI Can't Do It

We might eventually discover that reliable task decomposition simply doesn't generalize.

Every real-world problem may be too different.
Maybe there is no universal architecture that makes AI + LLM + harness work reliably.
And maybe, in some cases, hiring an expert will simply be cheaper.

That's a perfectly valid outcome.

Maybe We Get 10–15%

AI might reliably handle only a portion of the work.
Suppose we save 10–15% of someone's time.
That may not sound revolutionary.
But 12.5% is roughly one employee out of eight.
For a large organization, a 10% reduction in labor costs is already a very serious business case.

Maybe AI Becomes an Operator

Perhaps we can delegate a significant portion of operational processes to AI.
Humans supervise the system and step in when necessary.

Think about a power plant operator.

They don't manually control every parameter every second.
They sit at the control panel, monitor the system, and intervene when something requires human judgment.

Maybe software systems will work the same way.

Maybe AI Runs Almost Everything

Take aviation as another analogy.

There are situations where modern aircraft can handle an enormous amount of the actual flight work.

The pilots remain responsible for critical decisions and exceptional situations.

In a future version of this model, humans might spend most of their time monitoring, diagnosing, and handling edge cases rather than continuously operating the system.

Or Maybe We Go All the Way

And there is an even more extreme possibility.
AI handles almost everything. Human experts don't disappear. Instead, they become highly specialized service providers.

A company might only need an expert every couple of weeks rather than having one embedded in the organization full-time.

I don't know which of these futures is correct.
That's exactly what I want to find out.

Let's Build Something and See

This is where I want to start a discussion.
I realize the idea is controversial.
That's fine.
Maybe I'm wrong.
Maybe someone will take these ideas, throw half of them away, and build something much better.

I'd be perfectly happy with that outcome.
But there is one shift in perspective that I think is important.

Code Isn't the Product!

I'd like to move one level higher in abstraction.
We tend to think of code as the final product.
I increasingly think that's the wrong abstraction.
Code is a means of solving a problem.
The actual product is the solution to the client's problem.

If a clinic comes to you because patients are waiting too long, the deliverable isn't "50,000 lines of Python."

The deliverable is fewer problems with patient flow.
The code is just one of the tools used to get there.
That distinction becomes extremely important when we start delegating parts of software engineering to AI.

A Library of Solutions

This leads to another idea I've been exploring:

a library of proven solutions.

Imagine that instead of asking an AI to reinvent everything from scratch, we give it access to a collection of immutable, reusable implementations.

The AI can:

1. reference an existing solution;
2. reuse it directly;
3. adapt it when necessary;
4. or decide that it doesn't apply.

And each solution could accumulate real-world statistics.

• How often was it used?
• Did it work?
• How often did it fail?
• In what environments?
• What versions were successful?
• That information could become part of the solution itself.

Blockchain might potentially play a role here by providing an immutable record of provenance and history.

I'm not claiming blockchain is automatically the answer.

The important idea is the immutable, verifiable history of a solution, not the technology used to implement it.

Exchanging Knowledge, Not Just Documents

Experts in different domains could exchange knowledge without having to exchange huge theoretical documents. Instead, they could exchange deterministic agents and executable artifacts that demonstrate how their ideas actually work.
Imagine an expert in healthcare building a workflow that captures how a particular clinical process should operate.
Another organization could potentially import that knowledge, inspect it, test it, and adapt it.

Not:

"Here is a 200-page document explaining our methodology."

But:

"Here is a deterministic artifact that demonstrates the methodology. Run it. Inspect it. Test it. Decide whether it is useful."

That feels much closer to engineering knowledge than traditional documentation.

And perhaps that's the bigger opportunity.

Maybe the goal isn't to build an AI that knows everything.
Maybe the goal is to build an environment where AI can reliably use what humans already know.

1. Divide the problem.
2. Make every piece explicit.
3. Let the LLM do what it is good at.
4. And let deterministic systems, artifacts, and verification handle the rest.