Every Abstraction Is a Bet on the Future

Your architecture shouldn't solve requirements that don't exist yet.

Software developers love abstractions.

I know. I'm one of them.

Give us a small enough problem and eventually somebody will propose an interface, a factory, a plugin architecture, dependency injection, and perhaps a message bus just in case the three functions need to communicate asynchronously someday.

You know. For flexibility.

I've been thinking about this while building Arid, a Python duplicate-code checker written in Rust.

Arid has a deliberately narrow job:

Detect duplicated Python source code quickly and accurately.

That's it.

It doesn't format Python.

It doesn't sort imports.

It doesn't type-check anything.

It doesn't detect dead code.

It doesn't scan JavaScript.

And unless something changes dramatically, it isn't going to make coffee either.

That narrow scope created an interesting architectural question:

How much architecture does a small tool actually need?

My answer has increasingly become:

Enough to make the current problem clean. No more.

That sounds like YAGNI.

It is.

But I think there's a more useful way to look at it.

Every Abstraction Is a Prediction

Suppose Arid had started with this:

Language
    ├── Python
    ├── JavaScript
    ├── TypeScript
    └── ...

Looks reasonable.

Maybe even responsible.

After all, duplicate-code detection isn't inherently a Python problem. Why couple the architecture to Python?

So we introduce something like:

LanguageFrontend
    parse()
    normalize()
    classify()

Python implements it today.

JavaScript can implement it tomorrow.

Look at us. Future-proof already.

Except there is currently no JavaScript version of Arid.

There is no TypeScript version.

There is no requirement for either one.

There isn't even a roadmap item for another language.

What exactly did the abstraction buy us?

It bought us an interface.

It bought us indirection.

It bought us a contract that future implementations now have to fit.

It bought us tests for a generic concept that currently has exactly one implementation.

And perhaps most importantly, it quietly made a prediction:

Arid will need multiple language frontends.

Maybe that's true someday.

Maybe it isn't.

Either way, we've paid for part of that future before we know whether it's coming.

That's the thing about abstractions.

They're not free flexibility. They're bets about where the software is going.

Architecture Has Carrying Costs

The cost of an abstraction isn't just the code required to create it.

Code is usually the cheap part.

The real cost is that somebody has to understand it.

Consider a hypothetical detector architecture:

DuplicateDetector
    ├── ExactDetector
    ├── StructuralDetector
    ├── SemanticDetector
    └── FuzzyDetector

Nice.

Except Arid has one detector.

It performs exact duplicate detection after configurable Python-aware normalization.

There is no structural detector.

There is no semantic detector.

There is no fuzzy detector.

In fact, those are explicitly outside Arid's current scope.

So if I introduce DuplicateDetector, what have I modeled?

Not the software that exists.

I've modeled software I can imagine.

That's a subtle but important difference.

Now every developer reading the code has additional questions:

Why is this an interface?

Are there other implementations?

Can the implementation change at runtime?

Am I expected to add new detectors this way?

What guarantees does the abstraction make?

Which behavior belongs to the interface and which belongs to the implementation?

The abstraction has increased the number of concepts required to understand the system without increasing what the system can do.

That's architectural debt too.

We just don't usually call it that because the code looks clean.

But Don't Just Put Everything in main()

There is an obvious bad interpretation of this argument:

Small software doesn't need architecture.

I don't believe that.

Arid 1.1.0 has an architecture.

The main application pipeline is roughly:

discover
   ↓
read
   ↓
normalize
   ↓
build corpus
   ↓
detect duplicates
   ↓
apply baseline
   ↓
build report
   ↓
render output

Those are real boundaries because they represent different responsibilities in the problem.

File discovery shouldn't know how suffix arrays work.

Duplicate detection shouldn't know how Python comments are parsed.

Normalization shouldn't know how Markdown reports are rendered.

Reporting shouldn't decide what counts as a duplicate.

Those separations aren't predictions about hypothetical future products.

They're properties of the problem Arid solves today.

That's the distinction I care about.

Good architecture separates things that are actually different. Overarchitecture separates things because they might become different someday.

Those are not the same thing.

Concrete Is Not a Dirty Word

At some point, "concrete" became suspicious in software design.

If one module directly calls another module, perhaps we're too tightly coupled.

Better introduce an interface.

But coupling isn't automatically bad.

Incorrect coupling is bad.

Arid's duplicate detector depends on Arid's corpus representation.

Of course it does.

That's the data it detects duplicates in.

The normalization layer produces Arid's normalized representation.

Again: yes.

That's its job.

Those relationships aren't architectural mistakes waiting to be abstracted away.

They're the architecture.

The question shouldn't be:

How do I eliminate coupling?

It should be:

Are these things coupled for a reason that belongs to the domain?

If the answer is yes, hiding that relationship behind another interface doesn't necessarily improve anything.

Sometimes it just makes the coupling harder to see.

One Implementation Is a Clue

I don't subscribe to a hard rule that an interface must always have multiple implementations.

There are legitimate reasons to put a boundary in front of a single implementation.

External systems are an obvious example.

Testing can be another.

A meaningful architectural seam can exist before the second implementation arrives.

But one implementation should at least make you ask a question:

What variation am I modeling?

If the answer is:

Well, someday we might...

I become suspicious.

Someday is responsible for a lot of software.

Someday we'll support another database.

Someday we'll have multiple cloud providers.

Someday this will become a distributed system.

Someday users will write plugins.

Someday we'll support seventeen programming languages.

Maybe.

When someday becomes a requirement, we can design for someday with considerably more information than we have now.

And if the current design makes that future literally impossible without rewriting the entire system, that's worth considering.

But there is an enormous amount of territory between:

Don't make the future impossible.

and:

Implement the future now.

We seem to confuse those surprisingly often.

The Plugin System Nobody Asked For

Plugin systems are one of my favorite examples.

Imagine adding plugins to Arid.

What can a plugin do?

Add a language?

Change normalization?

Replace duplicate detection?

Add output formats?

Filter findings?

Modify configuration?

Now we need a plugin API.

Then we need to decide which internal concepts are public.

Then those concepts need stability guarantees.

Then plugins need version compatibility.

Then failures need isolation.

Then documentation.

Then testing.

Then somebody writes a plugin that depends on behavior we thought was an implementation detail.

Congratulations.

Our tiny duplicate-code checker now has an ecosystem to govern.

For what requirement?

There isn't one.

A plugin architecture would not make Arid more flexible today.

It would create an obligation to remain flexible tomorrow.

That's different.

Generalization Can Make the Current Problem Worse

There's another cost to premature abstraction that bothers me more than the extra code.

It can make the abstraction less correct.

Arid's frontend is Python-aware for a reason.

A comment isn't simply "text following a comment delimiter."

A docstring isn't simply "a string."

A function signature has Python-specific syntax.

Structural context depends on Python syntax.

If I had started by demanding a language-neutral abstraction, I would have needed to decide what all programming languages have in common before I had completely solved the Python problem.

What is a Function in the generic model?

What is a Comment?

What is a Docstring in a language that doesn't have docstrings?

What is StructuralScope across Python, Rust, JavaScript, SQL, and whatever somebody asks for next?

Now we're not merely implementing duplicate detection.

We're designing a theory of programming languages.

All because somebody might want TypeScript someday.

No thanks.

Arid can understand Python correctly.

If another language becomes a real requirement later, then we can compare two concrete implementations and discover which concepts are genuinely shared.

That's usually a much better time to generalize.

The second implementation teaches you things the first one can't.

Duplication Isn't Always Worse Than the Wrong Abstraction

This is where DRY can get us into trouble.

We're trained to see duplication and eliminate it.

I'm literally building a tool that finds duplicated code, so I'm probably supposed to be careful here.

But eliminating duplication by creating the wrong abstraction can be worse than the duplication itself.

Two pieces of code can look similar today and evolve for completely different reasons tomorrow.

Combine them too early and you've coupled their futures.

The same thing happens architecturally.

We see two concepts that might eventually share behavior, so we manufacture a common parent before we understand either one.

Then reality arrives.

One implementation needs a special case.

Then another.

The abstraction starts accumulating flags.

Then optional methods.

Then configuration.

Eventually the "generic" abstraction is mostly a complicated description of the differences it was supposed to hide.

Sometimes duplication is information.

It tells you:

These things look similar.

It does not necessarily tell you:

These things are the same concept.

That's a decision we still have to make.

Extensibility Is a Feature

We often talk about extensibility as though every system should have as much of it as possible.

I don't think that's true.

Extensibility is a product capability.

Like any other capability, it has users, requirements, costs, and tradeoffs.

If third-party developers need to extend your system without modifying it, extensibility may be essential.

If your organization has five implementations behind a stable contract, abstraction may be essential.

If you're publishing a framework whose entire purpose is to support unknown use cases, flexibility may be the product.

But Arid is a CLI that finds duplicate Python code.

Its value isn't proportional to the number of ways it can be extended.

Its value comes from doing its one job correctly, quickly, and predictably.

That changes the architecture I want.

Small Doesn't Mean Crude

Architectural restraint doesn't mean throwing everything into one file until it becomes unbearable.

Look at Arid's internal model.

A prepared file owns its original source, normalized source, normalized lines, and segments.

A normalized line carries things the detector and reporting pipeline genuinely need: its range in normalized text, original source line, whether it's effective, and its structural context and scope.

A duplicate occurrence identifies a file and a normalized range.

A duplicate group contains its effective size and occurrences.

These are small types.

They exist because the domain has those concepts.

That's very different from introducing types whose primary purpose is to make the architecture look sophisticated.

The test I increasingly like is:

Can I explain why this abstraction exists without talking about a hypothetical future?

If I can say:

We need this boundary because parsing Python and detecting repeated normalized sequences are different responsibilities.

Good.

If I say:

We need this because eventually we may support arbitrary parser backends selected dynamically from third-party plugins...

I'm going to need considerably more evidence.

What Happens When Requirements Change?

The obvious objection is:

Isn't this shortsighted? What happens when Arid needs another language?

Then I change the architecture.

Seriously.

Architecture is not a one-time ceremony performed before implementation begins.

It's the structure of a living system.

If Arid someday has a legitimate requirement to support Rust source, I'll have something incredibly valuable that I don't have today:

a second real language implementation.

Then I can look at Python and Rust and ask:

What is actually common?

What must vary?

Where does the boundary belong?

Which assumptions in the Python implementation were language-specific?

Which concepts really are universal?

The abstraction designed from those answers is likely to be better than the one I invent today while staring at a single Python implementation.

Will refactoring cost something?

Of course.

So does maintaining an unnecessary abstraction for three years waiting for a requirement that never arrives.

Architecture is tradeoffs.

There is no option where we pay nothing.

You Can Always Add Code Later

This sounds ridiculously obvious, but software development sometimes behaves as if there's a code shortage.

There isn't.

If a requirement appears later, we're allowed to write more code.

If a second implementation appears, we're allowed to extract an interface.

If users need plugins, we're allowed to design a plugin model.

If the pipeline needs asynchronous execution, we're allowed to introduce it.

If Arid needs another language, we're allowed to refactor the frontend.

We don't receive bonus points for having predicted every requirement five years early.

In fact, predictions made too early can make the real requirement harder to implement because now it has to fit the imaginary one.

The goal isn't to avoid changing the architecture.

The goal is to make the architecture easy to change when reality gives us a reason to change it.

Those are very different objectives.

Build the Architecture You Can Defend

Arid 1.1.0 isn't architecturally small because I don't care about architecture.

It's small because I do.

Its pipeline has boundaries.

Its Python-specific parsing is isolated from duplicate detection.

Its internal representation carries the information downstream stages actually need.

Detection doesn't decide presentation.

Structural metadata describes findings without changing duplicate identity.

Those are architectural decisions.

But there is no generic language framework.

There is no detector hierarchy.

There is no plugin system.

There is no dependency-injection framework.

There is no async runtime.

Not because those things are bad.

Because Arid doesn't currently have problems that they solve.

That's the standard I want to apply more often:

Don't ask whether an abstraction could be useful. Ask what requirement makes it necessary.

If you can't name one, maybe don't build it yet.

Your small tool doesn't need a framework.

It needs an architecture that makes the problem it actually solves obvious.

Build that.

When the problem changes, change the architecture.

---

Arid is an open-source Python duplicate-code checker written in Rust. This article discusses the architecture of Arid 1.1.0. Its intentionally small application pipeline can be seen in lib.rs, and its core domain representation is in model.rs.

About the Author

Bob Taylor is a software engineer and architect who builds developer tools and AI systems. He is currently developing Arid and Polaris.

GitHub