Software Design: Deep Modules

I'm reading A Philosophy of Software Design, by John K. Ousterhout, a professor of computer science at Stanford University and the creator of the Tcl programming language.

According to this other review he has almost two decades of real world software experience, so he seems to know a thing or two about software design.

I love software design, and I love reading different takes on it. I admit I haven't finished the book yet, but so far I love his simple yet complete approach.

So far, I'd recommend it, even though I both agree and disagree on what he calls Classitis.

Classitis

I don't want to spoil or copy too much from his book but to make things short, let's say classes are a type of module, and he encourages modules to be deep, instead of shallow.

A shallow module is module is one with a big public interface, compared to it's implementation. A deep module, is one with a small public interface, compared to it's implementation.

   DEEP MODULE
  ┌────────────┐
  │            ├─────► Interface
  ├────────────┤
  │            │
  │            │
  │            ├─────► Implementation
  │            │
  │            │
  │            │
  └────────────┘

  SHALLOW MODULE
  ┌────────────┐
  │            ├─────► Interface
  │            │
  │            │
  │            │
  ├────────────┤
  │            │
  │            ├─────► Implementation
  │            │
  └────────────┘

His argument is that shallow modules don't help to manage complexity, because the benefit they provide (hiding implementation) is dwarfed by the cost of having to learn a big, complicated public interface. Thus, they must be avoided when possible.

I think he makes a great point. The example he gives (a perfect one, I must add) is the Java File API:

FileInputStream fileStream = new FileInputStream(fileName);
BufferedInputStream bufferedStream = new BufferedInputStream(fileStream);
ObjectInputStream objectStream = new ObjectInputStream(bufferedStream);

The complexity of the ObjectInputStream interface is huge. You need to know a lot of things in order to use that class. And that unknown, is complexity.

Sure, the class is very flexible, but the API is not great. He calls having several small classes like this Classitis, and says it must be avoided.

But... Small classes are good, right?

Small classes are a staple of OOP languages like Smalltalk, and to some extent, Ruby inherited that.

Authors like Sandi Metz, a Ruby consultant with 30+ years of experience, and a Smalltalk background, strongly advises for small classes and small methods.

Small objects seem to make following the Object Oriented Design Principles easier.

So, how can two well-respected authors have polar opposite opinions? Well, for one, because software is hard, but also, because writing good, maintainable software is more an art than a mathematical formula you can blindly apply.

Different people with different backgrounds and different experience reach the goal in different ways. Shocker right? 😉

A case for small objects

I am biased. Being a Ruby developer, and sharing Sandi's philosophy, I love small objects with tiny interfaces. But I know sometimes, they can make things more complex.

Something Sandi and John have in common is that they both care a lot about abstractions. Abstractions are very important, and they require constant refactor, in order to accommodate them to the software we are writing.

Sandi says "it's better to have duplication, than the wrong abstraction". And in this sense, we can see that it's not enough to blindly follow some rules. And that is the trick to it.

Whether you approach it from the right or from the left, whether you prefer small objects or deep modules, you need a critical eye, and always be watching the design of your software.

Take time to refactor, accommodate the abstractions, think about different solutions, and sometimes, recognize that you just can't come up with a good solution, in which case, it's better to leave it as it is for now, until you have more code. The more repetition you have, the easier it is to notice the pattern, and abstract it away.

Back to the Java example

Remember this?

FileInputStream fileStream = new FileInputStream(fileName);
BufferedInputStream bufferedStream = new BufferedInputStream(fileStream);
ObjectInputStream objectStream = new ObjectInputStream(bufferedStream);

Ugly right? One could look at it and blame the small classes. But you could also look at it, and realize you are looking at an implementation, not an interface.

What if you used a builder object to abstract it away? I don't know much about the Java API, but in Ruby-land, it could look like this:

stream = StreamBuilder.build(buffered: true)

Knowing the name of the classes and what it takes to instantiate them is a dependency. You can abstract those away, think of them as implementation details. The consumers of StreamBuilder don't even need to know they exist.

We now exposed a small interface -- only a constructor -- and hide the implementation details, which is the name of classes and how to arrange them all together.

You will still need to know the name of the builder class, and what it expects in the constructor, but that can be easily documented.

Complexity

It is true small classes have issues. A class will always be more complex than just a function, and debugging OOP code can feel like following Alice through the rabbit hole. You look at one object which uses another which uses another.

But they also have advantages. For example, you don't have to hold several objects in your head at once, but you might need to hold a lot of state if you are debugging one big method.

Also, small classes force you to separate the algorithm into smaller parts. A fundamental part of your problem could easily be intermingled and hidden away, you might not even know it exist, if it was just one big method or massive class.

Yet another advantage is that the average complexity of your code will be smaller. It might not be perfect, but it will be consistent. It will allow your software to not be consumed by it's own inevitable complexity.

There's a great talk by Sandi called All the Little Things which explains this in detail.

So, what's better? It depends. We know that both extremes are wrong, so it's up to you to come with a happy middle! What do you prefer?

Comments

[hidden_dude]

I think the points raised are very interesting.

However, I wonder if in Java OutputStream they were trying to share an abstracted interface or a framework for streaming related functionality. Where they trying to hide details or arrange them?

An example of this is:

builder = StreamBuilder.new(buffered: true)

Ok. That looks nice, but I'd argue the builder isn't giving you much here why not do this:

stream = StreamFactory.newInstance(buffered: true)

But that being said, say I want to create a new kind of stream like RarCompressingOutputStream. Since the designers of Java didn't know I was going to create that you can hardly do:

stream = StreamFactory.newInstance(rar: true)

So the Java way, while admittedly convoluted and complex is maximizing EXTENSIBILITY over ABSTRACTION. It is giving us ways for our own implementations to work together, much the same way a UI framework might do.

Whether this was justified in Java or not is a different question. And it certainly wasn't the best option for people trying to learn how to program for the first time.

So maybe the answer is that we need to understand what we are designing for. Certainly a "deep module" is nice because it hides a lot of unnecessary details we don't need.

But when we are building "extensible utility classes" that may be a different story.

[Federico Ramirez]

The Java way surely is maximizing the extensibility. I think it uses the decorator pattern, so it ends up looking like an onion. I also think it's more object-oriented, but also more complex. I'm guilty of making interfaces like this, and I realize it can get hard to understand if you come back to it a few months later. I still think it's worth because of the flexibility you have, but of course, it depends on your use-case. If you will only have one type of streams, a design like this would certainly be over-engineering.

The builder and factory patterns are quite similar in this context, and I think any of those could be used. I like factories to create several instances of related classes, and builders to arrange several more generic objects together.

As for my example, I think I made a mistake there, it should have been something like stream = MyBuilder.build(some: :parameter), thanks for pointing that out :) I was thinking about doing a DSL example but that might be too Ruby-ish and it would miss the point.

I like something the author mentions. He says something like "The default use-case should be easy for the consumer". In the Java example, while it's super flexible, if the user will want a buffered stream most of the time, why not just make that the default, and allow other decorators to be applied if needed?

So maybe the answer is that we need to understand what we are designing for.

Yes! And because it's impossible to know upfront what we are designing for, the design needs to change and adapt as the software grows.

[hidden_dude]

I think the first step to designing is to find out what you are designing for.

[Federico Ramirez]

You surely need something before designing, but that thing will surely change as your software grows, and then you'll need to adapt the design. That's why the waterfall model is dead, unless you really know what you are doing (eg: you are writing v3 of something).