DEV Community

Riccardo Odone
Riccardo Odone

Posted on • Originally published at odone.io

Decomposing Features into Pipelines

You can keep reading here or jump to my blog to get the full experience, including the wonderful pink, blue and white palette.


What is a feature?

Sometime it's as an impure action:

--> do_something -->
Enter fullscreen mode Exit fullscreen mode

Sometimes it's a pure calculation:

input --> output
Enter fullscreen mode Exit fullscreen mode

Most of the times it's a mix of the two:

input --> do_something --> output
Enter fullscreen mode Exit fullscreen mode

In other words, a feature is just an algorithm: data in, action, data out. With that in mind, we can go back to that scary huge feature and model it that way:

a_ton_of_input --> do_a_ton_of_stuff --> a_ton_of_output
Enter fullscreen mode Exit fullscreen mode

Done, profit!

Well, not really. It's not that simple.

a_ton_of_input could be complicated, including nested structures, optional data and varying schema. do_a_ton_of_stuff could entail network requests, integrations with other processes and complex calculations. a_ton_of_output could require a strange shape, different formats depending on the requests and conditionals all over the place. And that is just the happy path!

Breaking down a problem is a fundamental tool in the developer's toolbox. We need to be able to go from:

input --> do_something --> output
Enter fullscreen mode Exit fullscreen mode

To a pipeline like:

input --> do_something --> do_something --> do_something --> output
Enter fullscreen mode Exit fullscreen mode

But hey, now we have four problems: three behaviours plus the recomposition. Yes! Four much simpler problems than what we started with. As soon as a problem is broken down, the actions and calculations should be straightforward to implement. And the composition doesn't have to be more complicated than piping in Bash (i.e. |).

The nice part is that we do not need a fuctional language to model a problem this way:

data_1 = do_something_1(input)
data_2 = do_something_2(data_1)
output = do_something_3(data_2)
Enter fullscreen mode Exit fullscreen mode

Surprisingly, the hard part is breaking down the problem into subproblems. If anything, as developers, we often make the mistake of immediately pulling a ticket, writing code and refactoring. This is not breaking down a problem. This is coding a solution and then praying refactoring will make it nice and correct.

Finding subproblems means exploring the problem space. Not contaminating our minds with any specific solutions will keep us creative. Maybe to solve the problem there's no need to write any code at all? Furthermore, refactoring on a whiteboard is pure joy. Especially, when it turns out that jumping into code without a thought made us solve the problem in the wrong way.

Flowchart until you think you understand the problem. Write code until you realize that you don't.

–NATO science committee (1968)

I got a challenge for us all. For the next few tickets we pull, let's setup a timer and spend some time in problem space instead of starting from the solution. Should you embark on the journey, please share your experience.


Get the latest content via email from me personally. Reply with your thoughts. Let's learn from each other. Subscribe to my PinkLetter!

Top comments (0)