Mutable Pipeline Pattern — Revisiting My Thoughts on Dynamic Pipelines

Background

A while back, I wrote an article called The Dynamic Pipeline Pattern. The idea was to make method chains reconfigurable at runtime.

At the time, I just thought "this might be useful" and wrote it up. But since then, I've had opportunities to use similar structures in a few projects, and my thinking has evolved a bit.

So I'd like to take this chance to reorganize my thoughts under the name Mutable Pipeline Pattern.

The "Rigidity" of Traditional Method Chains

TypeScript

data
  .processA()
  .processB()
  .processC();

Python

(data
  .process_a()
  .process_b()
  .process_c())

This style is readable, and I think many people like it. I do too.

But there's one constraint: you can't change its shape at runtime.

• Sometimes you want to skip processB under certain conditions
• Sometimes you want to temporarily inject logging
• Sometimes you want to reorder steps based on the situation

When these requirements come up, we usually end up with more if statements and flags. I think many of you have experienced this.

In a sense, this is like an iron pipe. You can change the data flowing through it, but the pipe's shape is fixed.

The Idea: Treating Pipelines as Data

The core idea from my previous article was this:

Instead of treating a pipeline as "code," hold it as an "array of processes" and reconfigure it at runtime with add/remove operations.

If you hold [A, B, C] as data, then changing the configuration becomes a data operation, not a code change.

Back then I called it "Dynamic Pipeline," but now I feel Mutable Pipeline fits better. The essence is "mutability" rather than "dynamism."

Here's what it looks like in practice:

TypeScript

const pipeline = new Pipeline<Data>()
  .add('a', processA)
  .add('b', processB)
  .add('c', processC);

// Runtime modification
pipeline.remove('b');
pipeline.addBefore('a', 'log', logStage);

// Execute
const result = pipeline.execute(inputData);

Python

pipeline = (Pipeline()
    .add('a', process_a)
    .add('b', process_b)
    .add('c', process_c))

# Runtime modification
pipeline.remove('b')
pipeline.add_before('a', 'log', log_stage)

# Execute
result = pipeline.execute(input_data)

Characteristics of the Mutable Pipeline Pattern

Let me organize this more clearly:

Aspect	Method Chaining	Mutable Pipeline
Structure	Fixed	Mutable
When changes happen	At deploy time	At runtime
Conditional logic	Expressed with if statements	Expressed as configuration changes
Focus	Data flow	Pipeline structure itself

The last point about "focus" is important. In this pattern, the emphasis is on managing the configuration rather than just flowing data through it.

Where This Might Be Useful

Here are some scenarios where I felt this pattern works well.

Temporary Debugging in Production

Add an inspection stage only when a specific condition is met, then remove it when the investigation is done. No redeployment needed.

This is similar to what I mentioned about real-time processing in the original article.

Self-Adjusting Flows

Observe execution time and failure rates for each stage, then remove inefficient stages or insert caching stages upstream.

The program can adjust its own configuration while running.

Expressing State Through Configuration Instead of Flags

In games or simulations, character states are often managed with flags. With Mutable Pipeline Pattern, you could:

• Poisoned state → Add damage processing stage
• Silenced state → Remove magic processing stage

This way, state is expressed through the pipeline configuration itself. Sometimes this gives better visibility than combinations of flags and if statements.

Things to Keep in Mind

This pattern is powerful, but there are some considerations.

Keep data immutable: Even if the pipeline is mutable, keeping the data passed between stages immutable helps predictability. If both are mutable, things can get chaotic.

Make configuration visible: Having something like a snapshot() function to check the current configuration makes debugging much easier.

Closing

I've reorganized the ideas from my previous Dynamic Pipeline article under the name Mutable Pipeline Pattern.

When conditional logic and flag management start getting messy, "making the configuration itself mutable" might be one option to consider.

Of course, this doesn't work for every situation, and there might be better approaches out there. Just something to keep in mind.