A simple explanation of functional pipe in JavaScript

Sometimes I'm asked why we don't have "dot-chaining" in RxJS anymore, or why RxJS made the switch to use pipe. There are a lot of reasons, but this is really something that needs to be looked at from a higher level than just RxJS.

The need for piping functions comes from two problems butting heads: The desire to have a broad set of available development APIs for simple types (like Array, Observable, Promise, etc), and the desire to ship smaller apps.

The size problem

JavaScript is a very unique language with a problem that most other programming languages do not have: Usually, JavaScript is shipped over a network, parsed, and executed at the exact moment the user wants to use the app the JavaScript is powering. The more JavaScript shipped, the longer it will take to download and parse, thus slowing down your app's responsiveness. Something that can have a HUGE impact on the user experience.

This means that trying to keep JavaScript apps small is critically important. Fortunately, we have a lot of great tools to do this nowadays. We have a lot of "build time" bundlers and optimizers that can do things like tree-shaking in order to get rid of unused code prior at build time, so we can ship the least amount of JavaScript possible to the users.

Unfortunately, tree-shaking doesn't remove code if it can't statically be sure that the code isn't being used somewhere.

Providing broad APIs

For types to be as useful as possible, it is nice to have a well-groomed set of known functionality attached to the type. Especially in such a way that it can be "chained" by making calls left-to-right on that type.

The "built-in" way for JavaScript to provide broad APIs for a given type is prototype augmentation. This means that you would add methods to any given type's prototype object. So if we wanted to add a custom odds filter to array, we could do it like this:

Array.prototype.odds = function() {
  return this.filter(x => x % 2 === 1)
}

Array.prototype.double = function () {
  return this.map(x => x + x);
}

Array.prototype.log = function () {
  this.forEach(x => console.log(x));
  return this;
}

Prototype augmentation is problematic

Mutating global variables. You're now manipulating something that everyone else can touch. This means other code could start depending on this odds method being on Array, without knowing that it actually came from a third party. It also means that another bit of code could come through and trample odds with it's own definition of odds. There are solutions to this, like using Symbol, but it's still not ideal.

Prototype methods cannot be tree-shaken. Bundlers will not currently attempt to remove unused methods that have been patched onto the prototype. For reasoning, see above. The bundler has no way of knowing whether or not a third party is depending on using that prototype method.

Functional programming FTW!

Once you realize that the this context is really just a fancy way to pass another argument to a function, you realize you can rewrite the methods above like so:

function odds(array) {
  return array.filter(x => x % 2 === 0);
}

function double(array) {
  return array.map(x => x + x);
}

function log(array) {
  array.forEach(x => console.log(x));
  return array;
}

The problem now is that you have to read what's happening to your array from right-to-left, rather than from left-to-right:

// Yuck!
log(double(odds([1, 2, 3, 4, 5])))

The advantage, though, is that if we don't use double, let's say, a bundler will be able to tree-shake and remove the double function from the end result that is shipped to users, making your app smaller and faster.

Piping for Left-to-right readability

In order to get better left-to-right readability, we can use a pipe function. This is a common functional pattern that can be done with a simple function:

function pipe(...fns) {
  return (arg) => fns.reduce((prev, fn) => fn(prev), arg);
}

What this does is return a new higher-order function that takes a single argument. The function that that returns will pass the argument to the first function in the list of functions, fns, then take the result of that, and pass it to the next function in the list, and so on.

This means that we can now compose this stuff from left-to-right, which is a little more readable:

pipe(odds, double, log)([1, 2, 3, 4, 5])

You could also make a helper that allowed you to provide the argument as the first argument to make it even more readable (if a bit less reusable) like so:

function pipeWith(arg, ...fns) {
  return pipe(...fns)(arg);
}

pipeWith([1, 2, 3, 4, 5], odds, double, log);

In the case of pipeWith, now it's going to take the first argument, and pass it to the function that came right after it in the arguments list, then it will take the result of that and pass it to the next function in the arguments list, and so on.

"Pipeable" functions with arguments

To create a function that can be piped, but has arguments, look no further than a higher order function. For example, if we wanted to make a multiplyBy function instead of double:

pipeWith([1, 2, 3, 4, 5], odds, multiplyBy(2), log);

function multiplyBy(x) {
  return (array) => array.map(n => n * x);
}

Composition

Because it's all just functions, you can simplify code and make it more readable by using pipe to create other reusable, and pipeable, functions!

const tripleTheOdds = pipe(odds, multiplyBy(3));


pipeWith([1, 2, 3, 4, 5], tripleTheOdds, log)

The larger JS ecosystem and the Pipeline Operator

This is roughly the same pattern that is used by RxJS operators via Observable pipe method. This was done to get around all of the issues listed with prototype above. But this will clearly work with any type.

While prototype augmentation may be the "blessed" way to add methods to types in JavaScript, in my opinion, it is a bit of an antipattern. JavaScript needs to start embracing this pattern more, and ideally we can get a simple version of the pipeline operator proposal to land in JavaScript.

With the pipeline operator, the above code could look like this, but be functionally the same, and there wouldn't be a need to declare the pipe helper.

pipeWith([1, 2, 3, 4, 5], odds, double, log);

// becomes

[1, 2, 3, 4, 5] |> odds |> double |> log

Comments

[Manuel Romero]

Real world use case: slugify a String:

const slugify = s => pipe(
toString,
toLower,
split(' '),
filter(removeDashes),
join('-')
);

[Matej Svajger]

Very cool article!

I gave an attempt at implementing your proposal for the slugify function if anyone cares to play around: codesandbox.io/s/js-functional-pip...

[Dmitri Zaitsev]

Forgotten (s) at the end?

[Manuel Romero]

Yes. It was like pseudo code

[Leonardo Alves]

We can create another function to let pipe even more readable than pipeWith with the following function:

function pipeValue(args) {
    return {
        to: (...fns) => pipe(...fns)(args)
    }
}

const array = [1,2,3,4];
pipeValue(array).to(
    odds,
    double,
    log
);

[Dmitri Zaitsev]

Or use the curried pipeline where you can pass your arguments directly.

[jesse]

This API really resonates with me. Thanks for sharing.

[Joe Graham]

Not a fan, this breaks functional composition

[Ben Halpern]

Awesome first post Ben, welcome! Sorry I grabbed the premium username space.

[Dmitri Zaitsev]

It looks like the pipeline in general provides a really convenient and reusable syntax, also battle-proof from decades of use in Unix and other languages. But the pipeline proposal seems to get stale with multiple disagreements and can take years to complete.

The pipeWith function keeps the linear flow but mixes arguments with functions inside its arguments and requires to begin with array:

pipeWith([1, 2, 3, 4, 5], odds, double, log);

Why not instead use a curried pipeline function with identical functionality:

pipeline(1, 2, 3, 4, 5)(odds, double, log)

Among advantages: no need to wrap arguments in an array and typing the comma is quicker than "|>" that requires 2 Shift + 2 keys (4 in total vs 1 for the comma)?

I have been searching for cleanest and most reusable patterns to write and compose Continuation-Passing-Style functions and this pipeline curried function seems to do "the best job", but I'd be curious to put this claim to test and hear other thoughts.

[Thiago Romão Barcala]

From one side it is sad to have to wait so long for this improvement, but from the other side it is better to wait than to have a modification in the language syntax that doesn’t work well in the future.

Your efforts to find the perfect syntax are definitely valuable, but I think it requires a lot of analysis of the consequences to avoid superficial argumentation. Removing the need to wrap arguments in an array doesn’t necessarily makes things easier, because it is more likely that the argument being piped will be already coming as a variable from somewhere else. So we would rarely construct arrays to pipe, but pipe some existing array. And in this case you would need to spread the array to be able to use it in the pipe. Besides that, the pipeline operator can be used with anything, not only arrays. It is just a different way of writing nested function calls.

Also, the amount of keys pressed to type the code is not that important, because we spend more time reading code than actually typing, so maybe less parentheses is preferable instead of one extra character.

[puiu]

Not data last.

[Magne]

Why is data last important? If the function composition grows (multi-line), it can be annoying to go to the end just to see what is passed in at the beginning. Chronological ordering is more readable. If it doesn't pose a clear disadvantage, I don't see why not use it.

[mister. jones]

From what I understand of functional programming, currying typically uses a data-last approach.

It's nice, because it allows you to create functions from composed functions. So, yes, if you're piping a lot of functions into your pipeline, it could be annoying to have to find the end-of-the-line to check what data is actually being piped through. But, I'd argue that if you're piping that many functions through, you should consider making it into a named function anyway, so that readers don't have to parse through every step to understand what it's ultimately doing.

Manuel Romero already posted a slugify function (above) as an example, so I'll riff on that:

const pipe = (...fns) => x => fns.reduce((y, f) => f(y), x);

const toLowerCase = str => str.toLowerCase()
const split = separator => str => str.split(separator)
const join = separator => arr => arr.join(separator)

const slugify = pipe(
  String,
  toLowerCase,
  split(' '),
  join('-')
)

fooBarSlug = slugify('Foo Bar Fizz Buzz')
console.log(fooBarSlug)  // foo-bar-fizz-buzz

[Hopsken]

Great article Ben! When I moved to RxJS 6, I was always wondering why we need the pipe operator, this article really explains.

[Nando]

smart and beautiful! kudos, Ben!

a little typo maybe in "Functional programming FTW!" paragraph:
function double(array) {
return arr.map(x => x + x);
}
I think it should be:
return array.map ...
;)

have a good week-end!!

[Lorefnon]

Unfortunately typescript doesn't provide a good way to impose type constraints on functions with unlimited variadic arguments.

So pipeline function implementations like the above can be made type safe only by repeated overrides upto an arbitrarily determined maximum arity. Example.

I am curious why the RxJS team didn't opt for a fluent API (like this one) which does not suffer from a similar issue.

[Asti]

They originally did use a fluent API. Then for RxJS5... bad decisions were made.

[Anthony]

Really good article. Very insightful!

But! Just to be a nitpicky jerk (and because it's oddly amusing to me to notice this):

Your three initial Array prototype extensions are odds, double, and log. I don't know if there's a defined principle for this, but method naming should be consistent enough to allow for a certain amount of expectation to be met by the developer. My immediate assumption was that log() would return an array of the log of each current array member, consistent with double() (this is why I'm amused with myself, because that's such an unlikely method that has way less value). This speaks to the value of namespacing for the value of grouping methods by functionality. Maybe an Array.Math object would have any math related extensions while Array.Utils could have log(). Or maybe I'm wrong?

Obviously it's not central or relevant to your article, just noticed and thought it is its own interesting topic.

[Sammy Israwi]

Fantastic! I wondered about this but I don't think I've ever actually understood it.

I've seen the pipeline operator a lot, though I haven't followed the proposal status. I'm not in love with the syntax but the functionality will be very much a gift!

[Hersh]

I think meant for
function double(array) {
return arr.map(x => x + x);
}
to be
function double(array) {
return array.map(x => x + x);
}