DEV Community 👩‍💻👨‍💻

Cover image for 5 JavaScript Pipelining Techniques
Conan
Conan

Posted on • Updated on

5 JavaScript Pipelining Techniques

Photo by Quinten de Graaf on Unsplash

Pipelining using 5 different techniques, current and future.

We'll refactor two chunks of code lifted from the TC39 pipeline proposal:

i) "Side-effect" chunk

const envarString = Object.keys(envars)
  .map(envar => `${envar}=${envars[envar]}`)
  .join(' ')
const consoleText = `$ ${envarString}`
const coloredConsoleText = chalk.dim(consoleText, 'node', args.join(' '))
console.log(coloredConsoleText)
Enter fullscreen mode Exit fullscreen mode

ii) "Pure" chunk

const keys = Object.keys(values)
const uniqueKeys = Array.from(new Set(keys))
const items = uniqueKeys.map(item => <li>{item}</li>)
const unorderedList = <ul>{items}</ul>
return unorderedList
Enter fullscreen mode Exit fullscreen mode

Each has a "chain" of operations used one after the other against the previous value.

The first chunk logs the final value, the second returns it:

  1. envars > envarString > consoleText > coloredConsoleText > log
  2. values > keys > uniqueKeys > items > unorderedList > return

In both cases, the final value is the only one we're really interested in, so this makes them candidates for pipelining!

Let's start with...

i) The "Side-effect" chunk

1. Using let tmp

The simplest way to drop those temporary variables is to declare a mutable let tmp and continuously reassign it:

let tmp = envars
tmp = Object.keys(tmp)
tmp = tmp.map(envar => `${envar}=${envars[envar]}`)
tmp = tmp.join(' ')
tmp = `$ ${tmp}`
tmp = chalk.dim(tmp, 'node', args.join(' '))
console.log(tmp)
Enter fullscreen mode Exit fullscreen mode

It'll work, but maybe there are less error-prone ways of achieving the same thing. Also, mutable variables aren't exactly en vogue these days. 🤔

2. Using Promise

We can use Promise.resolve and a sequence of then's to keep the scope of each temporary variable under control:

Promise.resolve(envars)
  .then(_ => Object.keys(_))
  .then(_ => _.map(envar => `${envar}=${envars[envar]}`))
  .then(_ => _.join(' '))
  .then(_ => `$ ${_}`)
  .then(_ => chalk.dim(_, 'node', args.join(' ')))
  .then(_ => console.log(_))
Enter fullscreen mode Exit fullscreen mode

No polluting the enclosing scope with tmp here! A Promise carries the idea of "piping" from envars all the way to logging the final colourised output without overwriting a temporary variable.

Not quite how we'd typically use Promise perhaps, but since many of us are familiar with how they chain together it's a useful jumping-off point for understanding pipelining for those not already familiar.

By the way, we could have used Object.keys and console.log first-class:

Promise.resolve(envars)
  .then(Object.keys) // instead of: _ => Object.keys(_)
  .then(console.log) // instead of: _ => console.log(_)
Enter fullscreen mode Exit fullscreen mode

But I'll avoid using this "tacit" style here.

I'm also intentionally avoiding:

Promise.resolve(
  Object.keys(envars)
    .map(envar => `${envar}=${envars[envar]}`)
    .join(' ')
)
  .then(_ => `$ ${_}`)
  .then(_ => chalk.dim(_, 'node', args.join(' ')))
  .then(console.log)
Enter fullscreen mode Exit fullscreen mode

Instead I'll try to keep the first level of indentation equal, as I think it helps convey the full pipelined-operation a bit better.

Anyway, using a Promise isn't ideal if we want a synchronous side-effect.

Popping an await before the whole chain is possible of course, but only if the pipeline sits inside an async function itself, which might not be what we want.

So let's try some synchronous pipelining techniques!

3. Using pipe()

With this magic spell:

function pipe(x, ...fns) {
  return fns.reduce((g, f) => f(g), x)
}
Enter fullscreen mode Exit fullscreen mode

...we can have:

pipe(
  envars,
  _ => Object.keys(_),
  _ => _.map(envar => `${envar}=${envars[envar]}`),
  _ => _.join(' '),
  _ => `$ ${_}`,
  _ => chalk.dim(_, 'node', args.join(' ')),
  _ => console.log(_)
)
Enter fullscreen mode Exit fullscreen mode

We dropped all those .then()'s and left the lambdas (arrow-functions) behind as arguments to pipe that will run in sequence, with the first argument providing the starting value to the first lambda.

Handy!

4. Using Hack-pipes

If you're using Babel or living in a future where the TC39 pipeline proposal has landed, you can use Hack-pipes:

envars
  |> Object.keys(^)
  |> ^.map(envar => `${envar}=${envars[envar]}`)
  |> ^.join(' ')
  |> `$ ${^}`
  |> chalk.dim(^, 'node', args.join(' '))
  |> console.log(^)
Enter fullscreen mode Exit fullscreen mode

Terse! And starting to look like an actual pipe on the left there, no?

Notice that a token ^ acts as our "previous value" variable when we use |>, just like when we used _ or tmp previously.

5. Using the Identity Functor

Let's cast another magic spell:

const Box = x => ({
  map: f => Box(f(x))
})
Enter fullscreen mode Exit fullscreen mode

...and make a pipeline with it:

Box(envars)
  .map(_ => Object.keys(_))
  .map(_ => _.map(envar => `${envar}=${envars[envar]}`))
  .map(_ => _.join(' '))
  .map(_ => `$ ${_}`)
  .map(_ => chalk.dim(_, 'node', args.join(' ')))
  .map(_ => console.log(_))
Enter fullscreen mode Exit fullscreen mode

Looks suspiciously like the Promise pipeline, except then is replaced with map. 🤔

So that's 5 different pipelining techniques! We'll apply them now in reverse-order for...

ii) The "Pure" chunk

Here's the reference code again as a reminder:

const keys = Object.keys(values)
const uniqueKeys = Array.from(new Set(keys))
const items = uniqueKeys.map(item => <li>{item}</li>)
const unorderedList = <ul>{items}</ul>
return unorderedList
Enter fullscreen mode Exit fullscreen mode

To start, we'll first make Box a monad:

const Box = x => ({
  map: f => Box(f(x)),
  chain: f => f(x) // there we go
})
Enter fullscreen mode Exit fullscreen mode

By adding chain we can return the JSX at the end of a pipeline without transforming it into yet another Box (which didn't really matter in the side-effect chunk since we weren't returning anything):

return Box(values)
  .map(_ => Object.keys(_))
  .map(_ => Array.from(new Set(_)))
  .map(_ => _.map(item => <li>{item}</li>))
  .chain(_ => <ul>{_}</ul>)
Enter fullscreen mode Exit fullscreen mode

Kinda feels like the Promise.resolve pipeline if it had an await at the beginning, eh? Instead it's a Box with a chain at the end. 🤔

And synchronous too, like pipe()!

Speaking of which, let's go back and use it now:

Using pipe()

return pipe(
  values,
  _ => Object.keys(_),
  _ => Array.from(new Set(_)),
  _ => _.map(item => <li>{item}</li>),
  _ => <ul>{_}</ul>
)
Enter fullscreen mode Exit fullscreen mode

Fairly similar to the side-effect chunk, except to reveal that yes, pipe will indeed give us back the value returned by the last lambda in the chain. (That lovely <ul /> in this case.)

Using Promise

Back in the land of async, does it make sense to return JSX from a Promise? I'll leave the morals of it up to you, but here it is anyway:

return await Promise.resolve(values)
  .then(_ => Object.keys(_))
  .then(_ => Array.from(new Set(_)))
  .then(_ => _.map(item => <li>{item}</li>))
  .then(_ => <ul>{_}</ul>)
Enter fullscreen mode Exit fullscreen mode

(await thrown-in just to communicate intention, but it's not required.)

Lastly, let's bring it right back to let tmp:

Using let tmp

let tmp = values
tmp = Object.keys(tmp)
tmp = Array.from(new Set(tmp))
tmp = tmp.map(item => <li>{item}</li>)
tmp = <ul>{tmp}</ul>
return tmp
Enter fullscreen mode Exit fullscreen mode

And that's where we came in!

Conclusion

All in all we covered 5 different ways of pipelining: A way of transforming one value into another in a sequence of steps without worrying about what to call the bits in between.

  1. let tmp
  2. Promise#then
  3. pipe(startingValue, ...throughTheseFunctions)
  4. Hack |> pipes(^)
  5. Identity Functor/Monad (Box#map/chain)

If you learned something new or have something to follow-up with, please drop a comment below. In any case, thanks for reading!

Top comments (0)

Visualizing Promises and Async/Await 🤯

async await

☝️ Check out this all-time classic DEV post