Elixir-style Pipelines in 9 Lines of Ruby

Elixir pipelines are an elegant construct for sequencing operations in a readable way. Fortunately, 9 lines is all it takes to implement them in Ruby.

Background: << and >>

Ruby offers some pipelining primitives. Proc and Method respond to #<< and #>>, which can be used in pipelines:

FindByLogin = proc { |login| ... }
ConfirmUserAccount = proc { |user| ... }
SendConfirmationNotification = proc { |user| ... }

(FindByLogin >>
  ConfirmUserAccount >>
  SendConfirmationNotification).call("gregnavis")

This approach has several drawbacks:

1. The Proc returned by #>> cannot be called using result(...), but only via proc.call(...) or proc.(...) or proc[...], which is inconsistent with regular method calls. Admittedly, this is unavoidable, but can be made to matter less.
2. The pipeline argument comes at the end, but having it at the front would be more readable and more consistent with the pipeline structure.
3. Operations taking more than one parameter must be implemented as higher-order procs or use currying. Introducing or eliminating additional parameters entails switching between regular procs and higher-order or curried procs.

To illustrate the last problem, let's make SendConfirmationNotification take an argument determining the type of notification: e-mail or SMS. It has to be rewritten as:

# Using a higher-order proc.
SendConfirmationNotification = proc do |method|
  proc { |user| ... }
end

# Using currying; notice .curry after the block
SendConfirmationNotification = proc do |method, user|
  ...
end.curry

Unfortunately, the pipeline still suffers from the problem of argument coming at the end:

(FindByLogin >>
  ConfirmUserAccount >>
  SendConfirmationNotification[:sms])["gregnavis"]

The rest of the article shows how to use Ruby refinements, a built-in but relatively obscure facility, to make the code below work:

"gregnavis" >>
  FindByLogin >>
  ConfirmUserAccount >>
  SendConfirmationNotification[:sms]

Let's start with operator definitions. Monkey-patching will be used initially, but will be replaced with refinements by the end of the article.

Step 1: Defining Parameterized Operations

Parameterized and parameterless operations should be defined the same way. A new Kernel method will help here:

module Kernel
  def operation(...) = proc(...).curry
end

Basically, operation is an automatically curried proc. The pipeline can now be defined as:

FindByLogin = operation { |login| ... }
ConfirmUserAccount = operation { |user| ... }

# Notice user comes last.
SendConfirmationNotification = operation { |method, user| ... }

Due to currying, the first argument to SendConfirmationNotification can be provided in the pipeline, while the execution is "paused" until user is provided, too. The code below now works as expected:

(FindByLogin >>
  ConfirmUserAccount >>
  SendConfirmationNotification[:sms])["gregnavis"]

The next goal is moving the pipeline to the front.

Step 2: Piping Arguments into Callables

The following two expressions should be equivalent:

# When we write this:
argument >> callable

# we actually mean this:
callable.call(argument)

The snippet hints at what needs to be done: all objects must respond to >> and that method must call call. A top-level class (Object or BasicObject) must be modified to make #>> available on all objects, resulting in the following patch:

class Object
  def >>(callable) = callable.call(self)
end

Finally, we're able to write:

"gregnavis" >>
  FindUserByLogin >>
  ConfirmUserAccount >>
  SendConfirmationNotification[:sms]

The patches on Kernel and Object must be turned into a refinement to avoid global monkey patching.

Step 3: Introducing Refinements

Refinements are a topic for a separate article, but in short they are monkey-patches that can enabled inside a specific module or class by calling Module#using. Let's approach the problem outside in by starting with how we want the code to be used.

Suppose we're working inside a Rails controller. We'd like to be able to write code like this:

class UsersController < ApplicationController
  using Pipelines

  def confirm
    params[:login] >>
      FindUserByLogin >>
      ConfirmUserAccount >>
      SendConfirmationNotification[:sms]
  end
end

using is built into Ruby, and Pipelines is the refinement to be defined. It's an ordinary Ruby module that refines (i.e. patches) Kernel and Object:

module Pipelines
  refine Kernel do
    def operation(...) = proc(...).curry
  end

  refine Object do
    def >>(callable) = callable.call(self)
  end
end

That's it! Pipelines are now enabled only in UsersController, and no other code will be affected. Keep in mind you need to be using the refinement when defining operations too (so that operation is available).

Summary

That pipeline implementation would fit on a napkin. Let's have a critical look at this approach.

First, calling a curried proc with no arguments keeps the execution "paused", so missing an argument can make the pipeline return a curried proc, instead of the expected return value. This will likely lead to difficult to understand errors later in the program.

Second, procs are difficult to inspect. Seeing #<Proc:0x...> in the terminal is unhelpful when debugging. It is possible to inspect parameters passed to an operation via operation_object.binding.local_variables and operation_object.binding.local_variable_get(name) for parameters of interest. It'd be more helpful if inspecting an operation produced something along the lines of SendConfirmationNotification[method: :sms].

Next Steps

The above was my second approach to implementing pipelines. The first approach was object-oriented and didn't have the drawbacks mentioned above at the expense of slightly more complex implementation. I'll cover it an an upcoming article.