The Origin of the Programming "Paradigm" by the Combination of Enum Functions and Pipeline Operators

Interviewee: José Valim, the creator of Elixir, Chief Adoption Officer at Dashbit;
Interviewer: Susumu Yamazaki, an associate professor at Univ. of Kitakyushu, an organizer of ElixirConf JP.

Japanese Translation

Dec. 17, 2021. To: José Valim

Hi, José,

Masakazu Mori (He made a presentation at ElixirConf US 2021 [1]) and I lecture the course "Programming Theory" at the University of Kitakyushu, including the mainstream programming paradigms: imperative programming, OOP, and functional programming, relating to the history of the architecture of the computer systems. Of course, one of the course's main topics is "Why Elixir?". In Programming Elixir [2], Dave Thomas said, "I don't want to hide data. I want to transform it." I identified this consideration as a new programming paradigm instead of functional programming: The data transformation paradigm.

So, I want to interview you about the history of what you consider about designing Elixir by email, including this topic. I plan to edit and publish this interview as one of the 10th-anniversary articles of the foundation of Elixir.

[1] Masakazu Mori: Live coding a Membership Site in 20min by Phoenix and phx_gen_auth, ElixirConf 2021. The movie is available at https://youtu.be/t5TT0-mI2O4
[2] Dave Thomas: Programming Elixir 1.6: Functional |> Concurrent |> Pragmatic |> Fun, 2nd edition, The Pragmatic Bookshelf, 2018. https://pragprog.com/titles/elixir16/programming-elixir-1-6/

First of all, an essential feature of Elixir, which brings a base to the data transformation, is a combination of Enum functions and pipeline operators. As you know, this brings intense pleasure to programming! In fact, it is one of the reasons that someone chooses Elixir. However, the first version of Elixir 10 years ago is not functional but object-oriented. Then, I guess your first ideation of Elixir may not include such a concept towards data transformation. So, my first question is about your consideration process towards the feature: When did you form the idea, and what brought it?

Note: José said that Elixir was on born May 24, 2012.

Dec. 28, 2021. From: José Valim

When did you form the idea, and what brought it?

I don't think there was one specific moment when the idea was formed. Rather, it was the exposure to several ideas and concepts that slowly unraveled Object Orientation. One of such moments was a talk by Rich Hickey, the creator of Clojure, called Simple Made Easy. The other one was while reading a book called "Concepts, Techniques, and Models of Computer Programming", by Peter Van Roy and Seif Haridi.

In the book, they build a programming language by introducing new concepts and their benefits one by one. When it comes to Object Orientation, they argue that Object Orientation is nothing more than syntax sugar for dispatching to a known module. For example, if you have:

car = new Car()
car.method()

That's the same as:

car = new Car()
Car.method(car)

If you see Object Orientation as syntax sugar, you have to answer the question: is the syntax sugar worth it? The trouble with this syntax sugar (and Object Orientation) is that it couples state and behaviour. The state (car) can only be handled by a given entity (the class Car). This is often sold as a feature but the truth is that developers spend a lot of time trying to undo or reason about this coupling. The need for inheritance, for example, is caused directly by this coupling. However, introducing inheritance brought its own issues, leading languages to introduce concepts such as multiple inheritance (mixins), open-classes (monkey patching), etc. All with their own flaws too!

The other part of the puzzle is that functional programming has shown the complexity of a software is not in its computations and algorithms. If the system has no shared state, if the system has no side-effects, it becomes much easier for both humans and compilers to reason about your code. Therefore, by encapsulating state, objects have taught us to hide the complex parts of our system. Not only that, we often split this state into several objects, which makes understanding and visualizing how our applications work very hard!

Finally, the industry has learned something that was known in academy for decades: the properties that are positively associated to Object Orientation, such as encapsulation, abstraction, and polymorphism, are not actually specific to objects, and can be leveraged, sometimes even with more success, in other paradigms.

So, going back to our original questions: is this syntax sugar worth it? Is object orientation worth it? The answer to me is a clear no, the downsides are many more than the upsides. And, without objects, all we have is state (data) and functions (transformations), as separate entities. The Elixir programming modal comes as a direct consequence of it.

Elixir also provides the pipeline operator, which can also be seen as syntax sugar. It transforms this:

Enum.frequencies(String.graphemes("Elixir"))

into this:

"Elixir" |> String.graphemes() |> Enum.frequencies()

In some sense, it can be seen as the "." in Object Oriented languages, but without the coupling of the state and the behaviour. The state, in this case it starts as the "Elixir" string, can be sent to any behaviour that accepts the string type.