bchamagne

Posted on May 25, 2023 • Edited on May 27, 2023

Don't use structs, use records instead!

#elixir

Structs are omnipresent in Elixir projects. And for good reasons, they're great, they have numerous advantages over maps.

Autocompletion
Dot Syntax
Enforce keys
Dialyzer friendly
Pattern matching

I use them everywhere. I try to only use maps when the keys are dynamic values.

But I have one issue with them. Something that I believe is a MUST-HAVE for the maintainability of a project:

Struct are public, they can't be private

This probably raise 2 questions in your head:

What is a private structure?

A private structure is a structure that can only be CRUD (create/read/update/delete) from a single module. The module must expose accessors to manipulate it.

What are the benefits?

Decoupling

Modify the struct with peace of mind. It will only impact current module. No risk of breaking code from other places.

Enforce business rule at a single place

Don't worry about someone else overriding a business rule in some part of the code because she/he created a struct by her/him-self.

The solution: records to the rescue

Records exist since the beginning of Elixir but no one talks about them. As you might have peeked in the reference, the Record module exposes a defrecord macro but more importantly a defrecordp macro...

The defrecordp macro allow us to create a private record. Hooray!

The syntax

Record syntax is a bit weird because the macro generates 3 macro to manipulate the record.

Record.defrecord(:user, [:name, :age])

generates:

user/0 will create the struct with default values
user/1 will create the struct from given values
user/2 will either update a struct or access a field

See examples on the reference page

Demonstration: TMNT Game

I'll demonstrate the benefits with an example. Let's say that my business rule is to be able to play a game with 1 of the 4 famous ninja turles.

I'll present you the same example twice but first coded with a struct and secondly coded with a record.

The struct version

defmodule Turtle do
  defstruct [:weapon, :name, :color]

  @type color() :: :blue | :red | :purple | :orange
  @type t() :: %__MODULE__{
          weapon: String.t(),
          name: String.t(),
          color: color()
        }

  ######################################################################
  @spec new_by_color(color()) :: t()
  def new_by_color(:blue),
    do: %__MODULE__{color: :blue, name: "Leonardo", weapon: "Katana"}

  def new_by_color(:red),
    do: %__MODULE__{color: :red, name: "Raphael", weapon: "Sai"}

  def new_by_color(:purple),
    do: %__MODULE__{color: :purple, name: "Donatello", weapon: "Staff"}

  def new_by_color(:orange),
    do: %__MODULE__{color: :orange, name: "Michelangelo", weapon: "Nunchaku"}

  ######################################################################
  @spec battle_cry(t()) :: String.t()
  def battle_cry(t) do
    color = turtle_color(t.color)
    "#{t.name}: #{color}Cowabunga!"
  end

  ######################################################################
  defp turtle_color(:blue), do: IO.ANSI.blue()
  defp turtle_color(:red), do: IO.ANSI.red()
  defp turtle_color(:purple), do: IO.ANSI.magenta()
  defp turtle_color(:orange), do: IO.ANSI.yellow()
end

defmodule Game do
  @spec play(Turtle.color()) :: :ok
  def play(color) do
    player = Turtle.new_by_color(color)
    tick(player)
  end

  defp tick(turtle) do
    IO.puts(Turtle.battle_cry(turtle))
  end
end

Looks good, no? Everything looks great, dialyzer pass. It's easy to read and easy to use.

Now... Imagine months pass and a coworker, fan of Dumas, creates this function in Game.ex:

def play2() do
  player = %Turtle{color: :blue, name: "Porthos", weapon: "Balizarde"}
  tick(player)
end

This code just broke my business rule. And nothing prevented that! How annoying! Who knows the impact of this change?

The record version

defmodule Turtle do
  require Record

  Record.defrecordp(:turtle, [:weapon, :name, :color])

  @type color() :: :blue | :red | :purple | :orange
  @type t() :: record(:turtle, 
          weapon: String.t(), 
          name: String.t(),
          color: color()
        )

  ######################################################################
  @spec new_by_color(color()) :: t()
  def new_by_color(:blue),
    do: turtle(color: :blue, name: "Leonardo", weapon: "Katana")

  def new_by_color(:red),
    do: turtle(color: :red, name: "Raphael", weapon: "Sai")

  def new_by_color(:purple),
    do: turtle(color: :purple, name: "Donatello", weapon: "Staff")

  def new_by_color(:orange),
    do: turtle(color: :orange, name: "Michelangelo", weapon: "Nunchaku")

  ######################################################################
  @spec battle_cry(t()) :: String.t()
  def battle_cry(t) do
    color = turtle_color(turtle(t, :color))
    name = turtle(t, :name)
    "#{name}: #{color}Cowabunga!"
  end

  ######################################################################
  defp turtle_color(:blue), do: IO.ANSI.blue()
  defp turtle_color(:red), do: IO.ANSI.red()
  defp turtle_color(:purple), do: IO.ANSI.magenta()
  defp turtle_color(:orange), do: IO.ANSI.yellow()
end

defmodule Game do
  @spec play(Turtle.color()) :: :ok
  def play(color) do
    player = Turtle.new_by_color(color)
    tick(player)
  end

  defp tick(turtle) do
    IO.puts(Turtle.battle_cry(turtle))
  end
end

As you can see, the code is very similar. Game.ex did not change at all.

Now... let's try to recreate the same scenario as before. Months pass and a coworker tries to add the play2/0 function from above... Impossible! There's no way to create a turtle record from outside the Turtle.ex module.

Actually there is a way by modifying the tuple directly, which is something no sane person will attempt.

Obviously, this example is very explicit but I believe it makes my point. In a real world example, the implicit coupling is much more subtle and usually creates friction when maintaining the code.

For the curious, the internal representation of a record is a tuple. Whereas structs are maps. Which is yet another benefit in the record side. Don't take my word for it, read Maps or Records? for the details.

So what do you think?

EDIT: Forgot to mention, there is a @opaque attribute. But unfortunately, it does not prevent the creation of the struct from outside the module. It only forbid to peek at the structure.

EDIT2: Please check the comments section, it's worth it.

Top comments (9)

Ben Munat • May 27 '23

The awesome and thorough comment from Qqwy goes into way more detail than I could but I would just offer up a more basic answer: in the 5 years I've been doing Elixir professionally I have never run into an app using records over structs. As Qqwy points out, they are mostly there for Erlang inter-op. I would argue that following established Elixir conventions and common practices is a lot more important than attempting to hide data from yourself and your coworkers.

I would also argue, but maybe with a bit less fervor, that encapsulation mania in OOP went overboard. The beauty of functional programming is that we can stop trying to create smart objects that hide things from us and instead focus on carefully constructed functions which, when connected in different pipelines produce different but expected results. Your data, for the most part, wants to be free and available to everyone. The functions that work on that data are grouped into modules and contexts that keep things sane and organized.

Also note that -- following good Domain-Driven Design practices -- you may well have the same entity modeled by more than one struct/type in the app. For example, Order as it pertains to sales/e-commerce vs Order as it pertains to reporting.

But ultimately -- and again, like Qqwy said -- there are established patterns (overriding the inspect protocol, the foo naming pattern, etc) for making some parts of your structs "private". It's pretty much always best to stick to convention unless you're really breaking new programming ground. Cheers!

Qqwy / Wiebe-Marten • May 26 '23 • Edited

Hi there! Thanks for writing this article.

Unfortunately, I think the main conclusion you are making is somewhat flawed, and while I do not wish to discourage you, I do want to clear things up a little so people new to Elixir do not get confused.

Structs are 'just' maps that happen to have a __struct__ field containing a particular atom.
Similarly, Records are 'just' tuples whose first element is a particular atom.

In both cases, the full content of the datastructure is visible for all to see, and nothing is stopping anyone from messing with it.

So I do not think that 'just use records' is good advice to give to anyone who cares about data-hiding.

Here are some recommendations to do instead:

Of course, mention that the structure is opaque in your documentation.
Override the Inspect protocol to make it clear in IEx, doctests and stacktraces that the internals of the struct are not to be relied upon. _{Elixir's own docs recommends this.}
Use Dialyzer to define your struct type as @opaque or @typep. This will not help at runtime, but it will give compile-time warnings (either in an IDE if elixir-ls is configured properly, or when dialyzer is run manually or as part of a CI build). _{Elixir's own docs recommends this.}
Use the excellent Boundary package/Mix plug-in which will restrict what modules can call what other modules. This enforces not only data hiding but full separation-of-concerns all throughout your project.
If you really want to enforce 100% correctness of the datatypes passed to your functions at runtime, you can use the TypeCheck package _{(disclaimer: TypeCheck was written by me)} which extracts the Dialyzer types to create guards for your functions.

Of course, all of these techniques still are a bit on a 'best effort' basis. If someone really really wants to circumvent these techniques, they can.

But that is an inherent property of any dynamically-typed programming language. (And, frankly, most statically-typed languages also have escape hatches by the way.)

Side note: Experimental techniques which are 'possible'

There are some more archaic approaches that go even further to do data hiding, but I would not recommend you use these anywhere other than in toy projects as they come with large trade-offs ^_^':

the calculus package implements private datastructures by leveraging closures ('lambda types'). Unfortunately, using these is 3x-10x slower than using structs or records.
You can define custom data structures in a statically-type language like Rust and expose these to Elixir using the NIF (Natively Implemented Function) interface, such as using rustler. This 'works' but it makes your code less portable, requires separate Rust compilation, requires you to work in both Elixir and Rust, and there are many other limitations with NIFs.

So, to conclude:

Just use structs, combined with above recommendations, if you care about hiding the internals of your data structures.
(Only) prefer records if you need Erlang interop, or if profiling shows that a part of your code currently is too slow and using records might make it fast enough.

Cheers!

~Marten / Qqwy

bchamagne • May 27 '23

Thank you Marten for this amazing reply!

I wonder how I missed the @typep, but as far as I can tell it doesn't help. Yes, we can't use Turtle.t() outside of the module but I am still able to create a %Turtle{} in Game and feed it to a Turtle module function and it'll work. Dialyzer does not complain.

But I realize also that I missjudged the @opaque attribute. I thought it was not good because I could create the struct outside the module but after more testing, you can't actually use it because dialyzer will warn you:

lib/game.ex:10:call_without_opaque
Function call without opaqueness type mismatch.

Which is great! Exactly what I wanted! I'll try to switch some @type to @opaque in the following weeks and see how it goes.

Someone also suggested boundary on reddit, and I'll definitely check it out, because in the end, what I am currently looking for is separation of concerns.

Cheers!