DEV Community: Rickard Andersson

You don't need the community

Rickard Andersson — Mon, 29 May 2023 10:58:13 +0000

If you're relatively new to software development and tend to keep up with developments in different areas of it you are probably convinced that community is important. I think you're probably overestimating the value of community.

Why are we here?

How did we end up with this view that a good language has a massive community and that it derives its value from this community?

Simple: Visibility.

A community that hosts conferences, has user groups, has thousands upon thousands of libraries readily available is visible. We can observe the mighty power of this language and technology and since these are some of the easiest things to observe we tend to associate this with high quality that is trending upwards. The community is sure to produce more libraries for us that we can use to build more things faster.

What can't we observe as easily?

We can't actually observe how many of these libraries are thrown away after a month because they have massive bugs, take forever to compile or are otherwise not good enough to use in our code bases.

We can't actually observe how many programmers hate coming in to work because the popular language they use has massive holes and is super easy to accidentally misuse. Likewise we can't observe how many other offerings the users of a technology have been exposed to, so we don't know whether their frame of reference even holds any reasonable alternatives. We have extremely imprecise proxies for some of these things but they aren't good enough.

What am I suggesting?

Recently there has been a series of controversies around the Rust language and its governance. I'm here to tell you that even before this you should've been unconcerned with most other people. The community doesn't matter; only you and your usage of the technology matters.

Learn your fundamentals, learn lower-level programming and how to express those things in Rust and engage minimally with everyone else. Perhaps engage with a community but not with the community. Staying on this path will help your objectivity as you will not add libraries to your projects "because everyone else uses this serialization library that adds 20 seconds to their compile time", you won't tolerate bad things just because it's the status quo of the community.

In short, you'll learn to think for yourself and evaluate both language features and libraries based on their actual merit and usefulness to you, not anyone else. When you need something, you'll be more prepared to build it unless you can easily find an alternative that actually works.

Knock-on effects

One of the best things about detaching yourself and your engagement from a community is that you free yourself up to simply do other things. Time spent engaging with the community can be spent doing actually productive things, spent researching alternatives that have unique upsides you didn't see before because you were so entrenched in this one technology and its community.

You can do all of these things without having to engage in spectacles like "Why I'm leaving X" posts, because you're not leaving anything. You're just also using other tools. You are a programmer, a software developer, you're not tied to any one language or technology.

Hopefully when you've disconnected from any one language and community you'll also be ready to learn things that simply aren't about languages, but rather an application of ideas such as network programming, 3D rendering, game engine design, etc.

purerl - Integrating PureScript into Elixir projects

Rickard Andersson — Wed, 11 Jan 2023 09:35:41 +0000

What is PureScript?

PureScript is a Haskell-like language aimed at providing an alternative to TypeScript for statically typed programming in the JavaScript space. I highly recommend taking a look at PureScript for your compile-to-JavaScript needs outside of the use case we'll be talking about in this post.

PureScript is most easily compared to TypeScript where it could be summed up as having many of the defaults you'd want in most TypeScript projects anyway, then layered with effects being noted in the types of functions on top.

What is Erlang?

Erlang is a language developed from the 80's onwards for the purposes of writing highly concurrent software. The primary purpose from the beginning was telephony systems where fault tolerance, concurrency and failure isolation were important concepts. These happen to be very important concepts in essentially all networked services nowadays.

What is Elixir?

Elixir is a collection of libraries, a syntax and a slim set of language features that enable a somewhat nicer workflow over Erlang. The libraries come into play when we want to use already created functionality in our PureScript code, as it might not be available in Erlang.

What is `purerl`?

purerl is a compiler for turning PureScript code into Erlang code, so that you're able to write BEAM (the Erlang virtual machine) applications using it.

This enables us to use it together with both Elixir and Erlang, which in a beautiful symbiosis extends the power of all three languages.

Our project setup

We'll start with an Elixir application that we'll add automatic compilation of *.purs files to, which will provide the perfect basis for getting things up and running quickly, easily and with a robust foundation to build on.

`mix`

In this particular example we'll use mix new, but mix phx.new will work exactly the same way and it's likely that any other new template would work as well. We're using mix because it's the standard tool for managing Elixir projects.

$ mix new purerl_up_and_running
...

When our project has been created we want to make sure that the versions we have of everything are clearly described. For this purpose we'll use asdf, a general version management tool we can use for all the tools we'll need in this project.

`asdf`

$ asdf plugin add erlang && asdf install erlang 25.1.2 && asdf local erlang 25.1.2
...
$ asdf plugin add elixir && asdf install elixir 1.14.2-otp-25 && asdf local elixir 1.14.2-otp-25
...
$ asdf plugin add purescript && asdf install purescript 0.15.3 && asdf local purescript 0.15.3
...
$ asdf plugin add spago && asdf install spago 0.20.9 && asdf local spago 0.20.9
...
$ asdf plugin add purerl && asdf install purerl 0.0.17 && asdf local purerl 0.0.17
...
$ asdf plugin add rebar && asdf install rebar 3.20.0 && asdf local rebar 3.20.0

The process for each tool here is to install the asdf plugin, a version of the tool and then set that version as the locally used one for our project. This ensures that other contributors can use the correct versions.

PureScript bits via `spago`

Initial setup

First we run spago init in our project folder:

$ spago init
...

This sets up our project to have basic PureScript files. The source directory for PureScript is src, so out of the box we are able to keep it separate from our Elixir code, which lives in lib.

Minor changes to our project files

We'll have to modify our spago.dhall and packages.dhall files just a bit in order for our project to point to the correct purerl things:

`packages.dhall`

diff --git a/packages.dhall b/packages.dhall
index e13009d..28bcb0c 100644
--- a/packages.dhall
+++ b/packages.dhall
@@ -99,7 +99,6 @@ in  upstream
 -------------------------------
 -}
 let upstream =
-      https://github.com/purescript/package-sets/releases/download/psc-0.15.3-20220712/packages.dhall
-        sha256:ffc496e19c93f211b990f52e63e8c16f31273d4369dbae37c7cf6ea852d4442f
+      https://github.com/purerl/package-sets/releases/download/erl-0.15.3-20220629/packages.dhall
 in  upstream

The change we're making here is that we are pointing the package set, the set of packages known to work together for this PureScript compiler version, to the purerl package set, i.e. a package set that uses Erlang code as the implementation language.

`spago.dhall`

The definition of our PureScript side of the project is done in spago.dhall and we'll add just one minor bit here:

diff --git a/spago.dhall b/spago.dhall
index 41a8576..a9b223a 100644
--- a/spago.dhall
+++ b/spago.dhall
@@ -14,4 +14,5 @@ to generate this file without the comments in this block.
 , dependencies = [ "console", "effect", "prelude" ]
 , packages = ./packages.dhall
 , sources = [ "src/**/*.purs", "test/**/*.purs" ]
+, backend = "purerl"
 }

The change above tells PureScript that we want to use a special compiler backend, i.e. purerl, in order to compile our PureScript code.

Elixir code needs to understand our project

In order to compile our PureScript code we'll rely on Elixir managing the compilation process as part of its normal compilation, via mix compilers.

`mix.exs`

diff --git a/mix.exs b/mix.exs
index 6ba9745..657162c 100644
--- a/mix.exs
+++ b/mix.exs
@@ -7,7 +7,9 @@ defmodule PurerlUpAndRunning.MixProject do
       version: "0.1.0",
       elixir: "~> 1.14",
       start_permanent: Mix.env() == :prod,
-      deps: deps()
+      deps: deps(),
+      compilers: [:purerl] ++ Mix.compilers(),
+      erlc_paths: ["output"]
     ]
   end

@@ -21,6 +23,7 @@ defmodule PurerlUpAndRunning.MixProject do
   # Run "mix help deps" to learn about dependencies.
   defp deps do
     [
+      {:purerlex, "~> 0.4.2"}
       # {:dep_from_hexpm, "~> 0.3.0"},
       # {:dep_from_git, git: "https://github.com/elixir-lang/my_dep.git", tag: "0.1.0"}
     ]

The package we are using is called purerlex and works out of the box as long as we also add that Erlang source files can be found in the output folder. We need to run mix deps.get in our project folder to get this dependency as well.

Adding PureScript code

We are now ready to add PureScript code that can be executed on the BEAM. We'll try this functionality out in iex, the interactive Elixir session that we can use to evaluate expressions.

`OurModule.purs`

Let's add a file in src/OurModule.purs containing the following:

module OurModule where

hello :: String
hello = "Hello!"

`iex`

Let's run iex -S mix in the project folder and execute the following:

iex(1)> :ourModule@ps.hello()
"Hello!"

Note here that our module name is being translated in a particular way. First of all it's converted into camel-case from the PascalCase standard that PureScript modules use. On top of that modules are also compiled with a @ps suffix that is very close to the Elixir. prefix that Elixir modules are compiled with, to distinguish the modules from modules generated with other languages.

Where to go from here

We're now able to compile and run PureScript code so that we can use it in the rest of our Elixir code. This means we can write entire subsystems in PureScript instead and start them in our application supervisor, meaning they are started from the root of our application. If we have a subsystem that is best expressed in a statically typed language this is now something we can cover as a use case.

In the next article we'll go over how to write one of these subsystems so that we can take advantage of this newfound ability to compile and run PureScript code in our project.

ERTS & OTP - The Erlang Runtime System and core library

Rickard Andersson — Sun, 18 Dec 2022 17:06:48 +0000

Since I will be writing posts about how to do things on the BEAM with different languages I thought it might be prudent to create a short intro to the systems that power the programs we write.

If you prefer to look at things yourself, the code that will be referenced later is available here.

BEAM

The name "BEAM" comes originally from Bogdan's/Björn's Erlang Abstract Machine, named after the first and subsequent implementer & maintainer of the virtual machine. There were previous, distinct versions of this concept before, notably JAM (Joe's Abstract Machine), created by Joe Armstrong.

BEAM is a bytecode interpreter, meaning it takes a well-defined, high-level description of logic and executes that on whatever platform we are running on, as opposed to compiling to native code immediately. This is often implicit in the phrase "virtual machine". You can imagine this as a adapter machine running on top of the one actually executing things.

We won't go over the internals of the BEAM in this article, but rather focus on the capabilities offered by the Erlang Runtime System. If you're interested in how the BEAM works, this book by Erik Stenman is a great place to start.

Erlang Runtime System

The Erlang Runtime System is the software that runs your code on top of the BEAM. Since the BEAM is just a bytecode interpreter, any logic that needs to manage the scheduling and de-scheduling of code on top of that needs to have a system taking care of it. That is exactly what ERTS is.

Lightweight processes

Processes are threads managed by ERTS and all code that runs on it is executed in these processes.

Processes:

Take up very little heap space by default; about 1232 bytes
Have message queues (the primary way to interact with them is messages)
Don't really use any processing power unless they have waiting messages (see below)

Preemptive scheduling

ERTS has a scheduler that decides which processes are allowed to do work at the moment. It does so via a list of ready and waiting processes. A process is put in a ready state if it has a message in its message queue, and put in the waiting state if it asks for incoming messages again, or consumes too many resources while processing a message.

This last part is distinct from most models of concurrency and parallelism; we do not usually have a limit on how much we can do in a thread before it gets de-scheduled. ERTS has an internal concept of reductions. A process has a set amount of reductions it can consume/do before the scheduler determines that it has to let other processes run. You can view this concept as having a set amount of fuel to consume before you have to make a pit stop.

The above concept means that threads are scheduled fairly and without any user intervention on the BEAM. We don't ever have to "yield" a thread, it's a concept managed entirely by the runtime itself. The absence of preemptive scheduling can be seen in languages where starting a thread can completely shut off a core from doing anything else while it's running, because we are locking that core up entirely until we are done, or until we yield.

Because we don't use a lot of memory for each process and that a process doesn't use any processing power unless it has incoming messages it is very common to start very high numbers of them if we need to do so. We can be confident that they are managed as they should be by the runtime.

OTP

We've learned that code running in the BEAM is intrinsically tied to the concept of processes and that the way we do things with processes is by way of messages; pieces of data that a process can react and reply to. So how do we deal with processes from a code perspective?

OTP, the Open Telecom Platform, is a library for starting, managing, communicating with and stopping processes. We'll go over some of the most commonly used components of this library to give a sense of what it means to use it.

`gen_server` / `GenServer`

As the name suggests, gen_server is for generic servers. You can view a server here as something that accepts and potentially responds to messages and manages an internal state.

As an example of a server, let's look at how a very simple server for storing a list of things and getting the current list might look:

gen_server (Erlang)

-module(beam_intro_lister).

-export([start_link/0, init/1, add/1, get/0, handle_call/3,
         handle_cast/2, handle_info/2]).

-behaviour(gen_server).

start_link() ->
  % We specify how to start the process. In this case we don't use the
  % arguments coming in at all, and pass down only the symbol `ok` to
  % our `init` function, which is used for determining the initial
  % state of the process.
  gen_server:start_link({local, ?MODULE}, ?MODULE, ok, []).

add(Thing) ->
  gen_server:cast(?MODULE, {add, Thing}).

get() ->
  gen_server:call(?MODULE, get).

init(_Args) ->
  % The initial state of the process is a map where the key `contents`
  % is associated with an empty list.
  {ok, #{contents => []}}.

% We use pattern matching here to pull out the `contents` value so we
% can use it in our logic. When a thing is added, we prepend it to our
% internal list of things.
handle_cast({add, Thing}, #{contents := OldContents}) ->
  {noreply, #{contents => [Thing | OldContents]}}.

% When someone requests the contents of our process, we reply to their
% call with the `contents` value in our state.
handle_call(get, _From, #{contents := Contents} = State) ->
  {reply, Contents, State}.

handle_info(_Info, State) ->
  {noreply, State}.

GenServer (Elixir)

defmodule BeamIntro.Lister do
  use GenServer

  def start_link(_args) do
    # We specify how to start the process. In this case we don't use
    # the arguments coming in at all, and pass down only the symbol
    # `ok` to our `init` function, which is used for determining the
    # initial state of the process.
    GenServer.start_link(__MODULE__, :ok, name: __MODULE__)
  end

  def add(thing) do
    GenServer.cast(__MODULE__, {:add, thing})
  end

  def get() do
    GenServer.call(__MODULE__, :get)
  end

  def init(:ok) do
    # The initial state of the process is a map where the key
    # `contents` is associated with an empty list.
    {:ok, %{contents: []}}
  end

  # We use pattern matching here to pull out the `contents` value
  # so we can use it in our logic. When a thing is added, we prepend
  # it to our internal list of things.
  def handle_cast({:add, thing}, %{contents: contents} = state) do
    {:noreply, %{state | contents: [thing | contents]}}
  end

  # When someone requests the contents of our process, we reply to
  # their call with the `contents` value in our state.
  def handle_call(:get, _from, %{contents: contents} = state) do
    {:reply, contents, state}
  end
end

gen_server follows a fairly simple request/response model and if we can answer some key questions we can construct one:

start_link & init: How is the server started and what does that mean for the initial state?
Interface functions: Which messages do we want to send when other processes interact with the server?
handle_call, handle_cast & handle_info: When those messages come in, how does our internal state change and what do we reply with?

With these key questions answered we can cover a surprisingly large area in the design space of a system.

`supervisor` / `Supervisor`

Supervisors are processes that manage the starting, restarting and stopping of other processes.

supervisor (Erlang)

-module(beam_intro_lister_supervisor).

-export([start_link/0, init/1]).

-behavior(supervisor).

start_link() ->
  supervisor:start_link({local, ?MODULE}, ?MODULE, []).

init([]) ->
  Flags =
    #{% We specify here that the supervisor restarts a single process when it
      % dies, not all the processes attached to the supervisor.
      strategy => one_for_one,
      % If 3 children die in 5 seconds, the supervisor will terminate. If this
      % supervisor is a child of another supervisor, the parent supervisor will
      % react to that termination as specified.
      intensity => 3,
      period => 5},
  % This describes how our child process is started.
  ListerSpec =
    #{id => beam_intro_lister,
      start => {beam_intro_lister, start_link, []},
      restart => permanent,
      shutdown => 5000,
      type => worker},
  {ok, {Flags, [ListerSpec]}}.

Supervisor (Elixir)

defmodule BeamIntro.Lister.Supervisor do
  use Supervisor

  def start_link(_args) do
    Supervisor.start_link(__MODULE__, :ok, name: __MODULE__)
  end

  def init(:ok) do
    children = [
      {BeamIntro.Lister, []}
    ]

    Supervisor.init(
      children,
      # We specify here that the supervisor restarts a single process
      # when it dies, not all the processes attached to the supervisor.
      strategy: :one_for_one,
      # If 3 children die in 5 seconds, the supervisor will terminate.
      # If this supervisor is a child of another supervisor, the parent
      # supervisor will react to that termination as specified.
      max_restarts: 3,
      max_seconds: 5
    )
  end
end

Supervisors allow us to talk about how a process is started, under which circumstances it should be restarted as well as how many failures we can tolerate before we need to restart the entire supervisor.

Supervisors come with a simple but featureful set of configuration options that we can use to design our system:

Restart strategies: Should we restart only the process that has failed, or should we restart all the supervisor's children, or perhaps all the ones that come after the failed one in our child list?
How many process failures in how much time do we tolerate before we restart the entire supervisor and all its children?
Which processes are important to always have up even though they may have shut down normally? Are some processes fine to not restart if they shut down normally, but should be restarted if they crashed?

Answering the above questions is key to understanding what role a supervisor and its children have in a system.

Supervision trees

Supervision trees are an emergent property of the above questions. Because we can have supervisors as children of other supervisors, this means we can segment our systems up in sub-systems that might be important to keep together.

If one sub-system depends on another, we might for example say that our restart strategy is rest_for_one (restart all subsequent children when one fails) and have them be part of the same supervision tree. This allows us to guarantee that the dependent one is always started after the one it depends on.

If you take the time to look, you'll find that most BEAM applications are actually started by a root supervisor process usually called some variant of application/Application. This means that even the most minimal system will tend towards some level of supervision tree structure.

Seeing the forest and not just the trees

What these things and more boil down to is that on the BEAM we are able to use high-level specifications and libraries to talk about the most interesting parts of a system:

Which parts go together?
How do the parts communicate change and effect?

Being allowed to take on this high-level view is very liberating and is why I personally feel that the BEAM & ERTS is a great place to be as a creator and maintainer of systems.

API constraints a'la carte in Haskell & PureScript

Rickard Andersson — Wed, 08 May 2019 21:41:31 +0000

The example in this post is 100% compatible in PureScript as well, with the exception of the effect monad which is called Effect in PureScript. Replacing IO with Effect is all you have to do, as well as use the libraries available there.

I think everyone who's in and around Haskell & PureScript hear some variant of "Every program you make always ends up having a bunch of IO () in it all the time anyway." and they're not necessarily wrong. The complaint is that you have this neat type system that should constrain your effects but all you get is IO or Effect, which just plain permits everything.

It's not untrue, but it's not exactly the full story either. Consider the following function:

downloadFile' :: Link -> IO (Response ByteString)
downloadFile' (Link link) =
  Http.get link `Exception.catch`
  (\(HttpExceptionRequest req (StatusCodeException resp bytestring)) -> do
     case resp ^. responseStatus . statusCode of
       404 -> putStrLn $ "ERROR: File not found (404) for " <> link
       code ->
         putStrLn $ "ERROR: Unknown error with code " <> show code <> " for " <>
         link
     pure $ fmap (const (LBS.fromStrict bytestring)) resp)

We can of course see that we're trying to download a file. But there's a lot going on that doesn't really have anything to do with downloading files in here. In total, the effects we are dealing with:

We're using HTTP functions: Http.get
We're dealing with exceptions: Exception.catch
We're printing to the terminal: putStrLn

How can we be clearer in our types about what we're doing in a lightweight way?

Well, let's make some constraints:

class MonadTerminalIO m where
  putStrLnM :: String -> m ()

instance MonadTerminalIO IO where
  putStrLnM = putStrLn

class MonadHttp m where
  httpGetM :: String -> m (Response LBS.ByteString)

instance MonadHttp IO where
  httpGetM = Http.get

Exception.catch already has an associated type class/constraint, so we don't need to make that one.

We can now transform our function to the following:

downloadFile ::
     (MonadHttp m, MonadTerminalIO m, Exception.MonadCatch m)
  => Link
  -> m (Response ByteString)
downloadFile (Link link) =
  httpGetM link `Exception.catch`
  (\(HttpExceptionRequest req (StatusCodeException resp bytestring)) -> do
     case resp ^. responseStatus . statusCode of
       404 -> putStrLnM $ "ERROR: File not found (404) for " <> link
       code ->
         putStrLnM $ "ERROR: Unknown error with code " <> show code <> " for " <>
         link
     pure $ fmap (const (LBS.fromStrict bytestring)) resp)

We're now being a lot clearer with what we're doing in our type signature and all it took was a couple of type classes and a generic return monad type.

Because we now return m (Response ByteString) and have a set of constraints on m instead, we've guaranteed that there can't be any random IO or other effects in our function, because those are in fact not valid for any m the type system can imagine. When we try to add something that can talk to the network, for example, it would have to have an associated constraint called MonadNetwork, for example, and we would have to add it to our constraints to make those functions available in that scope.

If we find ourselves wanting better type signatures, they could be just a few small constraints / type classes away. It's a very effective way to limit the capabilities of a function and be very clear about what's happening inside of it.

This is all possible because of type classes which serve as constraints on generic type variables, as well as higher-kinded types that allow us to talk generically about types wrapping types and together they form something I really like about Haskell & PureScript.