DEV Community: Marcio Lopes de Faria

A very simple scriptable Elixir server with tests

Marcio Lopes de Faria — Sun, 19 Sep 2021 02:18:06 +0000

Sometimes I miss the old days of Sinatra. It was so simple to build a really simple server, that we can do it in less than ten lines into two files, the Gemfile and the server.rb file.

We need to work a little more than expected with Elixir, but it's doable, and with a plus, it's not hard to bake tests inside your script.

Thanks to Mix.install/2 that is available since Elixir v1.12.0, its possible even to make it in one single file. Just place the snippet bellow in a single file, for example, server.exs

# Place this content on server.exs file

Mix.install([
  {:plug_cowboy, "~> 2.0"}
])

defmodule Server do
  use Plug.Router
  plug(:match)
  plug(:dispatch)

  get "/hello-world" do
    send_resp(conn, 200, "Hello World!")
  end

  match _ do
    send_resp(conn, 404, "Nothing to see here")
  end
end

ExUnit.start(autorun: false)

defmodule ServerTest do
  use ExUnit.Case, async: true
  use Plug.Test

  @opts Server.init([])

  test "returns hello world" do
    # Create a test connection
    conn = conn(:get, "/hello-world")

    # Invoke the plug
    conn = Server.call(conn, @opts)

    # Assert the response and status
    assert conn.state == :sent
    assert conn.status == 200
    assert conn.resp_body == "Hello World!"
  end

  test "returns 404 to any another route" do
    for path <- ["/foo", "/bar"] do
      conn = conn(:get, path)
      conn = Server.call(conn, @opts)

      assert conn.state == :sent
      assert conn.status == 404
      assert conn.resp_body == "Nothing to see here"
    end
  end
end

case ExUnit.run() do
  %{failures: 0} ->
    Plug.Cowboy.http(Server, [], port: 4000)
    System.no_halt(true)

  _ ->
    :error
end

And now just call in your shell:

elixir server.exs

Visit your browser at http://localhost:4000/hello-world to see the result.

This is another small and interesting tip to the mix of useful things that we can do with Elixir.

Happy Hacking Folks!

Inspector, a better way to debug Elixir Code

Marcio Lopes de Faria — Fri, 17 Sep 2021 20:03:10 +0000

EDIT: Since Elixir 1.14, there is a new macro dbg/2 that substitute the Inspector package.

I have been tinkering with better ways to improve my Elixir workflow and during this journey I realized that I am, somehow, trying to bring things which I'm used with from other environments, the same way a foreign citizen does when living in a different country.

Elixir prizes highly the Developer Experience, so that we have the astounding tools to generate documentation, first quality package management, and an excellent and mature build tool, and a REPL, iex that allows fast experiments, but I think that may I have pushed a lot the expected usage of it, in a not intended way as it was planned by the creators.

We can feel it during a IEx.pry/0 usage, it's a pretty limited tool, if you are accustomed to use Ruby pry for example, but given that Elixir doesn't hold mutable state, maybe it isn't necessary at all, so I started to think about the root cause of my dissatisfaction and I found it.

Debugging

Elixir is blaze fast, and we can notice it clearly when running a test suite with a lot of tests, they run really fast and the maintainers are working hard to make it even faster, so the so called Puts debugging is more than enough.

// Detect dark theme var iframe = document.getElementById('tweet-690689000617603072-855'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=690689000617603072&theme=dark" }

But what really hurts is to type always and everyplace the same think, IO.inspect/2, passing ad-hoc variables. It's not uncommon to make several calls to IO.inspect/2 during a debugging session, so if we improve it, enhancing the IO.inspect/2?

This is how Inspector was born, a library that makes debugging less annoying. You need only to place it in the calling site, like this:

# If this code is on lib/my_module.ex
defmodule MyModule do
  def my_function(args) do
    awesome_var = :foo
    amazing_var = :bar
    require Inspector; Inspector.i
  end
end

MyModule.my_function(:hey)

And you will see in the stdout:

inspector: %{
  bindings: [
    args: :hey,
    awesome_var: :foo,
    amazing_var: :bar
  ],
  file: "lib/my_module.ex:5",
  location: "MyModule.my_function/1"
}

If you uses Vim or Neovim, like me, you can even make it better with a key-map:

nnoremap <Leader>i orequire Inspector; Inspector.i<esc>

I hight recommend use it running this simple command:

// Detect dark theme var iframe = document.getElementById('tweet-1363791170745348097-482'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=1363791170745348097&theme=dark" }

Or with mix test.watch found here

That is it folks. Happy debugging!

Execute and Watch Elixir tests from iex.

Marcio Lopes de Faria — Tue, 14 Sep 2021 23:22:22 +0000

EDIT: I came up with a even better way to debug Elixir code than using iex, check it here.

Interactive Driven Development, or REPL Driven Development is a really good way to approach programming, specially for developers that get distracted easily with time being expend out the main development loop or flow.

A way to do it in Elixir is just to use the fswatch as presented in Elixir Mix Test documentation, or using the excellent mix.test-watch library, but them don't work well with iex, the Elixir REPL, so why not do something else.

So I created some modules that can be imported on your iex through the ~/.iex.exs file and get the same benefits.

Just insert it on your ~/.iex.exs file:

And create a ~/.iex directory, placing this file on it:

There is an explanition about how to use it on the @moduledoc, but briefly you just need to call iex prepending the ~/.iex directory on the Elixir path:

  ELIXIR_ERL_OPTIONS="-pa $HOME/.iex" MIX_ENV=test iex -S mix

The IExWatchTests.Helpers that is imported on ~/.iex.exs allows to call f and s and a to run failed, stale and all tests respectively.

You can call w to watch tests and uw to unwatch.

There is a really simple throttle mechanism that disallow run the suite concurrently.

That is all. Enjoy!

Small tweaks to run a Ruby on Rails project locally

Marcio Lopes de Faria — Thu, 02 Sep 2021 01:38:18 +0000

Sometimes you want just to tweak the way you run a Rails project, but without commit those changes upstream.

There are some tools that allow one to do it without much problem.

Direnv is your Ally

Dir env allows you to just ignore the rbenv .ruby-version or asdf .tool-version in a very easy way. For example if the project uses a ruby version 2.7.1 but you have installed the version 2.7.2, all you need to do is to put this on your .envrc file:

  load_prefix $HOME/.asdf/installs/ruby/2.7.2
  layout ruby

With this, you can force the version 2.7.2 upon the Project.

Gemfile manipulation

Imagine that the Project mandates you to use Passenger, but you want to run the project locally using Puma, but without commit it to the original Gemfile. One way to do it is just to create a Gemfile.local with:

eval_gemfile "Gemfile"

gem 'passenger', require: false
gem 'puma'

After it just, copy the original Gemfile.lock to Gemfile.local.lock and place add more one line to your .envrc file:

# what we add previously
load_prefix $HOME/.asdf/installs/ruby/2.7.2
layout ruby

# a new bundle gemfile
BUNDLE_GEMFILE=Gemfile.local

With this in place, just run the commands as usual:

bundle install
bundle exec rails server puma

Aditional notes

It's important to always resync the Gemfile.local.lock when some major modification on the original Gemfile.lock happens.
It's important to make sure that your .gitignore_global file will ignore properly the .envrc and Gemfile.local and Gemfile.local.lock files.

Acme Pub Sub, an Elixir exercise

Marcio Lopes de Faria — Mon, 23 Aug 2021 00:14:37 +0000

Recently I was rejected by a company that uses Elixir ostensively after expending a lot of energy engaging in the interview process, so I was thinking of ways to get something positive. I completed a specific exercise, but one hour is a small time-box to do something solid, and I think that the way I finished it was not what they expected, so I'm here learning in public, attempting a different approach.

The exercise mandates to transform the echo server https://elixir-lang.org/getting-started/mix-otp/task-and-gen-tcp.html#echo-server, into a broadcast pub-sub server.

My first approach was to make use of an Agent to maintain all accepted sockets. When some server receives a message from one client, it iterates over all other accepted sockets persisted on the Agent, sending each message back for each client socket. Pretty Simple!

This are all the Snippets of changes made to make this first attempt work.

# in lib/acme_pub_sub/application.ex
  def start(_type, _args) do
    port = String.to_integer(System.get_env("PORT") || "4040")

    children = [
      {Task.Supervisor, name: AcmePubSub.TaskSupervisor},
      %{
        id: AcmePubSub.ClientStorage,
        start: {Agent, :start_link, [fn -> [] end, [name: AcmePubSub.ClientStorage]]}
      }, # An supervised Agent was included to hold all clients
      Supervisor.child_spec({Task, fn -> AcmePubSub.accept(port) end}, restart: :permanent)
    ]

    opts = [strategy: :one_for_one, name: AcmePubSub.Supervisor]
    Supervisor.start_link(children, opts)
  end

# in lib/acme_pub_sub.ex

  defp loop_acceptor(socket) do
    {:ok, client} = :gen_tcp.accept(socket)
    Logger.info("Accepted new connection")

    {:ok, pid} = Task.Supervisor.start_child(AcmePubSub.TaskSupervisor, fn -> serve(client) end)

    # All clients are stored here.
    Agent.update(
      AcmePubSub.ClientStorage, 
      fn clients -> 
        [client | clients] 
      end
    )

    :ok = :gen_tcp.controlling_process(client, pid)

    loop_acceptor(socket)
  end

  defp serve(socket) do
    received_message = read_line(socket)
    Logger.info("received message #{received_message}")

    # get all clients
    clients = Agent.get(
      AcmePubSub.ClientStorage, 
      &Function.identity/1
    )

    # iterate over all clients, broadcasting all messages
    for client <- clients,
        socket != client do
      write_line(received_message, client)
    end
  after
    serve(socket)
  end

This first version is on github under the first-version branch:

https://github.com/marciol/acme_pub_sub/tree/first-version

A better approach

But it'd be better to encapsulate the access to client sockets so that only the specific process can read and write to their sockets.

An aside note: Writing tests for this specific kind of application is challenging, because of the need to ensure that all receiving test clients are set up just before starting to broadcast. Thanks to Elixir Tasks it is possible to start all receiving clients asynchronously and get all received messages directly on the caller test process, and with a little trick, get all moving parts synchronized.

So, with this working solution, it'd be nice to make it even better, establishing a way to broadcast messages without writing directly on those sockets.

One interesting approach would be to separate two kinds of servers, an input server responsible for listening to all received messages and dispatch them to the correspondent output servers.

The input server must know about the corresponding output server, so that if the client closes the socket, it sends a close signal to output server.

# in lib/acme_pub_sub.ex

  defp loop_acceptor(socket) do
    {:ok, client_socket} = :gen_tcp.accept(socket)
    Logger.info("Accepted new connection")

    # the output server task
    {:ok, output_pid} =
      Task.Supervisor.start_child(
        AcmePubSub.TaskSupervisor,
        fn ->
          output_server(client_socket)
        end
      )

    # the input server task
    {:ok, input_pid} =
      Task.Supervisor.start_child(
        AcmePubSub.TaskSupervisor,
        fn ->
          input_server({client_socket, output_pid})
        end
      )

    # all connected input and output corresponding clients
    Agent.update(
      AcmePubSub.ConnectedClients,
      fn clients ->
        [{input_pid, output_pid} | clients]
      end
    )

    loop_acceptor(socket)
  end

  defp input_server({client_socket, output_pid}) do
    case read_line(client_socket) do
      {:ok, data} ->
        # it will dipatch the message to all clients
        dispatch(data)
        input_server({client_socket, output_pid})

      {:error, :closed} ->
        # when closed, it will remove the client tasks
        Agent.update(
          AcmePubSub.ConnectedClients,
          fn clients ->
            List.delete(clients, {self(), output_pid})
          end
        )

        # it will send the close signal to the
        # corresponding output server
        send(output_pid, :close)

        :ok
    end

  # dispatching function
  defp dispatch(data) do
    clients = Agent.get(AcmePubSub.ConnectedClients, &Function.identity/1)

    # for each input/output process, different from itself
    # it will dispatch the message
    for {input_pid, output_pid} <- clients,
        input_pid != self() do
      send(output_pid, data)
    end
  end


  # the output server
  defp output_server(client_socket) do
    receive do
      # it will close if the input server was closed
      :close ->
        :ok

      # receive the dispatched message, 
      # sending out to client
      data ->
        write_line(data, client_socket)
        output_server(client_socket)
    end
  end

One alternative approach could be to encapsulate all this stuff in a GenServer, but it's an exercise for the reader, but I like this last solution because it shows how much it's possible with the Elixir primitives, Agent and Tasks.

The final solution is on the main branch of this repo:

https://github.com/marciol/acme_pub_sub