DEV Community: bakenator

Visualize NodeJS Errors in Real Time with Llama Logs

bakenator — Wed, 26 Aug 2020 02:20:11 +0000

What?

Have you ever wondered what is going on inside your program? Wanted a visual way to inspect its inner workings?

The gif above shows an example of Llama Logs. It is a new tool I created to let you see the inner workings of your applications in real time. It is ready for you to start using in your app for free today!

Below I will walk through an example of using Llama Logs to show and debug errors in a basic express app as they happen.

The Setup

I am going to write a basic express app that takes in a user's email through a url param and saves it to the database if the email is a llamalogs.com domain.

The basic logic will look like this

app.get('/', (req, res) => {
  let customerEmail = req.query.email
  let isDomainOk = domainCheck(customerEmail)

  if (isDomainOk) {
      saveEmail(customerEmail)
  }

  res.send('We received your email')
})

Now the doozy is that I am going to write some horrible code that errors if the user forgets to include the @domain part of their email.

const domainCheck = (customerEmail) => {
  // toLowerCase will fail if the [1] value is undefined!
  const domain = customerEmail.split("@")[1].toLowerCase()
  const domainIsOk = domain === "llamalogs.com"
  return domainIsOk
}

Visualizing with Llama Logs

Llama Logs is incredibly simple to set up. Once you sign up on llamalogs.com, all you need to do is install the client via npm and then start logging. Llama Logs will automatically turn your logs into an interactive graph.

So for example lets update the domainCheck method to the following

const domainCheck = (customerEmail) => {
  try {
    const domain = customerEmail.split("@")[1].toLowerCase()
    const domainIsOk = domain === "llamalogs.com"

    LlamaLogs.log({ sender: 'Server', receiver: 'Domain Check' })

    return domainIsOk

  } catch (e) {
    LlamaLogs.log({ 
      sender: 'Server', 
      receiver: 'Domain Check', 
      message: `input: ${customerEmail}; Error: ${e}`,
      isError: true
    })
  }
}

So we have added a logging case for both the success and failure outcomes. Llama Logs will then use the names provided in the "sender", "receiver", and "isError" attributes to automatically visualize the activity in your app as a series of points moving between components.

In the graph below we can see the result of running a few calls to the server with valid emails and ones that cause errors.

Debugging

Even better than visualizing the activity in the graph, Llama Logs will let you get data from your errors in real time.

Remember in the domainCheck method we attached this attribute to the Llama Log?

message: `input: ${customerEmail}; Error: ${e}`,

By using this message attribute, that means that when we hover on the red error point, it will show the message. The image below comes shows me hovering on an error. The request it represents had the email param == "jd", missing the email domain.

By using Llama Logs to visualize errors in your system, you can identify the sources of bugs faster and easier than ever before!

More Info

Please visit LlamaLogs.com for more information if you are interested. The app is free and ready to use today. Please feel free to contact me at andrew@llamalogs.com if you have any questions.

Full Code

This is such a small express app I think it is easiest to include all of the code within this blog post.

const express = require('express')
const { LlamaLogs } = require('llamalogs');

LlamaLogs.init({
    accountKey: 'YOUR_ACCOUNT_KEY',
    graphName: 'YOUR_GRAPH_NAME'
});

const app = express()
const port = 3000

app.get('/', (req, res) => {
  LlamaLogs.log({ sender: 'User', receiver: 'Server' })

  let customerEmail = req.query.email
  let isDomainOk = domainCheck(customerEmail)

  if (isDomainOk) {
      saveEmail(customerEmail)
  }

  res.send('We received your email')
})

app.listen(port, () => {
  console.log(`Example app listening at http://localhost:${port}`)
})


const domainCheck = (customerEmail) => {
  try {
    const domain = customerEmail.split("@")[1].toLowerCase()
    const domainIsOk = domain === "llamalogs.com"

    LlamaLogs.log({ sender: 'Server', receiver: 'Domain Check' })

    return domainIsOk

  } catch (e) {
    LlamaLogs.log({ 
      sender: 'Server', 
      receiver: 'Domain Check', 
      message: `input: ${customerEmail}; Error: ${e}`,
      isError: true
    })
  }
}

const saveEmail = (customerEmail) => {
    // pretend we are saving to a database here
    LlamaLogs.log({ sender: 'Domain Check', receiver: 'Database' })
}

Visualize Your Elixir/Phoenix App with Llama Logs

bakenator — Mon, 17 Aug 2020 01:03:06 +0000

About

Do you ever wonder what is going on inside your Phoenix application? Struggle to remember exactly how that data is processed? Llama Logs can bring an end to these problems.

Llama Logs is a brand new app that can automatically create real time interactive graphs of the activity within your Phoenix application. Llama Logs also seamlessly works with JS, Go, and Python. It is ready for you to start using in your application for free today!

Below I will show you how to add Llama Logs to an example Phoenix application and visualize whats going on in the server.

Example App

For this post, I will set up and use a new Phoenix API application. This is a very simple Phoenix app. It will have only one resource, Projects. Projects will have 3 attributes. The type attribute will either be "Internal" or "External".

  schema "projects" do
    field :name, :string
    field :status, :string
    field :type, :string

    timestamps()
  end

Adding Llama Logs

Llama Logs is very simple to add to an application. There is a full getting started guide here: https://llamalogs.com/docs/getting-started

Once Llama Logs is initialized, just a few log statements need to be added to generate a real time visual graph.

For this application I thought it would be cool to visualize the incoming traffic based on whether the projects were internal or external. This can be accomplished by adding Llama Log statements into the controller.

Here is an example of the changes to the create function in the project_controller.ex file:

def create(conn, %{"project" => project_params}) do
    with {:ok, %Project{} = project} <- HelloProjects.create_project(project_params) do
      LlamaLogs.log(%{sender: "User", receiver: project.type})
      LlamaLogs.log(%{sender: project.type, receiver: "#{project.type} Create"})
      LlamaLogs.log(%{sender: "#{project.type} Create", receiver: "Database"})

      conn
      |> put_status(:created)
      |> put_resp_header("location", Routes.project_path(conn, :show, project))
      |> render("show.json", project: project)
end

Note that the values used for sender and receiver are evaluated dynamically. This makes it possible to use one statement for multiple data paths.

Llama Logs Graph

The Llama Logs client will automatically aggregate and send the data from those statements into the Llama Logs servers. Once there, a graph of the information is created in real time to show the ongoing activity in the server.

The graph below shows the activity within the example project with users creating, viewing, and updating their project models.

Each component in the graph is auto generated based on the strings in your Llama Log statements.

Each shape moving between components represents multiple events. This way, up to millions of events can be visualized simultaneously. The key of what each shape represents is shown below.

Errors

The key shows error logs as well. This is a neat feature of Llama Logs that lets you dynamically catch and visualize errors in your Phoenix application.

With these lines added to the example app, errors during the project creation call can be caught. Note the is_error field in the third statement.

  def create(conn, %{"project" => project_params}) do
    with {:ok, %Project{} = project} <- HelloProjects.create_project(project_params) do
...
    else 
      result -> 
        LlamaLogs.log(%{sender: "User", receiver: project_params["type"]})
        LlamaLogs.log(%{sender: project_params["type"], receiver: "#{project_params["type"]} Create"})
        LlamaLogs.log(%{sender: "#{project_params["type"]} Create", receiver: "Database", is_error: true})
        conn 
        |> put_status(:bad_request)
        |> render("error.json", %{})
    end
  end

I set the name field on a project to be unique, so errors can be triggered by creating a project with a pre existing name.
And then below we can see in real time the errors that I set up while creating external projects.

Resources

All the code used for this example Phoenix app can be found in this repo:
https://github.com/llamalogs/PhonixBlogExample

More information on Llama Logs and the Elixir/Phoenix developer guide can be found at:
https://llamalogs.com/docs

Please reach out if you have any questions or need help getting Llama Logs set up in your Phoenix application. I can be reached at andrew@llamalogs.com

Intro to OOP in Elixir

bakenator — Wed, 01 May 2019 20:41:50 +0000

Who is This For?

Like many others I came to the Elixir language through the Phoenix framework and then became interested in writing Elixir outside of Phoenix. But with its immutability and lack of class methods, my biggest challenge was figuring out how to get anything done.

This post is meant to go over some of the big lessons I learned about how to transition my OOP programming knowledge into productive Elixir code. Most of these points are very simple, they just need to "click" in your thinking.

Functions and Data are separate

In most OOP languages you use instances of classes (objects) to do the work of our program. Each instance has a self contained state and then functions that can do work by referencing that state. For instance a simple Ruby class may look like this.

class Person
   def initialize(name)
      @cust_name = name
   end

   def name_backwards
      @cust_name.reverse
   end
end

person_1 = Person.new('Ted')
person_1.name_backwards
# outputs "deT"

Elixir functions can not have any references to this, @, or any other instance variables. So the way to do something similar in Elixir is by having a function take the data (instance) as the first parameter. Here is the Ruby code above rewritten in Elixir.

person = %{name: "Ted"}

defmodule Person.Functions do
  def name_backwards(person) do
    String.reverse(person.name)
  end
end

Person.Functions.name_backwards(person)

In the code above we can see that the data of a person instance is kept completely outside of the Person.Functions module. When we want to use a function on the person instance, we pass it into a function that takes a person instance as the first parameter.

In practice a module would never be named Person.Functions because by definition the module is not tied to a person instance. The name_backwards function could be used on any object with a name field.

Update by Replacement

One of the most common things to do in OOP is to use an instance of an object to track some state. For example in Ruby

class Person
   def initialize(age)
      @person_age = age
   end

   def add_year
      @person_age = @person_age + 1
   end

   def get_age
      @person_age
   end
end

person_1 = Person.new(30)
person_1.add_year
person_1.add_year
person_1.get_age
# outputs 32

In Elixir all objects are immutable, so how on earth can we update a field of this person instance? It can be done by making a new person instance and replacing the old one.

person = %{age: 30}

defmodule Person.Functions do
  def add_year(person) do
    new_age = person.age + 1
    %{age: new_age}
  end

  def get_age(person) do
    person.age
  end
end

person = Person.Functions.add_year(person)
person = Person.Functions.add_year(person)
Person.Functions.get_age(person)

So instead of changing the value of the field within the instance, we return a new instance in the line %{age: new_age}.

Global Objects are stored in Processes

I am not advocating for the use of global state, however I do think this idea is fundamentally important to learning the basics of Elixir. Also this may be a contrived example, but I think it will show the challenge of translating OOP to Elixir.

I added a ticket class which dispenses tickets. It keeps track as each ticket goes out so that each person gets a unique ticket. Fairly simple in Ruby since we can use the static field @@ticket_count to keep track of the tickets.

class Tickets
  @@ticket_count = 1
  def self.get_ticket
    next_ticket = @@ticket_count
    @@ticket_count = @@ticket_count + 1
    next_ticket
  end
end

class Person
   def initialize()
      @ticket_num = -1
   end

   def get_ticket
    @ticket_num = Tickets.get_ticket
    @ticket_num
   end
end

person_1 = Person.new()
puts person_1.get_ticket
#outputs 1
person_2 = Person.new()
puts person_2.get_ticket
#outputs 2

So how would we keep track of the ticket counts in Elixir? By using what is called a GenServer.

defmodule Tickets do
  use GenServer

  def start_link(start_ticket_num) do
    GenServer.start_link(Tickets, start_ticket_num, name: Tickets)
  end

  def get_ticket() do
    GenServer.call Tickets, :get_ticket
  end

  def handle_call(:get_ticket, _from, curr_ticket_num) do
    {:reply, curr_ticket_num, curr_ticket_num + 1}
  end
end

defmodule Person.Functions do
  def get_ticket(person) do
    next_ticket = Tickets.get_ticket()
    %{ticket_num: next_ticket}
  end
end

Tickets.start_link(1)
person_1 = %{ticket_num: -1}
IO.inspect Person.Functions.get_ticket(person_1)
#output %{ticket_num: 1}
person_2 = %{ticket_num: -1}
IO.inspect Person.Functions.get_ticket(person_2)
# %{ticket_num: 2}

Now this is getting more advanced, but it is the real beauty of Elixir. In the start_link(1) function, what we did was start another process with an instance of a GenServer named Tickets with an initial value of 1.

GenServers can do a lot of things, but here we are using it to store the current ticket number. Each time we call get_ticket the code flows through to the response {:reply, curr_ticket_num, curr_ticket_num + 1} which is an order to the GenServer basically formatted as {command_to_genserver, response_value, new_state_value}. Now any Elixir module can get a ticket just by calling the Tickets.get_ticket function.

In most OOP languages, starting another process is a very high level task. In fact a dev can go for years more or less only working in the main process. In Elixir however working with other processes is a fundamental skill.

If you want to learn more about GenServers I would highly recommend writing your own! Here is a tutorial

Wrap Up

These were the first few things that came to my mind when thinking back about the challenges of learning Elixir. Hope one of them was helpful for you.

Coding in Elixir is Fun!

A Tour of Task Async and Await

bakenator — Thu, 18 Apr 2019 02:16:50 +0000

One great thing about Elixir is that the source code for all the language modules is easily examinable. For a personal project I was recently considering using the Task module instead of a spawn, but then I asked myself...

How do Tasks Work?

If you look at the documentation for the Task module in Hex Docs it says the most common use case for tasks is to convert sequential code into concurrent code by computing a value asynchronously, then it provides the example below.

task = Task.async(fn -> do_some_work() end)
res = do_some_other_work()
res + Task.await(task)

In order to understand how Tasks work we can dive into the Task source code for each of the two functions used above.

Task.async

First there is a call to Task.async. Below is the source code for both definitions of the function.

  def async(fun) when is_function(fun, 0) do
    async(:erlang, :apply, [fun, []])
  end

  def async(module, function_name, args)
    when is_atom(module) and is_atom(function_name) and is_list(args) do
    mfa = {module, function_name, args}
    owner = self()
    {:ok, pid} = Task.Supervised.start_link(get_owner(owner), get_callers(owner), :nomonitor, mfa)
    ref = Process.monitor(pid)
    send(pid, {owner, ref})
    %Task{pid: pid, ref: ref, owner: owner}
  end

Task.async/1 is a catch all function that passes the param function to Task.async/3 with :erlang.apply.

{:ok, pid} = Task.Supervised.start_link(get_owner(owner), get_callers(owner), :nomonitor, mfa)

In this line of Task.async/3 the current pid (owner) and original param function (mfa) is passed to the Task.Supervised as params to the Task.Supervised.start_link function. start_link then spawns a new process and returns its pid.

send(pid, {owner, ref})

Next a copy of the owner_pid (owner) and a monitor reference (ref) is sent to the new Task.Supervised process. These are what the Task.Supervised process will use to identify itself to Task.await when it finishes running the function.

%Task{pid: pid, ref: ref, owner: owner}

Finally Task.async/3 returns a Task struct to be used in Task.await/2 for identification.

Next lets take a look what was going on inside Task.Supervised that was spawned

Task.Supervised

def start_link(owner, callers, monitor, fun) do
  {:ok, :proc_lib.spawn_link(__MODULE__, :reply, [owner, callers, monitor, fun])}
end

defp reply(owner, owner_pid, mref, timeout, mfa) do
    receive do
      {^owner_pid, ref} ->
        _ = if mref, do: Process.demonitor(mref, [:flush])
        send(owner_pid, {ref, invoke_mfa(owner, mfa)})
    ...

{:ok, :proc_lib.spawn_link(__MODULE__, :reply, [owner, callers, monitor, fun])}

Task.Supervised.start_link starts a new process and calls its own reply function.

receive do
{^owner_pid, ref} ->

In the lines copied above, we can see that the first thing reply does is to wait for the {owner, ref} tuple that was sent up in the Task.async function.

send(owner_pid, {ref, invoke_mfa(owner, mfa)})

After it is received Task.Supervised runs the original user provided function in invoke_mfa(owner, mfa). invoke_mfa basically uses apply(module, fun, args) internally with additional error handling.

The result of the original function is then sent back to the owner process (wherever we ran Task.async). This message back to the owner process will be picked up by Task.await

Task.await

  def await(%Task{ref: ref, owner: owner} = task, timeout \\ 5000) when is_timeout(timeout) do
    ...
    receive do
      {^ref, reply} ->
        Process.demonitor(ref, [:flush])
        reply
    ...

def await(%Task{ref: ref, owner: owner} = task, timeout \\ 5000) when is_timeout(timeout) do

Task.await takes a Task struct and pulls out the ref attribute.

receive do
{^ref, reply} ->

The function then calls receive and waits for any incoming messages. Each message from Task.Supervised has a ref it uses as an id to identify itself to Task.await.

reply

If the ref is a match, the result is returned.

Summary of Task.async/Task.await lifecycle

The function fn -> do_some_work() end was:

Given to Task.async as a param
Within Task.async, given to a new Task.supervised process as a param
Run in the Task.supervised process
The result was sent in a message from the Task.supervised process back to the original process
The message from Task.Supervised is caught in Task.await and returned

Why Use a Ref for Identification?

Task.Supervised processes identify themselves with a ref which was initialized all the way back in the line ref = Process.monitor(pid). Why does it do this when we already have a variable to identify a process, pid?

To be honest I dug around and looked through the source for a while and could not find a specific reason to use ref as an identifier instead of the pid. It has been set up this way since the start of the Elixir source code. If anyone has insights on this, I would be interested to know.

Out of Order Tasks?

From this inspection of Tasks I learned something very valuable about messages between processes.

Messages may not be processed in the order in which they are received!

In the Task.await function there is no base case for the receive pattern matching. If the message ref does not match, then there is no match. What happens in this case? An example:

task1 = Task.async(fn ->
  :timer.sleep(10_000)
  1 
end)

task2 = Task.async(fn -> 2 end)

IO.inspect Task.await(task1)
IO.inspect Task.await(task2)

Here we can see that Task 1 is going to stall for 10 seconds before it returns. So Task 2 is going to finish and send its response before Task 1. But Task 1 is the first to block waiting for its response. What is the output?

1
2

Even though the messages with the function results were received in the order [task2, task1] everything still worked.

This is because if receive finds no match for the latest incoming message, it simply puts that message back in the queue and keeps on waiting. Then each time a new message is received it checks again and keeps blocking until a match is found.

This also means that if there are already matching messages in the queue, receive does not need to wait at all. It will check all the existing messages in the queue and move forward if one is found to match. That happens in this case within Task.await(task2)

Hope you found this dive into the Task async/await functionality interesting.

Coding in Elixir is Fun!

Source code for the Task files can be found here:
Task Module
Task.Supervised Module

A Deep Dive into Mix

bakenator — Thu, 11 Apr 2019 01:40:30 +0000

This post started out its life as a short example of bubble sort in elixir. But I encountered an interesting message when I tried to run my script with mix run bubble.exs

** (Mix) Cannot execute "mix run" without a Mix.Project, please ensure you are running Mix in a directory with a mix.exs file or pass the --no-mix-exs flag

In this case since the file is a true one page script with no dependencies, I could have run elixir bubble.exs

But this error message caught my attention and led to some bigger questions. How does this flag work? How does Mix work?

Mix Tasks

In this example we are using mix run so I thought I would try to track down the run command in the mix source code. Mix is a sub application within the Elixir source code. So I went to the Elixir repo and tried a search for "def run".
Repo: https://github.com/elixir-lang/elixir/tree/master/lib/mix/lib

After poking around for a while I got lucky and found the module Mix.Tasks.Run. It appears that the "run" command is a Task within Mix. This fits with the high level overview of mix provided in the top level Mix module file.

Basically each of the default commands that you can provide to Mix in the command line is a separate task. Each task has a corresponding module with a "run" method that is called.

Tracing a Mix Command

Now that we know Mix is passing our run command to the Mix.Tasks.Run module, lets dig in.

The file starts with a catch all run method:

  def run(args) do
    {opts, head} =
      OptionParser.parse_head!(
        args,
        aliases: [r: :require, p: :parallel, e: :eval, c: :config],
        strict: [
          parallel: :boolean,
          require: :keep,
          eval: :keep,
          config: :keep,
          mix_exs: :boolean,
          halt: :boolean,
          ...
        ]
      )

    run(args, opts, head, &Code.eval_string/1, &Code.require_file/1)
    unless Keyword.get(opts, :halt, true), do: System.no_halt(true)
    Mix.Task.reenable("run")
    :ok
  end

This method parses the command line options and passes the parsed data to a run method with more parameters. Interesting that the --no-halt option is handled at this level.

Also deep in the OptionParser file, I found this line interesting.
defp tag_option("no-" <> option = original, config) do
The no- syntax is used as a switch for options. I always assumed that --halt and --no-halt would just be identified as separate strings

Next up we have the run method with expanded parameters.
def run(args, opts, head, expr_evaluator, file_evaluator) do
First it does some more processing of the command line args, then we reach this crucial code block.

cond do
      Mix.Project.get() ->
        Mix.Task.run("app.start", args)

      "--no-mix-exs" in args ->
        :ok

      true ->
        Mix.raise(
          "Cannot execute \"mix run\" without a Mix.Project, " <>
            "please ensure you are running Mix in a directory with a mix.exs file " <>
            "or pass the --no-mix-exs option"
        )
    end

Here we can see exactly where the --no-mix-exs option comes into play.

First it checks if we are in a normal Mix project directory with Mix.Project.get(). If that returns false it checks the args for --no-mix-exs. Finally it will throw the error we saw at the start. Mystery Solved!

Going Deeper

Mix.Project.get()

It took me forever to track down how this thing works!

When the Mix app is first invoked the Mix.CLI module tries to compile the mix.exs file in the current directory. By default that file will have the line use Mix.Project. Once the file is compiled it runs this post compile hook in Mix.Project

  def __after_compile__(env, _binary) do
    push(env.module, env.file)
  end

This hook pushes the env.module name onto an internal stack that is checked in the Mix.Project.get() command.

Mix.Task.run("app.start", args)

If a mix.exs file is available in the current directory, the run function moves onto the line Mix.Task.run("app.start", args). This starts a new task, as you may guess the task is in the Mix.Tasks.App.Start module.

This module starts in its run method and progresses down several functions until it reaches the line case Application.start(app, type) do This passes the app down to the erlang call :application.start(app, type) to start the app.

Back to Mix.Tasks.Run.run

After checking if we are in a project directory, this function still has a few more tricks up its sleeve.

    process_load(opts, expr_evaluator)

    if file do
      if File.regular?(file) do
        file_evaluator.(file)
      else
        Mix.raise("No such file: #{file}")
      end
    end

I wanted to include this because I think it is especially neat. If you run a command like mix run -e "IO.inspect 123" Mix will run the inline code. This happens in the process_load function. It checks to see if the -e or eval opts were provided and then uses the Code.eval_string/1 function to run the code on the fly.

Finally Mix will run any individual file given in the command line using the &Code.require_file/1 function.

Wrap Up

Thanks for making it this far. It was a challenge to track down exactly what was going on in the Mix app, but it was well worth the effort. Mix is probably the most used Elixir app there is!

Now that you know all about Mix Tasks, maybe you will be inspired to write your own? https://dev.to/drumusician/things-you-could-do-with-mix-2ni3

Coding in Elixir is fun!

Web Monitor with Email Notifications in Elixir

bakenator — Wed, 03 Apr 2019 20:15:08 +0000

Why Build This In Elixir?

First and foremost this app is a tool to learn more about writing elixir code outside of Phoenix.

But! Elixir is an excellent language for a website monitoring tool. By using a supervisor and outside mailing service, we get a highly resiliant tool from only a few lines of code.

The Monitor

To check our website we will use a single function module that performs the following steps:

checks the response from the website
sends an email out depending on whether or not the response was ok
waits 1 hr
starts the function over

I also added a limit on the successful case so that emails are only sent once per day if the website is up. However when the site is down, email notifications will go out every hour.

One of the most interesting things to note is how we can recur the same function with a timer. In most other languages you would have to use a cron job to get a scheduled recurring function. Elixir can due this thanks to the power of its tail call optimization.

Here is the full code for the module

defmodule WebMonitor.SimpleMonitor do
  def start_link() do
    # boilerplate to prepare for :httpc.request
    Application.ensure_all_started(:inets)
    Application.ensure_all_started(:ssl)

    IO.inspect "Monitor Started"
    # Could use the Task module here, but spawn_link used for educational reasons
    pid = spawn_link(WebMonitor.SimpleMonitor, :check_site, [])
    {:ok, pid}
  end

  def check_site() do
    # make web request and set timeout to 10 seconds
    case :httpc.request(:get, {'https://bakerbaker.dev', []}, [{:timeout, 10000}], []) do
      {:ok, {{'HTTP/1.1', 200, 'OK'}, _headers, _body}} -> 
        if Time.utc_now().hour == 6 do
          # send email for alive website 
          WebMonitor.Mailer.send_good_email()
        end
      _ -> 
        # send email for down website
        WebMonitor.Mailer.send_bad_email()
    end
    # sleep for 1 hr
    :timer.sleep(60 * 60 * 1000)
    # run check function again
    check_site()
  end

  # returning informational object to the supervisor
  def child_spec([]) do
     %{
        id: __MODULE__,
        start: {__MODULE__, :start_link, []}
      }
  end
end

There are several places in this module where we could beef up our code. One way would be to use a Genserver and submit recursive calls to the check_site function. I chose the explicit spawn_link here to provide an example of manually supervising an independent process.

We could have also used a Task instead of spawn_link to get better debugging/error handling capabilities.

Finally I used the :httpc module over HTTPoison as a chance to learn. I was inspired by this blog post: https://virviil.github.io/2018/04/06/elixir-do-you-have-http-requests-you-are-doing-them-wrong/

Sending Emails

In the SimpleMonitor code I use a Mailer module for sending the emails. This module was made with the very easy to use MailGun lib at https://github.com/chrismccord/mailgun

Setting it up requires creating a free mailgun account. For free accounts you have to confirm each recipient, but since I am the only recipient this is not a problem.

I followed the set up tutorial and used the default Mailer module in the library as a guide. Once simplified it ended up as this for my app

defmodule WebMonitor.Mailer do
  # the :web_monitor atom is specific to this app's name
  @config domain: Application.get_env(:web_monitor, :mailgun_domain),
          key: Application.get_env(:web_monitor, :mailgun_key)

  use Mailgun.Client, @config

  @from "site_checker@bakerbaker.dev"

  def send_good_email() do
    send_email to: "andrew@bakerbaker.dev",
               from: @from,
               subject: "Site is up",
               html: "<strong>The site is good!</strong>"
  end

  def send_bad_email() do
    send_email to: "andrew@bakerbaker.dev",
               from: @from,
               subject: "Site is DOWN!",
               html: "<strong>The site is DOWN!</strong>"
  end
end

Pretty much all of the work is done by the Mailgun library through macros.

Starting the App

The SimpleMonitor needs to be added to the supervision tree so that it will be called when the app starts.

Here is the full Application code

defmodule WebMonitor.Application do
  @moduledoc false

  use Application

  def start(_type, _args) do
    # List all child processes to be supervised
    children = [
      {WebMonitor.SimpleMonitor, []}
    ]

    opts = [strategy: :one_for_one, name: WebMonitor.Supervisor]
    Supervisor.start_link(children, opts)
  end
end

Now we can run test the app with iex -S mix or create a Release to run permanently on a server

Going Further

I wanted to provide this app as an example for anyone looking to move from Phoenix to more manual control of an Elixir app. With automated email and a timed function, this app could be used as a start for building many other notifier apps. For example an application that checks a web API on a recurring schedule and emails the results.

Hope you enjoyed and learned something! Coding in Elixir is fun!

Measuring Stack Size in Elixir

bakenator — Fri, 29 Mar 2019 21:19:16 +0000

Why Measure?

There are many good articles written on tail call optimization in Elixir. This is what allows Elixir to recursively call a function forever without overloading the stack.

Here is a list of some of them: https://dev.to/search?q=elixir%20tail

While most of these articles say that Elixir optimizes the recursion, I wanted to see it with my own eyes.

The Code

Below is a minimum amount of code to see an optimized recursive function in action. It simply prints out information about the current process, then recurs with a new number.

defmodule MyApp.Example do
  def recursiveFunc(number \\ 1) do
    proc_info = Process.info(self()) 
    IO.inspect proc_info
    recursiveFunc(number + 1)
  end
end

If the function is run in iex, it will print the process information very quickly in the terminal. Within that information object it will list the current stack_size and heap_size. Here is what I got on my machine.

total_heap_size: 15143,
heap_size: 4185,
stack_size: 46,

What we can see is that even though it is calling the function over and over, the stack_size should remain exactly the same. The heap_size will move around but it should not increase very much.

Non Optimized Example

Here is some code that will not be automatically optimized since the recursive function call is not the last thing evaluated in the function

defmodule MyApp.Example do
  def nonOptimizedFunc(number \\ 1) do
    proc_info = Process.info(self()) 
    IO.inspect proc_info
    1 + nonOptimizedFunc(number + 1)
  end
end

When this is run the stack size increases by one for every iteration just like we expected. The heap_size will also grow quickly as each new iteration adds an int to the heap.

Will This Crash?

What was most interesting about this non optimized code is how large the stack can get.

In JavaScript land the stack limit can be reached quite quickly.

Running the non optimized example on my Mac, I get to a stack_size of 100k within a minute and the process keeps on trucking!

This is because by default there is no maximum stack size for an Elixir process. It will keep on growing the stack and heap until the process runs out of memory. By default an Elixir process is allowed however much memory it wants, all the way until it crashes the BEAM VM.

If we wanted to stop this from happening, we can add the following line to our code
Process.flag(:max_heap_size, 100_000)
This will kill the process once it has gobbled up too much memory for our liking, but before crashing our whole app.

Thanks for reading! Coding in Elixir is Fun!

Deploying Elixir and Docker in Docker

bakenator — Tue, 26 Mar 2019 18:56:49 +0000

Recently I have been building a CI (continuous integration) app in elixir.

Why would I do that when there are plenty of good SaaS options? For Fun!

Along the way I have found out about a few interesting things that I thought I could share. This post will focus solely on Elixir Release and Docker in Docker. I will post another article on the elixir app itself.

What Does the CI App Do?

This is a simple CI application. It waits for Github webhook notifications, downloads and tests the new code, then sends the results back to Github.

Working with github is detailed here: https://developer.github.com/v3/guides/building-a-ci-server/

How Does it Work?

Here is where things start getting tricky. Because we want to isolate the code being tested, we will need to run the tests inside a docker container.

But also for portability of the CI app, our elixir code will also need to be running in a docker container.

This brings us to the ultimate mind bender: Docker in Docker!

Lets jump right in.

Dockerizing an Elixir Application

Best practice to send elixir apps off to production is to use the built in Erlang Releases concept. For elixir apps, this can be easily done through the Distillery library. https://elixirschool.com/en/lessons/libraries/distillery/

One hiccup is that the release needs to be built on an operating system matching the operating system where it will be deployed. Docker can come to the rescue here because it allows you to build your elixir releases inside of docker to guarantee the deployed OS is identical to the build OS.

A good post on the subject is here: https://medium.com/@pentacent/getting-started-with-elixir-docker-982e2a16213c
And the officially recommended Dockerfile example is here: https://hexdocs.pm/distillery/guides/working_with_docker.html

Docker in Elixir in Docker

Remember, we want the OS with our elixir app to include docker so that we can launch containers for testing. The guides above show how to build an elixir release in docker, but not with the OS needed for docker in docker.

That OS is called "Docker in Docker"!

The official image is docker:dind, but there are several variants on dockerhub. We will be using a specific version docker:18.09.0-dind so that we are sure it matches the elixir:1.7.2-alpine OS used to build the release

From the hexdocs.pm example, make a new Dockerfile and change the line
FROM alpine:${ALPINE_VERSION}
to
FROM docker:18.09.0-dind

Dockerfile CMD Bug

Through trial and error I found that the line
CMD trap 'exit' INT; /opt/app/bin/${APP_NAME} foreground
was overriding the CMD from the docker:18.09.0-dind image.
This results in the docker daemon not starting, which makes the image useless.

To fix this I tracked down what the docker:18.09.0-dind CMD is and put it in a shell script.

So in the top level project directory add the following file start.sh
The sleep command is there to allow the docker daemon to start up before the elixir app starts.

#!/bin/sh
dockerd-entrypoint.sh & 
sleep 10
/opt/app/bin/ci foreground

Then update the end of the DockerFile to

COPY ./start.sh .
RUN ["chmod", "+x", "start.sh"]
ENTRYPOINT /start.sh

Inner Docker Image

We've sorted out the container that runs the elixir app on a docker:dind image. Now lets prepare the docker image that the tests will use to run.

Create a folder at test_runner
Within this folder my Dockerfile looks like this

FROM node:11.12.0-alpine
RUN apk update && apk upgrade && apk add --no-cache bash git openssh
COPY . .
RUN ["chmod", "+x", "start.sh"]
ENTRYPOINT /start.sh

and the start.sh file looks like this (with personal details replaced)

#!/bin/sh
git clone --depth 1 -b branch_name https://$TOKEN@github.com/$USER_NAME/$PROJECT.git
cd $PROJECT_NAME
npm install
npm run test

Your inner docker image can be whatever you want. And note it doesn't need any special docker:dind OS to run.

Getting the Image into Your App

So we've made the testing image, but now we need the elixir app to have access to it inside the docker:dind system.

We can do this using the docker save and docker load commands. These allow us to save a docker image to a file and then load it back up.

If we run docker save test_runner > test_runner.tar in the test_runner folder, we will get a new test_runner.tar file.

Then in our main ./Dockerfile we can add the line (right before ENTRYPOINT)
COPY ./test_runner/test_runner.tar .
This brings the file into our Elixir image.

Finally we add this line to the main ./start.sh file
docker load < ./test_runner.tar
This adds the image to our docker application so that is can be fired up from the elixir app.

The final ./start.sh file should look like this

#!/bin/sh
dockerd-entrypoint.sh & 
sleep 10
docker load < ./test_runner.tar
/opt/app/bin/ci foreground

Starting a Container in Elixir

Now that we have set up all the infrastructure for our apps, we are ready to use the container in the elixir app.

Wherever it is needed you can use
System.cmd("docker", ["run", "test_runner"])
That will run the docker image and return you the output and exit status in a tuple.

Don't forget that we can tailor the test runner on the fly by passing in environment variables as well.

Conclusion

Thank you if you've made it this far. No one said docker in docker would be easy!

I realize that elixir deployment methods can be contentious, and docker in docker even more so. I welcome any comments on alternative/better systems we could use here instead.

Now here is a build script I wrote to automate all of the build steps after you set up your elixir project, Dockerfiles, and start.sh scripts.

Final note, you must use the --privileged flag to make use of docker in docker.

#!/bin/sh
cd test_runner
docker build -t test_runner .
docker save test_runner > test_runner.tar
docker image rm test_runner
cd ../
docker build -t elixir_app .
rm test_runner/test_runner.tar
docker run -p $OUTER_PORT:$INNER_PORT --privileged -d elixir_app

Supervisor Intensity, what is it?

bakenator — Fri, 22 Mar 2019 22:01:58 +0000

Did you know that Elixir Supervisors will stop trying to restart a child process if they detect something has gone haywire in the child?

They do!

This leads to the next question, how do they know when a child process is a lost cause?

Supervisor Intensity!

What is it?

The intensity setting of a Supervisor is how many failures it can tolerate from a child process within a certain period of time. If more failures are recorded, then the Supervisor stops all child processes and fails itself.

The intensity and period are optional settings that can be set in Elixir like so

opts = [..., max_restarts: 3, max_seconds: 5]
Supervisor.start_link(children, opts)

Note that I have used the values that Elixir currently defaults to, 3 failures over a 5 second period.

When Do You Run Into This?

I figured this out when stepping through a nice bug. At the top of one of my child processes I had this line

def start_link(port: port, dispatch: dispatch) do
    {:ok, socket} = :gen_tcp.listen(port, active: false, packet: :http_bin, reuseaddr: true) do
    ...

I was crashing the process on purpose and subsequently the entire app would shutdown. This confused me, because shouldn't the Supervisor restart the process?
This issue was that :gen_tcp.listen returns an error if the port is already in use by another socket.

So after the first time, this module throws an error on the first line of start_link. The Supervisor very quickly tries and fails three times on that same bug and the Supervisor is shut down.

Why Do Supervisors Do This?

In short it is to prevent infinite loops of processes trying to be restarted over and over.

However note that the Supervisor does not just stop trying to restart the child. If the intensity limits are surpassed the Supervisor shuts down itself.

Why?

It shuts down so that any possible Supervisor of that Supervisor can be notified that something wacky is going on and try to fix the issue. The original Supervisor already gave it its best effort and is now basically passing the issue up a level to ask for help.

Try at Home!

Here is a nice little module you can try out in your own app.

Drop this line into your Supervisor child start list

{FailTwoSeconds, []}

And add this module to your project

defmodule FailTwoSeconds do
    def start_link([]) do
        IO.inspect "New Process Starting"

        pid = spawn_link(fn -> 
            Process.sleep(2000)
            raise "Failing Now"
        end)
        {:ok, pid}
    end

    def child_spec(opts) do
        %{
            id: FailTwoSeconds, 
            start: {FailTwoSeconds, :start_link, [opts]}, 
        }
    end
end

We can see that the module exists just to start up, wait 2 seconds, and crash.

But crashing every 2 seconds is within the default intensity settings for Elixir. So this module will go on crashing and being restarted til the cows come home.

If you switch the sleep time to 1000, then it will trip the Supervisor's intensity limits and the Supervisor will say "I've had too much!" and shutdown.

Thanks for reading and hope this saves you a headache one day!