How to Track Errors in Oban for Elixir Using AppSignal

When developing an Elixir app, you'll often need to handle tasks in a way that does not interrupt the normal user request-response cycle.

Tasks like sending emails are great examples of jobs that should be delegated to a capable background job processing service. In the Elixir ecosystem, Oban is one such background job processing library.

In this article, we'll learn what Oban is, how it works, and how to instrument it using AppSignal.

Prerequisites

To follow along with this tutorial, I recommend you have the following:

• An AppSignal account - you can sign up for a free trial.
• An Elixir app. In my case, I am using a Phoenix LiveView app with a landing page where a user can subscribe to an email newsletter with a couple of background jobs to schedule and send out emails. You can go ahead and fork the app.
• The AppSignal package added and configured for your app.
• Elixir, Phoenix, and PostgreSQL installed on your local development environment.
• Some basic experience of working with Elixir and Phoenix.

And with that, we can get started.

Introducing Oban

Oban is a stable and high-performing background job processing library for Elixir applications. Unlike other job processing systems that might require their own infrastructure to run background jobs, such as RabbitMQ and Sidekiq (which needs a key-value store like Redis), Oban uses your app's existing PostgreSQL or SQLite database.

With Oban, you can define, enqueue, and process jobs in the background. Another important thing to note is that Oban jobs are persistent, which means that even if something unexpected happens in the server environment, jobs will be retried for you.

Installing and Configuring Oban in Elixir

First, add Oban to your app's mix.exs like so:

# mix.exs

defmodule EmailSubscriptionApp.MixProject do
  ...

  defp deps do
    [
      ...
      {:oban, "~> 2.17"}
    ]
  end

  ...
end

Then run mix deps.get in the terminal to install the package and its dependencies.

Before moving on, it's important to cover the database your app is using since it will determine how you configure Oban.

If you've already configured PostgreSQL for your app, the Postgres package is likely already installed, and you'll not have to add it manually. However, if your app uses SQLite, you'll have to add the EctoSQLite3 package to mix.exs and configure it accordingly. In this article, the featured app uses PostgreSQL for the database, and Oban is installed to run on top of that.

Next, generate a migration file that will create all the database tables Oban needs to run and keep tabs on job queues.

mix ecto.gen.migration add_oban_jobs_table

Then modify the generated file to look like this:

# priv/repo/migrations/timestamp_add_oban_jobs_table.exs

defmodule EmailSubscriptionApp.Repo.Migrations.AddObanJobsTable do
  use Ecto.Migration

  def up do
    Oban.Migration.up(version: 12)
  end

  def down do
    Oban.Migration.down(version: 1)
  end
end

Run the migration with mix ecto.migrate.

With that done, you next need to configure Oban to run with the Postgres engine by adding the following lines to config.exs:

# config/config.exs

config :email_subscription_app, Oban,
  engine: Oban.Engines.Basic,
  queues: [default: 10],
  repo: EmailSubscriptionApp.Repo

It's also recommended that you prevent Oban from running jobs in testing mode by adding the lines below to text.exs:

# config/text.exs

config :email_subscription_app, Oban, testing: :inline

Finally, we'll need to add Oban to the app's list of supervised children since they run as isolated supervision trees. To do this, add Oban to the app supervisor like so:

defmodule EmailSubscriptionApp.Application do
  use Application

  @impl true
  def start(_type, _args) do
    children = [
      ...
      EmailSubscriptionApp.Repo,
      {Oban, Application.fetch_env!(:email_subscription_app, Oban)},
      ...
  end
  ...
end

Then, to finish up the installation and configuration, open up a new interactive Elixir shell with iex -S mix. Run the command Oban.config() to return an output similar to the one below:

iex(1)> Oban.config()
%Oban.Config{
  dispatch_cooldown: 5,
  engine: Oban.Engines.Basic,
  get_dynamic_repo: nil,
  insert_trigger: true,
  log: false,
  name: Oban,
  node: "...",
  notifier: {Oban.Notifiers.Postgres, []},
  peer: {Oban.Peers.Postgres, []},
  plugins: [],
  prefix: "public",
  queues: [default: [limit: 10]],
  repo: EmailSubscriptionApp.Repo,
  shutdown_grace_period: 15000,
  stage_interval: 1000,
  testing: :disabled
}

Before moving on to defining a few jobs and instrumenting them using AppSignal, let's touch on how instrumentation actually works.

Automatic Instrumentation of Oban Using AppSignal

When you added the AppSignal package to your app, Oban instrumentation was automatically added. The AppSignal package provides instrumentation for Oban workers and will also collect metrics on how background jobs are performing.

With that in mind, let's write a few jobs and see how they appear on the AppSignal dashboard.

Defining the First Background Job

Our practice Phoenix application is all about receiving a user's email in an input on the homepage, then sending that user a series of emails (starting with a welcome email that is sent immediately when the user subscribes, a follow-up email sent ten minutes later, and then a final email sent thirty minutes later).

Let's start by defining the background job that will trigger the first email.

# lib/email_subscription_app/workers/welcome_email_worker.ex

defmodule EmailSubscriptionApp.Workers.WelcomeEmailWorker do
  use Oban.Worker, queue: :default

  alias EmailSubscriptionApp.Emails.{EmailSender, Mailer}

  @impl Oban.Worker
  def perform(%Oban.Job{args: %{"email" => email}}) do
    email
    |> EmailSender.welcome_email()
    |> Mailer.deliver()

    :ok
  end
end

Let's dig into this example code a bit to understand how Oban workers work. This information will prove helpful when troubleshooting job errors.

You have the module definition — in this case, the WelcomeEmailWorker — then one or more job-performing functions, usually called perform/1.

The perform/1 function receives an Oban.Job struct containing the specific job's arguments which, in our case, is the email key.

Even though we haven't shown it in this example code, Oban also allows for job scheduling using the schedule_at and schedule_in options.

And like we mentioned before, AppSignal automatically instruments Oban and displays some default dashboards, as you can see in the examples below.

Here are our Oban workers listed on AppSignal:

And details of a worker's instrumentation:

Now we've installed Oban, defined a simple background worker job, instrumented Oban with AppSignal, and visualized the output. Let's turn to when things go wrong with Oban and how we can use AppSignal to help us track errors.

Oban Errors

Although Elixir systems are well-known for their fault tolerance, errors and bugs can affect them, resulting in a poor user experience if not addressed. In this section, we'll look at some common errors that could affect your Oban workers, as well as how you can instrument errors with AppSignal and see them on your dashboard.

Many different errors could affect an Oban worker, including:

• Database connection errors - Since Oban depends on accessing a PostgreSQL or SQLite 3 database, any connection issues between your app and its database will definitely affect any workers you have defined.
• External service failures - For example, if there is an issue that affects emails being sent through a third-party email service provider in an email subscription app, the background jobs we've defined could fail to send emails, which would result in Oban job errors. Further, if you call third-party API services from your app, any rate-limiting efforts by the API service provider could result in connected Oban jobs failing to complete and resulting in errors.
• Job timeout errors - These errors occur if a job's actual execution time exceeds the configured time.
• Job queue congestion errors - These happen when there are too many jobs in the queue, which could result in delays in job execution.

Instrumenting Oban Errors Using AppSignal for Elixir

One of the major benefits of AppSignal is that most common errors are automatically instrumented for you. For example, see the code snippet below:

defmodule EmailSubscriptionApp.Workers.WelcomeEmailWorker do
  use Oban.Worker, queue: :default
  alias EmailSubscriptionApp.Emails.{EmailSender, Mailer}

  @impl Oban.Worker
  def perform(%Oban.Job{args: %{"email" => email}}) do
    email
    |> EmailSender.welcome_email()
    |> Mailer.deliver()
    |> case do
      {:ok, _} -> :ok
      {:error, reason} ->
        raise "Mailer delivery error: #{inspect(reason)}"
  end
end

If an error occurs within the perform/1 function, AppSignal will catch it and give you a dashboard view with details of the error, as you can see in the screenshot below:

Before wrapping up this article, let's take a look at the error details AppSignal shows you in the dashboard:

• Error message - The actual error message associated with the raised error.
• Backtrace - The backtrace gives you fine-grained context on what happens before an error occurs in your app.
• Parameters - AppSignal also shows you any parameters involved when the error occurred.
• Deploy - Finally, you also get information on whether the error occurred during a deployment or not.

And that's that!

Wrapping Up

In this article, we looked at the background job processing package Oban. We learned how to install it, configure background workers, simulate an error, and instrument errors using AppSignal. I recommend using this information as a foundation for using Oban and AppSignal in your Elixir apps.

Happy coding!

P.S. If you'd like to read Elixir Alchemy posts as soon as they get off the press, subscribe to our Elixir Alchemy newsletter and never miss a single post!