DEV Community: Aestimo K.

How to Track Errors in Oban for Elixir Using AppSignal

Aestimo K. — Tue, 26 Nov 2024 11:14:49 +0000

When developing an Elixir app, you'll often need to handle tasks in a way that does not interrupt the normal user request-response cycle.

Tasks like sending emails are great examples of jobs that should be delegated to a capable background job processing service. In the Elixir ecosystem, Oban is one such background job processing library.

In this article, we'll learn what Oban is, how it works, and how to instrument it using AppSignal.

Prerequisites

To follow along with this tutorial, I recommend you have the following:

An AppSignal account - you can sign up for a free trial.
An Elixir app. In my case, I am using a Phoenix LiveView app with a landing page where a user can subscribe to an email newsletter with a couple of background jobs to schedule and send out emails. You can go ahead and fork the app.
The AppSignal package added and configured for your app.
Elixir, Phoenix, and PostgreSQL installed on your local development environment.
Some basic experience of working with Elixir and Phoenix.

And with that, we can get started.

Introducing Oban

Oban is a stable and high-performing background job processing library for Elixir applications. Unlike other job processing systems that might require their own infrastructure to run background jobs, such as RabbitMQ and Sidekiq (which needs a key-value store like Redis), Oban uses your app's existing PostgreSQL or SQLite database.

With Oban, you can define, enqueue, and process jobs in the background. Another important thing to note is that Oban jobs are persistent, which means that even if something unexpected happens in the server environment, jobs will be retried for you.

Installing and Configuring Oban in Elixir

First, add Oban to your app's mix.exs like so:

# mix.exs

defmodule EmailSubscriptionApp.MixProject do
  ...

  defp deps do
    [
      ...
      {:oban, "~> 2.17"}
    ]
  end

  ...
end

Then run mix deps.get in the terminal to install the package and its dependencies.

Before moving on, it's important to cover the database your app is using since it will determine how you configure Oban.

If you've already configured PostgreSQL for your app, the Postgres package is likely already installed, and you'll not have to add it manually. However, if your app uses SQLite, you'll have to add the EctoSQLite3 package to mix.exs and configure it accordingly. In this article, the featured app uses PostgreSQL for the database, and Oban is installed to run on top of that.

Next, generate a migration file that will create all the database tables Oban needs to run and keep tabs on job queues.

mix ecto.gen.migration add_oban_jobs_table

Then modify the generated file to look like this:

# priv/repo/migrations/timestamp_add_oban_jobs_table.exs

defmodule EmailSubscriptionApp.Repo.Migrations.AddObanJobsTable do
  use Ecto.Migration

  def up do
    Oban.Migration.up(version: 12)
  end

  def down do
    Oban.Migration.down(version: 1)
  end
end

Run the migration with mix ecto.migrate.

With that done, you next need to configure Oban to run with the Postgres engine by adding the following lines to config.exs:

# config/config.exs

config :email_subscription_app, Oban,
  engine: Oban.Engines.Basic,
  queues: [default: 10],
  repo: EmailSubscriptionApp.Repo

It's also recommended that you prevent Oban from running jobs in testing mode by adding the lines below to text.exs:

# config/text.exs

config :email_subscription_app, Oban, testing: :inline

Finally, we'll need to add Oban to the app's list of supervised children since they run as isolated supervision trees. To do this, add Oban to the app supervisor like so:

defmodule EmailSubscriptionApp.Application do
  use Application

  @impl true
  def start(_type, _args) do
    children = [
      ...
      EmailSubscriptionApp.Repo,
      {Oban, Application.fetch_env!(:email_subscription_app, Oban)},
      ...
  end
  ...
end

Then, to finish up the installation and configuration, open up a new interactive Elixir shell with iex -S mix. Run the command Oban.config() to return an output similar to the one below:

iex(1)> Oban.config()
%Oban.Config{
  dispatch_cooldown: 5,
  engine: Oban.Engines.Basic,
  get_dynamic_repo: nil,
  insert_trigger: true,
  log: false,
  name: Oban,
  node: "...",
  notifier: {Oban.Notifiers.Postgres, []},
  peer: {Oban.Peers.Postgres, []},
  plugins: [],
  prefix: "public",
  queues: [default: [limit: 10]],
  repo: EmailSubscriptionApp.Repo,
  shutdown_grace_period: 15000,
  stage_interval: 1000,
  testing: :disabled
}

Before moving on to defining a few jobs and instrumenting them using AppSignal, let's touch on how instrumentation actually works.

Automatic Instrumentation of Oban Using AppSignal

When you added the AppSignal package to your app, Oban instrumentation was automatically added. The AppSignal package provides instrumentation for Oban workers and will also collect metrics on how background jobs are performing.

With that in mind, let's write a few jobs and see how they appear on the AppSignal dashboard.

Defining the First Background Job

Our practice Phoenix application is all about receiving a user's email in an input on the homepage, then sending that user a series of emails (starting with a welcome email that is sent immediately when the user subscribes, a follow-up email sent ten minutes later, and then a final email sent thirty minutes later).

Let's start by defining the background job that will trigger the first email.

# lib/email_subscription_app/workers/welcome_email_worker.ex

defmodule EmailSubscriptionApp.Workers.WelcomeEmailWorker do
  use Oban.Worker, queue: :default

  alias EmailSubscriptionApp.Emails.{EmailSender, Mailer}

  @impl Oban.Worker
  def perform(%Oban.Job{args: %{"email" => email}}) do
    email
    |> EmailSender.welcome_email()
    |> Mailer.deliver()

    :ok
  end
end

Let's dig into this example code a bit to understand how Oban workers work. This information will prove helpful when troubleshooting job errors.

You have the module definition — in this case, the WelcomeEmailWorker — then one or more job-performing functions, usually called perform/1.

The perform/1 function receives an Oban.Job struct containing the specific job's arguments which, in our case, is the email key.

Even though we haven't shown it in this example code, Oban also allows for job scheduling using the schedule_at and schedule_in options.

And like we mentioned before, AppSignal automatically instruments Oban and displays some default dashboards, as you can see in the examples below.

Here are our Oban workers listed on AppSignal:

And details of a worker's instrumentation:

Now we've installed Oban, defined a simple background worker job, instrumented Oban with AppSignal, and visualized the output. Let's turn to when things go wrong with Oban and how we can use AppSignal to help us track errors.

Oban Errors

Although Elixir systems are well-known for their fault tolerance, errors and bugs can affect them, resulting in a poor user experience if not addressed. In this section, we'll look at some common errors that could affect your Oban workers, as well as how you can instrument errors with AppSignal and see them on your dashboard.

Many different errors could affect an Oban worker, including:

Database connection errors - Since Oban depends on accessing a PostgreSQL or SQLite 3 database, any connection issues between your app and its database will definitely affect any workers you have defined.
External service failures - For example, if there is an issue that affects emails being sent through a third-party email service provider in an email subscription app, the background jobs we've defined could fail to send emails, which would result in Oban job errors. Further, if you call third-party API services from your app, any rate-limiting efforts by the API service provider could result in connected Oban jobs failing to complete and resulting in errors.
Job timeout errors - These errors occur if a job's actual execution time exceeds the configured time.
Job queue congestion errors - These happen when there are too many jobs in the queue, which could result in delays in job execution.

Instrumenting Oban Errors Using AppSignal for Elixir

One of the major benefits of AppSignal is that most common errors are automatically instrumented for you. For example, see the code snippet below:

defmodule EmailSubscriptionApp.Workers.WelcomeEmailWorker do
  use Oban.Worker, queue: :default
  alias EmailSubscriptionApp.Emails.{EmailSender, Mailer}

  @impl Oban.Worker
  def perform(%Oban.Job{args: %{"email" => email}}) do
    email
    |> EmailSender.welcome_email()
    |> Mailer.deliver()
    |> case do
      {:ok, _} -> :ok
      {:error, reason} ->
        raise "Mailer delivery error: #{inspect(reason)}"
  end
end

If an error occurs within the perform/1 function, AppSignal will catch it and give you a dashboard view with details of the error, as you can see in the screenshot below:

Before wrapping up this article, let's take a look at the error details AppSignal shows you in the dashboard:

Error message - The actual error message associated with the raised error.
Backtrace - The backtrace gives you fine-grained context on what happens before an error occurs in your app.
Parameters - AppSignal also shows you any parameters involved when the error occurred.
Deploy - Finally, you also get information on whether the error occurred during a deployment or not.

And that's that!

Wrapping Up

In this article, we looked at the background job processing package Oban. We learned how to install it, configure background workers, simulate an error, and instrument errors using AppSignal. I recommend using this information as a foundation for using Oban and AppSignal in your Elixir apps.

Happy coding!

P.S. If you'd like to read Elixir Alchemy posts as soon as they get off the press, subscribe to our Elixir Alchemy newsletter and never miss a single post!

A Complete Guide to Phoenix for Elixir Monitoring with AppSignal

Aestimo K. — Tue, 01 Oct 2024 12:20:29 +0000

For Phoenix developers, maintaining the health of your applications is critical. AppSignal offers a powerful solution to gain deep insights into your application's performance and stability.

In this introductory guide, we'll walk through the process of setting up AppSignal in your Phoenix app, instrumenting your code for detailed monitoring, handling errors effectively, and utilizing AppSignal's features to maintain and improve your application's performance. The lessons you learn from this article will help you get started using AppSignal to monitor your Phoenix app.

But before proceeding, let's see what you'll need to follow along effectively.

Prerequisites

An AppSignal account — you can sign up for a free 30-day trial.
A Phoenix app to work with. If you don't have one, you can fork this one that we'll use throughout the tutorial.
Some experience working with Elixir/Phoenix applications.

Let's get started by learning how to add AppSignal to an existing Phoenix application.

Adding AppSignal to a Phoenix Application

Adding AppSignal's Phoenix package is very easy — just open up mix.exs and add the package as shown below:

# mix.exs
...
defp deps do
  [
    {:phoenix, "~> 1.7.14"},
    {:phoenix_ecto, "~> 4.5"},
    ...
    {:appsignal_phoenix, "~> 2.0"}
  ]
end
...

After that, run mix deps.get to add the package to the application, then run the command below to install it:

mix appsignal.install <YOUR API KEY HERE>

Tip: You can find your API key in the AppSignal dashboard.

When you run the installer, you'll be prompted for an application name — go ahead and enter an appropriate name:

Validating Push API key: Valid! 🎉
What is your application's name? [counter_live_app]: Counter Live App

Next up, you'll need to choose how you want the AppSignal configuration handled for your app:

There are two methods of configuring AppSignal in your application.
  Option 1: Using a "config/appsignal.exs" file. (1)
  Option 2: Using system environment variables.  (2)

You can choose whatever suits you, but I prefer option 1, which uses a config file that you can customize to your liking. Once you make your choice, the AppSignal configuration should now be working, and if you visit the dashboard, you can see some default graphs (these might be empty initially as it takes a little bit of time for your app's data to be sent over).

Before moving on, let's take a look at how you can customize the AppSignal config file.

Customizing the AppSignal Config File

A config file tells AppSignal which app and environment to instrument.

The code snippet below shows a minimal configuration:

# config/appsignal.exs

import Config

config :appsignal, :config,
  otp_app: :counter_live_app,
  name: "Counter Live App",
  push_api_key: "API KEY HERE",
  env: Mix.env

As you can see, the most important settings include:

Your application's OTP name.
The name of the app as you want it to appear on AppSignal.
Your AppSignal API key.
The environment you want to instrument for, which could be development, production, or test environments.

The config file can be further customized using a variety of other options.

With the installation and configuration working, let's shift gears to monitor our Phoenix app's performance.

Monitoring the Performance of a Phoenix App

When you add the AppSignal package to your Phoenix application, the package will take care of default HTTP instrumentation needs and send data to a dashboard:

And part of the automatic instrumentation includes performance monitoring:

The default instrumentation will also include important details for slow-performing requests, like what's shown below:

And all of this is possible without any extra work from you.

Next up, let's see how you can customize some of the performance instrumentation. For example, we can add a custom slow function to the Posts index and see the same on the AppSignal dashboard.

First, let's edit the index action in the posts controller as shown below:

# lib/counter_live_app_web/controllers/posts_controller.ex

defmodule CounterLiveAppWeb.PostController do
  ...

  def index(conn, params) do
    really_slow()
    ...
  end
  ...
  def really_slow do
    Appsignal.instrument("really slow index request", fn ->
      :timer.sleep(1000)
    end)
  end
end

And with that, we can visualize the instrumentation on the dashboard:

Tracking Phoenix App Errors Using AppSignal

In the previous section, we saw how adding AppSignal to our Phoenix app provides us with a lot of information about app performance. We also went ahead and added some simple custom instrumentation to simulate a really slow request to the posts index action, and visualized the results on the AppSignal dashboard. Now, let's see how we can view and track Phoenix errors using AppSignal.

Our first example will be a simple call to a non-existent route, as shown below:

Again, as with the default performance metrics, without extra work on our part, AppSignal is able to pick up this error and display it on the dashboard:

If you click on the error, you can see more detailed information, which is really helpful for debugging purposes:

I won't go into more detail on how you can customize exception instrumentation in the scope of this post. If you are interested in going down that rabbit hole, check out the AppSignal for Elixir documentation on exception handling.

Next up, let's set up alerts and notifications.

Setting up Effective Alerts and Notifications

As useful as an application performance monitoring solution like AppSignal is, you wouldn't want to spend all your time staring at dashboards. Instead, it would be better if you received automated alerts for any significant events.

By default, whenever an error or performance event happens, it's recorded as an error or performance incident by AppSignal. Depending on the notification settings you've set up, you'll get an email (the default notification channel) or a message through any of the other available notification channels.

You can customize how AppSignal will notify you of events and errors by setting up any of the five notification defaults available:

Never notify - when set to this, AppSignal will not notify you whenever an error or incident occurs. This setting is useful for non-critical incidents.
Every occurrence - Let's say you'd like to be notified every time an incident happens. This is the setting you would use. You can use this setting for incidents that are critical in nature.
First in deploy - Here, a notification will be sent whenever an incident occurs when you first deploy your app. You will need to set up deploy markers that will tell AppSignal when a deployment occurs. For example, let's say you've set up post-deploy hooks of some kind. In that case, this setting helps you catch errors if these hooks don't work as expected.
First after close - This setting tells AppSignal to send a notification after an incident recurs, immediately after you close a previous similar one. This setting would be useful for incidents that are not entirely due to a bug in your code, (for example, third-party API connection issues and so forth).
Every nth hour or day - Here, you tell AppSignal not to send a notification every nth hour or day. This could be very useful if you are facing an increased bill due to an increase in notification events.

So far, everything we've looked at is quite basic, but your AppSignal setup is capable of a lot more.

Next, let's briefly look at one or two examples of custom monitoring and alert notification setups.

Custom Alert Notification Setup

By default, every application that is being monitored by AppSignal will have an email notifier set up automatically.

Even so, you might want to be notified via other channels, and with AppSignal, these are a breeze to set up. The first step is to click on the "Add integration" button.

For example, if you wanted a Discord notifier, just choose it from the list of integrations and input the required information in the resulting dialog like this:

Even with all the bells and whistles that AppSignal makes available for alerting and notifications, it's important to really think about the kinds of notifications you would like to receive and the ones you'd be safe ignoring.

Some good rules of thumb to guide your decision in this would include:

Consider error thresholds for critical versus non-critical errors. For non-critical errors, you could set very high thresholds. For critical errors, you might set finer thresholds (for example, send an alert if a critical error exceeds 5% and lasts for more than 3 minutes).
Set up alerts and notifications for memory consumption. One of the key resources any app uses is memory. If an app is consuming more-than-normal memory, you want to make sure an alert is sent to you early.
Uptime and availability notifications. This is self-explanatory, but you definitely want to be notified any time your app is unavailable.
It would also make sense to set up alerts for custom events that are key to the functionality of your app, or to the user experience (for example, a bunch of failed payments over a certain amount of time).

With these rules in mind, and considering AppSignal's robust alerting infrastructure, you have all you need to be notified whenever things go wrong. If you need further information, go through AppSignal's documentation on setting up notifications and alerts.

Monitoring of a Phoenix App Using AppSignal: An Example

We'll use AppSignal's function decorators to set up a custom error span that will be fired when we try to request a non-existent post.

First, we modify the posts controller to use AppSignal decorators like this:

# lib/counter_live_app_web/controllers/post_controller.ex

defmodule CounterLiveAppWeb.PostController do
  use CounterLiveAppWeb, :controller
  use Appsignal.Instrumentation.Decorators # add this line

  ...
end

Then we modify the show action with the custom instrumentation as shown below:

# lib/counter_live_app_web/controllers/post_controller.ex

...
  def show(conn, %{"id" => id}) do
    try do
      post = Blog.get_post!(id)
      render(conn, :show, post: post)
    rescue
      e in Ecto.NoResultsError ->
        Appsignal.send_error(e, __STACKTRACE__, fn span ->
          Appsignal.Span.set_name(span, "Custom Post Not Found Error")
          Appsignal.Span.set_sample_data(span, "custom_data", %{
            post_id: id,
            controller: "PostController",
            action: "show"
          })
        end)

        conn
        |> put_status(:not_found)
        |> put_view(CounterLiveAppWeb.ErrorView)
        |> render("404.html")
    end
  end
...

Here, we use AppSignal's send_error/3 to define a custom span with the name "Custom Post Not Found Error", which is called if we make a request to a non-existent post.

As you can see from the screenshots below, the error is captured as expected on AppSignal:

Clicking on the error link gives you access to the error details:

And that's it!

Wrapping Up

In this article, we've looked at how to set up AppSignal to monitor a Phoenix application. We also covered how errors and performance incidences are instrumented and displayed on the dashboard.

Even though this is enough to get you started with monitoring your Phoenix application, AppSignal offers a lot more options for you to explore (such as Ecto-specific monitoring to cover your app's Ecto queries, plug monitoring, and even in-depth custom instrumentation to cover almost any monitoring scenario you might have).

Until next time, happy coding!

P.S. If you'd like to read Elixir Alchemy posts as soon as they get off the press, subscribe to our Elixir Alchemy newsletter and never miss a single post!

Custom Instrumentation for a Phoenix App in Elixir with AppSignal

Aestimo K. — Tue, 20 Aug 2024 12:04:14 +0000

In the first part of this series, we saw that even if you just use AppSignal’s default application monitoring, you can get a lot of information about how your Phoenix application is running.

Even so, there are many ways in which a Phoenix application may exhibit performance issues, such as slow database queries, poorly engineered LiveView components, views that are too heavy, or non-optimized assets.

To get a grasp on such issues and peer even more closely into your app’s internals, you’ll need something that packs more punch than the default dashboards: custom instrumentation.

In this article, we'll go through a step-by-step process to build custom instrumentation using AppSignal for a Phoenix application.

Pre-requisites

Before we get into custom instrumentation, here's what you need to follow along:

A Phoenix LiveView app with the AppSignal package installed. If you have an app ready but haven't installed the AppSignal package, just follow this guide. Or you can clone the Phoenix app we'll be using throughout the tutorial.
Basic experience working with Elixir and the Phoenix framework.
An AppSignal account. Sign up for a free trial if you don't have one already.

What Is Custom Instrumentation, and Why Do We Need It?

Custom instrumentation means adding additional telemetry emission and monitoring functionality to your Phoenix app beyond what is provided by the default instrumentation. Specifically, it involves the use of functions and macros (provided through the AppSignal package) in specific places in your codebase that you want to investigate further. You can then view the collected data on a custom AppSignal dashboard.

If you have a complex or mission-critical Phoenix app, it might not be possible to identify all its potential performance issues using the default instrumentation. It's important to have deeper insights into what's going on inside your app.

With custom instrumentation, you can easily structure exactly what you need to identify and visualize these custom data collections within the AppSignal dashboard.

Getting Started with Custom Instrumentation in AppSignal

You can implement custom instrumentation using AppSignal in two ways: through function decorators or instrumentation helpers.

Function Decorators

A function decorator is a higher-order function that you wrap around a function from which you want to collect data. They give you more granular control than instrumentation helpers, but compared to the latter, they are not as flexible.

Instrumentation Helpers

AppSignal also provides instrumentation helper functions, which you can use to instrument specific parts of your code manually. These helper functions can start and stop custom measurements, help you track custom events, and add custom metadata to metrics reported via AppSignal dashboards. This approach gives you more flexibility in instrumenting your code but requires more manual intervention.

In the following sections, we'll use what we've learned to implement custom instrumentation for a Phoenix application.

How to Use an Instrumentation Helper

To implement the first custom instrumentation, we'll consider a controller action that calls a potentially slow function in a simple Phoenix app featuring games and reviews.

As you can see below, the games context has a method for fetching an individual game and preloading any reviews:

# lib/game_reviews_app/games.ex

...
def get_game_with_reviews(id), do: Repo.get(Game, id) |> Repo.preload([:reviews])
...

And the subsequent action using this method in the game controller is:

# lib/game_reviews_app_web/controllers/game_controller.ex

...
def show(conn, %{"id" => id}) do
  game=Games.get_game_with_reviews(id)
  render(conn, :show, game: game)
end
...

Now let's modify this controller action with a custom instrumentation helper to check how many times it is called and its response times. We can do this using AppSignal's instrument/2 function, which takes two parameters:

The function name
The function being instrumented

Note: If you use instrument/3 instead, it's possible to add an event group name as an optional parameter. Usually, whenever you use instrument/2, measurements will be collected and categorized under the "other" event group within the event groups section. But if you wanted another more descriptive name, instrument/3 allows you to pass in an additional parameter for the event group name.

# lib/game_reviews_app_web/controllers/game_controller.ex

defmodule GameReviewsAppWeb.GameController do
  use GameReviewsAppWeb, :controller

  ...

  def show(conn, %{"id"=>id}) do
    game = Games.get_game_with_reviews(id)
    am_i_slow() # add this line
    render(conn, :show, game: game)
  end
  ...
end

Let's define the am_i_slow function as a private module:

# lib/game_reviews_app_web/controllers/game_controller.ex

defmodule GameReviewsAppWeb.GameController do
  use GameReviewsAppWeb, :controller

  ...
  defp am_i_slow do
    Appsignal.instrument("Check if am slow", fn->
      :timer.sleep(1000)
    end)
  end
end

Specifically, we're defining a custom trace span with an event sample called "Check if am slow". This will show up in our dashboard under the other (background) event namespace, as you can see in the screenshots below:

But what if you wanted a different event group name from "other" or "background" for the event namespace? Just use the instrument/3 function:

# lib/game_reviews_app_web/controllers/game_controller.ex

defmodule GameReviewsAppWeb.GameController do
  use GameReviewsAppWeb, :controller

  ...
  defp am_i_slow do
    Appsignal.instrument("Really Slow Queries","First Slow Query", fn->
      :timer.sleep(1000)
    end)
  end
end

Here, we give the span a category name of "Really Slow Queries" and a span name of "First Slow Query", for a sample view like this:

Here's another example of using function helpers:

# lib/game_reviews_app_web/controllers/game_controller.ex

defmodule GameReviewsAppWeb.GameController do
  use GameReviewsAppWeb, :controller

  ...
  def index(conn,_params) do
    games = Appsignal.instrument("Fetch games", fn->
    Games.list_games()
  end)

  render(conn, :index, games: games)
  end
end

Using the instrument/2 function, we define a span called "Fetch games" which results in this event trace:

With that, you can easily visualize the function's response time and throughput.

There are more options available to customize AppSignal's instrumentation helpers than what I've shown here. I highly encourage you to check out the possibilities.

Next, let's see how you can use function decorators.

Using Function Decorators

As you've probably noticed when using instrumentation helpers, you end up modifying existing functions with the helper code you add. If you don't want to do this, you can use function decorators instead.

Let's continue working with the game controller and instrument the index method using a decorator. First, we will add the decorator module Appsignal.Instrumentation.Decorators:

# lib/game_reviews_app_web/controllers/game_controller.ex

defmodule GameReviewsAppWeb.GameController do
  ...
  use Appsignal.Instrumentation.Decorators
  ...
end

You now have access to the decorator's functions. Let's decorate the index method as shown below:

# lib/game_reviews_app_web/controllers/game_controller.ex

defmodule GameReviewsAppWeb.GameController do
  ...
  use Appsignal.Instrumentation.Decorators

  def show(conn, %{"id"=>id}) do
    am_i_slow()
    game = Games.get_game_with_reviews(id)
    render(conn,:show,game:game)
  end

  # add the decorator function
  @decorate transaction_event()
  defp am_i_slow do
    :timer.sleep(1000)
  end

end

This will create a transaction event, which you can visualize in your Events dashboard, as shown below:

You get all the information you need, namely:

a. The resource where the function decorator was called
b. A sample breakdown showing how long Ecto queries took, how long the templates took to load, and so forth.
c. An event timeline with a breakdown of the transaction time of everything involved in that function.

Finally, let's take a look at instrumenting Phoenix channels.

Instrumenting Phoenix LiveViews and Channels

In the example below, we have the welcome live view as shown:

# game_reviews_app_web/live/welcome_live.ex

defmodule GameReviewsAppWeb.WelcomeLive do
  use GameReviewsAppWeb, :live_view

  def mount(_params,_session,socket) do
    {:ok,assign(socket, current_time: DateTime.utc_now())}
  end

  def render(assigns) do
    ~H"""
      <div class="container">
        <h1>Welcome to my LiveView App</h1>
        <p>Current time: <%= @current_time %></p>
      </div>
    """
  end
end

Let's use an instrumentation helper to see how this live view performs:

# game_reviews_app_web/live/welcome_live.ex

defmodule GameReviewsAppWeb.WelcomeLive do
  ...

  import Appsignal.Phoenix.LiveView,only:[instrument: 4]

  def mount(_params,_session,socket) do
    instrument(__MODULE__,"Liveview instrumentation", socket, fn->
      :timer.send_interval(1000,self(),:tick)
      {
        :ok,
        assign(socket,current_time:DateTime.utc_now())
      }
    end)
  end

  ...
end

And as you can see, the transaction times and throughput are now available for inspection on our dashboard:

It's also worth noting that the AppSignal for Elixir package enables you to instrument Phoenix channels using a custom function decorator:

defmodule GameReviewsAppWeb.VideoChannel do
  use GameReviewsAppWeb, :channel

  # first add the instrumentation decorators module
  use Appsignal.Instrumentation.Decorators

  # then add the decorator function
  @decorate channel_action()
  def join("videos:" <> video_id, _params, socket) do
    {:ok,assign(socket, :video_id, String.to_integer(video_id))}
  end
end

And that's it!

Wrapping Up

In part one of this series, we looked at how to set up AppSignal for an Elixir app and AppSignal's error tracking functionality.

In this article, we've seen how easy it is to use AppSignal's Elixir package to implement custom instrumentation for a Phoenix application. We've also learned how to use instrumentation helpers and function decorators.

With this information, you can now easily decide when to use a decorator versus an instrumentation helper in your next Phoenix app.

Happy coding!

P.S. If you'd like to read Elixir Alchemy posts as soon as they get off the press, subscribe to our Elixir Alchemy newsletter and never miss a single post!

Setting Up Custom Metrics with Effective Alerts for a Ruby App in AppSignal

Aestimo K. — Wed, 14 Aug 2024 07:45:23 +0000

Most of the time, the default application monitoring metrics, graphs, and visualizations provided by AppSignal will do for your Ruby app. However, you might be the kind of user who likes a bit of control over what is measured, how it’s displayed, and how critical information about your app should be relayed.

AppSignal allows you to customize app metrics and dashboards as you wish. In this guide, we’ll learn all about AppSignal's custom metrics, including:

What custom metrics are
The different types of custom metrics you can set up
How to customize graph visualizations
How to set up effective alerts

And more!

But before we dive in, you'll need a couple of things to follow along.

Prerequisites

An AppSignal account: If you don't have one, sign up for a 30-day free trial.
A Ruby application: This app can be based on any supported Ruby framework, like Rails, Sinatra, or just plain Ruby. Additionally, it can be a production or development app. If you don't want to spin up your own app, clone the code for the example Sinatra app we'll be using in this tutorial.

Note: If you're using your own app to follow along with this tutorial, make sure your app is configured to use the latest AppSignal Ruby gem, as the examples used in this tutorial assume this is the case.

What Are Custom Metrics?

Other than measuring your app's error rates, throughput, and performance, you might be interested in measuring custom data specifically tailored for your own app. For example, you could be interested in how many visitors signed up to your app in a particular time period, how your app's websocket layer is performing, and so forth.

For such customized cases, you might be hard-pressed to find a standard measuring tool within AppSignal. Instead, you'll need to use a custom metric. Custom metrics are additional metrics that you define alongside the default ones for deeper context on how your app is running.

Next up, let's learn how to set up our first custom metric.

Setting Up Custom Metrics

You can set up a custom metric for almost any use case within your application. Let's start with a simple example to help you understand how everything fits together.

The first step is to define a custom metric that will be tracked on AppSignal. You can define a custom metric by using the different metric types available:

Gauges
Counters
Distributions

The Gauge Custom Metric

In AppSignal, a gauge custom metric is useful for measuring metrics that increase and decrease over time.

Let's set up a simple gauge custom metric to measure the total number of posts in our example Sinatra app:

# app.rb

require 'sinatra/base'
require 'dotenv/load'
require 'sinatra/reloader'
require 'sinatra/activerecord'
require './models/post.rb'
require 'appsignal/integrations/sinatra'

class App < Sinatra::Base
  ...

  get '/' do
    posts = Post.all

    # set a gauge custom metric
    posts_count = posts.count
    Appsignal.set_gauge("all_posts", posts_count)

    posts.to_json
  end
...

In the code shown above, we use the Appsignal::Helpers::Metrics module and call the set_gauge method, which accepts three arguments:

key: the name of the custom metric. In the example, this would be all_posts.
value - The metric or "thing" to be measured. In the example shown above, this is simply the total post count.
tags - Additional and optional metadata that can be added to a custom metric and are useful for labeling the data being measured however you want. For example, we could easily tag the posts_count metric to account for the environment, as shown below:

# app.rb

...

get '/' do
  posts = Post.all
  posts_count = posts.count
  Appsignal.set_gauge("all_posts", posts_count, environment: "development")

  posts.to_json
end

...

Great, we've just added our first custom gauge metric! But if you were to go back to AppSignal, your new custom metric will not be visible. Instead, you're likely to see the default dashboard, as shown below:

So what do you need to do to make the custom metric appear? You need to add a dashboard. Start with creating a new dashboard:

Then name your new dashboard with a descriptive title and description:

With the custom dashboard added, you'll now need to add a graph for the custom metric:

Then define the new graph:

Here's a breakdown of the fields to set up the new graph:

a. Title - Enter a descriptive title for the new graph.
b. Description - This is optional, but you can enter a description for the new graph.
c. Metrics - Here is where you define the metric that will be measured and displayed by the new graph. This is the name of the custom metric, or the first argument defined in the set_gauge method: all_posts. In this section, you can also define tags (for example, the tag environment is also included as shown).
d. Graph display - This is where you choose the type of graph display for your new graph.
e. Legend label - You can customize the label for the chart legend here.
f. Data format - Define the data type used for the graph display. You can choose from a number of formats, including number, percentage, throughput (in requests/minute or hour), duration (in milliseconds), or file size (in bytes).

Once you've properly defined the new graph's properties, you should get a graph for the custom metric. This is similar to what is shown below:

Moving forward, let's look at the next custom metric type: the counter.

The Counter Custom Metric

A counter custom metric is great for measuring how many times an event occurs. Using the example application, we can apply a counter metric to measure every time the home (root) page is visited.

To begin with, edit the root method to include the code shown below:

# app.rb
...
get '/' do
  ...
  Appsignal.increment_counter("visits_count", 1)
end
...

Here, we use AppSignal's increment_counter method and pass it visits_count as the first argument. The increment step is the integer 1, passed as the second argument. You could also add a tags hash as the third argument, but we'll leave it as is (since this was covered in the previous section).

Now go ahead and follow the steps as outlined for the gauge metric type. Add a custom graph for this counter metric to give you a similar graph to the one shown:

Let's switch gears to the distribution custom metric.

The Distribution Custom Metric

The AppSignal distribution custom metric is useful for measuring something per unit of time: for example, how many seconds it takes to generate a PDF report, or how long a background job takes to execute.

Using the example application, let's modify the main file to include a call to an open API endpoint. Then, we'll use a custom distribution to measure how long the API call takes in milliseconds.

# app.rb
...

require 'httparty'

class App < Sinatra::Base
  ...

  Thread.new do
    while true do
      start_time = Time.now
      url = 'https://openlibrary.org/search/authors.json?q=clavell'
      $response = HTTParty.get(url)
      duration = (Time.now - start_time) * 1000
      Appsignal.add_distribution_value("fetch_books_duration", duration)
    end
  end

  get '/' do
    ...
    $response.to_json
  end

  ...
end

Now, if we go back to AppSignal, we can view the custom distribution as a graph.

Tip: You can follow the steps outlined in the gauge section to set up the custom graph visualization.

Now that you've learned how to create custom metrics and the accompanying graph visualizations, you might have noticed that it's not very convenient to keep going back to the AppSignal dashboards to see what's going on with your app. Instead, it would be very handy if you could get a notification for your custom metrics, right?

Let's learn how to set up notifications for your custom metrics next.

Notification Alerts

By default, whenever an error or performance event occurs, AppSignal will open an incident for that event and place it in the relevant section. For example, if it's an error, you'll find it in the errors list, while performance incidents will be in the performance list.

Additionally, AppSignal sends out incident notifications via email (the default notification channel). You can also set up other notification channels, such as:

Discord
Google Hangouts
Intercom
Microsoft Teams
Slack
Webhook

And more.

But before we set up a notification trigger for one of our custom metrics, it's important to be aware of the various notification options available to you.

To begin with, you can set up a notification for:

Every time - Here, a notification will be sent out every time an incident occurs.
First deploy - This indicates that a notification will be sent the first time an incident occurs after a deployment.
First after close - Here, a notification is sent out whenever an incident re-occurs after the previous one was closed.
Never notify - As the name suggests, in this case, a notification will never be sent, but the error or performance incident will still be tracked on AppSignal.
Every nth hour or day - With this option, you can specify how many alerts will be sent to you within an hour or a day. This option is perfect for keeping a balance between getting notified of important events and having too many notifications (which could easily overwhelm you or your team).

I highly suggest you dig into AppSignal's notification settings documentation to get more information on these options.

Let's see how to set up a notification for one of the custom metrics we made earlier. This will be a trivial example, but it will illustrate the steps you need to go through for your own use case.

Setting Up Notification Alerts for Your Custom Metrics

For this example, we'll use the distribution metric that measured the API call's duration earlier in this post. Let's say we'd like to get an email alert whenever the mean duration exceeds a certain number (in milliseconds).

The steps for setting this up are shown below:

Firstly, begin by hitting the Triggers link in the left-hand menu.

Give your trigger a relevant name, then select the measurement for which this notification is intended. In this example, we are using the fetch_books_duration distribution custom metric. You can also add tags if you wish to.

Next, define the comparison operator and the value to check for. For example, let's say we want to receive an alert whenever the duration exceeds 1600 milliseconds. For this, we would select the comparison operator more than, then, a value of 1600.

Finally, you'll need to define the alert warm-up and cooldown settings. Provide a description for the alert, a link to the dashboard to be included in the alert message (if needed), and, finally, the notification method (with email being the default).

With that done, you'll receive a notification whenever an incident occurs matching the settings you input here.

And that's it!

Wrapping Up

In this article, we learned how to set up custom metrics for a Ruby application with matching dashboards and graph visualizations on AppSignal.

The custom metrics functionality that AppSignal offers can be fine-tuned for very powerful applications. Take a deep dive into AppSignal's custom metrics documentation to discover more possibilities.

Until next time, happy coding!

P.S. If you'd like to read Ruby Magic posts as soon as they get off the press, subscribe to our Ruby Magic newsletter and never miss a single post!

Track Errors in Phoenix for Elixir with AppSignal

Aestimo K. — Tue, 23 Jul 2024 12:38:29 +0000

AppSignal is a powerful error tracking and performance monitoring tool that can help you maintain reliability and speed in your Elixir applications.

In this tutorial, the first of a two-part series, you'll learn how to integrate AppSignal into your Elixir application, configure it for error tracking, interpret error reports, and leverage AppSignal's features to debug and resolve issues.

Whether you're a seasoned Elixir developer looking to improve your application's error handling or a beginner who wants to understand how error tracking works, this step-by-step tutorial will give you a firm foundation for using AppSignal to track errors in your Elixir app.

Prerequisites

To follow along with this tutorial, you need:

An Elixir development environment. If you don't have Elixir installed, follow the steps in this guide for your operating system.
An existing Elixir app. If you don't have one, go ahead and fork this app.
Some experience working with Elixir/Phoenix.

Let's get started!

Integrating AppSignal Into a Phoenix App

Quick tip: As of writing this article, AppSignal currently supports pure Elixir, Plug, and Phoenix apps. The examples used for this tutorial are centered around a Phoenix app featuring video games, users, and reviews, but the steps taken should be relatively similar for the other supported flavors.

AppSignal provides excellent support for the Phoenix framework and the installation process is very seamless. First, sign up for a free 30-day trial, no card details required.

Then open the Phoenix app's dependencies file and add AppSignal's Phoenix package:

# mix.exs

...
defp deps do
  [
    ...
    {:appsignal_phoenix, "~> 2.0"}
  ]
end
...

Run mix deps.get to add the package to your app. Once it's added successfully, install it by running the command below:

mix appsignal.install <PUSH_API_KEY>

Note: You can get your PUSH_API_KEY from the AppSignal dashboard.

When you run this command, you'll be presented with some options to customize the installation:

Validating Push API key: Valid! 🎉
What is your application's name? [game_reviews_app]: <APP NAME>

There are two methods of configuring AppSignal in your application.
  Option 1: Using a "config/appsignal.exs" file. (1)
  Option 2: Using system environment variables.  (2)

The first option is your app's name as you want it to appear on the AppSignal dashboard.
The second option is how to configure AppSignal: via a configuration file or a system variable.

Configuring AppSignal

For your app to send data to AppSignal, you should have a minimal configuration in config/appsignal.exs, as shown below:

# config/appsignal.exs

import Config

config :appsignal, :config,
  otp_app: :game_reviews_app,
  name: "Phoenix API app",
  push_api_key: <AppSignal Push API key>,
  env: Mix.env

Or you can set this up using system environment variables. One thing to note is that this file can be configured to your liking using a variety of options.

Assuming the configuration proceeds without a hitch, you should now see your app automatically added to the AppSignal dashboard, as shown below:

Note: It may take a couple of minutes for your app's data to start showing.

Great, now let's see how we can instrument an error in AppSignal.

Instrumenting Elixir Errors

This example app is a simple Phoenix LiveView app featuring a video game and reviews.

Once you've cloned the application to your development environment, run mix deps.get to install all the dependencies, then you can run the app with:

mix phx.server

This will make the app available on http://localhost:4000.

Go ahead and test it by creating a few games with the following command:

curl --location --request POST 'http://localhost:4000/api/games' \
--header 'Content-Type: application/json' \
--header 'Accept: */*' \
--header 'Host: localhost:4000' \
--header 'Connection: keep-alive' \
--data-raw '{
    "game": {
        "title": "Hell divers",
        "genre": "Action",
        "publisher": "Arrowhead Studios"
    }
}'

Having made a couple of games this way, you can now make a GET request for the games list (I use Insomnia for this, but you can use whatever you like):

Which should execute successfully, as you can see above.

Now, let's deliberately cause an error to see how it will be shown on the AppSignal dashboard.

Firstly, create a function defining a custom exception:

# lib/exceptions/plug_exception.ex

defmodule GameReviewsApp.PlugException do
  defexception [:plug_status, message: ""]
end

Then, modify the index action in the games controller to call this function:

defmodule GameReviewsAppWeb.GameController do
  ...
  def index(conn, _params) do
    raise GameReviewsApp.PlugException, plug_status: 403, message: "You're not allowed!"
    games = Games.list_games()
    render(conn, :index, games: games)
  end
  ...
end

Now when we make a request to the games list, this custom exception is raised:

And the same is shown on the AppSignal dashboard:

If you click on the displayed error link, AppSignal gives you more context on the error, which should help you resolve it:

Next up, let's learn how we can track Ecto queries inside AppSignal.

Instrumenting Ecto queries in AppSignal

Ecto is the most popular database abstraction library for Elixir apps.

AppSignal provides native support for Ecto by hooking into the Ecto query telemetry layer, giving you query profiling, identification of slow queries, and more, with no extra work on your part (as long as the otp_app name within the AppSignal configuration file matches the name of your app in config.exs):

# config/config.exs

...
# General application configuration
import Config

config :game_reviews_app,
...

# config/appsignal.exs

import Config

config :appsignal, :config,
  otp_app: :game_reviews_app,
  ...

With our configuration done, AppSignal's instrumentation will now automatically pick up Ecto queries and display them on this dashboard:

Now this might be good enough for normal Ecto queries. But to pick up query types such as preloads in the right way, you'll need to edit your app's repo:

# lib/game_reviews_app/repo.ex

defmodule PhxRestApiApp.Repo do
  # use Ecto.Repo,       #...replace this default line with the one below...
  use Appsignal.Ecto.Repo,
    otp_app: :game_reviews_app,
    adapter: Ecto.Adapters.SQLite3
end

Next, let's learn how to track Phoenix LiveViews using AppSignal.

Instrument Phoenix LiveView with AppSignal

With the latest AppSignal for Elixir package (version 2.12.0 as of writing), you can instrument LiveView dashboards in AppSignal.

Simply adding the package is enough to automatically add your app's live views using the AppSignal package's telemetry handlers. Or you can configure the instrumentation manually using helpers made available by the package.

The code below shows how easy it is to manually configure LiveView instrumentation by adding a single line within application.ex:

# lib/game_reviews_app/application.ex

...
def start(_type, _args) do
  Appsignal.Phoenix.LiveView.attach() # <= ...added this line

  children = [
    ...
  ]
  ...
end
...

And with that, you're able to view LiveView mount, handle_event, and handle_param events in the AppSignal dashboard, like so:

Additionally, with the AppSignal LiveView instrumentation helpers, you can wrap LiveView action definitions within an instrument/3 block for even more granular visualizations, like so:

# live/welcome_live.ex

defmodule GameReviewsAppWeb.WelcomeLive do
  use GameReviewsAppWeb, :live_view

  import Appsignal.Phoenix.LiveView, only: [instrument: 4]

  def mount(_params, _session, socket) do
    instrument(__MODULE__, "timer instrumentation", socket, fn ->
      :timer.send_interval(1000, self(), :tick)
      {
        :ok,
        assign(socket, current_time: DateTime.utc_now())
      }
    end)
  end

  def render(assigns) do
    ~H"""
    <div class="container">
      <h1>Welcome to my LiveView App</h1>
      <p>Current time: <%= @current_time %></p>
    </div>
    """
  end
end

When sending metrics to AppSignal, you can use a gauge, a counter, or a distribution.

In the code above, we define a counter measurement called "timer instrumentation" to send current time measurements to the AppSignal dashboard continuously.

As you can see below, by switching the dashboard view from "web" to "live_view", "timer instrumentation" is now available on the dashboard as a sample (with all the details you need to dig further):

Finally, I'll highlight a nifty feature that makes AppSignal really stand out in the application performance monitoring space: automated dashboards.

AppSignal's Automated Dashboards for Elixir

Whenever you add AppSignal to an app built using Elixir, it automatically creates a dashboard.

When we add the AppSignal package to the Phoenix app, we get an automated dashboard and notification — in this case, an Erlang Virtual Machine dashboard:

Check out the AppSignal for Elixir documentation for more information.

Wrapping Up

In this article, we've seen how to track Ecto queries and Phoenix LiveViews using AppSignal for Elixir. Consider this an introduction to what is possible when it comes to error tracking in Elixir with AppSignal.

Next time, we'll deep dive into custom instrumentation using Phoenix and Ecto's in-built telemetry features, learning how you can craft and send custom metrics to AppSignal for monitoring even more powerful use cases.

Happy coding!

P.S. If you'd like to read Elixir Alchemy posts as soon as they get off the press, subscribe to our Elixir Alchemy newsletter and never miss a single post!

Monitor the Performance of Your Ruby on Rails Application Using AppSignal

Aestimo K. — Wed, 26 Jun 2024 12:00:00 +0000

In the first part of this article series, we deployed a simple Ruby on Rails application to DigitalOcean's app platform. We also hooked up a Rails app to AppSignal, seeing how simple errors are tracked and displayed in AppSignal's Errors dashboard.

In this part of the series, we'll dive into how to set up the following for your Ruby on Rails application using AppSignal:

Performance monitoring
Rails background jobs monitoring, including how to monitor simple API calls
Logging
Notification alerts

Let's get into it!

Monitoring Rails App Performance Using AppSignal

When your uptime monitor shows that your app is up, it may be tempting to think that everything is okay. But, in reality, trouble could be brewing under the hood in the form of slow processes, unoptimized database queries, and long-running service calls.

This is a very important matter when you consider that there's a correlation between slow-loading web pages and a low visitor conversion rate. In a nutshell, those slow-running processes will cost you a lot if left unchecked. But with so many moving parts to a Ruby on Rails app, the important questions are: what should you look for, and where?

That's where AppSignal comes into play. Let's look at how you can use AppSignal to keep track of your Rails app's performance.

Tracking Response Times

From the default dashboard, AppSignal provides two graphs that can give you a quick overview of how slow (or fast) your Rails app is running: the Throughput and Response time graphs.

Throughput - This basically measures how many requests per second your app is currently processing (not to be confused with how many requests per second your app can handle overall). The basic rule of thumb is: the more requests per second, the better. However, if your app server handles too many requests per second, it might be time to scale your server resources to handle this.
Response time - The average time a browser response takes in milliseconds. The more time a response takes, the worse your app is running. A good rule of thumb here is that if your Rails web app has sub-100ms response times, you can consider it fast, while 300ms+ response times can be considered slow.

Now, let's say we want to track our app's response times and throughput over a seven-day window. That's very easy to do — just go to the Graphs sub-menu under Performance located on the left-side menu, then use the time filter buttons on top of the graphs, as shown below:

With that, you can easily see if your app runs slow on some days compared to others.

While on this view, it's also important to check out the Event groups graph, located below the Response time and Throughput graphs:

This graph gives you details on how fast or slow your app is running in the controller layer and the view layer. From here, you can find out what's causing slow response times and fix the issues accordingly.

Let's now switch gears to tracking database queries using AppSignal.

Tracking Database Queries

It is generally true that you can squeeze a lot more speed from your app by optimizing response times and throughput. However, more often than not, slow, unoptimized database queries are the worst offenders when it comes to sluggish app behavior.

Let's take an example of the infamous N+1 query. In the expense tracker app introduced in part 1, the index method in the expenses controller looks like this:

# app/controller/expenses_controller.rb

class ExpensesController < ApplicationController
  ...
  def index
    @expenses = Expense.all
  end
  ...
end

This might look innocent, but if you take a look at the Issues list in your AppSignal dashboard, you'll see something interesting:

An N+1 query has been highlighted in the expenses controller's index method.

If we dig further by clicking on the issue link, we get to an Issue details screen like this:

Notice how AppSignal offers an explanation of the issue, including a link to a more detailed blog post showing you how to deal with it.

Let's move on and learn how AppSignal helps you out with background jobs.

Keeping Track of Rails Background Jobs with AppSignal

Background jobs are a common feature in many production Ruby on Rails applications, with a variety of job queue processing gems and libraries for you to choose from.

AppSignal supports tracking and monitoring various background job processing libraries, including Sidekiq, Que, Delayed Job, Resque, and others.

Let's say we want a feature where users get daily currency exchange rates in their dashboard.

For this to work, we need an API call to a service providing such rates (we'll use one that doesn't require too much upfront configuration or payment, like this one, to fetch currency rates). We also need a background job to queue the call to this service at regular intervals.

The background processing gem we'll use in the expense tracker app is GoodJob, but you're free to use whatever suits you.

Let's create a background job to fetch rates:

# app/jobs/fetch_rates.rb
...
class FetchRates < ApplicationJob
  self.queue_adapter = :good_job

  def perform
    # fetch rates
    response = HTTParty.get('https://open.er-api.com/v6/latest/USD')

    if response['result'] == "success"
      Rate.create(
        base_rate_name: response['base_code'],
        eur: response['rates']['EUR'],
        cad: response['rates']['CAD'],
        aud: response['rates']['AUD'],
        gbp: response['rates']['GBP']
        )
    end
  end
end

And then set GoodJob to run a cron for this job every few minutes (or however you see fit). AppSignal automatically detects the presence of the background service and includes a nifty filter so you can separate it from the default web dashboards:

Without the need for much tooling upfront, AppSignal will also detect API calls and keep track of any issues:

If you want to dive deeper into monitoring Rails background jobs with AppSignal, I recommend you check out the documentation. For now, let's turn our attention to uptime monitoring.

Uptime Monitoring

Another matter that concerns many Rails developers is the continuous availability of their application for end users. Any downtime might mean loss of revenue and have other negative consequences.

Even so, you wouldn't want to poll the uptime status of your app manually. You can set up uptime monitoring using AppSignal.

Setting it up is a breeze. Begin by clicking on the Uptime monitoring link in the left-side menu.

Then, when you click on the create uptime monitor button, you should get a dialog similar to the one shown below:

Complete your uptime monitor by adding the most important settings:

Name - Give your uptime monitor an appropriate name.
URL - Provide the full URL to the uptime route. Beginning with Rails 7.1, the default uptime route is https://your-app-domain/up.
Region - Select the regions where you want to check for uptime.
Notification - Choose the notification channel/s to receive alerts on.

With that done, go ahead and create the monitor!

One thing to note with the AppSignal uptime monitor is that since polling is done almost every minute or so, it's very easy to hit your plan's limits. But there's a very good workaround to this which you can read more about in their docs.

Before moving on to how AppSignal can help with logging, there's another important feature to complete the uptime monitoring layer: free public status pages.

Click on the creating a public status page link as shown below:

Then click on the New status page button, bringing you to this:

Go ahead and fill in all the required information. Note that if you use a custom domain, you'll need to add a CNAME directive pointing to cname.appsignal-status.com in your custom domain's DNS settings.

Logging with AppSignal

Logging is a relatively new feature in AppSignal, but one that is timely and welcome. Using AppSignal, you don't have to run application monitoring and logging as separate services.

To get started with logging, make sure you are running the latest version of the AppSignal gem. If you have an older version of the gem, update it with:

bundle update appsignal

Then create a new initializer and edit it like so:

# config/initializers/appsignal_logging.rb

appsignal_logger = Appsignal::Logger.new("rails")
Rails.logger.broadcast_to(appsignal_logger)

Now you can access your app's logs like so:

Notice that you also get a nice filter to filter log records by severity, as well as access to live logs.

You can go through AppSignal's logging for Ruby documentation here.

Other Notable Features

There are a couple of other notable features you can use in AppSignal: anomaly detection and custom metrics.

Anomaly Detection

Let's say you are concerned with your app running out of memory. You can easily set up a trigger that sends an email notification if your app has a notable drop in available memory.

Go to the Anomaly detection link and create a new trigger, then configure it:

Metric - In my case, I am interested in memory usage, but you could choose any other metric that is specific to your use case.
Trigger details - Here, you'll configure the details relevant to your trigger. In my case, the trigger will fire if the host memory goes below 200MB.

Custom Metrics

AppSignal also gives you the tools to capture and visualize custom metrics. This feature alone warrants an entire article. You can read all about it in How to Monitor Custom Metrics with AppSignal and check out AppSignal's docs too.

Wrapping Up

In part one of this article series, we set up a Rails app hosted on DigitalOcean and monitored it for errors using AppSignal.

In this second and final part, we looked at some of AppSignal's great features for your Rails app, including performance monitoring, uptime monitoring, logging, and more.

There's so much more to AppSignal than could effectively be covered by this article series. I highly encourage you to check it out for your Rails app.

Happy coding!

P.S. If you'd like to read Ruby Magic posts as soon as they get off the press, subscribe to our Ruby Magic newsletter and never miss a single post!

Getting Started: Your Ruby On Rails App Hosted On DigitalOcean With AppSignal

Aestimo K. — Wed, 12 Jun 2024 08:45:16 +0000

Imagine this: you’ve just finished working on your brand new Rails app and have deployed it to a cloud provider like DigitalOcean. Like any developer, you’re very proud of your work but you still have lots of questions, like:

How well your new app will handle traffic
Whether the optimizations you put in place will actually work, etc.

Your goal is to provide the best user experience. You want to be notified whenever errors or other important events occur so you can take care of them fast.

It would be great to have a setup that automatically monitors your application. Enter AppSignal! In this article, the first part of a two-part series, we'll set AppSignal up so that you can effectively monitor your Rails app hosted on DigitalOcean.

Prerequisites

To follow along, make sure you have:

A local installation of Ruby (this tutorial uses version 3.3.0).
A local PostgreSQL installation (you can use a Docker version or a locally installed version).
A DigitalOcean account to deploy the application.
An AppSignal account (a free 30-day trial is available).

An Introduction To Our Ruby On Rails App

For this tutorial, we'll be using a simple Rails 7 expense tracker app. Users will be able to sign up and create entries for their personal expenses, to track their expenses over time.

We'll deploy this app to DigitalOcean, then configure AppSignal's monitoring solution to keep track of what is going on under the hood.

You can grab the source code or fork the app from here to follow along.

Deploying Your Rails App To DigitalOcean

We'll use DigitalOcean's app platform to deploy our application. The steps below assume that you've already forked the expense tracker app described above and that you have a DigitalOcean account ready to go.

After logging in, we'll create an app and get it up and running. Since we want to control some parts of this process, though, the first step is to create a database for the app.

Creating the App Database

Click on the Databases link on the left-hand menu, then click on the Create Database link to create a new PostgreSQL database:

After completing the database settings in the next screen, take note of the database connection string. You'll use it as an environment variable when creating the app in the next step:

At this point, you'll likely get a warning that your database is open to all incoming connections. Follow the accompanying link to take care of this security warning.

Creating and Deploying the App

Finally, create the app by first clicking on the Apps link on the left side menu:

Then connect the app's code repository resource as shown below:

Continue to the environment variables screen and insert the copied database URL string. Also, add the RAILS_MASTER_KEY as an environment variable since it will be used in the deployment process.

Then, with these environment variables in place, click on the Next button to deploy the app:

With the app successfully deployed, we'll now set up monitoring using AppSignal.

Setting up AppSignal to Monitor Your Rails App

First, log in to your AppSignal account and choose 'Ruby & Rails'.

Now add the AppSignal gem to the Rails app. This nifty gem will collect errors, exceptions, and relevant performance data, and port it over to AppSignal for analysis.

Open up the app's Gemfile and add the gem:

# Gemfile
gem "appsignal"

Then run bundle install.

Finally, you'll need to run the installation script that comes with your account-specific API key attached. When you run this install script, you should get something similar to the below:

 bundle exec appsignal install <redacted>

#######################################
## Starting AppSignal Installer      ##
## --------------------------------- ##
## Need help?  support@appsignal.com ##
## Docs?       docs.appsignal.com    ##
#######################################

Validating API key...
  API key valid!

Installing for Ruby on Rails

  Your app's name is: 'ExpenseTracker'
  Do you want to change how this is displayed in AppSignal? (y/n): n
How do you want to configure AppSignal?
  (1) a config file
  (2) environment variables
  Choose (1/2): 1

Writing config file...
  Config file written to config/appsignal.yml

#####################################
## AppSignal installation complete ##
#####################################

  Sending example data to AppSignal...
  Example data sent!

There are a couple of things to note when you run the script:

You'll get the option to change your app's name as it will appear on AppSignal's dashboard or leave the default option.
You'll also get the option to choose how you want AppSignal configured for your app. In my case, I went with the config file option, creating a config file in config/appsignal.yml, with the below content:

# config/appsignal.yml

default: &defaults
  push_api_key: "<%= ENV['APPSIGNAL_PUSH_API_KEY'] %>"

  name: "ExpenseTracker"

development:
  <<: *defaults
  active: true

production:
  <<: *defaults
  active: true

Here, we define:

A push_api_key which connects the app to AppSignal.
The app's name as it will appear on AppSignal.
The environments in which AppSignal will monitor the app (in this case, both development and production environments).

By the way, if you'd like to turn off AppSignal monitoring on the development environment, just set the active flag to false, like so:

...
development:
  <<: *defaults
  active: false
...

Tip: Both the config file and environment variable options do the same thing. They define how AppSignal will connect to your app, the app name, and which environment will be monitored.

Assuming everything goes to plan, you should see a screen showing that AppSignal is receiving data from your app!

An Introduction to AppSignal's Dashboard for Rails

Now that everything is set up correctly and AppSignal is receiving data from your app, you can access the default app monitoring dashboard view as shown below:

Tip: If you set up monitoring for both development and production environments, you can easily switch between them from the selection shown by the arrow.

From the default view, you have access to the following default charts:

Error rate - This will show the rate of errors occurring in your app per unit of time.
Throughput - Here, you'll get a snapshot of the throughput your app is able to handle in terms of requests per minute.
Response time - This will show your app's response times.
Latest open errors - This section will list the latest errors that will have happened within your app.
Latest open performance measurements - This section lists the latest performance measurements, including method calls, API requests, etc.

Next, we'll see how to set up proper error tracking for a Rails application using AppSignal.

Monitoring Errors with AppSignal

There are more than ten error types that could affect a running Ruby on Rails app. Obviously, some are more common than others. In this section, we'll simulate a few of these errors and see how AppSignal handles them. Let's start with a simple example first.

Since the expense tracker app is using Devise for authentication, let's add a check for whether a user is signed in. Instead of using the recommended user_signed_in? method, let's use the erroneous user_logged_in? method, which should trigger an ActionView::Template::Error:

<!-- app/views/shared/_navbar.html.erb -->
...
<% if user_logged_in? %>
    <%= link_to 'Logout', destroy_user_session_path %>
<% else %>
    <%= link_to new_user_session_path do %>
    Login
    <% end %>
<% end %>
...

Deploy this change, reload the production app, then go into the production environment dashboard in AppSignal and watch how this error shows up:

AppSignal makes it a breeze to get more details on any errors that happen in an application. For example, we can get details about the ActionView template error by clicking on it from the 'Latest Open Errors' dashboard panel.

You can clearly see that the error is caused by an undefined method which points us in the right direction to fix it.

AppSignal's Errors dashboard also gives you extra information through the Logbook and Settings panels:

Logbook - Here, you or your team members can add comments to an error being tracked by AppSignal. The Logbook panel also shows a chronological breakdown of any actions taken regarding the error in question.
Settings - From this panel, you can assign an error to another team member, change alert settings (we'll check these out in detail in the second part of this tutorial), and set error severity.

And that's it for this part of the series!

Wrapping Up

In this tutorial, we deployed a simple Rails application to DigitalOcean's app platform and hooked it up to AppSignal's application monitoring platform. We also went through how errors are monitored and displayed in AppSignal's Errors dashboard.

Obviously, this is just scratching the surface of what's possible with AppSignal. In the second part of this series, we'll dive deeper into performance measurements, anomaly detection, uptime monitoring, and logging.

In the meantime, happy coding!

P.S. If you'd like to read Ruby Magic posts as soon as they get off the press, subscribe to our Ruby Magic newsletter and never miss a single post!

Should You Use Ruby on Rails or Hanami?

Aestimo K. — Wed, 08 May 2024 10:23:08 +0000

Ruby on Rails is the most popular web framework in the Ruby ecosystem and has a large user base, ranging from freelancers to large established companies. With an active user community and wide-ranging documentation, it can be used to build everything from simple applications to complex web platforms.

That said, a new contestant is taking on Rails’ dominance for the full-stack Ruby framework title: Hanami. It is a fast, modular Ruby framework with improved performance and maintainability compared to Rails.

In this article, we'll explore the strengths and weaknesses of each framework in terms of performance, features, testing, and more. So whether you are looking to build a customer-facing web app, an internal tool, or a massively scalable API, you should leave being better informed about what to use for your next project.

Let's get started with a brief introduction to each framework.

Introducing Ruby on Rails

Ruby on Rails is the most well-known Ruby web application development framework with a mission to enhance developer productivity (by making a bunch of assumptions about how apps should be built, often referred to as the "Rails way").

Some of these assumptions include making sure developers do not spend time configuring stuff, or what is termed as "convention over configuration", and an emphasis on using DRY ("don't repeat yourself") principles. DRY principles encourage a developer to avoid repeating code over and over again, but instead use single and focused representations of app functionality to ensure maintainability and organization.

Introducing Hanami

While Rails is very well-known in the Ruby community, Hanami is less so. It's a fairly new modern Ruby framework trying to take on Rails' dominance of the full-stack web framework space.

Hanami is built from the ground up to have a small memory footprint and a focus on modularity, which, in turn, makes for a very fast and nimble framework.

Obviously, these brief introductions won't really give you all the information you need to decide on the framework that suits you best. For that, we'll need to dive deeper into each, starting with how they are structured.

Structure and Architecture of Rails and Hanami

Rails and Hanami are somewhat similar in that both are Ruby frameworks. However, how each is built and their application architecture is where you'll find most of the differences.

For starters, with Rails, you have fewer files or abstractions (the building blocks you use to build your app), which tend to grow in size as you develop your app. On the other hand, Hanami takes abstractions to a whole new level, with lots of files that tend to be smaller in size.

The diagrams below illustrate the point more clearly. Let's start with Rails.

Now compare the Rails abstraction diagram with the Hanami one below.

As you can see, each framework generally follows the model-view-controller (MVC) structure. However, Hanami takes abstraction to the next level.

Here's a simplified outline of how each framework is organized:

Routes - Each framework has a routes definition with your app's endpoints.
Controllers vs. actions - In Rails, you get controllers with one or a couple of actions in them. Controllers receive and respond to route requests by directing them to the relevant actions. In Hanami, there are no controllers. Instead, you go straight to self-contained actions (each action has its own self-contained class).
Models and persistence - The Rails model layer takes care of data validation and database communication, including any queries your app may have. In Hanami, each of these functionalities is contained in a different part of the persistence layer: data validations are handled by contracts, database communication is done by the repository (repo), while relations and scoping are the responsibility of relations.
View rendering - As with the persistence layer, Ruby on Rails views tend to contain everything you need for rendering data to the outside world in one place. This includes all the HTML structure, view helpers, and view logic. When it comes to view rendering in Hanami, things are more abstract. For starters, you have views that utilize any view helpers the app has and also render the template. The template handles the actual HTML structure, while the parts take care of any presentation logic.

So, what does all this mean when it comes to building an app? With fewer abstractions, Rails is a good choice for getting your app off the ground and is more beginner-friendly (as we shall see later on in this article). With Rails, you can build very robust monolith apps, but your code will get more and more complex as you scale. On the other hand, Hanami's more complex structure can be daunting to learn, but it will allow you to build massively scalable applications and could make for much better code organization.

Next, let's take a look at each framework's ecosystem.

Ecosystem and Community

As we mentioned earlier, Rails has a more established framework than Hanami. Rails has been around for longer and, as such, has a bigger and more mature community.

The differences in ecosystem and community are summarized below:

Documentation - It doesn't matter what you're trying to build, if you use Rails, you have access to very well-done documentation. You are guided through all parts of an app build, from authentication, to data persistence, view presentation, and everything in between. Additionally, a wide range of third-party tutorials cover almost everything imaginable in terms of building an application.

That said, things are not so established on the Hanami side of things. Being a relatively newer framework, Hanami has a comparatively limited documentation base. The Hanami team has done an impressive job with the official guides, but compared to the Rails documentation base, it doesn't come close.

Community - Here, as with the documentation, Rails is the clear winner. Rails has been around much longer than Hanami, so it's been adopted the most. It doesn't matter whether you are a beginner or advanced developer, you'll find a Rails community on relevant subreddits, Slack groups, Discords, and so forth. On the other hand, Hanami is still growing, meaning it has a much smaller community.
Gems and libraries - Since both Hanami and Rails are Ruby frameworks, it could be argued that gems that work in Rails will work in Hanami. Although this is technically true, you'll often find that Rails has a more established base of gems and libraries for all sorts of specialized functions.

That said, because of Hanami's focus on abstractions and specialization, it has some very advanced gems that could take your Rails apps to the next level. For example, the default dry-rb gems in Hanami could bring better code organization and abstraction when used in Rails apps.

Moving on, let's compare each framework's learning curve and adoption.

Ease of Use, Adoption, Governance, and Learning Curve

For the very reasons outlined in the previous section, Ruby on Rails easily beats Hanami in terms of ease of use, industry adoption, and learning curve, specifically:

Learning - Imagine you're a complete beginner who's looking to learn a new programming language. Your very first thought will be to check out online learning resources. If a framework has widely available learning materials covering the app-building process from start to finish, you'll likely choose that language over others. And because Rails has a more established documentation base, it trumps Hanami as the beginner's choice of framework. In addition, Rails is much more friendly for beginners to pick up since it makes so many assumptions under the hood (compared to Hanami, which offers a more abstract way of building apps). Hanami's abstractions can be daunting even for established Rails developers.
Industry adoption - Without including the adoption of other popular and more established frameworks like Python, React, C#, and others, if we consider the adoption of Ruby frameworks, Rails easily eclipses Hanami. The Rails homepage lists some big-name organizations using the framework. On the other hand, as the new kid on the block, Hanami is not so widely adopted. We'll have to wait and see whether that will change in the future.
Job market prospects - In terms of job prospects, you'll find more job openings for Rails developers than Hanami developers.
Governance - Another important aspect that should not be overlooked is governance. A changing landscape guides how open-source frameworks evolve and advance. However, more than that, the core team members of these frameworks often wield immense powers and can dictate what goes into a framework, how it develops, etc.

A good example is January's announcement by David Heinemeier Hansson (DHH), the creator of Rails. He said that, in the future, he would push for Rails to offer first-class support for building full-stack progressive web applications and native notifications. These features would make Rails very attractive to mobile developers.

In reaction, many developers, some outside the Ruby ecosystem and even others who had dropped Rails over the years, overwhelmingly responded in the positive, saying they would gladly pick up the framework or learn it if DHH kept his word. This example just goes to show you that governance is a very important consideration when deciding what framework to pick.

Let's now switch gears and look into something more technical, application performance.

Performance: Ruby on Rails vs. Hanami

As a developer, you will be concerned about your app's reliability, responsiveness, and how it will utilize server resources once deployed in production. These are fundamental concerns that can greatly affect your choice of framework.

You could use many methods to run performance tests on your app, one of the most popular being Apache JMeter. Let's use the benchmark numbers to compare Hanami and Rails.

Here's a quick benchmark test showing the number of requests each framework can handle per second:

Hanami beats Rails hands down, with the ability to handle 3 times more requests than Rails can handle.

This next screenshot shows the average latency values for each framework:

Again, Hanami beats Rails with an average latency time that's about 3 times shorter.

If you're looking for a Ruby framework that is blazingly fast (let's say, you need to work on a fast and hugely scalable API), then you'll be hard-pressed to find anything better than Hanami in the Ruby landscape.

Testing

When it comes to testing code, both frameworks are very much comparable since you can test either using the versatile RSpec library.

RSpec is included via the hanami-rspec gem for Hanami apps, whereas you need to install the rspec-rails gem to use it in your Rails app. The example below shows a very basic test spec that is included with your new Hanami app:

# spec/requests/root_spec.rb

RSpec.describe "Root", type: :request do
  it "is successful" do
    get "/"

    expect(last_response).to be_successful
    expect(last_response.body).to eq("Hello from Hanami")
  end
end

Running it with $ bundle exec rspec spec/requests/root_spec.rb should result in a passing test (assuming you've not edited the default view template):

.

Finished in 0.01812 seconds (files took 0.47734 seconds to load)
1 example, 0 failures

On the Rails side, you need to handle some things by yourself. Firstly, you need to add the RSpec gem to the development/test block in the Gemfile as shown below, then run bundle:

# Gemfile
...
group :development, :test do
    gem 'rspec-rails'
end

The next step is to run the RSpec install with the command bundle exec rails generate rspec:install to get it ready for your Rails project.

Finally, before writing and running any tests, you'll likely need to install another gem to define test data for your Rails app: FactoryBot.

And assuming you've set up your test suite properly and defined a few tests, you can run a test just as you would in Hanami, with bundle exec rspec spec/models/user_spec.rb. You can get your test results just as you would with a Hanami app:

Finished in 0.04421 seconds (files took 0.88106 seconds to load)
1 example, 0 failures

Finally, let's see how the frameworks compare when it comes to deployment.

Deployment

Nowadays, there are lots of options for deploying Ruby apps to production.

To begin with, you could go with a Platform-as-a-Service (PaaS) provider like Heroku, or Fly for a more seamless experience. You can also do a bit of DevOps: set up a Docker installation on a VPS and deploy your app there.

Whichever option you choose, you can expect deployment to be relatively similar in both cases. The only caveat is the lack of extensive documentation and tutorials for deploying Hanami apps.

As we pointed out before, being a newer framework, Hanami still doesn't have extensive documentation covering the deployment process and the challenges that might crop up. If you don't mind hacking around this, then you should be okay.

Wrapping Up

In this article, we've looked at two Ruby frameworks — Ruby on Rails and Hanami — comparing them in terms of features, architecture, performance, and more. Ultimately, we have to answer the question: "Which framework should I use, and why?". We can summarize as follows:

If you're a beginner Ruby developer, start with Rails: it's easier to pick up and learn. If you're a more advanced Ruby developer, develop a Hanami skillset, as it will help you develop even more robust Ruby applications.
If you need to develop an app that should be fast, say an API serving several clients, Hanami's small memory footprint, blazing fast responses, and low latency will serve you well. However, if you just need to develop a monolithic full-stack app to validate a SaaS idea, Rails will definitely get you there faster.

Ultimately, what you end up choosing is really up to you. It isn't a bad idea to acquire skills in both frameworks. That way, you can decide on the best framework to go for, depending on the needs of your project.

Happy coding!

P.S. If you'd like to read Ruby Magic posts as soon as they get off the press, subscribe to our Ruby Magic newsletter and never miss a single post!

P.P.S. Did you know AppSignal has an integration for Rails and Hanami?

Active Record or Sequel: Which Best Fits The Needs of Your Ruby App?

Aestimo K. — Wed, 20 Mar 2024 14:00:00 +0000

When it comes to choosing an object-relational mapping (ORM) library for your Ruby application, Active Record is usually the favorite choice. It's an easy-to-use ORM library that allows for lots of data wrangling without resorting to SQL. All the same, you might wonder: "Is Active Record the only Ruby ORM library I can use?"

In this article, we'll compare some Active Record features to its lesser-known but powerful cousin, Sequel. There are too many points of comparison to cover everything (such as how each library handles CRUD operations, table joins, associations, database replication and sharding, etc). Instead, we'll scratch the surface of a few database operations — namely, filtering, database locking, and transactions — and show how each library handles them.

By the end, you'll have a better idea about how to use each library's strengths to the fullest.

But first, let's learn what object-relational mapping is all about.

What Is Object-Relational Mapping (ORM)?

Object-relational mapping is a way to pass data between an application and its data store, usually a relational database like SQLite, PostgreSQL, or MySQL. Without ORM techniques, you would be forced to write raw SQL queries for all operations to get data in or out of your Ruby app's database. But with an ORM, you get access to class objects and methods, making it much easier to read/write data to your database without using raw SQL.

Let's introduce the two ORM libraries we're looking at, starting with Active Record, and then moving on to Sequel.

Introducing Active Record and Sequel for Ruby

Let's quickly examine what Active Record and Sequel can do.

Active Record

Active Record is the most well-known Ruby object-relational mapping library. It defines class objects that are directly mapped to tables in a database and provides convenient methods for manipulating data using these classes.

The library was first described by the pioneering computer scientist Martin Fowler in his book Patterns of Enterprise Architecture. It comes bundled with Rails, which could explain why it's more popular than Sequel. That said, Active Record is suitable for:

Object-oriented database operations
Performing validations on models before persisting them in the database
Representing models, their data, and relationships

Sequel

On the other hand, Sequel is the lesser-known of the two. It includes a powerful DSL for manipulating data within a variety of databases.

Sequel can:

Represent database records as Ruby objects
Handle simple and even complex associations in a concise way
Do database sharding, database replications, and more

One standout feature that makes Sequel so powerful is its dataset. A dataset is Sequel's way of directly representing an SQL query and making it readily available for a developer to use.

Of course, a lot more could be said of each of these ORM libraries, but we'll leave it at that. Let's dive straight into our first example to see how Active Record and Sequel deal with record filtering.

Filtering Records

Filtering data is the process of refining data results by applying specific filters. By using filters, you can fetch exactly what you want from a dataset. Let's see how each ORM library achieves this, starting with Active Record.

Using Active Record

Let's say you want to filter data to return all orders of products priced under $20. How would you achieve this using Active Record?

You can apply conditions to your Active Record query. Active Record has many methods that act as conditions for filtering data, including where, limit, and where.not. While it's not possible to get into all of the conditions in this article, we'll sample a few and see how they stack up against Sequel's filter methods.

For Active Record, you'll first need to set up the proper model associations like so:

# models/order.rb
class Order < ActiveRecord::Base
  belongs_to :product
end

Then the Product model:

# models/product.rb

class Product < ActiveRecord::Base
  has_many :orders
end

Now for the Active Record query:

product_orders_under_20 = Order.joins(:product).where('products.price < ?', 20)

In this example, we've used a joins to include results from the products table and then applied a where condition to filter for the products that fit our desired criteria.

Using Sequel

Sequel's use of SQL representations in the form of datasets makes filtering a breeze. Let's consider how Sequel would handle the example we used for Active Record.

Just like we did with Active Record, you need to define some models first:

# models/product.rb

class Product < Sequel::Model
  one_to_many :orders
end

The Order model:

# models/order.rb

class Order < Sequel::Model
  many_to_one :product
end

And this is how you'd run the query with Sequel:

product_orders_under_20 = DB[:orders].join(:products, id: :product_id).where { price < 20 }

Next up, let's look at database locks.

Database Locking

Imagine you have a content management system where a user with the editor role can edit other users' posts. Let's say there are several editors in the organization and it just so happens that two editors end up working on the same post at the same time. Such a scenario raises several questions, such as which editor's work is saved, assuming both commit changes at the same time.

This challenge is solved through the use of database locks.

Using Active Record

Active Record allows for two types of locks: optimistic locks and pessimistic locks. In optimistic locking, you need to include a version column in the respective database table. This is so that a record is checked against a matching version to ensure that another user or process doesn't modify it. If it is, an error is raised.

The example below shows how optimistic locking happens:

invoice1 = Invoice.find(1)
invoice2 = Invoice.find(1)

invoice1.total_amount = 100
invoice1.save # 100

invoice2.total_amount = 500
invoice2.save # raises ActiveRecord::StaleObjectError

When it comes to pessimistic locks, Active Record can implement them at the row level if your database allows for it. There are several ways of achieving pessimistic locking with Active Record, but we won't go into the details now. Instead, the example below should give you an idea of how this is done:

Invoice.lock.find(1) # applies a pessimistic lock on the record so that an UPDATE is done

Using Sequel

On the other hand, Sequel implements optimistic locking through a plugin.

You first add the plugin to the model like so:

# invoice.rb

class Invoice < Sequel::Model
  plugin :optimistic_locking
end

Then test this with two concurrent queries:

invoice1 = Invoice.find(1)
invoice2 = Invoice.find(1)

invoice1.update(status: 'Sent') # record will be updated

invoice2.update(status: 'Void') # raises Sequel::NoExistingObject error

For pessimistic locking, Sequel implements the lock when the records (or record) are fetched using the following methods: lock_style, lock, and for_update. Again, we won't provide details on this. You can read the documentation or Jeremy Evans' Pessimistic Locking blog post to learn more.

Here's an example using the for_update method to implement a pessimistic lock:

dataset = DB[:invoices]
invoice1 = dataset.where(id: 1).for_update # will apply a pessimistic lock on invoice1

Next, let's take a look at database transactions.

Database Transactions

To understand the concept of database transactions, consider a peer-to-peer lending app where Person A sends some money to Person B who can then withdraw the money and use it. Here, the app needs to first withdraw money from Person A's account and then send that amount over to Person B.

In such a scenario, you wouldn't want the app to send money to Person B if there's an error with the withdrawal from Person A's account. The ideal solution would be to wrap the two processes in a "database transaction", so if one of the processes fails (usually the first one), then the database can be rolled back to its original state before any of the transactions happened.

Using Active Record

With Active Record, implementing these queries without the use of database transactions might look something like this:

person_a = Person.find_by(name: 'A')
person_b = Person.find_by(name: 'B')

withdrawal = person_a.accounts.first.withdraw(250)
person_b.accounts.first.account_total # 100

person_b.find(B).accounts.first.deposit(withdrawal)
person_b.accounts.first.account_total # 450

Although this could work, you also need to account for something going wrong with the withdrawal and take steps to take care of it. Let's now use a database transaction to improve this example:

a_account = Person.find_by(name: 'A').accounts.first
b_account = Person.find_by(name: 'B').accounts.first

Account.transaction do
  a_account.withdraw(250)
  b_account.deposit(250)
end

Here, we use ActiveRecord::Transactions to wrap the two queries in a transaction so that the database can be rolled back if something goes wrong.

What about Sequel?

Using Sequel

Just like in Active Record, if you want to implement transactions in Sequel, you'll need to be explicit (although there are situations where this is enabled by default).

With the same example we used for Active Record, this is how to wrap our queries using a transaction in Sequel:

DB.transaction do
  Person.first(name: "A").accounts.first.withdraw(250)
  Person.first(name: "B").accounts.first.deposit(250)
end

From this example, we see that Sequel's dataset feature allows for complex transactional queries to be built using a few commands.

When To Choose Active Record Vs. Sequel for Ruby

When should you use Active Record, and when might Sequel be a better choice? Here's a quick summary of each:

Active Record is very beginner-friendly, in that, it conveniently hides complex SQL queries under the hood. On the other hand, if you are an advanced user who's comfortable using SQL, then this may be a bit limiting. Additionally, if you're looking for more powerful features like proxying queries, you'll likely need to add complementary gems on top of Active Record.
Sequel is very powerful, especially for those who master its dataset feature. That said, you can expect a steeper learning curve compared to Active Record. If you get through that, though, you will be rewarded with a flexible ORM that is perfect for almost anything you could throw at it.

Wrapping Up

In this article, we've compared two Ruby object-relational mapping libraries, Active Record and Sequel. We explored how each handles filtering, database locking, and transactions. Finally, we touched on when you might want to use one ORM over the other.

Until next time, happy coding!

P.S. If you'd like to read Ruby Magic posts as soon as they get off the press, subscribe to our Ruby Magic newsletter and never miss a single post!

P.P.S. Did you know that AppSignal offers an Active Record integration? Find out more.

Advanced Multi-tenancy for Elixir Applications Using Ecto

Aestimo K. — Tue, 19 Dec 2023 11:16:25 +0000

Welcome to part two of this series. In the previous tutorial, we learned about multi-tenancy, including different multi-tenancy implementation strategies. We also started building a multi-tenant Phoenix link shortening app and added basic user authentication.

In this final part of the series, we'll build the link resource, associate users to links, and set up the redirect functionality in our app. As we finalize the app build, we'll learn about features (like Ecto custom types) that make Phoenix such a powerful framework for Elixir.

Let's now turn our attention to link shortening.

Adding `Link` Resources

As a reminder, here's how link shortening will happen in our app:

Logged-in user enters a long URL
A random string is generated and associated with the long URL
Whenever this short URL is visited, the app will keep a tally of the number of visits

We'll get started by generating a controller, schema, migration, and views for a Link resource:

mix phx.gen.html Links Link links url:string visits:integer account_id:references:accounts

This should generate most of the boilerplate code we need to work with the Link resource.

You may notice that we haven't included a field for the short random string referencing the long URL. We'll cover this next.

Creating Short URLs Using the Ecto Custom Type in Phoenix

By default, Ecto models have an id field used as a model's primary key. Usually, it's in the form of an integer or, in some cases, a binary ID. In either case, when present in a model, this field is automatically incremented for every additional model created in an app.

A core function of our app is generating short, unique strings to represent long URLs. We could write a custom function to generate such strings, but since we are on a quest to learn Elixir, let's use a more creative approach.

Let's substitute the id primary key in the Link model with a custom Ecto type called HashId.

Using this approach, we'll:

Learn how to create and use Elixir custom types.
Automatically generate unique string hashes to form the short URL field for links. Ecto will manage the process whenever a new link model is created.

First, create a new file to represent this new type:

# lib/ecto/hash_id.ex

defmodule Urlbot.Ecto.HashId do
  @behaviour Ecto.Type
  @hash_id_length 8

  # Called when creating an Ecto.Changeset
  @spec cast(any) :: Map.t
  def cast(value), do: hash_id_format(value)

  # Accepts a value that has been directly placed into the ecto struct after a changeset
  @spec dump(any) :: Map.t
  def dump(value), do: hash_id_format(value)

  # Changes a value from the default type into the HashId type
  @spec load(any) :: Map.t
  def load(value), do: hash_id_format(value)

  # A callback invoked by autogenerate fields
  @spec autogenerate() :: String.t
  def autogenerate, do: generate()

 # The Ecto type that is being converted
  def type, do: :string

  @spec hash_id_format(any) :: Map.t
  def hash_id_format(value) do
    case validate_hash_id(value) do
      true -> {:ok, value}
      _ -> {:error, "'#{value}' is not a string"}
    end
  end

  # Validation of given value to be of type "String"
  def validate_hash_id(string) when is_binary(string), do: true
  def validate_hash_id(_other), do: false

  # The function that generates a HashId
  @spec generate() :: String.t
  def generate do
    @hash_id_length
    |> :crypto.strong_rand_bytes()
    |> Base.url_encode64
    |> binary_part(0, @hash_id_length)
  end
end

Check out the Ecto custom types documentation, which covers the subject in a much more exhaustive way than we could in this tutorial.

For now, what we've just done allows us to use the custom data type HashId when defining a field's data type.

Let's use this new data type to define the short URL field for the Link resource next.

Using the Custom Ecto Type in Phoenix

Open up the Link schema and edit it to reference the new Ecto type we've just defined:

# lib/urlbot/links/link.ex

defmodule Urlbot.Links.Link do
  use Ecto.Schema
  import Ecto.Changeset
  alias Urlbot.Ecto.HashId # Add this line

  @primary_key {:hash, HashId, [autogenerate: true]} # Add this line
  @derive {Phoenix.Param, key: :hash} # Add this line

  ...
end

Before moving on, let's briefly explain the use of @derive in the code above.

In Elixir, a protocol defines an API and its specific implementations. Phoenix.Param is a protocol used to convert data structures into URL parameters. By default, this protocol is used for integers, binaries, atoms, and structs. The default key :id is usually used for structs, but it's possible to define other parameters. In our case, we use the parameter hash and make its implementation derivable to actually use it.

Let's modify the links migration to use this new parameter type:

# priv/repo/migrations/XXXXXX_create_links.exs

defmodule Urlbot.Repo.Migrations.CreateLinks do
  use Ecto.Migration

  def change do
    create table(:links, primary_key: false) do
      add :hash, :string, primary_key: true
      add :url, :string
      add :visits, :integer
      add :account_id, references(:accounts, on_delete: :delete_all)

      timestamps()
    end

    create index(:links, [:account_id])
  end
end

Then run the migration to create the links table:

mix ecto.migrate

We now need to associate Link and Account— but before we do, let's ensure that only an authenticated user can make CRUD operations for Link.

Updating `Link` Routes in Phoenix for Elixir

In the router, we need to create a new pipeline to process routes that only authenticated users can access.

Go ahead and add this pipeline:

# lib/urlbot_web/router.ex

defmodule UrlbotWeb.Router do
  use UrlbotWeb, :router
  use Pow.Phoenix.Router

  ...

  pipeline :protected do
    plug Pow.Plug.RequireAuthenticated, error_handler: Pow.Phoenix.PlugErrorHandler
  end

  ...
end

Then add the scope to handle Link routes:

# lib/urlbot_web/router.ex

defmodule UrlbotWeb.Router do
  use UrlbotWeb, :router
  use Pow.Phoenix.Router

  ...

  pipeline :protected do
    plug Pow.Plug.RequireAuthenticated, error_handler: Pow.Phoenix.PlugErrorHandler
  end

  scope "/", UrlbotWeb do
    pipe_through [:protected, :browser]

    resources "/links", LinkController
  end

  ...
end

Now, if you try to access any of the protected routes, Pow should redirect you to a login page:

Our app build is progressing well. Next, let's associate Link and Account since this forms the basis of tenant separation in our app.

Assigning `Link` to `Account` in Elixir

Take a look at the Link schema below. We want to make sure account_id is included, since this will form the link to Account:

# lib/urlbot/links/link.ex

defmodule Urlbot.Links.Link do
  use Ecto.Schema
  import Ecto.Changeset
  alias Urlbot.Ecto.HashId

  ...

  schema "links" do
    field :url, :string
    field :visits, :integer
    field :account_id, :id

    timestamps()
  end
  ...

end

Next, edit the changeset block, adding account_id to the list of allowed attributes and those that will go through validation:

# lib/urlbot/links/link.ex

defmodule Urlbot.Links.Link do
  use Ecto.Schema
  import Ecto.Changeset
  alias Urlbot.Ecto.HashId

  ...

  def changeset(link, attrs) do
    link
    |> cast(attrs, [:url, :visits, :account_id])
    |> validate_required([:url, :visits, :account_id])
  end

end

Then, edit the Account schema and add a has_many relation as follows:

# lib/urlbot/accounts/account.ex

defmodule Urlbot.Accounts.Account do
  use Ecto.Schema
  import Ecto.Changeset
  alias Urlbot.Users.User
  alias Urlbot.Links.Link

  schema "accounts" do
    field :name, :string

    has_many :users, User
    has_many :links, Link # Add this line

    timestamps()
  end

  ...

end

We also need to edit the Link context to work with Account. Remember, scoping a resource to a user or account is the foundation of any multi-tenant structure.

Edit your file as shown below:

# lib/urlbot/links.ex

defmodule Urlbot.Links do
  ...

  def list_links(account) do
    from(s in Link, where: s.account_id == ^account.id, order_by: [asc: :id])
    |> Repo.all()
  end

  def get_link!(account, id) do
    Repo.get_by!(Link, account_id: account.id, id: id)
  end

  def create_link(account, attrs \\ %{}) do
    Ecto.build_assoc(account, :links)
    |> Link.changeset(attrs)
    |> Repo.insert()
  end

  ...

end

Adding a `Current_Account`

Our edits to the Link context show that most related controller methods use the account variable. This variable needs to be extracted from the currently logged-in user's account_id and passed on to the respective controller action.

So we need to make a custom plug to add to the conn. Create a new file — lib/plugs/set_current_account.ex — and edit its content as shown below:

# lib/plugs/set_current_account.ex

defmodule UrlbotWeb.Plugs.SetCurrentAccount do
  import Plug.Conn
  alias Urlbot.Repo
  alias Urlbot.Users.User

  def init(options), do: options

  def call(conn, _opts) do
    case conn.assigns[:current_user] do
      %User{} = user ->
        %User{account: account} = Repo.preload(user, :account)
        assign(conn, :current_account, account)
      _ ->
        assign(conn, :current_account, nil)
    end
  end
end

Next, let's use this new plug by adding it to the router (specifically to the protected pipeline since user sessions are handled there):

# lib/urlbot_web/router.ex

defmodule UrlbotWeb.Router do
  use UrlbotWeb, :router
  use Pow.Phoenix.Router

  ...

  pipeline :protected do
    plug Pow.Plug.RequireAuthenticated, error_handler: Pow.Phoenix.PlugErrorHandler
    plug UrlbotWeb.Plugs.SetCurrentAccount # Add this line
  end

  ...

  scope "/", UrlbotWeb do
    pipe_through [:protected, :browser]

    resources "/links", LinkController
  end

  ...

end

Then, edit the Link controller to pass the current_account in every action where it's required, like so:

# lib/urlbot_web/controllers/link_controller.ex

defmodule UrlbotWeb.LinkController do
  use UrlbotWeb, :controller

  alias Urlbot.Links
  alias Urlbot.Links.Link

  def index(conn, _params) do
    current_account = conn.assigns.current_account  # Add this line
    links = Links.list_links(current_account)
    render(conn, :index, links: links)
  end
  def create(conn, %{"link" => link_params}) do
    current_account = conn.assigns.current_account # Add this line
    case Links.create_link(current_account, link_params) do
      {:ok, link} ->
        conn
        |> put_flash(:info, "Link created successfully.")
        |> redirect(to: ~p"/links")

      {:error, %Ecto.Changeset{} = changeset} ->
        render(conn, :new, changeset: changeset)
    end
  end

  def edit(conn, %{"id" => id}) do
    current_account = conn.assigns.current_account  # Add this line
    link = Links.get_link!(current_account, id)
    changeset = Links.change_link(link)
    render(conn, :edit, link: link, changeset: changeset)
  end

  def update(conn, %{"id" => id, "link" => link_params}) do
    current_account = conn.assigns.current_account  # Add this line
    link = Links.get_link!(current_account, id)

    case Links.update_link(link, link_params) do
      {:ok, link} ->
        conn
        |> put_flash(:info, "Link updated successfully.")
        |> redirect(to: ~p"/links/#{link}")

      {:error, %Ecto.Changeset{} = changeset} ->
        render(conn, :edit, link: link, changeset: changeset)
    end
  end

  def delete(conn, %{"id" => id}) do
    current_account = conn.assigns.current_account  # Add this line
    link = Links.get_link!(current_account, id)
    {:ok, _link} = Links.delete_link(link)

    conn
    |> put_flash(:info, "Link deleted successfully.")
    |> redirect(to: ~p"/links")
  end

end

That's it! Now, any created Link will be associated with a currently logged-in user's account.

At this point, we have everything we need for users to create links that are properly scoped to their respective accounts. You can see this in action when you consider the list index action:


defmodule UrlbotWeb.LinkController do
  use UrlbotWeb, :controller

  alias Urlbot.Links
  alias Urlbot.Links.Link

  def index(conn, _params) do
    current_account = conn.assigns.current_account
    links = Links.list_links(current_account)
    render(conn, :index, links: links)
  end

  ...
end

Here, a user should only see links that are associated with them.

For example, this is a list index view for one user:

And this is a view for a different user (notice the logged-in user's email and the different link URLs):

Next, let's build the link redirection feature.

Redirecting Links and Incrementing Link Views in Phoenix

Let's begin by adding a new controller to handle the redirect. This controller will have one show action:

# lib/urlbot_web/controllers/redirect_controller.ex

defmodule UrlbotWeb.RedirectController do
  use UrlbotWeb, :controller

  alias Urlbot.Links

  def show(conn, %{"id" => id}) do
    short_url = Links.get_short_url_link!(id)
    Links.increment_visits(short_url)
    redirect(conn, external: short_url.url)
  end

end

We also need to modify the Link context with a custom get function that does not reference the current user:

# lib/urlbot/links.ex

defmodule Urlbot.Links do
  ...

  def get_short_url_link!(id) do
    Repo.get!(Link, id: id)
  end

  ...

end

Next, modify the router accordingly:

# lib/urlbot_web/router.ex

defmodule UrlbotWeb.Router do
  use UrlbotWeb, :router
  use Pow.Phoenix.Router

  ...

  scope "/", UrlbotWeb do
    pipe_through :browser

    get "/", PageController, :home
    get "/links/:id", RedirectController, :show # Add this line
  end

  ...

end

Finally, to handle the link view visits, we'll add a custom function to the Link context:

# lib/urlbot/links.ex

defmodule Urlbot.Links do
  ...

  def increment_visits(%Link{} = link) do
    from(s in Link, where: s.id == ^link.hash, update: [inc: [visits: 1]])
    |> Repo.update_all([])
  end

end

That's it! Now, when we visit a link like http://localhost:4000/links/rMbzTz3o, this should redirect to the original long URL and increment the link's view counter.

Wrapping Up

This two-part series has taken you on a journey to build a multi-tenant Elixir app using the Phoenix web framework. You've learned how to implement a simple tenant separation system using foreign keys and related models within a shared database and shared schema.

Although we've tried to be as exhaustive as possible, we couldn't possibly capture the whole app-building process, including testing, deployment options, or even the use of the amazing LiveView.

All the same, we hope this tutorial provides a foundation for you to build your very own Elixir app that helps solve serious problems for your users.

Happy coding!

P.S. If you'd like to read Elixir Alchemy posts as soon as they get off the press, subscribe to our Elixir Alchemy newsletter and never miss a single post!

Setting Up a Multi-tenant Phoenix App for Elixir

Aestimo K. — Tue, 05 Dec 2023 11:18:16 +0000

Apps built with Elixir can support massive scalability, real-time interactivity, great fault tolerance, and the language's syntax is actually a joy to use. Elixir is a natural fit for applications such as chat apps, data dashboard apps, and anything needed to support a large userbase.

In this article, we'll use Elixir — specifically, the web framework Phoenix — to build a multi-tenant link shortening app.

Let's get started!

What's Needed for Our Phoenix Project Built with Elixir

With the growing popularity of social media, there's been an increase in the use of URL shortening services. As a budding entrepreneur, you believe there's a market for yet another URL shortening service. And since such an app lives in between a generated short URL and the long version, it's necessary that the app be fast and scalable.

These features and more are already built into Phoenix, which makes it a great choice for this project, enabling you to stand out in the market.

Takeaways

After this tutorial, you will:

Understand the internal workings of an Elixir/Phoenix app (for example, how to create and use a custom Ecto type, how to handle authentication, and more).
Understand the concept of multi-tenancy, including the different strategies available.
Build a multi-tenant link shortening Elixir/Phoenix app that can be extended into something more advanced with real-world use.

To follow along with the tutorial, let's set up a few things.

Prerequisites

Elixir installed on your development machine. If you don't have it installed, follow this guide. For this tutorial, we're using Elixir version 1.14.0.
Since we'll be using Phoenix for the web app build, you'll also need to have it installed on your local development machine. We're using Phoenix version 1.7.2 in this tutorial.
PostgreSQL - Ensure you have PostgreSQL installed, as it's the default database used by Elixir/Phoenix apps.

With that in place, let's see what kind of app we'll be building next.

An Overview of the Phoenix App We'll Build

The app that we'll build is a simple multi-tenant URL shortening app with the following features:

User authentication - users can register and log in to the application.
Multi-tenant separation - users can create shortened links within their own separate accounts.
URL shortening - users can input normal links and generate shortened versions that redirect back to the original link when clicked.
View metrics - users can see how many times a link was clicked.

Implementing these features should add to your experience in working with Elixir and provide a good foundation to build even more complex Elixir apps.

Next, let's understand what multi-tenancy is and how it works.

An Overview of Multi-tenancy

At the most basic level, a multi-tenant app is one where the same application and database serve several tenants, with each client's data kept separate from that of other clients. There are thousands of real-world multi-tenant app examples, including AppSignal, Gmail, and others.

In our app, a user will be able to register and create shortened links that are associated with their account. Other users will also be able to sign up and create shortened links associated to them, but users won't be able to see other users' links. This is a simplified but very good example of multi-tenancy in action.

In the next section, we'll outline the different strategies a developer can use when building a multi-tenant app. It's important to point out that these strategies aren't exclusive to Elixir. Rather, think of them as universal building blocks for developing multi-tenant apps regardless of the programming language used.

Multi-tenancy Strategies

Several multi-tenancy strategies are available, including:

Separate databases - Here, each tenant gets their own database for complete isolation of each client's data. The beauty of this approach is that it's very scalable and great for applications where data security and isolation is key: for example, patient medical records, financial records, and other similar use cases. As you can imagine, one of the biggest disadvantages with this approach is how complex and costly it is to build and maintain apps using this strategy.
A shared database with separate schema - In this strategy, each client gets a separate schema within a shared database. This approach allows for a scalable system that is not too complex to build and maintain. That said, having separate schemas for each client is not something you can easily use to handle massive scale.
A shared database with shared schema - Here, all tenants share a common database and schema. It is a great choice for low-to-medium traffic apps and offers a convenient way to get started with multi-tenancy. Many SaaS startups are often built using this strategy. The biggest disadvantage of this strategy is that it's not built for scale or speed.
Hybrid strategy - A not-so-common approach where you have both the shared database and shared schema for some groups of users (say, the ones that pay the least in a SaaS app), and a common database with a separate schema for premium customers. This approach offers some level of scaling but is very complex to build and maintain.
Containerization - Similar to the separate databases approach, here, each tenant is provided with a completely separate and isolated app container. The obvious advantages are speed and scalability, but this is complex to build and maintain.

You now have a good overview of what strategy to use when you build your own Elixir app next time.

For this app project, we'll be using the shared database with shared schema approach and an app stack described in the next section.

Our Elixir Application Stack

To build our app project, we will use the up-and-coming Elixir stack called "PETAL", short for:

P - Phoenix
E - Elixir
T - Tailwind CSS
A - Alpine JS
L - Liveview - LiveView is an Elixir library that you include as a dependency in a Phoenix app to handle interactivity and other real-time flows characteristic of single-page applications (SPAs).

Since our goals are simple, employing the full PETAL framework in this case is overkill. Instead, we'll use the simple stack you get when you generate a new Phoenix web application:

Phoenix
PostgreSQL
HTML and Tailwind CSS

We are now ready to jump into the build, but before we do, it's necessary that we understand how link shortening actually works. This will form the basis for the steps we'll take during the build.

How Link Shortening Works

Link shortening is actually very simple. The link shortening app lies between a long URL and a generated short URL. When a visitor hits a short URL, their request is received by the app and a quick database search is done to find the matching long URL. Once that long URL is found, the visitor is redirected to it, and the link visits counter updates.

Obviously, this is a very simplified outline, but it will suffice for now. Next, let's start off our build.

Generating a New Phoenix Application

Begin by generating a new Phoenix application using the following mix command:

mix phx.new urlbot

Then open your project directory in a text editor and edit the dev.exs file with your development environment's database settings:

# config/dev.exs

import Config

config :shorten_my_links_app, ShortenMyLinksApp.Repo,
  username: "DATABASE USERNAME", # Edit this line
  password: "DATABASE PASSWORD", # Edit this line
  hostname: "localhost",
  database: "DATABASE NAME", # Edit this line
  stacktrace: true,
  show_sensitive_data_on_connection_error: true,
  pool_size: 10

  # ...

Create the database for the application:

mix ecto.create

Once that is done, run mix phx.server in the terminal to compile the application and spin up an instance of the compiled app on localhost:4000, where you can see the default Phoenix home page:

Next up, let's set up the multi-tenant structure and user authentication.

Setting Up Multi-tenancy for Elixir

The tenant structure needs to be implemented as early as possible into a project build, since doing it later can result in all sorts of challenges. To put the shared database with shared schema multi-tenant strategy that we chose into actuality, we'll make use of foreign keys and related models.

Specifically, we'll have two main models: User and Account (this can also be called Organization, Team, or even Company); a User belongs to the Account model.

We'll also have a third Link model which will belong to the Account model. This way, the User model can be used exclusively for authentication purposes, while resource ownership will be handled by Account.

This structure is represented in the diagram below:

With this structure in place, all resources created, updated, or deleted in the app can be scoped to a specific Account and User. And just like that, we can achieve our goal of having separate tenant resources in a shared database with shared schema setup.

An additional benefit to using this structure is that you can easily expand it to invite other users to an account as "teammates" and assign them different roles. However, we won't cover this feature in this tutorial.

Let's begin by generating the Account context.

Generating an `Account` Context

Generate the Account context, schema, and migration file with the command below:

mix phx.gen.context Accounts Account accounts name:string

Then run the migration with:

mix ecto.migrate

Next, let's move on to the User.

Building a `User` Context and Authentication with Elixir

The User context will be used for authentication purposes. Instead of generating the User context and trying to integrate it into an authentication flow from scratch, Elixir gives us several libraries we can use, including phx.gen.auth, Coherence, and Pow, a modular and extendable authentication library for Phoenix apps.

We first add Pow to the project's dependencies:

# mix.exs

defp deps do
  [
    # ...
    {:pow, "~> 1.0.30"}
  ]
end

And fetch it:

mix deps.get

Then finish by installing it:

mix pow.install

With that, we get a User context, schema, and migration file. User authentication routes are also appended to the router.

At this point, we could run the migration, but there are a couple of changes we should make to our generated user files first to ensure they are properly related.

Adding the `User` and `Account` Relationship

Begin by adding a belongs_to association to User:

# lib/shorten_my_links_app/users/user.ex

defmodule Urlbot.Users.User do
  use Ecto.Schema
  use Pow.Ecto.Schema

  schema "users" do
    pow_user_fields()

    belongs_to :account, Urlbot.Accounts.Account # Add this line

    timestamps()
  end
end

Let's also modify Account to include the has_many relationship:

# lib/urlbot/accounts/account.ex

defmodule Urlbot.Accounts.Account do
  use Ecto.Schema
  import Ecto.Changeset

  schema "accounts" do
    field :name, :string

    has_many :users, Urlbot.Users.User # Add this line

    timestamps()
  end

  ...
end

Next, let's modify the users migration to add the account_id field as a foreign key. We also need to indicate that a user will be deleted whenever their related account is deleted:

# priv/repo/migrations/XXXXXXXXXX_create_users.exs

defmodule Urlbot.Repo.Migrations.CreateUsers do
  use Ecto.Migration

  def change do
    create table(:users) do
      add :email, :string, null: false
      add :password_hash, :string

      # Add this line
      add :account_id, references(:accounts, on_delete: :delete_all), null: false

      timestamps()
    end

    create index(:users, [:account_id]) # Add this line
    create unique_index(:users, [:email])
  end
end

Finally, it's nice to have an Account automatically created the first time a user registers on the app.

Creating an `Account` on User Registration

We need a way to capture the one attribute of the Acccount model (the account_name in the user registration form) and pass this attribute to a modified user creation process which will create a related Account for us.

That sounds like a lot, but let's go step-by-step.

First, add the account_name attribute to the user registration form. Since we are working with Pow which comes with pre-built view templates, we need to generate the templates by running the command:

mix pow.phoenix.gen.templates

This will generate Pow's view templates, but we are only interested in the user registration view for now. Edit it by adding the account_name field:

# lib/urlbot_web/controllers/pow/registration_html/new.html.heex

<div class="mx-auto max-w-sm">
  ...

  <.simple_form :let={f} for={@changeset} as={:user} action={@action} phx-update="ignore">
    ...

    <!-- Add this line -->
    <.input field={f[:account_name]} type="text" label="Account name" />

    ...
  </.simple_form>
</div>

By the way, instead of adding the account_name like this, we can use a nested form for working with Account from User (but this method should work just as well).

Next, we want to add account_name as a virtual attribute in the User schema. The reason it's a virtual attribute is simply because it's not an attribute that is built into the User schema, but we still need to use it in there. You can read more on virtual attributes.

# lib/urlbot/users/user.ex

defmodule Urlbot.Users.User do
  use Ecto.Schema
  use Pow.Ecto.Schema

  schema "users" do
    pow_user_fields()

    # Add this line
    field :account_name, :string, virtual: true

    belongs_to :account, Urlbot.Accounts.Account

    timestamps()
  end
end

At this point, you can run mix ecto.migrate to create the user table.

Next, we want to make sure that the virtual attribute we've just added is passed on to the User changeset, which we'll add since it's not included by default when we run the Pow generator.

We'll also add a custom private function to create an account at the same time a user is created:

# lib/shorten_my_links_app/users/user.ex

defmodule Urlbot.Users.User do
  use Ecto.Schema
  use Pow.Ecto.Schema

  # Add this line since we'll be using Ecto's changeset in this schema
  import Ecto.Changeset

  # Add this line to reference Account since it's used in the private function
  alias Urlbot.Accounts

  ...

  # Also add this block of code to add a changeset
  def changeset(user, attrs) do
    user
    |> pow_changeset(attrs)
    |> cast(attrs, [:account_name])
    |> validate_required([:account_name])
    |> create_user_account(user)
    |> assoc_constraint(:account)
  end

  # Add the custom private function
  defp create_user_account(%{valid?: true,  changes: %{account_name: account_name}} = changeset, %{account_id: nil} = _user) do
    with {:ok, account} <- Accounts.create_account(%{name: account_name}) do
      put_assoc(changeset, :account, account)
    else
      _ -> changeset
    end
  end

  defp create_user_account(changeset, _), do: changeset

end

Let's break down what's going on. First, we define a changeset block passing in a user and the user attributes. Then we cast the virtual attribute we added, followed by a validation rule to make sure it's present. We then call the private function create_user_account (defined just below the changeset block) and finalize with an assoc_constraint which checks that the associated field, account_id, exists.

With all that done, whenever a new user registers, a connected account will be automatically created. Any link resources created by the logged-in user will always be scoped to them. For example, an index view under /links will only display links created by the user, and not list other users' links, even if they are available. This resource separation at the account or user level is the foundation of a multi-tenant structure.

Up Next: Adding `Link` Resources and Custom Ecto Types

We've explored how multi-tenancy works and discussed a few multi-tenancy strategies. We created a Phoenix app and set up multi-tenancy. Finally, we added accounts, users and authentication, and associated users and accounts.

In the next and final part of this series, we'll look at:

Generating the link resource
Using Ecto custom types
Generating a current_account
Assigning a current_account to links
Redirecting links and updating the views counter

Until then, happy coding!

P.S. If you'd like to read Elixir Alchemy posts as soon as they get off the press, subscribe to our Elixir Alchemy newsletter and never miss a single post!

Advanced Usages of Action Policy for Ruby on Rails

Aestimo K. — Wed, 25 Oct 2023 09:36:18 +0000

In part one of this series, we looked at some basic usages of Action Policy. Now we'll leverage Action Policy for more advanced authorization use cases.

First up, let's explore applying pre-checks.

Pre-checks

Let's say we want users with "editor" status to have access to a post's show, update, and destroy actions, like so:

# app/policies/posts_policy.rb

class PostPolicy < ApplicationPolicy
    def show?
      user.editor || user.reader || user.id == record.user_id
    end

    def update?
      user.editor || (user.id == record.user_id)
    end

    def destroy?
      user.editor || user.id == record.user_id
    end
end

We can refactor this policy code using an Action Policy pre-check that extracts common rules into pre-checks:

# app/policies/posts_policy.rb

class PostPolicy < ApplicationPolicy
  pre-check :allow_editors

  def show?
    user.reader || user.id == record.user_id
  end

  def update?
    user.id == record.user_id
  end

  def destroy?
    user.id == record.user_id
  end

  private

  def allow_editors
    allow! if user.editor
  end
end

Scopes

Another Action Policy feature that deserves our attention is scoping. Scopes filter data depending on any authorization rules you've set.

Using our blog app as an example, let's say we want to apply the following rules to the posts index action:

List all posts for all users with the "editor" role
List only posts that a user has created if they have the "author" role

Without using any Action Policy authorization, the posts controller index action looks like any generic index action:

# app/controllers/posts_controller.rb

class PostsController < ApplicationController
  ...

  def index
    @posts = Post.all
  end

  ...
end

We'll need to utilize Action Policy scoping to refactor this action and apply the outlined access rules. Scoping rules are defined within a policy class and applied in the respective controller using the authorized_scope or authorized methods.

First modify the Post policy, like so:

# app/policies/posts_policy.rb

class PostPolicy < ApplicationPolicy
  ...

  relation_scope do |scope|
      if user.editor?
        scope
      end
    end
  end

  relation_scope(:own) do |scope|
      scope.where(user: user)
    end
  end

  ...
end

Then the posts controller's index action, as shown below:

# app/controllers/posts_controller.rb

class PostsController < ApplicationController

  ...

  def index
    @posts = authorized_scope(Post, type: :relation, as: :own)
  end
  ...

end

Caching in Action Policy for Ruby and Rails

Action Policy is a performant authorization library, partly thanks to its efficient use of caching.

It has several cache layers for you to leverage, including memoization and external caching.

Memoization

Consider a situation where the same rule is called several times on an object instance. For example, let's say we need to load all comments associated with a post within the index action:

# app/controllers/posts_controller.rb

class PostsController < ApplicationController

  ...

  def index
    @posts = authorized_scope(Post.includes(comments), ...)
  end
  ...

end

Now, imagine there's an "edit" link for every comment a post has within the index view. As you can see, this is a resource-intensive undertaking since we need to check if the current user is allowed to first access the post index, and then edit a post's comments. If a post has multiple comments, this would mean loading the authorization policy multiple times.

In a situation like this, Action Policy will re-use the required policy instance — specifically, the record.policy_cache_key — as one of the identifiers in the local store.

This kind of caching works well for short-lived requests, but if you need to cache resource-intensive rules that will be persisted across requests, using an external cache store is a better option.

Using an External Cache Store

If you need to run access control rules that utilize complex database queries, for example, you can use an external cache store such as Redis. Your rules cache will be made available across requests. You just need to remember to explicitly define which rules to cache within the policy class:

# app/policies/post_policy.rb

class PostPolicy < ApplicationPolicy
  cache :index?

  def show?
    # some complex database heavy rules...
  end

  ....
end

And configure the cache store for your app:

# config/application.rb

Rails.application.configure do |config|
  config.action_policy.cache_store = :redis_cache_store
end

Quick tip: There's a lot more to this. Dig into the Action Policy caching documentation for more information.

Aliases in Action Policy for Rails

An alias is an alternative way to name policies so that they make more sense.

Let's check out an example using our blog app. Consider this Post policy:

# app/policies/post_policy.rb

class PostPolicy < ApplicationPolicy
  alias_rule :destroy?, :update?, to: :manage_post?

  ...

  def manage_article?
    user.editor || user.id == record.user_id
  end

  ...
end

Here, we combine the update? and destroy? rules into one alias called manage_post?.

Then we use it in the controller, like so:

# app/controllers/posts_controller.rb

class PostsController < ApplicationController

  ...

  def update
    authorize! :manage_post?, @post
    ...
  end

  # DELETE /posts/1 or /posts/1.json
  def destroy
    authorize! :manage_post?, @post
    ...
  end
  ...
end

Finally, let's quickly look at how to handle unauthorized access.

Handling Unauthorized Access in a Ruby and Rails App

If a user tries to access a resource they shouldn't, an ActionPolicy::Unauthorized error is raised.

You need to explicitly handle this error in your app's ApplicationController, like so:

# app/controllers/application_controller.rb

class ApplicationController < ActionController::Base
  rescue_from ActionPolicy::Unauthorized do
    redirect_to root_path, alert: 'Access denied.'
  end
end

And that's it!

Wrapping Up

In this post, we explored advanced Action Policy features, including pre-checks, scopes, caching, aliases, and finally, handling unauthorized access.

From basic rules to complex conditional scenarios, Action Policy has everything you need to handle almost any authorization scenario. Use this library in your next project and see how powerful it is.

Happy coding!

P.S. If you'd like to read Ruby Magic posts as soon as they get off the press, subscribe to our Ruby Magic newsletter and never miss a single post!