DEV Community: Janko Marohnić

Passkey Authentication with Rodauth

Janko Marohnić — Mon, 24 Jul 2023 08:28:41 +0000

Passkeys are a modern alternative to passwords, where the user's device performs the authentication, usually requiring some form of user verification (biometric identification, PIN). Passkeys are built on top of WebAuthn specification, which is based on public-key cryptography. Keypairs are created for each website, and the public key is sent to the server, while the private key is securely stored on the device. This makes passkeys:

stronger than any password
safe from data breaches
safe from phishing attacks

WebAuthn credentials are bound to the device that created them (think security keys like YubiKey), which is good for privacy since no company has your data, but losing your device could lock you out of your account. Passkeys add the ability to be backed up to the cloud and synchronized between multiple devices, which reduces the risk of passkeys getting lost.

Rodauth provides first class support for passkeys, implemented on top of the excellent webauthn-ruby gem. It enables using passkeys as a multifactor authentication method, or for passwordless login and registration. In addition to routes, views and database storage, it also provides the complete JavaScript part that interacts with Web Authentication API for zero configuration.

In this article, I would like to show how to set each of these up in a Rails app that uses rodauth-rails. I'll be using Safari on macOS Ventura, and have iCloud Keychain sync enabled, which is a requirement for Apple passkeys.

Multifactor authentication

As I mentioned before, Rodauth supports registering passkeys as a multifactor authentication method, in addition to TOTP, recovery codes and SMS codes it already provides.

We'll start by creating the necessary database tables:

$ rails generate rodauth:migration webauthn
$ rails db:migrate

class CreateRodauthWebauthn < ActiveRecord::Migration
  def change
    # stores WebAuthn user identifiers
    create_table :account_webauthn_user_ids, id: false do |t|
      t.integer :id, primary_key: true
      t.foreign_key :accounts, column: :id
      t.string :webauthn_id, null: false
    end
    # stores WebAuthn credentials
    create_table :account_webauthn_keys, primary_key: [:account_id, :webauthn_id] do |t|
      t.references :account, foreign_key: true
      t.string :webauthn_id
      t.string :public_key, null: false
      t.integer :sign_count, null: false
      t.datetime :last_use, null: false, default: -> { "CURRENT_TIMESTAMP" }
    end
  end
end

Next, we'll enable the webauthn feature in our Rodauth configuration:

# app/misc/rodauth_main.rb
class RodauthMain < Rodauth::Rails::Auth
  configure do
    enable :webauthn
  end
end

This will add routes for setting up, authenticating via and removing passkeys:

$ rails rodauth:routes
# ...
# GET/POST  /webauthn-auth    rodauth.webauthn_auth_path
# GET/POST  /webauthn-setup   rodauth.webauthn_setup_path
# GET/POST  /webauthn-remove  rodauth.webauthn_remove_path
# ...

Now when the user navigates to the page for managing multifactor authentication methods, they should see a link for setting up WebAuthn authentication.

This page will show a button for registering a WebAuthn credential, which is already hooked up with the necessary JavaScript code, so clicking on it should show a native browser dialog for creating a new passkey. Once I verify biometric identification, my 2nd factor is set up.

When this user is logging in the next time, once they've authenticated with 1st factor, they're given the option to authenticate via a passkey for 2nd factor.

Just like for registering passkeys, the page for authenticating via a passkey is already hooked up with the necessary JavaScript code, so clicking on the submit button should show a dialog for authenticating with the previously created passkey. Once I verify biometric identification, I'm authenticated with 2nd factor.

Passwordless login

Once the user has created a passkey for your website, in addition to multifactor authentication, they can also use it for passwordless login. This functionality is provided by the webauthn_login feature:

# app/misc/rodauth_main.rb
class RodauthMain < Rodauth::Rails::Auth
  configure do
    enable :webauthn_login
  end
end

This will automatically enable multi-phase login, where the user first enters their email, and then they can choose to authenticate via a password or a passkey. Note that the password field won't be displayed if the user didn't set one when creating their account.

Verifying the passkey will log the user in. If they're using multifactor authentication, by default this will only authenticate 1st factor, so they'll still need 2nd factor (for pages that require it). However, passkeys are generally considered multi-factor authentication, because the user presents something they "have" (device) and – if user verification took place – something they "are" (biometrics) or "know" (PIN).

We can tell Rodauth to consider user verification as 2nd factor. If there was no user verification, e.g. a security key was used that only requires user presence but doesn't have biometrics or PIN, then only 1st factor will get authenticated.

# app/misc/rodauth_main.rb
class RodauthMain < Rodauth::Rails::Auth
  configure do
    webauthn_login_user_verification_additional_factor? true
  end
end

To make the login UX even better, WebAuthn protocol supports autofill UI for passkeys when the email field is focused. Rodauth once again has this built in via the webauthn_autofill feature (which happens to be a feature I added 😊):

# app/misc/rodauth_main.rb
class RodauthMain < Rodauth::Rails::Auth
  configure do
    enable :webauthn_autofill
  end
end

When the user opens the login page again and focuses the email field, they should see their passkey being offered. This is because stored passkeys include the user's email address for identification.

There is normally a drop shadow, but my Mac's screen capture removed it.

When the user selects their passkey and verifies it, they'll be automatically logged in, without even having to type their email address. In fact, they don't even have to select the passkey, they can just scan their fingerprint as soon as they focus the email field.

In addition to passwordless login, Rodauth also supports passwordless registration via the webauthn_verify_account feature. The user just needs to enter their email address in the create account form, and when they follow the link in the verification email, they're required to register a passkey in order to verify their account.

Multiple credentials

Rodauth supports registering multiple passkeys for a single account; the user can just visit the WebAuthn setup page again and create another credential. This is useful for people wanting to register multiple devices for backup.

By default, credentials can only be differentiated by their last used timestamp, which is what's displayed on the WebAuthn remove page by default.

Let's add the ability for the user to choose a nickname for their credentials, so that they can more easily differentiate between them. We'll start by adding a new nickname column to the account_webauthn_keys table:

$ rails generate migration add_nickname_to_account_webauthn_keys nickname:string
$ rails db:migrate

class AddNicknameToAccountWebauthnKeys < ActiveRecord::Migration
  def change
    add_column :account_webauthn_keys, :nickname, :string
  end
end

Next, we'll import the view templates used by the WebAuthn feature, and add a nickname field to the setup form:

$ rails generate rodauth:views webauthn
# create  app/views/rodauth/webauthn_auth.html.erb
# create  app/views/rodauth/webauthn_setup.html.erb
# create  app/views/rodauth/webauthn_remove.html.erb

<!-- app/views/rodauth/webauthn_setup.html.erb -->
<!-- ... -->
  <div class="form-group mb-3">
    <%= form.label :nickname, "Nickname", class: "form-label" %>
    <%= form.text_field :nickname, value: params[:nickname], class: "form-control #{"is-invalid" if rodauth.field_error("nickname")}", aria: ({ invalid: true, describedby: "nickname_error_message" } if rodauth.field_error("nickname")) %>
    <%= content_tag(:span, rodauth.field_error("nickname"), class: "invalid-feedback", id: "nickname_error_message") if rodauth.field_error("nickname") %>
  </div>
<!-- ... -->

Now we'll hook into the WebAuthn setup request, validate that the nickname param was filled in and persist it on the new credential:

# app/misc/rodauth_main.rb
class RodauthMain < Rodauth::Rails::Auth
  configure do
    before_webauthn_setup do
      throw_error_status(422, "nickname", "must be set") if param("nickname").empty?
    end
    webauthn_key_insert_hash do |credential|
      super(credential).merge(nickname: param("nickname"))
    end
  end
end

Finally, let's also modify the remove form to display nicknames instead of last used timestamps (we're using the Account#webauthn_keys association defined by rodauth-model):

<!-- app/views/rodauth/webauthn_remove.html.erb -->
<!-- ... -->
  <fieldset class="form-group mb-3">
    <% current_account.webauthn_keys.each do |webauthn_key| %>
      <div class="form-check">
        <%= form.radio_button rodauth.webauthn_remove_param, webauthn_key.webauthn_id, id: "webauthn-remove-#{webauthn_key.webauthn_id}", class: "form-check-input #{"is-invalid" if rodauth.field_error(rodauth.webauthn_remove_param)}", aria: ({ invalid: true, describedby: "webauthn_remove_error_message" } if rodauth.field_error(rodauth.webauthn_remove_param)) %>
        <%= form.label "webauthn-remove-#{webauthn_key.webauthn_id}", webauthn_key.nickname, class: "form-check-label" %>
        <%= content_tag(:span, rodauth.field_error(rodauth.webauthn_remove_param), class: "invalid-feedback", id: "webauthn_remove_error_message") if rodauth.field_error(rodauth.webauthn_remove_param) && webauthn_key == current_account.webauthn_keys.last %>
      </div>
    <% end %>
  </fieldset>
<!-- ... -->

JavaScript side

While the JavaScript for passkey registration & authentication that ships with Rodauth is convenient when getting started, sooner or later you'll probably want to customize it. The original Web Authentication API isn't very user-friendly, but the [@github/webauthn-json] package makes it really simple to use.

The following is the simplest functional implementation using Stimulus:

// app/javascript/controllers/webauthn_controller.js
import { Controller } from "@hotwired/stimulus"
import * as WebAuthnJSON from "@github/webauthn-json"

export default class extends Controller {
  static targets = ["result"]
  static values = { data: Object }

  connect() {
    if (!WebAuthnJSON.supported()) alert("WebAuthn is not supported")
  }

  async setup() {
    const result = await WebAuthnJSON.create({ publicKey: this.dataValue })

    this.resultTarget.value = JSON.stringify(result)
    this.element.requestSubmit()
  }

  async auth() {
    const result = await WebAuthnJSON.get({ publicKey: this.dataValue })

    this.resultTarget.value = JSON.stringify(result)
    this.element.requestSubmit()
  }
}

<!-- app/views/rodauth/webauthn_setup.html.erb -->
<% cred = rodauth.new_webauthn_credential %>

<%= form_with url: request.path, method: :post, data: { controller: "webauthn", webauthn_data_value: cred.as_json.to_json } do |form| %>
  <%= form.hidden_field rodauth.webauthn_setup_param, data: { webauthn_target: "result" } %>
  <%= form.hidden_field rodauth.webauthn_setup_challenge_param, value: cred.challenge %>
  <%= form.hidden_field rodauth.webauthn_setup_challenge_hmac_param, value: rodauth.compute_hmac(cred.challenge) %>

  <% if rodauth.two_factor_modifications_require_password? %>
    <div class="mb-3">
      <%= form.label "password", rodauth.password_label, class: "form-label" %>
      <%= form.password_field rodauth.password_param, autocomplete: rodauth.password_field_autocomplete_value, required: true, class: "form-control #{"is-invalid" if rodauth.field_error(rodauth.password_param)}" %>
      <%= content_tag(:span, rodauth.field_error(rodauth.password_param), class: "invalid-feedback") if rodauth.field_error(rodauth.password_param) %>
    </div>
  <% end %>

  <%= form.submit rodauth.webauthn_setup_button, class: "btn btn-primary", data: { action: "webauthn#setup:prevent" } %>
<% end %>

<!-- app/views/rodauth/webauthn_auth.html.erb -->
<% cred = rodauth.webauthn_credential_options_for_get %>

<%= form_with url: rodauth.webauthn_auth_form_path, method: :post, data: { controller: "webauthn", webauthn_data_value: cred.as_json.to_json } do |form| %>
  <%= form.hidden_field rodauth.webauthn_auth_param, data: { webauthn_target: "result" } %>
  <%= form.hidden_field rodauth.webauthn_auth_challenge_param, value: cred.challenge %>
  <%= form.hidden_field rodauth.webauthn_auth_challenge_hmac_param, value: rodauth.compute_hmac(cred.challenge) %>

  <%= form.hidden_field rodauth.login_param, value: params[rodauth.login_param] if rodauth.valid_login_entered? %>

  <%= form.submit rodauth.webauthn_auth_button, class: "btn btn-primary", data: { action: "webauthn#auth:prevent" } %>
<% end %>

The flow works in a way that the server first generates parameters for the Web Authentication API, then when user clicks on the submit button, a JavaScript call is made with those parameters to register/authenticate a passkey on the client device. Once the browser flow is finished, the JavaScript response is submitted with the form, where the server verifies it and handles the outcome (saving passkey information in the database, logging the user in etc).

Closing words

Passkeys still need wider support in browsers and operating systems before they can become mainstream, but they look very promising. I like that I can use devices I already have, as opposed to having to buy a separate piece of hardware such as a YubiKey. I also feel safer that passkeys are synced automatically, and it's convenient that I don't have to remember on which Apple device I created a passkey for a given website.

The fact that Rodauth provides such advanced support for passkeys with zero configuration shows that it's really keeping up with authentication trends. It also speaks to its incredibly flexible design, as passkeys can be combined with existing authentication methods, acting both as 1st and 2nd factor. The whole flow is highly configurable, as can be seen from the vast list of configuration methods.

Resources

WebAuthn.io
FIDO Alliance documentation
Passkeys.dev
Passkeys: What the Heck and Why? (CSS Tricks)
WebAuthn vs Passkeys (Passwordless.ID)

Upgrading from Selenium to Cuprite

Janko Marohnić — Sun, 04 Jun 2023 09:51:09 +0000

When I joined my current company, the system tests for our Rails app used Selenium as the Capybara driver. I didn't have good experiences with Selenium in the past, mostly it was tedious to have to keep chromedriver up-to-date with the auto-updating Chrome. In this project, I was frequently hitting maximum number of open file descriptors on my OS when running system tests, probably in combination with Spring. We're using the Webdrivers gem, and we also needed to ignore its download URLs in VCR and WebMock. But my primary issue was that the system tests just seemed kind of slow in general.

I stumbled across an episode of The Bike Shed podcast, where it was mentioned that Selenium can add considerable overhead, so I decided it was worth trying out Cuprite. For those not familiar, Cuprite is a Capybara driver that interacts with Chrome directly using CDP. This is in contrast to Selenium, which goes through chromedriver/geckodriver command-line tool.

I have to say I did not expect the performance improvements to be so significant. Running individual system tests was about 30-50% faster on my M1 MacBook Air, while our overall system test suite went from 9 minutes to 6 minutes, which is a 30% speedup. Just from changing the Capybara driver. For someone who cares about fast tests, this was a considerable win 🤘

Our Capybara configuration ended up being the following:

require 'capybara/rspec'
require 'capybara/cuprite'

Capybara.default_max_wait_time = 5
Capybara.disable_animation = true

RSpec.configure do |config|
  config.before(:each, type: :system) do
    driven_by(:cuprite, screen_size: [1440, 810], options: {
      js_errors: true,
      headless: %w[0 false].exclude?(ENV["HEADLESS"]),
      slowmo: ENV["SLOWMO"]&.to_f,
      process_timeout: 15,
      timeout: 10,
      browser_options: ENV["DOCKER"] ? { "no-sandbox" => nil } : {}
    })
  end

  config.filter_gems_from_backtrace("capybara", "cuprite", "ferrum")
end

Flaky test failures

While moving to Cuprite made tests faster, we started getting dozens of new flaky test failures. After looking at them more closely, I found that each failure was caused by improper waiting for JavaScript (we're using Hotwire). With Selenium, those issues were just masked, because the overhead was big enough that the race conditions never manifested.

Most failures were around clicking on a link and then waiting for text to appear, but that text was already present on the previous page, so Capybara didn't actually wait for the new page to get navigated to. In some cases we needed to manually scroll to elements before interacting with them, where Selenium appears to have automatically scrolled.

click_on "Some link"
# make sure this text was NOT present on the previous page
expect(page).to have_content("Some text")

Disabling CSS transitions

We're using Bootstrap, and I noticed that some modal clicks were failing. It turned out that due to CSS transitions used by Bootstrap modals, Capybara would sometimes attempt to click on a moving target. Since we don't actually need animations in tests, I was looking for a way to disable them. By sheer luck I discovered a handy Capybara configuration that takes care of this:

# disable CSS transitions and jQuery animations
Capybara.disable_animation = true

One problem is that our flash messages are set to fade away after 3 seconds, so this caused them not to disappear automatically. This was fine most of the time, but in some cases they were covering content we needed to interact with. To address this, I created a helper method that retrieves the flash message and immediatelly closes the alert:

def flash_message
  message = find(".flash").text.split("\n").last
  find(".flash .close").click # close alert
  message
end

# ...
click_on "Create Device"
expect(flash_message).to eq "Device was successfully created"

Disabling Turbo previews

In a previous project, I found that Turbo previews were the cause of one flaky test, so I disabled them by adding the following into <head> of the layout:

<!-- disable cached Turbo previews when navigating -->
<meta name="turbo-cache-control" content="no-preview">

Raising Stimulus errors

When a JavaScript errors occur inside Stimulus lifecycle callbacks or actions, they are caught by Stimulus and logged. This avoids an error from one Stimulus controller halting JavaScript execution and preventing other Stimulus controllers from being executed (see Sam Stephenson's explanation for more details).

While this is useful for production, in tests we want to get alerted when JavaScript errors occur. After configuring Cuprite to convert any JS errors into Ruby exceptions by setting js_errors: true, we overrode Stimulus application's error handler in tests to propagate errors:

import { Application } from "@hotwired/stimulus"

const application = Application.start()

// Works with Webpacker (in Vite we'd use `import.meta.env.MODE === "test"`).
if (process.env.RAILS_ENV === "test") {
  // propagate errors that happen inside Stimulus controllers
  application.handleError = (error, message, detail) => {
    throw error
  }
}

Precompiling assets

We were initially hitting timeout errors on CI from Cuprite on the first test. The reason was that Webpacker would compile assets on the first request. We tried increasing :process_timeout in Cuprite, but that didn't help.

Instead of letting Webpacker compile assets on-the-fly, we chose to precompile them in advance, which fixed the issue.

$ bundle exec rake assets:precompile
$ bundle exec rspec spec/system

However, when JS errors would occur, I noticed the JavaScript source was now minified. This not only made it more difficult to locate where the error occurred, but on CI the error message was completely absent. This is because the webpack:compile rake task defaults NODE_ENV to production. First I tried setting NODE_ENV=test, but that didn't skip minification, and I later found out this is intended for JavaScript tests. Setting NODE_ENV=development on CI worked, which is what's used for on-the-fly compilation.

I would prefer Webpacker not to merge assets into a single file in tests, but I think migrating to Vite would help with this.

Toggling headless mode

Cuprite runs Chrome in so-called "headless" mode by default, which means the browser doesn't open up while tests are being run. However, if you're debugging a failing test, it's not always enough to look at the captured screenshots, sometimes you need to see what's happening on the page.

Our Cuprite configuration allowed us to pass HEADLESS=0 environment variable to the rspec command to disable headless mode. If the interaction was happening too fast to make sense of anything, we could additionally set e.g. SLOWMO=0.5 to add 0.5s of overhead to every click.

$ HEADLESS=0 SLOWMO=0.5 bin/rspec spec/system/something_spec.rb

Docker handling

Some team members prefer to run the Rails app locally through Docker. To make Cuprite work, we set DOCKER=true in our docker-compose.yml, and then based on that environment variable passed the no-sandbox option to Chrome.

Closing words

The performance benefits alone will definitely make me advocate for using Cuprite every time. The only feature I found it doesn't support yet is drag-and-drop, though initial support has been merged to master.

Flaky test failures have always been a challenge for me when writing system tests. What helped me is to trust that the root cause is always figureoutable. I would previously attribute them to complex Selenium internals, but Cuprite is much less magical in comparison, so I can make better sense of why something is happening.

Social Login in Rails with Rodauth

Janko Marohnić — Tue, 06 Dec 2022 10:00:24 +0000

OmniAuth provides a standardized interface for authenticating with various external providers. Once the user authenticates with the provider, it's up to us developers to handle the callback and implement actual login and registration into the app. There is a wiki page laying out various scenarios that need to be handled if you want to support multiple providers, showing that it's by no means a trivial task.

While Devise provides a convenience layer around OmniAuth, it does nothing to actually sign the user into your app. When I started writing the OmniAuth integration for Rodauth, I wanted to go one step further and actually handle things like persistence of external identities, account creation and login, while still allowing the developer to customize the behaviour. That's how rodauth-omniauth was created. ✨

In this article, I will show how to add social login to an existing Rails app that's using Rodauth, and extend the default behaviour with some custom logic. If you're looking to get started with Rodauth, check out my previous article. With that out of the way, let's dive in.

Setup

We'll start by installing the Rodauth extension and the desired OmniAuth strategies:

$ bundle add rodauth-omniauth omniauth-facebook omniauth-google_oauth2

You don't need to install any gems for CSRF protection of OmniAuth request endpoints, because rodauth-omniauth will automatically use whichever CSRF protection mechanism Rodauth was configured with, which in case of Rails will be ActionController::RequestForgeryProtection.

If you haven't already, create the necessary OAuth apps, and configure the callback URL to be https://localhost:3000/auth/{provider}/callback, where {provider} is either facebook or google. We'll save the credentials for the OAuth apps into our project:

$ rails credentials:edit

# ...
facebook:
  app_id: "<YOUR_APP_ID>"
  app_secret: "<YOUR_APP_SECRET>"
google:
  client_id: "<YOUR_CLIENT_ID>"
  client_secret: "<YOUR_CLIENT_SECRET>"

Next, we'll need to create the table that rodauth-omniauth will use for storing external identities:

$ rails generate migration create_account_identities
$ rails db:migrate

class CreateAccountIdentities < ActiveRecord::Migration
  def change
    create_table :account_identities do |t|
      t.references :account, null: false, foreign_key: { on_delete: :cascade }
      t.string :provider, null: false
      t.string :uid, null: false
      t.index [:provider, :uid], unique: true
    end
  end
end

In the Rodauth configuration, we can now enable the omniauth feature and register our strategies:

# app/misc/rodauth_main.rb
class RodauthMain < Rodauth::Rails::Auth
  configure do
    enable :omniauth

    omniauth_provider :facebook,
      Rails.application.credentials.facebook[:app_id],
      Rails.application.credentials.facebook[:app_secret],
      scope: "email"

    omniauth_provider :google_oauth2,
      Rails.application.credentials.google[:client_id],
      Rails.application.credentials.google[:client_secret],
      name: :google # rename it from "google_oauth2"
  end
end

Finally, we can import the view templates for the login form, and add the social login links there:

$ rails generate rodauth:views login
#  create  app/views/rodauth/_login_form.html.erb
#  create  app/views/rodauth/_login_form_footer.html.erb
#  create  app/views/rodauth/_login_form_footer.html.erb
#  create  app/views/rodauth/_login_form_header.html.erb
#  create  app/views/rodauth/login.html.erb
#  create  app/views/rodauth/multi_phase_login.html.erb

<!-- app/views/rodauth/_login_form_footer.html.erb -->
<!-- ... -->
  <li>
    <%= button_to "Login via Facebook", rodauth.omniauth_request_path(:facebook),
      method: :post, data: { turbo: false }, class: "btn btn-link p-0" %>
  </li>
  <li>
    <%= button_to "Login via Google", rodauth.omniauth_request_path(:google),
      method: :post, data: { turbo: false }, class: "btn btn-link p-0" %>
  </li>
<!-- ... -->

We're using POST form submits, because OmniAuth doesn't allow GET requests for the request phase anymore by default. You'll notice we had to disable Turbo for the request links, as those redirect to an external authorize URL, which don't support AJAX visits.

Some OAuth authorizations require that the web app is served over HTTPS. Assuming you're using Puma, a quick way to enable this locally would be to install the localhost gem, and tell the Rails server you want to use SSL:

$ bundle add localhost --group development
$ rails server -b ssl://localhost:3000

Now you should be able to open https://localhost:3000/login, and see the Rodauth login page with social login links.

Login & Registration

When we visit the Facebook login link, and authorize the OAuth app, upon returning to the app a new verified account with your Facebook email address should be automatically created, along with the external identity, and you should be logged in.

You should see something like this in the database:

account = Account.last
#=> #<Account id: 123, status: "verified", email: "janko.marohnic@gmail.com">
account.identities
#=> [#<Account::Identity id: 456, account_id: 123, provider: "facebook", uid: "350872771">]

A problem I often experienced as a user was forgetting which social provider I initially logged into on a certain app, and whether I had even logged in with a social provider (though I can now easily determine the latter by checking my password manager). If I would sign in with the wrong provider, this would usually result in a new account being created for me.

Wouldn't it be nice if the app could detect that the email address of my existing account matches the one of the external identity, and automatically assign that identity to the existing account? Well, rodauth-omniauth does that automatically. If I were to authenticate with Google the next time, I would get logged into my existing account, with the new Google identity connected to it.

account.identities #=> [
#   #<Account::Identity id: 456, account_id: 123, provider: "facebook", uid: "350872771">,
#   #<Account::Identity id: 789, account_id: 123, provider: "google", uid: "987349876343">,
# ]

If for whatever reason you want to change or disable this behaviour, you can override account_from_omniauth, which is what searches existing accounts when authenticating with a provider for the first time:

class RodauthMain < Rodauth::Rails::Auth
  configure do
    # ...
    account_from_omniauth do
      # this is roughly the default implementation
      account_table_ds.first(email: omniauth_info["email"])
    end
    # OR
    account_from_omniauth {} # new identity = new account
  end
end

Storing additional data

Let's say users in our app can fill in their full name, and we decided to inherit it from their external identity when possible, to save them extra work.

We'll assume we have a separate profiles table associated to accounts, and we're already creating a profile record on normal registration:

# in a migration:
create_table :profiles do |t|
  t.references :account, null: false, foreign_key: true
  t.string :name
  t.timestamps
end

class RodauthMain < Rodauth::Rails::Auth
  configure do
    # create profile after normal registration
    after_create_account { Profile.create!(account_id: account_id) }
  end
end

We can override the hook for creating the account via OmniAuth login, and create the profile with the name prefilled:

class RodauthMain < Rodauth::Rails::Auth
  configure do
    # create profile after registration through OmniAuth login
    after_omniauth_create_account do
      Profile.create!(account_id: account_id, name: omniauth_info["name"])
    end
  end
end

Let's say we now want to store additional data on identities, which by default have only the mandatory provider and uid columns. We might want to store created_at & updated_at timestamps, as well as an email for each identity. We can start by creating the necessary columns:

# in a migration:
add_timestamps :account_identities
add_column :account_identities, :email, :string

We can now ensure created_at is set the first time we authenticate with a provider, and that updated_at and email are updated each time we authenticate with the provider:

class RodauthMain < Rodauth::Rails::Auth
  configure do
    # store `created_at` only when the identity is created
    omniauth_identity_insert_hash do
      super().merge(created_at: Time.now)
    end
    # update `updated_at` and `email` each time the identity is updated
    omniauth_identity_update_hash do
      super().merge(updated_at: Time.now, email: omniauth_email)
    end
  end
end

Now our account & identity data might look as follows:

account = Account.last
#=> #<Account id: 123,
#    status: "verified",
#    email: "janko@hey.com",
#    name: "Janko Marohnić">

account.identities.first
#=> #<Account::Identity
#    id: 789,
#    account_id: 123,
#    provider: "google",
#    uid: "987349876343",
#    email: "janko.marohnic@gmail.com",
#    created_at: Fri, 11 Nov 2022 13:11:85 UTC,
#    updated_at: Fri, 02 Dec 2022 08:01:26 UTC>

Multiple account types

If your app has different account types which require potentially different authentication rules, you'll be glad to know that rodauth-omniauth supports distinct configurations.

Let's say we have an admin account type, for which we want to provide logging in via GitHub. Assuming we've already created the OAuth app, we'll install the OmniAuth strategy gem:

$ bundle add omniauth-github
$ rails credentials:edit

# ...
github:
  client_id: "<YOUR_CLIENT_ID>"
  client_secret: "<YOUR_CLIENT_SECRET>"

To protect the admin section, we'll only allow users that are members of the company's GitHub organization. For this, we might have the following Rodauth configuration:

# app/misc/rodauth_admin.rb
class RodauthAdmin < Rodauth::Rails::Auth
  configure do
    enable :omniauth

    prefix "/admin"
    session_key_prefix "admin_"

    omniauth_provider :github,
      Rails.application.credentials.github[:client_id],
      Rails.application.credentials.github[:client_secret]

    before_omniauth_callback_route do
      if omniauth_provider == :github && !organization_member?(omniauth_info["nickname"])
        set_redirect_error_flash "User is not a member of our GitHub organization"
        redirect "/admin"
      end
    end
  end

  private

  def organization_member?(username)
    # ... check if user is a member of company's GitHub organization ...
  end
end

Since we've set admin routes to be prefixed with /admin, OmniAuth routes will be prefixed as well, so the request phase will be at /admin/auth/github. This ensures authentication doesn't overlap with the main account type.

If we had decided to use a separate table for admin accounts (e.g. admins), we can also use a separate identities table:

# in a migration:
create_table :admin_identities do |t|
  t.references :admin, null: false, foreign_key: { on_delete: :cascade }
  t.string :provider, null: false
  t.string :uid, null: false
  t.index [:provider, :uid], unique: true
end

class RodauthAdmin < Rodauth::Rails::Auth
  configure do
    # ...
    omniauth_identities_table :admin_identities
    omniauth_identities_account_id_column :admin_id
  end
end

Future work

Separate registration step

The current automatic registration assumes the external login will always return the user's email address, which is not always the case (hello Twitter). It also assumes we don't need additional information from the user before creating their account.

After external login, I would like to support having a separate registration step, with fields like email address already being filled in. The main challenge is preventing an attacker from signing up with another person's identity. I would definitely welcome any contributions. 🙏

Connecting additional identities

Once the user is signed in, it would be useful to allow them to connect additional external identities, as well as disconnect already linked identities, to make future logins more reliable.

GitLab account interface

This feature seems more straightforward to implement. However, it's tricky to handle the scenario when a logged in user wants to authenticate via a different provider, because by default Rodauth doesn't disallow access to the login page in this case. There are also questions around connecting identities that are currently assigned to a different account.

Closing words

This was definitely the most challenging Rodauth extension I've built. I had been working on it periodically for 2 years, and was only able to release it once I decided to postpone the mentioned features. It took a while to find the right balance between respecting OmniAuth configuration and Rodauth conventions, support JSON API with JWT, handle inheritance, figure out the OmniAuth 2.0 upgrade, and implement a customizable callback phase.

I was able to do all this thanks to the strong foundation Rodauth provides. Its layered design allowed me to hook at the right level to make it work well with other authentication features, while the configuration DSL made it easy to make any part customizable. Because Rodauth supports feature dependencies, I was able to extract the pure OmniAuth integration for standalone usage, for those who don't need any of the database logic.

While it was previously possible to use OmniAuth directly, I'm happy that Rodauth now has a social login story. Given that this is a much more integrated solution compared to what Devise offers, and other people might have more custom authentication flows, I'm curious to get everyone's feedback. 😉

What It Took to Build a Rails Integration for Rodauth

Janko Marohnić — Wed, 12 Oct 2022 14:23:57 +0000

When Rodauth came out, I was excited to finally have a full-featured authentication framework that wasn't tied to Rails, given that existing solutions required either Rails (Devise, Sorcery), or at least Active Record (Authlogic). Even though I mainly develop in Rails, I want other Ruby web frameworks to be viable alternatives, so I'm naturally drawn to generic solutions that everyone can use.

Even though Rodauth is built on top of Roda and Sequel, it can work as a Rack middleware in any Ruby web framework. In the beginning, there was a demo app showing how Rodauth can be used in Rails, which leveraged the (now discontinued) roda-rails gem. However, the integration felt fairly raw, and definitely lacked the ergonomics Rails developers are used to.

Rodauth has a vast feature set, but if it was going to compete with other authentication solutions, it needed to match their level of convenience in context of Rails. That meant deeply integrating into the Rails framework, having clear drawers where code goes, and defaults that are easy to get started with. At the start of 2020, I set on a mission to make using Rodauth in Rails easy.

Initial spike

I first created a demo Rails app, and started setting up Rodauth there. In the early iterations, I managed to hook up view rendering, flash messages, CSRF protection, and email delivery to use Rails instead of Roda, with significantly less code compared to roda-rails. I also managed to make Rodauth code reloadable by inserting a proxy Rack middleware that just calls the Roda app.

# app/misc/rodauth_app.rb
class RodauthApp < Roda
  plugin :rodauth, csrf: false, flash: false do
    enable :rails, :login, :create_account, :verify_account, :reset_password, :logout
    # ... rodauth configuration ...
  end

  route do |r|
    r.rodauth # handle rodauth requests
  end
end

# lib/rodauth/features/rails.rb
module Rodauth
  Feature.define(:rails, :Rails) do
    # ... rails integration ...
  end
end

# config/initializers/rodauth.rb
class RodauthMiddleware
  def initialize(app)
    @app = app
  end

  def call(env)
    RodauthApp.new(@app).call(env) # keeps RodauthApp reloadable
  end
end

Rails.application.config.middleware.use RodauthMiddleware

Once I felt things were functioning well enough, I extracted the glue code into the rodauth-rails gem and added tests. I also included an install generator, which created the initial skeleton with sensible default configuration. A new Roda superclass provided a convenience configure method for loading the Rodauth plugin together with the rails feature.

$ bundle add rodauth-rails
$ rails generate rodauth:install

# app/misc/rodauth_app.rb
class RodauthApp < Rodauth::Rails::App
  configure do # automatically loads the rails feature
    enable :login, :create_account, :verify_account, :reset_password, :logout
    # ... rodauth configuration ...
  end

  route do |r|
    r.rodauth # handle rodauth requests
  end
end

By default, Rodauth uses database authentication functions for password matching, which coupled with a two-user database setup allows protecting password hashes even in case of SQL injection (read here on how this works). However, I felt this complexity could be daunting when getting started, so I changed the default to do password matching in ruby instead.

use_database_authentication_functions? false
account_password_hash_column :password_hash

Rodauth also optionally uses HMACs for signing tokens, providing additional security. Since this is quite important, rodauth-rails turns this on by setting the HMAC secret to the Rails' secret key base.

hmac_secret { Rails.application.secret_key_base }

Active Record

While Rodauth uses Sequel for database interaction, most Rails apps are using Active Record. This meant Rodauth needed to work seamlessly alongside Active Record.

One idea was to develop a Rodauth extension that replaces all Sequel calls with Active Record code. However, given Rodauth's advanced SQL usage, that would've been a monumental effort. It would also have been a huge maintenance burden, since the extension would break with new Rodauth changes, and 3rd-party Rodauth extensions would need to maintain their own Active Record integrations.

So, the initial integration simply connected Sequel to the same database Active Record was connected to, which was then picked up by Rodauth.

db_config = ActiveRecord::Base.connection_db_config

Sequel.connect(adapter: db_config.adapter, database: db_config.database)

However, I noticed that this breaks features such as maintaining test schema, which requires temporarily disconnecting from the database in order to re-create the test database; even though Active Record connections were closed, Sequel was still holding an open connection, which was blocking database dropping. To address this, I extended Active Record to connect & disconnect Sequel in lockstep with Active Record.

This worked, but I quickly encountered a new kind of problem. Because Active Record and Sequel used separate database connections, Active Record code couldn't reference records created within a Sequel transaction, because to that connection the record simply doesn't exist (remember, database transactions are tied to a connection).

class Profile < ActiveRecord::Base
  belongs_to :account
end

class RodauthApp < Rodauth::Rails::App
  configure do
    # we're still in a Sequel transaction that created the account record
    after_create_account do
      # creating an associated record with Sequel worked:
      # db[:profiles].insert(account_id: account_id, name: "New User")

      # but with Active Record it failed with a foreign key constraint violation:
      Profile.create!(account_id: account_id, name: "New User") # ~> account record not found
    end
  end
end

My goal was for developers not to have to care that Rodauth uses Sequel, so calling Active Record inside Rodauth should just work. Moreover, Bruno Sutic warned me if Sequel uses a separate database connection, it would mean production databases would have up to twice as many open connections. It became apparent this approach could never achieve the desired developer experience.

Just browsed the repo. Sequel connection is a bummer.
— Josef Strzibny (@strzibnyj) April 23, 2020

For the integration to work, I would need to make Sequel reuse Active Record's database connection. I discussed this idea with Jeremy Evans (the lead Sequel maintainer), and he provided me with some guidance, thanks to which I was able to come up a solution. It was a Sequel extension that retrieved Active Record connections, kept transaction state and callbacks synchronized between Sequel and Active Record, integrated SQL instrumentation, and reconciliated adapter differences (see my previous article for more details).

$ bundle add sequel-activerecord_connection

DB = Sequel.postgres(extensions: :activerecord_connection)
DB[:accounts].all # uses Active Record's database connection

Model

Unlike Devise or Sorcery, Rodauth is completely decoupled from models, and any calls need to go through the Rodauth object. If you need to perform Rodauth actions outside of an HTTP request, you can use the internal request feature, which makes an actual Rack call to the Rodauth app:

# performing rodauth actions (class level)
RodauthApp.rodauth.create_account(login: "user@example.com", password: "secret123")
RodauthApp.rodauth.verify_account(account_login: "user@example.com")

# calling rodauth methods (instance level)
rodauth = Rodauth::Rails.rodauth(account_login: "user@example.com")
rodauth.get_password_reset_key(account_id) #=> "DS6dtRNnvzSCWzm8jg4lltOzBE5vTN_xflNdToIPw3A"
rodauth.recovery_codes #=> ["30GRJkr1BheZztvFZcDeRSNy6yhzigXH6zB-yvzP4Io", ...]

While I love this decoupling, it would still be nice to be able to at least create accounts and retrieve associations directly through the model. So, I created the rodauth-model gem, which provides an interface similar to Active Record's has_secure_password, and defines associations based on your Rodauth configuration (together with associated models).

class Account < ActiveRecord::Base
  include Rodauth::Rails.model
end

# generating a password hash
account = Account.create(email: "user@example.com", password: "secret123")
account.password_hash #=> "$2a$12$k/Ub1I2iomi84RacqY89Hu4.M0vK7klRnRtzorDyvOkVI.hKhkNw."

account.password_reset_key #=> #<Account::PasswordResetKey id: 1, key: "DS6dtRNnvzSCWzm8jg4lltOzBE5vTN_xflNdToIPw3A" ...>
account.recovery_codes #=> [#<Account::RecoveryCode id: 1, code: "30GRJkr1BheZztvFZcDeRSNy6yhzigXH6zB-yvzP4Io">, ...]

Rodauth stores account statuses (unverified, verified, closed) as integers, but we can use ActiveRecord::Enum to map them to strings, which is what the install generator now does.

class Account < ActiveRecord::Base
  # ...
  enum :status, unverified: 1, verified: 2, closed: 3
end

account = Account.find(123)
account.status #=> "unverified"

account.verified? #=> false
account.status = "verified"
account.verified? #=> true

Account.closed #=> [#<Account id: 456, status: "closed" ...>, ...]

Routes introspection

In this architecture, Rodauth routes are handled by the Rack middleware that's sitting in front of the Rails router. This has the benefit of allowing you to perform authentication actions such as requiring authentication, checking active sessions, and remembering from cookie in a single place, thus better encapsulating authentication logic.

However, since Rodauth routes aren't registered within the Rails router, they don't show up in rails routes. Rails' routes introspection doesn't currently have capability of registering custom routes, and Roda's routing is dynamic, so it's not possible to retrieve its routes programmatically.

Eventually I managed to implement a rodauth:routes Rake task, which retrieves the route paths from the Rodauth app, and parses the source code for HTTP verbs. It's not ideal, but it should be good enough.

$ rails rodauth:routes
# Routes handled by RodauthApp:
# 
#   GET/POST  /login                   rodauth.login_path
#   GET/POST  /create-account          rodauth.create_account_path
#   POST      /email-auth-request      rodauth.email_auth_request_path
#   GET/POST  /email-auth              rodauth.email_auth_path
#   GET/POST  /logout                  rodauth.logout_path
#   GET/POST  /reset-password-request  rodauth.reset_password_request_path
#   GET/POST  /reset-password          rodauth.reset_password_path
#   GET/POST  /change-password         rodauth.change_password_path
#   GET/POST  /change-login            rodauth.change_login_path
#   GET/POST  /verify-login-change     rodauth.verify_login_change_path
#   GET/POST  /close-account           rodauth.close_account_path
#   GET/POST  /verify-account-resend   rodauth.verify_account_resend_path
#   GET/POST  /verify-account          rodauth.verify_account_path
# 
#   GET       /admin/multifactor-manage      rodauth(:admin).two_factor_manage_path
#   GET       /admin/multifactor-auth        rodauth(:admin).two_factor_auth_path
#   GET/POST  /admin/multifactor-disable     rodauth(:admin).two_factor_disable_path
#   GET/POST  /admin/otp-auth                rodauth(:admin).otp_auth_path
#   GET/POST  /admin/otp-setup               rodauth(:admin).otp_setup_path
#   GET/POST  /admin/otp-disable             rodauth(:admin).otp_disable_path
#   GET/POST  /admin/recovery-auth           rodauth(:admin).recovery_auth_path
#   GET/POST  /admin/recovery-codes          rodauth(:admin).recovery_codes_path
#   POST      /admin/unlock-account-request  rodauth(:admin).unlock_account_request_path
#   GET/POST  /admin/unlock-account          rodauth(:admin).unlock_account_path

Generators

Before working on this gem, I didn't realize how important code generators are for convenience, and also how difficult they are to get right. There are currently three generators rodauth-rails ships with:

rodauth:install – sets up initial skeleton with default auth features and migrations
rodauth:views – imports ERB view templates for customization
rodauth:migration – generates migrations for selected authentication features

The generators needed to handle various scenarios, such as RSpec vs Minitest, fixtures vs factory_bot, Active Record vs Sequel, different SQL adapters, API-only mode, UUID primary keys, and more.

Schema migrations

I wanted to remove any Sequel code from sight, so I translated Sequel migrations into Active Record code, and generated those in Active Record migrations. If Sequel happens to be used as the main database library, we'd generate Sequel migrations instead.

$ rails generate rodauth:migration otp active_sessions
# create  db/migrate/20221012110706_create_rodauth_otp_active_sessions.rb

class CreateRodauthOtpActiveSessions < ActiveRecord::Migration[7.0]
  def change
    # Used by the otp feature
    create_table :account_otp_keys do |t|
      t.foreign_key :accounts, column: :id
      t.string :key, null: false
      t.integer :num_failures, null: false, default: 0
      t.datetime :last_use, null: false, default: -> { "CURRENT_TIMESTAMP" }
    end

    # Used by the active sessions feature
    create_table :account_active_session_keys, primary_key: [:account_id, :session_id] do |t|
      t.references :account, foreign_key: true
      t.string :session_id
      t.datetime :created_at, null: false, default: -> { "CURRENT_TIMESTAMP" }
      t.datetime :last_use, null: false, default: -> { "CURRENT_TIMESTAMP" }
    end
  end
end

View templates

The built-in view templates use Tilt's interpolated string engine, which avoids ERB dependency, but requires work to adapt for Rails. So, rodauth-rails' views generator imports already converted ERB view templates that use familiar Rails' form helpers.

$ rails generate rodauth:views login create_account lockout 
# create  app/views/rodauth/_login_form.html.erb
# create  app/views/rodauth/_login_form_footer.html.erb
# create  app/views/rodauth/_login_form_header.html.erb
# create  app/views/rodauth/login.html.erb
# create  app/views/rodauth/multi_phase_login.html.erb
# create  app/views/rodauth/create_account.html.erb
# create  app/views/rodauth/unlock_account_request.html.erb
# create  app/views/rodauth/unlock_account.html.erb

<%= form_with url: rodauth.unlock_account_path, method: :post, data: { turbo: false } do |form| %>
  <%== rodauth.unlock_account_explanatory_text %>

  <% if rodauth.unlock_account_requires_password? %>
    <div class="mb-3">
      <%= form.label "password", rodauth.password_label, class: "form-label" %>
      <%= form.password_field rodauth.password_param, value: "", id: "password", autocomplete: rodauth.password_field_autocomplete_value, required: true, class: "form-control #{"is-invalid" if rodauth.field_error(rodauth.password_param)}", aria: ({ invalid: true, describedby: "password_error_message" } if rodauth.field_error(rodauth.password_param)) %>
      <%= content_tag(:span, rodauth.field_error(rodauth.password_param), class: "invalid-feedback", id: "password_error_message") if rodauth.field_error(rodauth.password_param) %>
    </div>
  <% end %>

  <div class="mb-3">
    <%= form.submit rodauth.unlock_account_button, class: "btn btn-primary" %>
  </div>
<% end %>

Because not all Rodauth actions are Turbo-compatible (multi-phase login and viewing recovery codes return a 200 response on form submits), I have chosen to disable Turbo by default for all HTML forms, to ensure everything works.

Future plans

I would like the migration generator to support database authentication functions, to make it easier for developers to better secure their password hashes. I have been working on it on & off, but it's fairly complex to generate correct migration code, especially since different SQL databases (PostgreSQL, MySQL, SQL Server) require different setups.
Default Rodauth view templates use Bootstrap markup, but Ben Koshy has been working on adding Tailwind CSS support, which will be a nice addition. You'll be able to pass --css=tailwind, which will be the default when the tailwindcss-rails gem is used.
I want to make it easier for 3rd-party Rodauth extensions to provide migrations/views for Rails generators. The rodauth-oauth gem currently provides its own generators (rodauth:oauth:install and rodauth:oauth:views), but it would be nice if they didn't have to be duplicated.
Rodauth methods are called through the Rodauth object, and you're encouraged to define convenience controller/view helpers that suit your application's needs. That being said, having some default helpers like Devise has (and even something like Devise groups) probably might go a long way. I was also considering more convenient routing helpers, perhaps even a builder for defining custom ones.

  # config/routes.rb
  Rails.application.routes.draw do
    # this is what we have today
    constraints Rodauth::Rails.authenticated do
      # ...
    end

    # but maybe Devise syntax is worthwhile
    authenticate do
      # ...
    end
  end

Closing words

There are many more things the Rails integration takes care of, such as ignoring asset requests from Sprockets or Propshaft, instrumentation/logging of Rodauth requests, executing controller callbacks & rescue handlers, handling background emails, testing helpers, and Rails 4.2+ & JRuby support. But I think I rambled on for long enough.

The purpose for this article wasn't to bolster on my achievements, but to share my realization about the price of convenience, and how much work it can actually take integrate a generic library into Rails, especially when it does the work of a Rails engine. I cannot thank Jeremy Evans enough for discussing, reviewing, and merging every one of my additions to Rodauth that helped enable a smooth Rails integration 🙏

I hope I fulfilled my goal of making Rodauth easy to work with in Rails. That being said, I'm grateful for the continuous feedback I'm getting from people that are using Rodauth. I'm trying to convert many of my answers into a wiki page, a how-to guide or a screencast, to grow the knowledge around handling common use cases.

Anything I Want With Sequel And Postgres

Janko Marohnić — Mon, 29 Mar 2021 15:11:37 +0000

At work I was tasked to migrate our time-series analytics data from CSV file dumps that we've been feeding into Power BI to a dedicated database. Our Rails app's primary database is currently MariaDB, but we wanted to have our analytics data in a separate database either way, so this was a good opportunity to use Postgres which we're most comfortable with anyway.

We're using Active Record for interaction with our primary database, which gained support for multiple databases in version 6.0. However, given that we expected the queries to our analytics database would be fairly complex, and that we'd probably need to be retrieving large quantities of time-series data (which could be performance-sensitive), I decided it would be a good opportunity to use Sequel instead.

Thanks to Sequel's advanced Postgres support, I was able to utilize many cool Postgres features that helped me implement this task efficiently. Since not all of these features are common, I wanted to showcase them in this article, and at the same time demonstrate what Sequel is capable of. 🤘

Table partitioning

I mentioned that our analytics data is time-series, which means that we're storing snapshots of our product data for each day. This results in a large number of new records every day, so in order to keep query performance at acceptable levels, I've decided to try out Postgres' table partitioning feature for the first time.

What this feature does is allow you to split data that you would otherwise have in a single table into multiple tables ("partitions") based on certain conditions. These conditions most commonly specify a range or list of column values, though you can also partition based on hash values. Postgres' query planner then determines which partitions it needs to read from (or write to) based on the SQL query. This can drammatically improve performance for queries where most partitions have been filtered out during the query planning phase.

Sequel supports Postgres' table partitioning out-of-the-box. In order to create a partitioned table (i.e. a table we can create partitions of), we need to specify the column(s) we want to partition by (:partition_by), as well as the type of partitioning (:partition_type). In our app, we wanted to have monthly partitions of product data for each client, so our schema migration contained the following table definition:

create_table :products, partition_by: [:instance_id, :date], partition_type: :range do
  Date    :date,        null: false
  Integer :instance_id, null: false # in our app "instances" are e-shops
  String  :product_id,  null: false

  # Postgres requires the columns we're partitioning by to be part of the
  # primary key, so we create a composite primary key
  primary_key [:date, :instance_id, :product_id]

  jsonb :data         # general data about the product
  jsonb :competitors  # data about this product from other competitors
  jsonb :statistics   # sales statistics about the product
  jsonb :applied_rule # information about our repricing of the product
end

The partitioned table above acts as sort of an abstract table, in the sense that it won't contain any data by itself, but instead it allows partitions to be created from it, which will be the ones holding the data. For example, let's create a partition of this table which will hold data for an e-shop with ID of 10 for March 2021:

create_table? :products_10_202103, partition_of: :products do
  from 10, Date.new(2021, 3, 1)
  to 10, Date.new(2021, 4, 1) # this end is excluded from the range
end

The arguments we pass to from and to are the values of columns we've specified in :partition_by on the partitioned table (we have two arguments because we specified two columns – :instance_id and :date). The name of the table partition is custom, in this example I've just chosen a products_<INSTANCE_ID>_YYYYMM naming convention. Given that we're creating these partitions on-the-fly (as opposed to in a schema migration), I've used Sequel's create_table? to handle the case when the partition already exists, which generates a CREATE TABLE IF NOT EXISTS query.

Once we've created the partitions and populated them with data, we can just reference the main table in our queries, and Postgres will know which partition(s) it should direct the queries to.

# queries partition `products_10_202101`
DB[:products].where(instance_id: 10, date: Date.new(2021, 1, 1)).to_a

# queries partitions `products_29_202102` and `products_29_202103`
DB[:products].where(instance_id: 29, date: Date.new(2021, 2, 1)..Date.new(2021, 3, 31)).to_a

# creates the record in partition `products_13_202012`
DB[:products].insert(instance_id: 13, date: Date.new(2020, 12, 25), product_id: "abc123", ...)

Upserts

We have 4 types of product data, each of which is retrieved, aggregated, and stored in a separate background job. Previously, each background job was writing to a separate CSV file, but now they would all be writing to a single table, either creating new records or updating existing records with new data.

The simplest option which is also concurrency-safe was to use Postgres' INSERT ... ON CONFLICT ..., also known as "upsert". Sequel supports upserts with all its parameters via #insert_conflict:

DB[:products]
  .insert_conflict # by default ignores insert that fails unique constraint violation
  .insert(instance_id: 10, date: Date.new(2021, 1, 1), product_id: "abc123")

# INSERT INTO products (instance_id, date, product_id)
# VALUES (10, '2021-01-01', 'abc123')
# ON CONFLICT DO NOTHING

In my task, I needed each background job to only store data it is responsible for, and that these jobs can be executed in any order. So, the background job which was responsible for storing general product data into the analytics database had the following code:

product_data #=>
# [
#   { instance_id: 10, date: Date.new(2021, 1, 1), product_id: "111", data: { ... } },
#   { instance_id: 10, date: Date.new(2021, 1, 1), product_id: "222", data: { ... } },
#   { instance_id: 10, date: Date.new(2021, 1, 1), product_id: "333", data: { ... } },
#   ...
# ]

product_data.each_slice(1000) do |values|
  DB[:products]
    .insert_conflict(
      target: [:date, :instance_id, :product_id],
      update: { data: Sequel[:excluded][:data] }
    )
    .multi_insert(values)
end

# INSERT INTO products (...) VALUES (...)
# ON CONFLICT (date, instance_id, product_id) DO UPDATE data = excluded.data

The above inserts values in batches of 1,000 records, and when the record already exists, only the data column value is replaced. In general, when a conflict happens, Postgres exposes the values we've tried to insert under the excluded qualifier. So, in the DO UPDATE clause we were able to do data = excluded.data, which updates only the data column. In this case, Postgres also requires us to specify the column(s) involved in the unique index, which in our case are date, instance_id, and product_id that form the primary key.

COPY

Now that we've covered the important bits involved in modifying the code to write new data into Postgres, what remains is efficiently migrating all the historical data from our CSV files into Postgres.

The fastest way to import CSV data into a Postgres table is using COPY FROM, which Sequel supports via #copy_into:

DB.copy_into :records,
  format: "csv",
  options: "HEADERS true",
  data: File.foreach("records.csv")

In my case, I couldn't import the CSV files directly into the products table, because I wanted to write most of the fields into JSONB columns. So I first imported the CSV data into a temporary table whose columns matched the CSV data, and then copied the data from that table into the end products table in the desired format.

data = File.foreach("products_10.csv")
columns = File.foreach("products_10.csv").first.chomp.split(",")
temp_table = :"products_#{SecureRandom.hex}"

DB.create_table temp_table do
  columns.each do |column|
    String column.to_sym
  end
end

DB.copy_into temp_table, format: "csv", options: "HEADERS true", data: data

DB[temp_table].paged_each.each_slice(1000) do |products|
  DB[:products].insert products.map { |product| ... } # transform into desired format
end

DB.drop_table temp_table

Inserting from SELECT

Notice how in the last example we were fetching data from the temporary table, transforming it in Ruby, then writing the result in batches into the destination table. This is a common way people copy data, but it's actually pretty inefficient, both in terms of memory usage and speed.

What we can do instead is transform the data via a SELECT statement, and then pass it directly to INSERT. This way we avoid retrieving any data on the client side, and we allow Postgres to determine the most efficient way to copy the data.

INSERT INTO my_table (col1, col2, col3, ...)
SELECT val1, val2, val3, ... FROM another_table WHERE ...

Sequel's #insert method supports this feature by accepting a dataset object:

DB[:products].insert [:instance_id, :date, :product_id, :data], DB[temp_table].select(...)

I've covered this topic in more depth in my recent article, which includes a benchmark illustrating the performance benefits of this approach.

Unlogged tables

Lastly, writing data into a temporary table does create some overhead, which we can reduce by making the temporary table "unlogged". With this setting, data written to this table is not written to Postgres' write-ahead log (used for crash recovery), which makes the writing speed considerably faster than in ordinary tables.

Sequel allows creating unlogged tables by passing the :unlogged option to #create_table:

DB.create_table temp_table, unlogged: true do
  # ...
end
# CREATE UNLOGGED TABLE products_5ea6fe37d2fde562 (...)

Loose count

During this migration, I've often wanted to check the total number of records, to verify that the migration was performed for all of our customers. The problem is that the regular SELECT count(*) ... query can be slow for larger amounts of records.

# can take some time:
DB[:products].where(instance_id: 10).count
# SELECT count(*) FROM products WHERE instance_id = 10

Luckily, Postgres stores a rough number of records for each table, which can be retrieved very fast, and in my case that was more than sufficient. I wouldn't have found about this Postgres feature if I hadn't come across Sequel's pg_loose_count extension:

DB.extension :pg_loose_count
DB.tables
  .grep(/products_10_.+/) # select only partitions for e-shop with ID of 10
  .sum { |partition| DB.loose_count(partition) } # fast count

Conclusion

With Sequel and Postgres I was able to use table partitioning to store time-series data in a way that's efficient to query, import large amounts of historical data from CSV files into a temporary unlogged table, and transform it and write it into the destination table all in SQL, while checking the data migration progress with Postgres' loose record counts.

All these Postgres features helped me to efficiently handle time-series data and import historical data, and I didn't have to make any comporomises, thanks to Sequel supporting me every step of the way.

Interesting throw/catch behaviour in Ruby

Janko Marohnić — Sun, 24 Jan 2021 17:27:01 +0000

When I was working on integrating Rodauth with OmniAuth authentication, I noticed an error warning after upgrading to Rails 6.1, when Rodauth was redirecting inside a Rails controller action:

class RodauthController < ApplicationController
  def omniauth
    # ...
    rodauth.login("omniauth") # logs the session in and redirects
  end
end

Could not log "process_action.action_controller" event.
/path/to/actionpack-6.1.1/lib/action_controller/log_subscriber.rb:26:in `block in process_action': undefined method `first' for nil:NilClass (NoMethodError)

Since I want the integration between Rodauth and Rails to be as smooth as possible, I decided to investigate.

Diving in

Let's see the ActionController::LogSubscriber source code where the error happens:

# lib/action_controller/log_subscriber.rb
module ActionController
  class LogSubscriber < ActiveSupport::LogSubscriber
    # ...
    def process_action(event)
      # ...
        status = payload[:status]

        if status.nil? && (exception_class_name = payload[:exception].first) # <==== the exception happens here
          status = ActionDispatch::ExceptionWrapper.status_code_for_exception(exception_class_name)
        end
      # ...
    end
    # ...
  end
end

We can see that the issue happens because :exception data is missing from the instrumentation event payload. Let's look at ActionController::Instrumentation next, which is in charge of instrumenting controller actions:

# lib/action_controller/metal/instrumentation.rb
class ActionController
  module Instrumentation
    def process_action(*)
      # ...
      ActiveSupport::Notifications.instrument("process_action.action_controller", raw_payload) do |payload|
        result = super # <=== this calls our controller action
        payload[:response] = response
        payload[:status]   = response.status
        # ...
      end
    end
  end
end

We can see that, if our controller action raises an exception, the :status data will never be set. This ties to the status.nil? check we've seen in the ActionController::LogSubscriber.

The remaining part is to find where :exception is being set. Knowing that instrumentation is implemented in Active Support, I quickly found ActiveSupport::Notifications::Instrumenter:

# lib/active_support/notifications/instrumenter.rb
module ActiveSupport
  module Notifications
    class Instrumenter
      # ...
      def instrument(name, payload = {})
        # ...
        begin
          yield payload if block_given?
        rescue Exception => e
          payload[:exception] = [e.class.name, e.message] # <==== the exception is set here
          payload[:exception_object] = e
          raise e
        ensure
          finish_with_state listeners_state, name, payload
        end
      end
    end
  end
end

The problem

When Rodauth redirects, what is actually doing is throwing :halt with the rack response. This is how Roda implements redirection, and it's common practice in non-Rails web frameworks (Sinatra and Cuba do it too). In our case, throwing exits from controller action and is caught by the Roda middleware.

Does throwing act the same way as raising an exception does? Initially it would appear so:

begin
  throw :halt
rescue Exception => exception
  puts "rescue: #{exception.inspect}"
  raise
ensure
  puts "ensure"
end

rescue: #<UncaughtThrowError: uncaught throw :halt>
ensure
~> uncaught throw :halt (UncaughtThrowError)

This makes sense to me, because uncaught throw is an exception. But then why wasn't the rescue block that was supposed to set the :exception in the event payload being executed?

The picture starts getting clearer when we wrap the code with a catch block:

catch(:halt) do
  begin
    throw :halt
  rescue Exception => exception
    puts "rescue: #{exception.inspect}"
    raise
  ensure
    puts "ensure"
  end
end

ensure

We see that in this case the rescue block isn't being executed, and this is precisely our scenario. This actually makes sense when you think about it, because a throw with a matching catch is not anything erroneous, it's just a way to do an early return.

The solution

Now we know where the issue is, which is that Rails just wasn't correctly handling a throw/catch scenario when processing controller actions. Fixing it was the easy part.

throw/catch is probably something you'll rarely use, but it does have its use cases. I hope this article taught you a bit more about this lesser known Ruby feature.

Adding Multifactor Authentication in Rails 6 with Rodauth

Janko Marohnić — Mon, 21 Dec 2020 14:33:23 +0000

Multi-factor authentication or MFA (generalized two-factor authentication or 2FA) is a method of authentication where the user is required to provide two or more pieces of evidence ("factors") in order to be granted access. Typically the user would first prove knowledge of something only they know (e.g. their password), and then prove posession of something only they own (e.g. another device). This provides an extra layer of security for the user's account.

Most common multifactor authentication methods include:

TOTP (Time-based One-Time Passwords) – user has an app installed on their device that displays the authentication code, which is refreshed every 30 seconds
SMS codes – user receives authentication codes on their phone via SMS when the application requests them
Recovery codes – user is given a fixed set of one-time codes they can enter when logging in (this is typically used as a backup method)
WebAuthn – user authenticates themselves using a security key or built-in platform biometric sensors (e.g. fingerprint)

In this article, I want to show you how to add multifactor authentication to a Rails app using Rodauth, which has built-in support for each of the multifactor authentication methods mentioned above. Compared to alternatives¹, Rodauth provides a much more integrated experience by shipping with complete endpoints, default HTML templates, session management, lockout logic and more². To keep the tutorial focused, we'll be implementing just the first three methods, as they're by far the most common.

We'll be using the rodauth-rails gem, and we'll be continuing off of the application we started building in my previous article. The goal functionality: allow users to set up TOTP as their primary MFA method, and use SMS codes and recovery codes as backup MFA methods.

TOTP

The TOTP functionality is provided by Rodauth's otp feature. It depends on the rotp and rqrcode gems, so let's first install those:

$ bundle add rotp rqcode

Next, we need to create the required database table. For this we'll use the migration generator provided by rodauth-rails:

$ rails generate rodauth:migration otp
# create  db/migrate/20201214200106_create_rodauth_otp.rb

$ rails db:migrate
# == 20201214200106 CreateRodauthOtp: migrating =======================
# -- create_table(:account_otp_keys)
# == 20201214200106 CreateRodauthOtp: migrated ========================

Now we can enable the otp feature in our Rodauth configuration:

# app/lib/rodauth_app.rb
class RodauthApp < Rodauth::Rails::App
  configure do
    # ...
    enable :otp
  end
end

This adds the following routes to our application:

/otp-auth – authenticate via TOTP code
/otp-setup – set up TOTP authentication
/otp-disable – disable TOTP authentication
/multifactor-manage – set up or disable available MFA methods
/multifactor-auth – authenticate via available MFA methods
/multifactor-disable – disable all MFA methods

To allow the user to configure MFA, let's display a link to the /multifactor-manage route for managing MFA methods in our views:

<!-- app/views/application/_navbar.html.erb -->
<% if rodauth.logged_in? %>
  <!-- ... --->
  <%= link_to "Manage MFA", rodauth.two_factor_manage_path, class: "dropdown-item" %>
  <!-- ... --->
<% end %>

Now when the user logs in and clicks on "Manage MFA", they'll get redirected to the OTP setup page that Rodauth provides out-of-the-box³:

The user can now scan the QR code using an authenticator app such as Google Authenticator, Microsoft Authenticator or Authy, and enter the OTP code (along with their current password) to finish setting up OTP.

When the user logs in the next time, we want to redirect them automatically to the OTP auth page, and generally require logged in users that have MFA setup to authenticate with 2nd factor. This can be achieved with the following configuration:

# app/lib/rodauth_app.rb
class RodauthApp < Rodauth::Rails::App
  configure do
    # ...
    # redirect the user to the MFA page if they have MFA setup
    login_redirect do
      if uses_two_factor_authentication?
        two_factor_auth_required_redirect
      else
        "/"
      end
    end
  end

  route do |r|
    # ...
    # require MFA if the user is logged in and has MFA setup
    if rodauth.logged_in? && rodauth.uses_two_factor_authentication?
      rodauth.require_two_factor_authenticated
    end
  end
end

Recovery codes

After the user sets up TOTP, it's recommended to also generate a set of "recovery" codes for them to save somewhere, which they can use on login in case they lose access to their TOTP device. This functionality is provided by Rodauth's recovery_codes feature.

Let's start by creating the required database table:

$ rails generate rodauth:migration recovery_codes
# create  db/migrate/20201214200106_create_rodauth_recovery_codes.rb

$ rails db:migrate
# == 20201217071036 CreateRodauthRecoveryCodes: migrating =======================
# -- create_table(:account_recovery_codes, {:primary_key=>[:id, :code]})
# == 20201217071036 CreateRodauthRecoveryCodes: migrated ========================

And enabling the recovery_codes feature in our Rodauth configuration:

# app/lib/rodauth_app.rb
class RodauthApp < Rodauth::Rails::App
  configure do
    # ...
    enable :otp, :recovery_codes
  end
end

This adds the following routes to our app:

/recovery-auth – authenticate via a recovery code
/recovery-codes – view & add recovery codes

We'll now override the after_otp_setup hook to display recovery codes to the user after they've successfully set up TOTP, instead of the default redirect. We'll override the default Rodauth template to display the recovery codes in a nicer way and add a download link for convenience.

# app/lib/rodauth_app.rb
class RodauthApp < Rodauth::Rails::App
  configure do
    # ...
    # auto generate recovery codes after TOTP setup
    auto_add_recovery_codes? true
    # display recovery codes after TOTP setup
    after_otp_setup do
      set_notice_now_flash "#{otp_setup_notice_flash}, please make note of your recovery codes"
      response.write add_recovery_codes_view
      request.halt # don't process the request any further
    end
  end
end

<!-- app/views/rodauth/add_recovery_codes.html.erb -->
<div class="border border-info rounded px-3 py-2">
  <% rodauth.recovery_codes.each_slice(2) do |code1, code2| %>
    <div class="row text-info text-center">
      <div class="col-lg my-1 text-monospace"><%= code1 %></div>
      <div class="col-lg my-1 text-monospace"><%= code2 %></div>
    </div>
  <% end %>
</div>

<p class="my-3 text-center">
  Copy these recovery codes to a safe location.
  You can also download them <%= link_to "here", download_recovery_codes_path %>.
</p>

<!-- Used for filling in missing recovery codes later on -->
<% if rodauth.can_add_recovery_codes? %>
  <h2>Add Additional Recovery Codes</h2>
  <%= rodauth.render("recovery-codes").html_safe %>
<% end %>

# config/routes.rb
Rails.application.routes.draw do
  # ...
  controller :rodauth do
    get "download-recovery-codes"
  end
end

# app/controllers/rodauth_controller.rb
class RodauthController < ApplicationController
  def download_recovery_codes
    send_data rodauth.recovery_codes.join("\n"),
      filename: "myapp-recovery-codes.txt",
      type: "text/plain"
  end
end

When the user now sets up TOTP, they will be shown a page like this:

And when they log into their account the next time, on the multifactor auth page they can choose to enter a recovery code instead of TOTP.

SMS codes

In addition to TOTP, it's good practice to also provide the ability to use SMS codes for 2nd factor authentication. Rodauth provides a specialized sms_codes feature for this.

To set it up, we again create the required database table:

$ rails generate rodauth:migration sms_codes
# create  db/migrate/20201219173710_create_rodauth_sms_codes.rb

$ rails db:migrate
# == 20201219173710 CreateRodauthSmsCodes: migrating ==================
# -- create_table(:account_sms_codes)
# == 20201219173710 CreateRodauthSmsCodes: migrated ===================

And enable the sms_codes feature in the Rodauth configuration:

# app/lib/rodauth_app.rb
class RodauthApp < Rodauth::Rails::App
  configure do
    # ...
    enable :otp, :recovery_codes, :sms_codes
  end
end

This adds the following routes to our app:

/sms-request – request the SMS code to be sent
/sms-auth – authenticate via an SMS code
/sms-setup – set up SMS codes authentication
/sms-confirm – confirm the provided phone number
/sms-disable – disable SMS codes authentication

When an SMS code is requested, Rodauth calls the sms_send method with the configured phone number and a corresponding text message. This method isn't defined by default, since Rodauth doesn't know how we want to send the SMS, instead we're expected to implement sms_send:

# app/lib/rodauth_app.rb
class RodauthApp < Rodauth::Rails::App
  configure do
    # ...
    sms_send do |phone, message|
      # we need to implement this
    end
  end
end

We'll use Twilio for sending SMS messages. Assuming we've set up an account, we'll add the account SID, auth token, and phone number to Rails credentials:

$ rails credentials:edit

twilio:
  account_sid: <YOUR_ACCOUNT_SID>
  auth_token: <YOUR_AUTH_TOKEN>
  phone_number: <YOUR_PHONE_NUMBER>

Next, we'll install the twilio-ruby and dry-initializer gems, and create a wrapper class for the Twilio client:

$ bundle add twilio-ruby dry-initializer

# app/lib/twilio_client.rb
class TwilioClient
  Error              = Class.new(StandardError)
  InvalidPhoneNumber = Class.new(Error)

  extend Dry::Initializer

  option :account_sid,  default: -> { Rails.application.credentials.twilio[:account_sid] }
  option :auth_token,   default: -> { Rails.application.credentials.twilio[:auth_token] }
  option :phone_number, default: -> { Rails.application.credentials.twilio[:phone_number] }

  def send_sms(to, message)
    client.messages.create(from: phone_number, to: to, body: message)
  rescue Twilio::REST::RestError => error
    # more details here: https://www.twilio.com/docs/api/errors/21211
    raise TwilioClient::InvalidPhoneNumber, error.message if error.code == 21211
    raise TwilioClient::Error, error.message
  end

  def client
    Twilio::REST::Client.new(account_sid, auth_token)
  end
end

Finally, we'll implement sms_send using our new TwilioClient class, converting SMS sending errors into validation errors:

# app/lib/rodauth_app.rb
class RodauthApp < Rodauth::Rails::App
  configure do
    # ...
    sms_send do |phone, message|
      twilio = TwilioClient.new
      twilio.send_sms(phone, message)
    rescue TwilioClient::InvalidPhoneNumber
      throw_error_status(422, sms_phone_param, sms_invalid_phone_message)
    rescue TwilioClient::Error
      throw_error_status(500, sms_phone_param, "sending the SMS code failed")
    end
  end
end

When the user now visits the SMS authentication setup page on the multifactor manage page, they can enter their phone number and password, and then enter the SMS code they received to finish the SMS authentication setup.

Afterwards, when the user logs in the next time, in addition to authenticating via TOTP or a recovery code, they'll now also be able to choose to authenticate via SMS.

Disabling multifactor authentication

In addition to setup and authentication, Rodauth also provides endpoints for disabling any MFA method, which require the user to confirm their password:

/otp-disable – disable OTP authentication
/sms-disable – disable multifactor authentication
/multifactor-disable – disable all multifactor methods

The links for disabling MFA methods that have previously been set up are automatically displayed on the multifactor manage page:

Disabling a MFA method will take care of deleting any records associated to that account from the corresponding database table.

Closing words

In this tutorial we've shown how to add multifactor authentication functionality in Rails with Rodauth and rodauth-rails. We've enabled the user to set up TOTP as their primary MFA method, after which they receive a set of recovery codes, and have the possibility to also set up SMS as a backup MFA method.

We've seen that Rodauth ships with complete endpoints and default HTML templates for managing multiple MFA methods, and generally provides a much more integrated experience compared to the alternatives. Given that multifactor authentication is becoming an increasingly common requirement, it's very useful to have a framework that supports it with the same level of standard as the other authentication features.

At the time of writing, most popular alternatives are devise-two-factor, active_model_otp, and two_factor_authentication. ↩
See the source code for OTP, SMS Codes, Recovery Codes, and Two Factor Base for more details. ↩
You can override the default template by running rails generate rodauth:views otp and modifying app/views/rodauth/otp_setup.html.erb. ↩

Adding Authentication in Rails 6 with Rodauth

Janko Marohnić — Thu, 19 Nov 2020 16:50:59 +0000

In this tutorial, we'll show how to add fully functional authentication and account management functionality into a Rails 6 app, using the Rodauth authentication framework. Rodauth has many advantages over the mainstream alternatives such as Devise, Sorcery, Clearance, and Authlogic, see my previous article for an introduction.

We'll be working with a fresh Rails app that has basic posts CRUD and Bootstrap installed:

$ git clone https://gitlab.com/janko-m/rails_bootstrap_starter.git rodauth_blog
$ cd rodauth_blog
$ bin/setup

Installing Rodauth

Let's start by adding the rodauth-rails gem to our Gemfile:

$ bundle add rodauth-rails

Next, we'll run the rodauth:install generator provided by rodauth-rails:

$ rails generate rodauth:install

# create  db/migrate/20200820215819_create_rodauth.rb
# create  config/initializers/rodauth.rb
# create  config/initializers/sequel.rb
# create  app/lib/rodauth_app.rb
# create  app/controllers/rodauth_controller.rb
# create  app/models/account.rb

This will create the Rodauth app with default authentication features, configure Sequel which Rodauth uses for database interaction to reuse Active Record's database connection, and generate a migration that will create tables for the loaded Rodauth features. Let's run the migration:

$ rails db:migrate

# == CreateRodauth: migrating ====================================
# -- create_table(:accounts)
# -- create_table(:account_password_hashes)
# -- create_table(:account_password_reset_keys)
# -- create_table(:account_verification_keys)
# -- create_table(:account_login_change_keys)
# -- create_table(:account_remember_keys)
# == CreateRodauth: migrated ===========================

If everything was installed successfully, we should be able to open the /create-account page and see Rodauth's default registration form.

Adding authentication links

Rodauth configuration generated by rodauth-rails provides several routes for authentication and account management:

$ rails rodauth:routes

# /login                   rodauth.login_path
# /create-account          rodauth.create_account_path
# /verify-account-resend   rodauth.verify_account_resend_path
# /verify-account          rodauth.verify_account_path
# /logout                  rodauth.logout_path
# /remember                rodauth.remember_path
# /reset-password-request  rodauth.reset_password_request_path
# /reset-password          rodauth.reset_password_path
# /change-password         rodauth.change_password_path
# /change-login            rodauth.change_login_path
# /verify-login-change     rodauth.verify_login_change_path
# /close-account           rodauth.close_account_path

Let's use this information to add some main authentication links to our navigation header:

<!-- app/views/application/_navbar.html.erb -->
<!-- ... --->
<% if rodauth.logged_in? %>
  <div class="dropdown">
    <%= link_to current_account.email, "#", class: "btn btn-info dropdown-toggle", data: { toggle: "dropdown" } %>
    <div class="dropdown-menu dropdown-menu-right">
      <%= link_to "Change password", rodauth.change_password_path, class: "dropdown-item" %>
      <%= link_to "Change email", rodauth.change_login_path, class: "dropdown-item" %>
      <div class="dropdown-divider"></div>
      <%= link_to "Close account", rodauth.close_account_path, class: "dropdown-item text-danger" %>
      <%= link_to "Sign out", rodauth.logout_path, method: :post, class: "dropdown-item" %>
    </div>
  </div>
<% else %>
  <div>
    <%= link_to "Sign in", rodauth.login_path, class: "btn btn-outline-primary" %>
    <%= link_to "Sign up", rodauth.create_account_path, class: "btn btn-success" %>
  </div>
<% end %>
<!-- ... --->

Rodauth doesn't define the #current_account method, so let's copy-paste the example provided in the rodauth-rails README:

# app/controllers/application_controller.rb
class ApplicationController < ActionController::Base
  before_action :current_account, if: -> { rodauth.logged_in? }

  private

  def current_account
    @current_account ||= Account.find(rodauth.session_value)
  rescue ActiveRecord::RecordNotFound
    rodauth.logout
    rodauth.login_required
  end
  helper_method :current_account
end

Now our application will show login and registration links when the user is not logged in:

While logged in users will see some basic account management links:

Requiring authentication

Now that we have working authentication, we'll likely want to require the user to be authenticated for certain parts of our application. In our case, we want to authenticate the posts controller.

We could add a before_action callback to the controller, but Rodauth allows us to do this inside the Rodauth app's route block, which is called before each Rails route. This way we can keep our authentication logic contained in a single place.

# app/lib/rodauth_app.rb
class RodauthApp < Rodauth::Rails::App
  # ...
  route do |r|
    # ...
    if r.path.start_with?("/posts")
      rodauth.require_authentication
    end
  end
end

Now visiting the /posts page will redirect the user to the /login page if they're not logged in.

We'll also want to associate the posts to the accounts table:

$ rails generate migration add_account_id_to_posts account:references
$ rails db:migrate

# app/models/account.rb
class Account < ApplicationRecord
  has_many :posts
  # ...
end

And scope them to the current account in the posts controller:

# app/controllers/posts_controller.rb
class PostsController < ApplicationController
  # ...
  def index
    @posts = current_account.posts.all
  end
  # ...
  def create
    @post = current_account.posts.build(post_params)
    # ...
  end
  # ...
  private
    def set_post
      @post = current_account.posts.find(params[:id])
    end
    # ...
end

Adding new fields

To have something other than an email address to display our users, let's require users to enter their name during registration. This will also give us an opportunity to see how Rodauth can be configured.

Since we'll need to edit the registration form, let's first copy Rodauth's HTML templates into our Rails application:

$ rails generate rodauth:views

# create  app/views/rodauth/_field.html.erb
# create  app/views/rodauth/_field_error.html.erb
# create  app/views/rodauth/_login_field.html.erb
# create  app/views/rodauth/_login_display.html.erb
# create  app/views/rodauth/_password_field.html.erb
# create  app/views/rodauth/_submit.html.erb
# create  app/views/rodauth/_login_form.html.erb
# create  app/views/rodauth/_login_form_footer.html.erb
# create  app/views/rodauth/_login_form_header.html.erb
# create  app/views/rodauth/login.html.erb
# create  app/views/rodauth/multi_phase_login.html.erb
# create  app/views/rodauth/logout.html.erb
# create  app/views/rodauth/_login_confirm_field.html.erb
# create  app/views/rodauth/_password_confirm_field.html.erb
# create  app/views/rodauth/create_account.html.erb
# create  app/views/rodauth/_login_hidden_field.html.erb
# create  app/views/rodauth/verify_account_resend.html.erb
# create  app/views/rodauth/verify_account.html.erb
# create  app/views/rodauth/reset_password_request.html.erb
# create  app/views/rodauth/reset_password.html.erb
# create  app/views/rodauth/_new_password_field.html.erb
# create  app/views/rodauth/change_password.html.erb
# create  app/views/rodauth/change_login.html.erb
# create  app/views/rodauth/close_account.html.erb

We can now open the create_account.erb template and add a new name field:

<!-- app/views/rodauth/create_account.erb -->
<%= form_tag rodauth.create_account_path, method: :post do %>
  <!-- new "name" field -->
  <div class="form-group">
    <%= label_tag "name", "Name" %>
    <%= render "field", name: "name", id: "name" %>
  </div>

  <%= render "login_field" %>
  <%= render "login_confirm_field" if rodauth.require_login_confirmation? %>
  <%= render "password_field" if rodauth.create_account_set_password? %>
  <%= render "password_confirm_field" if rodauth.create_account_set_password? && rodauth.require_password_confirmation? %>
  <%= render "submit", value: "Create Account" %>
<% end %>

Since the user's name won't be used for authentication, let's store it in a new profiles table, and associate the profiles table to the accounts table.

$ rails generate model Profile account:references name:string
$ rails db:migrate

# app/models/account.rb
class Account < ApplicationRecord
  has_one :profile
  # ...
end

We now need our Rodauth app to actually handle the new name parameter. We'll validate that it's filled in and create the associated profile record after the account is created.

# app/lib/rodauth_app.rb
class RodauthApp < Rodauth::Rails::App
  configure do
    # ...
    before_create_account do
      # Validate presence of the name field
      throw_error_status(422, "name", "must be present") unless param_or_nil("name")
    end
    after_create_account do
      # Create the associated profile record with name
      Profile.create!(account_id: account_id, name: param("name"))
    end
    after_close_account do
      # Delete the associated profile record
      Profile.find_by!(account_id: account_id).destroy
    end
    # ...
  end
end

Now we can update our navigation header to use the user's name instead of their email address:

-     <%= link_to current_account.email, "#", class: "btn btn-info dropdown-toggle", data: { toggle: "dropdown" } %>
+     <%= link_to current_account.profile.name, "#", class: "btn btn-info dropdown-toggle", data: { toggle: "dropdown" } %>

Sending emails asynchronously

Rodauth will send emails as part of account verification, email change, password change, and password reset. By default, these emails will be sent synchronously via an internal mailer, but for performance reasons we should send these emails asynchronously inside a background job.

Since we'll want to modify Rodauth's default email templates eventually, let's create our own mailer with the default templates:

$ rails generate rodauth:mailer

# create  app/mailers/rodauth_mailer.rb
# create  app/views/rodauth_mailer/email_auth.text.erb
# create  app/views/rodauth_mailer/password_changed.text.erb
# create  app/views/rodauth_mailer/reset_password.text.erb
# create  app/views/rodauth_mailer/unlock_account.text.erb
# create  app/views/rodauth_mailer/verify_account.text.erb
# create  app/views/rodauth_mailer/verify_login_change.text.erb

class RodauthMailer < ApplicationMailer
  def verify_account(recipient, email_link)
    # ...
  end
  def reset_password(recipient, email_link)
    # ...
  end
  def verify_login_change(recipient, old_login, new_login, email_link)
    # ...
  end
  def password_changed(recipient)
    # ...
  end
end

Now, to have our mailer automatically called by Rodauth and to deliver emails in the background, let's uncomment the following lines in our Rodauth app:

# app/lib/rodauth_app.rb
class RodauthApp < Rodauth::Rails::App
  configure do
    # ...
    send_reset_password_email do
      mailer_send(:reset_password, email_to, reset_password_email_link)
    end
    send_verify_account_email do
      mailer_send(:verify_account, email_to, verify_account_email_link)
    end
    send_verify_login_change_email do |login|
      mailer_send(:verify_login_change, login, verify_login_change_old_login, verify_login_change_new_login, verify_login_change_email_link)
    end
    send_password_changed_email do
      mailer_send(:password_changed, email_to)
    end
    auth_class_eval do
      def mailer_send(type, *args)
        db.after_commit do
          RodauthMailer.public_send(type, *args).deliver_later
        end
      end
    end
    # ...
  end
end

We enqueue the email deliveries after the database transaction commits, to ensure that any database changes made before Rodauth called into our mailer have been applied when the background job is picked up.

Closing words

In this tutorial we've gradually built out a complete authentication and account management flow using the Rodauth authentication framework. It supports login & logout, account creation with email verification and a grace period, password change & password reset, email change with email verification, and close account functionality. We've seen how to require authentication for certain routes, add new fields to the registration form, and send authentication emails asynchrously.

I'm personally very excited about Rodauth, as it has an impressive featureset and a refreshingly clean design, and also it's not tied to Rails. I've been working hard on rodauth-rails to make it as easy as possible to get started with in Rails, so hopefully it will help Rodauth gain more traction in the Rails community.

Inserting from Datasets with Sequel

Janko Marohnić — Mon, 02 Nov 2020 08:30:55 +0000

At a previous company, I was working on an internal app for managing and distributing video content. Content curators would create playlists of videos, submit them for approval, and once playlists were approved they would be automatically published to target devices.

Both the approval and the publishing flows consisted of multiple steps, so we were creating logs for these events and storing them in PostgreSQL. We were using Sequel for database interaction, and our logs table looked roughly like this:

create_table :activity_logs do
  primary_key :id
  foreign_key :playlist_id, :playlists, null: false
  foreign_key :user_id, :users
  String :event, null: false
  String :action, null: false
  String :message
  String :target
  Time :created_at, null: false, default: Sequel::CURRENT_TIMESTAMP
end

The table was populated with log records that looked like this:

[
  ...,
  {
    id:          23,
    playlist_id: 103,
    user_id:     30,
    event:       "approval",
    action:      "approve",
    message:     "Looks good!",
    target:      nil,
    created_at:  Time.utc(2020, 10, 1, 5, 29, 39),
  },
  {
    id:          25,
    playlist_id: 423,
    user_id:     nil,
    event:       "publication",
    action:      "published",
    message:     nil,
    target:      "Video Wall 1",
    created_at:  Time.utc(2020, 10, 1, 7, 38, 11),
  },
  ...
]

What we eventually realized was that we're actually storing two types of records in the same table, where not all columns are applicable to both types:

the approval flow, done by a person (uses user_id and message columns), and
the publication flow, is done by the system (uses target column).

Instead storing both types of events in the activity_logs table, we've decided to extract the publication logs into a new publication_logs table.

1. Creating the new table

We've started by creating our desired publication_logs table:

create_table :publication_logs do
  primary_key :id
  foreign_key :playlist_id, :playlists, null: false
  String :action, null: false
  String :target
  Time :created_at, null: false, default: Sequel::CURRENT_TIMESTAMP
end

We've ommitted the event column from activity_logs, as well user_id and message, since they were only applicable for approval actions done by a person (publication is done automatically by our system).

Next up was migrating the existing publication logs from the activity_logs table into the new publication_logs table.

2a. Migrating records one by one

The easiest approach of migrating data would be looping through each record, transforming the record into the desired format, and inserting it into the new table, then deleting the data from the old table.

# select records we want to move
publication_logs = from(:activity_logs).where(event: "publication")

# insert each record individually into the new table
publication_logs.each do |log|
  from(:publication_logs).insert(
    playlist_id: log[:playlist_id],
    action:      log[:action],
    target:      log[:target],
    created_at:  log[:created_at],
  )
end

# delete records from the old table
publication_logs.delete

We can gain an additional performance improvement here if we switch to using prepared statements for the inserts:

# select records we want to move
publication_logs = from(:activity_logs).where(event: "publication")

prepared_insert = from(:publication_logs).prepare :insert, :insert_publication_data,
  playlist_id: :$playlist_id, action: :$action, target: :$target, created_at: :$created_at

# insert each record individually into the new table
publication_logs.each do |log|
  prepared_insert.call(
    playlist_id: log[:playlist_id],
    action:      log[:action],
    target:      log[:target],
    created_at:  log[:created_at],
  )
end

# delete records from the old table
publication_logs.delete

This strategy would usually perform well enough on small to medium tables, but it so happens ours was a logs table with lots of records (about 200,000 IIRC). Since long-running migrations can generally be problematic, let's find a better approach.

2b. Migrating records in bulk

Most SQL databases support inserting multiple records in a single query, which is significantly faster. In PostgreSQL, the syntax looks like this:

INSERT INTO my_table (col1, col2, col3, ...)
VALUES ('a1', 'b1', 'c1', ...),
       ('a2', 'b2', 'c2', ...),
       ...

With Sequel, we can utilize the multi-insert feature via #import, which accepts the list of columns as the 1st argument, and the arrays of values as the 2nd argument:

# select the records we want to move
publication_logs = from(:activity_logs).where(event: "publication")

# insert all new data in a single query
from(:publication_logs).import [:playlist_id, :action, :target, :created_at],
  publication_logs.map { |log| log.fetch_values(:playlist_id, :action, :target, :created_at) }

# delete records from the old table
publication_logs.delete

This already provides a big improvement in terms of speed. However, one problem here is that we're loading all the data into memory before inserting it, which can spike our memory usage. Furthermore, inserting so many records at once can put significant load on the database.

To fix both issues, we can break the multi-insert down into smaller batches:

# select the records we want to move
publication_logs = from(:activity_logs).where(event: "publication")

# bullk-insert new records in batches
publication_logs.each_slice(1000) do |logs|
  from(:publication_logs).import [:playlist_id, :action, :target, :created_at],
    logs.map { |log| log.fetch_values(:playlist_id, :action, :target, :created_at) }
end

# delete records from the old table
publication_logs.delete

2c. Migrating records in the database

While the previous approach should work well enough for most cases, retrieving data from the database only to send it back slightly modified does seem a bit wasteful. Wouldn't it be great if we could do all of that on the database side?

It turns out we can. Typically, we use an INSERT statement by passing it raw values. However, an INSERT statement can also receive values directly from a SELECT statement:

INSERT INTO my_table (col1, col2, col3, ...)
SELECT a1, b1, c1, ... FROM other_table WHERE ...

In Sequel, the #import method supports this feature by accepting a dataset object in place of raw values. This allows us to rewrite our migration one last time:

# select the records we want to move
publication_logs = from(:activity_logs).where(event: "publication")

# form a dataset with the new data and insert from that dataset
from(:publication_logs).import [:playlist_id, :action, :target, :created_at],
  publication_logs.select(:playlist_id, :action, :target, :created_at)

# delete records from the old table
publication_logs.delete

3. Removing old columns

What's left was removing columns that were specific to publication from the activity_logs table:

alter_table :activity_logs do
  drop_column :event           # this table will only hold approval logs now
  drop_column :target          # this was specific to publication logs
  set_column_not_null :user_id # only publication logs didn't have user id set
end

Measuring performance

I've created a script which populates the activity_logs table with 100,000 approval logs and 100,000 publication logs, and measures execution time and memory allocation of all 5 migration strategies we've talked about (database is PosgreSQL 12).

Here are the results:

Strategy	Duration	Objects allocated	Memory allocated
individual inserts	35.7 s	610k	478 MB
individual prepared inserts	23.8 s	480k	634 MB
bulk insert	8.4 s	21k	162 MB
batched bulk insert	7.9 s	21k	158 MB
database insert	2.0 s	94	0 MB

As expected, inserting records individually is the slowest strategy. I was a bit surprised to see that it's allocating significantly more memory than the bulk insert strategy, but I believe it's because we're allocating hashes for each record, while with bulk inserts we're allocating value arrays. It's nice to see that prepared statements provided a ~33% speedup, though at a cost of increased memory usage.

For bulk inserts, I expected the batching variant to be slower, because I imagined that we're trading off speed for reduced load. But I'm positively surprised that it performs at least as fast as the non-batched version.

Inserting from a dataset performed the best, since there the database does all the work. In our case it was 4x faster than the bulk insert strategy. It allocates zero additional memory on the application side, because everything is happening on the database side.

Closing thoughts

For the past several years, I've become more and more interested in finding ways to make the database do the majority of the work I need, especially since I've started using Sequel. I love that Sequel allows me to do anything I want without compromises.

We've compared the performance of migrating records (a) individually, (b) in bulk, and (c) from a dataset. Inserting from a dataset was by far the fastest strategy, and the code wasn't any more complex than the other strategies.

This article showed a fairly simple scenario, in many cases our data migrations might require more complex data transformations. In this case it's tempting to just do the transformations in Ruby, as we know Ruby much better than SQL. However, I think that learning some common SQL functions can come a long way in being able to make your database do the work.

The Complexity of Active Record Transactions

Janko Marohnić — Mon, 28 Sep 2020 11:32:04 +0000

I've recently picked up the sequel-activerecord_connection gem again to make some reliability improvements around database transactions. For context, this gem extends Sequel with the ability to reuse Active Record's database connection, which should lower the barrier for trying out Sequel in apps that use Active Record.

After pushing some fixes, I was thinking how working on this gem has greatly increased my familiarity with the internals of database transaction implementations of both Active Record and Sequel. Since there aren't any existing articles on this topic, I thought it would be useful to share the knowledge I gathered over the past few months.

This article will compare the transaction API implementation between Active Record and Sequel, and assumes the reader is already familiar with Active Record's transaction API usage. As the title suggests, I will be critical of Active Record's implementation. I know some will perceive this as "not nice", but I think it's important to be aware of the internal complexity of libraries we're using every day (myself included).

Model vs Database

Active Record

Active Record transactions are typically called on the model, which is shown in the official docs as well. I think this can be misleading to novice developers, as it suggests that database transactions are tied to specific database tables, when in fact they're applied to any queries made by the current database connection.

# opens a connection-wide transaction (unrelated to the `accounts` table)
Account.transaction do
  balance.save!
  account.save!
end

Active Record provides transaction callbacks as part of a model's lifecycle, allowing you to execute code after the transaction commits or rolls back. This, for example, allows you to spawn a background job after a record is persisted, but wait until the transaction commits to ensure the record is up-to-date when the background job is picked up.

class Account < ActiveRecord::Base
  after_create_commit :send_welcome_email

  private

  def send_welcome_email
    AccountMailer.welcome(self).deliver_later
  end
end

In my opinion, this approach has several issues. For one, it encourages putting business logic into your Active Record models, and generally increases complexity of the model lifecycle. It's unfortunately not trivial to use transaction callbacks outside of models, because they're coupled to models (although there are gems that work around that).

Transaction callbacks can also negatively impact memory usage if you're allocating many model instances within a transaction, as references to these model instances are held until the transaction is committed or rolled back, which prevents Ruby from garbage collecting them beforehand. Active Record will do this for any model that has any transaction callbacks defined.

class Comment < ActiveRecord::Base
  after_commit :deliver_new_mentions, on: [:create, :update], if: :body_changed?

  private

  def deliver_new_mentions
    MentionNotificationJob.deliver_later(self)
  end
end

ActiveRecord::Base.transaction do
  author.comments.find_each do |comment|
    # Even though we're not triggering the `after_commit` callback here, Active
    # Record will still keep references to these model instances, preventing
    # Ruby from garbage collecting them until the transaction is closed.
    comment.update(author: new_author)
  end
end

Sequel

In Sequel, the transaction API is implemented on the database object, which is completely decoupled from models.

DB = Sequel.connect(adapter: "postgresql", database: "myapp") #=> #<Sequel::Database ...>

# calling #transaction on the database object communicates it's connection-wide
DB.transaction do
  balance.save
  account.save
end

Sequel also has transaction hooks, but they too are defined on the database object, and aren't tied to models in any way – they're just blocks of code that get executed after the transaction is committed or rolled back. This makes them possible to use in business logic that lives outside of models (of course, in that case one can also just move the code outside of the transaction block).

class CreateAccount
  def call(attributes)
    DB.transaction do
      account = Account.create(attributes)
      send_welcome_email(account)
      account.update(api_key: SecureRandom.hex)
    end
  end

  private

  def send_welcome_email(account)
    # queue email delivery after the enclosing transaction commits
    DB.after_commit do
      AccountMailer.welcome(account).deliver_later
    end
  end
end

And if you really want to register transaction hooks on the model level, you can do that inside regular model lifecycle hooks:

class Account < Sequel::Model
  def after_create
    db.after_commit { AccountMailer.welcome(self).deliver_later }
  end
end

By giving us the ability to compose APIs this way, the Sequel::Model class was able to remain unaware of the existence of transaction hooks (which keeps it simpler), but we were still able to achieve the same functionality as we have with Active Record.

Note that the examples above will still keep references to the model instances until the transaction is closed. However, Sequel's API gives us the necessary control to change that. For example, we can choose to register a transaction hook only if a certain condition holds (useful in use cases like file attachments):

class Comment < Sequel::Model
  def after_save
    if column_changed?(:body)
      db.after_commit { MentionNotificationJob.deliver_later(self) }
    end
  end
end

DB.transaction do
  author.comments_dataset.paged_each do |comment|
    # The transaction hooks aren't registered, so Ruby can garbage collect
    # these model instances while the loop is running.
    comment.update(author: new_author)
  end
end

We can also register a transaction hook in a way where it will only keep the reference to the record id instead of the whole record instance:

class MentionNotifications
  def self.enqueue(comment_id)
    db.after_commit { MentionNotificationJob.deliver_later(comment_id) }
  end
end

class Comment < Sequel::Model
  def after_save
    NotificationMentions.enqueue(id)
  end
end

Transaction state

Active Record

Active Record maintains transaction state on the connection level, but a lot of transaction-related state is also maintained at the model level. While the transaction manager is implemented pretty decently, the ActiveRecord::Transactions module is incredibly complex, and has been the source of numerous issues.

The reason for this complexity is that every new incoming bug has generally been solved by adding yet another tweak, yet another conditional, yet another instance variable. And some of these instance variables even leak outside of the ActiveRecord::Transactions module, which indicates a leaky abstraction.

Honestly, for me this reached a state where I don't consider Active Record's transaction callbacks to be safe enough for production, and I try to avoid them whenever possible.

Sequel

Sequel stores all the transaction state in a single @transactions instance variable on the database object. Models don't have access to the transaction state, which keeps transactions fully decoupled from models.

DB.transaction do |conn|
  DB.after_commit { ... }
  DB.transaction(savepoint: true) do
    DB.instance_variable_get(:@transactions)[conn] #=>
    # {
    #   after_commit: [
    #     <Proc...> # the block we've registered above
    #   ],
    #   savepoints: [
    #     { ... }, # transaction data
    #     { ... }  # savepoint data
    #   ]
    # }
  end
end

If you're reading Sequel's transaction code, you'll notice that all of it is contained in a single file and single context (including transaction hooks). In my experience this made the logic much easier to grok.

Lazy transactions

Active Record

In version 6.0, Active Record introduced a performance optimization that makes transactions lazy. What this means is that Active Record will issue BEGIN/COMMIT queries only if there was at least one query exected inside the transaction block.

ActiveRecord::Base.transaction do
  ActiveRecord::Base.connection.execute "SELECT 1"
end
# BEGIN
# SELECT 1
# COMMIT

ActiveRecord::Base.transaction do
end
# (no queries were executed)

The main use case behind this addition seems to be saving lots of records whose attributes didn't change, where each attempted update would execute empty BEGIN/COMMIT statements (even though no UPDATE was issued), which didn't perform well. A workaround at the time would be to call record.save if record.changed? instead.

article = Article.find(id)
article.published #=> true
article.update(published: true) # executed empty BEGIN/COMMIT prior to Active Record 6.0

However, as Sean Griffin had pointed out in the pull request review, this added significant complexity for very little gain. In addition to requiring additional transaction state, each Active Record adapter is now also responsible for materializing transactions when necessary.

Sequel

In Sequel, opening a transaction will always execute BEGIN/COMMIT statements (if the transaction commits), regardless of whether any queries were made inside the block or not.

DB.transaction do
end
# BEGIN
# COMMIT

Sequel::Model#save behaves differently than ActiveRecord::Base#save, in terms that it always executes an UPDATE statement for an existing record (updating all columns). To update only changed attributes, you would use Sequel::Model#save_changes, which doesn't execute UPDATE if no attributes have changed. And Sequel::Model#update calls #save_changes under the hood:

article = Article.find(id)
article.published #=> true
article.update(published: true) # no queries executed

Unlike ActiveRecord::Base#save, Sequel::Model#save_changes doesn't open a transaction if it won't execute the UPDATE statement. This seems like a much more elegant solution to the problem Active Record's lazy transactions intended to solve, but with none of the complexity.

Final words

I really care that libraries I'm using at work have sufficiently straightforward internals that I can understand when debugging an issue. When it comes to database transactions, Active Record's internal complexity is just too overwhelming for me (and that's coming from someone who contributes to open source on a daily basis).

On the other hand, the Sequel's transaction implementation was fairly straightforward to understand, which is all the more impressive considering that it's more feature-rich compared to Active Record (see the docs). And this is not an exception – I regularly see this pattern whenever I'm reading Sequel's source code 😉

Hopefully this article will add another point towards Sequel for people starting new Ruby/Rails projects.

Rodauth: A Refreshing Authentication Solution for Ruby

Janko Marohnić — Tue, 18 Aug 2020 15:26:24 +0000

If you're working with Rails, chances are your authentication layer is implemented using one of the popular authentication frameworks – Devise, Sorcery, Clearance, or Authlogic. These libraries provide complete authentication and account management functionality for Rails, giving you more time to focus on the core business logic of your product.

One characteristic these authentication frameworks have in common is that they're all built on top of Rails. This means that they plug into the existing Rails components (models, controllers, routes), and generally try to follow "The Rails Way" of doing things.

However, libraries that build on top of Rails often tend to inherit some of Rails' anti-patterns as well, such as having high coupling, burdening models and controllers with additional responsibilities, and overusing Active Record callbacks. Having had my fair share of experience with the Ruby ecosystem outside of Rails' default menu, I've come to learn that taking the effort to make your library's implementation decoupled from Rails can provide significant advantages:

breaking away from Rails gives you mental space to design your library better
your library can now be used with other Ruby web frameworks too 😇
supporting new Rails versions becomes easier too

If one wanted to implement authentication in another Ruby web framework, the most frequent advice seems to be to use Warden. Warden is a Rack-based library that provides a mechanism for authentication, with support for multiple strategies. However, the problem is that Warden doesn't actually do anything by itself. You still need to implement login, remembering, registration, account verification, password reset and other functionality yourself (which is what Devise does), and I'd argue this is actually the hard part 😓

Enter Rodauth

We've actually had a better solution for some time now, it just hasn't received enough attention. Jeremy Evans – a Ruby Hero, a Ruby committer, and an author of numerous Ruby libraries (most popular of which is Sequel and Roda) – has been developing a new full-featured authentication framework for the past several years, called Rodauth. Recently I've finally had the opportunity to integrate Rodauth into a Rails app at work, and I can safely say that its tagline – "Ruby's most advanced authentication framework" – is in no way exaggerated 👌

One of the first things you'll notice is that, in constrast to other authentication frameworks which are built on top of Rails and Active Record, Rodauth is implemented using Roda and Sequel. As a big fan of both libraries, for me personally this was exciting, but at the same time I was worried this meant Rodauth cannot be used with Rails. However, it turns out Rodauth can in fact be used with any Rack-based web framework, including Rails.

Integrating Rodauth into Rails for the first time definitely wasn't trivial, but once all the kinks had been worked out, I extracted the necessary glue code into rodauth-rails. It comes with generators, controller & view integration, mailer support, CSRF & flash integration, HMAC security and more. Additionally, to make Sequel coexist better with Active Record, I've created the sequel-activerecord_connection gem, which enables Sequel to reuse Active Record's database connection. All this should make Rodauth as easy to get started with as its Rails-based counterparts 💪

And with its recent 2.0 release, it's time to finally take a proper look into Rodauth and see what makes it so special ✨

Encapsulated authentication logic

As we've touched on before, what characterizes most Rails-based authentication frameworks is that they implement their authentication behaviour directly on the MVC layer. While this approach keeps things close to Rails, it also shoves additional responsibilities into already-heavy Rails components, and generally causes the authentication logic to be spread out across multiple application layers.

With Rodauth, all authentication behaviour is encapsulated in a special Rodauth::Auth object, which is created inside a Roda middleware and has access to the request context. It handles everything from routing requests to Rodauth endpoints to performing authentication-related commands and queries. We configure it as a Roda plugin and can we use it in Roda's routing block to perform any actions before the request reaches the main app. This design enables us to keep our authentication logic contained in a single file.

class RodauthMiddleware < Roda
  # define your Rodauth configuration
  plugin :rodauth do
    # load authentication features you need
    enable :login, :logout, :create_account, :verify_account, :reset_password
    # change default settings
    password_minimum_lenth 8
    login_return_to_requested_location? true
    reset_password_autologin? true
    logout_redirect "/"
    # ...
  end
  # handles requests before they reach the main app
  route do |r|
    # handle Rodauth paths (/login, /create-account, /reset-password, ...)
    r.rodauth
    # require authentication for certain routes
    if r.path.start_with?("/dashboard")
      rodauth.require_authentication
    end
  end
end

When we add the above Roda app to our middleware stack, the route block will be called for each request before it reaches our main app, yielding the request object. The r.rodauth call will handle any Rodauth routes, while rodauth.require_authentication will redirect to the login page if the session is not authenticated. When the end of the routing block is reached, the request proceeds onto our main app.

If you're using Rails with rodauth-rails, the Rodauth instance will remain available in your controllers and views as well, so you can do things like require authentication at the controller level if you prefer to:

class PostsController < ApplicationController
  before_action -> { rodauth.require_authentication }
  # ...
end

or render authentication links in the views:

<% if rodauth.authenticated? %>
  <%= link_to "Sign out", rodauth.logout_path, method: :post %>
<% else %>
  <%= link_to "Sign in", rodauth.login_path %>
  <%= link_to "Sign up", rodauth.create_account_path %>
<% end %>

There are many more useful authentication methods defined on the Rodauth instance that give us additional flexibility and introspection:

rodauth.auhenticated_by                   # e.g. ["password", "otp"]
rodauth.session_value                     # returns account id from the session
rodauth.account_from_login("foo@bar.com") # retrieves account with given email address
rodauth.password_match?("secret")         # returns whether given password matches current account's password
rodauth.login("password")                 # logs the account in and redirects with a notice flash
rodauth.logout                            # logs the session out
# ...

Feature maturity

Rodauth has all of the essential features you already know from other authentication frameworks:

login/logout and remember
create account with email verification (and a grace period)
reset password and change password
change email with email verification
lockout and close account

You'll also find most industry-standard security features the devise-security extension provides:

There are many other useful features as well:

HTTP Basic authentication
email authentication
password confirmation dialog (with a grace period)
audit logging
...

Multifactor authentication

In addition to the features above, Rodauth also provides multifactor authentication functionality out-of-the-box, supporting multiple MFA methods (TOTP, SMS codes, recovery codes, and WebAuthn). It includes complete endpoints for setup, authentication, and disabling MFA methods, along with the related HTML templates.

Here is an example setup that allows a user to enable TOTP verification for their account, along with a backup SMS number and recovery codes (assuming we've created the necessary database tables):

class RodauthApp < Roda
  plugin :rodauth do
    enable :otp, :sms_codes, :recovery_codes, ...
    # use Twilio to send SMS messages
    sms_send do |phone, message|
      twilio = Twilio::Rest::Client.new("<ACCOUNT_SID>", "<AUTH_TOKEN>")
      twilio.messages.create(body: message, to: phone, from: "<APP_PHONE_NUMBER>")
    end
  end
end

<!-- somewhere under account settings: -->
<% if rodauth.uses_two_factor_authentication? %>
  <%= link_to "Manage MFA", rodauth.two_factor_manage_path %>
  <%= link_to "Disable MFA", rodauth.two_factor_disable_path %>
<% else %>
  <%= link_to "Setup MFA", rodauth.two_factor_manage_path %>
<% end %>

Having full-featured multifactor authentication built into the framework is a game changer, as it means it will work well with other features and will remain compatible with future Rodauth releases 🙌

JSON API

Another cool feature of Rodauth is its built-in support for JWT, which provides token-based JSON API access for each authentication feature. Here is how we can configure the JWT feature:

class RodauthApp < Roda
  plugin :rodauth, json: :only do # 1) enable Roda's JSON support and only allow JSON access
    enable :login, :create_account, :change_password, :close_account, :jwt # 2) load JWT feature
    jwt_secret "abc123" # 3) set secret for the JWT feature
    require_login_confirmation? false
    require_password_confirmation? false
  end
end

We can now trigger Rodauth actions via a JSON requests, using the Authorization header for authentication. Here is an example flow using http.rb:

# 1) create an account
response = HTTP.post("https://myapp.com/create-account", json: { login: "foo@example.com", password: "secret" })
token = response.headers["Authorization"]
# 2) change the password
response = HTTP.auth(token).post("https://myapp.com/change-password", json: { password: "secret", "new-password": "new secret" })
# 3) login with the new password
response = HTTP.post("https://myapp.com/login", json: { login: "foo@example.com", password: "new secret" })
token = response.headers["Authorization"]
# 4) close the account
http.auth(token).post("https://myapp.com/close-account", json: { password: "new secret" })
# 5) try to login again
response = HTTP.post("https://myapp.com/login", json: { login: "foo@example.com", password: "new secret" })
response.status.to_s # => "401 Unauthorized"

Other authentication frameworks haven't yet standardized JSON API support:

Devise has multiple solutions (DeviseTokenAuth, Devise::JWT, SimpleTokenAuthentication)
Sorcery currently has a few unmerged pull requests (#239, #167, #70)
Clearance doesn't currently support JSON (comment)

Uniform configuration DSL

If we look at how Devise is customized, we'll notice there are several different layers on which we can configure authentication behaviour: global settings, model settings, controller settings, and routing settings. Some of these settings can be configured dynamically (based on either model or controller state), while other can only be configured statically. And some before/after hooks are triggered on the model level, while for others you need to override controller actions. This is pretty inconsistent 👎

In contrast, Rodauth provides a uniform configuration DSL that allows changing virtually any authentication behaviour, which is all defined on the Rodauth::Auth class. You can override a configuration method either by providing a static value, or by passing a dynamic block that gets evaluated in context of a Rodauth::Auth instance (and you can call super to get original behaviour).

class RodauthApp < Roda
  plugin :rodauth do
    # each feature adds its own set of configuration methods
    enable :login, :create_account, :verify_account_grace_period, :reset_password

    # examples of static values:
    login_redirect "/dashboard"        # redirect to /dashboard after logging in
    verify_account_grace_period 3.days # allow unverified access for 3 days after registration
    reset_password_autologin? true     # automatically log the user in after password reset

    # examples of dynamic blocks:
    password_minimum_length { MyConfig.get(:min_password_length) } # change minimum allowed password length
    login_valid_email? { |login| TrueMail.valid?(login) }          # override email validation logic
    verify_account_redirect { login_redirect }                     # after account verification redirect to wherever login redirects to
  end
end

Internaly, Rodauth also provides a DSL for writing new features, which streamlines adding new configuration methods and encourages making the feature behaviour as flexible as possible.

Hooks

Rodauth consistently provides hooks for virtually any action, which we can override inside our configuration block. We can do something before/after specific operations:

after_login           { remember_login }
before_create_account { throw_error("company", "must be present") if param("company").empty? }
after_login_failure   { LoginAttempts.increment(account[:email]) }

before specific routes:

before_change_login_route   { require_password_authentication }
before_create_account_route { redirect "/register" if param("type").empty? }

or before each Rodauth route:

before_rodauth { AuthLogger.call(request) }

Enhanced security

When I would talk to people about Rodauth, one common concern was whether it's secure enough, given that alternatives such as Devise are more widely-used. While I'm not qualified enough to provide a direct answer, what I can say is that there are multiple indications that Jeremy takes Rodauth's security very seriously:

Rodauth incorporates some additional security measures that are not common in other authentication frameworks (more on this below)
Jeremy proactively patches Rodauth when new security issues are found in other authentication frameworks
Jeremy's talks showcase some very advanced knowledge on web application security

Tokens

Many authentication features (remembering logins, password reset, account verification, email change verification etc.) generate random unique tokens as part of their functionality. Since these tokens give permissions for performing sensitive actions, we don't want others having access to them.

When hmac_secret is set (rodauth-rails sets it automatically), Rodauth will sign the tokens sent via email using HMAC, while the raw tokens will be stored in the database. This will make it so if the tokens in the database are leaked (e.g. via an SQL injection vulnerability), they will not be usable without also having access to the HMAC secret.

hmac_secret "abc123"

For better bruteforce protection, Rodauth tokens also include the account id. This way an attacker can only attempt to bruteforce the token for a single account at a time, instead of being able to bruteforce tokens for all accounts at once.

<account_id>_<random_token>

Protecting password hashes

Because cracking password hashes is always getting faster, to additionally protect password hashes in case of a database breach, Devise and Sorcery enable you to use a password pepper, a secret key added to the password before it's hashed. This is pretty secure, provided that compromise of database doesn't also imply compromise of application secrets. A downside of peppers is that they're not easy to rotate, as changing the pepper would invalidate all existing passwords stored in the database.

bcrypt(password + pepper)

Rodauth offers an alternative approach to protecting password hashes, which relies on restricting database access. Password hashes are stored in a separate table, for which the application database user has INSERT/UPDATE/DELETE access, but not SELECT access. This means an attacker cannot retrieve password hashes even in case of an SQL injection or a remote code execution exploit.

# full access by the app database user
create_table :accounts do
  primary_key :id
  String :email, null: false, unique: true
end

# *restricted* access by the app database user
create_table :account_password_hashes do
  foreign_key :id, :accounts, primary_key: true
  String :password_hash, null: false
end

That's great, but without the ability to retrieve password hashes, how is your app then able to check whether an entered password matches the one stored in the database on login? The answer: via a database function, which takes an account id and a password hash, and returns whether the given password hash matches the stored password hash for the given account id. This is effectively a limited SELECT query, but it's all that's needed for login functionality, and it's all the attacker gets in case of a database breach.

SELECT rodauth_valid_password_hash(123, "<some_bcrypt_hash>") -- returns true or false

You can read about this in more detail here. Note that this mode is completely optional, Rodauth works just as well with storing password hashes in the accounts table; in this case it does password matching in Ruby, just like we're used to with other authentication frameworks.

Other design highlights

Decoupled authentication features

Rodauth really did a great job in achieving loose coupling between its authentication features. Additionally, each Rodauth feature is contained entirely in a single file (and a single context), with the code being loaded only if the feature is enabled. All this makes it significantly easier to understand how each individual feature works.

enable :create_account,              # create account and automatically login                -- lib/features/rodauth/create_account.rb
       :verify_account,              # require email verification after account creation     -- lib/features/rodauth/verify_account.rb
       :verify_account_grace_period, # allow unverified login for a certain period of time   -- lib/features/rodauth/verify_account_grace_period.rb

       :change_login,                # change email immediately                              -- lib/features/rodauth/change_login.rb
       :verify_login_change,         # require email verification before change is applied   -- lib/features/rodauth/verify_login_change.rb

       :password_complexity,         # add password complexity requirements                  -- lib/features/rodauth/password_complexity.rb
       :disallow_common_passwords,   # don't allow using a weak common password              -- lib/features/rodauth/disallow_common_passwords.rb
       :disallow_password_reuse,     # don't allow using a previously used password          -- lib/features/rodauth/disallow_password_reuse.rb

Database table per feature

Rodauth features are decoupled not only in code, but also in the database. Instead of adding all columns to a single table, in Rodauth each feature has a dedicated table. This makes it more transparent which database columns belong to which features.

create_table :accounts do ... end                    # used by base feature
create_table :account_password_reset_keys do ... end # used by reset_password feature
create_table :account_verification_keys do ... end   # used by verify_account feature
create_table :account_login_change_keys do ... end   # used by verify_login_change feature
create_table :account_remember_keys do ... end       # used by remember feature
# ...

Conclusion

Rodauth is one of those libraries that come and renew my interest in the Ruby ecosystem 😍 Its advanced and comprehensive set of authentication features (including multifactor authentication and JSON support), backed by a powerful configuration DSL that provides immense flexibility, provide overwhelming advantages over other authentication frameworks.

As someone who enjoys working in other Ruby web frameworks, I strongly believe in focusing on libraries that are accessible to everyone. And Rodauth is the first full-featured authentication solution that can be used in any Ruby web framework (including Rails).

Hopefully this overview will encourage you to try Rodauth out in your next project 😉

sequel-activerecord_connection now fully supports Sequel's transaction API

Janko Marohnić — Fri, 24 Jul 2020 11:56:27 +0000

A while ago I created the sequel-activerecord_connection gem, which allows you to use Sequel alongside ActiveRecord and reuse the same database connection. This makes it easier to try Sequel out or use libraries that depend on Sequel (e.g. Rodauth) in ActiveRecord-based projects.

Originally the transaction support was partially implemented, where the most common Sequel transaction options were emulated with the ActiveRecord API. In the latest version I've rewritten the transaction code to fully support Sequel's transaction API, including transaction/savepoint hooks.

require "sequel"

DB = Sequel.postgres(test: false) # avoid creating a database connection
DB.extension :activerecord_connection # use ActiveRecord's database connection

DB.transaction(isolation: :serializable) do # handles isolation levels
  DB.after_commit { ... } # runs after transaction commits
  DB.transaction(savepoint: true) do # creates a savepoint
    DB.after_rollback { ... } # runs after savepoint rolls back
  end
  DB.in_transaction? #=> true
end