DEV Community: Juraj Kostolanský

How to wrap Rails mailer previews in a database transaction

Juraj Kostolanský — Fri, 04 Sep 2020 08:24:24 +0000

Most of the web applications today need to send some kind of email. In Ruby on Rails, this is an easy task. Just use the Action Mailer, create the email content, set up the SMTP configuration and you are ready to go.

The hardest part of this process is the content creation. Especially if you are building a multi-language application, seeing the big picture just from the source code can be quite hard. Fortunately, Rails provides a way to see how these emails look right in a web browser, using the Action Mailer Previews. Just create a class that inherits from the ActionMailer::Preview and call the required mailer action.

# test/mailers/previews/user_mailer_preview.rb

class UserMailerPreview < ActionMailer::Preview
  def confirmation
    user = User.new(email: "tester@example.com")
    UserMailer.with(user: user).confirmation
  end
end

However, sometimes your mailer expects the ID of a record in the database, not the object itself. And sometimes you need to prepare this record, for example to create a new email confirmation token.

# test/mailers/previews/user_mailer_preview.rb

class UserMailerPreview < ActionMailer::Preview
  def confirmation
    user = User.first
    user.prepare_token!(:confirmation)
    UserMailer.with(user_id: user.id).confirmation
  end
end

But this should be just a temporary change. Sure, you can wrap the whole method in a database transaction and use the database rollback command. But… How exactly?

Fortunately, there is an easy way to accomplish this using some Rails monkey patching. Just create the following module:

# lib/core_extensions/mailer_preview_rollback

module CoreExtensions
  module MailerPreviewRollback
    def preview
      ActiveRecord::Base.transaction do
        super
        raise ActiveRecord::Rollback
      end
    end
  end
end

Create an initializer in your Ruby on Rails application and prepend this module:

# config/initializers/mailer_preview_rollback.rb

Rails.configuration.after_initialize do
  require "core_extensions/mailer_preview_rollback"
  Rails::MailersController.prepend CoreExtensions::MailerPreviewRollback
end

And voila! All of your mailer previews are now safely wrapped in a database transaction.

Original article: How to wrap Rails mailer previews in a database transaction

How to upgrade PostgreSQL from 11 to 12

Juraj Kostolanský — Thu, 03 Oct 2019 19:40:06 +0000

The new PostgreSQL 12 has just been released. There are multiple ways to upgrade from the old version 11, and the easiest one is by using the pg_upgrade tool. Here is a quick tutorial for Ubuntu (or Debian) systems. And, please, do not forget to back up your data!

Update packages and install the new PostgreSQL 12.

sudo apt-get update
sudo apt-get install postgresql-12 postgresql-server-dev-12

Check if there are any differences in the config files.

diff /etc/postgresql/11/main/postgresql.conf /etc/postgresql/12/main/postgresql.conf
diff /etc/postgresql/11/main/pg_hba.conf /etc/postgresql/12/main/pg_hba.conf

Stop the PostgreSQL service.

sudo systemctl stop postgresql.service

sudo su postgres

Check clusters (notice the --check argument, this will not change any data).

/usr/lib/postgresql/12/bin/pg_upgrade \
  --old-datadir=/var/lib/postgresql/11/main \
  --new-datadir=/var/lib/postgresql/12/main \
  --old-bindir=/usr/lib/postgresql/11/bin \
  --new-bindir=/usr/lib/postgresql/12/bin \
  --old-options '-c config_file=/etc/postgresql/11/main/postgresql.conf' \
  --new-options '-c config_file=/etc/postgresql/12/main/postgresql.conf' \
  --check

Migrate the data (without the --check argument).

/usr/lib/postgresql/12/bin/pg_upgrade \
  --old-datadir=/var/lib/postgresql/11/main \
  --new-datadir=/var/lib/postgresql/12/main \
  --old-bindir=/usr/lib/postgresql/11/bin \
  --new-bindir=/usr/lib/postgresql/12/bin \
  --old-options '-c config_file=/etc/postgresql/11/main/postgresql.conf' \
  --new-options '-c config_file=/etc/postgresql/12/main/postgresql.conf'

Go back to the regular user.

exit

Swap the ports for the old and new PostgreSQL versions.

sudo vim /etc/postgresql/12/main/postgresql.conf
# ...and change "port = 5433" to "port = 5432"

sudo vim /etc/postgresql/11/main/postgresql.conf
# ...and change "port = 5432" to "port = 5433"

Start the PostgreSQL service.

sudo systemctl start postgresql.service

sudo su postgres

Check the new PostgreSQL version.

psql -c "SELECT version();"

Run the generated analyze_new_cluster script.

./analyze_new_cluster.sh

Back to normal user.

exit

Check which old PostgreSQL packages are installed.

apt list --installed | grep postgresql

Remove the old PostgreSQL packages (from the listing above).

sudo apt-get remove postgresql-11 postgresql-server-dev-11

Remove the old configuration.

sudo rm -rf /etc/postgresql/11/

sudo su postgres

Finally, drop the old cluster data.

./delete_old_cluster.sh

Done!

Original article: How to upgrade PostgreSQL from 11 to 12

Using PostgreSQL advisory locks to control concurrency

Juraj Kostolanský — Thu, 04 Jul 2019 11:00:00 +0000

We often need to control concurrent access to databases in Ruby on Rails applications. We need to make sure that a piece of code will run synchronously, to guarantee atomicity of multiple non-atomic operations.

PostgreSQL provides various techniques to control concurrent access to data. For example, we can use the Serializable transaction isolation level. In that case, any concurrent execution of a set of Serializable transactions is guaranteed to produce the same effect as running them one at a time in some order.

There are also explicit row-level locks that prevent two transactions from modifying a resource in conflicting ways. We can choose pessimistic locking if a database transaction conflict is very likely to happen. It locks the record until the transaction is done, and any other transaction trying to acquire the same lock has to wait until the lock is released.

Otherwise, we can use optimistic locking. It uses a version number of the record to check whether another process has made changes to it since it was opened, and throws an exception if that has occurred and the update is ignored.

Another way to control concurrency is advisory locking.

Advisory locking

Advisory locks by PostgreSQL are not tied to database tables or rows. We can use them, for example, to restrict the concurrency of a specific code in our application that works with a record from a specific table, and the other parts of our application that are accessing that table wouldn't notice that.

Advisory locks are application enforced locks. That means PostgreSQL isn't using them implicitly, and it is up to the application to call provided statements to obtain and release these locks, and to give a meaning to these locks.

We can even use advisory locking to control synchronization of a code that isn't managing data from the database - for example, to make API calls, access files on a disk, or anything else.

Advisory locks can be acquired at session level or at transaction level. The PostgreSQL documentation explains the difference:

Once acquired at session level, an advisory lock is held until explicitly released or the session ends. Unlike standard lock requests, session-level advisory lock requests do not honor transaction semantics: a lock acquired during a transaction that is later rolled back will still be held following the rollback, and likewise an unlock is effective even if the calling transaction fails later. A lock can be acquired multiple times by its owning process; for each completed lock request there must be a corresponding unlock request before the lock is actually released.

Transaction-level lock requests, on the other hand, behave more like regular lock requests: they are automatically released at the end of the transaction, and there is no explicit unlock operation. This behavior is often more convenient than the session-level behavior for short-term usage of an advisory lock.

There are multiple functions provided by PostgreSQL to acquire and release exclusive or shared, session or transactional level advisory locks. Again, check out the official documentation for more information.

Here are some of the provided functions for session level locks:

pg_advisory_lock(key bigint) obtains exclusive session level advisory lock. If another session already holds a lock on the same resource identifier, this function will wait until the resource becomes available. Multiple lock requests stack, so that if the resource is locked three times it must then be unlocked three times.
pg_try_advisory_lock(key bigint) obtains exclusive session level advisory lock if available. It's similar to pg_advisory_lock, except it will not wait for the lock to become available - it will either obtain the lock and return true, or return false if the lock cannot be acquired immediately.
pg_advisory_unlock(key bigint) releases an exclusive session level advisory lock.

And here are some for transaction level locks:

pg_advisory_xact_lock(key bigint) obtains exclusive transaction level advisory lock. It works the same as pg_advisory_lock, except the lock is automatically released at the end of the current transaction and cannot be released explicitly.
pg_try_advisory_xact_lock(key bigint) obtains exclusive transaction level advisory lock if available. It works the same as pg_try_advisory_lock, except the lock is automatically released at the end of the transaction and cannot be released explicitly.

A simple LockManager

We can use advisory locks in Ruby on Rails by creating a small class responsible for obtaining and releasing the lock. Let's call it the LockManager.

One small inconvenience is that advisory locks in PostgreSQL use a number as an argument. However, locking a specific number isn't too readable and self-explanatory. It would be better if we could use a string for this purpose.

A possible solution is to use the Zlib.crc32 hash function. It's commonly used for checksums, but we can use it to convert any string into the number required by PostgreSQL.

class LockManager
  def self.with_transaction_lock(lock_name)
    lock_id = Zlib.crc32(lock_name.to_s)
    ActiveRecord::Base.transaction do
      ActiveRecord::Base.connection.execute("SELECT pg_advisory_xact_lock(#{lock_id})")
      yield
    end
  end

  def self.with_session_lock(lock_name)
    lock_id = Zlib.crc32(lock_name.to_s)
    begin
      ActiveRecord::Base.connection.execute("SELECT pg_advisory_lock(#{lock_id})")
      yield
    ensure
      ActiveRecord::Base.connection.execute("SELECT pg_advisory_unlock(#{lock_id})")
    end
  end
end

And, for example, we can use this class in the following way:

LockManager.with_transaction_lock("subscription_usage_#{user_id}") do
  user = User.find(user_id)
  new_usage = user.current_subscription_usage
  user.update_attributes!(subscription_usage: new_usage)
end

Original article: Using PostgreSQL advisory locks to control concurrency

How to migrate primary keys from bigint to UUID in Rails

Juraj Kostolanský — Wed, 13 Mar 2019 13:00:00 +0000

Rails uses bigint as a default primary key type for PostgreSQL databases. This is a solid option, but you may consider using UUID as primary keys. There is and endless debate about pros and cons of each of them, so I'm not going to repeat the arguments here. However, if you've decided to migrate to UUIDs, here is one possible solution.

Let's say we have two tables in our database - users and comments:

create_table :users do |t|
  # ...
end

create_table :comments do |t|
  t.references :user, null: false, foreign_key: true
  # ...
end

A possible way to migrate these tables to UUIDs is:

Add UUID columns to tables
Migrate associations
Drop ID columns, rename UUID to ID and use them as primary keys

This migration process is in the following example, which is pretty self-explanatory.

class MigrateToUuid < ActiveRecord::Migration[5.2]
  def up
    # Add UUID columns
    add_column :users,    :uuid, :uuid, null: false, default: -> { "gen_random_uuid()" }
    add_column :comments, :uuid, :uuid, null: false, default: -> { "gen_random_uuid()" }

    # Add UUID columns for associations
    add_column :comments, :user_uuid, :uuid

    # Populate UUID columns for associations
    execute <<-SQL
      UPDATE comments SET user_uuid = users.uuid
      FROM users WHERE comments.user_id = users.id;
    SQL

    # Change null
    change_column_null :comments, :user_uuid, false    

    # Migrate UUID to ID for associations
    remove_column :comments, :user_id
    rename_column :comments, :user_uuid, :user_id

    # Add indexes for associations
    add_index :comments, :user_id    

    # Add foreign keys
    add_foreign_key :comments, :users    

    # Migrate primary keys from UUIDs to IDs
    remove_column :users,    :id
    remove_column :comments, :id
    rename_column :users,    :uuid, :id
    rename_column :comments, :uuid, :id
    execute "ALTER TABLE users    ADD PRIMARY KEY (id);"
    execute "ALTER TABLE comments ADD PRIMARY KEY (id);"

    # Add indexes for ordering by date
    add_index :users,    :created_at
    add_index :comments, :created_at
  end

  def down
    raise ActiveRecord::IrreversibleMigration
  end

The gen_random_uuid() function is from pgcrypto extension, so make sure it's enabled in your databse before running this migration:

CREATE EXTENSION IF NOT EXISTS pgcrypto;

You can also easily write the down method to make the migration reversible.

Original article: How to migrate primary keys from bigint to UUID

Localize your Rails enums

Juraj Kostolanský — Tue, 26 Feb 2019 16:00:00 +0000

When Rails 4.1 was released, it came out with a bunch of new features. One of those was ActiveRecord enums.

class Conversation < ApplicationRecord
  enum status: [:active, :archived]
end

conversation = Conversation.create(status: :active)
conversation.status # => "active"

This is a handy feature. However, if you try to localize your Rails app, for example for some select boxes, or just to show the current value, you will quickly realize that there is no built-in feature to translate the enum values.

Starting from Rails 5, all models will inherit from ApplicationRecord. We can use that to create our universal solution.

class ApplicationRecord < ActiveRecord::Base
  self.abstract_class = true

  def human_enum_name(enum_name, enum_value)
    enum_i18n_key = enum_name.to_s.pluralize
    I18n.t("activerecord.attributes.#{model_name.i18n_key}.#{enum_i18n_key}.#{enum_value}")
  end
end

This little method allows us to use the locale files in the following way:

en:
  activerecord:
    attributes:
      conversation:
        statuses:
          active: Active conversation
          archived: Archived conversation

Then, we can simply use the class method human_enum_name:

Conversation.human_enum_name(:status, :active) # => "Active conversation"

And with an instance:

conversation = Conversation.create(status: :active)
Conversation.human_enum_name(:status, conversation.status) # => "Active conversation"

Actually, I use this approach quite often, so I've built a little gem called human_enum_name. Feel free to use it!

And, if you want to manage your YAML translation files in a clever way, check out my project LocaleData.com.

Original article: Localize your Rails enums

How to expose your local development server

Juraj Kostolanský — Mon, 11 Feb 2019 08:00:00 +0000

If you need to expose your local development web server to the Internet, there are multiple tools you can use. For example: ngrok, pagekite, forward, localtunnel, ...

Or, you can build your own for free, if you have your own VPS. It can support SSL and works behind a NAT. Here is how.

Server setup

You will need an nginx server installed on your VPS and a domain name pointing to it, like dev.example.com.

Then add this (or similar) nginx configuration:

server {
    listen 80;
    server_name dev.example.com;
    rewrite ^ https://dev.example.com$request_uri? permanent;
}

server {
    listen 443 ssl http2;

    server_name dev.example.com;

    access_log off;
    error_log off;

    location / {
        proxy_pass http://127.0.0.1:8000;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto https;
        proxy_set_header Host $http_host;
    }

    ssl_certificate /path/to/ssl/dev.example.com/fullchain.pem;
    ssl_certificate_key /path/to/ssl/dev.example.com/privkey.pem;
}

The interesting part is the proxy_pass. It will point dev.example.com to the local port 8000 on the server.

Make sure that the following option is allowed in the /etc/ssh/sshd_config and reload the SSH server if needed:

GatewayPorts clientspecified

Local setup

Now you need to create an SSH tunnel betweeen your local machine and the server.

On your development machine, put this function inside your .bashrc file (or .zshrc if you are using ZSH). In this example, me is a remote linux user for the SSH access into the VPS.

function devtunnel {
    ssh -nNT -R "127.0.0.1:8000:localhost:${1}" me@dev.example.com
}

This will forward the remote port 8000 to the local port specified as an argument; -n prevents reading from stdin, -N means that you do not want to execute remote commands, -T disables pseudo-tty allocation, and -R means reverse port forwarding.

Usage

After this setup, exposing your local development server is easy. If you are a Rails developer, you are running your local web server on the port 3000. To expose it publicly, run this locally:

devtunnel 3000

After that, you (or someone else) can visit dev.example.com to access your local server. Enjoy!

Original article: How to expose your local development server

The unexpected after_commit behaviour

Juraj Kostolanský — Wed, 23 Jan 2019 20:00:00 +0000

Today I've learned something interesting. The new Rails callbacks after_create_commit, after_update_commit and after_destroy_commit can behave in a way I didn't expect.

The after_commit callback is a well-known part of the Ruby on Rails framework. It's called after a record has been created, updated, or destroyed and after the corresponding database transaction has been commited. It's the primary method to use if we want to trigger a background job associated with a record.

class User < ActiveRecord::Base
  after_commit :schedule_welcome_email, on: :create  

  def schedule_welcome_email
    WelcomeEmailJob.perform_later(id)
  end
end

The Ruby on Rails 5 came with some new after_*_commit callbacks. Before it looked like this:

after_commit :action1, on: :create
after_commit :action2, on: :update
after_commit :action3, on: :destroy

And now we can use:

after_create_commit  :action1
after_update_commit  :action2
after_destroy_commit :action3

The problem

Let's say we want to trigger the broadcast method after a record has been created or destroyed. We can try using the new callbacks:

class Comment < ActiveRecord::Base
  after_create_commit  :broadcast
  after_destroy_commit :broadcast

  def broadcast
    BroadcastJob.perform_later(id)
  end
end

That looks good! The after_destroy_commit works as expected. However, for some reason, the first after_create_commit is never triggered. But why?

The reason

Let's take a look at the source code of the after_create_commit method:

def after_create_commit(*args, &block)
  set_options_for_callbacks!(args, on: :create)
  set_callback(:commit, :after, *args, &block)
end

As you can see, these methods are effectively aliases for the old after_commit callback with the :on option specified. And subsequent after_commit declarations override former declarations for the same method. That can be pretty surprising!

The solution

To solve this issue, we can use the old callback. The :on option supports an array of multiple life cycle events, so the solution is simple and looks like this:

after_commit :broadcast, on: [:create, :destroy]

Original article: The unexpected after_commit behaviour

The Story Behind LocaleData

Juraj Kostolanský — Fri, 09 Nov 2018 11:32:55 +0000

Identifying a pain point

I work as a software engineer for a young ed-tech company that develops a worldwide Ruby on Rails SaaS app. We collaborate with our external translators who help us localize it into multiple languages. As you may already know, the Rails framework uses YAML file format for storing translations. This is a really nice default approach, but it has some cons, too. YAML files are best for computers and good-enough for programmers, but they are terrible for collaboration with non-technical translators. So we tried to find a simple and cheap solution.

Rails allows developers to use multiple different backends to store their translations. Therefore, a possible solution could be to use an SQL database for this purpose and to build a simple web interface for editing our content. But we have a problem here. We deploy an isolated instance of our app for every customer. That means every instance has its own database and we would need (A) to edit translations for each instance or (B) to introduce higher coupling into our infrastructure — by syncing translations, using a master database, or something like that. None of this was a suitable solution.

A better approach would be a micro-service for editing our existing YAML files. What we really need was a centralized interface for our external translators. We wanted to (1) upload our YAML files, (2) invite some translators, (3) let them do their job, (4) download the result. We were able to find and try such solutions. Unfortunately, they provide really cool features which we didn’t need, therefore they were too pricey for us.

I’ve been always interested in micro-services built as side hustles. I’ve read the “build something for yourself, be your first customer” advice too many times. This seemed to be a hot candidate for such an experiment. So, okay, challenge accepted! I bought the localedata.com domain in May 2017 and started working on it.

Building an MVP

After I decided to solve this issue on my own in May 2017, it was time to choose the right technologies and start typing the code. I picked the stack that I was already very familiar with, nothing fancy:

Ruby on Rails as an application framework (obviously)
Turbolinks with some vanilla JS for a better user experience
PostgreSQL as a database
Redis with Sidekiq for background jobs

There isn’t too much to talk about. The core of this project is pretty straightforward and I’ve been working on it alone. I haven’t use any wireframes or mockups. I just put a bunch of tasks and ideas into my Trello board and…

It takes about six months from the first project idea to the first public deployment. This seems to be a very long time, however, my productivity during the summer was quite low (among other things, I was walking the beautiful Wales Coast Path). It could probably be finished much sooner, but there was no reason to rush it.

Besides programming LocaleData, I’ve been working full-time for the company I am employed in. Sometimes it was quite hard to push myself to work on a side project in the evening, but the key seems to be just to do something regularly, preferably daily. New feature, minor enhancement, or even a small bug fix. Five minutes or five hours. That doesn’t matter. I just need to keep that momentum going, to see some real progress. I’ve found that this really helps me stay motivated.

I publicly deployed the first release in November 2017. I canceled the third-party subscription our company was using, uploaded our translation files into LocaleData and introduced this new tool to my teammates and to our translators. Long story short: They like it even better than the previous pricey solution (and, to my surprise, not only for its zero-price).

Do you want to try?

Just go to www.localedata.com and create your new account. If you have any questions or if you miss a feature, just send me a message.

Original article: The Story Behind LocaleData.com