DEV Community: Alberto Hernandez Cerezo

Configure Stimulus with esbuild and Babel — Rails & Javascript

Alberto Hernandez Cerezo — Sun, 26 Feb 2023 16:18:08 +0000

What will you learn?

Understand how Javascript works in Rails. Learn to configure a Rails project with Stimulus, using esbuild to bundle your code efficiently and effectively. Set up a simple script to automatically load your Stimulus controllers and bundle your Javascript code when editing your code.

Introduction

Rails server-side HTML rendering enables the creation of powerful web interfaces without using javascript. For more complex cases which require it, Rails provides Stimulus, a minimalist framework to embed javascript in your views.

Both Rails and Stimulus aim to minimize your project's javascript code. This is desired since javascript significantly increases maintenance costs due to its web browser execution environment.

With multiple web browsers in the market, each with multiple versions, running in numerous OS of different devices, ensuring a javascript application works as expected on each platform is challenging. Compared to a Rails server, running on a specific OS on specific machines where developers can observe and diagnose errors, the javascript run environment is out of the developers' control.

To mitigate this problem, the javascript ecosystem provides tools to transpile and bundle code to maximize code compatibility and optimization. We are talking about bundlers such as esbuild and transpilers like Babel.

However, the more we dig into these solutions, the more we overload our stack with javascript code. On top of that, these tools are complex and require a good understanding of their purpose, how they work, and how to configure them properly. Testimony of this is the length of the present post.

This document is designed with newcomers in mind. To properly configure your javascript stack, it is important to understand its functioning and parts. Because of that, we will spend a significant part of the post explaining all these agents in an easy-to-understand way; to then put the theory into practice with the real configuration of a Rails project.

If you are already familiar with bundlers and are just seeking a solution to integrate Stimulus and esbuildin your project, I recommend you to jump to the solution section or even to the PR links where the final configuration is available and can be easily integrated into any other Rails project.

The Basics

How does Javascript work in Rails?

A Rails web application is an application with a web-based interface. Web interfaces are rendered in web browsers.

When we open a Rails application in our web browser, the navigator performs HTTP requests to Rails backend server in HTML format. In response, the server returns an HTTP response in HTML format. This response is rendered by the web browser, displaying the application interface.

The server generates the HTTP response content dynamically based on the request parameters. It consists of a standard HTML document. This document comprises a body section, which defines the layout of the interface (what we see on the screen), and a head section.

The head contains document meta information (name, viewport configuration, etc.) and links. These links point to static assets the application requires to render the document correctly. For example, the .css code to style our HTML or, more relevant for us, the javascript code to run our front-end business logic. Per each link present, the browser will perform an additional HTTP request to download each asset and load it. Javascript assets (.js files) are loaded by executing their inner code in a similar way we could do manually by copying the file content and pasting it into the javascript console of the web browser.

Unlike server HTML responses, asset requests are static: they always return the same content. This allows us to generate our application asset responses (the asset files) once and use them in all asset requests (this is why asset files are precompiled, generated, before launching a Rails application).

The Rails asset pipeline generates the assets. The process is simple: the pipeline processes all the asset files in the assets folder, generates the final asset files, and places them under the public folder. This folder is, as the name says, public. Web browsers can access the resources available via the link tags in the HTTP responses' head.

The Javascript code of a Rails application is located in two folders: the app/javascript folder and the node_modules folder, which contains our javascript project dependencies. To serve our javascript code as a .js asset, we must pack all this code in a set of self-contained files that the asset pipeline can process and the web browsers load. We need to bundle our javascript code.

Javascript bundling in Rails

Quoting the definition from Next.js: Bundling is the process of resolving the web of dependencies and merging (or ‘packaging’) the files (or modules) into optimized bundles for the browser. We can bundle our javascript projects with javascript bundlers.

Most of the bundlers work similarly. They provided an API with methods to generate bundles and optional parameters for customized outputs. Depending on the tool you use, these sets of parameters might vary. Most of them can also extend their capabilities with additional plugins.

When selecting a bundler for your project, there are two key aspects to consider:

Speed: the faster the bundler bundles your code, the better. In production environments, where the code does not change, speed is irrelevant (you bundle your code once). In development, however, you want to test changes in your code as fast as possible. Since you need to re-bundle all javascript code on each change for it, having a fast bundler is critical for a smooth development experience.
Optimization: bundlers use multiple optimization techniques to keep your bundle small and to take the most advantage of web browser caching capabilities. The bundle is an asset your application will serve in every HTTP request: smaller bundles and the use of caching lead to faster load times and smoother end-user experiences per request.

Rails applications are bundler-agnostic. They do not care how you bundle your javascript code. It just expects whatever comes from the bundler to be placed under app/assets, so the asset pipeline processes it. We can see this in the official jsbundling-rails gem, which consists of scripts to install different bundlers and configure a default npm build command to generate our bundles—no interaction whatsoever with the Rails configuration. This black-box bundler logic allows us to change and update our bundler system without tuning any other aspect of our Rails application.

So to bundle our javascript code in Rails we need to install a bundler, run it, and ensure the resulting bundle is placed under any of the asset paths configured in our application so that the asset pipeline can digest it. Understanding this, the question is now, what do we bundle?

Bundling Stimulus

Stimulus is a minimalist Javascript framework. It can be classified in the same category as other frameworks, such as React, Vue, or Ember. These ones are complex tools with multiple modules and utilities to build complex web applications: components, routers, HTML template engines, services, etc. Stimulus omits all these framework parts and relies on already rendered HTML (server-side HTML), and HTML data attributes to attach simple javascript logic to it.

It works with controllers, javascript classes that can be attached to any part of our page. When a Stimulus application is launched, controller classes must be registered with an identifier. The application will observe changes in the page HTML. When an element includes a data-controller attribute with a specific identifier, the application will identify the corresponding controller class and generate an instance of it attached to the element. This approach makes it significantly easy to reuse our javascript logic in our views since it is decoupled from our HTML (in comparison with traditional component-oriented approaches from other frameworks).

Since Rails renders HTML in the backend, Stimulus is the best solution to sprinkle our views with simple yet powerful javascript code. Controllers are placed under the app/javascript/controllers folder. To run the Stimulus application, we define a javascript initialization script. This is usually done under app/javascript/application.js. This script creates an instance of a Stimulus application and then registers a list of imported controllers.

We must pass this application.js file to our bundler to bundle a Stimulus application since it contains the reference to our controllers and the logic of how the application can be launched. The bundler will go from there through all the imported controllers and their specific dependencies to generate a bundle containing all the required code to run the application in the browser.

And let’s not forget about the transpiler!

We have already introduced all the key concepts to start working with javascript in our Rails application. However, there is still a missing piece in our puzzle to bundle ready-for-production javascript code in our applications: the transpiler.

Many Rails developers consider that the less code you have in your Rails applications, the better. The main reason to think this is that javascript can be hard to test.

Javascript web-browser execution environment requires browser-makers to keep their browsers updated with the latest ECMAScript specification releases. This means that newer (and not so new) released javascript language features might not be compatible with some commercial web browsers or even behave differently from the specification. As a result, developing javascript code that runs in all web browsers can be challenging. Imagine going one by one testing your code not just in all commercial web browsers but in different versions of each one (which might have different implementations of the javascript language).

To solve part of this problem, we can use transpilers. A transpiler rewrites javascript code targeting a specific ECMAScript version. By doing so, we can take advantage of the latest javascript features without compromising the browser compatibility of our code. Transpilers use polyfills, which specify how new javascript features can be translated into javascript code compatible for older versions.

In our case, we will use Babel with CoreJS to transpile our code. This transpiler can be attached to most of the bundlers, running the transpiling process in parallel with the bundling, generating both highly optimized and compatible javascript code.

The Problem

Configure a Rails application with Stimulus, esbuild and babel.

The Solution

esbuild

We will use esbuild in our project. esbuild is a bundler that stands for its speed. It does not have the same level of maturity or popularity as other available bundlers, such as webpacker, and it still misses some useful optimization options (such as code splitting, which is still in the project roadmap at the time I write this post). Still, its growing popularity and blazing-fast bundling times are reasons enough to choose it as our preferred option.

Start by installing esbuild in your project. Open a terminal window, navigate to your Rails project, and install the esbuild package:

yarn add esbuild

Once installed, the esbuild CLI should be available:

# In your terminal window running this command:
esbuild --version                              
# Should return the installed esbuild version number
0.15.10

We can bundle javascript code with esbuild --bundle. It requires two parameters: entryPoints, the list of javascript files we want to bundle, and outdir, the directory path where the bundler outcome will be placed.

# Produces dist/entry_point.js and dist/entry_point.js.map
esbuild --bundle entry_point.js --outdir=dist

The command has multiple optional parameters to optimize the resulting bundle to your needs. You can find the entire list in the official documentation. To properly configure the bundle with the right options, it is important to understand how each customization option works and what your needs are.

For example, minify optimizes your bundle size using different compression techniques, like removing whitespaces from code. In production environments, the minify option is a must-use; we want to reduce the weight of our assets as much as possible. In development, however, removing whitespace leads to code that is harder to read and thus more difficult to debug, so you should not use this option.

A good way to learn about the different options and their outcomes is also by experimenting with them. Let’s start by bundling something simple. We will set up the javascript entry point for our application, consisting of a simple script that prints the current date-time in an infinite loop in the console:

// app/javascript/application.js

// Entrypoint for our Javascript code. It might already exists in your project.
// This is just a simple example to show how bundling works.
console.log('Loading Javascript asset')

function printTime() {
  console.log(new Date())
}

function setTimeLogger() {
  setInterval(() => printTime(), 1000)
}

console.log('Starting script')
setTimeLogger()

And now, let’s bundle it. We will point the bundle directory to the app/assets folder, as the Rails asset pipeline requires:

esbuild --bundle app/javascript/application.js --outdir=app/assets/builds --minify

  app/assets/builds/application.js      200b 
  app/assets/builds/application.js.map  536b 

⚡ Done in 14ms

The resulting bundle looks like this:

(()=>{console.log("Loading Javascript asset");function o(){console.log(new Date)}function e(){setInterval(()=>o(),1e3)}console.log("Starting script");e();})();

The minify option removes all line breaks and whitespace and renames functions and variables with shorter names. This results in a significantly smaller bundle than the one generated without the minification option:

esbuild --bundle app/javascript/application.js --outdir=app/assets/builds

  app/assets/builds/application.js  278b 

⚡ Done in 4ms

At the same time, this refactoring makes it extremely hard to read the code and map variables and methods from the bundle to the actual code. If there are bugs in our code, error messages will be hard to understand and trace back in our code. Remember, in case of doubt, to play with the different esbuild parameters and inspect the resultant bundle when exploring new bundle optimizations in your project.

The bundle generated is ready to be processed by the pipeline and loaded by the web browser. If you want to test how it works on the browser, you can always copy its code and paste it into the browser console:

To automate bundling, we can define a npm build command that runs esbuild.

// package.json
...
"scripts": {
    ...
    "build:esbuild": "esbuild --bundle app/javascript/application.js --outdir=app/assets/builds --minify --sourcemap",
    ...
...

This makes it possible to bundle our code by running the command yarn build:esbuild in our terminal, which is way better than typing the previous esbuild command. Still, this bundling system is not optimal. The CLI approach can lead to very long bundle commands, hard to read and maintain. Instead, we can use the esbuild javascript API to define our bundle script in javascript, way easier to work with:

// config/esbuild.mjs

import path from 'path'
import esbuild from 'esbuild'
import rails from 'esbuild-rails'
import babel from 'esbuild-plugin-babel'

esbuild
  .build({
    bundle: true,
    // Path to application.js folder
    absWorkingDir: path.join(process.cwd(), 'app/javascript'),
    // Application.js file, used by Rails to bundle all JS Rails code
    entryPoints: ['application.js'],
    // Destination of JS bundle, points to the Rails JS Asset folder
    outdir: path.join(process.cwd(), 'app/assets/builds'),
    // Enables watch option. Will regenerate JS bundle if files are changed
    watch: process.argv.includes('--watch'),
    // Split option is disabled, only needed when using multiple input files
    // More information: https://esbuild.github.io/api/#splitting (change it if using multiple inputs)
    splitting: false,
    chunkNames: 'chunks/[name]-[hash]',
    // Remove unused JS methods
    treeShaking: true,
    // Adds mapping information so web browser console can map bundle errors to the corresponding
    // code line and column in the real code
    // More information: https://esbuild.github.io/api/#sourcemap
    sourcemap: process.argv.includes('--development'),
    // Compresses bundle
    // More information: https://esbuild.github.io/api/#minify
    minify: process.argv.includes('--production'),
    // Removes all console lines from bundle
    // More information: https://esbuild.github.io/api/#drop
    drop: process.argv.includes('--production') ? ['console'] : [],
    // Build command log output: https://esbuild.github.io/api/#log-level
    logLevel: 'info',
    // Set of ESLint plugins
    plugins: [
      // Plugin to easily import Rails JS files, such as Stimulus controllers and channels
      // https://github.com/excid3/esbuild-rails
      rails(),
      // Configures bundle with Babel. Babel configuration defined in babel.config.js
      // Babel translates JS code to make it compatible with older JS versions.
      // https://github.com/nativew/esbuild-plugin-babel
      babel()
    ]
  })
  .catch(() => process.exit(1))

This is an example of our project’s esbuild bundle configuration. As you can see, we can use javascript to add comments and keep the bundle options better organized. Via process.argv we can use different option values based on custom build parameters. This can be useful to define multiple configurations in the same code (for example, enabling or disabling minification in development or production). Please make sure you define your bundle configuration in a .mjs file. This is required since the script uses the import method to require the esbuild dependencies.

With the javascript code set, we can now update our build script in our package.json file:

// package.json
...
"scripts": {
    ...
    "build:esbuild": "node config/esbuild.config.mjs",
    ...
...

You can now run yarn build:esbuild again to check the configuration works properly and the code is bundled.

Babel

Babel is the most popular Javascript transpiler. The transpiler refactors our bundle code to maximize its compatibility with web browsers. To install Babel, go to your project folder, open the terminal and run:

# in your terminal window:
yarn add --dev core-js @babel/core @babel/preset-env esbuild-plugin-babel

We listed multiple packages here. Let’s quickly describe their purpose:

@babel/core: the official Babel npm package.
esbuild-plugin-babel: this package allows us to integrate the Babel transpiling process in the esbuild pipeline. We already imported and used this package in our esbuild configuration file:

  // config/esbuild.mjs
  ...
  plugins: [
        ...
        // Configures bundle with Babel. Babel configuration defined in babel.config.js
        // Babel translates JS code to make it compatible with older JS versions.
        // https://github.com/nativew/esbuild-plugin-babel
        babel()
      ]
  ...

@babel/preset-env: Babel uses repositories of plugins and configuration options to transpile javascript code. These repositories are called presets. The preset-env module is an official Babel preset for compiling ES2015+ syntax.
core-js: this is a special kind of preset, a polyfill. The polyfill contains a library of javascript features and defines the code refactor needed for each one to make it compatible with older javascript versions.

Since we already told esbuild to use Babel, we need to configure Babel. To do so, create a babel.config.js file in your project root and import the defined presets:

const presets = [
  [
    // Javascript 2015+ syntax transpiler
    '@babel/preset-env',
    // Javascript Polyfill
    {
      useBuiltIns: 'usage',
      // Make sure version number matches the Core.js version installed in your project
      corejs: '3.28.0'
    }
  ]
]

module.exports = {
  presets,
  // Fixes Core-JS $ issue: https://github.com/zloirock/core-js/issues/912
  exclude: ['./node_modules']
}

Presets can be configured with multiple optional parameters. Similar to what we mentioned for esbuild, we encourage you to read the docs and find the setup that adapts best to your needs.

To finish the configuration, we must tell Babel the web browsers we want our code to be compatible with. Create a new file called .browserslist.rc and specify your desired browser compatibility. You can find all the possible ways to specify the list of compatible browsers in the official docs. We will use a simple configuration for our project:

# .browserslist.rc

# Babel Preset configuration
# --------------------------
# Defines web-browser compatibility parameters for Babel to transpile your JS code.
# This configuration is used by babel.config.js.
# More information in here.
# https://github.com/browserslist/browserslist

# Support browsers with a market share higher than 5%
>10%

With everything set, you can run esbuild and see how Babel transpiles your code. To do a little experiment, we will update our script with a new method that uses a technique called destructuring assignment to define its parameter. Its syntax is not available in some older javascript versions:

// app/javascript/application.js

...
// https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/Destructuring_assignment
function destructuringAssignment({ param }) {
  return param
}
...

Now compiling our bundle, we can see how the method is refactored. The new code does the same destructuringAssigment does. However, its syntax is now compatible with all web browsers which do not implement this kind of assignment:

(() => {
...
  function destructuringAssignment(_ref) {
    let {
      param
    } = _ref;
    return param;
  }
...
})();
//# sourceMappingURL=application.js.map

Stimulus

Rails provides multiple commands and generators to create applications with Stimulus pre-installed or install the framework from scratch. In this post, we will configure stimulus from scratch without using those helpers.

Let’s start installing the stimulus package. Open a terminal window in your project folder and execute:

yarn add @hotwired/stimulus

Let’s add a sample controller. We will use the example from the official Stimulus documentation. In a Rails application, Stimulus controllers are placed under the app/javascript/controllers folder. Each controller file must be suffixed with_controller.js (it is a convention).

// app/javascript/controllers/hello_controller.js
import { Controller } from '@hotwired/stimulus'

export default class extends Controller {
  connect() {
    this.element.textContent = 'Hello World!'
  }
}

Now we need to create the launch script. This script is responsible for creating a Stimulus application instance, registering all our project controllers, and launching the application in the web browser. The official Stimulus documentation also provides an example of how this script should look like. Since the script launches the application, we should place the code in application.js:

// app/javascript/application.js
import { Application } from "@hotwired/stimulus"

import HelloController from "./controllers/hello_controller"

// Creates & launches a Stimulus application instance
window.Stimulus = Application.start()

// Registers controller in Stimulus application instance
Stimulus.register("hello", HelloController)

Each controller in our project must be registered in the Stimulus application instance with an identifier. We chose the identifier of a controller by its file and path name, excluding the _controller.js filename suffix.

This script has multiple problems. Registering our controllers one by one can be tedious: any time a new controller is created, deleted or renamed, we need to update the file. On top of that, the idea of manually writing the identifiers can lead to multiple problems: what happens if someone makes a type in the identifier? What happens if different developers use different naming conventions?

We can solve this problem by automating the controller registering process with a script: the script could look into the controllers folder, get the list of controller files, import the respective classes defined, and register them with the identifier corresponding to the file name. Luckily, the esbuild-rails package already provides the resources to implement this script.

Installing the package first:

yarn add esbuild-rails

And then using its esbuild plugin in our bundle configuration:

// config/esbuild.mjs
...
plugins: [
      ...
      // Plugin to easily import Rails JS files, such as Stimulus controllers and channels
      // https://github.com/excid3/esbuild-rails
      rails()
    ]
...

We can now define our script to import our controllers automatically:

// Entry point for the build script in your package.json
import '@hotwired/turbo-rails'
import { Application } from '@hotwired/stimulus'

// General Controllers
// -------------------
// import all Stimulus controller files under the controllers folder
import controllers from './**/*_controller.js'

// Auxiliary Methods
// -----------------
// Infer Stimulus controller name from its file
function controllerName(defaultName) {
  const namespaces = [
    ...new Set(
      defaultName
        .split('--')
        .filter((ns) => !['..', 'controllers'].includes(ns))
    )
  ]
  return namespaces.join('--')
}

const application = Application.start()

// Set flag to true to get debbug information in the web browser console
application.debug = true
window.Stimulus = application

controllers.forEach((controller) => {
  Stimulus.register(controllerName(controller.name), controller.module.default)
})

Bundling our javascript code, we can see how the controllers are attached to the bundle, indicating we successfully integrated Stimulus in our Rails application:

// app/assets/builds/application.js
...
// We can see our controllers & the initialization script bundled
  application.debug = true;
  window.Stimulus = application;
  controller_default.forEach((controller) => {
    Stimulus.register(controllerName(controller.name), controller.module.default);
  });
  controller_default2.forEach((controller) => {
    Stimulus.register(componentControllerName(controller.name), controller.module.default);
  });
})();
...

Finally, copying the bundle file content into our web browser console and with the application.debug flag set to true; we can see how the Stimulus application is launched. If then, we type window.Stimulus in the console, we can access the application instance object:

Last touches

We already explained how to bundle our javascript code. But we are missing the last part: linking it in our HTML.

To attach a link to our application.js asset we need to use the javascript_include_tag helper in app/views/layout/application.html. This file defines a wrapper for all the views in our application. By placing the javascript helper in the head section, we make sure every HTML formatted HTTP response includes in its head a link to the application javascript asset:

app/views/layout/application.html.haml

head
  ...
  = javascript_include_tag("application", media: "all", "data-turbolinks-track" => true)
  ...

The helper renders a link tag pointing to the corresponding .js asset. By setting the name parameter to application, we point to our generated bundle.

Once this is set, you can start developing and testing your javascript logic. Remember to run the yarn build:esbuild command with the --watch option. This will enable the esbuild watch flag, which triggers automatic bundles when you change your controller files.

yarn build --watch --development

To deploy your application, you need to run the asset pipeline before launching your Rails application:

# In a terminal window
bundle exec rails assets:precompile

The command will try to bundle the javascript code by running the build script. Since we only defined the script for development, we need to add the missing one for production, so the pipeline can generate the bundle properly:

# package.json
"scripts": {
    ...
    "build": "yarn build:esbuild --production",
    ...
...

Running the pipeline, we will see the command being executed, the bundle being generated and placed under the public folder, from which web browsers will request the asset:

bundle exec rails assets:precompile
yarn install v1.22.19
[1/4] 🔍  Resolving packages...
success Already up-to-date.
✨  Done in 0.29s.
yarn run v1.22.19
$ yarn build:esbuild --production
$ node esbuild.config.mjs --production
Dynamic import can only be supported when transforming ES modules to AMD, CommonJS or SystemJS. Only the parser plugin will be enabled.

  app/assets/builds/application.js  150.7kb

✨  Done in 1.75s.
yarn install v1.22.19
[1/4] 🔍  Resolving packages...
success Already up-to-date.
✨  Done in 0.34s.
yarn run v1.22.19
$ yarn build:tailwind-config-viewer && yarn build:tailwind
$ tailwind-config-viewer export ./public/tailwind-config-viewer -c ./config/tailwind.config.js
$ npx tailwindcss --postcss -i ./app/assets/stylesheets/application.tailwind.scss -o ./app/assets/builds/application.css -c ./config/tailwind.config.js --minify

Rebuilding...

Done in 838ms.
✨  Done in 3.07s.
I, [2023-02-25T23:36:12.163440 #5705]  INFO -- : Writing /Users/alberto-mac/Desktop/Projects/Albert/Albert/public/assets/application-dd70b2d6bf41af42627f148cd9a084b6c0a0b0961780833472ec6faf19b199e7.js
I, [2023-02-25T23:36:12.163717 #5705]  INFO -- : Writing /Users/alberto-mac/Desktop/Projects/Albert/Albert/public/assets/application-dd70b2d6bf41af42627f148cd9a084b6c0a0b0961780833472ec6faf19b199e7.js.gz

Conclusions

Let’s try to wrap up all the lessons of this post. Javascript code is embedded in Rails views as assets via link tags in HTML response heads. To turn our javascript code into assets, we need to bundle it first, so Rails can process the resulting bundle in the asset pipeline and serve it to the web browser.

We can bundle our code with javascript bundlers such as esbuild. The bundler merges all our code with its dependencies in optimized files. These optimizations can be configured via different bundling parameters and third-party plugins, allowing us to shrink the size of our bundles and maximize their compatibility with web browsers, among other possibilities.

Finally, with Stimulus, Rails developers can write powerful javascript code with a simple framework that relies heavily on Rails server-side rendering to keep the entire javascript stack of your project as minimal as possible.

Remember always to keep your javascript code to the minimum possible and rely on CSS capabilities for simple UI interfaces. The disparity of javascript implementations present on the different web browsers in the market can make it difficult to build and test code that works in all possible scenarios.

And last but not least: read, experiment, and get familiar with the bundlers, transpilers, and other additional tools you find on your way to configuring your Rails application. It can be an overwhelming task due to all the different technologies involved and how complex they are, but definitely, the effort will pay off.

Code in Github

152 basic application layout by AlbertoHdezCerezo · Pull Request #166 · Albert-Markdown-Editor/Albert

References

Any Questions?

If you have reached this far, thank you so much for reading my article. It has been quite challenging to wrap up so many concepts, tools, and ideas in a single post, and I am pretty sure there is still much I can improve. This is why I keep writing more and more (apart from the fun, of course).

Did you find the post clear enough? Or did you get lost or stuck at any point? As always, feel free to write down in the comments or reach me for any support you need to set up your own bundling process. I will be happy to give a hand if possible.

Best regards, and see you in the next post!

Multistep Form (Wizard) — Rails Recipes

Alberto Hernandez Cerezo — Sat, 11 Feb 2023 13:37:48 +0000

Multistep Form (Wizard) — Rails Recipes

Anatomy of a Multistep Form

A multistep form is a variant of a wizard. A wizard is a usability tool. It represents a task as a set of steps arranged in a prescribed order. Users must complete each step in the given order to achieve its goal.

Multistep forms split complex forms into smaller parts, grouping their fields by different criteria. This results in a less daunting form-filling experience for end users. When combined with instructions, multistep forms significantly increase the ease of use of our applications.

Their main drawback is efficiency. Filling a multistep form usually takes more time than filling a single form (if you are familiar with the process). This is due to the navigation between steps, which does not exist in standard forms.

How you implement this navigation can significantly impact the user experience: will you allow users to navigate back and forth to any step in the wizard or just to the next and previous steps? is previous step information visible in the current step? Bad design choices can lead to slow and frustrating form-filling experiences that discourage users from filling out our form and even using our system.

This recipe will teach you to create reusable and customizable multistep forms in Rails.

The Challenge

In our application we manage Projects. A project is defined by a name and a description. A Project can have multiple Books associated to it. A Book is defined by a title and a description. We want to implement a multistep form to create Projects. The first step will require the project basic information, and the second, optional, information about a Book associated to the Project.

The Recipe

Prerequisites

(Required) Rails **ViewComponent **gem installed: ViewComponent wraps Rails views into objects. This makes it easier to test our views and provides additional methods for a better developer experience working with them.
(Optional) *TailwindCSS *PostCSS plugin installed: this CSS framework removes the need to write CSS in our code by providing a powerful set of CSS classes.

Model

In Rails, we use forms to create and update records. These operations are performed via POST, PUT, and PATCH HTTP requests submitted from a single form. Requests are processed following the same process:

Rails Controller layer parses form request data and asks the Model layer to perform the requested operation with the extracted information.
Rails Model layer processes request data, validates it, and performs the requested operation if it is valid. The result of the operation is sent back from the Model to the Controller.
Rails Controller layer requests the View layer to render a response based on the operation outcome. Successful operations render a view of the operation results. Unsuccessful operations render the submitted form with the form request data and error messages.

In a multistep form, operations only succeed when all steps are completed. Unsuccessful operations will re-render the multistep form, displaying the next step to complete. Once the last step is completed and submitted, the operation is then performed.

Our first task is to implement the logic to validate whether all steps in a multistep form are completed.

Rails ActiveRecord::Validations API, included in ApplicationRecord Model classes, defines the valid? method. This method is executed on every creation *and *update operation. It evaluates all Model validation conditions, returning true when all validations pass. Only valid? operations can persist information in the database.

The validation for our multistep form must check the current step index in the form request corresponds to its last step. We need two attributes to track these variables: current_step to store the current step index and total_steps to store the number of steps in a multistep form. Since both the validation and the attributes are the same for all models, we can encapsulate them in a concern that can be reused across our Model classes:

# app/models/concerns/multistep_form_model.rb

module MultistepFormModel
  ERROR_ATTRIBUTE = :current_step
  ERROR_DETAIL = :incompleted_multistep_form

  extend ActiveSupport::Concern

  included do
    # Defines two new attributes in including Model class
    attr_accessor :current_step, :total_steps

    # Adds new validation rule in including Model class
    validate :all_multistep_form_steps_completed
  end

  private

  def all_multistep_form_steps_completed? = current_step == total_steps

  def all_multistep_form_steps_completed
    errors.add(ERROR_ATTRIBUTE, ERROR_DETAIL) unless all_multistep_form_steps_completed?
  end
end

The all_multistep_form_steps_completed validation will be checked in all creation and update operations. Since there might be scenarios where you want to update records without using a multistep form, we need to ensure the validation is only checked when requests are submitted from a multistep form.

# app/models/concerns/multistep_form_model.rb

module MultistepFormModel
  ...
  included do
    # Adds new validation rule in including Model class
    validate :all_multistep_form_steps_completed,
      if: -> { current_step.present? && total_steps.present? }
  end
  ...
end

Adding this if statement, we ensure the validation is executed only when both current_step and total_steps attributes have a value (which will happen when using multistep forms only).

Finally, we can update our models by including our multistep form concern so that we can use them with our multistep form:

# app/models/project.rb

class Project < ApplicationRecord
  ...
  # Now our Project class has current_step and total_steps attributes, and the multistep validation rule
  include MultistepFormModel
  ...
end

Controller

Controllers are responsible for parsing form request data and passing it down to the Model class. To work with multistep forms, we need to update the controller’s strong parameters to ensure the **curret_step **and **total_steps **parameters are parsed and passed to the Model class.

# app/controllers/projects_controller.rb

class ProjectsController < ApplicationController
  ...
  private
  ...
  def project_params
    params
      .require(:project)
      .permit(:name, :description, :total_steps, :current_step, books_attributes: [:title, :description])
  end
  ...
end

View

Let’s decompose a multistep form in ViewComponents. A multistep form is a form split into multiple parts (steps). In terms of components, this can be modeled as a MultistepFormComponent whose content is composed of multiple StepComponents.

Each step contains a set of form fields. A step is completed when all step-required fields are filled with valid data. A **StepComponent **renders the fields corresponding to a multistep form step and provides a simple interface to know, given the form data, if the step it represents is completed.

We previously talked about the valid? method and how it determines whether form data is valid. In addition to this behavior, **valid? **also updates the **Model **object with a dictionary containing validation errors. It can be accessed via model.errors, which returns an ActiveModel::Errors object.

To determine if a StepComponent is completed, we can use the form.object.errors object. Given a step with a set of input fields, we can confirm the step is completed if the form.object.errors object does not include any error associated with the step fields.

# How to check whether a step is completed.
def completed?(form_object)
  input_attributes.none? { |attr| form_object.errors.key?(attr) }
end

This method also reveals the information a step needs to be rendered:

An ActionView::Helpers::FormBuilder object to render the step fields and fetch the form data via the .object method.
A list of the input_attributes fields it renders.
Not revealed from the method but obvious, we also need to know the index of the step in the multistep form.

With this in mind, we can write down our StepComponent class:

# app/components/common/multistep_form_component/step_component.rb

module Common
  class MultistepFormComponent::StepComponent < ApplicationComponent
    class << self
      def title = raise "StepComponent #{name} does not define title method"

      def input_attributes = raise "StepComponent #{name} does not define list of input attributes"

      def completed?(form_object)
        input_attributes.none? { |attr| form_object.errors.key?(attr) }
      end
    end

    attr_reader :multistep_component, :index, :form

    def initialize(multistep_component:, index:, html_attributes: {})
      @index = index
      @form = multistep_component.form
      @multistep_component = multistep_component
      super(html_attributes:)
    end

    def call
      raise "
        #{self.class.name} is a StepComponent. StepComponents are abstract by default.
        Components inheriting from it must define its own template or call method.
      "
    end

    def current_step? = index == multistep_component.current_step

    def completed? = self.class.completed?(multistep_component.form.object)
  end
end

We slightly changed our previous specification and required a MultistepFormComponent instance instead of a FormBuilder object to initialize the component. This will allow our StepComponents to access additional information from the parent component.

The **StepComponent **is an abstract component. It is not designed to be rendered but extended by other StepComponent classes. For our Project form steps, we will define two new components, each one for each steps we want to render:

# app/components/common/projects/form_component/project_details_component.rb

module Common
  class Projects::FormComponent::ProjectDetailsComponent < Common::MultistepFormComponent::StepComponent
    class << self
      def title = "Project Details"

      def input_attributes = %w[name]
    end
    ...
  end
end

# app/components/common/projects/form_component/project_deliverables_component.rb

module Common
  class Projects::FormComponent::ProjectDeliverablesComponent < Common::MultistepFormComponent::StepComponent
    class << self
      def title = "Project Deliverables"

      # Example of a step component with nested attributes / nested form fields
      def input_attributes = %w[books.title]
    end
    ...
  end
end

The **MultistepFormComponent **is also an abstract component. It defines the list of StepComponents which compose the multistep form and is responsible for rendering them in the given order. Internally the component renders a form in which steps are placed. To generate the form, the component requires:

The URL where the form data will be submitted.
And the form data itself. This data represents the record we want to create or update. In each iteration of the multistep form cycle, the data will represent the information provided by the user in the previous form submission.

The multistep form must also render hidden fields for the current_step and total_step attributes, ensuring these attributes are present in the submission and triggering the model validation logic:

# app/components/common/multistep_form_component.rb

module Common
  class MultistepFormComponent < ApplicationComponent
    class << self
      def steps = raise "#{name} MultistepFormComponent does not define steps class model"
    end

    delegate :steps, to: :class

    attr_reader :form_url, :back_url, :model, :form

    def initialize(form_url:, back_url:, model:, html_attributes: {})
      @form_url = form_url
      @back_url = back_url
      @model = model
      super(html_attributes:)
    end

    def call
      form_with(url:, model: model) do |form|
        @form = form

        concat(form.hidden_field(:total_steps, value: steps.count))
        concat(form.hidden_field(:current_step, value: current_step_index))

        steps.each_with_index do |step, index|
          concat(render step.new(multistep_form: self, index:))
        end
      end
    end
  end
end

In the code above, the call method renders all steps StepComponents in the given order. When the form is rendered, we store a reference to the form builder object in the @form attribute. This will allow each step component instance to access the builder object to render their form fields.

We can now define our project FormComponent by extending the class and specifying the step of StepComponent classes that define its form:

# app/components/common/projects/form_component.rb

module Common
  class Projects::FormComponent < Common::MultistepFormComponent
    class << self
      def steps = [ProjectDetailsComponent, ProjectDeliverablesComponent]
    end
  end
end

Let’s now define the logic for navigating between steps. Our current component displays all form steps at once. The goal is to show one step at a time and be able to show and hide others’ steps when navigating back and forth in the form.

There are multiple ways to navigate between steps: some only allow navigation forward until all steps are completed; in others, it is possible to navigate back and forth one step and a time, and others even allow displaying multiple previous steps at once. Our multistep form should be flexible enough to implement all possible navigation strategies. However, it should not implement any navigation strategy. We will delegate it to the components extending the MultistepFormComponent class.

We can hide and show steps using simple checkbox and radio HTML inputs. A checkbox is a simple box that can be checked. A radio is a group of checkboxes in which only one can be checked. CSS can target the checked status of these elements via the :checked selector. This allows styling HTML elements based on the status of a checkbox or radio.

We can assign a radio and checkbox input to each step in our form and style the step according to its state so it is only visible when the corresponding input is checked. We will define a new method to wrap each step instance within a tag containing a radio and checkbox to achieve it.

# app/components/common/multistep_form_component.rb

...
  def call
    ...
      steps.each_with_index do |step, index|
        concat(step_wrapper(step, index) { render step.new(multistep_form: self, index:) })
      end
    ...
  end

  def radio_id(step_index) = "#{object_id}_radio_#{step_index}"

  def checkbox_id(step_index) = "#{object_id}_radio_#{step_index}"

  def step_wrapper(step, index, html_attributes: {}, &content)
    tag.div(**html_attributes) do
      concat(
        radio_button_tag(
          radio_id(nil),
          1,
          current_step_index == index,
          id: radio_id(index),
          class: "hidden peer/radio"
        )
      )
      concat(
        check_box_tag(
          checkbox_id(index),
          1,
          current_step_index == index,
          id: checkbox_id(index),
          class: "hidden peer/checkbox"
        )
      )
      concat(capture(&content))
    end
  end
...

The step_wrapper method yields the steps in a div tag together with a checkbox and radio element. Via two new id methods, we assign an id and name to them. This allows targeting the inputs from buttons that can check and uncheck the input they point to when clicked. Finally, we use the tailwind hidden class to hide the inputs and the peer class to style the sibling step container based on the input checked status.

Component extending the MultistepFormComponent can update this method defining common elements present in all steps and using the peers to show and hide steps based on the checked status.

# app/components/common/projects/form_component.rb

...
def step_wrapper(step, index, html_attributes: {}, &content)
  selected = index == current_step_index

  options = (selected ? { "aria-current": "step" } : {})

  super(step, index, html_attributes: options) do
    concat(
      tag.label(
        for: index <= current_step_index ? radio_id(index) : "",
        class: "
          w-full inline-flex items-center px-4 py-2 gap-4 border-2 text-zinc-500 border-zinc-500
          peer-checked/radio:bg-zinc-500 peer-checked/radio:text-white
        "
      ) do
        concat(tag.p(index + 1, class: "h-6 w-6 text-center rounded-full text-white bg-zinc-500"))
        concat(tag.p(step.title))
      end
    )
    concat(
      tag.div(
        class: "
          hidden px-4 py-6
          peer-checked/radio:block
        "
      ) do
        concat(capture(&content))
        concat(
          tag.div(class: "mt-5 w-full inline-flex gap-2 items-center justify-end") do
            concat(link_to("cancel", back_url, class: "px-4 py-2 bg-zinc-500 text-white")) if index == 0
            concat(tag.label("previous", for: radio_id(index - 1), class: "px-4 py-2 bg-zinc-500 text-white")) if index > 0
            concat(form.submit("next", class: "cursor-pointer px-4 py-2 bg-zinc-500 text-white")) if index < (steps.count - 1)
            concat(form.submit("create project", class: "cursor-pointer px-4 py-2 bg-zinc-500 text-white")) if index == (steps.count - 1)
          end
        )
      end
    )
  end
end
...

The project FormComponent extends the step_wrapper method to wrap our steps with a header and a set of navigation buttons. In lines 13 and 24 we can see some peer-checked/radio prefixed CSS classes. They style the step wrapper when the radio checkboxes are checked, revealing it. Our steps are hidden by default and only revealed when the sibling radio checkbox is checked. We use the radio to display one step at a time. If we want to display multiple steps simultaneously, we should target the checkbox instead.

In line 31 there is the logic to navigate a step back in the form. We use the MultistepFormComponent radio_id method to map the label to the radio input from the previous step. When we click the label, the radio button will be checked. This will hide the current step and reveal the previous one.

To navigate forward, we always need to submit the form. If, after form submission, all previous steps are completed, the multistep will render the next step. Otherwise, the form will step back to the incompleted step.

# app/components/common/multistep_form_component.rb
...
def first_incompleted_step_index
  steps.find_index { |s| !s.completed?(model) }
end

def current_step_index
  previous_step = model.current_step&.to_i

  @current_step_index =
    if model.current_step.nil?
      0
    else
      next_step = previous_step + (steps[previous_step].completed?(model) ? 1 : 0)
      [next_step, first_incompleted_step_index].compact.min
    end
end
...

Updating the MultistepFormComponent with these methods provides a way for steps to determine which is the current step in the form. If a specific multistep form defines a different, forward navigation logic, the current_step_index can be overwritten to implement that specific navigation logic. The MultistepFormComponent can use this method to set the status of the radio and checkboxes:

# app/components/common/multistep_form_component.rb
...
def step_wrapper(step, index, html_attributes: {}, &content)
  tag.div(**html_attributes) do
    concat(
      radio_button_tag(
        ...
        # If true radio is checked
        current_step_index == index,
        ...
      )
    )
    concat(
      check_box_tag(
        ...
        # If true checkbox is checked
        current_step_index == index,
        ...
      )
    )
    concat(capture(&content))
  end
end
...

Finally, we need to handle errors. Every time the form is submitted, all validations are checked. This can lead to errors associated with form fields in steps the user did not visit, resulting in error messages displayed before filling out the form. On top of that, the current_step error is always registered. We use this error to trigger the rendering of new steps, but we must avoid displaying its associated error message (the user should be unaware of it).

We will add a new method responsible for removing all error messages related to steps the user did not visit and another to delete the current_step error. To achieve this, we will add a third attribute responsible for tracking the latest step in the form the user filled in so we can determine which steps remain unvisited.

# app/components/common/multistep_form_component.rb
...
def call
    ...
    concat(form.hidden_field(:total_steps, value: steps.count))
    concat(form.hidden_field(:latest_step, value: latest_step_index))
    concat(form.hidden_field(:current_step, value: current_step_index))
    ...
  end
end
...
def latest_step_index
  @latest_step_index =
    if model.latest_step.nil?
      0
    else
      [current_step_index, model.latest_step].max
    end
end
...

Tests

There are two key elements to test for our multistep form: our Model concern and the MultistepFormComponent. Additional unit and system tests for the controllers, project components, and the whole user flow of creating projects are also essential. Still, they do not involve any special configurations as these others do.

To test the MultistepFormModel concern, we might be tempted to test the current_step validations via the Project model. This is not a good practice since it could lead to flaky tests: we must not test concerns via Models that might not include or extend them in the future.

Instead, we can define an auxiliary Model class with the only purpose of testing the concern, and then run the tests with it. This approach has two benefits: testing the concern without relying on Model classes from our business logic and documenting how the concern can be used in our Model classes.

class MultistepFormModelTest < ActiveSupport::TestCase
  class ValidMultistepFormMode
    include ActiveModel::Model
    include MultistepFormModel
  end

  def setup
    @model = ValidMultistepFormMode.new
  end

  # Validations
  # -----------
  test "MultistepFormModel does not apply step validation if total and current attibutes are not specified" do
    assert_nil @model.current_step
    assert_nil @model.total_steps
    assert @model.valid?
  end

  test "MultistepFormModel is valid only when total and current steps are the same" do
    @model.current_step = 1
    @model.total_steps = 2
    refute @model.valid?

    @model.current_step = 3
    @model.total_steps = 3
    assert @model.valid?
  end

  test "MultistepFormModel sets custom error attribute and detail when total and current step attributes are invalid" do
    @model.current_step = 1
    @model.total_steps = 2
    refute @model.valid?

    assert_equal @model.errors.first.attribute, MultistepFormModel::ERROR_ATTRIBUTE
    assert_equal @model.errors.first.detail, MultistepFormModel::ERROR_DETAIL
  end
end

To test the MultistepFormModel we can follow the same approach, defining a set of test components that inherit from the MultistepFormComponent and StepComponent classes just for testing. Here we will check the component sets the correct current_step according to our logic by passing different Model data. Since we do not need to interact with the form (click buttons, run javascript code, etc), we can use unit tests instead of slower system tests.

class Common::MultistepFormComponentTest < ViewComponent::TestCase

  class SampleModel

    include ActiveModel::Model

    include MultistepFormModel

attr_accessor :attribute_1, :attribute_2, :attribute_3

validates_presence_of :attribute_1
validates_presence_of :attribute_2
validates_presence_of :attribute_3



end

class TestMultistepFormComponent < Common::MultistepFormComponent

    class << self

      def steps = [FirstTestStepComponent, SecondTestStepComponent, ThirdTestStepComponent]

    end

  end

class FirstTestStepComponent < Common::MultistepFormComponent::StepComponent

    class << self

      def title = "First Step"

      def input_attributes = %i[attribute_1]

    end

def call
  tag.div(**wrapper_attributes)
end



end

class SecondTestStepComponent < Common::MultistepFormComponent::StepComponent

    class << self

      def title = "Second Step"

      def input_attributes = %i[attribute_2]

    end

def call
  tag.div(**wrapper_attributes)
end



end

class ThirdTestStepComponent < Common::MultistepFormComponent::StepComponent

    class << self

      def title = "Third Step"

      def input_attributes = %i[attribute_3]

    end

def call
  tag.div(**wrapper_attributes)
end



end

def setup

    @component = TestMultistepFormComponent

  end

test "Renders multistep form" do

    render_inline(@component.new(form_url: "test", back_url: "test", model: SampleModel.new))

    assert_selector(".#{@component.component_class}")

  end

test "Renders multistep form steps" do

    render_inline(@component.new(form_url: "test", back_url: "test", model: SampleModel.new))

    assert_selector(".#{FirstTestStepComponent.component_class}", visible: :all)

    assert_selector(".#{SecondTestStepComponent.component_class}", visible: :all)

    assert_selector(".#{ThirdTestStepComponent.component_class}", visible: :all)

  end

test "Selects first step as current step if current_step value is nil" do

    model = SampleModel.new

    model.valid?

    result = @component.new(form_url: "test", back_url: "test", model:).current_step_index

    assert_equal 0, result

  end

  ...

end

Conclusions

Multisteps in Rails are tightly coupled to the controller and model layers. To keep our business logic as separate as possible from this component, we rely on concerns and abstract components. These tools work as a plug & play logic that makes it extremely easy to use multisteps in any part of our application.

The abstract nature of the MultistepFormComponent simplifies the task of implementing forms with different designs and navigation logics, by delegating the implementation of this logic to the components that extend it.

Finally, by writing some generic tests that rely on agnostic test Model and Component classes, we ensure the correct functioning of the multisteps across all our applications and provide a detailed and well-documented example of how developers can make use of the multistep logic in the future.

Code in Github

140 multistep form by AlbertoHdezCerezo · Pull Request #150 · Albert-Markdown-Editor/Albert

References

Any Questions?

If you find yourself lost at any point in the article or try to replicate my solution with no result, you can reach me in the comments section. I will be glad to provide additional information and try to clarify open questions. Soon I will implement additional channels for sharing feedback and questions.

If you have reached this point, thank you for your time and patience. I hope you found my tutorial helpful!

Database Views & Rails Active Record: defining new Model classes out of views

Alberto Hernandez Cerezo — Mon, 23 Jan 2023 19:01:25 +0000

Database Views & Rails Active Record: defining new Model classes out of views

Learn about database views, their use, benefits and how to integrate them in your Rails application. Through a simple scenario you will understand their capability to bring new concepts to your data model based on information already present in it.

Introduction

A Database view is a persistent SQL query that other SQL queries can reference. views make queries reusable, reducing redundancy and complexity in our business logic. Its persistent nature implies their result tables behave as “virtual tables“: they are not persisted in the database but can be referenced anytime from the view. From a conceptual point of view, views can reveal hidden relations in our data model, defined from other existing models and represented by these result tables.

In Rails, database tables are mapped to Model classes. With this in mind, we can define new Model classes out of views virtual tables, consisting of the combination of attributes from multiple models. This allows working with different models simultaneously to perform certain operations more effectively and efficiently than using them separately.

In this post, you will learn about database views, their integration with Rails applications, and how they can help you enhance your data model via virtual tables and their respective read-only Model classes.

The Basics

SQL queries, subqueries, and views

We use the Structured Query Language (SQL) to perform CRUD operations (Create / Read / Update / Delete) in relational databases. Relational databases are made of tables (also called relations).

The SQL language is extremely powerful and allows developers to fetch any data they need from the database. Read operations are performed via SELECT statement queries, and return result tables. Result tables are composed of columns from other table columns in the database (or new ones defined for the specific query) and filled with records from these tables, which also match the set of conditions specified by other SQL statements in the query (WHERE, HAVING, …).

Sometimes, queries for fetching complex information become too long and difficult to handle. The language provides two powerful tools to design simpler queries that can return the same results: subqueries and views.

A subquery is a SQL query nested inside of another query. They split long queries into smaller ones, which are easier to handle. One of the limitations of subqueries is that they only exist in the SQL query in which they are declared. They are “one use“ only.

A view is a reusable SQL query. views are persistent and can be referenced from other SQL queries to build more complex queries, behaving similarly to a subquery. They remove redundancy from our SQL code via reusable encapsulated queries. views definitions (their SQL queries) are persisted within the database, and their result tables are generated each time a query that includes the view is executed. A view variant, materialized views, stores views result tables in the database, acting as a cache, which we will ignore to keep this article simple.

views are a powerful tool. Besides their reusability benefits, they also bring the possibility of discovering new hidden concepts in our data model. While result tables out of SQL queries are ephemeral, views result tables can be accessed anywhere from the database as if they were persistent tables. This opens the possibility of creating new “virtual“ tables and enriching our data model with new concepts resulting from synergies between existing ones.

What are the consequences of this? Let’s briefly explore how the Rails Model layer works first!

Rails, Active Record & SQL requests

Rails applications present a Model-View-Controller architecture. The Model layer is responsible for handling the data exchange with the database. For that, the framework counts with ActiveRecord, Rails ORM implementation. Object-Relational Mapping (ORM) maps connect our business logic classes with their respective database tables representation. For ActiveRecord, this means generating Model class instances out of database result table records and vice versa.

ActiveRecord classes include the QueryInterface module. It consists of high-level methods to fetch information from the database (find, where, includes, joins, etc). We use these methods to build SQL queries that fetch the records we need for our business logic. When we invoke a QueryInterface method, we get a Relation object in return. These objects are composed of an SQL query and the QueryInterface methods. We can see this by calling the .to_sql method, which returns the SQL query associated with the method invoked. Since Relation objects also have QueryInterface methods, we concatenate them with additional methods to assemble more complex SQL queries.

irb(main):002:0> Project.joins(:deliverables).where(name: "My project").to_sql
=> "SELECT \"projects\".* FROM \"projects\" INNER JOIN \"project_deliverables\" ON \"project_deliverables\".\"project_id\" = \"projects\".\"id\" WHERE \"projects\".\"name\" = 'My project'"

ActiveRecord processes the database result table when the SQL query is performed and parses the data to one or more Model class records of the class or object from which we initially invoked the query methods. Rails convention maps the database table to its equivalent Model class via their names.

irb(main):004:0> relation_object = Project.where.not(name: nil)
  Project Load (0.4ms)  SELECT "projects".* FROM "projects" WHERE "projects"."name" IS NOT NULL
=>                                                                                                          
[#<Project:0x000000010c0764f0                                                                               
...

views **are persistent SQL queries, and result tables from these queries can be seen as “virtual tables“. Rails acknowledge this and allow us to map **Model **classes to **views result tables by their respective view name. As a result, we can create new classes out of existing ones. Since they are generated from views, these classes will be read-only, so no creation, update, or deletion operations can be performed on them.

Why `view`-based `Model` classes matter?

Model classes mapped from views are read-only, and their attributes are defined from other Model class attributes. This means view-based Models are useful for handling data from multiple models in single requests. How is it useful for our applications?

There are scenarios where we want to perform a certain operation over a heterogeneous data set. For example, we want to display a list of objects from different Model classes. In these situations, the convention one table one class Rails uses by default is ineffective since it requires us to fetch each set of Model class data separately. Things get even more complicated if we want to extend these lists with new features: how do we filter and sort these lists? what if we want to implement pagination? This would require implementing quite complex ad-hoc logic in our business model logic.

views can lower this burden. By defining a view-based Modelclass from the different Modelclasses we want to include in the list, we could fetch all the list items in one request. On top of that, additional operations such as those mentioned above could also be implemented within the fetch SQL request, resulting in a more efficient and simple list logic.

In conclusion, features requiring read operations over data sets including instances of different Model classes are potential candidates for using views. They significantly simplify the logic of handling records and provide an efficient way to do so.

The only additional cost of using views is their maintenance costs. views are persisted in the database, and as such, they require some maintenance. If the data model changes and tables associated with the view modify its composition, we might need to update our views. This will also require versioning our views and handling these updated versions via migrations, as we do with the rest of the database changes. This is a minor trade-off from views which is still important to mention before working with them.

Let’s now look at an example of how to implement views in Rails.

The Problem

We have an application that manages Projects. A Project can contain multiple Deliverables. Deliverables can be any sort of text based document: a Book, a BlogPost, etc. We want to create a list that displays all Deliverables of a Project, with the possibility to filter and sort them by multiple criteria.

The Solution

Model

This problem has two key concepts: the Project and the Deliverable. A Project contains multiple Deliverables. And what is a Deliverable? It can be anything. Right now, we identified Books and BlogPosts as deliverables, but what if we discover new kinds of deliverables in the future? We should be able to use them as Deliverables too. Each deliverable can be arbitrarily different from the rest. A Book can have pages, an index, different sections, etc. A BlogPost is way simpler, with its content, maybe the URL where it will be available, and not much more. Both of them might share some common attributes, like a title. So how do we define a Deliverable in our data model?

The answer is: we don’t! The Deliverable is just the concept of an artifact associated with a Project, which multiple classes can embody. The data of a Deliverable is its underlying Model class. It does not have specific data that would require us to define a new table in our data model. It is a model composed of data from other models, a perfect candidate to be represented by a view.

Let’s start generating our new Book and BlogPost model classes. We will add some fixtures for testing purposes and update the existing Project class with the corresponding associations.

# Rails generator commands run in terminal to generate new models
bundle exec rails g model BlogPost title:string summary:string release_date:date project:references
bundle exec rails g model Book title:string summary:string release_date:date project:references

# app/models/project.rb

class Project < ApplicationRecord
  ...
  # Associations
  has_many :blog_posts, dependent: :nullify
  has_many :books, dependent: :nullify
  ...
end

# app/models/book.rb

class Book < ApplicationRecord
  belongs_to :project, optional: true
end

# app/models/blog_post.rb

class BlogPost < ApplicationRecord
  belongs_to :project, optional: true
end

# app/test/fixtures/books.yml

# Book Fixture
recipes_book_project_book_1:
  title: How to cook pizza without burning your kitchen
  summary: Learn how make the best pizza in simple and secure steps
  release_date: <%= rand(10..360).days.from_now %>
  project: recipes_book

# app/test/fixtures/blog_posts.yml

# BlogPost Fixture
recipes_book_project_blog_post_1:
  title: "Introducting my new book: How to cook pizza without burning your kitchen"
  summary: Promo post to announce my new awesome book
  release_date: <%= rand(10..360).days.from_now %>
  project: recipes_book

# app/test/fixtures/projects.yml

# Project Fixture
recipes_book:
  title: My Recipes Book
  description: This is the best book about recipes you have ever read

To model our Deliverable class, we will need a view. We will use the popular scenic gem, which provides some useful generators for creating views with their respective migrations, and utilities to handle views versioning.

# gemfile
...
# DB
# --
# Database View Manager
gem "scenic"
...

# Running this command in the console will generate our Deliverables view
bundle exec rails g scenic:view project_deliverables

The view SQL query must return all Book and BlogPost records in our database. It must also handle commonly named attributes, returning the correct value for each data table record. Since it will return a result table as a response, we might also need a way to identify which table records are Books and which ones are BlogPosts.

All this can be easily achieved via a NATURAL OUTER JOIN and the COALESCE method. The JOIN method will merge both tables into a single one with the combined columns of each. We can then select the relevant columns for our final result table via the SELECT statement.

Since some columns share the same name, we can use the COALESCE method to select the corresponding column value for Book and BlogPost records. COALESCE admits an indefinite number of parameters, returning the first one which is not NULL. Joint table Book records will have BlogPost columns set to NULL and vice versa.

# app/db/views/project_deliverables_v01.sql

SELECT
    CASE
        WHEN books.id IS NOT NULL THEN 'Book'
        WHEN blog_posts.id IS NOT NULL THEN 'BlogPost'
    END as kind,
    COALESCE(books.id, blog_posts.id) as id,
    COALESCE(books.title, blog_posts.title) as title,
    COALESCE(books.summary, blog_posts.summary) as summary,
    COALESCE(books.release_date, blog_posts.release_date) as release_date,
    COALESCE(books.project_id, blog_posts.project_id) as project_id
FROM books NATURAL FULL OUTER JOIN blog_posts;

With our SQL view ready, we can now define the associated ActiveRecord Model class. Following the Rails conventions, it must have the same name as the view. And since views are read-only, our class must be read-only too. We will mark Model classes as read only via the readonly? method. Since we will likely define new view-based Model classes in the future, we will encapsulate this logic in a concern that we can later reuse for other new classes to make them view-based.

# app/models/concerns/view_based_model.rb

module ViewBasedModel
  extend ActiveSupport::Concern

  # View Models are made out of database views, so they are read only
  def readonly? = true
end

# app/models/project_deliverable.rb

class ProjectDeliverable < ApplicationRecord
  include ViewBasedModel

  # Ensures only deliverables with valid kinds are created
  enum kind: %w[Book BlogPost]
end

# app/models/project.rb

class Project < ApplicationRecord
  ...
  # Associations
  has_many :deliverables, class_name: "ProjectDeliverable"
  ...
end

All set; we can now test how the Project class instances can fetch all ProjectDeliverable instances associated with it. If we open the Rails console in the terminal (bundle exec rails c), we can see:

irb(main):001:0> Project.first.deliverables
  Project Load (1.1ms)  SELECT "projects".* FROM "projects" ORDER BY "projects"."id" ASC LIMIT $1  [["LIMIT", 1]]
  ProjectDeliverable Load (2.2ms)  SELECT "project_deliverables".* FROM "project_deliverables" WHERE "project_deliverables"."project_id" = $1  [["project_id", "f920b63a-a941-565f-92c4-6af6b0f107ad"]]
=> 
[#<ProjectDeliverable:0x00000001063e4188 kind: "Book", id: "ffbcc7d5-66cb-5095-b0d8-c3e5d2a16e29", title: "How to cook pizza without burning your kitchen", summary: "Learn how make the best pizza in simple and secure steps", release_date: Wed, 13 Sep 2023, project_id: "f920b63a-a941-565f-92c4-6af6b0f107ad">,
 #<ProjectDeliverable:0x00000001065b7730 kind: "BlogPost", id: "6acf2090-67dc-5d76-9dcc-c082ab0bc84b", title: "Introducting my new book: How to cook pizza without burning your kitchen", summary: "Promo post to announce my new awesome book", release_date: Sun, 06 Aug 2023, project_id: "f920b63a-a941-565f-92c4-6af6b0f107ad">]

The class returns all the Book and BlogPost records as ProjectDeliverable instances. Each instance has the attributes corresponding to its respective data table record. We can also take a look at the underlying SQL query and see how small it is in comparison to the view request, illustrating once more how views can simplify queries:

irb(main):004:0> Project.first.deliverables.to_sql
  Project Load (0.4ms)  SELECT "projects".* FROM "projects" ORDER BY "projects"."id" ASC LIMIT $1  [["LIMIT", 1]]
=> "SELECT \"project_deliverables\".* FROM \"project_deliverables\" WHERE \"project_deliverables\".\"project_id\" = 'f920b63a-a941-565f-92c4-6af6b0f107ad'"

Now, if we want to filter or sort ProjectDeliverables we can easily do it by just concatenating the .deliverable method invocation with the respective QueryInterface methods. For example, for sorting, we could use the following:

irb(main):007:0> Project.first.deliverables.order(name: :desc)
  Project Load (0.4ms)  SELECT "projects".* FROM "projects" ORDER BY "projects"."id" ASC LIMIT $1  [["LIMIT", 1]]
  ProjectDeliverable Load (0.4ms)  SELECT "project_deliverables".* FROM "project_deliverables" WHERE "project_deliverables"."project_id" = $1 ORDER BY "project_deliverables"."name" DESC  [["project_id", "f920b63a-a941-565f-92c4-6af6b0f107ad"]]
=>                                                                                                                 
[#<ProjectDeliverable:0x000000010816eeb0, kind: "BlogPost", id: "6acf2090-67dc-5d76-9dcc-c082ab0bc84b", name: "Introducting my new book: How to cook pizza without burning your kitchen", summary: "Promo post to announce my new awesome book", release_date: Sun, 06 Aug 2023, project_id: "f920b63a-a941-565f-92c4-6af6b0f107ad">,                                                             
 #<ProjectDeliverable:0x000000010816ed48 kind: "Book", id: "ffbcc7d5-66cb-5095-b0d8-c3e5d2a16e29", name: "How to cook pizza without burning your kitchen", summary: "Learn how make the best pizza in simple and secure steps", release_date: Wed, 13 Sep 2023, project_id: "f920b63a-a941-565f-92c4-6af6b0f107ad">]

Filtering is a more complex problem, but luckily I already wrote a detailed post about how to implement filters in Rails that you can follow to add filters to your application, and you can find the solution to it in the PR reference available here.

As a final touch, let’s rework our ViewBasedModel concern. Having an enum defining the Model compatible kinds is a pattern we want to do in all our view-based model classes. To document and enforce following this convention, we can extend our concern to define the enum for us and require developers to define a list of compatible kind classes per model. When you have to implement new views in the future, it will be easier to remember how to configure your models.

# app/models/concerns/view_based_model.rb

module ViewBasedModel
  extend ActiveSupport::Concern

  class_methods do
    def compatible_kinds
      raise "
        Model #{name} does not define `compatible_kinds` class method.
        Define method returning an array of strings corresponding to the model
        class names of the tables the view record is composed of. Check example
        in Project and ProjectDeliverable classes.
      "
    end
  end

  # View Models are made out of database views, they are read only
  def readonly? = true

  included do
    enum kind: compatible_kinds.index_with(&:to_s)
  end
end

# app/models/project_deliverable.rb

class ProjectDeliverable < ApplicationRecord
  class << self
    def compatible_kinds = %w[Book BlogPost]
  end

  include ViewBasedModel
end

As we can see, we ended up with a very simple, sustainable, and efficient solution to handle multiple Model classes at once. If, in the future, we want to add new deliverables to our Projects; we need to update the view SQL query with a new version and then set the proper model associations. If our existing ProjectDeliverable models change, we might need to update the query if any changes affect its fields.

Tests

To make sure the new feature works, we need to check Project instance deliverables method returns all associated deliverable records in it (books and blog_posts).

Rather than just using the Project fixtures to test .deliverables returns the same records as .books and .blog_posts together, we will use the ProductDeliverable .kinds method to get the list of available ProjectDeliverable kinds. Then we can use it to fetch all deliverable associations and compare them with the tested method. If new kinds are added in the future, the test will still be valid.

# app/test/models/project_test.rb

class ProjectTest < ModelTestHelper
  def setup
    @sample_project = projects(:recipes_book)
  end

  test ".deliverables return all project deliverables" do
    project_kinds = ProjectDeliverable.kinds.keys.map(&:underscore).map(&:pluralize)
    project_deliverables = project_kinds.map { |k| @sample_project.send(k) }.flatten.map { |d| d.id }.sort

    result = @sample_project.deliverables.pluck(:id).sort

    assert_equal project_deliverables, result
  end
  ...
end

Additional tests can be set to check that each deliverable kind is properly mapped to the ProjectDeliverable instance, but tests carry maintenance costs. The current test is good enough to ensure the system is working and very easy to maintain when new kinds are added to the view. If in the future we detect further issues in the logic, we can always add more tests to grant the correct functioning of it.

Finally, let’s add a test to our new concern. It will serve as a documented example of how to create a view-based model for us in the future:

# app/test/model/concerns/view_based_model_test.rb

require "test_helper"

class ViewBasedModelTest < ActiveSupport::TestCase

  class ValidViewBasedModel < ActiveRecord::Base

    def self.compatible_kinds = %w[Test]

  end

class InvalidViewBasedModel < ActiveRecord::Base

  end

test "ViewBasedModel concern requires included classes to define compatible_kinds class method" do

    assert_nothing_raised do

      ValidViewBasedModel.include(ViewBasedModel)

    end

assert_raises do
  InvalidViewBasedModel.include(ViewBasedModel)
end



end

test "ViewBasedModel concerns defines kinds enum in included class" do

    ValidViewBasedModel.include(ViewBasedModel)

assert ValidViewBasedModel.defined_enums.key?("kind")



end

test "ViewBasedModel concerns kinds enum includes the respective compatible_kind values" do

    ValidViewBasedModel.include(ViewBasedModel)

assert_equal ValidViewBasedModel.defined_enums.values.first.values.sort, ValidViewBasedModel.compatible_kinds.sort



end

end

Conclusion

views are a powerful tool for handling information from multiple relations in our database. Its persistent nature allows us to use them as “virtual tables“. As a result, we can model new result tables out of them to add new concepts to our data model without changing our database structure.

When working with Rails applications, QueryInterface methods are limited to fetching instances of one Model class at a time. With views, we can create new virtual models out of their respective result tables and define new Model classes from them. These classes can contain information from multiple classes, allowing us to overcome the limitation of one Model class per request.

This leads to more efficient use of database queries and a sustainable logic in our data model, which can be easily extended to include new kinds anytime needed. Considering the only additional cost of views is their maintenance (which should be zero if the data model is properly defined), this is one of the key tools to rely on when working with databases and Rails applications.

Code in Github

In response to the feedback gathered from my previous post, and to provide complementary material for better understanding my future post entries, I decided to share all the code associated to the work for each post in simple PR requests.

133 create a list with filter options for project items by AlbertoHdezCerezo · Pull Request #138 · Albert-Markdown-Editor/Albert

References

Effectively Using Materialized Views in Ruby on Rails
PostgreSQL Views Official Documentation
Announcing Scenic — Versioned Database Views for Rails

Filters in Rails with Query Object Pattern

Alberto Hernandez Cerezo — Wed, 16 Nov 2022 10:51:58 +0000

What will you learn?

Use the Query Object Pattern to implement a sustainable filter logic for your models and controllers. Connect your filter logic to your views by building a reusable filter component.

Introduction

Filtering is the process of refining a data set by excluding data according to certain criteria. Filters are frequently used to search for information in large datasets efficiently and effectively.

A filter component is an accessibility tool. It increases the accessibility and visibility of our data by providing a set of predefined options to select information. Designwise it consists of a form with multiple input fields, each corresponding to a filter condition. Filters are used in combination with lists and tables. When a filter form is submitted, the associated list or table is updated, displaying results that match the submitted filter form condition values.

It is important to complement our lists and tables with filters to make it easier for users to navigate through them. Implementing filters in your application will be a recurring issue. Each filter will vary depending on multiple aspects (the data model you work with, the list or table attached to the filter, the user needs, etc.). Thus, it is important to implement a filter logic that is easy to reuse in multiple contexts and with different sets of conditions.

The Basics

How to filter data?

From an implementation perspective, filtering consists of performing multiple custom queries to fetch data by different filter conditions.

Rails ApplicationRecord model classes count with the ActiveRecord query interface, consisting of a set of class methods to fetch records from the database. They work as an abstraction layer to build raw SQL queries easily. We can use these interface methods to implement our custom queries.



class Document < ApplicationRecord
  # (!) Filter methods are class methods too!

  # Filter document by title
  def self.filter_by_title(title)
    where("title ILIKE ?", "%#{title}%")
  end

  # Filter document by author
  def self.filter_by_author(name)
    joins(:author).where("author.name ILIKE ?", "%#{name}%")
  end
end

# SQL query generated by the query interface for one of our filters
Document.filter_by_title("My book").to_sql
# returns: "SELECT \"documents\".* FROM \"documents\" WHERE (title ILIKE '%My book%')"

Query interface methods return a relation. Relations also have access to query interface methods. This allows chaining multiple query method invocations to assemble complex SQL queries easily. To create our filter query, which will include all our custom queries, we just need to chain all our custom queries:



# Our document filter
...
  def self.filter(title, author_name)
    filter_by_title(title).filter_by_author(author_name)
  end
...

This query will always filter documents by their title and author. However, filter conditions are usually optional: when a filter value is missing, its respective condition is not considered for filtering results (for example, if I do not pass a title to the document filter, it should only filter results by author).

We could update our custom query methods, wrapping them in an if statement, so only when the filter value is present is the relation returned. However, returning a nil value instead of a relation will break the chain of custom queries.

The solution to this problem is using the scope class method, also part of the query interface. This method defines new model class methods for retrieving and querying data, such as the custom queries we previously defined. However, scope queries are lenient to nil values, always returning a relation. We can use scopes to make our custom queries conditional while preserving their chain ability.



class Document < ApplicationRecord
  ...
  scope :by_title, ->(title) { where("title ILIKE ?", "%#{title}%") if title.present? }
  scope :by_author, ->(name) { joins(:author).where("author.name ILIKE ?", "%#{name}%") if name.present? }
  ...

  def self.filter(title, author)
    by_title(title).by_author(author)
  end
end

# This will always return a relation
Document.filter(nil, nil)
Document.filter("My document", nil)
Document.filter(nil, "Leonardo Dantes")

Even though we come up with a nice way to define our custom queries and implement our filter query with them, our current approach presents multiple design flaws that need to be addressed:

Models' main responsibilities are exchanging data with the database and implementing parts of the business logic. Filters, on the other hand, answer an accessibility problem and thus are not related to model responsibilities. Models should not be responsible for handling our filter logic.
Filtering is a model-agnostic logic; all models filter results similarly. Placing our filter method in our model implies that each model must define this filter logic, which is redundant.
The definition of custom query scopes in model classes will lead to big model classes overloaded with filter queries.

The Query Object Pattern

The Query Object Pattern is a Design Pattern. It consists of defining a specialized object to encapsulate all information required to perform a specific data query logic (in our case, filter queries, but it can be for any other queries, such as sort queries, for example).

We will use this pattern for our solution, refactoring all our scopes and filter logic to a specialized object responsible for assembling our filter SQL queries.

Delegating our filter logic to a query object solves all the problems previously mentioned:

Our models do not need to know how to filter records; the query object will do. This reduces our models' responsibilities and removes all the filter-related code from them.
Our query objects can share a common filter logic, making our system less redundant.

In the next lines, we will explain how to implement the Query Object Pattern for a simple scenario.

The Problem

We have an application to write Documents. Documents can be organized in Projects (a Project has zero or many Documents, and a Document can belong to a Project). We want to allow users to filter both Documents and Projects by different criteria.

The Solution

Model

We already learned how we can define optional filter queries with scopes. Since all our filter custom queries should be optional, we must add a conditional clause to all of them. To avoid this redundancy, we will define our own filter_scope “scope“ method instead, which will only chain a custom query to the filter query when its value is present.



# model/concerns/filter_scopeable.rb

module FilterScopeable
  extend ActiveSupport::Concern

  def filter_scope(name, block)
    define_method(name) do |filter_value|
      return self if filter_value.blank?

      instance_exec(filter_value, &block)
    end
  end
end

This approach allows us to define a common filter logic for custom queries and share it across all our filtrable models.

Let’s start implementing our query object by moving our scopes out of the model. These methods only work in the context of their respective model class. To extract and use them, we will extend the default model class scope with our filter scopes via the extending method. This method requires a known scope (our class model) and a module with methods to extend the scope. Let’s define our first filter query object with that information:



# /models/concerns/filters/document_filter_proxy.rb
module Filters
  module DocumentFilterScopes
    extend FilterScopeable

    # We define scopes with out new method
    filter_scope :name, ->(name) { where("name ILIKE ?", "%#{name}%") }
    filter_scope :status, ->(status) { where(status:) }
  end

  class DocumentFilterProxy < FilterProxy
    def self.query_scope = Document
    def self.filter_scopes_module = Filters::DocumentFilterScopes
  end
end

Finally, let's create our FilterProxy class. It will define the logic to assemble the filter query for our DocumentFilterProxy and all other future filter proxies:



# /models/concerns/filters/filter_proxy.rb
module Filters
  class FilterProxy
    extend FilterScopeable

    class << self
      # Model Class whose scope will be extended with our filter scopes module
      def query_scope
        raise "Class #{name} does not define query_scope class method."
      end

      def filter_scopes_module
        raise "Class #{name} does not define filter_scopes_module class method."
      end

      def filter_by(**filters)
        # extend model class scope with filter methods
        extended_scope = query_scope.extending(filter_scopes_module)

        # The payload for filters will be a hash. Each key will have the
        # name of a filter scope. We will map each key value pair to its
        # respective filter scope.
        filters.each do |filter_scope, filter_value|
          if filter_value.present? && extended_scope.respond_to?(filter_scope)
            extended_scope = extended_scope.send(filter_scope, filter_value)
          end
        end

        # Final relation with all filter scopes from +filters+ payload
        extended_scope
      end
    end
  end
end

Our solution is very simple, easy to maintain, and to scale. All our query objects share a common filtering logic, and the set of filter queries is now isolated in a separate module. If in the future, you want to create filters for a new model, you can do it by simply creating a new module for your queries and a FilterProxy class with a query_scope and a filter_scopes_module.



# /models/concerns/filters/project_filter_proxy.rb
module Filters
  module ProjectFilterScopes
    extend FilterScopeable

    ...
  end

  class ProjectFilterProxy < FilterProxy
    def self.query_scope = Project
    def self.filter_scopes_module = Filters::ProjectFilterScopes
  end
end

Finally, let's integrate our proxy filter logic into our models. We will expand our models' default ActiveRecord query interface with a new filter_by class method to provide an easy way to filter records (MyModel.filter_by(...)).

Since this interface will be the same for all filterable models, we will define it in a concern:



# /models/concerns/filterable_model
module FilterableModel
  extend ActiveSupport::Concern

  def filter_proxy
    raise "
      Model #{name} including FilterableModel concern requires filter_proxy method to be defined.
      Method should return filter proxy class associated to model.
    "
  end

  delegate :filter_by, to: :filter_proxy
end

# /models/project
class Project < ApplicationRecord
  extend FilterableModel

  class << self
    def filter_proxy = Filters::ProjectFilterProxy
  end
end

# To filter projects now we can do
Project.filter_by(name: "My Personal Project")

Now each model expanding the FilterableModel concern will be able to use the filter_by method. This method will also be available in all our filterable model scopes (for example, Document.all.filter_by), which again will allow us to assemble our filter queries with other queries to create more complex queries easily:



Project.all.includes(:documents).filter_by(name: "My Blog").sort_by(name: :asc)

Controller

Controllers handle HTTP requests and return the proper output in response. Our controllers are responsible for defining the interface to filter records, which consists of defining the filter parameters. Each filterable controller will have its own set of filter parameters, but the logic to define and use them to fetch the results will be the same. Thus we can encapsulate it in a concern for better reusability:



# app/controllers/concerns/filterable_controller.rb
module FilterableController
  extend ActiveSupport::Concern

  def filter(scope, filters = filter_params)
    unless scope.respond_to?(:filter_by)
      raise "
        Controller #{self.class.name} tried to filter a scope of type #{scope.class.name}.
        Scope class does not extend FilterProxy interface.
      "
    end

    scope.filter_by(**filters)
  end

  def filter_params
    raise "FilterableModel controller #{self.class.name} does not define filter_params method."
  end

  included do
    helper_method :filter_params
  end
end

This concern defines a filter method that requires a scope with a filter_by method and a set of filters formatted as a Hash. Each key-value pair in the Hash will represent the name of the filter scope and the value to filter by, respectively.

Usually, controllers work with a unique set of filter_params. However, this is not always the case. For more flexibility using multiple filter parameters, the filter method admits an optional filters parameter.

Last, we defined our filter_params as a helper method. The idea is to make each request filter parameters accessible from our filterable controller views. Thanks to it, our filter components can fetch these parameters and update the form fields according to them. If you define your own filter parameters method, make sure also to declare it as a helper.

Now we can include this concern in our controllers and update our actions with filters.



# app/controllers/projects_controller.rb

class ProjectsController < ApplicationController
  ...
  include FilterableController

  # GET /projects
  def index
    @projects = filter(Project.all)
  end
  ...
  private
  ...
  def filter_params
    {
      name: params["name"],
      description: params["description"],
    }
  end
  ...
end

View

As mentioned at the beginning of this post, a filter component is just a form in which each input field represents a filter condition. Implementing a simple form is a straightforward task, thanks to Rails form_for helper method. There are only two implementation issues to be addressed here.

The first is loading the form with fields filled with the user request filter parameters. Thanks to the previously defined filter_params helper, we can fetch user-request filter parameters and inject them into our filter input fields.

The second is to specify the URL our form filter points to (where the form payload will be submitted). This can be done in two ways.

One is specifying the URL via named URL helpers, so we explicitly set the URL.

The other consist of using the URL helper method url_for. Filter requests attach the filter parameters as query strings to the URL, using the same URL with different parameters to fetch different results on the same page.

The url_for method returns the URL of the user request. Using this as our form URL instead of a specific route name helper, we ensure that the form will always work in all routes it is rendered.



# Solution implemented with ViewComponents & TailwindCSS

module Projects
  class IndexComponent::FilterComponent < ApplicationComponent
    # Tell the component filter_params is a helper and not a component method
    delegate :filter_params, to: :helpers

    def call
      form_with(url: url_for, method: :get, class: "px-8 py-6 inline-flex gap-6") do |form|
        concat(
          tag.div(class: "inline-flex gap-4") do
            concat(form.label(:name, "Name"))
            concat(
              form.text_field(
                :name,
                value: filter_params[:name],
                class: "border-b-2 focus:border-slate-400 focus:outline-none"
              )
            )
          end
        )
        concat(
          tag.div(class: "inline-flex gap-4") do
            concat(form.label(:description, "Description"))
            concat(
              form.text_field(
                :description,
                value: filter_params[:description],
                class: "border-b-2 focus:border-slate-400 focus:outline-none"
              )
            )
          end
        )
        concat(
          form.submit
        )
      end
    end
  end
end

Tests

We must ensure our filter scopes return database records that match the filter criteria and that FilterableControllers process the filter_parameters, when present, and return a response with the corresponding filtered results.

To test our filter scopes, we can create a TestCase per FilterProxy and test per proxy filter scope. Each test will define a filter payload, create a set of records matching the payload and invoke the filter_by method to assert the returned records match the created ones.



# test/models/concerns/project_filter_proxy_test.rb
require "test_helper"

class ProjectFilterProxyTest < ActiveSupport::TestCase
  test ".name filters documents by name" do
    filters = { name: "Test Project #{Time.now}" }

    projects = FactoryBot.create_list(:project, 3, **filters)

    results = ::Filters::ProjectFilterProxy.filter_by(filters)

    assert_equal documents.pluck(:id).sort, results.pluck(:id).sort
  end

  test ".description filters documents by status" do
    filters = { description: "Test project description #{Time.now}" }

    documents = FactoryBot.create_list(:project, 3, **filters)

    results = ::Filters::ProjectFilterProxy.filter_by(filters)

    assert_equal documents.pluck(:id).sort, results.pluck(:id).sort
  end
end

To test our FilterableControllers we will add a new test to their corresponding IntegrationTests to test that requests to actions which filter parameters return filtered results. We need to define a filter payload to pass to our controller request and the corresponding filter proxy and compare that the results they return are the same. This time we will use fixtures for efficiency.



# test/controllers/projects_controller_test.rb
require "controller_test_helper"

class ProjectsControllerTest < ControllerTestHelper
  ...
  test "GET index filters projects" do
    filter_params = { name: projects(:recipes_book).name }

    filtered_projects = Project.filter_by(filter_params)

    get projects_path

    # https://github.com/rails/rails-controller-testing
    result = assigns(:projects)

    assert_equal result.pluck(:id).sort, filtered_projects.pluck(:id).sort
  end
  ...
end

Finally, we must ensure our FilterProxy assembles filter queries as expected, combining all individual filter scope SQL clauses into a single query, ignoring all scopes with a blank value. Instead of testing arbitrarily complex filter request results, we will check the composition of the resultant SQL queries, which is more aligned with the behavior we want to test.

We already mentioned how the to_sql method returns the raw SQL associated with a scope. We will use it to check the resultant filter_by query contains all the filter scope WHERE clauses from the filter scopes present in the test payload, and that there is no trace of filter scopes with empty values.



require "test_helper"

class FilterProxyTest < ActiveSupport::TestCase

  class InvalidFilterProxy < ::Filters::FilterProxy

  end

def setup

    @proxy ||= ::Filters::DocumentFilterProxy

    @proxy_model_class = Document

    @proxy_scopes_module = "Filters::DocumentFilterScopes".constantize

    @proxy_scope = @proxy_model_class.extending(@proxy_scopes_module)

  end

test "#filter_by combines all filter scopes into a single query" do

    result = @proxy.filter_by({ name: "a name", status: :idea })

    assert result.to_sql.include?(sql_filter_clause(@proxy_scope.name("a name").to_sql))

    assert result.to_sql.include?(sql_filter_clause(@proxy_scope.status(:idea).to_sql))

  end

test "#filter_by ignores scopes with empty/blank values" do

    result = @proxy.filter_by({ name: "", status: :idea })

    assert_not result.to_sql.include?("name")

    assert result.to_sql.include?(sql_filter_clause(@proxy_scope.status(:idea).to_sql))

  end

test "#filter_by ignores undefined scopes" do

    result = @proxy.filter_by({ invalid_attribute: "invalid value", status: :idea })

    assert_not result.to_sql.include?("invalid_attribute")

    assert result.to_sql.include?(sql_filter_clause(@proxy_scope.status(:idea).to_sql))

  end

private

# Extracts filter match condition from SQL query, discarding the SELECT part

  def sql_filter_clause(query)

    query.match(/WHERE (.*)/).captures[0]

  end

end

Conclusion

In this post, we have explained how to create a sustainable filter logic via the Query Object pattern.

This pattern allows us to isolate common patterns to fetch data present in our business logic in sustainable modules that are easy to maintain and scale.

We illustrated the pattern's versatility with the case scenario of the filters, but you can find many other patterns in your business logic that can be strong candidates to be implemented with this solution (for example, sorting).

And that's it! I hope you found this article helpful. Please feel free to share any feedback or opinion in the comments, and thanks for reading it.

DEV Community: Alberto Hernandez Cerezo

Configure Stimulus with esbuild and Babel — Rails & Javascript

What will you learn?

Introduction

The Basics

How does Javascript work in Rails?

Javascript bundling in Rails

Bundling Stimulus

And let’s not forget about the transpiler!

The Problem

The Solution

esbuild

Babel

Stimulus

Last touches

Conclusions

Code in Github

References

Any Questions?

Multistep Form (Wizard) — Rails Recipes

Multistep Form (Wizard) — Rails Recipes

Anatomy of a Multistep Form

The Challenge

The Recipe

Prerequisites

Model

Controller

View

Tests

Conclusions

Code in Github

References

Any Questions?

Database Views & Rails Active Record: defining new Model classes out of views

Database Views & Rails Active Record: defining new Model classes out of views

Introduction

The Basics

SQL queries, subqueries, and views

Rails, Active Record & SQL requests

Why view-based Model classes matter?

The Problem

The Solution

Model

Tests

Conclusion

Code in Github

References

Filters in Rails with Query Object Pattern

What will you learn?

Introduction

The Basics

How to filter data?

The Query Object Pattern

The Problem

The Solution

Model

Controller

View

Tests

Conclusion

Why `view`-based `Model` classes matter?