DEV Community: Michael Roudnitski

Query multiple tables easily with Rails and Postgres Views

Michael Roudnitski — Wed, 31 Jul 2024 18:13:39 +0000

As your app grows in size, you'll likely find yourself having a few models with common attributes. You may eventually want to perform queries across these models as if they were one. For example, sort all of your articles, books and videos by when they were created. It would be easy to come up with a brute force way to do this, but luckily we have powerful tools in Rails and Postgres, which we'll talk about in this tutorial.

The Scenario

Imagine you have a content management system with articles, books and videos. You implemented each of these content types as their own model backed by their own database table. Though they have some common attributes like title, description and created_at.

This visual illustrates some attributes of articles, books and videos.

You can see how we would naturally want to develop some features like searching all our content types in one query, or sorting them all by created_at.

You could "brute force" this problem, which might involve querying each of these tables separately and combining the results in Ruby, but this is inefficient. A step up might be to create a UNION query to combine the results at the database level, but this has drawbacks as well.

Using Postgres Views in Rails

We can create an elegant solution using Postgres Views and a single Rails model.

Postges also offers Materialized Views. These actually store the results of their query. They can be useful if you're dealing with data that changes infrequently as they offer performance benefits. But in this tutorial, we'll only explore views.

A view is like a virtual table. We can query it, but it doesn't actually store any data. It can act like a reusable subquery and that's how we'll be using it in this case.

Additionally, thanks to the power of ActiveRecord, we'll only need 7 lines of code to implement the model in Rails. Let's see how it's done.

Setting up the view in our database

1) First, let's install the Scenic gem. It's not required, but it gives us nice ways to create and manage views.



bundle add scenic

2) Use scenic to generate the necessary files. We'll name our view search_results. The scenic gem will create a couple files for us through this command.



rails generate scenic:view search_results
      create  db/views/search_results_v01.sql
      create  db/migrate/20240731153045_create_search_results.rb

The search_results_v01.sql file defines our view, the migration will add the view to our database when we run rails db:migrate.

3) Implement the view. Add the following to db/views/search_results_v01.sql,



SELECT 'Article' as searchable_type, id AS searchable_id, title, description, created_at FROM articles
UNION ALL
SELECT 'Book' as searchable_type, id AS searchable_id, title, description, created_at FROM books
UNION ALL
SELECT 'Video' as searchable_type, id AS searchable_id, title, description, created_at FROM videos;

Then just run the migrations,



rails db:migrate

The above SQL unions our 3 tables and allows us to query against all the common columns as if we had all our articles, books and videos in a single table.

The column names searchable_type and searchable_id are important. They follow Rails naming conventions for polymorphic associations and will make it easy to go from a SearchResult to its associated Article, Book or Video. More on that in the next section.

Set up the SearchResult model

Now that we have a view in our Postgres database to simplify querying these 3 different tables, we need a way to query it with ActiveRecord in our Rails app.

To do this, we can setup a really simple, but powerful model.



# app/models/search_result.rb
class SearchResult < ApplicationRecord
  belongs_to :searchable, polymorphic: true

  def readonly?
    true
  end
end

The belongs_to :searchable, polymorphic: true tells ActiveRecord this model is associated to a "searchable" model. Here is where the searchable_type and searchable_id column names are handy as they follow Rails conventions for polymorphic associations. This single line of code is the key to unlocking lots of built in Rails features.

We also override the #readonly? method to tell ActiveRecord this is not a model we should be expecting to write to with methods like #save and #update. After all, this model is backed by a view, not a table.

Putting it Together

With this setup, we have a really easy and elegant way to perform otherwise messy queries. Here are some examples

Simple queries



# find all content with 'rails' in the title
SearchResult.where("lower(title) LIKE ?", "rails")
=> [#<SearchResult>, ...]

# find the 3 newest pieces of content
SearchResult.order(created_at: :desc).limit(3)
=> [#<SearchResult>, #<SearchResult>, #<SearchResult>]

We can see that for our needs, this design behaves like any regular Rails model backed by a table.

Using our polymorphic belongs_to

Lets also see how we can take advantage of the polymorphic association we set up.



# find and access the newest piece of content
search_result = SearchResult.order(created_at: :desc).limit(1).first
search_result.searchable
=> #<Video>

In this example, a Video was the most recently created piece of content. By calling #searchable on our SearchResult instance, we can get an instance of the Video itself.

This is incredibly useful for rendering partials, or even basic routing, as we could now add a link to the video with #polymorphic_path.



<%= link_to search_result.title, polymorphic_path(search_result.searchable) %>

Conclusion

Using Postgres Views with Rails provides an elegant solution for querying across multiple related models. This approach combines powerful features of both Postgres and Rails, resulting in clean, maintainable code. While it's not a silver bullet for all scenarios involving multiple models, it's a powerful tool to have in your Rails development toolkit. It's also worth noting this can all be achieved with MySQL as well. You can either create your view without scenic, or use the MySQL adapter.

Effortless Integration Testing in Rails: Mock External APIs with RSpec

Michael Roudnitski — Wed, 27 Sep 2023 16:04:47 +0000

It's not uncommon for our web apps to have features that make API calls to external services. In fact, if you have an application that is heavily reliant on external services, it can make integration testing a nightmare for you and your team.

In this guide, I'll show you how we can use built in RSpec features like Instance Doubles and Shared Contexts to make integration testing a breeze in our Ruby on Rails applications.

The Scenario

Imagine we're building a project management web app that heavily relies on GitLab for file management. When a user creates a new project in our app, we have to create a project on gitlab.com using their API.

In general, we never want to call an external API during a test. Remember, our tests are there to test that in a given scenario, our code behaves correctly. It would take longer to run, risk exposing API credentials and create unwanted data on our external services if we talked to them during our tests.

Furthermore, it is safe to assume that GitLab has already thoroughly tested that their create project endpoint works. There's no need for our tests to cover that as well.

So, how do we test this kind of code repeatedly and efficiently, without creating a bunch of test data on gitlab.com?

Instance Doubles

The solution is to use Instance Doubles, which are also commonly referred to as Stubs or Mocks. They allow us to modify the behaviour of our code during a test.

For example, let's consider GitLab's POST https://gitlab.com/api/v1/projects endpoint. In our application, we would use an instance of a Gitlab client to make this request

gitlab_client = Gitlab.client(Settings.gitlab.hosted_client)
gitlab_client.create_project({...})
=> {"id" => 1, ...}

So, let's tell RSpec that every time our code calls gitlab_client.create_project({...}), simply return some predefined hash instead of a real response from gitlab.com.

To implement this, we can simply define our expected response at the top of our spec like this:

# define our hardcoded expected response from GitLab
let(:project) { {"id" => 1, ...} }

# define an instance double
let(:gitlab_client) do
  # override the return for the #project and #create_project methods on every instance of Gitlab::Client
  instance_double(Gitlab::Client, project: project, create_project: project)
end

# Gitlab::Client.new will return our instance double from above, instead of a real gitlab client
before(:each) do
  allow(Gitlab::Client).to receive(:new).and_return(gitlab_client)
end

Reusable Instance Doubles as Shared Contexts

We just saw how we can override our gitlab client during a test. But, it would be really tedious to write this out every time our code makes an API call to some service. Luckily, it's pretty easy to write this once and reuse it across tests, thanks to Shared Contexts.

Building on our previous example, let's define a shared context that will allow us to easily reuse our instance double:

# spec/support/shared_contexts/gitlab_client.rb
RSpec.shared_context "with gitlab client" do
  # define our hardcoded expected response from GitLab
  let(:project) { {"id" => 1, ...} }

  # define an instance double
  let(:gitlab_client) do
    # override the return for the project and create_project methods on every instance of Gitlab::Client
    instance_double(Gitlab::Client, project: project, create_project: project)
  end

  # Gitlab::Client.new will return our instance double from above, instead of a real gitlab client
  before(:each) do
    allow(Gitlab::Client).to receive(:new).and_return(gitlab_client)
  end
end

That's it! To use it, we just need to remember to include it in any tests that interact with the gitlab client

RSpec.describe "/projects", type: :request do
  include_context "with gitlab client" # important

  describe "POST /projects" do
    context "with valid parameters" do
      it "creates a new project" do
        # thanks to the include_context above,
        # we can test our create project endpoint
        # without worrying about GitLab
        expect do
          post projects_url, params: { project: build(:project).attributes }
        end.to change(Project, :count).by(1)
      end
    end
  end
end

Infinite Scrolling with Zero JavaScript [Rails]

Michael Roudnitski — Sat, 23 Sep 2023 17:50:01 +0000

Infinite scrolling is a web design technique that loads content continuously as the user scrolls down the page, eliminating the need for pagination. It is a great way to improve user experience, providing a seamless way for users to consume content. In this article, we'll explore how to implement this feature in a Ruby on Rails application without writing a single line of JavaScript, thanks to the Hotwire library.

With Hotwire, we can leverage Turbo Frames and Streams to facilitate partial page updates, enhancing our user experience with very little code.

The Scenario

Imagine we are building a feature to display a list of user comments on a blog post. We want to continuously load these comments as the user scrolls, eliminating the need for them to click through pages. To achieve this, all we need is two turbo frames:



<%= turbo_frame_tag(:comments_container) do %>
  <%= turbo_frame_tag([:comments, 1], src: comments_path(page_size: 20)) do %>
    <!-- Skeleton Loading Indicator -->
    <div class="flex items-center space-x-4 animate-pulse">
      <div class="rounded-full bg-gray-200 h-8 w-8"></div>
      <div class="flex-1 space-y-2">
        <div class="h-8 bg-gray-200 rounded w-1/2"></div>
      </div>
    </div>
  <% end %>
<% end %>

This code sets up two turbo frames. The :comments_container is where we will be appending our lazy loaded page results, and the inner :comments, 1 frame is where we initially load the first page of comments. It will have an id in the DOM of comments_1.

Next, we need to write the response from our comments controller. For each page we load, we need to render the comments for the current page. Crucially, we also need to set up the turbo_frame_tag to lazy load the next page of results (or render a message if there are no more results to fetch).



<%= turbo_frame_tag([:comments, @page]) do %>
  <!-- Loop through each comment and render it on the page -->
  <% @comments.each do |comment| %>
    <%= render comment %>
  <% end %>

  <%= turbo_stream.append :comments_container do %>
    <!-- Check if there are more pages to load -->
    <% if @page < @total_pages %>
      <!-- Set up the next frame to load when it comes into view -->
      <%= turbo_frame_tag([:comments, @page + 1], src: comments_path(page_size: 20, page: @page + 1), loading: :lazy) do %>
        <!-- Skeleton Loading Indicator -->
        <div class="flex items-center space-x-4 animate-pulse">
          <div class="rounded-full bg-gray-200 h-8 w-8"></div>
          <div class="flex-1 space-y-2">
            <div class="h-8 bg-gray-200 rounded w-1/2"></div>
          </div>
        </div>
      <% end %>
    <% else %>
      <!-- Display a message if there are no more comments to load -->
      <%= @page == 1 && @comments.empty? ? "No comments yet" : "You've reached the end" %>
    <% end %>
  <% end %>
<% end %>

After a bit of scrolling in the browser, if everything is working our DOM should be structured like this:



<turbo-frame id="comments_container">

  <turbo-frame id="comments_1">

  <turbo-frame id="comments_2">

  <turbo-frame id="comments_3">

  <!-- ... -->

</turbo-frame>

Implementation Highlights

1. Lazy Loading:

Imagine you're reading a book, and the next page only appears when you’re ready to read it. That's what loading: :lazy does; it only loads the next set of comments when the user is ready to read them, i.e., when they come into view.

2. Skeleton Loading Indicator:

You know when you load a video and see a blurry gray blob before it fully loads? That's what the skeleton loading indicator does; it provides a visual hint that more content is on its way.

3. Smooth Sailing with Zero JavaScript:

The cool part? We achieved this modern user experience with just a few lines of code, without writing any JavaScript, using turbo frame tags instead.

Conclusion

By leveraging the Hotwire library in Ruby on Rails, implementing infinite scrolling is like a JavaScript-free walk in the park. It's neat, efficient, and developer-friendly. I hope this technique becomes a useful tool in your developer toolbox!

Looking Back at my IBM Internship (2020-21)

Michael Roudnitski — Tue, 04 Jan 2022 19:34:58 +0000

Today I'm picking up where I left off and starting my first day as a full time Software Engineer at IBM so here are 5 things I learned in my 16 month internship (2020-21)👇

⭐️ Show Initiative

Volunteer to work on things and do your best to own entire projects. This means volunteering to plan, design, develop and ship. Obviously this requires a lot of trust from your team so do your best to earn this trust.

🎰 Take Risks

This goes hand in hand with showing initiative. Take risks and you will be rewarded. They don't have to be big risks, just being confident enough to volunteer your ideas in a call will reward you.

🧠 Learn as much as you can from your mentors, and even fellow interns

Observing how your mentors operate day-to-day will give you a blueprint of where you need to be at. This is the target you are aiming for and you should do everything you can to help the other interns in your cohort achieve this, and vice-versa.

🙋‍♂️ Ask Questions... But not too many

At the end of the day as an intern, you're there to learn and enjoy your experience. Asking questions from experienced engineers who have been there before is so valuable. But you also have to remember their time is valuable, so spend time really thinking about your question. "It's not working" won't get you anywhere.

🎉 Have fun with it

This is the most important one. I never expected to work from home all 16 months, so I definitely missed out on lots of office ping pong and lunches. But that doesn't stop you from sending memes over Slack. If an office get-together is planned, attend it!

Lazy Loading With Turbo [Rails]

Michael Roudnitski — Mon, 26 Apr 2021 21:30:09 +0000

The speed of a single-page web application without having to write any JavaScript.

Overview

Turbo allows you to add SPA functionality to your web apps. It comes with three components,

Turbo Drive
Turbo Frames
Turbo Streams

To add lazy loading to our page, we'll be using Turbo Frames. You can install Turbo directly into your Rails application.

Why Use Lazy Loading

Let's say we have a blog and when a reader visits our blog, we want it to feel ⚡️ lightning quick; because we want them to come back someday. One way of accomplishing this is using a very common technique called lazy-loading.

What Is Lazy Loading

Our server can send out a light weight HTML page with loading indicators in place of our blog posts. Only after this light weight page fully loads, we can request our blog posts from the server.

You can see examples of lazy loading everywhere in the wild, here's an example from https://youtube.com where they lazy load their videos.

They're showing you skeleton loading indicators while you wait to see if Mr. Beast bought a country yet.

Using Turbo Frames To Lazy Load

So how can we use Turbo to lazy load our blog posts? It's really simple! We just need to add an action to our controller so we can separate blog post loading from our traditional index action.

Let's start with our erb templates:



# index.html.erb
<h1>Hello World! Check out my blog posts below 👇</h1>
<%= turbo_frame_tag "blog_posts", src: blog_posts_path do %>
  <h2>Hold on a sec while I fetch my blog posts...</h2>
  <%# once blog_posts_path responds, it will replace everything inside this turbo frame tag with blog_posts.html.erb %>
<% end %>

# blog_posts.html.erb
<%= turbo_frame_tag "blog_posts" %>
  <h2>Done loading! Here are my blog posts.</h2>
  <%= @blog_posts.each do |post| %>
    <%= post.title %>
    <%= post.description %>
  <% end %>
<% end %>

And now in our controller:



# GET /blog
def index; end

# GET /blog_posts
def blog_posts
  @blog_posts = BlogPost.all
  render layout: false # Don't forget this optimization!
end

And of course, don't forget to add a GET route to your new action in routes.rb.

This is all you need to do with Turbo Frames to accomplish lazy loading! We just had to add 1 action and 1 view.

Let's go through it: when a user lands on our index page, we send them a very small index.html.erb file that lets them know our blog posts are loading. Once this page is loaded in the browser, Turbo will see we have a turbo frame tag on the page and it will notice our src= attribute.



<%= turbo_frame_tag "blog_posts", src: blog_posts_path do %>

Turbo will then launch a request to blog_posts_path and when a response with a matching turbo frame tag comes in,



# notice the id matches our turbo frame tag in index.html.erb
<%= turbo_frame_tag "blog_posts" %>
...
<% end %>

Turbo will work it's magic and replace our loading indicator with our blog posts!

Tip

As of Turbo 7 you can now add the loading="lazy" attribute to your turbo frame, which will only load your frame once it becomes visible to the user (when they scroll down for example).

Conclusion

Turbo is super powerful and it can help you modernize your Rails app without writing JavaScript. And of course, Turbo doesn't have to be used with Rails! Everything I wrote about can even be applied to a JavaScript app if you're feeling especially crazy 😜.

For more info, check out the official handbook here.

How to Extend the LTI Consumer XBlock (edX)

Michael Roudnitski — Mon, 01 Mar 2021 16:46:51 +0000

In this guide I will show you how to create your own XBlock based on the xblock-lti-consumer from edX. This can be very useful if you have a ubiquitous LTI resource that requires configuration.

Take for example a video conferencing app that is launched through LTI. If you want to add this app to your course, you would use the LTI Consumer XBlock and spend some time configuring it.

Forking LTI Consumer XBlock

An easy way to streamline this is to fork xblock-lti-consumer and use some inheritance magic 🧙‍♂️ so that we can add an LTI resource to our course without configuration.

In our fork, we need just make a few changes!

In lti_xblock.py, create a new class which inherits LtiConsumerXBlock.

# lti_xblock.py

...

class MyXBlock(LtiConsumerXBlock):
  display_name = String(
         display_name=_("Display Name"),
         help=_("Describe me..."),
         scope=Scope.settings,
         default=_("My Custom XBlock"),
     )

What we just did is create a new XBlock, MyXBlock. It is exactly the same as LtiConsumerXBlock except for the display name.

Next, open setup.py and modify it like so

# setup.py
entry_points={
    'xblock.v1': [
        'lti_consumer = lti_consumer.lti_xblock:LtiConsumerXBlock',
        'my_xblock = lti_consumer.lti_xblock: MyXBlock' # add this
    ],
    ...
},

Finally, in lti-consumer/__init__.py, add the following line

# lti_consumer/__init__.py
from .lti_xblock import MyXBlock

Wrapping Up

Now that we've created the XBlock, we can add it to a course by going to "Advanced Settings" and adding "my_xblock" to the advanced module list.

You should now see "My Custom XBlock" in advanced components

How to Use Elixir Pipes

Michael Roudnitski — Tue, 05 Jan 2021 02:30:55 +0000

A while ago I was introduced to an awesome functional language called Elixir. It's a young ecosystem but has plenty of useful libraries and frameworks, like Phoenix Web Framework which we use on my team here at IBM.

When I was learning Elixir there were completely new concepts I hadn't seen in previous languages. The pipe operator is one of those and it's well worth learning.

The Pipe Operator

Is used to pipe output from one function to another via the 1st argument.

# traditional
myFunction(arg1, arg2, arg3)
# with a pipe
arg1 |> myFunction(arg2, arg3)

All it does is feed the left side into the right side -- riveting!

Example

It's best to teach something like this through examples. Let's say we want to write a function which filters an array and then sorts it.

In JavaScript, we may write something like this:

function filterSort(l) {
  l = l.filter(n => n < 3);
  l = l.sort();
  return l;
}

In Elixir, we can simplify using pipes

def filterSort(l) do
  l
  |> Enum.filter(fn n -> n < 3 end)
  |> Enum.sort()
  # return is not required in Elixir!
end

Here we needed 2 pipes. One to pass the original list into the filter function, and the second to pass our filtered list into sort. We can chain pipes as much as we like. It makes for some really clear and maintainable code when done right.

Debugging with pipes

We can debug effortlessly by adding IO.inspect() (Elixir's equivalent to console.log or print) to our chain of pipes. Continuing from our example, let's inspect the state of our list between each manipulation.

def filterSort(l) do
  l
  |> IO.inspect()
  |> Enum.filter(fn n -> n < 3 end)
  |> IO.inspect()
  |> Enum.sort()
  |> IO.inspect()
  # return is not required in Elixir!
end

This works because IO.inspect() will return whatever value you originally gave it; allowing the pipe to carry on your list to the next function in your chain.

Having a hard time understanding pipes? Feel free to ask questions in the comments!