Rails 7 pagination with Turbo Frame / Streams / Infinite Scrolling / Stimulus JS

Christophe Cholot — Tue, 15 Nov 2022 01:00:42 +0000

After working on a few projects using Rails 7 and Turbo, I noticed that they all used Turbo Frames and Turbo Streams in different ways.

Now that I have some perspective on this subject, I wanted to write down a guide on how to have a modular pagination system (effortlessly switching between next/prev+next/infinite scrolling) using Turbo Frames and Turbo Streams, and then add a tiny bit of javascript using Stimulus JS to polish things up ✨.

As of October 2022, I'd recommend reading those two amazing blog posts as we all used different approaches to achieve our goal:
👉 Infinite scrolling pagination with Rails, Hotwire, and Turbo from Bearer (❤️ what they're doing).
👉 Pagination and infinite scrolling with Rails and the Hotwire stack from Colby.so

Read-bait

If you want to achieve manual paging with prev/next buttons (carousel style), manual append-only or infinite scrolling in less than 100 lines of code (I told you it was a bait), you're at the right place.

1. Setup
2. Styling with TailwindCSS
3. Paginating with Pagy
4. Navigating with Turbo Frames and Turbo Streams
5. Next page only pagination
6. Prev/Next page pagination
7. Infinite scrolling
8. History.pushState
9. Non-technical conclusion

1. Setup

Generate a new Rails application using esbuild and tailwindcss to make things look fancier than the default scaffold. Make sure you're using a version of @hotwired/turbo >= 7.2.0 (via turbo-rails or directly) as Turbo Streams using GET requests were introduced on 7.2.0



$ rails new rails-demo-turbo-stream-pagination --javascript=esbuild --css=tailwind



$ rails -v
7.0.4

Add these two well-known gem in the Rails community to our Gemfile:



# Gemfile
gem "faker" # to populate our seed data
gem "pagy" # pagination gem

Scaffold our Article model



rails g scaffold Article title:string cover_url:string body:text

And create our Article instances



# seeds.rb
100.times do
  Article.create(
    title: Faker::Book.title,
    body: Faker::Lorem.paragraph,
    cover_url: "https://source.unsplash.com/random/800x600?book"
  )
end



$ rake db:create
$ rake db:migrate
$ rake db:seed
$ bin/dev

Take a trip to localhost:3000/articles, and your screen should look like this 👇

Looking good isn't it?

2. Styling with TailwindCSS

Let's setup a minimal set of styles so we can pretend to work on a real-life application. Thanks to TailwindCSS, we can get something decent in less than 2 minutes, just by updating those 3 files:

application.html.erb



<body class="bg-gradient-to-r from-pink-50 to-sky-50 container px-12 xl:px-0 py-24 mx-auto max-w-5xl antialiased">
  <%= yield %>
</body>

articles/index.html.erb




<div class="flex justify-between items-center mb-12">
  <h1 class="text-2xl font-bold font-mono">Articles</h1>
  <%= link_to "Write an article ✍️ ", new_article_path, class: "px-6 py-3 bg-white text-black rounded shadow border border-black" %>
</div>

<div class="md:grid md:grid-cols-2 lg:grid-cols-3 gap-8">
  <% @articles.each do |article| %>
    <%= render "article" %>
  <% end %>
</div>

articles/article.html.erb



<div id="<%= dom_id article %>" class="bg-white border border-black overflow-hidden rounded shadow flex flex-col">
  <%= image_tag article.cover_url, class: "w-full object-fill h-64" %>
  <div class="px-8 py-6 flex-1">
    <h2 class="font-mono text-xl font-bold"><%= article.title %></h2>
    <p class="mt-4"><%= article.body.truncate(128) %></p>
  </div>
  <p class="px-8 pb-6 text-gray-500">#<%= article.id %> @ <%= l(article.created_at, format: :short) %></p>
</div>

Hit refresh and you're now working on that look waaay nicer! Still can't believe this took us less than 5 min and a few lines of code.

3. Paginating with Pagy

We will follow the setup instructions from Pagy repository to keep going:

app/controllers/application_controller.rb



include Pagy::Backend

app/helpers/application_helper.rb



include Pagy::Frontend

Update app/controllers/articles_controller.rb to use pagy. I picked 3 as I wanted @articles to fit within my viewport.



def index
  @pagy, @articles = pagy(Article.all, items: 3)
end

And finally, update app/views/articles/index.html.erb add the pagy_nav helper method that will display our pagination links (we won't wast our time styling it cause we're going to get rid of it very soon!).

Your /app/views/articles/index.html.erb should now look like this:



<div class="flex justify-between items-center mb-12">
  <h1 class="text-2xl font-bold font-mono">Articles</h1>
  <%= link_to "Write an article ✍️ ", new_article_path, class: "px-6 py-3 bg-white text-black rounded shadow border border-black" %>
</div>

<div class="md:grid md:grid-cols-2 lg:grid-cols-3 gap-8">
  <% @articles.each do |article| %>
    <%= render "article" %>
  <% end %>
</div>

<div class="flex mt-12 items-center justify-center space-x-4 w-full">
  <%== pagy_nav(@pagy) %>
</div>
<p class="text-center mt-8 text-gray-500"><%== pagy_info @pagy %></p>

You should now have a page with a fully functioning pagination and extra information about the pagination cursor.

4. Navigating with Turbo Frames and Turbo Streams

I'll skip the whole part where you could just use Turbo Frame to avoid full page reload as it is extremely well described in both Bearer and Colby.so articles mentioned in the introduction.

Instead, I will focus on how to make our pagination system as flexible as possible, as you might need different types of pagination within the same application. Whether we'll be using infinite scrolling, carousel, or append-only pagination, we'll only need 2 distinct Turbo Frames 💥.

Placeholder/Skeleton

To begin, we'll add what is called a placeholder, or skeleton (If you're reading this article from 2010, this is just a replacement for what we use to name "spinner.gif" 😂) to keep our users waiting while our server will be fetching its response.

Create a new file article_placeholder.html.erb, then add the following code to create a blank article card.

app/views/articles/article_placeholder.html.erb



<div class="bg-white border border-black overflow-hidden rounded shadow flex flex-col">
  <div class="bg-gray-100 w-full h-64 block"></div>
  <div class="px-8 py-6 flex-1">
    <h2 class="font-mono text-xl font-bold"><span class="w-3/4 py-2.5 rounded animate-pulse bg-gray-200 block" /></h2>
    <div class="mt-8 space-y-4">
      <p class="w-full py-1.5 rounded animate-pulse bg-gray-200 block"></p>
      <p class="w-5/6 py-1.5 rounded animate-pulse bg-gray-200 block"></p>
      <p class="w-full py-1.5 rounded animate-pulse bg-gray-200 block"></p>
    </div>
    <p class="mt-8 w-5/6 py-1.5 rounded animate-pulse bg-gray-200 block"></p>
  </div>
</div>

Keep in mind that you want your placeholder to be meaningful to your user and extremely fast to load (do not load external resources). Hint: add sleep(2) to your ArticleController#index to observe your placeholders/skeletons.

Once again, thanks to TailwindCSS we can fire those things up super easily with a few lines of code. Rendering this partial should give you this exact article card placeholder.

Turbo Frame #1: Article collection

Role:
👉 Using manual pagination (with a next and/or prev button), we'll use this frame to append or replace our paginated articles collection.
👉 Using infinite scrolling, we'll just append our paginated articles into this frame.

Code:
We'll replace our @articles collection iteration from app/views/articles/index.html.erb with a <turbo-frame> tag using lazy loading to fetch articles as a GET turbo_stream request using the following lines:

We're replacing



<div class="md:grid md:grid-cols-2 lg:grid-cols-3 gap-8">
  <% @articles.each do |article| %>
    <%= render "article" %>
  <% end %>
</div>

By this



<%= turbo_frame_tag "articles_list", src: articles_url(format: :turbo_stream, page: params[:page]), loading: "lazy" do %>
  <div class="md:grid md:grid-cols-2 lg:grid-cols-3 gap-8" id="articles_placeholder">
    <!-- Adjust this to your Pagy default item count -->
    <% 3.times do %>
      <%= render "article_placeholder" %>
    <% end %>
  </div>
<% end %>

Turbo Frame #2: Pagination

Role:
👉 Using manual pagination (with a next and/or prev button), we'll use this frame to append or replace paginated articles.
👉 Using infinite scrolling, we'll just append paginated articles into this frame (we'll get back to this later).

Code:
Add to app/articles/index.html.erb



<%= turbo_frame_tag "articles_pagination" %>

Our app/views/articles/index.html.erb should looks like this:



<div class="flex justify-between items-center mb-12">
  <h1 class="text-2xl font-bold font-mono">Articles</h1>
  <%= link_to "Write an article ✍️ ", new_article_path, class: "px-6 py-3 bg-white text-black rounded shadow border border-black" %>
</div>

<%= turbo_frame_tag "articles_list", src: articles_url(format: :turbo_stream, page: params[:page]), loading: "lazy" do %>
  <div class="md:grid md:grid-cols-2 lg:grid-cols-3 gap-8" id="articles_placeholder">
    <!-- Adjust this to your Pagy default item count -->
    <% 3.times do %>
      <%= render "article_placeholder" %>
    <% end %>
  </div>
<% end %>

<%= turbo_frame_tag "articles_pagination" %>

Turbo Stream Response

Now that our app/views/articles/index.html.erb is setup with Turbo Frames, we're ready to accept a Turbo Stream format response from ArticlesController.

app/controllers/articles_controller.rb



def index
  @pagy, @articles = pagy(Article.all, items: 3)

  respond_to do |format|
    format.html 
    format.turbo_stream
  end
end

Simple enough right? That's because the magic 🪄 will happen in format.turbo_stream, which will resolve to app/views/articles/index.turbo_stream.erb.



$ touch app/views/articles/index.turbo_stream.erb

5. Next page only pagination

To achieve an append-only pagination, we will add the following code to the newly created app/views/articles/index.turbo_stream.erb:



<!-- remove placeholders -->
<%= turbo_stream.remove("articles_placeholder") %>
<!-- append @articles to the #article_list turbo_frame -->
<%= turbo_stream.append("articles_list") do %>
  <div class="mt-8 md:grid md:grid-cols-2 lg:grid-cols-3 gap-8">
    <% @articles.each do |article| %>
      <%= render article %>
    <% end %>
  </div> 
<% end %>
<!-- update the pagination turbo_frame with the current pagination offset -->
<%= turbo_stream.update "articles_pagination" do %>
  <% if @pagy.next %>
    <div class="text-center mt-12">
      <%= link_to "Next", articles_url(page: @pagy.next), data: { turbo_stream: true }, class: "bg-white rounded px-6 py-3 border border-black shadow" %>
      <p class="mt-8 text-gray-500"><%== pagy_info @pagy %></p>
    </div>
  <% end %>
<% end %>

We're using 3 Turbo Streams here to:

Remove skeletons
Append records to our #article_list Turbo Frame
Update pagination with the latests offsets and informations.

6. Prev/Next page pagination

To achieve a prev/next pagination, we will add the following code to app/views/articles/index.turbo_stream.erb:



<!-- update @articles collection -->
<%= turbo_stream.update("articles_list") do %>
  <div class="mt-8 md:grid md:grid-cols-2 lg:grid-cols-3 gap-8">
    <% @articles.each do |article| %>
      <%= render article %>
    <% end %>
  </div>
<% end %>
<!-- update the pagination turbo_frame with the current pagination offset -->
<%= turbo_stream.update "articles_pagination" do %>
  <div class="flex mt-12 items-center justify-center space-x-4 w-full">
    <% if @pagy.prev %>
      <%= link_to "Prev", articles_url(page: @pagy.prev), data: { turbo_stream: true }, class: "bg-white rounded px-6 py-3 border border-black shadow" %>
    <% end %>
    <% if @pagy.next %>
      <%= link_to "Next", articles_url(page: @pagy.next), data: { turbo_stream: true }, class: "bg-white rounded px-6 py-3 border border-black shadow" %>
    <% end %> 
  </div>
  <p class="text-center mt-8 text-gray-500"><%== pagy_info @pagy %></p>
<% end %>

We're using 2 Turbo Streams here to:

Update records from #article_list Turbo Frame. Since update is used, skeletons will get replaced!
Update pagination with the latests offsets and informations.

7. Infinite scrolling

Last but not least, we can achieve an infinite scroll pagination pretty easily thanks to the loading="lazy" attribute on a <turbo-frame> tag.
According to the documentation, our frame content will only be loaded if the frame is visible.
Let's use this to perform the following actions when the frame gets visible (or in other words, when we reach the bottom of our page):

👉 Add new content to our article collection Turbo Frame
👉 Paginate to the next offset as soon as we reach the bottom of the page.

Good news is that we've already wrote this piece of code, we just need to call it!

app/views/articles/index.html.erb:



<!-- remove skeletons -->
<%= turbo_stream.remove("articles_placeholder") %>
<!-- append @articles to the #article_list turbo_frame -->
<%= turbo_stream.append("articles_list") do %>
  <div class="mt-8 md:grid md:grid-cols-2 lg:grid-cols-3 gap-8">
    <% @articles.each do |article| %>
      <%= render article %>
    <% end %>
  </div>  
<% end %>
<!-- replace #article_pagination frame by our content loader when it gets visible (our turbo_frame_tag with loading="lazy" attribute -->
<% if @pagy.next.present? %>
  <%= turbo_stream.replace "articles_pagination" do %>
    <%= turbo_frame_tag "articles_pagination", src: articles_url(page: @pagy.next, format: "turbo_stream"), loading: "lazy" %>
  <% end %>
<% end %>

And we're done!

8. History.pushState

In order to perfect our paging system with Turbo Streams and Turbo Frames, we need to behave as close as possible to a traditional paging system. When clicking on a page link /articles?page=2, we expect our browser to update both its URL and history state. Let's see how we can bring back this behavior.

Problem: Our pagination links are 'framed' by our #article_pagination Turbo Frame. Every time click a link inside the frame, it will update the frame src attribute instead of our browser URL 🤷‍♀️.

Thankfully @Intrepidd wrote a Stimulus Controller doing the amazing job of observing an element attribute mutation, then push its src attribute to our history state using @hotwired/turbo navigator api.

Add the controller to your code base. You can name it just like his author did TurboFrameHistory / turbo_frame_history_controller (don't forget to register it in your in your Stimulus controllers)

and replace



<%= turbo_frame_tag "articles_pagination" %>



<%= turbo_frame_tag "articles_pagination", data: { controller: "turbo-frame-history" } %>

in app/articles/index.html.erb and there you go! Now every time a link from the #pagination_article Turbo Frame is clicked, following chain will happen:

the frame src's attribute is updated
mutate(entries) from our Stimulus controller is called
navigator.history.push is called

Note: This technique will NOT work with the infinite scroll code snippet as we're not triggering any click from inside the #article_pagination Turbo Frame.

If you really want the same behavior when using the infinite scroll, a solution could be to wrap the #article_pagination Turbo Frame and observe its childList mutation. Another solution could be to set a data-attribute on the replaced pagination turbo frame and manually trigger a mutation.



<%= turbo_stream.remove("articles_placeholder") %>
<%= turbo_stream.append("articles_list") do %>
  <div class="mt-8 md:grid md:grid-cols-2 lg:grid-cols-3 gap-8">
    <% @articles.each do |article| %>
      <%= render article %>
    <% end %>
  </div>  
<% end %>
<% if @pagy.next.present? %>
  <%= turbo_stream.replace "articles_pagination" do %>
    <%= turbo_frame_tag "articles_pagination", src: articles_url(page: @pagy.next, format: "turbo_stream"), loading: "lazy", data: { mutate_on_connect: true } %>
  <% end %>
<% end %>



// turbo_frame_history.js
connect () {
  useMutation(this, { attributes: true })
    if(this.element.dataset.mutateOnConnect) {
      this.mutate([{ type: 'attributes', attributeName: 'src' }])
    }
  }
}

And you should be good to go!

9. Non-technical conclusion

On a final note, here's an article on how Etsy.com purchases dropped by 22% when switching from traditional prev/next/pages to infinite scroll on their product listing page.

Whether you run an e-commerce store or a blog about cats, users behavior has a strategic influence on your website metrics (good metrics is a SEO boost, as well as a boost when using ads engines like Adwords or Facebook Ads), and so your revenue.

Don't just assume that X pagination style will be the best for your website just because it's fancier to code or because it's the trend. A/B testing is the way to go if you want to figure things out and we just saw how Rails with Turbo made it feels like a breeze.

🧑‍💻 You'll find the github repository of this article right here: https://github.com/Chwistophe/rails-demo-turbo-frame-stream-pagination

Cheers,

DEV Community: Christophe Cholot