Endless Scroll / Infinite Loading with Turbo Streams & Stimulus

Hotwire Turbo by the Ruby on Rails developers is the new solution to enhance server side rendered apps with interactive behavior without much Javascript at all.

In this post, I want to show you, how I built an infinite scrolling feature, meaning: When reaching the bottom of a list, load the next page and add it to the DOM. There are many many ways in handling this kind of solution, like, using a 3rd party library like “lazyload” and more. To get more familiar with the Hotwire Stack of Stimulus + Turbo, I decided to use Turbo Streams for handling the DOM-part, and Stimulus to connect with an Interaction Observer.

For this post, I will make the following assumptions:

• We hava a controller PostsController with an index action
• We already handling pagination via the great pagy Gem
• Turbo + Stimulus are all set up
• I use SLIM as the template language, because I like it’s brevity and clearness

Disclaimer: When this PR get’s released, this solution might be simplified much more, but just using <turbo-frame action="append"> with a little glue code.

Add stimulus to our posts

// index.html.slim

.list-group(data-controller="infinite-scroll")
  // If you need to enable Live Updates, you could connect to a
  // = turbo_stream_from current_user, :posts
  #posts
    = render @posts
  div(data-infinite-scroll-target='scrollArea')

  #pagination.list-group-item.pt-3(data-infinite-scroll-target="pagination")
    == pagy_bootstrap_nav(@pagy)

In this index we,

• wrap our posts with a Stimulus Controller and
• mark the posts into a div with id=posts (to later append to)
• add a scrollArea empty element div just below our posts list - This area will be used for our Intersection Observer later on
• add the pagy_nav or pagy_bootstrap_nav pagination tags on the bottom, also wrapped in a Stimulus Target to later on pick the next page’s link from it

Now, before we modify the controller to respond to Turbo events, we implement the Stimulus Controller

Stimulus controller

// app/javascript/controllers/infinite_scoll_controller.js

import { Controller } from "stimulus"

export default class extends Controller {
  static targets = ["scrollArea", "pagination"]

  connect() {
    this.createObserver()
  }
  createObserver() {
    const observer = new IntersectionObserver(
      entries => this.handleIntersect(entries),
      {
        // https://github.com/w3c/IntersectionObserver/issues/124#issuecomment-476026505
        threshold: [0, 1.0],
      }
    )
    observer.observe(this.scrollAreaTarget)
  }
  handleIntersect(entries) {
    entries.forEach(entry => {
      if (entry.isIntersecting) {
        this.loadMore()
      }
    })
  }
  loadMore() {
    const next = this.paginationTarget.querySelector("[rel=next]")
    if (!next) {
      return
    }
    const href = next.href
    fetch(href, {
      headers: {
        Accept: "text/vnd.turbo-stream.html",
      },
    })
      .then(r => r.text())
      .then(html => Turbo.renderStreamMessage(html))
      .then(_ => history.replaceState(history.state, "", href))
  }
}

• We define the areas ScrollArea + Pagination
• When loaded, the controller puts an Interaction Observer onto the scroll area
• When the scroll area get’s into the viewport, we loadMore posts, by picking the next page’s url from the pagination
• Import: We can’t use Turbo.visit but we are using fetch instead, because the Turbo-Stream Magic only works on POST/PATCH/… requests. But we want our controller to respond to this GET request with a specific Turbo Stream that handles the DOM manipulation. That’s why we use fetch manually with the special Accept: text/vnd.turbo-stream.html header here.
• When the fetch returns, we pipe the result manually to Turbo.renderStreamMessage which evaluates the html content for Turbo Stream actions.

PostsController

class PostsController < ApplicationController
  include Pagy::Backend
  helper Pagy::Frontend

  def index
    @pagy, @posts = pagy Post.order(:created_at)
    respond_to do |f|
      f.turbo_stream
      f.html
    end
  end

Straight forward Turbo: we respond with “turbo_stream” format, or with html (template at the top in this post). Let’s show the turbo_stream template:

index.turbo_stream.slim - Turbo Stream response

turbo-stream action="append" target="posts"
  template
    = render partial: 'posts/post', collection: @posts, formats: [:html]

turbo-stream action="update" target="pagination"
  template
    == pagy_bootstrap_nav(@pagy)

• We append this page’s posts to the div with id=posts
• We replace the pagination completely to reflect the new page
• Important to switch the format to html to get our ‘post’ partial, don’t know if intended or bug.

That’s it! Because the scroll area will be always on the bottom, it will trigger over and over, Stimulus will pick out the current next page’s url from the pagination part, and Rails will respond with a Turbo stream that updates both the new posts and replace the pagination.

Comments

[Marc Köhlbrugge]

Thanks for sharing this! I used it on buildinpublic.com

There seems to be a timing issue with it though. If a "next page" starts loading, but you navigate away before it's finished, then this "next page" might be end up being appended to the new page you're on.

So you might be browsing User A's profile, click over to User B, and then see User A's second page appended to User B's profile.

One solution would be to abort the loading when a page is navigated away from. I think this can be done by deleting the IntersectionObserver on disconnect although I haven't tried that yet.