A Generalized User-local Container for UI State in Kredis

In our last post, we persisted and restored a collapsed/expanded UI state with Kredis. However, the great pain point with this is that we have to invent a lot of Kredis keys.

This time, we'll see how we can avoid this by developing a generalized solution for any DOM node whose attribute states we want to track on the server side.

Let's dive straight in!

The Challenge

Remember that we had to concoct a Kredis set key (open_department_ids) on the user model in the previous post. Now, following our example, imagine a user might be associated to the products they bought via has_many. Assume that each of these products has a details page which also has some ephemeral state attached.

All of a sudden, the complexity increases tremendously: we now have N kredis_sets to cater for, prefixed with the product ID. Or we could handle it on the Product model, storing the state for each user ID. Either way, it is going to get cumbersome, and we'll likely have to resort to metaprogramming to smooth it out a bit.

This, you'll agree with me, is not a delightful challenge. It would be very beneficial for the app's architecture if we could abstract that away.

And this is precisely what we are going to attempt. We'll use an accordion on the product details page to illustrate this case and also its resolution.

Preparation

To start, we'll add two more text columns, description and specs, to our Product model:

$ bin/rails g migration AddDescriptionAndSpecsToProducts description:text specs:text
$ bin/rails db:migrate

Again, in the Rails console, we'll provide some seed data:

Product.find_each do |product|
  product.update(description: Faker::Lorem.paragraph, specs: Faker::Markdown.ordered_list)
    description: Faker::Lorem.paragraph,
    specs: Faker::Markdown.ordered_list
  )
end

Finally, let's amend the _product partial to include those two new properties in an "accordion", again making use of the <details> element.

  <!-- app/views/products/_product.html.erb -->

  <div id="<%= dom_id product %>">
    <p>
      <strong>Name:</strong>
      <%= product.name %>
    </p>

    <p>
      <strong>Department:</strong>
      <%= product.department_id %>
    </p>

+   <p>
+     <details>
+       <summary>Description</summary>
+
+       <%= product.description %>
+     </details>
+   </p>
+
+   <p>
+     <details>
+       <summary>Specs</summary>
+
+       <%= simple_format product.specs %>
+     </details>
+   </p>
  </div>

In our home index view, let's link to the products:

  <!-- app/views/home/index.html.erb -->

  <!-- ... -->

        <ul>
          <% child_dep.products.each do |prod| %>
-           <li><%= prod.name %></li>
+           <li><%= link_to prod.name, prod %></li>
          <% end %>
        </ul>

  <!-- ... -->

Here's the detail of the view this produces.

Observing UI Changes with JavaScript

Now we have to rewrite our UIState Stimulus controllers to listen for arbitrary DOM changes. We do this by placing a MutationObserver on its element, which looks for attribute changes after it connects. In its callback (mutateState), the following happens:

1. All attributes on the element, except for the data-controller attribute itself, are collected.
2. Their respective values are stored on an attributes object with their name as the key.
3. This object is appended to a new FormData object, along with a unique key that identifies the exact instance of the UI state (we'll get to that in a moment).
4. This is then sent over to the server as before, in the form of a PATCH request.

  // app/javascript/controllers/ui_state_controller.js

  import { Controller } from "@hotwired/stimulus";
- import { patch } from "@rails/request.js";
+ import { get, patch } from "@rails/request.js";

  export default class extends Controller {
    static values = {
-     departmentId: String,
+     key: String,
    };

-   async toggle() {
-     const body = new FormData();
-     body.append("open", this.element.open);
-     body.append("department_id", this.departmentIdValue);
-
-     await patch("/ui_state/update", {
-       body,
-     });
-   }

+   async connect() {
+     this.mutationObserver = new MutationObserver(this.mutateState.bind(this));
+
+     this.mutationObserver.observe(this.element, { attributes: true });
+   }
+
+   disconnect() {
+     this.mutationObserver?.disconnect();
+   }
+
+   async mutateState() {
+     const body = new FormData();
+
+     const attributes = {};
+     this.element
+       .getAttributeNames()
+       .filter((name) => name !== "data-controller")
+       .map((name) => {
+         attributes[name] = this.element.getAttribute(name);
+       });
+
+     body.append("key", this.keyValue);
+     body.append("attributes", JSON.stringify(attributes));
+
+     await patch("/ui_state/update", {
+       body,
+     });
+   }
  }

Persisting UI State with Unique Kredis Keys

To generalize persisting UI state, we have to accomplish two things:

1. Generate unique Kredis keys regarding the user, resource (in our case, the product), and the location in the view, respectively.
2. Use these keys to persist the attributes object received from the client.

Using Rails Helpers

So first, we have to solve the problem of coming up with unique keys. If you look at the requirements carefully, they read a lot like the built-in Rails fragment cache helper.

In short, this helper will build a cache key based on the template it's called from, and any cacheable Ruby objects passed to it. To quote the documentation:

views/template/action:7a1156131a6928cb0026877f8b749ac9/projects/123
      ^template path   ^template tree digest             ^class   ^id

We don't need the cache helper though, because we are not actually storing a fragment, we just need a key. Thankfully, the method it calls internally to generate one, cache_fragment_name, is also exposed.

Let's check out how we can make use of this. If we add the following line to our products/_product.html.erb partial:

<%= cache_fragment_name([current_user, product]) %>

Something like this is rendered:

["products/_product:0e55db8bd12c9b268798d6550447b303",
  [#<User id: 1, email: "[e-mail]", ...>,
  #<Product id: 68, name: "Durable Wool Wallet", ...>]
]

Clearly, this nested array needs some massaging, but we are already very close. What's still missing is a reference to the specific line in the template, but we can obtain this from the call stack's first entry representing a template. Let's extract this to a helper and convert it to a string:

# app/helpers/application_helper.rb

module ApplicationHelper
  def ui_state_key(name)
    key = cache_fragment_name(name, skip_digest: true)
      .flatten
      .compact
      .map(&:cache_key)
      .join(":")

    key += ":#{caller.find { _1 =~ /html/ }}"

    key
  end
end

Note: We are skipping the view digesting as including the caller location makes it redundant.

Called as ui_state_key([current_user, product]), this yields a string of the following format:

"users/1:products/68:/usr/app/ui-state-kredis/app/views/products/_product.html.erb:23:in `_app_views_products__product_html_erb___2602651701187259549_284440'"

Now there's one final piece to solve: because we will include this key in our markup (to be picked up by the Stimulus controller above), we'll want to obfuscate it. Otherwise, it would be easy for a bad actor to exchange the user ID and/or the product global ID using the browser's developer tools. A simple digest will do in our case, because we do not intend to decrypt it back.

  # app/helpers/application_helper.rb

  module ApplicationHelper
    def ui_state_key(name)
      key = cache_fragment_name(name, skip_digest: true)
        .flatten
        .compact
        .map(&:cache_key)
        .join(":")

      key += ":#{caller.find { _1 =~ /html/ }}"

-     key
+     ActiveSupport::Digest.hexdigest(key)
    end
  end

That's it, now we can obtain a digest of our UI fragment that will be unique to the location in the template and its MTIME, as well as user and resource:

"d45028686c0171e1e6e8a8ab78aae835"

Attach Stimulus Controllers to Your Rails App

Equipped with this, we can now attach the Stimulus controllers along with the respective keys to our <details> elements:

  <!-- app/views/products/_product.html.erb -->

  <div id="<%= dom_id product %>">
    <p>
      <strong>Name:</strong>
      <%= product.name %>
    </p>

    <p>
      <strong>Department:</strong>
      <%= product.department_id %>
    </p>

    <p>
      <details
+       data-controller="ui-state"
+       data-ui-state-key-value="<%= ui_state_key([current_user, product]) %>"
      >
        <summary>Description</summary>

        <%= product.description %>
      </details>
    </p>

    <p>
      <details
+       data-controller="ui-state"
+       data-ui-state-key-value="<%= ui_state_key([current_user, product]) %>"
      >
        <summary>Specs</summary>

        <%= simple_format product.specs %>
      </details>
    </p>
  </div>

We can now set out to generalize the UIStateController as well. For this, we obtain a new Kredis.json key using the transmitted key param. Whenever a UI state update occurs, all the attributes sent over as form data will be parsed and stored therein.

  # app/controllers/ui_state_controller.rb

  class UiStateController < ApplicationController
+   include ActionView::Helpers::SanitizeHelper

+   before_action :set_ui_state

    def update
-     if ui_state_params[:open] == "true"
-       current_user.open_department_ids << params[:department_id]
-     else
-       current_user.open_department_ids.remove(params[:department_id])
-     end
+     @ui_state.value = JSON.parse(params[:attributes]).deep_transform_values { sanitize(_1) }

      head :ok
    end

    private

    def ui_state_params
-     params.permit(:department_id, :open)
+     params.permit(:attributes, :key)
    end

+   def set_ui_state
+     @ui_state = Kredis.json params[:key]
+   end
  end

Rehydrating the UI

Since our UI state is now safely stored in an ephemeral Kredis key, the only piece of the puzzle that's missing is how to put the UI back together. This process is called rehydration. In this example, we'll use server-side rehydration.

For this, we will revisit the ui_state_key helper from above, and extend it with the rehydration logic.

   # app/helpers/application_helper.rb
   module ApplicationHelper
+    def remember_ui_state_for(name, &block)
+      included_html = capture(&block).to_s.strip
+      fragment = Nokogiri::HTML.fragment(included_html)
+
+      first_fragment_child = fragment.first_element_child
+
+      # rehydrate
+      ui_state = Kredis.json ui_state_key(name)
+
+      ui_state.value&.each do |attribute_name, value|
+        first_fragment_child[attribute_name] = sanitize(value, tags: [])
+      end
+
+      # add stimulus controller and create unique key
+      first_fragment_child["data-controller"] = "#{first_fragment_child["data-controller"]} ui-state".strip
+      first_fragment_child["data-ui-state-key-value"] = ui_state_key(name)
+
+      first_fragment_child.to_html.html_safe
+    end

     def ui_state_key(name)
       key = cache_fragment_name(name, skip_digest: true)
         .flatten
         .compact
         .map(&:cache_key)
         .join(":")

       key += ":#{caller.find { _1 =~ /html/ }}"

       key
     end
   end

The remember_ui_state_for helper takes a name argument that it will pass on to the ui_state_key helper from above. Before it does that, though, it will capture the HTML from the block passed to it and extract its first fragment child with Nokogiri.

Afterward, the actual rehydration logic starts:

1. the ui_state is read back from Kredis
2. in case such a key already exists, each attribute is put back on the HTML element obtained in the previous step. To prevent any XSS vulnerabilities, the attributes' values are also sanitized
3. finally, the Stimulus controller is attached to the element, along with the ui_state_key.

Any change to the attributes would now again be transmitted by the MutationObserver, and any page reload would invoke this helper again, rehydrating them upon the element.

Note: An additional security mechanism would be safelisting the allowed attributes, which I have excluded from this example for simplicity.

In our product partial, instead of wiring up the Stimulus controller manually, we must now use the new view helper:

  <!-- app/views/products/_product.html.erb -->

  <div id="<%= dom_id product %>">
    <p>
      <strong>Name:</strong>
      <%= product.name %>
    </p>

    <p>
      <strong>Department:</strong>
      <%= product.department_id %>
    </p>

    <p>
-     <details
-       data-controller="ui-state"
-       data-ui-state-key-value="<%= ui_state_key([current_user, product]) %>"
-     >
-       <summary>Description</summary>
-
-       <%= product.description %>
-     </details>
+     <%= remember_ui_state_for([current_user, product]) do %>
+       <details>
+         <summary>Description</summary>
+
+         <%= product.description %>
+       </details>
+     <% end %>
    </p>

    <p>
-     <details
-       data-controller="ui-state"
-       data-ui-state-key-value="<%= ui_state_key([current_user, product]) %>"
-     >
-       <summary>Specs</summary>
-
-       <%= simple_format product.specs %>
-     </details>
+     <%= remember_ui_state_for([current_user, product]) do %>
+       <details >
+         <summary>Specs</summary>
+
+         <%= simple_format product.specs %>
+       </details>
+     <% end %>
    </p>
  </div>

This is what the end result looks like.

It turns out that this approach is perfect for HTML elements or custom web components that reflect their state in one or more attributes (e.g. <details>, Shoelace, Lit, etc.).

Wrapping Up

In part one of this series, we described what Kredis can and cannot do, and developed an example use case for storing ephemeral UI state using a bespoke Redis key.

In this second and final part, we identified the drawbacks of this approach. We developed a generalized solution to apply to any DOM node whose state of attributes we want to track on the server side.

Finally, let me point out that these considerations inspired me to develop a library called solder, which I would be happy to receive feedback on!

Happy coding!

P.S. If you'd like to read Ruby Magic posts as soon as they get off the press, subscribe to our Ruby Magic newsletter and never miss a single post!