A few sneak peeks into Hey.com technology (III - Turbolinks frames)

This post covers an exciting new Turbolinks feature found on Hey - the Turbolinks frames. I think they allow devs to asynchronously update any part of the webpage with fresh server data without needing to write any JavaScript code! The update can be done automatically right after the page load or triggered interactively.

Please bear in mind that what is written here is all dug up from the HTML / JS sources of the Hey site and is limited by my current understanding of them. If you happen to find errors in my thinking, go ahead and tell me!

Update as of Dec 2020: now that the Hey support code has been extracted and officially released under the name Hotwire, we know that this article actually speaks about the Turbo Frames. Basecamp renamed and polished a few things but the principles stay the same as described here.

 Autoloading page parts after page load

When you open up the Developer tools Network tab while clicking on any Hey page, you’ll soon notice a pattern of HTML resources loading: only the most important page content is loaded with the first request while the less relevant stuff is loaded asynchronously afterwards.

For example, the initial request of the main page (Imbox) will load the main content (the page layout plus the mail listings, green area) but not the top yellow upgrade banner or the “Reply later” and “Set aside” feeds at the bottom (red areas) − these three areas are loaded async:

It looks like this in the Network tab:

Why even bother with such a pattern? Let’s emphasize a few things here:

• Loading only the page skeleton and some most relevant stuff is good for the speed. Neither the server, network, nor browser is slowed down by rendering the less important things during the initial request. The user can start scanning the page a bit sooner.

• The requests can be cached more easily. See the 304 status codes in the Network tab? Those are cached requests - the server still has to calculate and render their template but the browser does not have to download or paint anything as the response is all the same! Smaller requests covering only a portion of a page are more likely to stay the same and thus be cacheable. Again, this is a speed optimization!

• Above all, you don’t have to write a single line of JavaScript for this to work! Let me show you…

Let’s use the Page inspector picker tool to reveal the HTML code of the yellow upgrade banner. It shows something like this:

It’s a turbolinks-frame element, it has a src attribute pointing to the /account/trial/callouts back-end action which we saw in the Network requests listing above. OK, but how does it obtain its content, the banner div itself?

The answer lies in the fact that the turbolinks-frame tag is a Custom HTML element. This means a few important things that lead to the following investigation:

• The Custom element is closely tied to JavaScript code that determines its behavior. And indeed, we can find the elements/turbolinks_frame.ts file in the JS sources. The extension tells us that this is a TypeScript file.

  This is actually a very similar concept to the Stimulus framework, invented by the Rails team. The difference is that Custom elements are directly supported by (modern) browsers. On the other hand, Stimulus controllers are way more opinionated, so the JS code is usually smaller and (I’d say) more readable.

• The Web components standard states that the custom element must be define-d to bring it to life on a page. OK, we can find this define statement at the bottom of the turbolinks_frame.ts file. This statement connects the HTML tag to the TurbolinksFrameElement class in the same file.

• This class has quite some code to read through and it even cooperates with a few sibling classes… nevertheless we can ignore all that for now and focus on only one thing − the attributeChangedCallback. The docs say that this callback is automatically invoked by the browser whenever an “observed attribute” changes its value or is added to the custom element. For which attributes the change is noticed is specified in the observedAttributes method. A quick look into this method reveals that this custom element is observing its src attribute:

  

• So, whenever the src attribute of the custom element changes, the callback method will be invoked by the browser. The same happens when the whole element first appears on page. Remember that the src attribute contains the URL of the resource that defines the content of the page part. Let’s see what happens when the callback is invoked:

  

• Oh now we’re getting somewhere: when the browser discovers a new URL in the src attribute, it grabs that URL and calls a Turbolinks visit method which fetches the URL via AJAX and calls the FrameController.requestSucceededWithResponse callback upon success. Further jumping around the source code finally gets us to the loadFrameElement method which takes the response from the AJAX call and replaces the custom element with it. It looks we’ve just updated that page part with a new content from the server!

Summary

To sum up this workflow, this is all you need to do for autoloading a page part upon page load: add an empty <turbolinks-frame> tag to the page somewhere and fill in its src attribute. The tag contents will get auto-updated via an AJAX request just after the main page loads. I guess the Turbolinks team will provide some nice back-end helper, too, to make things here even simpler.

This also implies that the server should return plain old HTML in the response. No JavaScript, no SJR, no UJS, no JSON, just HTML!

Let’s show the upgrade banner server response that we got here:

See? Nothing but HTML! If you do need to add some JavaScript interactivity, just let the server add Stimulus controller attributes to the returned HTML tags and that’s it! Stimulus will notice that the attributes were added to the DOM and will call the appropriate JS controllers. Again, no JS is needed to be returned in the response, as all JS is pre-bundled in the Stimulus controllers code module.

Oh well, this got a little longer than expected, I will continue this topic in the next post about interactively loaded page parts, have a nice day and stay tuned…

Comments

[Yoel]

Very nice article!
"Stimulus will notice that the attributes were added to the DOM and will call the appropriate JS controllers"
I don't understand this part, is Stimulus called whenever something is updated in the DOM?

[Matouš Borák]

Thank you!

Regarding your question, yes. Stimulus uses the Mutation observer API in modern browsers so that it's all very speedy and lightweight.

[intrnl]

I think that Hey might be pushing it a little too far though. Lots of accessibility issues, and while some of them can be fixed easily, fixing the rest without adding a sprinkle of JS will be impossible without the platform giving us a better higher level widgets to work with.

[Matouš Borák]

Oh, I just read this thread that explains it well, interesting, thanks! twitter.com/devongovett/status/127...

I still think these issues should be solved in the HTML standards so that good accessibility is not dependant on JS.

[Matouš Borák]

Hi, can you be more specific about the accessibility issues and how more JS would solve them?

[leo liang]

Hey @intrnl , curious about what kind of accessibility issues the Hey approach would introduce? Normally, accessibility should be more related to html not JS right?

[leo liang]

Also, another thing to notice is they are using HTTP/2, which is not supported by Rack at the moment, not sure if they got a secret way to get around it

[Matouš Borák]

Hey, no secret needed, AFAIK they use some flavour of nginx as a reverse proxy in front of puma servers. And nginx handles http/2 well.

[Rares Portan]

Interesting read, thanks.
It would be also nice to know how the server-side works. How it's deployed and scaled, etc

[Matouš Borák]

Thanks, I would surely love to see the back-end code myself! I guess we’ll have to wait for the official blog posts about it. Hopefully they’ll come soon...