dev.to’s Frontend: a brain dump in one act

There is currently an issue open to improve the frontend documentation (see Frontend · DEV Docs) to get people onboarded quicker in the frontend. Big shout out to @rhymes for opening this issue!

Add developer documentation about the JavaScript frontend #2507

rhymes posted on Apr 21, 2019

Is your feature request related to a problem? Please describe.

On the heels of Thoughts on migrating to TypeScript and improving the overall quality of the frontend DEV codebase, DEV Notes: Don't Forget To Clear Cache! and https://github.com/thepracticaldev/dev.to/issues/2499 I thought it'd be nice and useful if there was some kind of overview of how the frontend pieces (especially the JavaScript layer) all fit together.

I know there are two sets of JS code bases, the "legacy" in app/assets/javascripts managed by Rails's sprockets and the one in app/javascript managed by webpack.

Other than that I'm not sure how everything works (I admit I haven't spent much time in the frontend, other than the occasional bug fix or small feature).

A few questions that a docs/frontend/javascript.md might answer:

• How does initialization work?
• Is the Preact layer totally ignorant of the vanilla JS layer?
• Do the two JS code bases interact with/call each other?
• How are they attached to the template pages? Does each webpage serves two sets of packed/minified JS files?
• What does the service worker do in the context of dev.to? Is it registered on the whole page? Is there more than one?
• How does edge caching fit in all of this?
• How does instant click fit in all of this?

Plus anything else deemed important. It doesn't have to be super in depth, just a treasure map to know what does what and how everything fits together.

Describe the solution you'd like

A documentation file that contains a description of how the JS frontend works

Describe alternatives you've considered

I haven't considered alternatives TBH, the status quo is okay, it's just going to be easier for contributors if there's some onboarding documentation about the code base, especially if the goal is to refactor it, modernize it or even adapt with TypeScript or other solutions.

View on GitHub

I decided to write this post because I’ll be contributing to this documentation issue and thought it would be beneficial for everyone, including myself. I’m hoping people will ask questions in the comments and/or fill in missing gaps in the post.

Vanilla JS

There is a lot of the frontend code base in the app/assets/javascripts folder. This part of the code base does not use ES modules. It loads scripts, runs stuff once the DOM has loaded, has stuff in the global scope and provides a lot of the functionality on the client-side for dev.to.

The assets are loaded through standard rails/fastly methods that add the <script /> tags to load the front-end code. Most, if not all of these scripts are deferred (See the defer attribute in <script>: The Script element - HTML).

Preact, webpacker & webpack

There is a more modern JavaScript portion of the application as well, but it is not a Single Page Application (SPA). It is a set of components that are dispersed in key locations, e.g. search, v2 editor, onboarding etc.

Preact components are managed using the webpacker gem and webpack. If you're curious about webpacker, @maestromac on the team is a great person to speak to.

Scripts for webpack entry points are added to Ruby ERB templates, but they use the webpacker javascript_pack_tag to add the script server-side. There is a webpack configuration file, but it is in yaml format. In that config, there are settings that determine where the code is and how entry points are defined.

dev.to/webpacker.yml at master · thepracticaldev/dev.to · GitHub

...
default: &default
  source_path: app/javascript
  source_entry_path: packs
...

Looking at the configuration above, this part of the frontend code base can be found in the app/javascript folder with webpack entry points found in the app/javascript/packs folder.

This represents the base configuration for webpack. If additional configuration is required for an environment, webpacker allows you to enhance the configuration via webpack configuration export.

dev.to/development.js at master · thepracticaldev/dev.to · GitHub

const environment = require('./environment');
const config = environment.toWebpackConfig();

// For more information, see https://webpack.js.org/configuration/devtool/#devtool
config.devtool = 'eval-source-map';

module.exports = config;

As the project continues to move forward, expect to see some more things client side becoming preactitized (I just made that up, boom!).

@ mention autocompletes in comment box #354

benhalpern posted on Aug 14, 2018

Feature Request or Task

As a user I want to be able to start typing with @ and have it deliver me a dropdown. This is expected behavior in this context.

This means that the box would have to become a content-editable div and the code should attach to each box. Making that part work could involve some yak shaving.

This should be written in Preact, and if think it makes sense to include a different library, let's discuss. We are critical of each dependency as we want to be efficient in this regard. It doesn't mean you definitely can't use a library, but let's just discuss the implications if you go that route.

The dropdown will use Algolia search, and we may need to add a proper new custom index on the User model to make this happen.

Definition of Done

This is done when all content boxes have a dropdown menu and the behavior works as expected on sites like GitHub and Twitter. Core team is here to help navigate the nuances of the issue.

View on GitHub

An example of how Preact works in the frontend codebase

1. The Search entry point script is loaded via webpacker’s javascript_pack_tag, e.g. <%= javascript_pack_tag "Search", defer: true %>.

dev.to/application.html.erb at master · thepracticaldev/dev.to · GitHub

<!DOCTYPE html>
<html lang="en">
  <head>
    <meta charset="utf-8">
    <% title = yield(:title) %>
    <title><%= title || "#{ApplicationConfig['COMMUNITY_NAME']} Community" %></title>
    <% if internal_navigation? %>
      <style>
            ...
      </style>
    <% else %>
      ...
      <style>
        ..
      </style>
      ...
      <%= javascript_pack_tag "Search", defer: true %>
...

1. The search bar is rendered server-side as well on initial page load. This is what I currently call ghetto server-side rendering (SSR) for Preact. I know that @ben wanted to add preact SSR at some point, but it wasn't that high a priority at the time. Maybe now it will rank higher as more components are created with preact.

dev.to/_top_bar.html.erb at master · thepracticaldev/dev.to · GitHub

...
    <div id="nav-search-form-root">
      <div class="nav-search-form">
        <form acceptCharset="UTF-8" method="get">
          <input class="nav-search-form__input" type="text" name="q" id="nav-search" placeholder="search" autoComplete="off" />
        </form>
      </div>
    </div>
...

1. On the client-side once the DOM content has loaded, Preact takes over.

dev.to/Search.jsx at master · thepracticaldev/dev.to · GitHub

import { h, render } from ‘preact’;
import { Search } from ‘../src/components/Search’;

document.addEventListener(‘DOMContentLoaded’, () => {
  const root = document.getElementById(‘nav-search-form-root’);

  render(<Search />, root, root.firstElementChild);
});

1. From there on in, all interactions with the Search box are client-side.

InstantClick

Like the tag line says, “InstantClick is a JavaScript library that dramatically speeds up your website, making navigation effectively instant in most cases.”. Basically the way it works is if a user hovers over a hyperlink, chances are their intentions are to click on it. InstantClick will start prefetching the page while a user is hovering over a hyperlink, so that by the time they do click on it, it's instantaneous. Note, on mobile devices, preloading starts on "touchstart".

Aside from prefetching pages, InstantClick also allows you to customize what happens when an InstantClick page changes.

dev.to/githubRepos.jsx at master · thepracticaldev/dev.to · GitHub

...
window.InstantClick.on('change', () => {
  loadElement();
});
...

You can also decide whether or not to reevaluate a script in an InstantClick loaded page via the data-no-instant attribute. I don’t believe there are any examples in the code base that blacklist script reevaluation. You can also blacklist a link. Here is an example from the codebase.

dev.to/buildCommentHTML.js.erb at master · thepracticaldev/dev.to · GitHub

...

function actions(comment) {
  if (comment.newly_created) {
    return '<div class="actions" data-comment-id="'+comment.id+'" data-path="'+comment.url+'">\
              <span class="current-user-actions" style="display: '+ (comment.newly_created ? 'inline-block' : 'none') +';">\
                <a data-no-instant="" href="'+comment.url+'/delete_confirm" class="edit-butt" rel="nofollow">DELETE</a>\
                <a href="'+comment.url+'/edit" class="edit-butt" rel="nofollow">EDIT</a>\
              </span>\
                <a href="#" class="toggle-reply-form" rel="nofollow">REPLY</a>\
            </div>';
  } else {
...

For more information on this, see the Events and script re-evaluation in InstantClick documentation.

Linting / Code Formatting

eslint & prettier

The project uses eslint with the Prettier plugin. This means that all eslint rules related to code formatting are handled by prettier. For the most part we use the out of the box rules provided by the configurations that we extend but there are some tweaks.

As well, as mentioned above, there are some objects that live in the global scope, e.g. Pusher. We need to tell eslint that it is defined otherwise it will complain that it is not defined. This is where the eslint globals section comes in handy.

...
  globals: {
    InstantClick: false,
    filterXSS: false,
    Pusher: false,
    algoliasearch: false,
  }
...

Husky, lint-staged

The code base comes with pre-commit hooks that allow us to do things like run eslint before things are committed. If there are listing issues that can be fixed, they will get auto fixed and committed. If there are issues that cannot be resolved, the commit fails and the changes need to be handled manually.

Storybook

The dev.to frontend codebase uses Storybook. This is used to develop/showcase components. There is custom configuration for it that can be found in dev.to/app/javascript/.storybook at master · thepracticaldev/dev.to · GitHub.

Writing a Storybook Story

The Storybook documentation is quite good, but if you're looking for some examples, see dev.to/app/javascript/src/components/stories at master · thepracticaldev/dev.to · GitHub.

Stuff to do for Storybook

This is currently not deployed to Netlify, but there is an issue open for it.

Deploy Storybook #338

nickytonline posted on Aug 13, 2018

Task

Storybook can be run locally, but I also put a script in place to generate a static site for it. I was waiting for the project to go open source before we did this. If you run npm run build-storybook, it will generate a static site for dev.to's Storybook. It currently builds to the ./storybook-static folder. You can deploy that folder wherever. I'm assuming Netlify as that's where the dev.to docs are.

As well, you'll need to add a DNS record for the name of the storybook site, e.g. storybook.dev.to

I also took the precautionary measure to add this folder to the .gitignore

Definition of Done

As part of the CI npm run build-storybook should run and if it fails (most likely because someone forgot to update them), the build should fail. When PRs are merged to master, the above should still happen, but as well the ./storybook-static static folder should be deployed to a service like Netlify that will resolve to a web site URL such as https://storybook.dev.to.

Feel free to ping me @maestromac or @benhalpern if you need to discuss any of this.

View on GitHub

This part of the code base probably needs some love. There is probably a lot of low hanging fruit in here for frontends interested in contributing as I believe there are several components that are not in Storybook.

Theming

Contributors Wanted: Night Mode/Themes on DEV

Ben Halpern for The DEV Team ・ Mar 29 '19

#contributorswanted #opensource #meta #css

I was not a part of this initiative, but I know it uses CSS variables heavily for theming, with fallbacks. A great way to do modern theming.

So all that is themeable always applies the CSS variables with whatever their current value is (unless all you have is the fallback because your browser doesn’t support CSS variables).

The magic of theme toggling can be seen in action in the user configuration. Here we can see some style being applied if it’s the night theme or if it’s the pink theme.

dev.to/_user_config.html.erb at master · thepracticaldev/dev.to · GitHub

<script>
  try {
    var bodyClass = localStorage.getItem('config_body_class');
    document.body.className = bodyClass;
    if (bodyClass.includes('night-theme')) {
            document.getElementById('body-styles').innerHTML = '<style>\
              :root {\
        --theme-background: #0d1219;\
        --theme-color: #fff;\
        --theme-logo-background: #0a0a0a;\
            ...
        --theme-social-icon-invert: invert(100)</style>'
    } else if (bodyClass.includes('pink-theme')) {
      document.getElementById('body-styles').innerHTML = '<style>\
      :root {\
      --theme-background: #FFF7F9;\
      --theme-color: #333;\
      --theme-logo-background: #fff7f9;\
            ...
      --theme-social-icon-invert: invert(0)</style>'
    }
  } catch(e) {
      console.log(e)
  }
</script>

So if you’re contributing to anything CSS related in the project, keep in the back of your head if you need theming applied to what you’re working on. Don't be shy, just ask if it's not obvious in the issue. @venarius has worked a lot on this, so he’s probably a good person to talk to about theming.

Unknowns

Service worker

I haven’t worked at all on anything service worker related in the codebase, so if someone can chime in on it’s usage, that’d be awesome 😺. I know it supports the offline page which is a lot of fun to draw on. Shout out to @aspittel for her great work on the off-line page! As well, I’m sure it also does a lot of caching, but again, I don’t know all the details in regards to this part of the code base.

Edge Caching and the frontend

I have not done any work in regards to edge caching, but I know that dev.to uses Fastly. I imagine all the frontend is heavily cached on a CDN worldwide. @ben I feel like you could probably elaborate more on this part. 😺


Hopefully this sheds some more light on the dev.to frontend for folks. 👋

Additional resources:

• Frontend · DEV Docs
• ESLint - Pluggable JavaScript linter
• Prettier · Opinionated Code Formatter
• GitHub - rails/webpacker: Use Webpack to manage app-like JavaScript modules in Rails
• webpack
• GitHub - typicode/husky: 🐶 Git hooks made easy
• GitHub - okonet/lint-staged: 🚫💩 — Run linters on git staged files
• Storybook: UI component workshop for frontend developers
• InstantClick — JS library to make your website instant

Photo by Milad Fakurian on Unsplash

Comments

[rhymes]

Great write up Nick! I noticed there's still jQuery lying around which is used in various pages (grep for $()

[Jess Lee]

Wow Nick, THANK YOU for this. I think we should definitely add this article somewhere on docs.dev.to if you haven't already submitted a PR!

[James Sral]

I've been using webpacker at work with the widget approach and it rocks! Stimulus had also been great for smaller, less render-heavy bits.

[yucer]

Basically the way it works is if a user hovers over a hyperlink, chances are their intentions are to click on it. InstantClick will start prefetching the page while a user is hovering over a hyperlink, so that by the time they do click on it, it's instantaneous.

I hope it has no equivalent in the mobile version, because there I use it mostly with data charge$$$ from my telecom company. Normally one doesn't want to pay for reading stuff that would eventually read.

Is there any feature to save an article to read it offline afterwards ? I mean like the "reading list". Something like the "offline reading list" ? That would be better for the mobile version because allows to explicitly select the future stuff to read.

Most of the time I have to read articles in the subway or another areas without internet coverage. For others applications what I do is that I save the content in my handy using home wifi when I know in advance that I would not be able to connect to internet. Does anybody else have the same uses cases ?

Or maybe the solution is to specify which tags can be prefetched in advances. If the link points to one article of those authorized tags then InstantClick might be used. At least is a little more of control.

Maybe an "interest" field by tag ? Or allow the user to sort the tag list and set the Instantclick threshold?

Also the prefetched articles by the hover action could be accessible from the "offline reading list".

Maybe I am proposing stuff that is already covered by Instantclick, in such a case just ignore it. :-)

[Nick Taylor]

For InstantClick, on mobile devices, preloading starts on “touchstart”. I'll update the post with this. For offline reading, that sounds like a great feature request.

[Ahmad Awais ⚡️]

This is Awesome! 💯

[Robert Cooper]

Nice, never heard of Instant Click, but it's a great idea for optimizing performance.