Raising the Limits on Developer Speed

How we used React/Redux, Jest, and automation to rebuild our dashboard in only three months

Up until five months ago, our dashboard was slow, limited, and (worst of all) written in Angular. We don’t have anything in particular against Angular itself, but nobody on our team had any significant experience with it. This meant that fixing even a tiny bug took a large amount of effort. We had big plans to add awesome features to our dashboard — organizations, multiple applications per user, queryable logs, and more! — but we weren’t convinced that building on top of our legacy code was the best use of our time. So we scrapped it.

Our goal was to build the new dashboard in a way that would enable us to develop quickly and effectively. We knew we wanted to use React, to keep in line with our other in-house JavaScript projects, but the dashboard has a considerable amount of application state, so we decided to use Redux as well. We started by taking a bunch of pointers from Bumpers’ Isn’t our code just the *BEST* 🙄 article.

Instead of having all of our store files directly at the /store level, as is common in many Redux applications, we split the store into several subdirectories, each one with their own actions, reducers, and selectors files. Each of these subdirectories correspond to one of our models, e.g. Account, Organization, Application, etc. This made development much faster by grouping relevant code together. Now, when someone wants to make changes to the Account state, they only have to navigate to /store/accountto find that code, rather than having to ctrl+f in massive files that bunch all of the code for different models together.

Sidenote: Another point we took from the Bumpers article was the addition of an endpoints file in each of these store subdirectories. The API calls can be handled in this file and called in-turn by the actions file. This is a nice separation-of-concerns that makes it easy to switch out either the actions or the API interface without affecting the other, and provides a clean stub interface for testing.

This separation made it extremely easy to modify stores, but unfortunately, adding new stores required a lot of prep work. For each new store, we had to create a new directory, add a bunch of files, and write the same initial skeleton code — all work that could be automated! We wrote a scaffolding script to do it for us and added it to our package.json file. We were able to run this script with Yarn via yarn scaffold <type> <name>. We added several other pieces of skeleton code to this script, so we could create quickly components and model definitions as well as stores. This made the process of adding new code super easy and fast. Most of the code only needed a few details to be fleshed out and then it was good to go. 👍

Another thing that we did to increase our development speed was to use Jest for testing. We believe testing is an important part of development, but writing tests can significantly slow down the development process. Jest’s visual output is fantastic (on par with pytest for Python), but the real kicker is its concept of snapshots.

Using Jest’s snapshots is as simple as expect(testData).toMatchSnapshot(). The first time this test is run, the user will be prompted to inspect testData and confirm that it has the expected value. Jest then creates a snapshot file with the confirmed data value. On each subsequent test run, Jest will compare the current value of testData to the value stored in the snapshot. If the values do not match, the console will show the differences between the two values and ask if the snapshot should be updated.

This is much faster than having to 1) figure out what the value of data should be when initially writing the test (particularly if it has a complex structure), and 2) having to go change a bunch of hard-coded test values when something does legitimately change (which happens a lot in early development stages). We were able to add skeleton tests to our scaffold script, so each React component automatically got a test like this:

// @flow
import React from 'react';
import renderer from 'react-test-renderer';
import NewComponent from './index';

test('renders as expected', () => {
  const component = renderer.create(<NewComponent />);
  const tree = component.toJSON();
  expect(tree).toMatchSnapshot();
});

This helped ensure that all of our components were tested. If it was a simple component, the skeleton test was all it needed. If it was a more complex component, the test would fail and prompt the developer to update the test to be more accurate. This effectively eliminated the case where developers forget to write a test for a component, and not having to do any extra work to test the simpler components was a huge plus.

Sidenote: Prettier, an auto-formatter that we talked about in our tooling blog post, also did wonders for our development speed. You don’t realize how much time you spend on spacing until you no longer have to do it.

As these changes were integrated with our workflow, we were able to develop faster and faster. We completed the entire dashboard project (along with the associated backend changes necessary to support the new features we wanted to implement) in an astounding 12 weeks. The key lesson we learned from this experience was that investing the initial time into building a strong developing foundation is well worth the effort! For this project, building that strong foundation involved grouping code together in a functional way and automating as many patterns as we could. Going forward, we’ll be sure to look out for ways that we can do the same in our other projects!

We had a rising trend in the number of commits to the dashboard project, until it neared completion. This was with a team of 4-5 engineers, depending on the week.

---

This post was originally published on the Nylas Engineering Blog.

Comments

[rhymes]

This post is so great, thank you for sharing it Halla. I'm building a web app in Vue and definitely didn't know about toMatchSnapshot(). Also I might end up adapting your scaffolding script later on ;-)

[Frank Carr]

It's an amazing world where an Angular app is considered legacy code. The legacy code I usually get introduced to in interviews is stuff in languages/environments like VB6, Access, Powerbuilder, Visual Foxpro, etc. that was originally written in the late 90's or early 00's.

[Dawn Messier]

I love the scaffolding script. But I'm having difficulty implementing it. You say to add it to your package.json. I'm new to custom scripts there. I added "scripts" { "scaffold": "node scaffold.js" } to package.json. But when I run 'yarn run scaffold main statefulComponentTemplate' or 'yarn run scaffold statefulComponentTemplate', it tells me it's an 'unknown command'. What am I missing?

[Halla Moore]

So command is perhaps a bit misleading, but it's referring to the first argument to the scaffold script. The pattern is yarn run scaffold {command} {names}. The command argument has to be one of the cases defined in the switch statement on line 257 (gist.github.com/hallamoore/bf4e18e...), which are 'store', 'component', 'layout', 'screen', 'container', and 'model'. There's also a --stateful option which will use the stateful template, so you might want yarn run scaffold component statefulComponentTemplate --stateful. Hope this helps!

[Halla Moore]

Also, just to clarify, you can use any custom name rather than matching the name of the template. It's the command argument and --stateful option that determines which template will be used. E.g. yarn run scaffold component MyStatefulComponent --stateful will create a component named MyStatefulComponent with the statefulComponentTemplate, while yarn run scaffold model User will create User with the modelTemplate

[Carlos Florêncio]

Excelent post! Also experienced that awful workflow of copy/pasting folders to create new features. Can you share your scaffolding solution?

[Halla Moore]

Of course! Here's a copy of the code in our scaffold.js script: gist.github.com/hallamoore/bf4e18e...