Making those React tests faster and nicer

I've never been the guy who would write a lot of tests, but I finally am getting my act together on it. The reason for my lack of interest on testing has been two-fold. As a hobbyist I never needed it. Instead I spent great deal of time learning to read and understand code well. When beginning as a pro I eventually shifted to writing code well enough you wouldn't need to fix it again later on.

Then the second reason. In my first workplace as a professional testing was one of the things that never really got the attention it should have. Thus tests were cumbersome, nobody really pushed them forward, and nobody was in the position to teach how to do testing. This resulted into slow tests and poor experience.

Speed was one thing that really kept me away from getting into testing properly. If you have tests that are slow to execute you don't want to even attempt to do TDD.

So now that I got my mind around actually focusing into testing I set a couple of goals. First of all I wanted things to be as clear and as minimalistic as possible. And after these I'd like the tests to be fast. Ideally in the milliseconds range, but there are a couple of things that make that impossible.

React and Jest together is a slow experience

I chose my old React Tabbordion project as the project to improve upon. It hasn't had the love and time it should have so it had a lot of improvements to be done. It is also a complex beast which makes it ideal for writing good tests for. Tests weren't the only thing that needed improving, there was a lot of need for fixing the internals as well.

Working with this project meant I wanted to do modern React which then meant I'm dependent on modern JavaScript and JSX. From the performance perspective this is awful! No matter what you do you have to suffer a penalty of transforming the code before executing tests. Every single time.

A typical test runner in the React world is Jest. I've never liked it, because it always takes ages for it to boot up and get into the actual testing. I know it compensates by running multiple tests in parallel, but with one single simple test a cold run around 9 seconds and 3 seconds when hot is an awful price to pay! I'd rather have my entire test suite be complete within a second if at all possible.

Alternatives to Jest

Since there seems to be no way to make Jest get itself up faster I had to get into learning all the various tools used in testing to see if I can find anything that would allow me to get something faster.

If we go for the absolute minimalist route we can find Baretest. It is very fast and simple, but this is also kind of it's downfall. You don't get tap output, no support for coverage, and you have to enforce good testing practises yourself. Baretest is great if you're going for minimalism with as few lines as JS as possible, no code compilation steps and no JSX. It certainly isn't for your typical React project.

When adding in features such as a need for coverage the next best on speed is Tape. It however is from the age before big advancements in JavaScript code which means there are some gotchas when using it, such as the asynchronous example where you have to t.plan(x) to incidate test comparisons will be called x times before it is ready. This is prone to mistakes so I'd rather have something that abstracts this issue away.

I then found Riteway. It is built on top of Tape. I have to say I dislike the name a bit and the author is very opioned on how he thinks tests should be done, but the main API looked very good to me:

// NOTE: pseudo test just to give the feel of it
describe('Component', async (assert) => {
    {
        const html = renderToStaticMarkup(<Component />)
        assert({
            given: 'no props',
            should: 'render a div',
            actual: html,
            expected: '<div></div>',
        })
    }

    {
        const html = renderToStaticMarkup(<Component className="test" />)
        assert({
            given: 'className prop',
            should: 'render a div with class attribute',
            actual: html,
            expected: '<div class="test"></div>',
        })
    }
})

I like the way this makes me think about tests. Everything you write has a purpose to be there and the encouragement to use block scope allows re-use of variable names without resorting to variable re-use.

The author however goes further with his ideals and encourages to write unit tests only for pure components, thus leaving out testing for features that depend on DOM and stateful components. This would limit tests running on Node to Server Side Render only. I get the impression that DOM and stateful components should be tested in real browser as functional end-to-end tests. This seems quite limiting.

In Tabbordion I have multiple components that depend on each other via the React Context API. Yet it is entirely possible to test these individually via unit tests. I would also like my unit and integration tests to cover all the logic as these tests are much faster to execute than booting up a functional test in a browser. Thus I'm going for as complete test suite running on Node as possible. I don't really care about hitting 100% coverage, but it would be good if most of the logic involving state changes is covered.

The downside of doing this is that you need DOM. There is no real minimalist solution here, the only way is to add dependency to JSDOM. This is unfortunate because importing JSDOM is a slow process when mangling code with Babel.

Finally there has to be helper utilities to work with testing the DOM. There would be Enzyme that also allows for shallow rendering and thus some limited testing of functionality even without depending fully on DOM. After playing around with it I decided it isn't worth the hassle, especially as importing Enzyme and it's React Adapter also has notable slowdown effect with Babel. Instead I found React Testing Library which is lightweight enough and focuses on testing React via DOM.

Speed comparisons

So where are we at this point?

1. Code: React JSX on modern JavaScript
2. Compiler: Babel
3. Test environment: Node with JSDOM
4. Test runner: Riteway (internally using Tape)
5. DOM testing utility: React Testing Library
6. Test output: tap-difflet (gives nice string diffs)

So here we have Jest replaced with Riteway. Do we get any benefits? For a simple benchmark I add in only one DOM test, because what matters to me most is the use case for testing a single component library and there won't be a lot of slow tests. The thing I want to reduce is time spent in all the other stuff than simply running the tests.

Jest version

import { fireEvent, render } from '@testing-library/react'
import React from 'react'

function Checkbox() {
    return <input type="checkbox" />
}

test('checkbox can be toggled', async () => {
    const { container } = render(<Checkbox />)
    expect(container.firstChild.checked).toEqual(false)
    fireEvent.click(container.firstChild)
    expect(container.firstChild.checked).toEqual(true)
})

Best time after a few runs: Done in 2.48s

Riteway version

import { fireEvent, render } from '@testing-library/react'
import dom from 'jsdom-global'
import React from 'react'
import { describe } from 'riteway'

function Checkbox() {
    return <input type="checkbox" />
}

describe('Checkbox', async (assert) => {
    const cleanup = dom()
    const { container } = render(<Checkbox />)
    const beforeClick = container.firstChild.checked
    fireEvent.click(container.firstChild)
    const afterClick = container.firstChild.checked
    assert({
        given: 'initial render with no props and then clicked',
        should: 'render unchecked checkbox and toggle to checked',
        actual: { beforeClick, afterClick },
        expected: { beforeClick: false, afterClick: true }
    })
    cleanup()
})

Best time after a few runs: Done in 1.87s

There is more code in the Riteway version. Some of it could be abstracted away. The remaining verbosity helps sharing understanding on what is being tested.

I'm also very happy with the reduction of total time spent, 600 ms is a big deal. But... can we do better? Well, we can! These days Babel is not the only option. There is a faster alternative called Sucrase. You can't use it for bundling, but it aims to be an option for development time. So it is a tool very valuable for testing.

Jest with Sucrase: Done in 1.93s.

Riteway with Sucrase: Done in 1.21s.

Conclusion

Working in modern React with all the bells and whistles has a drawback on performance and you can feel it through everything you do. You need to battle bundle sizes and boot times in the frontend, you need to wait for changes to happen when doing development, and you need to wait a bit of extra time when running tests. Most of the tests during development would run in milliseconds if there was no need to compile the code.

However there are ways to improve the status. By switching away from Jest to more lightweight solutions you can reduce time spent in the compile phase by simply having less code. Also switching Babel to Sucrase is a major improvement, and thanks to doing both I got from 2.48s (Babel+Jest) down to 1.21s (Sucrase+Riteway). And if I didn't need DOM I'd get it down by further 500 ms.

I'm not entirely happy with the time though. That 1.2 seconds is still an awful lot and I'd rather have my tests as instant as possible. Achieving faster times would require me to abandon compile step entirely which would also mean abandoning JSX and modern JavaScript.

What I am happy with is what my tests look like. They are much more readable and uniform than before, and improvements in tooling and docs have removed the mistake of focusing on testing implementation details.

Another major win is that because I use Microbundle for bundling I no longer have a need for tons of dependencies in my package.json! All the Babel stuff goes away and is replaced with a single sucrase import. There are also no more babel.config.js and jest.config.js. It is lovely to have less boilerplate stuff!

Comments

[Vesa Piittinen]

Note: after this I've noticed Riteway doesn't work with TypeScript and, last I checked, apparently the author isn't interested to fix the issue. I still like the overall simplicity though, the only problem is the ever-growing requirement of all libraries to have TypeScript support to be even considered for use.

[Rudy Nappée]

Have you tried vitest ? Same API than Jest but a lot faster ! And typescript/jsx enabled