In this post join me as I migrate my project’s test runner framework from Jest to Vitest, and check if it is really as they claim - “A blazing fast unit test framework”.
Vitest?
I’m pretty sure you’ve already heard about Vite which is Evan You’s recent promising project for frontend tooling. Benchmarks claim it is super fast in times where bundling and HMR present real challenges for dev velocity.
It was recently brought to my attention that there is another emerging project called Vitest which is supposed to bring that same disruption into the realm of test runners. My curiosity obviously could not let this pass.
Although “Vitest is still in development and not stable yet. It's not recommended to use it in production.” I was eager to find out just how fast we are talking about.
Here's my path for migrating a project using Jest into using Vitest along with some benchmarking.
Benchmarking
My guinea pig is (yet again) my React Word-Search game and its tests, but before I jump in I would like to measure how long it currently takes to run the project’s tests in order to do some comparison at the end.
The Project currently has 4 test files which have 37 tests between them. Not too many, I agree, but I believe it can give a good sense of the difference between Jest and Vitest.
I will measure the tests running time in 2 scenarios:
- Initial run - where I first lunch the test
- Watch run - where I change a single file and see how long it takes for the tests to run
For each framework I will run these scenarios a few times (with and without cleaning the cache for Jest for the initial run) to make sure I get the average time.
Here are the results for Jest I got:
- Initial run - 6.5s (without clearing Jest cache its 5.5s)
- Watch run - 5.5s
Migrating to Vitest
Vitest has a really comprehensive gitHub repo, with some good documentation (considering its age).
I could look into the code examples for React and React Testing Lib and monkey copy-paste it with fingers crossed, but I would like to know what’s really going on, and what is the exact minimum required for one to make this transition.
Before I even start to npm install anything, I’d like to try it as they suggest by running a single test. This following test is passing with Jest, now let’s try it with Vitest:
npx vitest src/components/Confirmation/index.test.js
After confirming the installation of ‘vitest’ we get our feedback from the test runner - who could have guessed, the first error (I must admit that Vitest has a more clearer way of displaying the errors and failed tests):
Error: Failed to parse source for import analysis because the content contains invalid JS syntax. If you are using JSX, make sure to name the file with the .jsx or .tsx extension.
While Jest has no issue with parsing these files, it appears that Vitest does not know how to parse them, and requires that we change the file names if they contain JSX.
Before I jump into introducing new configurations I would like to see if just changing the file name will help with this error, and it does - changing the name of my test file from index.test.js to index.test.jsx eliminates that error, but now I’m getting a new one:
describe is not defined
Well, Jest has these globals declared, but it seems that Vitest does not, and we need to import them explicitly. No worries, let’s do that, but before we do, we need to install Vitest. We at least know now that running Vitest just by using npx is not enough when migrating a project to work with it.
npm i vitest -D
Now let’s add the needed imports to our test file:
import { it, describe, expect } from 'vitest';
Oh my, now all my tests fail with a lot of errors flying, but that’s good. Let’s address them one by one:
document is not defined
This error comes from react-testing-library and it has to do with js-dom support of vitest. I’m going to look for some resources for this… yes, the docs do not fail - it says that adding a docblock or comment specifying the env as js-dom (or dom-happy) will do the trick. I will add it to my test and see how it goes:
/**
* @vitest-environment jsdom
*/
describe('Confirmation component', () => {
. . .
The tests run again, but still all of them are failing, now with new error:
Invalid Chai property: toBeInTheDocument
Chai? No, no, no… toBeInTheDocument is not a Chai property.
toBeInTheDocument is an API of the testing-library’s js-dom, and the part responsible to include it and append its assertions is the test setup file (in create react app it is the testSetup.js file on the project root).
In order to let vitest include this file as its own setup we need to create a vitest config, no escape there. Now is a good time to look at the configuration found on the example and check what’s going on in the configuration there. Again, I’m not blindly copy-pasting and so I pick what I know to be relevant to the problem I’m facing.
In order to use the configuration I need to install “vite”. I’m not very keen about it, but if that makes my tests run faster, so be it:
npm i vite -D
I create a configuration file named “vite.config.js” and set the configuration as follows:
import {defineConfig} from 'vite';
export default defineConfig({
test: {
globals: true,
setupFiles: 'src/setupTests.js',
},
});
As you can see I’m giving the setup file location, which loads the jest-dom needed, and also notice that I have the global property set to “true”. This means that I won’t need to import those global variables Jest comes with like “describe”, “expect” etc. I can remove that import from my test :)
(more information on the configuration can be found here)
Good progress, but do our tests pass now? No, some still don’t. We have another error:
jest is not defined
Well of course it isn’t. We’re using jest in this test for creating spy/stub functions with jest.fn(), but Vitest has another way of achieving this - it has the same implementation but under “vi”. So instead we need to use vi.fn()
import {vi} from 'vitest';
it('should be able to receive a handler for the "Cancel" button and execute it upon click', () => {
const onCancellationHandler = vi.fn();
. . .
});
Hurrah! We have a single test migrated into Vitest :)
I will now attempt to run the entire tests with vitest. I will start by changing my npm script for test to run vitest instead of jest:
"scripts": {
"test": "vitest",
. . .
},
Let's also add the environment: 'jsdom' to the configuration file so we can avoid adding the env docblock in each test.
Running npm tests, and as you probably guessed it, many tests fail, but the good news is that there is nothing new to the issues we already bumped into before.
It is time to do some benchmarking
Benchmark again and compare
Now it is time to take our statistics again for Vitest:
- Initial run - 5.30s (nice, but kinda the same as Jest with cache)
- Watch run 1.25s (wow!)
Let’s put it in a nice table:
| Framework | Initial run | Watch run |
|---|---|---|
| Jest | 6.50s | 5.5s |
| Vitest | 5.30s | 1.25s |
From this little benchmarking I did here on my own machine, it appears that although the initial runs are slightly in the favor of Vitest, the watch run is a lot faster!
As I see it, there is no question that once Vitest is ready for production you should really consider replacing your current test runner with it. My Word-Search game already has it ;)
As always, if you have any thoughts or comments about what's written here, please share with the rest of us :)
Hey! If you liked what you've just read check out @mattibarzeev on Twitter 🍻
Photo by Florian Steciuk on Unsplash
Top comments (25)
I've been using vitest because of remix stacks. Almost everything feels the same as jest but I notice a few things pragmatically.
Pros
Cons
.onlyand.skipdon't seem to work as well for mejest setup has been stressing me out all day, I'm happy that at I came across vitest today and then this article helped me solve the document is not defined issue that i ran into when using vitest. Thanks for writing
Glad I could help 🍻
My friends, family and colleagues told me I shouldn't run off after another test framework, but now I know we're right and they're wrong. Thanks for this article!
Love this ❤️
Thanks!
Thank you for this post!
Can I use vitest for company new project now?
I believe you can, but make sure it answers all your company's needs
Oh, that looked surprisingly simple!
I am still a bit unsure how enzyme would fit into the Vitest environment and how easy it is to setup with next.js - but the Vitest project sounds SO promising!
It is suggested to drop enzyme tests, because it is not supported any more. Keeping aside that it brings questionable value.
Thanks Emil. I have also come to that conclusion myself 🙂
Vitest is not that fast as it pretends to be. Jest is faster in most cases, because usually you expect isolation in tests, but Vitest has problems with performance. Here is a Github issue github.com/vitest-dev/vitest/issue....
I’m definitely keeping my eyes on vitest and this post makes me wanna give it another shot (forget exactly why but had trouble getting it working a few weeks back) but the whole philosophy of vitest is really compelling. Thanks for the nice case study!
With pleasure :)
You showed how to define
setupFilesin vite.config, but you did not reference the contents ofsetupTests.js.Could you include what should be in there for the Chai issue to resolve?
Remix Vite plugin can't detect preamble. Something is wrong.❯ app/routes/stored.tsx:2:8
1| export default function Comp(){
2| return <h1>jcs</h1>
Error: Remix Vite plugin can't detect preamble. Something is wrong.
here is my vite.config
///
///
import { vitePlugin as remix } from "@remix-run/dev";
import { installGlobals } from "@remix-run/node";
import { defineConfig } from "vite";
import tsconfigPaths from "vite-tsconfig-paths";
installGlobals();
export default defineConfig({
test:{
environment:"jsdom",
globals: true,
setupFiles: ["./set-up-test.env.ts"],
},
plugins: [
remix({
ignoredRouteFiles: ["*/.css"],
}),
tsconfigPaths(),
],
});
can you help me solve this
I hope vitest gets more traction and becomes a solid alternative to jest for vue and svelte also soon. thanks for the write-up
Hard to tell how much improvement there is, since we don't know how many & how complicated the tests are.
I mean he literally said it was 4 test files, and 37 tests. A small sample size, sure, but large enough to show relative progress.
I'd love to see some numbers for larger codebases, but that's tough since the software is still considered not production ready.
True. I won't use my little PoC here to determine whether or not you should use Vitest, but I do say it does look promising.
as always, love to read your articles!
their comment on not being ready for production and the tight coupling to vite will cause me to wait some more :)
Cheers :)
Yeah, I'd wait for it some a little bit more
Thank you for this post, just like a "TL,DR" for me to migrate to vitest.
Cheers :) Just remember that this is not production ready yet, according to their readme.
Great post!