loading...
Cover image for Jest Snapshot Testing: a blessing or a curse?

Jest Snapshot Testing: a blessing or a curse?

vpukhanov profile image Vyacheslav Pukhanov ・4 min read

Path to Jest

My current project at work is a React app. I've joined this team only a couple of months ago, but I've been happy to learn that my new colleagues have been considerate of the code quality: the codebase is 100% TypeScript, excluding some Node modules, which I always prefer over JavaScript, and most of the components and helper functions are covered by unit tests, which use the Jest framework.

At that moment in time, I was new to Jest. Of course, I liked using it for personal projects, whenever I was in the mood to write some tests, but I've always used Mocha and Chai at work.

Many of the existing unit tests in the codebase were snapshot tests. And very soon I fell in love with them and started using this approach as often as possible. What's not to like: just make some test data, pass it to a function or a component, and compare the output to the snapshot. Boom, the test is ready, and there's no "testing fatigue", where I get a bit tired and absentminded after writing a bunch of tests in a row.

But are there any disadvantages to this type of testing? They can't be a panacea after all.

What exactly is being tested?

Snapshot tests can make it harder to understand, what exactly is being verified by the test. Consider the following example without the snapshot testing first:

const rootState = { /* ... */ }; // initial state of the app for testing

test("increaseTemperature should increase initial temperature in all rooms by one degree", () => {
  const initialState = { ...rootState };
  const action = increaseTemperature();

  const reducedState = reduce(initialState, action);

  reducedState.rooms.forEach((room, index) =>
    expect(room.stats.temperature).toEqual(
      initialState.rooms[index].stats.temperature + 1
    )
  );
});

Now let's take a look at how this could be written using snapshot matching:

test("increaseTemperature should increase initial temperature by one degree", () => {
  const initialState = { ...rootState };
  const action = increaseTemperature();

  const reducedState = reduce(initialState, action);

  expect(reducedState).toMatchSnapshot();
});

At the first glance, this test is much easier to write and read, and the result should be the same - by running this test we are making sure that the behavior and the changes that the increaseTemperature action introduces are consistent. And while this is true, some of the benefits of the test were lost.

For example, now you can't see which fields exactly are expected to change after the increaseTemperature action is applied. Without the snapshot testing, you were clearly able to see, that this action changes the stats.temperature of every room, but this knowledge is now lost or at the very least obscured (since the purpose of the test is only available in its name now). As someone who considers unit tests to be a part of my projects' documentation, I consider this important.

Another point to think through is that changing the initial temperature state now requires you to update the snapshot. The prior assertion would have still passed even with this change because it explicitly checked the reduced value against the value in the initial state. And every snapshot update takes some of your precious time since you have to make sure that changing the initial temperature didn't accidentally change something in an unrelated part of the snapshot, especially if it's large.

Is it even automated anymore?

Let's imagine that we've developed a Greeter component, and we want to test that it renders consistently. With snapshot testing we might've written something along these lines:

test("Greeter should render correctly", () => {
  const { asFragment } = render(() => (
    <Greeter name="John" />
  ));
  expect(asFragment()).toMatchSnapshot();
});

This, of course, works just fine. But some time has passed, and after some seemingly unrelated changes, the test started failing.

To understand the source of the issue we have to manually check the difference between snapshots to understand if it fails because of some minor markup change (like extra or missing whitespace, or a couple of new div's), or if the Greeter component is actually broken.

And at this point, can this testing even be considered automated? This is an example where snapshot testing made it really quick to write the test and introduce it into the suite, but it actually increased our workload down the line.

This is how this test could've looked like with regular assertions:

test("Greeter should render correctly", () => {
  const { getByText } = render(() => <Greeter name="John" />);
  getByText("Welcome, John!");
});

This test is in multiple ways better than the original because it required us to explicitly state, what exactly we consider a "correctly rendered" Greeter (in this case, we thought that if the component greets the user, it's good enough for our needs). It also provides better error messages in case of failure, which will make it easier to find the reason for it - Jest framework tells us, which specific assertion has failed, so we don't need to manually look through the snapshot - yay, we got our automation back!

Should I never use snapshot testing?

No, of course not, snapshot testing is an important part of our toolbox, but it's a typical case of picking the right tool for a job. When it's easy to see, what you're actually checking with a test, even if you use a snapshot matcher, go for it! If you're testing a very simple component, which you don't expect to change too often, snapshot test seems to be the way to go as well.

I just wanted to remind you, that while Jest Snapshot tests are a powerful tool, they shouldn't be used just because they're quicker to write. You should always take a moment to consider, what you win and what you lose by using the snapshot matcher in the test.

Hopefully you've got some food for thought from this writeup! If you have something to add, I'll be glad to discuss this further.

Posted on Jun 20 by:

vpukhanov profile

Vyacheslav Pukhanov

@vpukhanov

Full Stack Web Developer and a hobbyist Mobile Developer

Discussion

markdown guide
 

Most criticisms of snapshot testing I've seen can usually be resolved by writing more thorough tests.
I think the biggest problem is that snapshots make it far too easy to be lazy: people think all they have to do is add expect(myRenderedComponent).toMatchSnapshot() and then they're done :/

The most common problems:

  • grabbing a snapshot of the whole component/object when only a small portion is relevant. Use a selector to better target the test subject; or extract the properties that are relevant and put those in the snapshot. You then make what is being tested explicit.
  • testing mounted components. Use shallow rendering unless absolutely necessary; otherwise changes in nested components can break your test

IMO this addresses both your main points. The test author makes the decision of how much markup to include in their snapshot; and they should base that decision on whether or not it's important to catch and review changes in that markup. Any automated test can fail and will require human intervention. It's the responsibility of the test author to consider this and write tests accordingly.

 

Thanks, Ben! I agree, the ease of writing snapshot test is both a huge benefit and a major pitfall because it makes it easy to ignore the important decision step, where the test author figures out, what exactly do they want to verify.

I think you're spot on about the shallow rendering. I also often recommend snapshot testing for simple or very rarely changed components. As for snapshotting an extracted subset of properties, I'd argue that it's very similar to just writing an explicit assertion, so it's probably down to personal preference.

 

Jest snapshots are great for testing APIs and json payloads. If you are doing full stack TS/JS, you should definitely use snapshots everywhere!

You're absolutely right that it can get a little weird sometimes if some component somewhere in your tree shifts slightly and now your test fails, but you should always look at the snapshot to make sure the change makes sense, not just commit it. My perspective is that I work with a lot of teams and a lot of developers and snapshots help make sure there is some kind of test.

If you dig into the matchers and leverage best practices with your component test harness (enzyme still? don't laugh at me - it's been a little while), then you can get great results with snapshots.

 

Thank you, Matt! That has been my experience as well, snapshot test is much better than no test at all. I just found that often explicit assertions provide much more context to the test, which can help solve some issues down the road.

Yeah, enzyme is a popular choice 👍🏻 We're using @testing-library/react though, but it's a matter of personal preference.