Snapshots are a great tool for testing. It enables you to ensure that something always results exactly the same thing as before, which is absolutely useful if you're unit testing pure functions. UI Components are (or should be) pure functions, so, why does the title of this article state that we shouldn't use it for UI components? Allow me to explain.
The problem
Let's imagine the following situation. You developed a card component showing an image and the title of your blog post on your personal blog. You then decide to write unit tests for this component to make sure that it shows both the image and the title.
That's an easy one, just snapshot it, and you're good to go, right?
Let's write it down:
describe('Card', () => {
it('should show image and title', () => {
const { asFragment } = render(() =>
<Card image={/*some url*/} title="Title of my Post" />)
expect(asFragment()).toMatchSnapshot()
})
})
Boom! Your snapshot now has the markup for the whole component. You're covered.
Now you want to add a button to the component so that your readers can actually go to the post and read it, cause you know, you actually want people to read your posts. You make the change, boot up the development server of your blog and it's there, working beautifully.
Then you run your tests and they fail...
You read the test description 'should show image and title', look at the development version of your blog and you clearly see that both the image and the title are being shown, plus the new shiny button.
I hear you saying: "Well, don't be stupid, just update your snapshot!"
Update snapshot
You're right, I forgot to update my snapshot. Now I have to look at the snapshot, compare the old and new markup, assess whether the changes are intended and update it.
I have one question for you: Who's making the assertion, is it you or your test?
It's easy to do it with one component, but what will it happen you have 50 different components using the changed component and all the snapshots tests break?
We write tests to assure that our components do what they need to, fulfill its contract. The moment you are the one making the assertion instead of your test, you're swapping roles. That's literally the same as doing a manual test.
Plus, this is such dangerous behavior. It put's you into a mindset of: "I made a markup change, just update the snapshot, no need to check". That's how you just slip in a buggy component.
Tests resilience
We can also talk about the resilience of our test. The test states that it shows both the image and the title. While the snapshot does show that both of them are there, it actually does way more than that. A snapshot makes sure that the output of your component is exactly the same and before. This makes your codebase resistant to refactoring, which is most certainly not a good thing.
Your tests shouldn't care about the implementation, they should care about the results and if it meets the specs. This way you can ensure that you don't have a false negative out of a test. This test should never fail if the image and the title are being shown on the final markup, regardless of how that's achieved.
The solution
I hope that by now you do understand my reasoning on why snapshotting UI is a bad idea.
The solution is simple: make assertions!
A couple of years ago that was annoying, I agree. But now we have @testing-library with super amazing queries like getByText, getByRole, and more. If you haven't heard of, take a look at it. It's really amazing.
Let's refactor using them:
describe('Card', () => {
it('should show image and title', () => {
const title = "Title of my post"
const url = "some url for the image"
const altText = "description of the image"
const { getByText, getByAltText } = render(() =>
<Card image={url} title={title} />)
getByText(title)
expect(getByAltText(altText)).toHaveAttribute('src', url)
})
})
A few considerations:
- Meaningful error messages. Snapshot delivers the job of finding out what's wrong with the component to you. You're the one making the comparison. You do get a nice diff, but that's it. With this refactor, now the error messages actually tell you what's wrong. Be it not finding a component, which means somehow you screwed up the rendering or you changed the API of the component and have not updated your tests to cover all the changes.
- No false alerts. Now, if by any means, you change the markup, add or remove anything other then the image and the title, the test won't fail and you can safely iterate on this component and refactor it to make it better in the future.
-
You are consuming the component as the user will. The queries provided by
dom-testing-libraryforce you to use your components just like a user would (e.g. looking for the text on the screen or looking for the alt text of an image).
Conclusion
Writing snapshot tests for your UI components has more cons than pros. It enforces a codebase that resists change. Testing for its behavior and making specific assertions, on the other hand, leads to no false alerts and more meaningful error messages.
How do you feel about this? Add up to the topic in the comments below. Let's all discuss and learn.
Top comments (13)
This post makes a good point about contract testing, which snapshot tests are not well suited for. However, UI components in particular also need to produce valid/intended HTML markup, which is very awkward to do with individual assertions, and very easy to accidentally break in subtle ways during development - snapshots are great for catching the stray empty
<span>, or a classname that should be present but was accidentally removed or misspelled. In UI code, ordering of elements can also be important, but it would be awkward to write an assertion to test for ordering, and the result would be something more fragile and difficult to maintain than a corresponding snapshot test.One way to frame this is that, although using assertions is more appropriate for ensuring that the component does what is expected for upstream code, in many cases snapshots are more appropriate for ensuring that the component correctly works with downstream (rendering) processes.
Either way, it's definitely valuable to think critically about which tool to use for a particular test - I just think that snapshots have particular value the closer you get to the front end - just make sure you're shallow rendering so that you're not double-testing nested components!
Even if I agree with what you said on how snapshot testing isn't great for testing individual components, I find it very useful to test side-effects. When you update a component, you know exactly how much components are affected by that change and you can easily check if that's wanted or not.
It's also great when you update dependencies, by just running the snapshot tests you can check if a minor update didn't break anything and what a major update broke.
I don't think you should do one or the other, they're very different and having both can help a lot : Assertions to test an individual component's behavior and Snapshots for side-effects.
Interesting. You realize you are essentially duplicating your tests when you snapshot the component AND the components who use it. If they have already been thoroughly tested, shouldn't you be able to trust the components? Shouldn't all the use cases be described in the component test suit? If you're using a third-party library and you update it and it meets all the requirements previously met at your tests, doesn't that mean it doesn't break anything? Don't you think having actual assertions for all the requirements of your components would suffice (aside from giving you more precise error messages and being more resilient to changes)?
Totally agree.
You usually write tests to be sure that your code changes are not breaking stuff and only changes the components you want.
Imagine you're working on a project along with another 40 small teams. You use some shared components, and you make the change in a shared component, some other teams are not expecting neither functional nor visual changes to component.
As to use your example. You have a card with image and text, you're using this component for listing of blog posts and you expect it to have a link so that clicking on it will lead to a blog post. Other team is using the same component to render the header of blog post on mobile view, with the same image and title, and they expect no link there. Here's where snapshot testing will help you out.
Also using your example, I'd say it makes more sense to have a new component that would be
clickableCardorcardWithLinkso that you don't change nor add functionality to a visual component.The most common complaint I've heard about snapshot tests is exactly this; but if this happens then you're doing something wrong. In my experience snapshot tests are absolutely fine if you shallow render the tested component and limit the scope of your snapshot. So, if you are testing a large component, write tests that target specific elements and snapshot those; rather than taking a snapshot of the whole component. Then when a test fails the diff is kept small enough to parse quickly and establish if the change is expected.
Of course there are times when you shouldn't use them; but for dumb components they can definitely be a massive time-saver and, as Michael McGahan says, they're the most effective way to test rendered markup if that happens to be critical to your application. I feel their only drawback is a lack of understanding of how to apply them effectively :/
Fair enough, if you limit the scope of your snapshots you can indeed minimize the problem and save time. I still feel like saving time on your tests means losing time later that you'd not have lost if you had better error messages.
Would you fill a single test with all the assertions required to properly test your component? Obviously not... No-one should be doing that with snapshot tests either.
To be fair one other problem with snapshot tests is a certain amount of hype they've attracted that means their value has been oversold. You still have to think carefully about how you structure your tests in order to be most effective. So no, a single snapshot test cannot magically replace multiple, properly targeted tests.
But I'd still contest the time-saving aspect. When I see the diff on a failed test it's fairly trivial to establish whether it's styling/layout related and quickly update where appropriate. And when output is not as expected do I need an error message that says the output doesn't match the expected value? No: that's also immediately obvious from the diff.
I've seen some terribly written assertion tests that unnecessarily increased the burden of maintenance; that could easily be replaced with equally (if not more) effective snapshot tests; but that doesn't mean I won't ever write assertion tests...
Don't blame the tool: just learn how and when to use it properly ;)
Alright, you do have a point. As usual, it's a matter of trade-offs. Thank you for taking the time to discuss it. I personally have been moving away from snapshots and shallow rendering and definitely felt the weight that they add when refactoring components on an old enough codebase. I don't think testing implementation on the front (e.g. markup) is healthy for a long term project. I've been favoring testing of behavior with
@testing-library/reactinstead. They are much more resilient. That said, they don't cover the visual aspect, but we could argue that a visual testing tool is more suited for this job.Thank you this validated my thoughts on snapshot testing.
I also disagree with both the examples and the conclusion.
There are other snapshot testing tools, that do a way better job at remedying the underlying problems of snapshot testing (noise and redundancy). Have a look at recheck-web (e.g. opensource.com/article/19/10/test-...). It's currently only implemented in Java, but a JS implementation if coming... any thoughts?
50 components. You shouldn't render the output of other components in your snapshot.
Author missed shallow rendering.
Two different topics I feel. I'd be interested in hearing your thoughts on it. I feel like the same arguments apply.