What’s wrong with snapshot tests
When Jest announced snapshot tests in 2016, I was very excited. It felt like a perfect solution for testing React components. Since then I’ve written fewer and fewer snapshot tests. In this article I’ll try to explain why.
I’ll talk mostly about testing React components, because that’s where I’ve used or have seen snapshots the most, but the same can be applied to testing other things.
Why snapshots are problematic
Easy to commit snapshots with bugs
When you write
expect(mycomponent).toMatchSnapshot(), Jest creates a snapshot file, like
mytest.spec.js.snap, with all snapshots of your test file. You don’t see this file unless you really want to.
That’s a problem: you don’t know if your code is correct yet and, unless you carefully read each line of your snapshot file, you may commit code with a bug and a snapshot that ensures that the bug is still there.
Failures are hard to understand
It’s often hard to see what changed by looking at the snapshot failure diff: did the snapshot fail because of your intended changes or because you’ve introduced a bug?
We tend to update snapshots without thinking
On a big project a simple change can lead to failures in dozens of snapshots in different parts of the codebase. Understanding each failure takes a lot of time, so often developers just update snapshots without looking at them at all, especially if it’s not their code.
Coupling with low level modules
A little change in a Button component may lead to a failure of hundreds of snapshots, even when everything works fine. A Button component is an implementation detail, and tests should pass if a button still behaves like a button.
Test intentions are hard to understand
Snapshot tests like “should render the default state of a component” are common but don’t really explain what exactly, if anything, they’re testing.
Such snapshots will break on any change to markup, even if the behavior remains the same.
A false sense of security
Snapshots don’t verify that the component looks the same as before and don’t verify that the component behavior is correct. Despite this it’s easy to believe that they do, and make tests that don’t actually test anything, but give you high test coverage.
Snapshots only verify that the component renders (meaning its HTML, not how it looks in the browser) the same thing, which is rarely important knowledge.
What to use instead of snapshots
Let’s look at a typical test (I’ve written similar tests myself):
We have two snapshots here: both will break on any markup change. Intention of both isn’t clear. The first is likely covered by other test cases, so we can remove it. In the second we likely want to know if the success message was shown instead of a form, which isn’t at all clear from the code. We can test exactly that:
The test intention is now clear: we click the submit button and instead of a form we see a success message. This test won’t fail because of primitive components or third-party dependencies updates. This test won’t fail because of the layout changes that don’t affect the behavior.
When to use snapshots
I think snapshots are still useful in some cases:
- Very short output with clear intent, like class names or error messages:
When you really want to verify that the output is the same:
Babel plugins are a good example of that, see babel-plugin-tester:
- Use inline snapshots to make snapshots visible inside the test file and to help you to keep them small.
- Use snapshot property matchers to avoid storing generated values, like dates or IDs.
- Use descriptive snapshot names to make it easier to spot when a snapshot is incorrect.
For example, compare:
With the latter it’s clear when a snapshot is wrong, like this:
- Avoid snapshotting file paths: they’ll fail on other platforms because of different delimiters (
- Avoid snapshotting dates and times: even if the date depends only on the input, it may be different in another time zone, and your tests will fail.
Snapshots are useful in some rare cases, but often they give you a false sense of security while making the codebase harder to change.
Check out the Effective Snapshot Testing article by Kent C. Dodds for more ideas on how to use snapshot tests.
Thanks to Oliver Joseph Ash, Ovidiu Cherecheș, Manjula Dube, Matt Hamlin, Patrick Hund, Oleg Isonen, Morgan Packard, Oliver Turner and Andy Wermke for their feedback.