Snapshot Testing in JavaScript & .NET

Earlier this year I discovered the fantastic Jest JavaScript unit testing library made by Facebook and commonly associated with React. Jest is built on top of Jasmine, but adds some very nice features, including something called Snapshot testing.

While unit tests are typically intended to be precision surgical tools focused on testing one or two very small things in isolation, snapshot tests are the polar opposites. Snapshot tests are effectively shotguns aimed at state verification. Snapshot tests take a look at a piece of state and compare it to a stored version of that state. If there is any difference whatsoever, the unit test will fail and a report of the differences in the JSON representation of that state will output as the test failure.

This is extremely useful for “pinning tests”. These tests are tests intended to capture the current application state and behavior, regardless of whether it’s right or wrong. If the same operation later produces a slightly different result, the test should fail and the developer will be confronted with the change in state — either an intentional change or an accidental one.
In the case of an accidental change, the developer fixes the potential bug they just introduced and the test goes back to passing. For intentional changes (such as fixing a bug that was previously represented in the snapshot), the snapshot is actually the thing that is wrong and so the developer updates the snapshot and the test passes again.

Snapshot tests are not very clear as to the exact behavior they test, but they do make for good safety nets for things that are not yet tested by specific unit tests.

JavaScript Snapshot Testing in Jest

In Jest, a snapshot test might look something like this:

myVariable.shouldMatchSnapshot();

That’s it.

The first time you run the test, a snapshot is generated (since none exists) and placed in a special __snapshots__ directory. This file should be placed under version control since it is vital to the behavior of the test.

Subsequent times, Jest checks against the stored snapshot file and passes or fails based on any differences.

If you need to update your snapshots, you just pass in an argument to Jest and it updates the snapshots automatically.

jest -u

The updated snapshot file is then committed to source control and subsequent test runs are made without the -u flag.

See https://jestjs.io/docs/en/snapshot-testing for more details on snapshot testing with Jest as well as steps for getting set up with the Jest testing framework.

Snapshot Testing in .NET with Snapper

Since .NET has a glorious history of plundering concepts, libraries, and techniques from other languages and I spend a decent amount of my time programming in .NET, I looked for something that offered Jest-like snapshot testing capabilities.

While there are a number of libraries available such as the very diverse and qualified ApprovalTests.Net, I chose to go with the simpler and more automated unit-test oriented Snapper for simplicity’s sake.

A sample test in Snapper looks like the following:

[Fact]
public void SamplePinningTest()
{
  // Arrange
  var resume = new ResumeInfo("Matt Eland");
  resume.Jobs.Add("Software Engineering Manager");
  // Act
  var result = Analyze(resume);
  // Assert
  result.ShouldMatchSnapshot();
}

This will look for the snapshot JSON file inside the __snapshots__ directory, perform a diff, and fail the test if any difference is detected.

Snapshot failures appear in unit test results like the following:

Snapper.Exceptions.SnapshotsDoNotMatchException : 
Snapshots do not match
- Snapshot
+ Received


{
  "Resume": {
-      "FullName": "Some Body",
+      "FullName": "Matt Eland",
    "Jobs": [
      {
-          "Title": "Test Engineer",
+          "Title": "Software Engineering Manager",
        "Organization": "Universal Exports",
        "MonthsInJob": 42,
...
    ]
  },
-    "Score": 42
+    "Score": 2147483647
}

In this, Snapper follows Jest’s conventions for the most part, but while Jest uses command-line arguments to update its snapshots, Snapper has to do something different.

Snapper requires you to add or remove the UpdateSnapshots attribute to your test method in order to generate the initial snapshot or update the snapshot.

I would argue that Snapper should be smart enough to auto-generate the snapshot if it is not already present as Jest does, but as of the date of this article, it does not do this.

In summary, Snapshot testing is a low-effort way to add a wide safety net to your application as you work to expand your test coverage to more specific and targeted unit tests. Jest and Snapper are easy to use and simple frameworks that make snapshot testing available in common programming languages.

Give snapshot testing a try and let me know what you think. Hopefully you will see its value as a specialized role in your suite of tests that reduces the odds of you unintentionally changing application behavior.