Unhandled Async Calls Cause Flaky Jest Tests

Daria Caraway — Mon, 12 Apr 2021 23:40:40 +0000

Have you ever come across a Jest test failure that seemed completely random and was incredibly frustrating? Most of the time when this happens to me, it's because I have an unhandled asynchronous error wreaking havoc on my testing suite.

The Symptoms

These are some symptoms that you might have a flakey failure due to mishandling an async call.

Different tests are "randomly" failing in the same file on different test runs.
When you run the tests individually, they all pass.
When you run a subset of the tests, they all pass.
When you give the tests more resources to run faster, they all pass.

The Cause

Say you have an asynchronous test:

it('should add 1 + 1', async () => {
  asyncFunctionFails() // kicks off on async call that will eventually throw
  await asyncFunction() // kicks off a successful async call that is awaited
  const testValue = synchronousAddOneFunction(1)
  expect(testValue).toBe(2) 
}) // test ends

asyncFunctionFails is an asynchronous function that does some work and eventually throws an exception in the testing environment.

asyncFunction is an asynchronous function that is correctly awaited before the test continues. When this function is called with await, the test yields the thread back to process asyncFunctionFails.

When run on its own, this test passes even though asyncFunctionFails will throw an exception. Why? The test process finishes before asyncFunctionFails has the chance to throw the error because nothing is telling the thread to wait for it, so Jest reports a success.

But what if you have other tests in the same file?

it('should add 1 + 1', async () => {
  asyncFunctionFails() // eventually throws
  await asyncFunction()
  const testValue = synchronousAddOneFunction(1)
  expect(testValue).toBe(2)
})

it('should add 2 + 1', async () => {
  await asyncFunction()
  const testValue = synchronousAddOneFunction(2)
  expect(testValue).toBe(3)
})

it('should add 3 + 1', async () => {
  await asyncFunction()
  const testValue = synchronousAddOneFunction(3)
  expect(testValue).toBe(4)
})

When you run this whole test file, one of them fails:

Why does the second test fail when the first test is the one that calls the problematic function?

Now that there are more tests, the Jest process has more time to run than when there was only one test, meaning asyncFunctionFails has a chance to process and throw an exception. So, when the exception is thrown, the Jest process has already moved past the first test and will attribute failure to whichever test happens to be running.

Sweet Race Condition!

This bug is one of the hardest to track down because depending on how many tests you have in the file or how fast the tests take to run, the failures might seem to pop up randomly.

Most of the time, too, the async calls are not as straightforward as this example. Maybe you are mounting a React component that kicks off 5 different hooks to fetch data before rendering in the dom. Or perhaps you are calling a function that fires events to 5 different listeners that each executes code.

The Solution

Make sure to await for the expected outcome or mock out any timers, so all of the code has a chance to run. The exception may still be thrown, but Jest will attribute the error to the correct test. Doing this will make everything much more straightforward to debug.

To address the exception, you may be able to mock out the asynchronous behavior. For example, if the call fails trying to get data from a server, mock the server.

The Yay!

I hope this post helps save you some time debugging a seemingly random test failure. Double-checking your async calls might be the key to stable passing tests :).

DEV Community: Daria Caraway