Unit Testing JavaScript's Asynchronous Activity

Concept

In some code that I was working on for a side-project, I was dealing with asynchronous activity.

I was actually working on a way to mock a Promise response for a particular test.

I went from something bulky and awkward (and, as I later found out, it is somewhat unstable in some scenarios) ...

it('expects ...', async () => {
  const someValue = 'anything';
  spyOn(key, 'asyncFunction').and.callFake(async function() {
    return await someValue;
  });
  // ...
});

.. to a second generation that was much leaner and more efficient. This code is actually more readable, in my opinion ...

it('expects ...', async () => {
  const someValue = 'anything';
  spyOn(key, 'asyncFunction').and.returnValue(Promise.resolve(someValue));
  // ...
});

This all got me thinking about the various asynchronous events I've dealt with over the years and how to test them.

The structure of this article loosely comes from my article JavaScript Enjoys Your Tears. In this article, I detail several activities (some asynchronous in JavaScript, others not) and how they are managed in JavaScript.

Index

This article will cover ...

1. Github Repo that proves all the code being presented in this article.
2. Patterns
3. False Positives and bad chaining
4. setTimeout
5. setInterval
6. Callbacks
7. ES2015 Promises
8. Event Listeners
9. Web Workers
10. ES2017 Async / Await

Github Repo

Here is the working code I put together to verify all the code in this article.

bob-fornal / testing-tears

TESTING-TEARS

This presentation is for testing JavaScript's Asynchronous Activity.

General Notes

• Generate Jasmine test results for all scenarios.

  • Concept Code
  • False Positive Code
  • setTimeout Code
  • setInterval Code
  • Callback Code
  • ES2015 Promise Code
  • Event Listener Code
  • Web Worker Code
  • ES2017 Async / Await Code

• Build a presenter similar to what the original Async Talk does:

  • Presenter with "comments" (markdown?)
  • "Test-Result Display" tab
  • "Code View" tab

See article on details for this presentation: Unit Testing JavaScript's Asynchronous Activity

View on GitHub

This repo will change as I prepare it to become a presentation; however, the core tests will remain.

Patterns

What I would really like to examine here are various means to Unit Test these activities without any additional tooling; staying "testing tool agnostic."

The core patterns that I will reference will take a few basic directions:

1. done(): Utilizing done() to ensure the test knows that there are asynchronous dependent expects.
2. Clock: Utilizing internal test suite tooling to "trick" the clock into moving forward in a way that the asynchronous code fires earlier.
3. Synchronous: Moving the synchronous activity into its own "testable" function.
4. Async / Await: Utilizing this pattern for more readable code.
5. Mocking: Mocking the asynchronous functionality. This is here for larger, existing unit tests and code-bases, and should be a "last resort."

While this article references these patterns in almost all of the categories, there may or may not be code, depending on the scenario. Additionally, the patterns may not always be presented in the order listed above.

False Positives

One of the main problems with asynchronous testing is that when it is not set up correctly the spec ends before the assertions get to run.

And, in most test suites, the test silently passes. By default, a test is flagged as passed when there is no expect in it.

The following code is one example of a false positive that can come from not taking into account asynchronicity in JavaScript ...

it("expects to fail", () => {
  setTimeout(() => {
    expect(false).toEqual(true);
  });
});

The test finishes before the setTimeout completes; hence, a false positive.

Solving False Positives

One means of dealing with this issue is simple and relatively straightforward. A parameter needs to be passed into the it specification; usually called done.

Passing in this parameter flags the spec within the test suite as asynchronous, and the test engine will wait for the function identified by the parameter to be called before flagging the test as passed or failed.

it('expects "done" to get executed', (done) => {
  setTimeout(() => {
    expect(true).toEqual(false);
    done();
  }, 0);
});

This test will now fail, as expected.

NOTE: In Jest, there is an additional means of protecting the code: expect.hasAssertions() which verifies that at least one assertion is called during a test. This is often useful when testing asynchronous code, to make sure that assertions in a callback actually got called. SEE DOCUMENTATION HERE

While this solution is quite simple, the issue itself is just the tip of a rather large iceberg. This issue, as simple as it is, can lead to severe issues in a test suite, because when the done parameter is not properly used the suite can become challenging to debug, at best.

Without examining at a ton of code, imagine dozens of tests ... all of them properly implementing done. However, one test added by another developer is not properly calling done. With all the tests happily passing ... you may not even know there is a broken test until some level of testing (integration, automated, or users in production) sees that there is actually an error that was not caught.

Bad Promise Chaining

The issue presented above is not the only possible problem. There is always the possibility of mistakes caused when assembling the promise chains in the tests.

const toTest = {
  get: () => {
    return Promise.delay(800).then(() => 'answer');
  },
  checkPassword: (password) => {
    if (password === 'answer') {
      return Promise.resolve('correct');
    }
    return Promise.resolve('incorrect');
  }
};

it('expects to get value and then check it', (done) => {
  toTest.get()
  .then(value => {
    toTest.checkPassword(value)
    .then(response => {
      // The issue is here. The .then immediately above is not
      // in the main promise chain
      expect(response).toEqual('wrong answer');
    });
  })
  .then(() => done())
  .catch(done);
});

The .then immediately after the toTest.checkPassword() is detached from the main promise chain. The consequence here is that the done callback will run before the assertion and the test will pass, even if it gets broken (we are checking for 'wrong answer' above and should be failing).

To fail properly, use something like this ...

it('expects "toTest" to get value and then check it', () => {
  toTest.get()
  .then(value => {
    return toTest.checkPassword(value);
  })
  .then(response => {
    expect(response).toEqual('wrong answer');
    done();
  })
  .catch(done);
});

setTimeout and setInterval

I have an article that addresses some of the testing in category: Testing setTimeout / setInterval.

Looking at the functionality embodied in setTimeout and setInterval, there are several ways to approach testing this code.

There is a reasonable patch documented in the article above. I do not recommend this type of option unless there is a significant about of test code already in place.

setTimeout

Looking into utilizing the done() parameter previously presented, here is some code that needs to be tested ...

var testVariable = false;
function testableCode() {
  setTimeout(function() {
    testVariable = true;
  }, 10);
}

While this is remarkably simple code, it focuses in on the asynchronous activity to be tested.

Using the done() pattern ...

it('expects testVariable to become true', function(done) {
  testableCode();

  setTimeout(function() {
    expect(testVariable).toEqual(true);
    done();
  }, 20);
});

This is a pattern that will work. Given a certain amount of time, the variable can be tested for the expected result. However, there is a huge issue with this type of test. It needs to know about the code being tested; not knowing how long the setTimeout delay actually was, the test would work intermittently.

The "internal synchronous" activity can be moved into its own testable function ...

var testVariable = false;
function changeTestVariable() {
  testVariable = true;
}
function testableCode() {
  setTimeout(changeTestVariable, 10);
}

This way, the setTimeout does not have to be tested. The test becomes very straightforward.

it('expects testVariable to become true', () => {
  changeTestVariable();
  expect(testVariable).toEqual(true);
});

Another approach is to use internal test tools, in this case, the jasmine.clock(). The code to test then becomes something like this ...

it('expects testVariable to become true', function() {
  jasmine.clock().install();

  testableCode();
  jasmine.clock().tick(10);

  expect(testVariable).toEqual(true);
  jasmine.clock().uninstall();
});

The use of the async / await pattern means we need a slight rewrite of the testableCode to become "await-able."

var testVariable = false;

const sleep = (time) => {
 return new Promise(resolve => setTimeout(resolve, time));
};

async function testableCode() {
  await sleep(10);
  testVariable = true;
}

Then, the code can be tested quite simply like this ...

it('expects "testable" code to set testVariable to TRUE', async () => {
  await testableCode();
  expect(testVariable).toEqual(true);
});

setInterval

Starting with a simple example similar to the setTimeout code used above ...

var testVariable = false;
function testableCode2(){
  var counter = 1;
  var interval = setInterval(function (){
    if (counter === 5) {
      testVariable = true;
      clearInterval(interval);
    }
    counter++;
  }, 500);
  return interval;
}

The patterns explored in setTimeout will carry over.

Using done() as a means to tell the test that the expect will be checked asynchronously ...

it('expects testVariable to become true', function(done) {
  testableCode2();

  setTimeout(function() {
    expect(testVariable).toEqual(true);
    done();
  }, 1000);
});

However, the timing issue is the same. The test code will have to know something about the code to be tested.

Additionally, the timer behavior can be mocked ... allowing jasmine to step the time forward.

it('expects testVariable to become true', function() {
  jasmine.clock().install();

  testableCode2();
  jasmine.clock().tick(4000);

  expect(testVariable).toEqual(true);
  jasmine.clock().uninstall();
});

Refactoring the synchronous code out of the setInterval is also a viable option ...

var testVariable = false;
var counter = 1;
var interval;
function testableAfterInterval() {
  if (counter === 5){
    testVariable = true;
    clearInterval(interval);
  }
  counter++;
}
function testableCode2() {
  counter = 1
  interval = setInterval(testableAfterInterval, 500);
  return interval;
}

With this simple refactor, the tests are much more focused ...

it('expects testVariable to become true', function() {
  counter = 5;
  testableAfterInterval();
  expect(testVariable).toEqual(true);
});

Now, additional refactoring will allow utilization of the async / await pattern.

var testVariable = false;
function waitUntil() {
  return new Promise(resolve => {
    var counter = 1;
    const interval = setInterval(() => {
      if (counter === 5) {
        testVariable = true;
        clearInterval(interval);
        resolve();
      };
      counter++;
    }, 1000);
  });
}

async function testableCode2() {
  await waitUntil();
}

... with the code being tested like this ...

it('expects testVariable to become true', async () => {
  await testableCode2();
  expect(testVariable).toEqual(true);
});

This is not the cleanest of code examples. The waitUntil function is long and prone to some issues. Given this type of scenario, the code should be reworked to use the setTimeout sleep() code discussed previously for a cleaner Promise chain pattern.

Callbacks

Callbacks are one of those areas that are at the same time, simpler, and more complex to test.

Starting with some code before digging into the details ...

const numbers = [1, 2, 3];
let answers = [];

const forEachAsync = (items, callback) => {
  for (const item of items) {
    setTimeout(() => {
      callback(item);
    }, 0, item);
  }
};

const runAsync = () => {
  forEachAsync(numbers, (number) => {
    answers.push(number * 2);
  });
};

Testing the callback by itself, there is no need to worry about the code's asynchronous nature. Simply pull out the function used as a callback and test the callback function itself.

const runAsyncCallback = (number) => {
  answers.push(number * 2);
};

runAsync = () => {
  forEachAsync(numbers, runAsyncCallback);
};

Given the above modification, the runAsyncCallback can now be tested independently of the forEachAsync functionality.

it('expects "runAsyncCallback" to add to answers', () => {
  runAsyncCallback(1);
  expect(answers).toEqual([2]);
});

However, if the forEachAsync functionality needs to be tested, other approaches will be necessary.

Next, looking at using the done() pattern; there is nothing clear to hook onto ...

it('expects "runAsync" to add to answers', (done) => {
  runAsync();
  setTimeout(() => {
    expect(answers).toEqual([2, 4, 6]);
    done();
  }, 100);
});

Using the clock pattern, the testing code should look something like this ...

it('expects "runAsync" to add to answers', function() {
  jasmine.clock().install();

  runAsync();
  jasmine.clock().tick(100);

  expect(answers).toEqual([2, 4, 6]);
  jasmine.clock().uninstall();
});

As a final scenario, the code has to be reworked to allow for use of the async / await pattern. Modifying the original set of code becomes ...

const numbers = [1, 2, 3];
let answers = [];

const sleep = (time) => {
 return new Promise(resolve => setTimeout(resolve, time));
};

const forEachAsync = async (items, callback) => {
  for (const item of items) {
    await sleep(0);
    callback(item);
  }
};

const runAsync = async() => {
  await forEachAsync(numbers, (number) => {
    answers.push(number * 2);
  });
};

With these adjustments, the test code then becomes ...

it('expects "runAsync" to add to answers', async () => {
  await runAsync();
  expect(answers).toEqual([2, 4, 6]);
});

ES2015 Promises

Beginning with a simple promise ...

let result = false;
function promise () {
  new Promise((resolve, reject) => {
    result = true;
    resolve(result);
  })
  .catch(err => console.log(err));    
}

The clear path to look at when testing this code is to use the done() pattern ...

it('expects variable to become true', (done) => {
  promise();

  setTimeout(() => {
    expect(result).toEqual(true);
    done();
  }, 50);
});

This is still an awkward way to test this code; the timeout adds an unnecessary delay to the test code.

Another pattern that is equally as awkward is using the clock pattern ...

  it('expects variable to become true', () => {
    jasmine.clock().install();

    promise();
    jasmine.clock().tick(50);

    expect(result).toEqual(true);
    jasmine.clock().uninstall();
  });

The synchronous pattern used is also awkward here because we would be pulling out a single line of code to reinject it in before the code resolves.

The final way to approach testing this code would be with async / await and should look like this ...

it('expects variable to become true', async () => {
  await promise();
  expect(result).toEqual(true);
});

This is a very clean pattern and easy to understand.

Event Listeners

Event Listeners are not asynchronous, but the activity against them is outside of JavaScript's synchronous code, so this article will touch on testing them here.

Given some really basic code ...

function dragStart(event) {
  event.dataTransfer.setData('text/plain', event.target.id);
}

function dragOver(event) {
  event.preventDefault();
  event.dataTransfer.dropEffect = 'move';
}

function drop(event) {
  const id = event.dataTransfer.getData('text');
  const element = document.getElementById(id);
  event.target.appendChild(element);
}

The first thing to notice when looking at this code is that an event is passed to each function. The test code can pass an object that can mock a real event, allowing for simplified testing to occur.

describe('drag-and-drop events', () => {
  it('expects "dragStart" to set data', () => {
    let resultType = '';
    let resultData = '';
    const mockId = 'ID';
    let mockEvent = {
      dataTransfer: {
        setData: (type, data) => {
          resultType = type;
          resultData = data;
        }
      },
      target: {
        id: mockId
      }
    };

    dragStart(mockEvent);
    expect(resultType).toEqual('text/plain');
    expect(resultData).toEqual(mockId);
  });

  it('expects "dragOver" to set drop effect', () => {
    let mockEvent = {
      preventDefault: () => {},
      dataTransfer: {
        dropEffect: null
      }
    };
    spyOn(mockEvent, 'preventDefault').and.stub();

    dragOver(mockEvent);
    expect(mockEvent.preventDefault).toHaveBeenCalled();
    expect(mockEvent.dataTransfer.dropEffect).toEqual('move');
  });

  it('expects "drop" to append element to target', () => {
    const data = 'DATA';
    const element = 'ELEMENT';
    let mockEvent = {
      dataTransfer: {
        getData: () => data
      },
      target: {
        appendChild: () => {}
      }
    };
    spyOn(mockEvent.dataTransfer, 'getData').and.callThrough();
    spyOn(document, 'getElementById').and.returnValue(element);
    spyOn(mockEvent.target, 'appendChild').and.stub();

    drop(mockEvent);
    expect(mockEvent.dataTransfer.getData).toHaveBeenCalledWith('text');
    expect(document.getElementById).toHaveBeenCalledWith(data);
    expect(mockEvent.target.appendChild).toHaveBeenCalledWith(element);
  });
});

Web Workers

This seemed like an area that could be problematic. Web workers run in a separate thread. However, while researching for this part of the article, I came across Testing JavaScript Web Workers with Jasmine.

The author clearly describes several clean methods to load and enable the web worker for testing. I'll leave out several of these methods since they are so well documented in the article above.

For the code in this article to be tested, this means that whether a runner is used to test in the browser or the tests are run in a headless browser, the "web worker" code can simply be loaded with the test code.

<script src="/js/web-worker.js"></script>
<script src="/spec/web-worker.spec.js"></script>

Given the web worker code ...

onmessage = function() {
  for (let step = 0, len = 10; step <= len; step++) {
    postMessage(step * 10);
    const start = Date.now();
    while (Date.now() < start + 1000) {};
  }  
}

The function postMessage (which is actually window.postMessage) can be mocked in a way to capture the responses from the code to be tested.

Testing this in the first round utilizing done(), the code would look like this ...

it('expects messages for 0 to 10', (done) => {
  spyOn(window, 'postMessage').and.stub();

  onmessage();
  setTimeout(() => {
    expect(window.postMessage).toHaveBeenCalledTimes(11);
    expect(window.postMessage).toHaveBeenCalledWith(0);
    expect(window.postMessage).toHaveBeenCalledWith(10);
    expect(window.postMessage).toHaveBeenCalledWith(20);
    expect(window.postMessage).toHaveBeenCalledWith(30);
    expect(window.postMessage).toHaveBeenCalledWith(40);
    expect(window.postMessage).toHaveBeenCalledWith(50);
    expect(window.postMessage).toHaveBeenCalledWith(60);
    expect(window.postMessage).toHaveBeenCalledWith(70);
    expect(window.postMessage).toHaveBeenCalledWith(80);
    expect(window.postMessage).toHaveBeenCalledWith(90);
    expect(window.postMessage).toHaveBeenCalledWith(100);
    done();
  }, 100);
});

Additionally, the test can be run using the clock method ...

it('eexpects messages for 0 to 10', function() {
  jasmine.clock().install();
  spyOn(window, 'postMessage').and.stub();

  onmessage();
  jasmine.clock().tick(100);

  expect(window.postMessage).toHaveBeenCalledTimes(11);
  expect(window.postMessage).toHaveBeenCalledWith(0);
  expect(window.postMessage).toHaveBeenCalledWith(10);
  expect(window.postMessage).toHaveBeenCalledWith(20);
  expect(window.postMessage).toHaveBeenCalledWith(30);
  expect(window.postMessage).toHaveBeenCalledWith(40);
  expect(window.postMessage).toHaveBeenCalledWith(50);
  expect(window.postMessage).toHaveBeenCalledWith(60);
  expect(window.postMessage).toHaveBeenCalledWith(70);
  expect(window.postMessage).toHaveBeenCalledWith(80);
  expect(window.postMessage).toHaveBeenCalledWith(90);
  expect(window.postMessage).toHaveBeenCalledWith(100);
  jasmine.clock().uninstall();
});

Since the core code is not, in itself, asynchronous ... this code will not be testable via async / await without a major rework.

ES2017 Async / Await

Testing the async / await functionality is pretty straight forward and does not have the need to go through the previously defined patterns. We can simply use the same functionality when testing; async / await.

Starting with this code ...

let variable = false;

const sleep = (time) => {
  return new Promise(resolve => {
    setTimeout(resolve, time);
  });
};

const testable = async () => {
  await sleep(10);
  variable = true;
};

Testing this code synchronously would have to account for the sleep time as well as pulling out the functional part of this code. Given, that the core code would need modified and that the testing code could not easily handle a changing time, this code becomes too hard to test this way.

Moving forward, this code tested with done() or with the timer have to account for a possibly changing time within the source code, as well.

The final pattern, utilizing async / await was literally made for this task. The test code would look something like this ...

it('expects varible to become true', async () => {
  await testable();
  expect(variable).toEqual(true);
});

While the other patterns could be used here, the simplicity shown in this test makes it the clear choice.

Conclusion

This article covered ...

1. Github Repo that proves all the code being presented in this article.
2. Patterns
3. False Positives and bad chaining
4. setTimeout
5. setInterval
6. Callbacks
7. ES2015 Promises
8. Event Listeners
9. Web Workers
10. ES2017 Async / Await

The core patterns referenced took a few basic directions:

1. done(): Utilizing done() to ensure the test knows that there are asynchronous dependent expects. This pattern, as we have seen would have to have some understanding of the underlying code.
2. Clock: Utilizing internal test suite tooling to "trick" the clock into moving forward in a way that the asynchronous code fires earlier. This pattern, as we have seen would also have to have some understanding of the underlying code.
3. Synchronous: Moving the synchronous activity into its own "testable" function. This can be a viable solution, but can be avoided if one of the other patterns provides a clear testable solution.
4. Async / Await: Utilizing this pattern for more readable code.
5. Mocking: Mocking the asynchronous functionality. This is here for larger, existing unit tests and code-bases, and should be a "last resort."

I am sure there are other scenarios that would provide additional clarity, as well as other testing patterns that could be used. However, these tests clearly cover the code in my previous article: JavaScript Enjoys Your Tears.