DEV Community: webpapaya

TDD for TypeScript type definitions

webpapaya — Fri, 20 Mar 2020 13:37:36 +0000

I recently wanted to add TS type definitions to hamjest, my assertion library of choice. If you don't know hamjest yet you can check out my post on writing better test assertions. Without any experience in writing type definitions I wanted a way to verify my type definitions. After doing some research I realised that there is a tool called tsd which can be used to verify type definitions.

TDD in a nutshell

TDD is an iterative development process where every change in production code is triggered by a failing test. The process is divided into three parts, which are repeated after every iteration:

Write a failing test
- watch the test fail
Write just as much code to make the test pass
- watch the tests pass
Refactor the code

I won't get into much details in here as there are some great resources on this topic out there:

7 minutes, 26 seconds by @jbrains
An Introduction to Test-Driven Development on freeCodeCamp
just use google for your language of choice

Getting started

First of all we need a very basic setup for TypeScript:

npm init
npm i tsd chokidar-cli -D
touch index.d.ts # types or implementation
touch index.test-d.ts # tests
echo "{\"lib\": [\"es2015\"]}" > tsconfig.json # add minimal tsconfig.json
./node_modules/.bin/chokidar "./**/*" -c "./node_modules/.bin/tsd && echo done" # start testing

To make this easier to understand we'll test the types of a math library which can do various arithmetic on numbers. After starting tsd and reading the docs of the library we see that there is a sum function which can take 2 arguments and returns the sum of those. So we could start with the first test in index.test-d.ts

// index.test-d.ts
import {expectType} from 'tsd'
import * as myModule from '.'

expectType<number>(myModule.sum(1, 2))

tsd contains a couple of assertions which can be used to verify ts types. In the given example we're using expectType to verify the return signature of the function. After saving we get back the following 2 errors:

// ✖  4:0   Parameter type number is not identical to argument type any.
// ✖  4:28  Property sum does not exist on type typeof import("/Users/tmayrhofer/Projects/hamjest/index").

The implementation could look like this:

// index.test-d.ts
import {expectType} from 'tsd'
import * as myModule from '.'

expectType<number>(myModule.sum(1, 2))

// index.d.ts
export function sum(a: number, b: number): number

From the documentation of the library we know that besides summing 2 numbers it’s possible to sum 3 numbers as well:

// index.test-d.ts
import {expectType} from 'tsd'
import * as myModule from '.'

expectType<number>(myModule.sum(1, 2))
expectType<number>(myModule.sum(1, 2, 3)) // <= new test case
// ERROR: ✖  7:38  Expected 2 arguments, but got 3.

The above error could be fixed like that:

// index.test-d.ts
import {expectType} from 'tsd'
import * as myModule from '.'

expectType<number>(myModule.sum(1, 2))
expectType<number>(myModule.sum(1, 2, 3))

// index.d.ts
export function sum (a: number, b: number, c?: number): number

Looking at the documentation again we see that an arbitrary amount of numbers could be summed by this function. So we'll add another test:

// index.test-d.ts
import {expectType} from 'tsd'
import * as myModule from '.'

expectType<number>(myModule.sum(1, 2))
expectType<number>(myModule.sum(1, 2, 3))
expectType<number>(myModule.sum(1, 2, 3, 4, /*...*/ 9)) // <= new test case
// ERROR: Expected 2 arguments, but got 9.

// index.d.ts
export function sum (a: number, b: number, c?: number): number

The type definition could be adapted like that:

// index.test-d.ts
import {expectType} from 'tsd'
import * as myModule from '.'

expectType<number>(myModule.sum(1, 2))
expectType<number>(myModule.sum(1, 2, 3))
expectType<number>(myModule.sum(1, 2, 3, 4, /*...*/ 9))

// index.d.ts
export function sum(...values: number[]): number

The documentation states that the sum function requires at least one argument or otherwise a runtime exception will be thrown. As we don't want the application to run into runtime exceptions we'll add another test case which should prevent a call to the sum function without any arguments:

// index.test-d.ts
import {expectType, expectError} from 'tsd'
import * as myModule from '.'

expectType<number>(myModule.sum(1, 2))
expectType<number>(myModule.sum(1, 2, 3))
expectType<number>(myModule.sum(1, 2, 3, 4, /*...*/ 9))
expectError(myModule.sum()) // <= new test case
// ERROR: ✖  16:0  Expected an error, but found none.

// index.d.ts
export function sum(...values: number[]): number

Our type could update to the following:

// index.test-d.ts
import {expectType, expectError} from 'tsd'
import * as myModule from '.'

expectType<number>(myModule.sum(1, 2))
expectType<number>(myModule.sum(1, 2, 3))
expectType<number>(myModule.sum(1, 2, 3, 4, /*...*/ 9))
expectError(myModule.sum())

// index.d.ts
type ArrayWithAtLeastOneElement<T> = {
  0: T
} & Array<T>

export function sum (...values: ArrayWithAtLeastOneElement<number>): number

The ArrayWithAtLeastOneElement type specifies a non empty array. We can add this type to our function and our tests are turning green again.

Another function this library exposes is called div. This function simply divides two numbers. Adding the first test could look like this:

// index.test-d.ts
import {expectType, expectError} from 'tsd'
import * as myModule from '.'

// ... tests for sum

expectType<number>(myModule.div(1, 2))
// ✖  18:0   Parameter type number is not identical to argument type any.
// ✖  18:28  Property div does not exist on type typeof import("/Users/tmayrhofer/Projects/hamjest/index").

The implementation should be similar to the first example of the add function. So we end up with the following:

// index.test-d.ts
import {expectType, expectError} from 'tsd'
import * as myModule from '.'

// ... tests for sum
expectType<number>(myModule.div(1, 2))

// index.d.ts
export function div(a: number, b: number): number

As dividing by zero is impossible we would like to catch this case by the type system itself. So the next test could look like the following:

// index.test-d.ts
import {expectType, expectError} from 'tsd'
import * as myModule from '.'

// ... tests for sum
expectType<number>(myModule.div(1, 2))
expectError(myModule.div(1, 0)) // <= new test case
// ERROR: ✖  19:0  Expected an error, but found none.

Fixing this test could be done by the following code:

// index.test-d.ts
import {expectType, expectError} from 'tsd'
import * as myModule from '.'

// ... tests for sum
expectType<number>(myModule.div(1, 2))
expectError(myModule.div(1, 0))


// index.d.ts
export function div<Divider extends number>(
  a: number,
  b: Divider extends 0 ? never : Divider
): number

So finally we end up with the following the following types and tests:

// index.d.ts
type ArrayWithAtLeastOneElement<T> = {
  0: T
} & Array<T>

export function sum (...values: ArrayWithAtLeastOneElement<number>): number
export function div<Divider extends number>(
  a: number,
  b: Divider extends 0 ? never : Divider
): number

// index.d.ts
expectType<number>(myModule.sum(1, 2))
expectType<number>(myModule.sum(1, 2, 3))
expectType<number>(myModule.sum(1, 2, 3, 4, /*...*/ 9))
expectError(myModule.sum())

expectType<number>(myModule.div(1, 2))
expectError(myModule.div(1, 0))

Conclusion

Using TDD for TS types makes it possible to refactor types without the fear of breaking them. Having a safe way to refactor types prevents accidental complexity. Working in baby-steps makes it possible to break down more complex types into easy to process chunks. Writing tests for types in the presented way is not 100% bulletproof as we can't be sure that our typings match the actual implementation. As the tests are simple TS/JS files those could still be executed so that the actual implementation could be verified as well. Especially for more complex libraries (eg. momentJS/jQuery) the TDD approach makes it easier to develop types. Despite enjoying this process to develop types like presented in this post, it should be applied only when there is no other way to alter the project. Having a comprehensive test suite written in TS would make it possible, to verify typings and the behaviour of the library with the same code.

If you want to learn more about TypeScript, TDD or maintainable software I'm always happy to have a chat via Twitter.

Mostly typeless TypeScript

webpapaya — Wed, 18 Mar 2020 12:36:38 +0000

Coming from a JavaScript and Ruby background I never thought about static typing in my code. Besides some algorithm and data structure lectures at university I had almost no touchpoints with statically typed languages. When coding in Ruby or JS I normally rely on TDD for business logic and some integration or "happy path" tests to verify the interaction between the components. Testing the correct wiring between function/method call on the integration level is almost impossible as the amount of tests grow exponentially with every additional code path (see Integrated tests are a scam) by @jbrains. In 2017 there had been quite some fuzz about Making Impossible States Impossible in elm and after seeing Scott Wlaschin talk about Domain Modeling Made Functional I began to realise that the missing piece to my integration test problem could be type checking.

In spring 2019 I was working at a company which decided to switch to TypeScript (abbr. TS) as their main language of choice for an upcoming project. Working on this project fixed the issue of testing interactions between components, but took a big toll on source-code readability. By accident I came across a library called validation.ts which can derive a TS type from validation functions. Seeing a type being derived completely from a JS data structure made me think that it might be possible to get all benefits of a strong type system without writing any types.

This post will cover the topic of type inference and how application code could be written without type annotations. The first half of this blog post will explain some of the main building blocks of TSs type system and in the second half will try to use those in a way to remove type annotations from the application code.

Type inference

In recent years functional programming (abbr. FP) gained traction and many object oriented languages adapted ideas from this paradigm. One of FPs features is type inference, which describes an automatic deduction of a type from an expression. Depending on the language the extent of type inference can vary from a very basic one where variables don't need to specify the type, to a very sophisticated one where type annotations are mostly used as a form of documentation. Given the following example:

let mutableValue = 10 // => type number
const constantValue = 10 // => type 10

The variable with the name mutableValue is inferred to a number. This means that it can't be assigned to anything else than the number type. The variable constantValue is automatically inferred to the constant 10, as the const keyword prevents a variable from being reassigned.

Generics

A generic is a way to reuse a component with a variety of types rather than a single type. A generic could be seen as a type level function which can be customised with other types. In TS generics are always written between pointy brackets (eg. Array<T>). T is the generic which can be replaced by any valid type (eg. Array<number>, Array<string>). This post won't go into details about generics as the TypeScript Handbook provides an in-depth overview about this topic. TypeScripts type system can also infer some generics like arrays:

const numberArray = [0,1,2,3] // => Array<number>
const stringArray = ['A','B','C','D'] // => Array<string>
const booleanArray = [true,false] // => Array<boolean>

There are situations where array items belong to more than one type (eg.: [1, 'A', true]). TS tries to find the best possible data type and in this case it automatically infers the value to the type Array<number | string | boolean>. The | character is called union or choice, which means that the values in the array could either be a string, a number or a boolean.

const mixedArray = [1, 'A', true] // Array<number | string | boolean>

Const assertion

JavaScript the language itself does not have immutable data types. An immutable datatype is an object whose state cannot be changed after it was created Source. In JS a variable defined as const can still be altered after its creation. That's because the reference to the value is defined as const but the object itself could still change. For arrays this means that items inside the array can be mutated, as well as additions and removals of individual elements.

const numberArray = [0,1,2,3] // => type Array<number>
numberArray[0] = 10;

In the example above the type is inferred to Array<number>. There are cases where this type is considered too wide, as the array won't be mutated and it always contains a well known list of elements (eg. 0,1,2,3). Starting with version 3.4. TS introduced a const assertion which solves the problem of type widening. This converts an object to be readonly and helps TS to better infer its type:

const mutableArray = [0,1,2,'three'] // => type Array<number | string>
mutableArray[2] // => type number

const constArray = [0,1,2,'three'] as const // => type readonly [0,1,2,"three"]
constArray[2] // => literal 2
constArray[3] // => literal "three"

constArray[4] = 4
// ^^^^^^^^^^
// ERROR: Index signature in type 'readonly [0, 1, 2, "three"]' only permits reading.

Adding const assertions makes it possible to get better type information and narrow the type from Array<number | string> to readonly [0,1,2,"three"].

Applying the theory

In order to make the content easier to approach, imagine building a simplified E-Commerce application. The application has 3 different products which can be listed and added to a shopping cart. The functionality of the shopping cart is critical to the business so we need to make sure that nothing besides the known products can be added to the shopping cart. After seeing the requirements we'll start modeling the domain with TS types:

type Product = 'Chair' | 'Table' | 'Lamp'
type ShoppingCart = Array<Product> // Array<'Chair' | 'Table' | 'Lamp'>

Displaying products

The Product is defined as a union of either Chair, Table or Lamp. The ShoppingCart is defined as an array of the Product type, which makes it possible to buy a product multiple times. Having a model definition we can proceed with the implementation of the products list:

type Product = 'Chair' | 'Table' | 'Lamp'

const products: Array<Product> = ['Chair', 'Table', 'Lamp']
const displayProducts = () => { /* irrelevant */}

Looking at this code already reveals one major flaw. Adding a fourth product to the application would require a change in two places. The Product type would need to adapt and in order to display the additional product on the product overview page it is required to alter the products as well. Keeping two constants in sync is an almost impossible challenge and as a result the products array will get out of sync eventually. As we've seen earlier in this post TS can derive types from expressions, so it might be possible to derive the Product type directly from the products array.

const products = ['Chair', 'Table', 'Lamp']
type Product = typeof products[number] // string
//                                        ^^^^^^^
// This type does not match our requirements, as the
// => type string is to loose.

typeof products[number] returns a list of all possible types in the array. Deriving the type of the shopping cart from the products array doesn't yield the expected results, as every possible string becomes a possible product. Earlier in this article TS const assertion was mentioned which would prevent this type widening.

const products = ['Chair', 'Table', 'Lamp'] as const // => type readonly ['Chair', 'Table', 'Lamp']
type Product = typeof products[number] // => 'Chair' | 'Table' | 'Lamp'

This yields the expected result and the implementation can't get out of sync with the types as both are fed from the same data.

Adding to the shopping cart

With the domain we modeled it is not possible to add invalid products to the application. A simplified version of the shopping cart could look like the following:

const products = ['Chair', 'Table', 'Lamp'] as const
type Product = typeof products[number]
type ShoppingCart = Array<Product>

const shoppingCart: ShoppingCart = []
shoppingCart.push('Chair')
shoppingCart.push('Table')
shoppingCart.push('Unknown product')
//                ^^^^^^^^^^^^^^^^^
// ERROR: Argument of type '"Unknown product"' is not assignable to
// parameter of type '"Chair" | "Table" | "Lamp"'.

All the business requirements are met as invalid products can't be added to the shopping cart. The code itself is fully typed but it comes at the cost of readability. Removing the types and converting the app back to JS makes the code easier to read but this also removes all the benefits we gained via TS.

const products = ['Chair', 'Table', 'Lamp']

const shoppingCart = []
shoppingCart.push('Chair')
shoppingCart.push('Table')
shoppingCart.push('Unknown product')
//                ^^^^^^^^^^^^^^^^^
// In JS this error can't be tracked...

What if it would be possible to still maintain type safety and remove almost all TS specific code?

const products = ['Chair', 'Table', 'Lamp'] as const
const shoppingCart = emptyArrayOf(products)

shoppingCart.push('Chair')
shoppingCart.push('Unknown product')
//                ^^^^^^^^^^^^^^^^^
// Argument of type '"Unknown product"' is not assignable to
// parameter of type '"Chair" | "Table" | "Lamp"'.

Besides the const assertion in line 1 it would be impossible to tell wether this is a TS or JS application. So in the following sections we'll have a look on how to convert the fully typed TS example to an almost TS free version. In order to get to the TS free version I tend to differ between business logic related code (without TS) and utility code which contains TS. In the example above the emptyArrayOf function would be considered as a utility function.

Before starting with the implementation of the emptyArrayOf function we need to take a step back and look at generics again. Similar to regular JS functions, TS generics make it possible to reuse certain type logic with different type arguments. Looking at the following function:

const emptyArrayOf = <TypeOfArrayItem>(): Array<TypeOfArrayItem> => {
  return []
}

const stringArray = emptyArrayOf<string>() // Array<string>
const numberArray = emptyArrayOf<number>() // Array<number>
const shoppingCart = emptyArrayOf<'Chair' | 'Table' | 'Lamp'>() // Array<'Chair' | 'Table' | 'Lamp'>

The emptyArrayOf function has a type signature of () -> Array<T>. This means that the function returns an array whose items are of type T.

const emptyArrayOf = <TypeOfArrayItem>(): Array<TypeOfArrayItem> => {
  //                 ^^^^^^^^^^^^^^^^^
  // Define a generic type argument called `TypeOfArrayItem`.
  // The generic type could be seen "type parameter/variable"
  // for later use. Any valid TS type could be used.
  return []
}
// ...

const emptyArrayOf = <TypeOfArrayItem>(): Array<TypeOfArrayItem> => {
  //                                      ^^^^^^^^^^^^^^^^^^^^^^
  // Use the generic type variable `TypeOfArrayItem` to tell TS
  // what the function is returning.
  return []
}

const shoppingCart = emptyArrayOf<'Chair' | 'Table' | 'Lamp'>() // Array<'Chair' | 'Table' | 'Lamp'>
//                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
// Pass a "type parameter" to the emptyArrayOf function. Due to signature
// of `emptyArrayOf` it returns an empty array of the type Array<Product>.
// This means that no unknown product can be pushed to this array.

Sadly this is not exactly what we wanted to archive, as there is still some TS specific code present. What if the original products array
would be passed in as an argument to the function and TS automatically derives the return type from this.

const emptyArrayOf = <TypeOfArrayItem>(possibleValues: TypeOfArrayItem[]): TypeOfArrayItem[] => {
  //                                                   ^^^^^^^^^^^^^^^^^
  // Introduce a new argument in order to derive the resulting type from it.
  return []
}

const products = ['Chair', 'Table', 'Lamp'] // Array<string>
const shoppingCart = emptyArrayOf(products) // Array<string>
//                               ^^^^^^^^^^    ^^^^^^^^^^^^^
// Pass the products of the shop to the emptyArrayOf function.
// The return type is automatically derived.

As seen above the emptyArrayOf function automatically derives it's type from the given array. Instead of returning an array of valid products the function now returns an Array of strings. Adding a const assertion to the products array should fix this issue.

const emptyArrayOf = <TypeOfArrayItem>(possibleValues: Readonly<TypeOfArrayItem[]>): TypeOfArrayItem[] => {
  //                                                   ^^^^^^^^
  // As const assertions convert an object to be readonly we need to adapt the incoming
  // type to be readonly.
  return []
}

const products = ['Chair', 'Table', 'Lamp'] as const
//                                          ^^^^^^^^^
// Adding the const assertion to the products makes it readonly and allows
// typescript to better infer its type.

const shoppingCart = emptyArrayOf(products) // Array<'Chair' | 'Table' | 'Lamp'>
//    ^^^^^^^^^^^^
// Finally the shopping cart contains the correct type and no invalid product
// can be added.

Removing all the clutter leaves us with the following implementation:

// Utility Code
const emptyArrayOf = <TypeOfArrayItem>(possibleValues: Readonly<TypeOfArrayItem[]>): TypeOfArrayItem[] => {
  return []
}

// Application Code
const products = ['Chair', 'Table', 'Lamp'] as const
const shoppingCart = emptyArrayOf(products)

shoppingCart.push('Chair')
shoppingCart.push('Unknown product')
//                ^^^^^^^^^^^^^^^^^
// ERROR: Argument of type '"Unknown product"' is not assignable to
// parameter of type '"Chair" | "Table" | "Lamp"'.

Conclusion

This post showed the power of type inference in TS and how to derive types from JS data structures. Deriving types from JS is a powerful tool as it adds the benefits of type safety without sacrificing maintainability of the application code. A better readability of the code is a nice side-effect of minimising TS specific application code.

This is the first post of a series of TS related posts I'm planning to write in the upcoming weeks. If you have questions don't hesitate to hit me a message on Twitter or use the comments.

Writing better test assertions

webpapaya — Mon, 11 Mar 2019 20:11:17 +0000

Doing TDD is an integral part of my day to day workflow. Tests help me to break down complex problems into smaller chunks which I can process more easy. This helps me to develop parts of the application in isolation and focus on the core business logic without the fear of breaking existing functionality. Getting fast feedback to my changes empowers me to move fast and build more robust systems. Having lots of small tests which check one behavior of a unit under test makes it easy to see what the application is capable of. Quite often those small tests cause a maintenance overhead as additional requirements make those tests break, even though functionality was only added and existing behavior was kept untouched. This leads to the issue that tests need to be altered even though their behavior didn't change. Another issue often arises when writing tests against external systems like databases.

If an order by clause is not specified, then the ordering of the rows of Q is implementation-dependent. (https://dba.stackexchange.com/a/6053)

Getting records back in a different order each test run is a common issue and might result in green suite locally but failing tests on CI. After some research I realized that the way my test assertions were written, might be the root cause of my brittle tests. In this post I'll share some of my finding on my journey to write better test assertions.

TLDR

Watch the talk from the Vienna JS Meetup in double speed.

What is a test assertion

An assertion is a boolean expression at a specific point in a program which will be true unless there is a bug in the program Source. A very basic implementation of an assertion might look similar to the following:

const assert = (value, message = 'assertion failed') => {
  if (!value) { throw new Error(message); }
}

assert(1 === 1, '1 should be equal to 1');
assert(1 === 2, '1 should be equal to 1'); // Throws exception

Whenever a falsy value is passed to the assert function an exception is thrown with an optional message. When an unhandled exception is thrown inside a test-case, it is automatically marked as failed. The above test assertion is very low level and not very expressive. Assertion libraries solve this problem by providing a variety of different high level assertions which make the test easier to read. Some common assertion libraries include:

chai.js (https://www.chaijs.com/)
node.js assert module (https://nodejs.org/api/assert.html)
should.js (http://shouldjs.github.io/)
hamjest (https://github.com/rluba/hamjest)

The scenario

To make the issue with "hard to maintain" tests easier to understand. I created an artificial application with different user stories. The application is an employee management system for a local supermarket. The owner wants to open the supermarket on Sundays and due to legal constraints not all employees are allowed to work on Sundays. To see who is allowed to work she asked to generate a special report of her employees. Out of simplicity the implementation focuses on the business logic in JS only. In a real world application one might query the database directly.

First User-Story

As shop owner I want to view a list of all employees, which are older than 18 years, so that I know who is allowed to work on Sundays.

After reading this requirement the following test case is generated.

import { assertThat, equalTo } from 'hamjest';

const employees = [
  { name: 'Max', age: 17 },
  { name: 'Sepp', age: 18 },
  { name: 'Nina', age: 15 },
  { name: 'Mike', age: 51 }
];

it('returns employees which are older than 18', () => {
  const result = listEmployees(employees);
  assertThat(result, equalTo([employees[1], employees[3]]));
});

After running the tests the following test fails:

❌ returns employees which are older than 18

To make this test green the following function is implemented:

const listEmployees = (employees) => employees
  .filter((employee) => employee.age >= 18);

After running the tests again the test shows green.

✔️ returns employees which are older than 18

Second User-Story

As shop owner I want the list of employees to be sorted by their name, so I can find employees easier.

Without looking to much on the existing test the next test case is added:

import { assertThat, equalTo } from 'hamjest';

const employees = [
  { name: 'Max', age: 17 },
  { name: 'Sepp', age: 18 },
  { name: 'Nina', age: 15 },
  { name: 'Mike', age: 51 }
];

it('returns employees which are older than 18', () => {
  const result = listEmployees(employees);
  assertThat(result, equalTo([employees[1], employees[3]]));
});

// New test Case
it('returns employees ordered by their name', () => {
  const result = listEmployees(employees);
  assertThat(result, equalTo([employees[3], employees[1]]));
});

✔️ returns employees which are older than 18
❌ returns employees ordered by their name

After watching the new test fail, the following is implemented:

const listEmployees = (employees) => employees
  .filter((employee) => employee.age >= 18)
  .sort((a, b) => a.name.localeCompare(b.name));

❌ returns employees which are older than 18
✔️ returns employees ordered by their name

The sorting functionality was implemented successfully but now the first already working test is failing. After comparing the test assertions it is obvious why the test fails. The test might be changed as follows:

// before
it('returns employees which are older than 18', () => {
  const result = listEmployees(employees);
  assertThat(result, equalTo([employees[1], employees[3]]));
});

// afterwards
it('returns employees which are older than 18', () => {
  const result = listEmployees(employees);
  assertThat(result, containsInAnyOrder(employees[1], employees[3]));
});

The containsInAnyOrder matcher fixes the previous issue by ignoring the sorting of the result. It verifies that the two elements need to be present independent their order. This change results in a green test suite.

️✔️ returns employees which are older than 18
✔️ returns employees ordered by their name

Third User-Story

As shop owner I want the list of employees to be capitalized, so I can read it better.

Starting from the test file again a new test is added:

import { assertThat, equalTo, containsInAnyOrder } from 'hamjest';

const employees = [
  { name: 'Max', age: 17 },
  { name: 'Sepp', age: 18 },
  { name: 'Nina', age: 15 },
  { name: 'Mike', age: 51 }
];

it('returns employees which are older than 18', () => {
  const result = listEmployees(employees);
  assertThat(result, containsInAnyOrder(employees[1], employees[3]));
});

it('returns employees ordered by their name', () => {
  const result = listEmployees(employees);
  assertThat(result, equalTo([employees[3], employees[1]]));
});

// New test case
it('returns employees whose names are capitalized', () => {
  const result = listEmployees(employees);
  assertThat(result[0].name, equalTo('MIKE'));
  assertThat(result[1].name, equalTo('SEPP'));
});

✔️ returns employees which are older than 18
✔️ returns employees ordered by their name
❌ returns employees whose names are capitalized

One possible implementation to satisfy the failing this looks like this:

const listEmployees = (employees) => employees
  .filter((employee) => employee.age >= 18)
  .sort((a, b) => a.name.localeCompare(b.name))
  .map((employee) => ({ ...employee, name: employee.name.toUpperCase() }));

After running the tests we see that the new behavior was added successfully but we broke all the other tests.

❌️ returns employees which are older than 18
❌️ returns employees ordered by their name
✔️ returns employees whose names are capitalized

The issue with the other tests is that hamjest can't compare the objects anymore because the capitalized names differ from the original ones. In this trivial example changing 2 tests might not be the biggest issue. In a more complex example figuring out if the the change broke the original behavior might take more time. In this example the test might be changed to:

// original test
it('returns employees which are older than 18', () => {
  const result = listEmployees(employees);
  assertThat(result, equalTo([employees[1], employees[3]]));
});

// first iteration
it('returns employees which are older than 18', () => {
  const result = listEmployees(employees);
  result.forEach((employee) => {
    assertThat(employee.age >= 18, equalTo(true));
  });
});

// final iteration
it('returns employees which are older than 18', () => {
  const result = listEmployees(employees);
  assertThat(result, everyItem(hasProperty('age', greaterThanOrEqualTo(18))));
});

By changing the assertion to the following we introduced one major issue to this test. The following implementation results in a green test.

const listEmployees = (employees) => []

So this assertion is now 'underspecified', which means that an invalid/broken implementation results in a green testsuite. By changing the assertion to the following, one can prevent this:

it('returns employees which are older than 18', () => {
  const result = listEmployees(employees);
  assertThat(result, allOf(
    hasProperty('length', greaterThanOrEqualTo(1)),
    everyItem(hasProperty('age', greaterThanOrEqualTo(18))),
  );
});

✔️ returns employees which are older than 18
❌️ returns employees ordered by their name
✔️ returns employees whose names are capitalized

The other test might be changed to:

// original implementation
it('returns employees ordered by their name', () => {
  const result = listEmployees(employees);
  assertThat(result, equalTo([employees[3], employees[1]]));
});

// final iteration
it('returns employees ordered by name', () => {
  const result = listEmployees(employees);
  assertThat(result, orderedBy((a, b) => a.name < b.name));
});

After those changes, all 3 tests are green. As the empty result issue is already checked by the previous test we don't test this behaviour in the other tests.

✔️ returns employees which are older than 18
✔️ returns employees ordered by their name
✔️ returns employees whose names are capitalized

Fourth User-Story

As shop owner I want the employees to be sorted by their names descending instead of ascending.

As there is already a test-case which verifies the order, we decide to change this test to match the new requirements.

import { 
  assertThat,
  greaterThanOrEqualTo, 
  everyItem, 
  orderedBy,
  hasProperty,
} from 'hamjest';

const employees = [
  { name: 'Max', age: 17 },
  { name: 'Sepp', age: 18 },
  { name: 'Nina', age: 15 },
  { name: 'Mike', age: 51 },
];

it('returns employees which are older than 18', () => {
  const result = listEmployees(employees);
  assertThat(result, everyItem(hasProperty('age', greaterThanOrEqualTo(18))));
});

// changed assertion
it('returns employees ordered by name descendent', () => {
  const result = listEmployees(employees);
  assertThat(result, orderedBy((a, b) => a.name > b.name));
});

it('returns employees whose names are capitalized', () => {
  const result = listEmployees(employees);
  assertThat(result[0].name, equalTo('MIKE'));
  assertThat(result[1].name, equalTo('SEPP'));
});

✔️ returns employees which are older than 18
️️❌ returns employees ordered by their name descendent
️️️✔️ returns employees whose names are capitalized

To make our test green again the following code is implemented:

const listEmployees = (employees) => employees
  .filter((employee) => employee.age >= 18)
  .sort((a, b) => b.name.localeCompare(a.name))
  .map((employee) => ({ ...employee, name: employee.name.toUpperCase() }));

The third test reports a failure now.

✔️ returns employees which are older than 18
✔️ returns employees ordered by their name descendent
️️️️️❌ returns employees whose names are capitalized

// original implementation
it('returns employees whose names are capitalized', () => {
  const result = listEmployees(employees);
  assertThat(result[0].name, equalTo('MIKE'));
  assertThat(result[1].name, equalTo('SEPP'));
});

// first iteration
it('returns employees whose names are capitalized', () => {
  const result = listEmployees(employees);
  assertThat(result, everyItem(hasProperty('name', matchesPattern(/[A-Z]*/))));
});

// second iteration
const inUpperCase = () => matchesPattern(/[A-Z]*/);
it('returns employees whose names are capitalized', () => {
  const result = listEmployees(employees);
  assertThat(result, everyItem(hasProperty('name', inUpperCase())));
});

We run the tests and see that all tests are green.

✔️ returns employees which are older than 18
✔️ returns employees ordered by their name descendent
️️️️️✔️ returns employees whose names are capitalized

Conclusion

This blog post showed that additional requirements might cause existing tests to fail even though their behavior didn't change. By expressing the exact desired result in an assertion makes the overall test-suite less brittle and easier to change. Having tests which don't depend on implementation details or previous tests makes it easier to add and remove functionality. For example a new feature request where employees should be returned randomized is not causing other tests to break. I've been using hamjest for the last couple of years and I can recommend to test it out.

Edit

I refactored the examples with jest and chai and pushed them to https://github.com/webpapaya/better-test-assertions. Both jest-expect and chai has issues when matching elements in an array. The API design from hamjest is easy to extend and makes it possible to write extremly complex matchers.

Edit2

The drawback section was replaced by a better matcher as it doesn't introduce a new test.