DEV Community: Henrique Barcelos

Flexbox Gotchas Reloaded

Henrique Barcelos — Mon, 20 Apr 2020 13:59:10 +0000

Important: this article describes a problem that cannot be reproduced with Google Chrome, only with Firefox and Edge (I did not test with other browsers). Spoiler alert: it seems like Chrome intentionally deviates from the spec standard.

Intro

Despite of all the jokes saying it is not even a real programming language, more that 20 years since its creation, CSS is still one of the most misunderstood web technologies.

While it allows us to describe complex layouts in a high-level fashion, sometimes we bump into issues with corner cases. Also the CSS spec is not getting any smaller with time, and while I certainly do not miss juggling around with float and clear, it is hard to deny that flexbox and grid brought a lot of complexity to CSS.

TL;DR

I stumbled upon an issue with flexbox and white-space: nowrap, causing a flex child width to be larger than its parents. Visit this codesandbox to view the solution.

Flexbox all the things

I must admit that I might be overusing flexbox, but the reality is that other methods for centering content both vertically and horizontally are way too hacky.

I had this screen from the app I am currently working on whose outer-most wrapper was a flex container. The screen is supposed to be responsive, so there was no limits for its width. Deeply nested into the HTML tree there is another flex container whose children contained text that could potentially spread across multiple lines.

I was able to create a minimum reproducible example, so please forgive me if it looks a bit contrived.

Here is the markup:

<div class="outer-flex-container">
  <div class="outer-flex-child">
    <div class="inner-flex-container">
      <div class="inner-flex-child">
        Lorem ipsum dolor sit amet, consectetur adipiscing elit. Donec nec
        blandit massa, sed fringilla odio. Quisque nec ipsum molestie,
        maximus.
      </div>
    </div>
  </div>
</div>

And here are the styles:

.outer-flex-container {
  border: 2px dashed red;
  display: flex;
  justify-content: center;
  align-items: center;
  height: 100vh;
  padding: 16px;
}

.outer-flex-child {
  border: 2px solid orangered;
  padding: 16px;
}

.inner-flex-container {
  border: 2px dashed teal;
  display: flex;
  justify-content: center;
}

.inner-flex-child {
  background: paleturquoise;
  padding: 16px;
}

The result:

The issue

Because of layout constraints, I need the text in .inner-flex-child to be displayed in a single line, so I happily went along with:

 .inner-flex-child {
   background: paleturquoise;
   padding: 16px;
+  overflow: hidden;
+  text-overflow: ellipsis;
+  white-space: nowrap;
 }

The result however surprised me:

Oddly the .inner-flex-child now stopped respecting its parent's width. Furthermore, it seems like it is forcing both .inner-flex-container and .outer-flex.child to overflow .outer-flex-container width.

Okay, let's just:

 .inner-flex-child {
   background: paleturquoise;
   padding: 16px;
   overflow: hidden;
   text-overflow: ellipsis;
   white-space: nowrap;
+  width: 100%;
 }

And that will do it!

Nop! That did not work at all. Results were still the same. I tried then every possible combination of width, max-width, min-width, flex-basis I could think of without success.

For example, if I set a fixed width for .inner-flex-child:

 .inner-flex-child {
   background: paleturquoise;
   padding: 16px;
   overflow: hidden;
   text-overflow: ellipsis;
   white-space: nowrap;
+  width: 500px;
 }

I would get:

Works as expected if the viewport is larger than 500px, but expectedly if I had a smaller screen, the problem would be back:

Switching width to max-width however did not do the trick.

Another attempt had me setting flex-basis, flex-grow and flex-shrink:

 .inner-flex-child {
   background: paleturquoise;
   padding: 16px;
   overflow: hidden;
   text-overflow: ellipsis;
   white-space: nowrap;
+  flex: 200px 0 1;
 }

The idea here was to set the flex-basis to a fixed width, do not allow .inner-flex-child to grow, but do allow it to shrink. However, the result was even weirder:

Even though the content size was respecting the 200px I set, .inner-flex-container was still overflowing its parents. My head was now a complete mess.

Halt! DuckDuckGo time

Defeated, I resorted to the knowledge of the world wide web. I started digging with DuckDuckGo (just like Google, but it respects your privacy) and found this example. Apparently setting min-width: 0; in .inner-flex-child was supposed to do the trick:

 .inner-flex-child {
   background: paleturquoise;
   padding: 16px;
   overflow: hidden;
   text-overflow: ellipsis;
   white-space: nowrap;
+  min-width: 0;
 }

But it did not! I was bummed out at first, but I kept on digging. Of course there was something regarding that in StackOverflow. However the solution pointed to the same min-width: 0 trick. I was almost dismissing it when I saw it had a link to the spec.

The linked spec session starts with:

Note: The auto keyword, representing an automatic minimum size, is the new initial value of the min-width and min-height properties. The keyword was previously defined in this specification, but is now defined in the CSS Sizing module.

From that I understood that flex children have the value of min-width and min-height default to auto if not specified.

Then it goes on:

To provide a more reasonable default minimum size for flex items, the used value of a main axis automatic minimum size on a flex item that is not a scroll container is a content-based minimum size; for scroll containers the automatic minimum size is zero, as usual.

That's a mouthful. I was in rush/lazy mode by the time I was reading this, so I gave up. This is a behavioral pattern I fallback into when I am tired and with enough training I learnt how to recognize this and went out — of the room, since I am writing this in the middle of the Covid-19 pandemic, I could not leave the apartment — for a while.

After a cup of coffee and trying to put myself in the right mental state, I went back to the same SO question. It actually does a good job "translating" the spec into something more digestible:

With regard to the auto value...

On a flex item whose overflow is visible in the main axis, when specified on the flex item’s main-axis min-size property, specifies an automatic minimum size. It otherwise computes to 0.

In other words:

The min-width: auto and min-height: auto defaults apply only when overflow is visible.

If the overflow value is not visible, the value of the min-size property is 0.

Hence, overflow: hidden can be a substitute for min-width: 0 and min-height: 0.

But here is the problem: .inner-flex-child is already a flex container, because it has overflow: hidden set. Damn you, CSS!

Reading on, I reached this section:

You've applied min-width: 0 and the item still doesn't shrink?

Nested Flex Containers

If you're dealing with flex items on multiple levels of the HTML structure, it may be necessary to override the default min-width: auto / min-height: auto on items at higher levels.

Basically, a higher level flex item with min-width: auto can prevent shrinking on items nested below with min-width: 0.

That was exactly the problem I was facing. I was only meddling with .inner-flex-child and .inner-flex-container the whole time. However, I had more flex content above in the DOM hierarchy. Dumbass!

So I tried the following:

 .outer-flex-child {
   border: 2px solid orangered;
   padding: 16px;
+  min-width: 0;
 }

Aaaaaand bingo!

I decided to play around with the working example and found out that the following approaches have the same effect:

 .outer-flex-child {
   border: 2px solid orangered;
   padding: 16px;
+ overflow: hidden;
 }

According to the spec, this fixes the issue because .outer-flex-child is now a scroll container, so min-width default value falls-back to 0. The downside of this is that if it contained any children that might be out of bounds — such as a tooltip or a modal with position: absolute — they would be clipped out.

 .outer-flex-child {
   border: 2px solid orangered;
   padding: 16px;
+  max-width: 100%;
 }

While not directly mentioned in the linked spec section, I noticed that .outer-flex-container width itself was not affected by the overflowing content. When using % as unit, it will be always relative to the parent's dimensions, so setting max-width: 100% does the trick as well, at least in this specific case.

Here is a codesandbox with the example I used throughout this article and the possible solutions.

That's all folks!

TDD made simple with Mocha and Chai

Henrique Barcelos — Thu, 06 Feb 2020 13:22:17 +0000

This article was orignally published in my personal blog.

Intro

From the dark ol’ days of writing an entire application and only then starting to test it (often, manually) till nowadays, I have scoured a painful path of unending bug-fixing in production through the nights, many times not even knowing what was causing those bugs.

Yeah, something crazy is going on!

Since I first heard of Test Driven Development, it changed the way I think about software development.

I will not be digressing about TDD philosophy and its implications here, because a lot of more qualified people have done it before me. So let’s get to the code!

First, the problem and its solution

A long time ago in a galaxy far far away, I ended up in a problem: I had to monitor a “stream” (more like a polling) of events that were being created at a certain application in my Node.JS backend. This “stream” was not uniform and, most of the time, no event occurred.

I could not use websockets, so I would have to buffer these events in my backend. I thought using a database (even an in-memory one like Redis) just for that was too much. Then I decided that I would keep the events in memory and as my application did not care for all events that ever happened, I would keep only the last N of them.

Since Node.JS arrays are dynamic, they did not fit my needs. I did not want a fixed-size array implementation, what I needed was a fixed-sized first-in/first-out (FIFO) data structure, AKA a queue, which instead of overflowing when full, should pop its first element and then add the new one at the end.

Expected behavior

You better do that!

The data structure described above is rather simple. Its expected behavior could be summarized as follows:

Adding elements:

When it is not full, it should add the new element to the end; its size should be increased by 1.
When it is full, it should remove the first element and then add the new element to the end; its size must not change.
- The removed element should be returned.

Removing elements:

When it is not empty, it should remove the first element and return it; its size should be decreased by 1.
When it is empty, it should throw an error.

A Mocha to go, please!

Looks delicious! ;9

From the docs:

Mocha is a feature-rich JavaScript test framework running on Node.js and in the browser, making asynchronous testing simple and fun. Mocha tests run serially, allowing for flexible and accurate reporting, while mapping uncaught exceptions to the correct test cases. Hosted on GitHub.

Installation

yarn add --dev mocha
# or with NPM:
# npm install --save-dev mocha

Writing tests

To create a test suite, you use a globally defined function called describe. To add test cases to a suite, you should use another global function it:

Suites can be nested indefinitely when you want to group your test cases. Mocha will collect all your suites recursively and execute all test cases it find within them in the order they are declared.

And that’s probably about all you need to tedknow about Mocha to get star (at least for basic usage). It excels so much for simplicity and extensibility, that it allows you to use whatever assertion library and other plugins you want.

Running tests

yarn mocha '<path-to-test-file>'
# or with NPM's npx:
# npx mocha '<path-to-test-file>'

Enter Chai

I used to think that Node.JS developers only like coffee… Guess I was not totally right (:

By default, Mocha can be used along with Node.js native assert module. It works just fine, however I don't find its developer experience to be exactly great. For that reason, we will use a 3rd-party assertion library called Chai.

From the docs:

Chai is a BDD / TDD assertion library for node and the browser that can be delightfully paired with any JavaScript testing framework.

Installation

yarn add --dev chai
# or with NPM:
# npm install --save-dev chai

Usage

Chai offers 3 different styles for writing assertions:

Chai allows you to pick your own poison.

All of them have the same capabilities, so choosing one or another is more a matter of preference than of objective facts. I like to use the expect interface.

Oh, tests! Oh, dreaded tests!

These bug’s bites hurt a lot; better kill’em before they kill you!

Going back to our original problem, let’s translate the expected behavior into mocha test suites. But first, let’s do some setup:

const chai = require("chai");
const expect = chai.expect;

const RoundQueue = require("./round-linked-queue");

describe("Round-Queue", () => {
});

Source

Testing queue creation

The main reason why we are creating this data structure is that it has to be a limited size, so let's make sure it has such property:

const chai = require("chai");
const expect = chai.expect;

const RoundQueue = require("./round-linked-queue");

describe("Round-Queue", () => {
  describe("When creating an instance", () => {
    it("Should properly set the maxLength property", () => {
      const queueLength = 3;

      const queue = new RoundQueue(queueLength);

      expect(queue.maxLength).to.equal(queueLength);
    });
  });
});

Source and diff

Next we implement just enough code to make the test above pass:

class RoundLinkedQueue {
  constructor(maxLength) {
    this._maxLength = maxLength;
  }

  get maxLength() {
    return this._maxLength;
  }
}

module.exports = RoundLinkedQueue;

Source

To run the suite, we do:

yarn mocha round-linked-queue.test.js

Keep moving and we must ensure that a queue is created empty:

it("Should initially set the length to zero", () => {
  const queueLength = 3;

  const queue = new RoundQueue(queueLength);

  expect(queue.length).to.equal(0);
});

Source

In order to make the new test pass, we can do as follows:

class RoundLinkedQueue {
  constructor(maxLength) {
    this._maxLength = maxLength;
    this._length = 0;
  }

  get maxLength() {
    return this._maxLength;
  }

  get length() {
    return this._length;
  }
}

Source and diff.

Testing adding elements

Next we create another test suite inside the top-level suite to test the behavior of adding elements to a queue.

Our base use case happens when the queue is empty and we want to add an element to it:

describe("When adding elements", () => {
  it("Should add an element to an empty queue", () => {
    const queue = new RoundQueue(3);
    const originalLength = queue.length;
    const elementToAdd = 1;

    queue.add(elementToAdd);

    // Element should've been added to the end of the queue
    expect(queue.last).to.equal(elementToAdd);
    // But since it is now the only element, it should also be the at beginning as well
    expect(queue.first).to.equal(elementToAdd);
    // Length should've been increased by 1
    expect(queue.length).to.equal(originalLength + 1);
  });
});

Source

If you run the test suite right now, you will get the following error:

D'oh!

The test failed because we didn't implement the add method yet. Now we add just enough code to make this first test case pass.

Important: the code bellow is not entirely correct, we will have to modify it further in order to make the add method work as expected. However, it does make our first test case "adding element to an empty queue" pass.

class RoundLinkedQueue {
  // ...

  add(element) {
    this._root = element;
    this._first = element;
    this._last = element;

    this._length += 1;
  }
}

Source and diff

Now let's try adding a test for when the queue is not empty anymore and yet we still want to add an element to it:

it("Should add an element to the end of a non-empty queue", () => {
  const queue = new RoundQueue(3);
  const previousElement = 1;
  const elementToAdd = 2;
  // Make the queue non-empty
  queue.add(previousElement);

  queue.add(elementToAdd);

  // Element should've been added to the end of the queue
  expect(queue.last).to.equal(elementToAdd, "last not properly set");
  // But the first pointer must remain the first element added
  expect(queue.first).to.equal(previousElement, "first not properly set");
  // Length should've been increased by 2
  expect(queue.length).to.equal(2, "length not properly set");
});

If we once again run the test suite without changing the implementation, we will get a failure:

The more attentive readers should probably be expecting this error because the way we implemented the add method before would simply overwrite the elements in the queue. To fix this, we will need some more code:

class RoundLinkedQueue {
  // ...

  add(element) {
    const node = {
      data: element,
      next: null,
    };

    if (!this._root) {
      this._root = node;
      this._first = node;
      this._last = node;
    } else {
      const previousLast = this._last;
      previousLast.next = node;

      this._last = node;
    }

    this._length += 1;
  }
}

Source and diff

We had to convert our _root, _first and _last into a node object containing data — the actual value of the item — and next — a pointer to the next node in the linked list.

Moving on, now it's time to something a little bit more challenging. Whenever our queue is at capacity, adding a new element should should cause the removal of the element that was first added:

it("Should remove the first element and add the new element to the end of a full queue", () => {
  const queue = new RoundQueue(3);
  queue.add(1);
  queue.add(2);
  queue.add(3);

  queue.add(4);

  // Element should've been added to the end of the queue
  expect(queue.last).to.equal(4, "last not properly set");
  // The second element should've been shifted to the first position
  expect(queue.first).to.equal(2, "first not properly set");
  // Length should still be the same
  expect(queue.length).to.equal(3, "length not properly set");
});

Source

Running tests once more we get:

Looks like we will need some conditionals to make the new test case pass along with the previous ones:

class RoundLinkedQueue {
  // ...

  add(element) {
    const node = {
      data: element,
      next: null,
    };

    if (this.length < this.maxLength) {
      if (!this._root) {
        this._root = node;
        this._first = node;
        this._last = node;
      } else {
        const previousLast = this._last;
        previousLast.next = node;

        this._last = node;
      }

      this._length += 1;
    } else {
      this._root = this._root.next;
      this._last.next = node;
      this._first = this._root;
      this._last = node;
    }
  }
}

Source and diff

Halt! Refactor time

So far we were writing code in a rather linear fashion: make a failing test, implement code to make it pass; make another failing test, write just enough code to make it pass, and so on.

In TDD jargon, creating a failing test is called the red phase, while implementing the code that will make it pass is the green phase.

In reality, things are not so pretty-neaty. You will not always get how to write the best code possible the first time. The truth is we've been cheating a little: we were skipping the refactor phase of the TDD cycle:

The gist of TDD

Right now I see some possible improvements in our data structure:

Having both _root and _first properties seem redundant.
There is some duplication of code in the add method (remember DRY?)

Because we already know the expected behavior, which is coded in our test suite, we are comfortable to refactor mercilessly.

class RoundLinkedQueue {
  // ...

  add(element) {
    const node = {
      data: element,
      next: null,
    };

    if (this.length < this.maxLength) {
      if (!this._first) {
        this._first = node;
        this._last = node;
      }

      this._length += 1;
    } else {
      this._first = this._first.next;
    }

    this._last.next = node;
    this._last = node;
  }
}

Source and diff

Hopefully, our tests are still green:

Taking some shortcuts

Now we are going to cheat a little bit.

The last requirement is that the add method should return the removed element when the queue is full. What to return when the queue is not full is not in the specification though. In JavaScript, uninitialized values have a special value called undefined. It makes sense to return that when adding to the queue does not remove any element, so we can add the following two test cases.

TDD purists gonna be triggered.

it("Should return the removed element from a full queue", () => {
  const queue = new RoundQueue(3);
  queue.add(1);
  queue.add(2);
  queue.add(3);

  const result = queue.add(4);

  expect(result).to.equal(1, "removed wrong element");
});

it("Should return undefined when the queue is not full", () => {
  const queue = new RoundQueue(3);

  const result = queue.add(1);

  expect(result).to.equal(undefined, "should not return an element");
});

Source

Cool, so let's just return the element from the node we just removed:

class RoundLinkedQueue {
  // ...

  add(element) {
    const node = {
      data: element,
      next: null,
    };

    let removedElement;

    if (this.length < this.maxLength) {
      if (!this._first) {
        this._first = node;
        this._last = node;
      }

      this._length += 1;
    } else {
      removedElement = this._first.data;
      this._first = this._first.next;
    }

    this._last.next = node;
    this._last = node;

    return removedElement;
  }
}

Source and diff

Look like we are are done with the add method!

Testing removing elements

Removing elements seems like a simpler operation. Our base use case is when the queue is not empty. We remove an element from it and decrease its length by one:

describe("When removing elements", () => {
  it("Should remove the first element of a non-empty queue", () => {
    const queue = new RoundQueue(3);
    queue.add(1);
    queue.add(2);
    queue.add(3);
    const lengthBefore = queue.length;

    const result = queue.remove();

    const lengthAfter = queue.length;

    expect(lengthAfter).to.equal(lengthBefore - 1, "length should decrease by 1");
    expect(result).to.equal(1, "first element should the one being removed");
    expect(queue.first).to.equal(2, "should shift the second element to the head of the queue");
    expect(queue.last).to.equal(3, "should not change the last element");
  });
});

Source

Running the tests will once again give us an error:

Now we add some code just to make the test pass:

class RoundLinkedQueue {
  // ...

  remove() {
    const removedElement = this.first;

    this._first = this._first.next;
    this._length -= 1;

    return removedElement;
  }
}

Source and diff

The only other use case is when the queue is empty and we try to remove an element from it. When this happens, the queue should throw an exception:

it("Should throw an error when the queue is empty", () => {
  const queue = new RoundQueue(3);

  expect(() => queue.remove()).to.throw("Cannot remove element from an empty queue");
});

Source

Running the test suite as is:

Ouch!

Adding some conditions to test for emptyness and throw the proper error:

class RoundLinkedQueue {
  // ...

  remove() {
    const removedNode = this._first;
    if (!removedNode) {
      throw new Error("Cannot remove element from an empty queue");
    }

    this._first = this._first.next;
    this._length -= 1;

    return removedNode.data;
  }
}

Source and diff

And that's it!

Testing edge cases

There are still some bugs in or code. When we wrote the add method, we included the first and last getters as well. But what happens if we try to access them when the queue is empty? Let's find out! first things first (ba dum tsss!):

describe("When accessing elements", () => {
  it("Should throw a proper error when acessing the first element of an empty queue", () => {
    const queue = new RoundQueue(3);

    expect(() => queue.first).to.throw("Cannot access the first element of an empty queue");
  });
});

Source

Running the tests:

Looks like the error message is not really helpful. In fact, it is a little too low level. Let's make it better:

class RoundLinkedQueue {
  // ...

  get first() {
    if (!this._first) {
      throw new Error("Cannot access the first element of an empty queue");
    }

    return this._first.data;
  }

  // ...
}

Source and diff

Lastly, for the last getter, we will do the same:

it("Should throw a proper error when acessing the last element of an empty queue", () => {
  const queue = new RoundQueue(3);

  expect(() => queue.last).to.throw("Cannot access the last element of an empty queue");
});

Source

First the failing result:

Then fixing the code:

class RoundLinkedQueue {
  // ...

  get last() {
    if (!this._last) {
      throw new Error("Cannot access the last element of an empty queue");
    }

    return this._last.data;
  }

  // ...
}

Source and diff

Aaaaaaand that's about it!

Conclusion

I tried to make this a comprehensive introduction to TDD with the Node.js/JavaScript ecosystem. The data structure we had to implement here was intentionally simple so we could follow the methodology as much as possible.

When doing TDD in real world applications, things are usually not so linear. You will find yourself struggling from time to time with the design choices you make while writing your tests. It can be a little frustrating in the beginning, but once you get the gist of it, you will develop a "muscle memory" to avoid the most common pitfalls.

TDD is great, but as almost everything in life, it is not a silver bullet.

Be safe out there!

T-t-th-tha-that's i-is a-a-all f-f-fo-f-folks!

Did you like what you just read? Why don’t you buy me a beer (or a coffee if it is before 5pm 😅) with tippin.me?

Give your logs more context - Part 2

Henrique Barcelos — Thu, 06 Feb 2020 12:43:01 +0000

This article was originally published in my personal blog.

Building a contextual logger

This is the continuation of my previous article about logging context. Check it out to better understand the purpose of what we will build.

Give your logs more context - Part 1

Henrique Barcelos ・ Feb 6 '20 ・ 8 min read

#node #logging

TL;DR

The code we are going to build on this story is on my Github. If you just want to check out the final version, you can get it at the master branch.

Intro

Last time we walked through a way of managing context through concurrent requests using pino and cls-hooked. Now let’s build a wrapper around pino that will automatically deal with this for us.

And now, it’s tiiiiiiiime!

What do we want to achieve?

We need to build a logger that will have base “global” context through cls-hooked, but will also allow us to augment such context when actually calling the logger methods.

To improve reusability and interoperability, we want to maintain the original default pino API, so we already have a good set of test cases to cover. Also, we need to provide a way for our application interact with the context.

How will we write our code?

We are going to implement this wrapper TDD style. However the tests we will write are not “unit” tests in a strict sense, because they will include pino itself and make assertions about the generated log data. This is possible because pino accepts a custom WritableStream as its destination.

As testing framework, we will use ava. Keep in mind that while ava transpiles test files by default, it doesn’t do that for the actual code without properly setting babel. To avoid adding more complexity to this solution, all code (including tests) will not use ES modules or any features that are not available in Node.js 10.9.0.

If you want do follow along the implementation, please check out the instructions in the Github repository:

hbarcelos / give-your-logs-more-context

Wrapper on top of pino which provides integration with cls-hooked for better context in log messages

Give your logs more context

This repository is the implementation of the ideas from "Give your logs more context" article series:

Requirements

Yarn

Setup

If you are following follow the article, after cloning this repository, I checkout the initial tag:

git clone https://github.com/hbarcelos/give-your-logs-more-context
git checkout initial

Each step on the article is a subsequent commit, which you can reference by the commit hash.

The final version is under the final tag, which is also the HEAD of the master branch.

git checkout final
# or...
git checkout master

After choosing what you want to see, run:

yarn install
yarn test

View on GitHub

I tried to make this the sequence as natural as possible, only eliminating some inner loops and struggles that happen in a regular coding session.

Implementation steps

Initial setup

yarn init -y
yarn add pino cls-hooked
yarn add --dev ava

A nice feature of pino accepts a custom WritableStream as its destination. This will make our lives easier when testing our custom logger.

Ensuring methods for log levels

For simplicity, lets stick with pino default log levels: trace, debug, info, warn, error and fatal.

The simplest way to achieve that is:

The logger.js is currently just a factory function that return plain pino instances. The logger.test.js file generates one test case for each available method to make sure we don’t break anything later.

parse-json-stream.js is a utility that will parse the log output stream and return plain Javascript objects to make it easier to run assertions against the log output.

stream-to-generator.js is there for convenience: ava doesn’t play well with stream-based APIs. To make tests more concise, we convert the logging stream to a generator that yields promises to the next log entry.

The later two are not important in the context of what we are trying to achieve, they are here only for reference. The remaining snippets won’t include them.

Keeping context on logger method call

Also, notice that pino allow us to pass local context to a log entry by prepending an object to the argument list. This is a behavior we want to keep.

So, lets add a test case that covers this scenario:

Since so far we are just creating a pino instance, the test will pass.

Adding CLS awareness

Now we start touching CLS. First we need to create namespace and expose it to the world:

Preventing CLS context sharing between instances

For some reason, we might want to have multiple loggers in a given application. When doing that, it’s important to not mix the namespaces of both. However, the way we implemented above, all instances will have the same namespace '@@logger', which could cause strange behavior laters.

The easiest way to fix this would be to have a counter variable that would increment whenever we call createLogger and append the counter value to the namespace name.

While counters are not the most safe bet to generate unique names, since they are reset when the application restarts, they work in this case because all logger instances would be recreated anyway when the server restarts. Also, this value is not exposed anywhere, it serves only for the purpose of creating different namespaces, so we are fine.

Sometimes less is more!

Here’s what’s changed:

Applying CLS context to logs

This one is a big leap, so bear with me. First, let’s see the changes in the code, then let’s discuss it:

Sorry, I couldn’t break this into smaller changes :/

The test code has nothing special about it, just notice that we must run our logging and assertion within the logger.cls.run method callback.

Things start to get interesting on the actual code though. We are leveraging Javascript Proxy to intercept log method calls and patch their arguments.

So, in line 52 we create a proxy for our logger object, whose handler is named a loggerObjectHandler— lines 34–43. The handler defines a get trap, that will intercept only the calls for the log methods — trace, debug, etc. What it does is wrap those methods into yet another proxy, whose handler is named logMethodHandler — lines 11–32.

The loggerMethodHandler gathers the current active context on CLS, excluding some irrelevant properties from it — lines 14–15. Then, based on the current argument list, it checks whether we have or not a local context on the method call. If we don’t, then we simply need to prepend the CLS context to the argument list — lines 20–23. Otherwise, we need to merge the local context into the CLS context — lines 24–28. Finally, we call the original method with the proper arguments — line 30.

Propagating changes to child loggers

A nice feature from pino is that it allows us to create child loggers through the .child() method. A child logger maintains all properties from its parent, but can also accept additional context. So, we need to make our child generation CLS aware too:

Again, the new tests are self-descriptive. Let’s focus on the implementation. First we extracted the wrapper creation into its own function, named createWrapper — lines 47–52. This allows us to create a wrapper for the child loggers as well.

Next, we define a childMethodHandler which will intercept the calls to .child() — lines 18–25. This handler will call createWrapper on the newly created child logger, passing the CLS context from the parent as a parameter. This will guarantee that parent and children (and children of children) all have the same context.

Lastly, we change the implementation of loggerObjectHandler to include the proxy for the .child() method as well — lines 30–45 — including some internal refactoring on the conditionals.

Further improvements

Seems like our code works so far, but it might not be optimal. An issue that is easy to spot is that we are creating new proxies on the fly for every call on the child and log methods. While this might not be an issue with the former — because we wouldn’t call .child() very often — that’s not true for the latter.

To prevent this problem, we could create the proxies for the desired methods by the time we create the logger itself and put them as properties of the logger object. When we call the methods, the loggerObjectHandler would just check to see if there is a proxy set for the current method. If so, it returns the proxy, otherwise, it returns the original property:

Integrating with our web application

So now we have our logger factory. Now we need to integrate it with our application. From the final example from the previous article, we could refactor to:

Outro

The code above is pretty much the same I have successfully used in production some times and have saved me and the teams I worked with a lot of time while debugging.

If you have any suggestions that could improve it, you are more than welcome.

Did you like what you just read? Buy me a beer with tippin.me.

Give your logs more context - Part 1

Henrique Barcelos — Thu, 06 Feb 2020 01:31:28 +0000

This article was originally published in my personal blog.

How to make sense out of your Node.js web app logs

Logging might be one of the most difficult things to do right when building a real world application. Log too little and you will be staring at your screen trying to make sense of them (or the charts generated from them). Log too much and you will end up lost in a swamp of useless information, still having no clue if everything is OK or if you have a problem.

Logs without the right amount of context look like…
st
Speaking specifically of the Node.js/Javascript ecosystem, the top 3 logging libraries — Winston, Bunyan and Pino — can help you to manage context way better than the good ol’ console.log could ever do.

For this article I will use Pino, but the ideas can be easily replicated for both Bunyan and Winston (or any other mainstream logging utility).

Use log levels wisely

Pino has 6 default log levels, with increasing severity: trace, debug, info, warn, error and fatal. Each one of these levels maps to an integer from 10 to 60. This makes it easy to analyze your logs later using tools like jq:

jq 'select(.level > 40)' # gets ERROR and FATAL logs

While Pino allows you to define custom log levels, I have never seen a use case where they would be necessary, so I tend to stick with the default ones.

Usually, for production, it is recommended to ignore trace and debug levels, unless you are explicitly trying to debug some production issue.

Pino has a configuration option that allows you to define the minimum required level for the log entry to be generated. You can use environment variables to avoid having to make a deploy just to change the log level:

const logger = pino({
  level: process.env.LOG_LEVEL || 'info'
});

Rules of thumb

Use trace for internal logging that has a potentially high throughput.
Use debug for eventual debugging sessions you might need, but remember to remove them after you are finished.
Use info for regular application workflow logs.
Use warn for expected and frequent error conditions (like user input validation).
Use error for expected but infrequent error conditions (like network failures, database timeouts).
Use fatal for unexpected error conditions.

Embrace request IDs

While we are still developing the application, running unit/integration tests, manually triggering some request to see if everything is running smoothly, it’s all good. The events being produced happen in a more or less predictable order, so it’s easy to follow.

However, once the production version is launched, things can go really crazy. Your app will most certainly process concurrent requests. If you have a few asynchronous steps — like querying a database or calling some external services — the order of each event will be completely unpredictable. In this case, if you are manually inspecting the logs (we all have done this at some point 😅), you can become very frustrated trying to find a thread of execution.

Some frameworks — like Hapi — already take care of this for you. But if you like me still rely on good ol’ express, you have to do it yourself. Defining a middleware that does that is as simple as:

function setRequestId(generateId) {
  return (req, res, next) => {
    req.id = generateId();
    next();
  };
}

Then use it:

let i = 0;
const generateId = () => i++;
app.use(setRequestId(generateId));</span>

Of course, this naive implementation would not work if you ever restart your server, since the counter would be reset to 0. For a real world application, it’s recommended to use a more robust ID generator, such as uuid or, my personal choice, cuid.

If you use a micro-services architecture (or want to be prepared to), you can leverage distributed tracing simply by allowing your services to forward and receive a given request ID:

function setDistributedRequestId(generateId) {
  return (req, res, next) => {
    const reqId = req.get('X-Request-Id') || generateId();
    req.id = reqId;
    res.set('X-RequestId', reqId);
    next();
  };
}

Now we can create another middleware that logs incoming requests:

function logIncomingRequests(logger) {
  return (req, res, next) => {
    logger.trace({ req, requestId: req.id}, 'Incoming request');
    next();
  }
}

And use it:

app.use(logIncommingRequests(pino()))</span>

The generated log entry would look like:

{"level":30, "time":1533749413556, "pid":15377, "hostname":"henrique-pc", "msg":"Incoming request", "req":{"method":"GET", "url":"/", "headers":{"host":"localhost:4004", "user-agent":"curl/7.61.0", "accept":"*/*"}},
"requestId":1, # <---- notice here!
"v":1}

So far, so good. We could even use express-pino-logger to further integrate the logger with our express app. The major problem here is that the request ID is tightly coupled with our web layer. Unless you define all your business logic within express handlers — something I urge you to please don’t — you won’t be able to access the request ID value in other layers.

Oh, I could store the IDs in-memory or in a Redis cache and then retrieve it when logging something in other layers!!!

Yeah, nice try. I thought that myself too, but it doesn’t work. The reason is that you can’t know which request you are currently processing when you have concurrent accesses. Or can you?

Meet Continuation Local Storage

Imagine that each request is an isolated “thread” of connected execution paths (function calls) that is discarded when the result of the original call is returned.

While Javascript doesn’t spawn real threads for handling user requests, it emulates this by registering callbacks that will be called in the proper sequence when the results of the function calls are available.

Luckily for us, Node.js provides a way to intercept the hops through this execution “thread”. Continuation Local Storage (or CLS for short) leverages this capability to keep data available within a given “thread”.

When you set values in continuation-local storage, those values are accessible until all functions called from the original function — synchronously or asynchronously — have finished executing. This includes callbacks passed to process.nextTick and the timer functions (setImmediate, setTimeout, and setInterval), as well as callbacks passed to asynchronous functions that call native functions (such as those exported from the fs, dns, zlib and crypto modules).

Me when I first discovered CLS…

Redefining our request ID middleware, we would have something like:

import { createNamespace } from 'cls-hooked';
import cuid from 'cuid';
const loggerNamespace = createNamespace('logger');

function clsRequestId(namespace, generateId) {
  return (req, res, next) => {
    const reqId = req.get('X-Request-Id') || generateId();

    res.set('X-RequestId', reqId);

    namespace.run(() => {
      namespace.set('requestId', reqId);
      next();
    });
  };
}

app.use(clsRequestId(loggerNamespace, cuid));

Breaking it down:

A namespace is roughly the CLS equivalent of a table from a relational database or a collection/key space from a document store. To create one, we simply need to identify it as a string.
Our “high order” middleware clsRequestId now needs two parameters: the namespace and the ID generator function.
namespace.run is the function that creates a new context, bounded to the execution “thread”.
namespace.set puts the request ID into local storage.
next will call the next express handler. IMPORTANT: to make this work as expected, next MUST be called inside the namespace.run callback.

Now, whenever we need to access this value, we can use getNamespace from cls-hooked:

import { getNamespace } from 'cls-hooked';
import pino from 'pino';
const logger = pino();

loggerNamespace = getNamespace('logger');

function doStuff() {
  // ...
  logger.info({ requestId: loggerNamespace.get('requestId') }, "Some message");
}

If function doStuff call was ultimately originated in one of the handlers from the express app which registered that clsRequestId middleware, the value will be available.

Putting everything toghether:

Here’s a sample output generated with autocannon:

{"level":30,"time":1533759930690,"msg":"App is running!","pid":4985,"hostname":"henrique-pc","endpoint":"http://localhost:4000","v":1}
{"level":30,"time":1533759933634,"msg":"Before","pid":4985,"hostname":"henrique-pc",**"requestId":"cjkll2awx0000uhwg9qh20e0b"**,"v":1}
{"level":30,"time":1533759933636,"msg":"Before","pid":4985,"hostname":"henrique-pc",**"requestId":"cjkll2awz0001uhwgoyiptfxv"**,"v":1}
{"level":30,"time":1533759935531,"msg":"Middle","pid":4985,"hostname":"henrique-pc",**"requestId":"cjkll2awz0001uhwgoyiptfxv"**,"v":1}
{"level":30,"time":1533759939590,"msg":"Middle","pid":4985,"hostname":"henrique-pc",**"requestId":"cjkll2awx0000uhwg9qh20e0b"**,"v":1}
{"level":30,"time":1533759941222,"msg":"After","pid":4985,"hostname":"henrique-pc",**"requestId":"cjkll2awz0001uhwgoyiptfxv"**,"v":1}
{"level":30,"time":1533759941228,"msg":"Before","pid":4985,"hostname":"henrique-pc",**"requestId":"cjkll2grw0002uhwgzz14qyb6"**,"v":1}
{"level":30,"time":1533759943632,"msg":"Before","pid":4985,"hostname":"henrique-pc",**"requestId":"cjkll2imo0003uhwgf4dutgz3"**,"v":1}
{"level":30,"time":1533759946244,"msg":"Middle","pid":4985,"hostname":"henrique-pc",**"requestId":"cjkll2grw0002uhwgzz14qyb6"**,"v":1}
{"level":30,"time":1533759949490,"msg":"After","pid":4985,"hostname":"henrique-pc",**"requestId":"cjkll2awx0000uhwg9qh20e0b"**,"v":1}
{"level":30,"time":1533759951621,"msg":"Middle","pid":4985,"hostname":"henrique-pc",**"requestId":"cjkll2imo0003uhwgf4dutgz3"**,"v":1}
{"level":30,"time":1533759952464,"msg":"After","pid":4985,"hostname":"henrique-pc",**"requestId":"cjkll2grw0002uhwgzz14qyb6"**,"v":1}
{"level":30,"time":1533759953632,"msg":"Before","pid":4985,"hostname":"henrique-pc",**"requestId":"cjkll2qcg0004uhwgnmgztdr7"**,"v":1}
{"level":30,"time":1533759954665,"msg":"Middle","pid":4985,"hostname":"henrique-pc",**"requestId":"cjkll2qcg0004uhwgnmgztdr7"**,"v":1}
{"level":30,"time":1533759955140,"msg":"After","pid":4985,"hostname":"henrique-pc",**"requestId":"cjkll2imo0003uhwgf4dutgz3"**,"v":1}
{"level":30,"time":1533759957183,"msg":"After","pid":4985,"hostname":"henrique-pc",**"requestId":"cjkll2qcg0004uhwgnmgztdr7"**,"v":1}

If you look closely you will see that, even though the call order of the logger function is non-linear, the requestId for each different request is mantained.

It’s complete maaaagic!

Now, whenever you want to see the logs from a single request in isolation, you can again use jq and run:

jq 'select(.requestId == "cjkll2qcg0004uhwgnmgztdr7")' <log_file>

The output will be:

{
  "level": 30,
  "time": 1533759953632,
  "msg": "Before",
  "pid": 4985,
  "hostname": "henrique-pc",
  "requestId": "cjkll2qcg0004uhwgnmgztdr7",
  "v": 1
}
{
  "level": 30,
  "time": 1533759954665,
  "msg": "Middle",
  "pid": 4985,
  "hostname": "henrique-pc",
  "requestId": "cjkll2qcg0004uhwgnmgztdr7",
  "v": 1
}
{
  "level": 30,
  "time": 1533759957183,
  "msg": "After",
  "pid": 4985,
  "hostname": "henrique-pc",
  "requestId": "cjkll2qcg0004uhwgnmgztdr7",
  "v": 1
}

Further improvements

While the structure presented in this story works, it’s not practical for everyday use. It would be very tedious having to manually get the namespace and retrieve all values you need like in the example code above:

const namespace = getNamespace('logger');                                                 logger.info({ requestId: namespace.get('requestId') }, 'Before')</span>

Next time we will build a wrapper around pino to handle all of this transparently.

Bye!

Did you like what you just read? Buy me a beer with tippin.me

Part 2 is now available here.

A better approach for testing your Redux code

Henrique Barcelos — Mon, 25 Nov 2019 00:25:48 +0000

This article was originally published in my blog.

TL;DR

When testing Redux, here are a few guidelines:

Vanilla Redux

The smallest standalone unit in Redux is the entire state slice. Unit tests should interact with it as a whole.
There is no point in testing reducers, action creators and selectors in isolation. As they are tightly coupled with each other, isolation gives us little to no value.
Tests should interact with your redux slice same way your application will. Use action creators and selectors, without having to write tests targeting them in isolation.
Avoid assertions like toEqual/toDeepEqual against the state object, as they create a coupling between your tests and the state structure.
Using selectors gives you the granularity you need to run simple assertions.
Selectors and action creators should be boring, so they won't require testing.
Your slice is somewhat equivalent to a pure function, which means you don't need any mocking facilities in order to test it.

Redux + `redux-thunk`

Dispatching thunks doesn't have any direct effect. Only after the thunk is called is when we will have the side-effects we need to make our application work.
Here you can use stubs, spies and sometimes mocks (but don't abuse mocks).
Because of the way thunks are structured, the only way to test them is by testing their implementation details.
The strategy when testing thunks is to setup the store, dispatch the thunk and then asserting whether it dispatched the actions you expected in the order you expected or not.

I have created a repo implementing the ideas above.

Intro

As a Software Engineer, I am always finding ways to get better at my craft. It is not easy. Not at all. Coding is hard enough. Writing good code is even harder.

Then there are tests. I think every single time I start a new project — professionally or just for fun — my ideas on how I should test my code change. Every. Single. Time. This is not necessarily a bad thing as different problems require different solutions, but this still intrigues me a little.

The Problem with Tests

As a ~most of the time~ TDD practitioner, I have learned that the main reason we write tests it not to assert the correctness of our code — this is just a cool side effect. The biggest win when writing tests first is that it guides you through the design of the code you will write next. If something is hard to test, there is probably a better way to implement it.

However, after if you have done this for some time, you realize that writing good tests are as hard as writing production code. Sometimes is even harder. Writing tests takes time. And extra time is something that your clients or the business people in your company will not give you so easily.

Ain't nobody got time for that! (Photo by Aron Visuals on Unsplash)

And it gets worse. Even if you are able to write proper tests, throughout the lifespan of the product/project you are working on, requirements will change, new scenarios will appear. Write too many tests, make them very entangled and any minor change in your application will take a lot of effort to make all tests pass again. Flaky tests are yet another problem. When it fails, you have no idea were to start fixing it. You will probably just re-run the test suite and if it passes, you are good to go.

Schrödinger's tests: sometimes they fail, sometimes they pass, but you cannot know for sure (Picture by Jie Qi on Flickr)

But how do you know if you are writing good tests? What the hell is a good test in the first place?

Schools of Testing

There is an long debate between two different currents of thoughts known as London School and Detroit School of Testing.

Summarizing their differences, while Detroit defends that software should be built bottom-up, with emphasis on design patterns and the tests should have as little knowledge as possible about the implementation and have little to no stubbing/mocking at all, London advocates that the design should be top-down, using external constraints as starting point, ensuring maximum isolation between test suites through extensive use of stubs/mocks, which has a side effect of having to know how the subject under test is implemented.

This is a very brief summary — even risking being wrong because of terseness — but you can find more good references about this two decades old conundrum here, here and here

Testing in the Real World

So which one is right, Londoners or Detrotians? Both of them and neither of them at the same time . As I learnt throughout the almost five years I have been a professional Software Engineer, dogmatism will not take you very far in the real world, where projects should be delivered, product expectations are to be matched and you have bills to pay.

What you really need is to be able to take the best of both worlds and use it in your favor. Use it wisely.

We live in a world where everybody seems obsessed with ~almost~ perfect code coverage, while the problem of Redundant Coverage is rarely mentioned — it is not very easy to find online references discussing this. If you abuse tests, you may end up having a hard time when your requirements suddenly change.

In the end we are not paid to write tests, we are paid to solve other people's problems through code. Writing tests is expensive and does not add perceivable value to the clients/users. One can argue that there is value added by tests, but in my personal experience it is very hard to make non-technical people to buy that.

What we as Software Engineers should strive for is to write the minimum amount of tests that yields enough confidence in code quality and correctness — and "enough" is highly dependent on context.

Redux Testing According to the Docs

Redux is known to have an outstandingly good documentation. In fact this is true. There is not only API docs and some quick examples, as there are also some valuable best practices advice and even links to more in depth discussions regarding Redux and its ecosystem.

However, I believe that the "Writing Tests" section leaves something to be desired.

Testing Action Creators

That section in the docs start with action creators.

export function addTodo(text) {
  return {
    type: 'ADD_TODO',
    text
  }
}

Then we can test it like:

import * as actions from '../../actions/TodoActions'
import * as types from '../../constants/ActionTypes'

describe('actions', () => {
  it('should create an action to add a todo', () => {
    const text = 'Finish docs'
    const expectedAction = {
      type: types.ADD_TODO,
      text
    }
    expect(actions.addTodo(text)).toEqual(expectedAction)
  })
})

While the test is correct and passes just fine, the fundamental problem here is that it does not add much value. Your regular action creators should be very boring, almost declarative code. You do not need tests for that.

Furthermore, if you use helper libraries like redux-act or Redux's own @reduxjs/toolkit — which you should — then there is absolutely no reason at all to write tests for them, as your tests would be testing the helper libs themselves, which are already tested and, more important, are not even owned by you.

And since action creators can be very prolific in a real app, the amount of test they would require is huge.

But how can we know for sure our plain-old action creators do not contain silly errors like typos on them?

Bear with me. More on that later.

Testing reducers

In Redux, a reducer is a function which given a state and an action, should produce an entirely new state, without mutating the original one. Reducers are pure functions. Pure functions are like heaven to testers. It should be pretty straightforward, right?

The docs gives us the following example:

import { ADD_TODO } from '../constants/ActionTypes'

const initialState = [
  {
    text: 'Use Redux',
    completed: false,
    id: 0
  }
]

export default function todos(state = initialState, action) {
  switch (action.type) {
    case ADD_TODO:
      return [
        {
          id: state.reduce((maxId, todo) => Math.max(todo.id, maxId), -1) + 1,
          completed: false,
          text: action.text
        },
        ...state
      ]

    default:
      return state
  }
}

Then the test:

describe('todos reducer', () => {
  it('should return the initial state', () => {
    expect(reducer(undefined, {})).toEqual([
      {
        text: 'Use Redux',
        completed: false,
        id: 0
      }
    ])
  })

  it('should handle ADD_TODO', () => {
    expect(
      reducer([], {
        type: types.ADD_TODO,
        text: 'Run the tests'
      })
    ).toEqual([
      {
        text: 'Run the tests',
        completed: false,
        id: 0
      }
    ])

    expect(
      reducer(
        [
          {
            text: 'Use Redux',
            completed: false,
            id: 0
          }
        ],
        {
          type: types.ADD_TODO,
          text: 'Run the tests'
        }
      )
    ).toEqual([
      {
        text: 'Run the tests',
        completed: false,
        id: 1
      },
      {
        text: 'Use Redux',
        completed: false,
        id: 0
      }
    ])
  })
})

Let's just ignore the fact that the suggested test case "should handle ADD_TODO" is actually two tests bundled together — with might freakout some testing zealots. Even though in this case I believe it would be best to have different test cases — one for an empty list and the other for a list with some initial values — sometimes this is just fine.

The real issue with those tests is that they are tightly coupled with the internal structure of the reducer. More precisely, the tests above are coupled to the state object structure through those .toEqual() assertions.

While this example is rather simple, it is very common for the state of a given slice in Redux to change over time, as new requirements arrive and some unforeseen interactions need to occur. If we write tests like the ones above, they will soon become a maintenance nightmare. Any minimal change in the state structure would demand updating several test cases.

So how exactly are we supposed to write those tests?

Testing Redux the right way

Disclaimer: I am not saying this is the best or the only way of testing your Redux application, however I recently came to the conclusion that doing it the way I suggest bellow yields the best cost-benefit that I know of. If you happen to know a better way, please reach out to me through the comments, Twitter, e-mail or smoke signs.

Here is a popular folder structure for Redux applications that is very similar to the ones that can be found in many tutorials and even the official docs:

src
└── store
    ├── auth
    │   ├── actions.js
    │   ├── actionTypes.js
    │   └── reducer.js
    └── documents
        ├── actions.js
        ├── actionTypes.js
        └── reducer.js

If you are like me and like to have test files colocated with the source code, this structure encourages you to have the following:

src
└── store
    ├── auth
    │   ├── actions.js
    │   ├── actions.test.js
    │   ├── actionTypes.js
    │   ├── reducer.js
    │   └── reducer.test.js
    └── documents
        ├── actions.js
        ├── actions.test.js
        ├── actionTypes.js
        ├── reducer.js
        └── reducer.test.js

I have already left actionTypes tests out as those files are purely declarative. However, I already explained why action creators should be purely declarative, and therefore should not be tested as well. That leaves us with testing the only reducer itself, but that does not seem quite right.

The problem here is what we understand as being a "unit" in Redux. Most people tend to consider each of the individual files above as being themselves a unit. I believe this is a misconception. Actions, action types and reducers must be tightly coupled to each other in order to function properly. To me, it does not make sense to test those "components" in isolation. They all need to come together to form a slice (e.g.: auth and documents above), which I consider to be the smallest standalone piece in Redux architecture.

For that reason, I am found of the Ducks pattern, even though it has some caveats. Ducks authors advocates everything regarding a single slice (which they call a "duck") should be placed in a single file and follow a well-defined export structure.

I usually have a structure that looks like more this:

src
└── modules
    ├── auth
    │   ├── authSlice.js
    │   └── authSlice.test.js
    └── documents
        ├── documentsSlice.js
        └── documentsSlice.test.js

The idea now is to write the least amount of test possible, while having a good degree of confidence that a particular slice works as expected. The reason why Redux exists in the first place is to help us manipulate state, providing a single place for our application state to lie in.

In other words, the value Redux provides us is the ability to write and read state from a centralized place, called the store. Since Redux is based on the Flux Architecture, its regular flow is more or less like this:

The Flux Architecture by Eric Eliott on Medium

Redux Testing Strategy

In the end of the day, what we want to test is that we are correctly writing to — through dispatching actions — and reading from the store. The way we do that is by given an initial state, we dispatch some action to the store, let the reducer to its work and then after that we check the state to see if the changes we expect were made.

However, how can we do that while avoiding the pitfall of having the tests coupled with the state object structure? Simple. Always use selectors. Even those that would seem dumb.

Selectors are you slice public API for reading data. They can encapsulate your state internal structure and expose only the data your application needs, at the granularity it needs. You can also have computed data and optimize it through memoization.

Similarly, action creators are its public API for writing data.

Still confused? Let's try with some code using @reduxjs/toolkit:

Here is my auth slice:

import { createSlice, createSelector } from '@reduxjs/toolkit';

export const initialState = {
  userName: '',
  token: '',
};

const authSlice = createSlice({
  name: 'auth',
  initialState,
  reducers: {
    signIn(state, action) {
      const { token, userName } = action.payload;

      state.token = token;
      state.userName = userName;
    },
  },
});

export const { signIn } = authSlice.actions;

export default authSlice.reducer;

export const selectToken = state => state.auth.token;
export const selectUserName = state => state.auth.userName;
export const selectIsAuthenticated = createSelector([selectToken], token => token !== '');

Nothing really special about this file. I am using the createSlice helper, which saves me a lot of boilerplate code. The exports structure follows more or less the Ducks pattern, the main difference being that I don't explicitly export the action types, as they are defined in the type property of the action creators (e.g.: signIn.type returns 'auth/signIn').

Now the test suite implemented using jest:

import reducer, { initialState, signIn, selectToken, selectName, selectIsAuthenticated } from './authSlice';

describe('auth slice', () => {
  describe('reducer, actions and selectors', () => {
    it('should return the initial state on first run', () => {
      // Arrange
      const nextState = initialState;

      // Act
      const result = reducer(undefined, {});

      // Assert
      expect(result).toEqual(nextState);
    });

    it('should properly set the state when sign in is made', () => {
      // Arrange
      const data = {
        userName: 'John Doe',
        token: 'This is a valid token. Trust me!',
      };

      // Act
      const nextState = reducer(initialState, signIn(data));

      // Assert
      const rootState = { auth: nextState };
      expect(selectIsAuthenticated(rootState)).toEqual(true);
      expect(selectUserName(rootState)).toEqual(data.userName);
      expect(selectToken(rootState)).toEqual(data.token);
    });
  });
});

The first test case ('should return the initial state on first run') is only there to ensure there is no problem in the definition of the slice file. Notice that I am using the .toEqual() assertion I said you should not. However, in this case, since the assertion is against the constant initialState and there are no mutations, whenever the state shape changes, initialState changes together, so this test would automatically be "fixed".

The second test case is what we are interested in here. From the initial state, we "dispatch" a signIn action with the expected payload. Then we check if the produced state is what we expected. However we do that exclusively using selectors. This way our test is more decoupled from the implementation

If your slice grows bigger, by using selectors when testing state transitions, you gain yet another advantage: you could use only those selectors that are affected by the action you dispatched and can ignore everything else. Were you asserting against the full slice state tree, you would still need to declare those unrelated state properties in the assertion.

An observant reader might have noticed that this style of testing resembles more the one derived from Detroit School. There are no mocks, stubs, spies or whatever. Since reducers are simply pure functions, there is no point in using those.

However, this slice is rather too simple. Authentication is usually tied to some backend service, which means we have to manage the communication between the latter and our application, that is, we have do handle side-effects as well as the loading state. Things start to get more complicated.

Testing a More Realistic Slice

The first step is to split our signIn action into three new: signInStart, signInSuccess and signInFailure. The names should be self-explanatory. After that, our state needs to handle the loading state and an eventual error.

Here is some code with those changes:

import { createSlice, createSelector } from '@reduxjs/toolkit';

export const initialState = {
  isLoading: false,
  user: {
    userName: '',
    token: '',
  },
  error: null,
};

const authSlice = createSlice({
  name: 'auth',
  initialState,
  reducers: {
    signInStart(state, action) {
      state.isLoading = true;
      state.error = null;
    },
    signInSuccess(state, action) {
      const { token, userName } = action.payload;

      state.user = { token, userName };
      state.isLoading = false;
      state.error = null;
    },
    signInFailure(state, action) {
      const { error } = action.payload;

      state.error = error;
      state.user = {
        userName: '',
        token: '',
      };
      state.isLoading = false;
    },
  },
});

export const { signInStart, signInSuccess, signInFailure } = authSlice.actions;

export default authSlice.reducer;

export const selectToken = state => state.auth.user.token;
export const selectUserName = state => state.auth.user.userName;
export const selectError = state => state.auth.error;
export const selectIsLoading = state => state.auth.isLoading;
export const selectIsAuthenticated = createSelector([selectToken], token => token !== '');

The first thing you might notice is that our state shape changed. We nested userName and token in a user property. Had we not created selectors, this would break all the tests and code that depends on this slice. However, since we did have the selectors, the only changes we need to do are in the selectToken and selectUserName.

Notice that our test suite is completely broken now, but that is because we fundamentally changed the slice. It is not hard to get it fixed though:

describe('auth slice', () => {
  describe('reducer, actions and selectors', () => {
    it('should return the initial state on first run', () => {
      // Arrange
      const nextState = initialState;

      // Act
      const result = reducer(undefined, {});

      // Assert
      expect(result).toEqual(nextState);
    });

    it('should properly set loading and error state when a sign in request is made', () => {
      // Arrange

      // Act
      const nextState = reducer(initialState, signInStart());

      // Assert
      const rootState = { auth: nextState };
      expect(selectIsAuthenticated(rootState)).toEqual(false);
      expect(selectIsLoading(rootState)).toEqual(true);
      expect(selectError(rootState)).toEqual(null);
    });

    it('should properly set loading, error and user information when a sign in request succeeds', () => {
      // Arrange
      const payload = { token: 'this is a token', userName: 'John Doe' };

      // Act
      const nextState = reducer(initialState, signInSuccess(payload));

      // Assert
      const rootState = { auth: nextState };
      expect(selectIsAuthenticated(rootState)).toEqual(true);
      expect(selectToken(rootState)).toEqual(payload.token);
      expect(selectUserName(rootState)).toEqual(payload.userName);
      expect(selectIsLoading(rootState)).toEqual(false);
      expect(selectError(rootState)).toEqual(null);
    });

    it('should properly set loading, error and remove user information when sign in request fails', () => {
      // Arrange
      const error = new Error('Incorrect password');

      // Act
      const nextState = reducer(initialState, signInFailure({ error: error.message }));

      // Assert
      const rootState = { auth: nextState };
      expect(selectIsAuthenticated(rootState)).toEqual(false);
      expect(selectToken(rootState)).toEqual('');
      expect(selectUserName(rootState)).toEqual('');
      expect(selectIsLoading(rootState)).toEqual(false);
      expect(selectError(rootState)).toEqual(error.message);
    });
  });
});

Notice that signInStart has less assertions regarding the new state, because current userName and token do not matter to it. Everything else is much in line with what we have discussed so far.

There is another subtlety that might go unnoticed. Even though the main focus of the tests is the reducer, they end up testing the action creators as well. Those silly errors like typos will get caught here, so we do not need to write a separate suite of tests to prevent them from happening.

The same thing goes for selectors too. Plain selectors are purely declarative code. Memoized selectors for derived data created with createSelector from reselect should not be tested as well. Errors will get caught in the reducer test.

For example, if we had forgotten to change selectUserName and selectToken after refactoring the state shape and left them like this:

// should be state.auth.user.token
export const selectToken = state => state.auth.token;

// should be state.auth.user.userName
export const selectUserName = state => state.auth.userName;

In that case, all test cases above would fail.

Testing Side-Effects

We are getting there, but our slice is not complete yet. It lacks the part that orchestrates the sign in flow and communicates with the backend service API.

Redux itself deliberately does not handle side-effects. In order to be able to do that, you need a Redux Middleware that will handle that for you. While you can pick your own poison, @reduxjs/toolkit already ships with redux-thunk, so that is what we are going to use.

In this case, the Redux docs actually has a really good example, so I basically took it and adapted to our use case.

In our authSlice.js, we simply add:

// ...
import api from '../../api';

// ...
export const signIn = ({ email, password }) => async dispatch => {
  try {
    dispatch(signInStart());
    const { token, userName } = await api.signIn({
      email,
      password,
    });
    dispatch(signInSuccess({ token, userName }));
  } catch (error) {
    dispatch(signInFailure({ error }));
  }
};

Notice that the signIn function is almost like an action creator, however, instead of returning the action object, it returns a function which receives the dispatch function as parameter. This is the "action" that will be triggered when the user clicks the "Sign In" button in our application.

This means that functions like signIn are very important to the application, therefore, they should be tested. However, how can we test this in isolation from the api module? Enter Mocks and Stubs.

Since this is basically an orchestration component, we are not interested in the visible effets it has. Instead, we are interested in the actions that were dispatched from within the thunk according to the response from the API.

So we can change the test file like this:

import configureMockStore from 'redux-mock-store';
import thunk from 'redux-thunk';
// ...
import api from '../../api';

jest.mock('../../api');

const mockStore = configureMockStore([thunk]);

describe('thunks', () => {
    it('creates both signInStart and signInSuccess when sign in succeeds', async () => {
      // Arrange
      const requestPayload = {
        email: 'john.doe@example.com',
        password: 'very secret',
      };
      const responsePayload = {
        token: 'this is a token',
        userName: 'John Doe',
      };
      const store = mockStore(initialState);
      api.signIn.mockResolvedValueOnce(responsePayload);

      // Act
      await store.dispatch(signIn(requestPayload));

      // Assert
      const expectedActions = [signInStart(), signInSuccess(responsePayload)];
      expect(store.getActions()).toEqual(expectedActions);
    });

    it('creates both signInStart and signInFailure when sign in fails', async () => {
      // Arrange
      const requestPayload = {
        email: 'john.doe@example.com',
        password: 'wrong passoword',
      };
      const responseError = new Error('Invalid credentials');
      const store = mockStore(initialState);
      api.signIn.mockRejectedValueOnce(responseError);

      // Act
      await store.dispatch(signIn(requestPayload));

      // Assert
      const expectedActions = [signInStart(), signInFailure({ error: responseError })];
      expect(store.getActions()).toEqual(expectedActions);
    });
  });

So unlike reducers, which are easier to test with Detroit School methodology, we leverage London School style to test our thunks, because that is what makes sense.

Because we are testing implementation details, whenever code changes, our tests must reflect that. In a real world app, after a sucessful signin, you probably want to redirect the user somewhere. If we were using something like connected-react-router, we would end up with a code like this:

+import { push } from 'connected-react-router';
 // ...
 import api from '../../api';

 // ...
     const { token, userName } = await api.signIn({
       email,
       password,
     });
     dispatch(signInSuccess({ token, userName }));
+    dispatch(push('/'));
   } catch (error) {
     dispatch(signInFailure({ error }));
   }
 // ...

Then we update the assert part of our test case:

+import { push } from 'connected-react-router';
 // ...

 // Assert
 const expectedActions = [
   signInStart(),
   signInSuccess(responsePayload),
+  push('/')
 ];
 expect(store.getActions()).toEqual(expectedActions);
 // ...

This is often a criticism against redux-thunk, but if you even so decided to use it, that is a trade-off you have to deal with.

Conclusion

When it comes to the real world, there is no single best approach for writing tests. We can and should leverage both Detroit and London styles to effectively test your applications.

For components which behave like pure functions, that is, given some input, produce some deterministic output, Detroit style shines. Our tests can be a little bit more coarse-grained, as having perfect isolation does not add much value to them. Where exactly we should draw the line? Like most good questions, the answer is "It depends".

In Redux, I have come to the conclusion that a slice is the smallest standalone unit that exists. It makes little to no sense writing isolated tests for their sub-components, like reducers, action creators and selectors. We test them together. If any of them is broken, the tests will show us and it will be easy to find out which one.

On the other hand, when our components exists solely for orchestration purposes, then London style tests are the way to go. Since we are testing implementation details, tests should be as fine-grained as they get, leveraging mocks, stubs, spies and whatever else we need. However, this comes with a burden of harder maintainability.

When using redux-thunk, what we should test is that our thunk is dispatching the appropriate actions in the same sequence we would expect. Helpers like redux-mock-store make the task easier for us, as it expose more of the internal state of the store than Redux native store.

T-th-tha-that's a-all f-fo-fo-folks!

DEV Community: Henrique Barcelos

Flexbox Gotchas Reloaded

Intro

TL;DR

Flexbox all the things

The issue

Halt! DuckDuckGo time

You've applied min-width: 0 and the item still doesn't shrink?

Nested Flex Containers

TDD made simple with Mocha and Chai

Intro

First, the problem and its solution

Expected behavior

A Mocha to go, please!

Installation

Writing tests

Running tests

Enter Chai

Installation

Usage

Oh, tests! Oh, dreaded tests!

Testing queue creation

Testing adding elements

Halt! Refactor time

Taking some shortcuts

Testing removing elements

Testing edge cases

Conclusion

Give your logs more context - Part 2

Building a contextual logger

Give your logs more context - Part 1

Henrique Barcelos ・ Feb 6 '20 ・ 8 min read

TL;DR

Intro

What do we want to achieve?

How will we write our code?

hbarcelos / give-your-logs-more-context

Wrapper on top of pino which provides integration with cls-hooked for better context in log messages

Give your logs more context

Requirements

Setup

Implementation steps

Initial setup

Ensuring methods for log levels

Keeping context on logger method call

Adding CLS awareness

Preventing CLS context sharing between instances

Applying CLS context to logs

Propagating changes to child loggers

Further improvements

Integrating with our web application

Outro

Give your logs more context - Part 1

How to make sense out of your Node.js web app logs

Use log levels wisely

Rules of thumb

Embrace request IDs

Meet Continuation Local Storage

Further improvements

A better approach for testing your Redux code

TL;DR

Vanilla Redux

Redux + redux-thunk

Intro

The Problem with Tests

Schools of Testing

Testing in the Real World

Redux Testing According to the Docs

Testing Action Creators

Testing reducers

Testing Redux the right way

Redux Testing Strategy

Testing a More Realistic Slice

Testing Side-Effects

Conclusion

Redux + `redux-thunk`