DEV Community: Tony Wallace

Capture Groups in Regular Expressions

Tony Wallace — Mon, 17 Apr 2023 13:00:00 +0000

Regular expressions (regex) are powerful tools for pattern matching and string parsing. While the syntax can sometimes be difficult to understand, it can make many tasks more efficient. In this post, I'm going to demonstrate a feature of regular expressions called capture groups.

A few notes before we get started:

JavaScript will be used for the examples in this post. The principles of regular expressions should be the same in other languages but some usage details may differ. If you aren't using JavaScript, refer to your language's documentation for specifics.
Any discussion of overall efficiency with regard to software has to take development time and maintenance into account along with the runtime performance of the code. If runtime performance is your ultimate priority, regular expressions may not always be the best choice, because their performance can vary greatly depending on the characteristics of the input, the regular expression pattern, and the regular expression engine. In terms of development time and maintenance, regular expressions can reduce the amount and complexity of code necessary to accomplish some tasks. This is often a balancing act, and you can test your solutions with a benchmarking app like jsbench.me to be sure they align with your priorities. I also suggest writing unit tests for critical parsers to guard against regressions during future maintenance.

What is a capture group?

A capture group is a pattern within a regular expression that will be included in the result of calling RegExp.prototype.exec, String.prototype.match or String.prototype.matchAll.

Lets start with some basic pattern matching. Say we want to parse a substring containing one or more digits from a pathname like /items/42. The basic regular expression for that would be /\/\d+/g. This pattern will search a string for all matches (the g or global flag) of a forward slash (\/ — the preceding backslash is an escape) followed by one or more digits (\d+). This works very well if you want to test that a string contains that pattern:

const re = /\/\d+/g;

re.test('/items/42'); // true
re.test('/items/42/1'); // true
re.test('/items/42/options/1'); // true
re.test('/items/new'); // false

RegExp.prototype.test returns true for any string that contains a forward slash followed by one or more digits, and false for any other string. Now let's try using RegExp.prototype.exec to get some information about the match:

re.exec('/items/42'); // [ '/42', index: 6, input: '/items/42' ]
re.exec('/items/42/1'); // [ '/1', index: 9, input: '/items/42/1' ]
re.exec('/items/42/options/1'); // [ '/1', index: 17, input: '/items/42/options/1' ]
re.exec('/items/new'); // null

RegExp.prototype.exec returns an array unless the string doesn't match the regex, in which case it returns null. The array for each match contains one item, which is the string that matched. Additional properties attached to the array describe the index at which the string appears and the original input string. (You can access these properties via subscripting or dot notation, just like the properties of any object.) In cases where the input string contains more than one occurrence of the pattern /\/\d+/ (e.g. '/items/42/options/1'), note that only the last occurrence has been matched. This is due to the use of the g flag on the regex. You may omit the g flag if you only care about the first match, but we'll benefit from it in a moment.

Now let's try String.prototype.match:

'/items/42'.match(re); // [ '/42' ]
'/items/42/1'.match(re); // [ '/42', '/1' ]
'/items/42/options/1'.match(re); // [ '/42', '/1' ]

These results include every occurrence of the pattern in each string. If we had not used the g flag on the pattern, we would have gotten the same result as calling RegExp.prototype.exec without the g flag — only the first match would be included, and the array would have additional properties for index and input. You can obtain the best of both approaches by using String.prototype.matchAll, which will return the input and index for every match:

Array.from('/items/42/options/1'.matchAll(re));

// [
//   [ '/42', index: 6, input: '/items/42/options/1' ],
//   [ '/1', index: 9, input: '/items/42/options/1' ]
// ]

There are tho things that are important to note:

We're using Array.from to coerce the result into an array. The return value of String.prototype.matchAll is an iterator but in many cases it's easier to work with an array.
String.prototype.matchAll requires a global regex, meaning that you must include the g flag.

We could actually work with this result if we wanted to hack around a bit. The following code will extract the digits from each match by replacing the forward slash with an empty string:

const matches = Array.from('/items/42/options/1'.matchAll(re));

// [
//   [ '/42', index: 6, input: '/items/42/options/1' ],
//   [ '/1', index: 9, input: '/items/42/options/1' ]
// ]

const params = matches.map((match) => match[0].replace('/', '')); // [ '42', '1' ]

This accomplishes our basic task of extracting the digits, but isn't particularly clean. Fortunately, there's a better way. We can use a capture group to isolate the digits. The capture group is indicated by parentheses surrounding the part of the pattern you with to capture. In this case, one or more digits as specified by \d+:

const re = `/\/(\d+)/g`;

If we use String.prototype.matchAll with this regex, we get a second array element for each match, which contains the digits we captured without the forward slash (because the forward slash was not included in the capture group). We can extract the digits simply by accessing the second element in the array for each match:

const matches = Array.from('/items/42/options/1'.matchAll(re));

// [
//   [ '/42', '42', index: 6, input: '/items/42/options/1' ],
//   [ '/1', '1', index: 9, input: '/items/42/options/1' ]
// ]

const params = matches.map((match) => match[1]); // [ '42', '1' ]

Capture groups give you an efficient way to extract substrings that match certain patterns from a larger string. There are many cases in which this will be sufficient, but sometimes the results can be awkward to work with. Maybe you need to relate the values extracted in the previous example to property names, where the first value should be called itemId and the second optionId. This would require more code to create an object containing these properties. One way to do this is to reduce the matches into an object, in which each value is keyed by the property name:

const matches = Array.from('/items/42/options/1'.matchAll(re));
const propertyNames = ['itemId', 'optionId'];

const properties = matches.reduce((obj, match, idx) => {
  const name = propertyNames[idx];
  const value = match[1];
  return { ...obj, [name]: value };
}, {});

// { itemId: '42', optionId: '1' }

Again, this works but isn't very graceful. Relying on code to relate the matches to the corresponding property names could be fragile in some cases, depending on the specific regex and string that are being parsed. This is where named capture groups may help.

What is a named capture group?

A named capture group is similar to a regular capture group, but the syntax allows us to embed a name for each capture group in the regex. The syntax for a named capture group with the name id and pattern \d+ will be (?<id>\d+). The name is placed within angle brackets, preceded by a question mark, and followed by the pattern. If we apply this to our previous example, we can obtain the named values in the groups property on each match:

const re = `/\/(?<id>\d+)/g`;

const matches = Array.from('/items/42/options/1'.matchAll(re));

// [
//   [ '/42', '42', index: 6, input: '/items/42/options/1', groups: { id: '42' } ],
//   [ '/1', '1', index: 9, input: '/items/42/options/1', groups: { id: '1' } ]
// ]

const params = matches.map((match) => match.groups.id); // [ '42', '1' ]

Now, you might have noticed that this doesn't solve our problem of needing extra code to resolve the values to property names, because the values are still distributed across separate matches and all have the same name. There's still no direct route to convert them to the object we want, which is { itemId: '42', optionId: '1' }. We can change that by redesigning our regular expression. The following example uses a much more specific regex that describes the complete path we expect to match, including named capture groups for itemId and optionId:

const re = /^\/items\/(?<itemId>\d+)\/options\/(?<optionId>\d+)$/g;

const matches = Array.from('/items/42/options/1'.matchAll(re));

// [
//   [
//     '/items/42/options/1',
//     '42',
//     '1',
//     index: 0,
//     input: '/items/42/options/1',
//     groups: { itemId: '42', optionId: '1' }
//   ]
// ]

We can obtain the named values simply by accessing the groups property of the first match:

const [ match ] = Array.from('/items/42/options/1'.matchAll(re));

if (match) {
  const { groups } = match;
  // Work with the groups
}

(It's safe to assume that we can take the first match, if there is one, because the regex is anchored to the beginning and end of the string with ^ and $.)

The benefit of using named capture groups in this scenario is that names can be assigned to the values at the same time as they are parsed.

Choose your own adventure

In the previous example, we gained the ability to parse named values from a string in one shot, with no code other than what was necessary to execute the regex and access the result. We also lost the ability to use the generic regular expression we started with, because of the need to assign a unique name to each value. In reality neither approach is perfect for all situations. Unnamed capture groups enable you to use generic regex patterns because you don't have to worry about providing a unique name for each pattern, but traversing the results can be messy and error prone. Named capture groups can make parsing much cleaner, but only if the strings you have to parse have a consistent structure. Regular expressions that contain named capture groups may also be harder to understand. So which should you choose? Asking a series of questions may help:

Would named capture groups provide a real benefit?
- If yes, go to 2.
- If no, go to 3.
Do the strings you need to parse have a consistent enough structure to make named capture groups feasible?
- If yes, go to 4.
- If no, go to 5.
Regular expressions that use unnamed capture groups are usually simpler and easier to understand, so it might be best to use them even if named capture groups might work in your scenario. Evaluate both options.
Sounds like you might have found a good use case for named capture groups! Go to 6.
Is there a reasonable workaround for the consistency problem? For example, could you design a regex that would cover all your input cases? (Without creating a nightmare for yourself and others?)
- If yes, go to 6.
- If no, go to 7.
Time to experiment! Test your workaround in comparison regular capture groups and any other approaches that you have in mind, and choose whichever one works best.
Named capture groups are probably out of the question, so see if unnamed capture groups can do the job, or if you need to find a different solution.

Combining and nesting capture groups

Capture groups may be combined and nested. Consider the following regular expression:

const re = /^\/items\/(?<itemId>\d+)(\/options\/(?<optionId>\d+))?$/g

This is similar to our previous regex, but a capture group has been added around the /options subpath: (\/options\/(?<optionId>\d+))?. The question mark after the closing parenthesis makes the capture group optional, which means that this regex will match the path with or without the /options subpath. (E.g. /items/42/options/1 or /items/42.)

If we use this regex to match the full path, we get the following result:

const [ match ] = Array.from('/items/42/options/1'.matchAll(re));

// [
//   '/items/42/options/1',
//   '42',
//   '/options/1',
//   '1',
//   index: 0,
//   input: '/items/42/options/1',
//   groups: { itemId: '42', optionId: '1' }
// ]

The array contains four elements:

The match for the full pattern ('/items/42/options/1')
The match for the itemId named capture group ('42')
The match for the unnamed capture group around the /options subpath ('/options/1')
The match for the optionId named capture group ('1')

The groups property contains the itemId and optionId values that were parsed from the path.

Now let's match the path without the /options subpath:

const [ match ] = Array.from('/items/42'.matchAll(re));

// [
//   '/items/42',
//   '42',
//   undefined,
//   undefined,
//   index: 0,
//   input: '/items/42',
//   groups: { itemId: '42', optionId: undefined }
// ]

Once again, the array contains four elements for the full pattern match, itemId, /options subpath and optionId, but the /options subpath and optionId are both undefined because they're optional and were not present in the input. The groups property contains the itemId and optionId values, but optionId is similarly undefined.

We could also use a named capture group around the /options subpath to include it in the groups:

const re = /^\/items\/(?<itemId>\d+)(?<subpath>\/options\/(?<optionId>\d+))?$/g

const [ match ] = Array.from('/items/42/options/1'.matchAll(re));

// [
//   '/items/42/options/1',
//   '42',
//   '/options/1',
//   '1',
//   index: 0,
//   input: '/items/42/options/1',
//   groups: { itemId: '42', subpath: '/options/1', optionId: '1' }
// ]

const [ match ] = Array.from('/items/42'.matchAll(re));

// [
//   '/items/42',
//   '42',
//   undefined,
//   undefined,
//   index: 0,
//   input: '/items/42',
//   groups: { itemId: '42', subpath: undefined, optionId: undefined }
// ]

Again, the subpath group is undefined in the second example because it wasn't present in the input.

The nested capture groups in these examples have enabled the regular expression to match variations on a path and parse the itemId and optionId whether the /options subpath is present or not, all in just a few lines of code. Try applying these techniques to simple string parsing tasks so you can get a feel for how they work.

Summary

In this post, we've learned how capture groups can make many string parsing tasks easier, but remember that regular expressions aren't the right solution. Some use cases that could be awkward or slow with regular expressions may be satisfied perfectly well by manipulating strings with code. Others may call for a more robust solution like pegjs. As always, experiment with all the tools you have at your disposal and see what works best for your application.

Automated Testing with Playwright

Tony Wallace — Tue, 28 Mar 2023 13:00:00 +0000

There are four basic types of software test that can be automated.

End-to-end testing of user flows with live data (integration testing)
End-to-end testing of user flows with mock data (isolated user interface testing)
Isolated testing of individual components
Unit testing

This article will focus on end-to-end testing, but first a few notes about the other types of tests.

Playwright has experimental support for testing components in React, Vue and Svelte. I have not yet been able to integrate this into one of RedBit's React projects because Playwright uses a different bundler than us. (Playwright uses Vite while we use Webpack.) Many of our components rely on specific Webpack configuration and some custom plugins that can't easily be replicated in Vite. While I'm sure this is a surmountable problem, I'm not sure it's worth the effort. It would likely take less time to build out pages to render components that would be testable with the regular Playwright APIs, and could also serve as a reference library for developers.

Playwright isn't first and foremost a unit test runner, so I won't discuss it in that context. RedBit uses Jest or unit testing in web projects. Jest uses a similar assertion syntax to Playwright, which helps reduce cognitive overhead for developers.

End-to-end user flow testing

End-to-end testing of user flows aims to simulate the actions a user would perform while using an app, and verify that those actions have the expected outcomes. This is Playwright's main purpose.

Automating a user flow

Writing automated end-to-end tests normally involves determining the sequence of actions that a user would perform in the app, translating them manually to code, then adding assertions to verify that the expected actions were actually performed. For example, you might navigate to a certain page in your app and verify that the browser's location is set to the expected url, then simulate a click on a link and verify that the browser's location has changed to the link's url. At points along the way you might want to verify that certain messages or other components are visible on screen.

Playwright provides a test generator that takes a lot of the drudgery out of automating tests for long user flows. It launches your app in a Chromium instance alongside a second process that records all the actions you perform in the app. You navigate through a user flow in your app and the test generator translates your actions to code. The test generator will also add some basic assertions, like testing that the browser location updates to the expected url when you click a link. You can then copy the test code to your project and add other assertions manually.

The test generator worked quite well for the flows that I automated, except that it failed to capture the browser back button. That resulted in tests that would fail unless modified to restore the missing navigation actions. Even if the code output by the test generator needs some work, it's still a win in my opinion. The effort necessary to fix the tests will likely be far less than the effort that would have been required to write them from scratch.

Testing with mock data

When you run the test generator, it launches your app, which is presumably backed by an API or some other data source. The data source might be a production environment (but hopefully not), or a remote test environemnt, or maybe a dev environment on your local computer. Either way, you're testing with live data. The problem with live data is that it's often subject to change, and when it changes your tests will probably fail. Consider the following scenario:

Navigate to a page /products that renders a list of products.
Click the first item to navigate to /products/<id>, which displays details about an individual product. In this example, <id> represents an id property that is assigned to the product's database record.
Assert that the details page url contains the correct id property.

The Playwright test generator will write code that performs these actions based on rendered data, which will look more like this:

Navigate to /products.
Click the link that contains the text "Cuisinart Food Processor".
Assert that the details page url is /products/34.

    test('Products list and detail navigation flow', async ({ page }) => {
      // Navigate to the products page:
      await page.goto('/products');
      await expect(page).toHaveURL('/products');

      // Click the "Cuisinart Food Processor" link:
      await page.getByRole('link', { name: 'Cuisinart Food Processor' }).click();
      await expect(page).toHaveURL('/products/34');

      // Navigate back to the products page:
      await page.goBack();
      await expect(page).toHaveURL('/products');
    });

This will only work as long as the first product in the list is "Cuisinart Food Processor" with an id of 34. If the list is updated and another product is now first in the list, or if you test in another environment where the products have different ids, the test will fail. There are two solutions to this.

Mocking API responses

The easiest solution is to fulfill API requests with mock data. Playwright provides a way to do this simply and cleanly by intercepting requests to a particular route:

    test('Products list and detail navigation flow', async ({ page }) => {
      // Fulfill the products list API request with mock data.
      await page.route('/api/products', (route) => {
        return route.fulfill({
          status: 200,
          body: JSON.stringify([
            { id: 34, name: 'Cuisinart Food Processor' },
            { id: 75, name: 'Vitamix Blender' },
          ]),
        });
      });

      // Navigate to the products page:
      await page.goto('/products');
      await expect(page).toHaveURL('/products');

      // Click the "Cuisinart Food Processor" link:
      await page.getByRole('link', { name: 'Cuisinart Food Processor' }).click();
      await expect(page).toHaveURL('/products/34');

      // Navigate back to the products page:
      await page.goBack();
      await expect(page).toHaveURL('/products');
    });

In this example, a request to /api/products will be fullfilled with the JSON-serialized test data. (It is assumed that we're mocking an API that sends JSON responses, but you can replace the test data with whatever is appropriate for your application.)

With reliable test data, you can be assured that the first item in the list of products will never change, unless you change it. As long as the app's behaviour remains the same, your tests will always pass. This is not to say that you should never test with live data. If you're running an integration test, you may need to verify a complex series of actions during which data must be written to, read from, and deleted from a database. However, there will be many situations in which you will only be concerned with testing one part of the system (e.g. the user interface) and you should be able to run your tests in isolation.

Testing based on structure, not content

Returning to the test case we looked at before, we can see that it relies on specific content to locate the first product on the page:

Navigate to /products.
Click the link that contains the text "Cuisinart Food Processor".
Assert that the details page url is /products/34.

    test('Products list and detail navigation flow', async ({ page }) => {
      // Navigate to the products page:
      await page.goto('/products');
      await expect(page).toHaveURL('/products');

      // Click the "Cuisinart Food Processor" link:
      await page.getByRole('link', { name: 'Cuisinart Food Processor' }).click();
      await expect(page).toHaveURL('/products/34');

      // Navigate back to the list:
      await page.goBack();
      await expect(page).toHaveURL('/products');
    });

Notice that this test doesn't care where the "Cuisinart Food Processor" link is rendered. We're expecting it to be in a list of products but the test doesn't verify that. It could be anywhere on the page. That may or may not be important to you, but it's worth pointing out.

We could rewrite this sequence to depend on page structure, instead:

Navigate to /products.
Extract the detail page url from the first link in the products list.
Click that same link.
Assert that the details page url matches the url from step 2.

A test written this way would be content-agnostic and target elements precisely:

    test('Products list and detail navigation flow', async ({ page }) => {
      // Navigate to the products page:
      await page.goto('/products');
      await expect(page).toHaveURL('/products');

      // Get the first link in the products list and extract the detail page url:
      const link = page.locator('.ul.products > li > a').nth(0);
      const url = await link.evaluate((node) => node.getAttribute('href'));

      // Navigate to the product detail page:
      await link.click();
      await expect(page).toHaveURL(url);

      // Navigate back to the products list:
      await page.goBack();
      await expect(page).toHaveURL('/products');
    });

The trade off is that you need technical knowledge of your app to write tests based on structure. This approach may not be feasible depending on who in your organization will be responsible for testing. It isn't a replacement for reliable test data, but provides another way to make your tests more accurate and resilient.

Testing API requests

You may have cases in which it is important to verify that your app makes specific API requests. For example, you might want to test that a new API request is made when the user selects a filter, and that the request is configured with the filter they selected. Playwright allows you to wait for a request and obtain information about it. The following example verifies that a GET request is made for the /products list with certain pagination and sort params:

    test('A request is made for the first page of products in descending order of creation', async ({ page }) => {
      const request = await page.waitForRequest('/products**');
      await page.waitForLoadState('networkidle');

      // Verify that the request was configured correctly:
      const url = new URL(request.url);

      // Expect a GET request to /products?offset=0&limit=10&orderBy=createdAt&order=desc
      await expect(request.method).toEqual('GET');
      await expect(url.searchParams.get('offset')).toEqual('0');
      await expect(url.searchParams.get('limit')).toEqual('10');
      await expect(url.searchParams.get('orderBy')).toEqual('createdAt');
      await expect(url.searchParams.get('order')).toEqual('desc');
    });

If the request is made with any other method, or any other values for the offset, limit, orderBy and order params, the request will fail.

Note: The wildcard (**) at the end of the url tells Playwright to match any request for /products regardless of the query params. Without it, the request would only be matched if it was made without any query params.

Testing rendering accuracy

If you have a reliable and stable source of test data (see Mocking API responses) it's possible to test that your data was rendered according to requirements. The process is as follows.

For each item in your test data:

Prepare the properties of the test data as you would expect them to have been rendered. For example, if you have a number formatter that renders a number (25.00) as a currency string ('$25.00'), apply it to the number. (If your user interface is localized, make sure your tests use the same locale as the app. If you're being thorough, you may want to run separate tests for each locale.)
Locate the element that corresponds to each property in the DOM tree and extract the rendered value.
Assert that the rendered values are equal to the formatted values.

    import { productsTestData } from './test-data';
    import { formatCurrency } from './utilities/currency';

    test('Products list renders as expected', async ({ page }) => {
      await page.goto('/products');
      await expect(page).toHaveURL('/products');

      for (let i = 0; i < productsTestData.length; i++) {
        const product = productsTestData[i];

        // Format the expected values.
        const expectedLink = `/products/${product.id}`;
        const expectedName = product.name;
        const expectedPrice = formatCurrency(product.price);

        // Get the DOM node that contains the product.
        const node = await page.locator('.ul.products > li').nth(i).evaluate((node) => node);

        // Get the rendered link href.
        const renderedLink = node.querySelector('a').getAttribute('href');

        // Get the rendered product name and price.
        // Trim the values to ignore any whitespace introducted during rendering.
        const renderedName = node.querySelector('.product-name').textContent.trim();
        const renderedPrice = node.querySelector('.product-price').textContent.trim();

        // Assert that the rendered values equal the expected values.
        expect(renderedLink).toEqual(expectedLink);
        expect(renderedName).toEqual(expectedName);
        expect(renderedPrice).toEqual(expectedPrice);
      }
    });

How much should you test?

The complexity of your end-to-end tests will more or less reflect your application's complexity. The more information you render, the more you have to test. The example above only expects the link, product name and product price to be rendered as specific strings. It doesn't test that the layout and styling are correct, or even that the elements are visible. It's possible to write more comprehensive tests, but doing so requires more development time. Your tests will likely be invalidated more often, which will result in more failures. There are costs to consider and questions to ask:

How mature is your product? Is your user interface subject to frequent design changes or is it stable?
Can you afford the impact that more complex tests and more frequent failures will have on your dev team's velocity?
At what point does the cumulative cost of test development and maintenance exceed the cost of human QA testing?
At what point does the added time pressure cause developers to give up and remove failing tests instead of fixing them, rendering your investment pointless?

There are no right answers to these questions. The testing strategy you choose should depend on your organization's priorities and may evolve over time. An early-stage startup might prioritize high-level testing of user flows and limit rendering tests to the critical path. As the organization matures and their capacity improves, they might start to add tests for other parts of their product, or make existing tests more comprehensive, or both.

Remember that the goals of automated testing are to reduce human time and effort, and to improve consistency. The highest value automated tests are those that have to be run most often and require the most attention to detail – complex flows on your application's critical path. Start by identifying opportunities to reduce labour in those areas and gradually increase yor test coverage from there.

Small Viewport Units in CSS

Tony Wallace — Mon, 09 Jan 2023 14:00:00 +0000

In this post, I'll give a short demonstration of how small viewport units in CSS can be used to build a grid that keeps a certain number of elements above the fold on all devices.

Introduction to Viewport Units

Let's begin with a quick primer on CSS viewport units. You may already be familiar with the vh and vw units, which have been around for a while and have broad browser support. These units specify dimensions as a percentage of the viewport height (vh) or width (vw). These are distinct from percentage (%) units, which specify an element's dimensions as a percentage of one of its ancestor's dimensions. For example, 100vh is 100% of the viewport height while 5vw is 5% of the viewport width. There are additional units like vmin and vmax, which express dimensions as a percentage of the minimum and maximum viewport size, respectively. For example, 50vmax will be equal to 50vh in portrait orientation or 50vw in landscape orientation.

The trouble with wh and vw is that the full dimensions of the viewport aren't always available to your application, especially on mobile devices. Browser controls like the location bar will occupy some of that space at least some of the time, which means that an element whose height is 100vh will overflow the available space and some of its content will be hidden when the browser controls are visible. There is a good explanation of this problem at web.dev.

If you're trying to keep certain content above the fold, you can’t reliably use 100vh as an indicator of how much space will be available before the user has to scroll. You also can’t reliably predict the amount of space that will be occupied by browser controls because it varies by operating system and browser. (Otherwise you could probably get away with a rule like height: calc(100vh - <location bar height in px>), but please don't do that. It's an invitation to have your site broken by a future OS or browser update.)

This problem can be solved by taking advantage of three new types of units:

The small viewport (svh/svw/svmin/svmax): These units give you the height, width, minimum dimension and maximum dimension of the viewport when the browser chrome is visible (e.g. on page load or after scrolling to the top of the page).
The large viewport (lvh/lvw/lvmin/lvmax): These units give you the height, width, minimum dimension and maximum dimension of the viewport when the browser chrome is hidden (e.g. after scrolling down the page).
The dynamic viewport (dvh/dvw/dvmin/dvmax): These units give you the small or large viewport dynamically, depending on whether the browser chrome is currently visible.

These units are relatively new to CSS but have, at the time of writing, been implemented in Chrome, Edge, Firefox and Safari. You can check the current support for this feature on caniuse. (This link is for dynamic viewport units, but browser support should be the same for the small and large viewport units.) If you need to support older browsers that haven't implemented these units, consider using fallback rules. This strategy will allow you to provide the best experience for newer browsers at the cost of delivering a less optimal (but still good) experience on older browsers.

The Grid Layout

HTML

The HTML for this demo is about as simple as it gets — an unordered list that contains twelve items, each of which contains a number.

index.html

<!DOCTYPE html>
<html>
  <head>
    <meta charset="utf-8" />
    <title>Grid</title>
    <link rel="stylesheet" type="text/css" href="./styles.css" />
  </head>
  <body>
    <ul class="svh-grid">
      <li>1</li>
      <li>2</li>
      <li>3</li>
      <li>4</li>
      <li>5</li>
      <li>6</li>
      <li>7</li>
      <li>8</li>
      <li>9</li>
      <li>10</li>
      <li>11</li>
      <li>12</li>
    </ul>
  </body>
</html>

CSS

The stylesheet does all the work of rendering the list as a grid and setting the size of the items so that a certain number of rows will be visible without scrolling. Before you continue, it will help to be familiar with custom properties, the calc function and CSS grid.

styles.css

:root {
  --page-height: 100svh;
  --gap-size: 1svh;

  --column-count-portrait: 2;
  --row-count-portrait: 3;
  --gap-height-portrait: calc((var(--row-count-portrait) + 1) * var(--gap-size));
  --content-height-portrait: calc(var(--page-height) - var(--gap-height-portrait));
  --row-height-portrait: calc(var(--content-height-portrait) / var(--row-count-portrait));

  --column-count-landscape: 3;
  --row-count-landscape: 2;
  --gap-height-landscape: calc((var(--row-count-portrait) + 1) * var(--gap-size));
  --content-height-landscape: calc(var(--page-height) - var(--gap-height-landscape));
  --row-height-landscape: calc(var(--content-height-landscape) / var(--row-count-landscape));
}

html,
body {
  padding: 0;
  margin: 0;
  background-color: #FFF;
}

ul.svh-grid {
  list-style: none;
  display: grid;
  grid-template-columns: repeat(var(--column-count-portrait), 1fr);
  grid-auto-rows: var(--row-height-portrait);
  grid-gap: var(--gap-size);
  padding: var(--gap-size);
  margin: 0;
}

ul.svh-grid > li {
  display: flex;
  justify-content: center;
  align-items: center;
  color: #FFF;
  background-color: #333;
  font-family: sans-serif;
  font-size: 10vmax;
}

@media (orientation: landscape) {
  ul.svh-grid {
    grid-template-columns: repeat(var(--column-count-landscape), 1fr);
    grid-auto-rows: var(--row-height-landscape);
  }
}

We start by defining several custom properties. These aren't absolutely necessary, but they will help keep the CSS rules clean and readable.

The --page-height property is set to 100% of the small viewport height. (To reiterate, the small viewport encompasses all the space available to your application when the browser chrome is visible.)
The --gap-size property represents the size of the gutters between grid items and the padding around the grid container. In this example, --gap-size is set to 1% of the small viewport height, but it could have any value and use any units. If you want gutters to be 20px or 1rem on all devices, you're free to do so.

The grid properties are defined separately for portrait and landscape orientation:

--column-count-portrait/--column-count-landscape: The number of columns we want in the grid.
--row-count-portrait/--row-count-landscape: The number of rows we want to render above the fold. The actual number of rows in the grid is determined by the html.
--gap-height-portrait/--gap-height-landscape: The sum of all the gaps between rows that we want to render above the fold. The number of gaps will be one more than the number of rows. If we want three rows above the fold, there will be four gaps including the top and bottom.
--content-height-portrait/--content-height-landscape: The total height available for content, calculated by subtracting the total gap height from the page height.
--row-height-portrait/--row-height-landscape: The row height, calculated by dividing the total content height by the number of rows we want to render above the fold.

Next, we use the custom properties in our CSS rules:

The html, body rule is a simple reset to make sure the browser doesn’t apply default padding or margins that would affect the layout. (You might already have a reset like this in your stylesheet, in which case you might not need this.)

html,
body {
  padding: 0;
  margin: 0;
  background-color: #FFF;
}

The ul.svh-grid rule unsets the default list styles and sets display: grid to enable CSS grid layout. grid-template-columns is set to render the list items in equal-width columns, with the number of columns defined by --column-count-portrait. grid-auto-rows is set to render as many rows as necessary, with the row height defined by --row-height-portrait. The grid’s gap and padding are set to the value of the --gap-size property. The margin is set to zero to remove any default margins that the browser might apply to the ul element.

ul.svh-grid {
  list-style: none;
  display: grid;
  grid-template-columns: repeat(var(--column-count-portrait), 1fr);
  grid-auto-rows: var(--row-height-portrait);
  grid-gap: var(--gap-size);
  padding: var(--gap-size);
  margin: 0;
}

The ul.svh-grid > li rule applies to the individual items in the grid. These styles affect the appearance of the list items and have no impact on the grid layout. Notice that the font size for the numbers is set using the vmax unit. This makes the font size 10% of the maximum viewport dimension, which is the same in both portrait and landscape orientation. This is a simple way to set the font size proportionally to the viewport without having it shift on orientation change.

ul.svh-grid > li {
  display: flex;
  justify-content: center;
  align-items: center;
  color: #FFF;
  background-color: #333;
  font-family: sans-serif;
  font-size: 10vmax;
}

Finally, we use a media query to override the grid styles in landscape orientation:

@media (orientation: landscape) {
  ul.svh-grid {
    grid-template-columns: repeat(var(--column-count-landscape), 1fr);
    grid-auto-rows: var(--row-height-landscape);
  }
}

In portrait orientation, these styles give us a two column layout where three rows appear above the fold.

On page load or when scrolled to the top of the page:

When scrolled to the bottom of the page:

In landscape orientation, these styles give us a three column layout where two rows appear above the fold:

On page load or when scrolled to the top of the page:

When scrolled to the bottom of the page:

Why Not Use the Dynamic Viewport?

You might wonder why we didn't use the dynamic viewport height (dvh) instead of the small viewport, which would have allowed us to optimize the size of the grid items to the height of the viewport regardless of whether or not the browser chrome is visible. As soon as the user scrolls down the page, the browser chrome transitions offscreen and the dynamic viewport compensates by switching from the small viewport to the large viewport. If the layout is based on the dynamic viewport, the grid items will increase in height when that transition occurs. In addition to that problem, the CSS specification doesn’t require that the viewport transition animates at a full 60fps, likely because that would be computationally expensive in many scenarios. Some browsers perform the transition quite abruptly which causes a visible shift in the layout. In this case, using the small viewport height produces a more natural feeling result when scrolling. The behaviour will vary depending on the OS, browser and individual use cases, though, so experiment with the different viewports to find what works best for you. (If you want to see how the dynamic viewport works in this scenario, simply change the --page-height property to 1dvh.)

Building a Custom YAML Loader for Webpack

Tony Wallace — Wed, 04 Jan 2023 14:45:00 +0000

One of our internal tools at RedBit uses yaml to provide structured data. We chose yaml because it's more human-friendly than json, and the data in question is created and modified by humans during development. Applications need to consume the data as JavaScript objects, so we need to convert it from yaml to json. We could do the conversion at runtime but that would affect performance, possibly to the point of degrading the user experience. Instead, we chose to convert the data during the build process, which is controlled by webpack. This required a custom webpack loader. I won't describe the implementation of our internal tool in detail because it wouldn't be particularly helpful or relevant. Instead, I'll show you how to build a simple yaml loader that you can modify to suit your own needs.

The Loader

// yaml-loader.js

const { getOptions } = require('loader-utils');
const validate = require('schema-utils');
const yaml = require('js-yaml');

const loaderOptionsSchema = {
  type: 'object',
  properties: {
    commonjs: {
      description: 'Use CommonJS exports (default: false)',
      type: 'boolean',
    },
  },
};

module.exports = (loaderContext, source) => {
  const callback = loaderContext.async();
  const loaderOptions = getOptions(loaderContext) || {};

  validate(loaderOptionsSchema, loaderOptions, {
    name: 'yaml-loader',
    baseDataPath: 'options',
  });

  const { commonjs = false } = loaderOptions;

  try {
    const data = yaml.load(source);

    // At this point you may perform additional validations
    // and transformations on your data...

    const json = JSON.stringify(data, null, 2);
    const ecmaFileContents = `export default ${json};`;
    const cjsFileContents = `module.exports = ${json};`;
    const fileContents = commonjs ? cjsFileContents : ecmaFileContents;
    callback(null, fileContents);
  } catch (error) {
    callback(error);
  }
};

We begin by defining a schema for the loader options so we can validate them with schema-utils. The schema is an object that describes properties that you can set when you integrate the loader in your webpack config:

const loaderOptionsSchema = {
  type: 'object',
  properties: {
    commonjs: {
      description: 'Use CommonJS exports (default: false)',
      type: 'boolean',
    },
  },
};

In this case, we have one option, commonjs, which is a boolean. If the commonjs option is true, the loader will generate JavaScript files that use CommonJS modules (e.g. module.exports). Otherwise, the loader will use ECMA modules (e.g. export default). You will likely want ECMA modules in most modern web applications, but the option gives you some flexibility to work in other environments. (Note that the loader itself is written as a CommonJS module because it always runs in Node. Using CommonJS avoids some compatibility issues with ECMA modules in older versions of Node.)

Next, we have the loader function itself:

module.exports = (loaderContext, source) => {
  const callback = loaderContext.async();
  const loaderOptions = getOptions(loaderContext) || {};

  validate(loaderOptionsSchema, loaderOptions, {
    name: 'yaml-loader',
    baseDataPath: 'options',
  });

  const { commonjs = false } = loaderOptions;

  try {
    const data = yaml.load(source);

    // At this point you may perform additional validations
    // and transformations on your data...

    const json = JSON.stringify(data, null, 2);
    const ecmaFileContents = `export default ${json};`;
    const cjsFileContents = `module.exports = ${json};`;
    const fileContents = commonjs ? cjsFileContents : ecmaFileContents;
    callback(null, fileContents);
  } catch (error) {
    callback(error);
  }
};

This function accepts two arguments. loaderContext is the context in which the loader runs. We'll use this to obtain some information about the loader, including the options. source is the contents of the input file as a string (a yaml string in this instance). The function performs the following tasks:

Call loaderContext.async() to tell the loader that the process will run asynchronously. loaderContext.async() returns a callback that we'll use to pass the results of the process back to the loader.
Obtain the loader options by calling getOptions(loaderContext), which is a function provided by loader-utils. We default the return value of getOptions to an empty object literal in case the webpack config doesn't include the options hash.
Validate the loader options against the schema we created earlier. This will throw an error if the options aren't specified correctly in the webpack config. Unpack the options, if desired.
Parse the source string. We're using js-yaml for this.
At this point the data is parsed and you can perform additional validations and transformations on it.
Json-serialize the data using JSON.stringify(). Set the indentation according to your preferences (2 spaces in this case).
Create the file contents for ECMA and CommonJS modules by appending the serialized json string to the export statement.
Execute the callback with the appropriate file content string based on the commonjs option.

The result of this process will be that you will be able to import a yaml file in your JavaScript code, and its contents will be made available as a JavaScript module instead of a yaml string.

import data from './data.yaml';

for (key in data) {
  console.log(`The value for key '${key}' is ${data[key]}`);
}

Integration

Integration with webpack is similar to any other loader. Add a new rule to the module.rules array in your webpack config. The rule should test files for the .yaml file extension, exclude any files in the node_modules directory and use your custom yaml loader:

// webpack.config.js

module: {
  rules: [
    {
      test: /\.yaml$/,
      exclude: /node_modules/,
      use: {
        loader: path.resolve('./yaml-loader'),
        options: {
          commonjs: false,
        },
      },
    },
  ],
},

You can use this loader for simple yaml files, or as a starting point for a more complex loader of your own.

Improving Render Performance in React

Tony Wallace — Mon, 21 Nov 2022 14:00:00 +0000

Improving Render Performance in React

Tony Wallace, November 3 2022

Note: This article begins with a discussion of class components in React. By this point in time you've probably migrated to functional components and hooks. That's great! Functional components and hooks are the way forward for React and we'll discuss them later in this article. We're starting with class components to give some historical perspective, and because you should try to be familiar with both styles. A lot of React apps have been written using class components and it's likely that you'll have to maintain one at some point in your career.

Conditional rendering in class components

If you have worked with class components in React, you may have come across the shouldComponentUpdate lifecycle method. This method runs before each render and accepts the next props and state objects. You can use shouldComponentUpdate to compare the next props and state to the current props and state, and decide whether to allow the component to render. (shouldComponentUpdate returns true by default if you don't override it in your component class, which always allows the component to render.)

In the following example, we have a user profile component that renders a list of the user's skills. Each skill has a name and a description. The description is served as markdown and we need to parse it to HTML before we render it. We can do this with a markdown parser (marked, in this case) and use dangerouslySetInnerHTML to inject the HTML into a <div> element. Note that using dangerouslySetInnerHTML may leave your site vulnerable to XSS attacks. In order to mitigate this risk, we need to sanitize the HTML before we inject it. We'll use DOMPurify for that task.

import React, { Component } from 'react';
import marked from 'marked';
import * as DOMPurify from 'dompurify';

class UserProfile extends Component {
  constructor (props) {
    super(props);
  }

  render () {
    const { userId, name, skills = [] } = this.props;

    return (
      <div className="user-profile">
        <h1>{name} ({userId})</h1>
        <h3>Skills</h3>
        {skills.map((skill) => (
          <div className="skill">
            <h5>{skill.name}</h5>
            <div dangerouslySetInnerHTML={{
              __html: DOMPurify.sanitize(marked.parse(skill.description)),
            }} />
          </div>
        ))}
      </div>
    );
  }
}

To recap, our process is:

Parse the skill description markdown to HTML
Sanitize the HTML
Inject the HTML into a <div> element

If we think through this process in the context of the component render cycle, we can see how it could have a negative effect on performance:

Perform two potentially expensive tasks in series (markdown parsing and HTML sanitization)
Perform those tasks in a loop, which is also executed in series
Repeat every time the component renders

We can't do much about the first and second issues — we have to parse the markdown, sanitize the resulting HTML and render it — but we can avoid the third issue. Why render if nothing has changed? Here is an updated version of the previous example that uses shouldComponentUpdate to block the component from rendering unless the userId prop has changed:

import React, { Component } from 'react';
import marked from 'marked';
import * as DOMPurify from 'dompurify';

class UserProfile extends Component {
  constructor (props) {
    super(props);
  }

  shouldComponentUpdate (nextProps) {
    return nextProps.userId !== this.props.userId;
  }

  render () {
    const { userId, name, skills = [] } = this.props;

    return (
      <div className="user-profile">
        <h1>{name} ({userId})</h1>
        <h3>Skills</h3>
        {skills.map((skill) => (
          <div className="skill">
            <h5>{skill.name}</h5>
            <div dangerouslySetInnerHTML={{
              __html: DOMPurify.sanitize(marked.parse(skill.description)),
            }} />
          </div>
        ))}
      </div>
    );
  }
}

We're comparing the userId prop because it's a simple equality comparison that will tell us when the user has changed. You might be tempted to try to perform a deep equality comparison of the skills array instead, but that will likely create a worse performance issue than the one we're trying to solve.

shouldComponentUpdate is intended to be used for performance optimization but should be used carefully and sparingly. Blocking a component from rendering can cause bugs in descendant components that can be difficult to fix, and there are usually better ways to improve performance.

Reducing the rendering workload

In the previous example, we tried to mitigate a potential performance problem by blocking a component from rendering. That was the wrong solution to take because it didn't address the real problem. The real problem isn't that we're allowing the component to render when it doesn't have to, it's that we're doing too much work in the render cycle. Therefore, the correct solution is not to block rendering entirely, but to reduce the amount of work we do while rendering.

Option 1: Store preprocessed data on state

One way of reducing the workload is to run the expensive process only when inputs change and store the results on state. This is simple and doesn't interfere with the render cycle. It works equally well in both class and functional components.

Class component

The following example updates our previous examples to preprocess the user's skills and store them on state, instead of reading directly from props in the render cycle. The preprocessing logic has been moved into the processSkills function, which is called in two places: componentDidMount, where it will run before the initial render; and componentDidUpdate, where it will run when the value of the userId prop changes. If the component renders for any other reason, it will continue to use the preprocessed data from state at no additional cost.

import React, { Component } from 'react';
import marked from 'marked';
import * as DOMPurify from 'dompurify';

const processSkills = (skills = []) => {
  return skills.map((skill) => ({
        ...skill,
        htmlDescription: DOMPurify.sanitize(marked.parse(skill.description)),
      }));
};

class UserProfile extends Component {
  constructor (props) {
    super(props);

    this.state = {
      processedSkills: [],
    };
  }

  componentDidMount () {
    this.setState({
      processedSkills: processSkills(this.props.skills),
    });
  }

  componentDidUpdate (prevProps) {
    if (this.props.userId !== prevProps.userId) {
      this.setState({
        processedSkills: processSkills(this.props.skills),
      });
    }
  }

  render () {
    const { userId, name } = this.props;
    const { processedSkills = [] } = this.state;

    return (
      <div className="user-profile">
        <h1>{name} ({userId})</h1>
        <h3>Skills</h3>
        {processedSkills.map((skill) => (
          <div className="skill">
            <h5>{skill.name}</h5>
            <div dangerouslySetInnerHTML={{
              __html: skill.htmlDescription,
            }} />
          </div>
        ))}
      </div>
    );
  }
}

Functional component (hooks)

The functional component produces the same result as the class component, but is more concise. The useEffect hook calls processSkills with the skills array from props when the value of the userId prop changes. The resulting array is stored on state and used to render the skills list.

import React, { useState, useEffect } from 'react';
import marked from 'marked';
import * as DOMPurify from 'dompurify';

const processSkills = (skills = []) => {
  return skills.map((skill) => ({
        ...skill,
        htmlDescription: DOMPurify.sanitize(marked.parse(skill.description)),
      }));
};

const UserProfile = (props) => {
  const { userId, name, skills = [] } = this.props;
  const [ processedSkills, setProcessedSkills ] = useState([]);

  useEffect(
    () => setProcessedSkills(processSkills(skills)),
    [userId]
  );

      return (
        <div className="user-profile">
          <h1>{name} ({userId})</h1>
          <h3>Skills</h3>
          {processedSkills.map((skill) => (
        <div className="skill">
          <h5>{skill.name}</h5>
          <div dangerouslySetInnerHTML={{
            __html: skill.htmlDescription,
          }} />
        </div>
      ))}
        </div>
      );
}

Option 2: Memoize preprocessed data

Another option is to memoize (not memorize) the results of the process. Memoization is a form of in-memory caching. We're only going to discuss this in the context of functional components where we can use the useMemo hook provided by React, but you may be able to achieve similar results in a class component using a third-party helper.

In this example, the useMemo hook is called on every render. On the initial render and any time the userId prop is updated, useMemo returns the result of calling processSkills with the skills array. If the userId prop hasn't changed since the last render, useMemo returns the previously cached result.

import React, { useMemo } from 'react';
import marked from 'marked';
import * as DOMPurify from 'dompurify';

const processSkills = (skills = []) => {
  return skills.map((skill) => ({
        ...skill,
        htmlDescription: DOMPurify.sanitize(marked.parse(skill.description)),
      }));
};

const UserProfile = (props) => {
  const { userId, name, skills = [] } = this.props;

  const processedSkills = useMemo(
    () => processSkills(skills),
    [userId]
  );

      return (
        <div className="user-profile">
          <h1>{name} ({userId})</h1>
          <h3>Skills</h3>
          {processedSkills.map((skill) => (
        <div className="skill">
          <h5>{skill.name}</h5>
          <div dangerouslySetInnerHTML={{
            __html: skill.htmlDescription,
          }} />
        </div>
      ))}
        </div>
      );
}

Which option should I choose?

If you're working with class components, I suggest preprocessing and storing data on state unless you think you have a strong use case for integrating a memoization helper (e.g. you need memoization throughout your app, not just in one or two places).

In functional components, memoization offers the same benefit as preprocessing and storing the result on state without having to maintain state. However, it isn't guaranteed to be predictable. From the React docs:

You may rely on useMemo as a performance optimization, not as a semantic guarantee. In the future, React may choose to “forget” some previously memoized values and recalculate them on next render, e.g. to free memory for offscreen components. Write your code so that it still works without useMemo — and then add it to optimize performance.

Storing data on state may be a better choice if the predictability of your performance optimizations is critical to your application. In many, if not most cases it won't be critical, and memoization will likely be the cleanest and simplest way to improve render performance.

Conditional Logging to the JavaScript Console

Tony Wallace — Mon, 14 Nov 2022 14:00:00 +0000

The JavaScript console is a very useful debugging tool, but it can also introduce a very simple security risk into your application: console method calls that aren't removed before code is deployed can leak sensitive information to the browser. That's why many developers use a linter to warn about console method calls and other issues. (On RedBit's web team, we use husky to run the linter on a pre-commit hook so that console method calls are never committed by accident. We've found that many small bugs can be avoided by making linting mandatory before committing code.)

The problem is that there are times when you want to leave console method calls in place. You can use eslint-disable comments to bypass the linter, but you still run the risk of leaking sensitive information if you forget to clean up before you deploy. You could also wrap your console method calls in a condition that checks a debug or environment flag and only executes the calls in development, but that's a bit awkward. It's also easy to forget so your users' security will depend on your memory. That certainly isn't a good strategy given that we all make mistakes from time to time.

A better solution would be to integrate the debug or environment condition into the console itself. We can do this by replacing the browser's window.console object with a modified version of the same. Example 1 shows the function that does this.

Example 1

// console.js

export const createDebugConsole = (consoleObj, options = {}) => {
  const nextConsoleObj = { ...consoleObj };
  const { enabled = true } = options;

  for (let key in nextConsoleObj) {
    /* eslint-disable no-prototype-builtins */
    if (
      nextConsoleObj.hasOwnProperty(key) &&
      typeof nextConsoleObj[key] === 'function'
    ) {
      const func = nextConsoleObj[key];
      nextConsoleObj[key] = function () {
        if (enabled) {
          func.apply(nextConsoleObj, arguments);
        }
      };
    }
    /* eslint-enable no-prototype-builtins */
  }

  return nextConsoleObj;
};

Notice that the createDebugConsole accepts the console object as an argument, rather than assuming it to be in scope. The function is designed this way for two reasons. First, it eliminates a dependency on the global scope which makes the function easier to unit test. Second, it makes it possible to use the function in non-browser environments that use a browser-like console object that may not be attached to the window global.

Let's examine this function line by line:

Accept the console object and a hash of options as arguments.
Make a shallow copy of the console object as nextConsoleObj, to avoid modifying the original.
Destructure the enabled boolean from the options hash. The default value is true, which means that the new console will allow logging unless you pass { enabled: false } in the options hash.
Enumerate the properties of the copied console object. For each key that is owned by the object (i.e. not inherited) and whose value is a function, take a reference to the value (func) and then replace it with a new function. If the enabled option is true, the new function calls func.apply with the new console object and the new function's arguments. (Note that we're using the traditional function declaration here, rather than an ES6 arrow function, to ensure that we have access to the arguments array.) If the enabled option is false, the new function does nothing. Any additional properties of the console object that are not functions will be left unchanged.
Return the new console object.

(Depending on your linter rules, you may or may not need the /* eslint-disable no-prototype-builtins */ and /* eslint-enable no-prototype-builtins */ comments.)

Integration

Example 2 shows how to use the function to replace the browser's default console with our new debug console. In this case, we're using process.env.NODE_ENV to enable logging in non-production builds. (This assumes that your build process replaces process.env.NODE_ENV with the value of the NODE_ENV environment variable.) You could also set the enabled option based on a value from a dotenv file or any other source of configuration.

Example 2

// index.js

import { createDebugConsole } from './console';

window.console = createDebugConsole(window.console, {
  enabled: process.env.NODE_ENV !== 'production'
});

console.log("This won't log in production!");

Call createDebugConsole once in your application's main index.js file or wherever you do other setup work. You can now use the global console object normally. If the condition that sets the enabled value is true, the console methods will behave normally. It it's false the methods will return without doing anything. You can now be more confident that your production applications won't leak data to the console.

You could also assign the return value of createDebugConsole to a new constant in case you need to leave the window.console global untouched, or if you just don't want to use the global console, as shown in Example 3:

Example 3

import { createDebugConsole } from './console';

const myConsole = createDebugConsole(window.console, {
  enabled: process.env.NODE_ENV !== 'production'
});

myConsole.log("This won't log in production!");

The approach you take should depend on your application and personal preferences. I prefer to use the window.console global for two reasons:

It allows me to use the console without having to import anything
The linter will still complain if I leave console method calls in my code without explicitly allowing them with eslint-disable no-console comments. This helps reduce unnecessary logging.

Testing

The console created by createDebugConsole will not be active in unit tests, so you won't be able to use it to prevent logging in your test environment. If you use jest you can silence console methods with the jest CLI.

Here is a short jest suite that you can use to test createDebugConsole:

import { createDebugConsole } from './console';

const getMockConsole = () => ({
  assert: jest.fn(),
  clear: jest.fn(),
  count: jest.fn(),
  countReset: jest.fn(),
  debug: jest.fn(),
  dir: jest.fn(),
  dirxml: jest.fn(),
  error: jest.fn(),
  group: jest.fn(),
  groupCollapsed: jest.fn(),
  groupEnd: jest.fn(),
  info: jest.fn(),
  log: jest.fn(),
  profile: jest.fn(),
  profileEnd: jest.fn(),
  table: jest.fn(),
  time: jest.fn(),
  timeEnd: jest.fn(),
  timeLog: jest.fn(),
  timeStamp: jest.fn(),
  trace: jest.fn(),
  warn: jest.fn(),
});

test('createDebugConsole returns a copy of the input console', () => {
  const consoleObj = getMockConsole();
  const nextConsoleObj = createDebugConsole(consoleObj);

  expect(consoleObj === nextConsoleObj).toBe(false);

  for (let key in consoleObj) {
    // eslint-disable-next-line no-prototype-builtins
    if (consoleObj.hasOwnProperty(key)) {
      expect(consoleObj[key] === nextConsoleObj[key]).toBe(false);
    }
  }
});

test('When a function is called on the debug console returned by createDebugConsole, the same function is called on the original console with the same arguments if the debug console is enabled', () => {
  const consoleObj = getMockConsole();
  const nextConsoleObj = createDebugConsole(consoleObj, { enabled: true });
  const args = ['test', 'test2', 'test3'];

  for (let key in nextConsoleObj) {
    // eslint-disable-next-line no-prototype-builtins
    if (nextConsoleObj.hasOwnProperty(key)) {
      nextConsoleObj[key](...args);
      expect(consoleObj[key]).toHaveBeenCalledTimes(1);
      expect(consoleObj[key]).toHaveBeenCalledWith(...args);
    }
  }
});

test('When a function is called on the debug console returned by createDebugConsole, the same function is not called on the original console if the debug console is disabled', () => {
  const consoleObj = getMockConsole();
  const nextConsoleObj = createDebugConsole(consoleObj, { enabled: false });
  const args = ['test', 'test2', 'test3'];

  for (let key in nextConsoleObj) {
    // eslint-disable-next-line no-prototype-builtins
    if (nextConsoleObj.hasOwnProperty(key)) {
      nextConsoleObj[key](...args);
      expect(consoleObj[key]).not.toHaveBeenCalled();
    }
  }
});

SQL Tip: Self-Referential Tables

Tony Wallace — Wed, 09 Nov 2022 13:48:52 +0000

Note: The examples in this post are written for MSSQL. Syntax may differ for other SQL variants.

I was recently tasked with designing a SQL database for a new app. One of the requirements was to be able to store a category tree of unknown complexity and depth. It wasn't sufficient to have categories and subcategories; subcategories could also have subcategories, which could have subcategories, and so on. Each branch of the tree could have a different structure and continue to a different depth. An example tree would look something like this:

Category 1
- Category 1.1
- Category 1.2
- Category 1.2.1
- Category 1.2.2
- Category 1.3
- Category 1.3.1
  - Category 1.3.1.1
- Category 1.3.2
Category 2
- Category 2.1
- Category 2.1.1
  - Category 2.1.1.1
  - Category 2.1.1.2
  - Category 2.1.1.2.1
- Category 2.2
- Category 2.2.1
- Category 2.2.2
- Category 2.2.3

It was clearly not going to be possible to satisfy these requirements with simple category and subcategory tables. I needed to be able to describe all the category relationships in one self-referential table.

Designing the self-referential table

Relationships in a self-referential table are defined by two fields:

id (primary key, uniqueidentifier, not nullable)
parentId (foreign key, uniqueidentifier, nullable)

Note that parentId is a foreign key that references the same table. If your table is called category, category.parentId has a foreign key constraint to category.id.

(You can also declare any other fields you might need to store the category name and other information. Those aren't relevant to this tip.)

When you insert a row into the table, you can set parentId to null to put the row at the top of the tree, or set parentId to the id of any other category to make the row a subcategory.

Selecting branches of the tree

Creating the tree is simple enough, but how do you select only the rows that are descendants of a given category id? For that, we can use a recursive common-table expression (recursive CTE).

The recursive CTE creates a virtual table ([tree]) that is the union of the rows that have a given parent, the rows that have that first set of rows as parents, the rows that have that next set of rows as parents, and so on until there are no more descendent rows to fetch. We then select all rows from the virtual table:

DECLARE @id UNIQUEIDENTIFIER;
SET @id = '<id of the category whose descendants you want to select>';

WITH [tree] AS (
    SELECT parent.* FROM [category] parent WHERE parent.id = @id
    UNION ALL
    SELECT child.* FROM [tree]
    INNER JOIN [category] child ON child.parentId = [tree].id
)

SELECT * from [tree]

We can modify the SELECT statement to fetch only the bottom-most descendants of a given category, omitting the upper levels of the tree:

DECLARE @id UNIQUEIDENTIFIER;
SET @id = '<id of the category whose descendants you want to select>';

WITH [tree] AS (
    SELECT parent.* FROM [category] parent WHERE parent.id = @id
    UNION ALL
    SELECT child.* FROM [tree]
    INNER JOIN [category] child ON child.parentId = [tree].id
)

SELECT * FROM [tree] WHERE NOT EXISTS (
    SELECT parentId FROM [category] WHERE [category].parentId = [tree].id
)

We can also invert the CTE to fetch categories that are ancestors of a given subcategory:

DECLARE @id UNIQUEIDENTIFIER;
SET @id = '<id of the subcategory whose ancestors you want to select>';

WITH [tree] AS (
    SELECT child.* FROM [category] child WHERE child.id = @id
    UNION ALL
    SELECT parent.* FROM [tree]
    INNER JOIN [category] parent ON parent.id = [tree].parentId
)

SELECT * from [tree]

By modifying the SELECT statement, we can find the top-most ancestor of any subcategory:

DECLARE @id UNIQUEIDENTIFIER;
SET @id = '<id of the subcategory whose ancestors you want to select>';

WITH [tree] AS (
    SELECT child.* FROM [category] child WHERE child.id = @id
    UNION ALL
    SELECT parent.* FROM [tree]
    INNER JOIN [category] parent ON parent.id = [tree].parentId
)

SELECT * from [tree] WHERE parentId IS NULL

This combination of a self-referential table and recursive common-table expressions provided a great solution to my requirements. The model is straightforward, and easy to maintain, and the selection of entire branches can be performed in a single query.

Using Array.reduce With Objects

Tony Wallace — Tue, 04 Jan 2022 14:23:48 +0000

This article will demonstrate a few ways that you can use JavaScript's Array.reduce method to transform objects.

An Array.reduce primer

Let's take a quick look at how Array.reduce works. You can skip this if you're already familiar with it.

Array.reduce reduces an array down to a single value. The resulting value can be of any type — it does not have to be an array. This is one way in which Array.reduce differs from other array methods like map and filter. Here's a reduce statement that returns the sum of an array of numbers:

Example 1:

const numbers = [1, 2, 3, 4, 5];
const sum = numbers.reduce((next, number) => {
  return next + number;
}, 0);

Array.reduce accepts two arguments:

A callback function that is executed for each item in the array in series and receives the following parameters:
- The accumulator (called next in the example), which is the working value. In the first iteration the accumulator is set to the initial value (0). For all subsequent iterations it's the value returned by the previous iteration.
- The current array item (number in the example).
- The current array index (not used in the example).
- The array that is being reduced (not used in the example).
The initial value for the accumulator. In Example 1, the initial value is 0.

The reduce statement in Example 1 will execute the callback function five times with the following values:

Accumulator (next): 0 (the initial value); Value (number): 1; Returns: 1;
Accumulator: 1; Value: 2; Returns: 3;
Accumulator: 3; Value: 3; Returns: 6;
Accumulator: 6; Value: 4; Returns: 10;
Accumulator: 10; Value: 5; Returns: 15;

The final value of sum will be 15.

Array.reduce applied to objects

Remember that Array.reduce can use initial and return values of any type, which makes it very flexible. Let's explore how we can use it to perform some common tasks with plain objects.

1. Converting an array of objects to a single object keyed by id:

Developers frequently have to look up a value in one array using a value from another array. Consider the following hypothetical example where we have an array of objects representing users and another array representing profiles. Each user has an id property and each profile has a userId property. We need to match each user with their profile, where user.id equals profile.userId. A basic implementation of this is shown in Example 2.

Example 2:

const users = [
  { id: 1, email: 'dcontreras@email.tld' },
  { id: 2, email: 'afeher@email.tld' },
  { id: 3, email: 'odj@email.tld' },
];

const profiles = [
  { userId: 1, firstName: 'Danielle', lastName: 'Contreras' },
  { userId: 2, firstName: 'Alfredas', lastName: 'Fehér' },
  { userId: 3, firstName: 'Orpheus', lastName: 'De Jong' },
];

const usersWithProfiles = users.map((user) => {
  const profile = profiles.find((profile) => (user.id === profile.userId));
  return { ...user, profile };
});

// usersWithProfiles:
// [
//   { id: 1, email: 'dcontreras@email.tld', profile: { userId: 1, firstName: 'Danielle', lastName: 'Contreras' } },
//   { id: 2, email: 'afeher@email.tld', profile: { userId: 2, firstName: 'Alfredas', lastName: 'Fehér' } },
//   { id: 3, email: 'odj@email.tld', profile: { userId: 3, firstName: 'Orpheus', lastName: 'De Jong' } },
// ]

The problem with Example 2 is that it uses Array.find inside Array.map which is inefficient. This might not matter for the short arrays in the example, but it will become more of a problem when working with longer arrays. The longer the profiles array is, the longer the potential lookup time will be for any given profile. We can solve this problem by transforming the profiles array into an object beforehand, using the userId property as the key:

Example 3:

const users = [
  { id: 1, email: 'dcontreras@email.tld' },
  { id: 2, email: 'afeher@email.tld' },
  { id: 3, email: 'odj@email.tld' },
];

const profiles = [
  { userId: 1, firstName: 'Danielle', lastName: 'Contreras' },
  { userId: 2, firstName: 'Alfredas', lastName: 'Fehér' },
  { userId: 3, firstName: 'Orpheus', lastName: 'De Jong' },
];

// Transform the profiles into an object keyed by the userId:
const profilesByUserId = profiles.reduce((next, profile) => {
  const { userId } = profile;
  return { ...next, [userId]: profile };
}, {});

// profilesByUserId:
// {
//   1: { userId: 1, firstName: 'Danielle', lastName: 'Contreras' },
//   2: { userId: 2, firstName: 'Alfredas', lastName: 'Fehér' },
//   3: { userId: 3, firstName: 'Orpheus', lastName: 'De Jong' },
// }

// Look up the profiles by id:
const usersWithProfiles = users.map((user) => {
  return { ...user, profile: profilesByUserId[user.id] };
});

// usersWithProfiles:
// [
//   { id: 1, email: 'dcontreras@email.tld', profile: { userId: 1, firstName: 'Danielle', lastName: 'Contreras' } },
//   { id: 2, email: 'afeher@email.tld', profile: { userId: 2, firstName: 'Alfredas', lastName: 'Fehér' } },
//   { id: 3, email: 'odj@email.tld', profile: { userId: 3, firstName: 'Orpheus', lastName: 'De Jong' } },
// ]

Example 3 produces the same result as Example 2 but will be much faster with long arrays.

2. Copying an object with filtered properties:

Sometimes you need a copy of an object that only includes certain properties from the original object, or that omits certain properties. This is a great use case for Array.reduce.

Example 4:

// Copy an object, retaining allowed properties:

const person = {
  firstName: 'Orpheus',
  lastName: 'De Jong',
  phone: '+1 123-456-7890',
  email: 'fake@email.tld',
};

const allowedProperties = ['firstName', 'lastName'];

const allKeys = Object.keys(person);
const result = allKeys.reduce((next, key) => {
  if (allowedProperties.includes(key)) {
    return { ...next, [key]: person[key] };
  } else {
    return next;
  }
}, {});

// result:
// { firstName: 'Orpheus', lastName: 'De Jong' }

Example 4 reduces the keys from the person object into a new object that only contains the properties whose keys are included in the allowedProperties array. If you add a new property to the person object it will not appear in the result unless you also add the property name to the allowed properties.

Example 5:

// Copy an object, excluding disallowed properties:

const person = {
  firstName: 'Orpheus',
  lastName: 'De Jong',
  phone: '+1 123-456-7890',
  email: 'odj@email.tld',
};

const disallowedProperties = ['phone', 'email'];

const allKeys = Object.keys(person);
const result = allKeys.reduce((next, key) => {
  if (!disallowedProperties.includes(key)) {
    return { ...next, [key]: person[key] };
  } else {
    return next;
  }
}, {});

// result:
// { firstName: 'Orpheus', lastName: 'De Jong' }

Example 5 reduces the keys from the person object into a new object that only contains the properties whose keys are not included in the disallowedProperties array. If you add a new property to the person object it will appear in the result unless you also add the property name to the disallowed properties. If you want to ensure that only certain properties will be included in the result, Example 4 is a better choice, but Example 5 is useful when you only need to ensure that certain properties will never included.

We can also create generic functions for the reduce statements in examples 4 and 5:

Example 6:

const filterAllowedObjectProperties = (obj, allowedProperties = []) => {
  return Object.keys(obj).reduce((next, key) => {
    if (allowedProperties.includes(key)) {
      return { ...next, [key]: obj[key] };
    } else {
      return next;
    }
  }, {});
}

const filterDisallowedObjectProperties = (obj, disallowedProperties = []) => {
  return Object.keys(obj).reduce((next, key) => {
    if (!disallowedProperties.includes(key)) {
      return { ...next, [key]: obj[key] };
    } else {
      return next;
    }
  }, {});
}

Merging two objects, preferring values from one:

Another common task is to merge an object with another object that contains fallback, or default values for some properties. Sometimes you can do this simply by using object spread, but this can have unexpected consequences when you have null or empty properties:

Example 7:

const obj1 = {
  key1: 'value 1.1',
  key2: null,
  key3: 'value 1.3',
  key4: ''
};

const obj2 = {
  key1: 'value 2.1',
  key2: 'value 2.2',
  key3: 'value 2.3',
  key4: 'value 2.4',
  key5: 'value 2.5'
};

const result = { ...obj2, ...obj1 };

// result:
//  {
//    key1: 'value 2.1',
//    key2: null,
//    key3: 'value 2.3',
//    key4: '',
//    key5: 'value 2.5'
//  };

Example 7 creates a new object containing the properties from obj2 overridden by the properties from obj1. Therefore, the values from obj2 serve as fallback or defaults for the values from obj1. Notice that the result retains the null and empty string values from obj1. That happens because null and an empty string are both defined values. We probably didn't want this result but Array.reduce offers a solution.

Example 8:

const obj1 = {
  key1: 'value 1.1',
  key2: null,
  key3: 'value 1.3',
  key4: ''
};

const obj2 = {
  key1: 'value 2.1',
  key2: 'value 2.2',
  key3: 'value 2.3',
  key4: 'value 2.4',
  key5: 'value 2.5'
};

// Spread the keys from both objects into an array.
const allKeys = [ ...Object.keys(obj1), ...Object.keys(obj2) ];

// Convert the array of keys to a set to remove duplicate values,
// then spread the unique values into a new array.
const uniqueKeys = [ ...new Set(allKeys) ];

// Reduce the unique keys into a new object containing the value
// for each key from obj1, falling back to the value from obj2 if
// obj1[key] is falsey.
const result = uniqueKeys.reduce((next, key) => {
  const value = obj1[key] || obj2[key];
  return { ...next, [key]: value };
}, {});

// result:
// {
//   key1: 'value 1.1',
//   key2: 'value 2.2',
//   key3: 'value 1.3',
//   key4: 'value 2.4',
//   key5: 'value 2.5',
// }

Note that Example 8 uses a naive strategy to decide when to use the fallback value. The fallback value (obj2[key]) is used if the preferred value (obj1[key]) is falsey. That means undefined, null, an empty string, 0 or false. This may not be appropriate for all cases, since any of these values might be acceptable in your application. Revise the fallback condition as necessary. For example, replacing const value = obj1[key] || obj2[key]; with const value = (obj1[key] !== undefined && obj1[key] !== null) ? obj1[key] : obj2[key]; will ensure that the fallback value is only used when the preferred value is undefined or null.

Parsing search/query strings:

Finally, let's look at another very common task that developers often use a third party library to accomplish: parsing search strings (sometimes called query strings). Modern web browsers provide URLSearchParams to make quick work of this, but maybe you aren't writing code for a browser, or you have to support Internet Explorer, or you just want to try doing it yourself because that's how we learn. Whatever your reason, Array.reduce is your friend.

First, we need a search string. You might get this directly from window.location.search in a browser or by parsing an URL. If you use React and react-router you might use the useLocation hook:

`const { search = '' } = useLocation();`

However you get the search string, you'll need to prepare it first:

Example 9a:

// Get a search string:
const search = '?key1=value%201&key2=value%202&key3=value%203';

// Remove the leading '?':
const query = search.replace(/^\?/, '');

// Split the string on the ampersand to create an array of key-value strings:
const pairs = query.split('&');

// pairs:
// [ 'key1=value%201', 'key2=value%202', 'key3=value%203' ];

Next, reduce the key-value strings into an object by splitting them on the equals sign. The string before = is the key and the remainder is the value. The value needs to be decoded with decodeURIComponent.

Example 9b:

const params = pairs.reduce((next, pair) => {
  const [ key, value ] = pair.split('=');
  const decodedValue = decodeURIComponent(value);
  return { ...next, [key]: decodedValue };
}, {});

// params:
// {
//   key1: 'value 1',
//   key2: 'value 2',
//   key3: 'value 3',
// }

The parser in Example 9a/9b will do the job in many cases, but it's incomplete. Search strings can contain multiple values for each key, and this parser will only retain the last value for each key. Let's fix that.

Example 10:

const search = '?key1=value%201&key2=value%202&key3=value%203.1&key3=value%203.2&key3=value%203.3';
const query = search.replace(/^\?/, '');
const pairs = query.split('&');

const params = pairs.reduce((next, pair) => {
  const [ key, value ] = pair.split('=');
  const decodedValue = decodeURIComponent(value);
  const previousValue = next[key];
  let nextValue;

  if (previousValue !== undefined) {
    if (Array.isArray(previousValue)) {
      nextValue = [ ...previousValue, decodedValue ];
    } else {
      nextValue = [ previousValue, decodedValue ];
    }
  } else {
    nextValue = decodedValue;
  }

  return { ...next, [key]: nextValue };
}, {});

// params:
// {
//   key1: 'value 1',
//   key2: 'value 2',
//   key3: [ 'value 3.1', 'value 3.2', 'value 3.3' ],
// }

Example 10 prepares the search string exactly the same way as Example 9a. The difference is how the reduce callback handles the value for each key. Here's a step by step breakdown of the callback function:

The key-value string (pair) is split on = to get separate strings for the key and value.
The value is decoded with decodeURIComponent.
The accumulator (next) is checked to determine if there is a previous value for the key.
If there is a previous value (previousValue !== undefined) it is checked to determine whether it's an array.
If the previous value is an array, the decoded value is appended to it. (nextValue = [ ...previousValue, decodedValue ];) If the previous value isn't an array, a new array is created containing the previous and decoded values. (nextValue = [ previousValue, decodedValue ];)
If there is no previous value, the next value is set to the decoded value. (nextValue = decodedValue;)

The resulting params object contains string values for key1 and key2, and an array containing the three strings for key3 in the order in which they appeared in the search string.

Like we did in Example 1, we can clarify the process by examining the state of each iteration:

Accumulator (next): {} (the initial value); Value (pair): 'key1=value%201; Returns: { key1: 'value 1' };
Accumulator: { key1: 'value 1' }; Value: 'key2=value%202; Returns: { key1: 'value 1', key2: 'value 2' };
Accumulator: { key1: 'value 1', key2: 'value 2' }; Value: 'key3=value%203.1; Returns: { key1: 'value 1', key2: 'value 2', key3: 'value 3.1' };
Accumulator: { key1: 'value 1', key2: 'value 2', key3: 'value 3.1' }; Value: 'key3=value%203.2; Returns: { key1: 'value 1', key2: 'value 2', key3: ['value 3.1', 'value 3.2'] };
Accumulator: { key1: 'value 1', key2: 'value 2', key3: ['value 3.1', 'value 3.2'] }; Value: 'key3=value%203.3; Returns: { key1: 'value 1', key2: 'value 2', key3: ['value 3.1', 'value 3.2', 'value 3.3'] };

Summary:

Array.reduce is sort of a Swiss army knife that you can use to solve a wide variety of problems. I encourage you to explore it and try applying it in situations you might not have considered.

Are you using Array.map correctly?

Tony Wallace — Tue, 04 Jan 2022 14:03:55 +0000

Array.map - Appropriate and Inappropriate Uses

Lately, I've noticed a trend towards the inappropriate use of Array.map, both in tutorials and production code. I'm not sure exactly why this is happening, but I think it may stem from the prevalence of Array.map in React JSX components. JSX code typically uses Array.map to render a component hierarchy for each item in an array.

Example 1:

import React from 'react';
import PropTypes from 'prop-types';

const ItemsList = (props) => {
  const { items = [] } = props;

  return (
    <ul className="items-list">
      {items.map((item) => (
        <li key={item.id}>{item.content}</li>
      ))}
    </ul>
  );
};

ItemsList.propTypes = {
  items: Proptypes.arrayOf(
    PropTypes.shape({
      id: PropTypes.string.isRequired,
      content: PropTypes.node.isRequired,
    })
  ).isRequired,
};

export default ItemsList;

The example above shows a component that renders an unordered list from an array of objects (named items). Each item in the list is rendered with the content property of the current object (item) as the items array is enumerated. If you've worked with React or similar rendering libraries before, this should be familiar. But a few details might not be completely obvious:

The items.map statement returns an array.
The returned array is used, even though it isn't assigned to a variable or constant.

This can made more explicit by rewriting the component as follows:

Example 2:

const ItemsList = (props) => {
  const { items = [] } = props;
  const listItems = items.map((item) => (
    <li key={item.id}>{item.content}</li>
  ));

  return (
    <ul className="items-list">
      {listItems}
    </ul>
  );
};

There is no behavioural difference between this and the previous version of ItemsList. The only change is that the items.map statement's return value is now assigned to const listItems before rendering. Whether you use one approach over the other in practice is mostly a question of style. The point of this example is to make the render flow more explicit. In either case, the items array is enumerated by the items.map statement, which returns an array of JSX components to be rendered.

The React/JSX example demonstrates the correct use of Array.map — as a means of transforming the contents of an array. Here are some conventional examples:

Example 3:

// Multiply each number in an array by 2:

const numbers = [1,2,3,4,5].map(n => n * 2);
console.log(numbers);

// Result:
// [2,4,6,8,10]

// Get full names for an array of people:

const guitarists = [
  { firstName: 'Bill', lastName: 'Frisell' },
  { firstName: 'Vernon', lastName: 'Reid' },
];

const names = guitarists.map((guitarist) => {
  const { firstName, lastName } = guitarist;
  return `${firstName} ${lastName}`;
});

console.log(names);

// Result:
// ['Bill Frisell', 'Vernon Reid']

// Add the full names to an array of people:

const guitaristsWithFullNames = guitarists.map((guitarist) => {
  const { firstName, lastName } = guitarist;
  return { ...guitarist, fullName: `${firstName} ${lastName}` };
});

console.log(guitaristsWithFullNames);

// Result:
/*
[
  { firstName: 'Bill', lastName: 'Frisell', fullName: 'Bill Frisell' },
  { firstName: 'Vernon', lastName: 'Reid', fullName: 'Vernon Reid' },
]
*/

Now that we've looked at some appropriate use cases for Array.map, let's look at an inappropriate use case:

Example 4:

[1,2,3,4,5].map((number) => console.log(number));

This is a trivial example that uses Array.map to execute a side effect for each item in an array. (In this case, the side effect is a call to console.log but that doesn't matter. You could substitute any other function.) This code works and is familiar to beginning JavaScript developers because it's used so frequently in JSX, so what's wrong with it?

To put it simply, just because something works doesn't always mean it's correct or appropriate. The use of Array.map in Example 4 is inappropriate because it doesn't conform to the method's intended purpose. True to its name, Array.map is intended to be used to map (or transform) data from one structure to another. All the appropriate use cases we looked at follow this pattern:

An array of data is mapped to an array of JSX components.
An array of numbers is mapped to an array of the same numbers multiplied by 2.
An array of objects representing people is converted to (or extended with) their full names.

Using Array.map for anything other than mapping creates a few problems. First and foremost, it makes the code less clear. A developer reading code should expect Array.map to perform some kind of transform and for the return value to be used. Second, the return value is always created whether you use it or not. In Example 4, the Array.map callback returns undefined. That means the Array.map statement returns an array containing an undefined value for each index — [1,2,3,4,5] maps to [undefined, undefined, undefined, undefined, undefined]. Aside from being messy, there are performance costs associated with the unnecessary creation and disposal of those unused return values. Used in this way, Array.map is slower than the appropriate alternative, Array.forEach.

Array.forEach

If we rewrite Example 4 using Array.forEach instead of Array.map, we eliminate these problems:

[1,2,3,4,5].forEach((number) => console.log(number));

Array.forEach is intended to be used this way and does not create an unnecessary return value for each item in the array. It's both clearer and faster.

Array.forEach vs. the for loop

Array.forEach is similar in concept to a traditional for loop, but has a few practical advantages. The obvious advantages are clarity and brevity. I think we can all agree that Array.forEach is easier to read and write:

Example 5:

// Array.forEach:

[1,2,3,4,5].forEach((number) => console.log(number));

// for loop:

const numbers = [1,2,3,4,5];

for (let i = 0; i < numbers.length; i++) {
  console.log(numbers[i]);
}

Another hidden advantage is that Array.forEach handles arrays with uninitialized or deleted indexes (sparse arrays) gracefully. Consider the following example, which enumerates an array whose third index is uninitialized:

Example 6:

const numbers = [1,2,,4,5];

// Array.forEach:

numbers.forEach((number, i) => console.log(`number: ${number}, index: ${i}`));

// Result:
// number 1, index 0
// number 2, index 1
// number 4, index 3
// number 5, index 4

// for loop:

for (let i = 0; i < numbers.length; i++) {
  console.log(`number: ${numbers[i]}, index: ${i}`);
}

// Result:
// number 1, index 0
// number 2, index 1
// number undefined, index 2
// number 4, index 3
// number 5, index 4

Notice that Array.forEach skips the uninitialized index, thereby avoiding any problems that may be created by working on an uninitialized (or deleted) value. In most cases this is a nice safety feature to have. The for loop is unforgiving and enumerates uninitialized values just like initialized values. If you want to ignore them, you have to do it manually. The side effect of having Array.forEach perform those checks for you automatically is that it's somewhat slower than a for loop written without checks. Before you go rewriting all your Array.forEach code with for loops to try to save a thousandth of a millisecond, keep in mind that the performance hit is negligible in the vast majority of real world use cases.

Summary:

Why should you choose Array.map, Array.forEach or a traditional for loop?

Choose Array.map if you need to create a new array containing transformations of the items in the source array.
Choose Array.forEach when you simply need to enumerate the values in an array.
Choose a for loop only if absolute performance is critical (at the expense of stability and readability) or you still have to support Internet Explorer 8 (in which case you have my sympathy).

React Function Components: Testable Code Patterns

Tony Wallace — Tue, 04 Jan 2022 13:52:16 +0000

The Problem

The advent of function components has introduced new ways to think about component design in React. We can write code that's cleaner and easier to understand, while dispensing with a lot of the boilerplate code required by class components. This should be a win for developers (and hopefully for future code maintainers) but the patterns that have been demonstrated in many tutorials and adopted by many developers leave something to be desired: testability. Consider the example shown in Example 1.

Example 1

import React, { useState } from 'react';
import PropTypes from 'prop-types';

const AddButton = (props) => {
  const { initialNumber, addNumber } = props;
  const [ sum, setSum ] = useState(initialNumber);

  const addToSum = () => {
    setSum(sum + addNumber);
  };

  return (
    <button onClick={addToSum}>
      Add {addNumber} to {sum}
    </button>
  );
};

AddButton.defaultProps = {
  initialNumber: 0,
  addNumber: 1,
};

AddButton.propTypes = {
  initialNumber: PropTypes.number.isRequired,
  addNumber: PropTypes.number.isRequired,
};

export default AddButton;

This is a trivial component that adds a number to a sum each time a button is pressed &emdash; the sort of thing you'll find in a typical tutorial. The component accepts an initial number and the number to add as props. The initial number is set as the initial sum on state and each press of the button updates the sum by adding the number to it. There isn't much to this component. The business logic consists of the addToSum function, which amounts to a simple math expression whose result is passed to the setSum state setter. It should be very easy to test that this produces the correct result, but it isn't because addToSum is declared within the component's scope and can't be accessed from outside the component. Let's make a few small changes to fix that. Example 2 moves the logic into a separate function, so we can test that the math is correct.

Example 2

// functions.js

export const add = (a, b) => {
  return a + b;
};

// functions.test.js

import { add } from './functions';

test('The add function calculates the sum of two numbers', () => {
  const sum = add(4, 5);
  expect(sum).toEqual(9);
});

// component.js

import React, { useState } from 'react';
import PropTypes from 'prop-types';
import { add } from './functions';

const AddButton = (props) => {
  const { initialNumber, addNumber } = props;
  const [ sum, setSum ] = useState(initialNumber);

  const addToSum = () => {
    setSum(add(sum, addNumber));
  };

  return (
    <button onClick={addToSum}>
      Add {addNumber} to {sum}
    </button>
  );
};

AddButton.defaultProps = {
  initialNumber: 0,
  addNumber: 1,
};

AddButton.propTypes = {
  initialNumber: PropTypes.number.isRequired,
  addNumber: PropTypes.number.isRequired,
};

export default AddButton;

This is slightly better. We can test that the sum will be calculated correctly but we still have that pesky addToSum function littering up our component and we still can't test that the sum is actually set on state. We can fix both of these problems by introducing a pattern that I call an effect function.

Introducing Effect Functions

An effect function is really just a closure &emdash; a function that returns another function &emdash; in which the inner function has access to the outer function's scope. This pattern is nothing new. It has been widely used as a solution to scope problems in JavaScript for a long time. We're just going to put it to use to improve the structure and testability of our React components. I call it an effect function because of how it integrates with React's useEffect hook and other event handlers, which we'll see later on.

Example 3 builds on Example 2 by moving all the logic into an effect function called addToSumEffect. This cleans up the component nicely and allows us to write more comprehensive tests.

Example 3

// functions.js

export const add = (a, b) => {
  return a + b;
};

// functions.test.js

import { add } from './functions';

test('The add function calculates the sum of two numbers', () => {
  const sum = add(4, 2);
  expect(sum).toEqual(6);
});

// effects.js

import { add } from './functions';

export const addToSumEffect = (options = {}) => {
  const { addNumber, sum, setSum } = options;
  return () => {
    setSum(add(sum, addNumber));
  };
};

// effects.test.js

import { addToSumEffect } from './effects';

test('addToSumEffect returns a function', () => {
  const addNumber = 4;
  const sum = 2;
  const setSum = jest.fn();
  const func = addToSumEffect({ addNumber, sum, setSum });
  expect(typeof func).toEqual('function');
});

test('The function returned by addToSumEffect calls setSum with the expected value', () => {
  const addNumber = 4;
  const sum = 2;
  const setSum = jest.fn();
  const func = addToSumEffect({ addNumber, sum, setSum });
  func();
  expect(setSum).toHaveBeenCalledTimes(1);
  expect(setSum).toHaveBeenCalledWith(6);
});

// component.js

import React, { useState } from 'react';
import PropTypes from 'prop-types';
import { addToSumEffect } from './effects';

const AddButton = (props) => {
  const { initialNumber, addNumber } = props;
  const [ sum, setSum ] = useState(initialNumber);

  return (
    <button onClick={addToSumEffect({ addNumber, sum, setSum })}>
      Add {addNumber} to {sum}
    </button>
  );
};

AddButton.defaultProps = {
  initialNumber: 0,
  addNumber: 1,
};

AddButton.propTypes = {
  initialNumber: PropTypes.number.isRequired,
  addNumber: PropTypes.number.isRequired,
};

export default AddButton;

The code has changed a lot compared to Example 1, so let's walk through it beginning with the component. The component imports addToSumEffect from a separate file and assigns its return value to the button's onClick prop. addToSumEffect is the closure's outer function. Its return value is the closure's inner function, which will be called when the button is pressed. addToSumEffect accepts an options hash containing the current values of addNumber and sum, as well as the setSum function. These arguments are unpacked in the outer function's scope, which makes them available to the inner function.

export const addToSumEffect = (options = {}) => {
  // Unpack arguments from the options hash in the outer function:
  const { addNumber, sum, setSum } = options;
  return () => {
    // The values are scoped into the inner function:
    setSum(add(sum, addNumber));
  };
};

The outer function is called on every render with the current addNumber, sum and setSum values, which generates a new inner function each time. This ensures that, whenever the button is pressed, it has access to the most up-to-date values from the component. This makes the inner function a sort of snapshot of the component values at the time the component was last rendered.

We can break this process down step by step for the sake of clarity:

The component renders
addToSumEffect is called with a hash of the current addNumber, sum and setSum values from the component
addToSumEffect returns a new function with the current addNumber, sum and setSum values in scope
The returned function is assigned to the button's onClick prop
The user presses or clicks the button and the returned function is called
The new sum is calculated from the current sum and addNumber values
The new sum is passed to setSum which updates the sum on the component's state
The component renders and the process begins again with the new value of sum

The behaviour of addToSumEffect should be stable and predictable for any given values of sum and addNumber. We can confirm this with tests.

Testing Effect Functions

Example 3 defines the two tests for addToSumEffect. The first test simply confirms that addToSumEffect returns a function, which means that it conforms to the expected pattern.

test('addToSumEffect returns a function', () => {
  const addNumber = 4;
  const sum = 2;
  const setSum = jest.fn();
  const func = addToSumEffect({ addNumber, sum, setSum });
  expect(typeof func).toEqual('function');
});

The second test calls the returned function, supplying a jest.fn() mock function for setSum, which enables us to test that setSum was called appropriately by the returned function. We expect setSum to have been called only once, with the sum of the addNumber and sum values. If the returned function calls setSum more than once (or not at all) or calls it with the incorrect value, the test will fail.

test('The function returned by addToSumEffect calls setSum with the expected value', () => {
  const addNumber = 2;
  const sum = 4;
  const setSum = jest.fn();
  const func = addToSumEffect({ addNumber, sum, setSum });
  func();
  expect(setSum).toHaveBeenCalledTimes(1);
  expect(setSum).toHaveBeenCalledWith(6);
});

Note that we aren't testing the effect function's internal logic. We only care that setSum is called once with the expected sum. We don't care how the effect function arrives at that result. The internal logic can change as long as the result remains the same.

Using Effect Functions with the `useEffect` Hook

There's one more small enhancement we can make to the component shown in Example 3. Currently, nothing happens if the initialNumber prop changes after the initial mount. If initialNumber changes, I'd like it to be set as the new value of sum on state. We can do that easily by declaring a new effect function called initializeSumEffect as shown in Example 4.

Example 4

// functions.js

export const add = (a, b) => {
  return a + b;
};

// functions.test.js

import { add } from './functions';

test('The add function calculates the sum of two numbers', () => {
  const sum = add(4, 2);
  expect(sum).toEqual(6);
});

// effects.js

import { add } from './functions';

export const addToSumEffect = (options = {}) => {
  const { addNumber, sum, setSum } = options;
  return () => {
    setSum(add(sum, addNumber));
  };
};

// NEW:
export const initializeSumEffect = (options = {}) => {
  const { initialNumber, setSum } = options;
  return () => {
    setSum(initialNumber);
  };
};

// effects.test.js

import { initializeSumEffect, addToSumEffect } from './effects';

// NEW:
test('initializeSumEffect returns a function', () => {
  const initialNumber = 4;
  const setSum = jest.fn();
  const func = initializeSumEffect({ initialNumber, setSum });
  expect(typeof func).toEqual('function');
});

// NEW:
test('The function returned by initializeSumEffect calls setSum with the value of initialNumber', () => {
  const initialNumber = 4;
  const setSum = jest.fn();
  const func = initializeSumEffect({ initialNumber, setSum });
  func();
  expect(setSum).toHaveBeenCalledTimes(1);
  expect(setSum).toHaveBeenCalledWith(initialNumber);
});

test('addToSumEffect returns a function', () => {
  const addNumber = 4;
  const sum = 2;
  const setSum = jest.fn();
  const func = addToSumEffect({ addNumber, sum, setSum });
  expect(typeof func).toEqual('function');
});

test('The function returned by addToSumEffect calls setSum with the expected value', () => {
  const addNumber = 4;
  const sum = 2;
  const setSum = jest.fn();
  const func = addToSumEffect({ addNumber, sum, setSum });
  func();
  expect(setSum).toHaveBeenCalledTimes(1);
  expect(setSum).toHaveBeenCalledWith(6);
});

// component.js

import React, { useState, useEffect } from 'react';
import PropTypes from 'prop-types';
import { initializeSumEffect, addToSumEffect } from './effects';

const AddButton = (props) => {
  const { initialNumber, addNumber } = props;
  const [ sum, setSum ] = useState(initialNumber);

  // New:
  useEffect(initializeSumEffect({ initialNumber, setSum }), [initialNumber]);

  return (
    <button onClick={addToSumEffect({ addNumber, sum, setSum })}>
      Add {addNumber} to {sum}
    </button>
  );
};

AddButton.defaultProps = {
  initialNumber: 0,
  addNumber: 1,
};

AddButton.propTypes = {
  initialNumber: PropTypes.number.isRequired,
  addNumber: PropTypes.number.isRequired,
};

export default AddButton;

Let's break the new additions down step by step:

The component updates with a new value for the initialNumber prop
initializeSumEffect is called with a hash of the current initialNumber and setSum values from the component
initializeSumEffect returns a new function with the current initialNumber and setSum values in scope
The returned function is assigned to the useEffect hook (note that the hook is configured to run only when initialNumber has changed, not on every render)
The component renders
useEffect runs, calling the returned function
The initialNumber value is passed to setSum which updates the sum on the component's state
The component renders

We also have new tests to confirm that initializeSumEffect returns a function, and that the returned function calls setSum with the expected value.

Notice how similar initializeSumEffect is to addToSumEffect despite being used in different contexts. This is one of the benefits of this pattern. It works equally well whether you're working with React hooks, JavaScript event handlers, or both.

A Less Trivial Example: API Integration

The examples above are simple, which made them a good introduction to the effect function pattern. Let's look at how to apply this pattern to more of a real world integration: an asychronous API request that updates component state upon completion.

The basic pattern for this is the same as the previous example. We'll use an effect function to perform the request when the component mounts, then set the response body (or error) on the component state. Everything the effect consumes will be passed in from the component, so the effect function won't have external dependencies that would make it harder to test.

Example 5

// effects.js

export const getDataEffect = (options = {}) => {
  const { url, getJson, setData, setError, setIsLoading } = options;
  return async () => {
    setIsLoading(true);
    try {
      const data = await getJson(url);
      setData(data);
      setError(null);
      setIsLoading(false);
    } catch (error) {
      setError(error);
      setIsLoading(false);
    }
  };
};

// component.js

import React, { useState, useEffect } from 'react';
import { getDataEffect } from './effects';
import { getJson } from './requests';
import { LoadingIndicator } from './loading';
import { DataView } from './data-view';

const DataPage = (props) => {
  const [ data, setData ] = useState({});
  const [ error, setError ] = useState(null);
  const [ isLoading, setIsLoading ] = useState({});

  useEffect(
    getDataEffect({
      url: 'https://api.myapp.com/data',
      getJson,
      setData,
      setError,
      setIsLoading
    }),
    []
  );

  return (
    <div className="data-page">
      {isLoading && <LoadingIndicator />}
      {error && (
        <p className="error-message">
          {error.message}
        </p>
      )}
      {!error && (<DataView data={data} />)}
    </div>
  );
};

export default DataPage;

Note that some elements in Example 5 are not described in detail because they don't fall within the scope of this discussion. getJson is an async function that makes an GET request for some data and returns the data or throws an error. LoadingIndicator is a component that displays loading activity or progress UI. DataView is a component that displays the requested data. I have omitted these from the example so we can focus on the pattern. Let's break down the flow:

The component mounts.
getDataEffect is called with the request url, request function (getJson) and setters for the data, error and isLoading state values. getDataEffect returns an async function.
The useEffect hook calls the async function that was returned by getDataEffect.
The async function sets the loading state to true, which causes the loading indicator to render.
The async function calls getJson with the request url and waits for a response.
Upon receiving a successful response, the async function sets the data on state, the error state to null and the loading state to false. The component stops rendering the loading indicator and passes the data to DataView to be rendered.
If getJson throws an error, the async function sets the error on state and the loading state to false. The component stops rendering the loading indicator and renders an error message.

Next, let's add tests for getDataEffect:

Example 6:

// effects.test.js

import { getDataEffect } from './effects';

test('getDataEffect returns a function', () => {
  const url = 'https://fake.url';
  const getJson = jest.fn();
  const setData = jest.fn();
  const setError = jest.fn();
  const setIsLoading = jest.fn();
  const func = getDataEffect({ url, getJson, setData, setError, setIsLoading });
  expect(typeof func).toEqual('function');
});

test('The function returned by getDataEffect behaves as expected when making a successful request', async () => {
  const url = 'https://fake.url';
  const data = { status: true };

  // Mock the async getJson function to resolve with the data:
  const getJson = jest.fn();
  getJson.mockReturnValue(Promise.resolve(data));

  // Mock the setter functions:
  const setData = jest.fn();
  const setError = jest.fn();
  const setIsLoading = jest.fn();

  // Run the effect:
  const func = getDataEffect({ url, getJson, setData, setError, setIsLoading });
  await func();

  // Test that getJson was called once with the provided url:
  expect(getJson).toHaveBeenCalledTimes(1);
  expect(getJson).toHaveBeenCalledWith(url);

  // Test that setData was called once with the expected data:
  expect(setData).toHaveBeenCalledTimes(1);
  expect(setData).toHaveBeenCalledWith(data);

  // Test that setError was called once with null:
  expect(setError).toHaveBeenCalledTimes(1);
  expect(setError).toHaveBeenCalledWith(null);

  // Test that setIsLoading was called twice, with
  // true the first time and false the second time:
  expect(setIsLoading).toHaveBeenCalledTimes(2);
  expect(setIsLoading.mock.calls[0][0]).toBe(true);
  expect(setIsLoading.mock.calls[1][0]).toBe(false);
});

test('The function returned by getDataEffect behaves as expected when making an unsuccessful request', async () => {
  const url = 'https://fake.url';
  const error = new Error(message);

  // Mock the async getJson function to reject with the error:
  const getJson = jest.fn();
  getJson.mockReturnValue(Promise.reject(error));

  // Mock the setter functions:
  const setData = jest.fn();
  const setError = jest.fn();
  const setIsLoading = jest.fn();

  // Run the effect:
  const func = getDataEffect({ url, getJson, setData, setError, setIsLoading });
  await func();

  // Test that getJson was called once with the provided url:
  expect(getJson).toHaveBeenCalledTimes(1);
  expect(getJson).toHaveBeenCalledWith(url);

  // Test that setData was not called:
  expect(setData).not.toHaveBeenCalled();

  // Test that setError was called once with the error:
  expect(setError).toHaveBeenCalledTimes(1);
  expect(setError).toHaveBeenCalledWith(error);

  // Test that setIsLoading was called twice, with
  // true the first time and false the second time:
  expect(setIsLoading).toHaveBeenCalledTimes(2);
  expect(setIsLoading.mock.calls[0][0]).toBe(true);
  expect(setIsLoading.mock.calls[1][0]).toBe(false);
});

The first test just validates that getDataEffect returns a function. It's the same basic sanity check we've used in all the other examples. The second test validates the entire flow for a successful request:

We define a fake request run and data.
We create a mock function for getJson that returns a promise, which will resolve with the expected data.
We create simple mock functions for the state setters.
We call getDataEffect to obtain the async function.
We call the function and wait for it to return.
We test that getJson was called once with the provided url.
We test that setData was called once with the expected data.
We test that setError was called once with null.
We test that setIsLoading was called twice, with true the first time and false the second time.

The third test validates the entire flow for an unsuccessful (error) request. It's similar to the second test but the expectations are different. The mock getJson function returns a promise, which will reject with an error. setError should be called with that error. setData should not be called.

Wrapping Up

We now have a consistent structure that keeps business logic out of our components and makes our code easier to read. We're also able to write comprehensive tests to validate that our code does the right thing, which can improve confidence in the codebase. (This assumes that you actually run your tests regularly and integrate them into your continuous integration pipeline, but that's a topic for another post.) This is one of many ways to structure your components. I hope it gives you some ideas to establish an architecture that suits your needs.