DEV Community: Miroslav Jonas

Speeding up computations in GitHub/CircleCI workflows with Affected Commands

Miroslav Jonas — Thu, 25 Aug 2022 08:47:01 +0000

One of the most exciting features of Nx is using the project graph to run the target commands. This article exposes a bit of magic behind the affected commands, explains the common use cases, and shows you how to effectively use them in your Continuous Integration (CI) environment. Finally, we will present you with two CI utilities we created recently that will help you solve this issue.

The affected commands in a nutshell

This feature saves tremendous processing time by running a target only on the projects that were affected by the change. But it's not always trivial what to choose as a reference point for change detection.

Let's assume our solution consists of one application, two UI libraries, and one utility library. The application uses both UI libraries and the second UI library uses the utility library. The graph for this structure looks something like this:

If we were to modify app.ts only the application would be affected. However, if we change lib.ts then not only would the utility library be affected, but also UI 2 and App. The affected command encapsulates the logic of detecting which project nodes were affected by the change.

The command has a simple form:

nx affected --target={your chosen target e.g. "test"} {...options}

The target is defined in the workspace.json (lint, test, build...). Affected has several handy options, such as parallel (to run tasks sequentially or in parallel), but the two most important ones are base and head. They are essential in providing information about the scope of the change.

Here's our Egghead tutorial showing the Nx affected commands in action:

https://egghead.io/lessons/javascript-scale-ci-runs-with-nx-affected-commands

Change detection logic

Let's look at this excerpt showing the logic for detecting which files were changed:

if (files) {
  return { files };
} else if (uncommitted) {
  return { files: getUncommittedFiles() };
} else if (untracked) {
  return { files: getUntrackedFiles() };
} else if (base && head) {
  return { files: getFilesUsingBaseAndHead(base, head) };
} else if (base) {
  return {
    files: Array.from(
      new Set([
        ...getFilesUsingBaseAndHead(base, 'HEAD'),
        ...getUncommittedFiles(),
        ...getUntrackedFiles(),
      ])
    ),
  }

If the user explicitly provided a list of files or requested only uncommitted or untrackedfiles, then only these files would be used to detect affected projects. We can also see, that if head is not provided, the default value HEAD will be used. A side-effect effect of not providing the head is that affected logic will also check for untracked and uncommitted files. Obviously, this only makes sense locally where you expect some untracked or uncommitted changes.

Let's see now what base defaults to:

base = nxJson.affected?.defaultBase || 'master';

We use git merge-base to find all the files changed between the base and the head:

If you are running affected on a branch, it will compare the point where the branch was created with the latest commit in the branch along with any of your uncommitted changes

On the diagram above, when we run nx affected (or nx affected --base=main --head=HEAD) we are comparing are all the files changed between commits #22 (orange) and #2 (blue).

If you are running affected on the main branch, your base and head points to the same commit. The most obvious choice for main branch would be to use HEAD~1 as base:

nx affected --target={target} --base=HEAD~1 --head=HEAD

which compares the latest commit with the previous one.

The problem

Take the above commit graph as an example. Our CI runs nx affected --target=build **on each new commit. Let's assume that **commit #3 contained a code that broke the Lib's build and now our main branch is in broken state. Another developer pushed the commit #4 that only changes the UI 2. Running affected would compare commits #3 and #4 and would run targets only on the App and UI 2. The CI would falsely report that the main branch is now fixed, because the last check was successful.

The commits therefore can't just be HEAD and HEAD~1. If a few workflows fail one after another, that means that we're accumulating a list of affected projects that are potentially still broken.

The solution

We want to include every commit since the last time we had a successful run. That way we ensure we don't accidentally skip checking the broken project. What we essentially want is to use the last successful commit on main branch as the base:

nx affected --target={target} --base={last successful commit} --head=HEAD

Luckily most of the CI platforms already provide APIs needed to detect the last successful runs, and using git we can extract the exact commits.

CircleCI Orb and GitHub Action

When using affected in our CI platforms, the usual steps in our pipelines are:

Checkout the code
Install dependencies
Run the command(s)

Most likely we would have two separate workflows/pipelines - one for the main branch using HEAD~1 as base and one for the PR branches using main branch as the base. We want to inject another step before the run command that would find the last successful commit and store it in the environment variable so that affected commands can use it.

For CircleCI we created the nrwl/nx orb that includes a useful set-shas command for deriving commit SHAs of HEAD and last successful commit:

version: "2.1"
orbs:
    nx: nrwl/nx@1.0.0
jobs:
    build:
        docker:
            - image: cimg/node:14.17-browsers
        steps:
            # ...CI steps like checkout, install...

            - nx/set-shas
            - run:
                command: nx affected --target=build --base=$NX_BASE

            # ...other CI steps

Similarly, on GitHub we provide nx-set-shas action that gives equal functionality for GitHub workflows:

jobs:
  check:
    runs-on: ubuntu-latest
    name: Check branch
    steps:
      # ...CI steps like checkout, install...

      - name: Derive appropriate SHAs for base and head for `nx affected` commands
        uses: nrwl/nx-set-shas@v2

      - run: nx affected --target=build --base=${{ env.NX_HEAD }}

      # ...other CI steps

Summary

Using affected can speed up your computation times but unless you use it properly it can also be a source of escaped issues.

In this post we looked at how the affected command works and how to use it, and what is the common issue when using it on the CI.

Our CircleCI orb and Github Action helps you to compare always against the last successful build and make sure you protect against breakages from committed code.

The Butterfly Effect - How we gave the linter a 100x boost

Miroslav Jonas — Thu, 25 Aug 2022 08:45:43 +0000

Recently we had a significant performance improvement in our linter rule. In some benchmarks, the increase was up to 100x! While one would expect such a dramatic change to result from implementing a sophisticated algorithm, the reality is that this resulted from minor tweaks.

Like the butterfly effect - changing a few lines of code created a ripple effect that tremendously improved the overall performance.

Discovery of the problem

We always knew we could improve the performance of the linter rule. Like in other parts of Nx, we keep finding ways to shave off another 100ms here and there. Improving performance has been an ongoing process.

While implementing a new feature, we realized this new feature was adding 20% on top of the rule's duration. That performance impact could not be justified, even for critical features, let alone a nice-to-have one. So we revisited the code of the implementation.

Not all loops are born equal

At a first glimpse, nothing seemed off. The code was logical, didn't have unnecessary nested loops, and used modern Array higher-order functions - filter, map, reduce, which helped the code be shorter and easier to read. We also used the popular trick with Set to extract the unique values from the array.

The code looked like this (we replaced some functions with pseudo-code for simplicity):

const allReachableProjects = Object.keys(graph.nodes)
  .filter(pathExistsBetweenSourceAndNode && graph.dependencies[project]);

const externalDependencies = allReachableProjects
  .map((project) =>
    graph.dependencies[project].filter(
      isDependencyExternal
    )
  )
  .filter((deps) => deps.length)
  .flat(1);

return Array.from(new Set(externalDependencies));

We filtered all the nodes reachable from our source node that had dependencies. We then further mapped each of those projects to arrays of their external dependencies, filtered out those that didn't have any external dependencies, and finally flattened the two-dimensional array to a plain array.

The last line - Array.from(new Set(…)) uses the above-mentioned trick to filter out duplicates.
The higher-order functions are a very nice way to encapsulate otherwise boilerplate code, but they come with a hidden cost. Each iteration of map, filter, and reduce creates a new array by combining items from the previous iteration and adding a new element.

This led to a hidden inner loop. What initially looked like a single loop, in reality, were two loops:

iterating over elements of the source array
iterating over elements of the temporary array in every step

To avoid this, we used for loops with Array.push. We start with an empty reachableProjects array, and every time project satisfies the condition, we push it to our array. We applied the same logic to externalDependencies. Instead of a map, filter, and flat, we use loops and if conditions. We introduce a hash table to ensure no duplicates in our resulting collection. Once we add the project to the array, we mark it in our hash table with true, so we can quickly check if we have added it already.

This is what the resulting code looked like:

if (!graph.externalNodes) {
  return [];
}
const reachableProjects = [];
const projects = Object.keys(graph.nodes);

for (let i = 0; i < projects.length; i++) {
  if (pathExistsBetweenSourceAndNode) {
    reachableProjects.push(projects[i]);
  }
}

const externalDependencies = [];
const externalDependenciesMap = {};
for (let i = 0; i < reachableProjects.length; i++) {
  const dependencies = graph.dependencies[reachableProjects[i]];
  if (dependencies) {
    for (let d = 0; d < dependencies.length; d++) {
      const dependency = dependencies[d];
      if (graph.externalNodes[dependency.target] && !externalDependenciesMap[dependency.target]) {
        externalDependencies.push(dependency);
        externalDependenciesMap[dependency.target] = true;
      }
    }
  }
}

return externalDependencies;

The benchmark showed a 2–3% slowdown, significantly better than the initial 20%. Combined with an optional toggle, this was now acceptable. The success with improving performance motivated us to check the remaining rule code and see if we can apply a similar approach elsewhere.

Single shelf principle

Unfortunately, there weren't too many filter or map functions to replace, and those there didn't make much difference. Most of the slowdown was coming from two functions:

finding the source project based on the file
finding the target project based on the import

Both were iterating the graph, looking for a project that contained a given file (import target was either mapped to file if it was a relative import or mapped to barrel entry file in case of cross-project imports).

The graph was already improved earlier for linting and used a slightly more ingenious structure than the default graph.

Before:

{
  "nodes": {
    "project-a": {
      // ... other fields
      "data": {
        // ... other fields
        "files": [
          { 
            "file": "packages/project-a/a.ts", 
            "hash": "...", 
            "ext": "ts"
          },
          // ...
        ]
      }
    },
    // ... other projects
  },
  "externalNodes": { ... },
  "dependencies": [...]
}

After (notice the files section):

{
  "nodes": {
    "project-a": {
      // ... other fields
      "data": {
        // ... other fields
        "files": {
          "packages/project-a/a": { 
            "file": "packages/project-a/a.ts", 
            "hash": "...", 
            "ext": "ts" 
          },
          // ...
        }
      }
    },
    // ... other projects
  },
  "externalNodes": { ... },
  "dependencies": [...]
}

Unfortunately, while this gave great results if your monorepo consisted of a large monolith app and several small additional libraries, importing from the last project in the graph in a monorepo with hundreds of projects was still very slow. We would iterate through all the projects repeatedly and check which project contained that file.

The solution was simple. Instead of modifying the files section of each project from array to lookup table, we added a new field to the graph that contained a table with all the files' paths, each pointing to its parent project:

{
  "nodes": {
    "project-a": {
      // ... other fields
      "data": {
        // ... other fields
        "files": [
          { 
            "file": "packages/project-a/a.ts", 
            "hash": "...", 
            "ext": "ts"
          },
      // ...
        ]
      }
    },
    // ... other projects
  },
  "externalNodes": { ... },
  "dependencies": [...],
  "allFiles": {
    "packages/project-a/a": "project-a",
    // ...
  }
}

Forgotten O(n²) spread

There were reports from some users that the linter is too slow for them. One of our clients reported incredible values where the enforce-module-boundaries rule would run for a minute per project and consumed up to 96% of all the linting work.

We were excited to test this new improvement on their solution. To our surprise, the improvement was "only" 6x. Linting the project still took almost 10sec.

We started digging and time-tracking each code block using the console.time until we realized that almost all that time was spent checking for the circular dependencies. The code that checked for circular dependencies was straightforward - we have an adjacency matrix:

Given each pair of the projects (A, B), if project B is reachable from project A, we mark that field in the matrix with true
To check if there is a circular dependency, we check if A is reachable from B, where B is a direct dependency of A

So that couldn't have been it. Also, the matrix was only generated once and used from memory until the graph changed. It turned out that single matrix generation was extremely slow in large monorepos with thousands of projects.
The code looked like this:

function buildMatrix(graph) {
  const dependencies = graph.dependencies;
  const nodes = Object.keys(graph.nodes);
  const adjList = {};
  const matrix = {};

  const initMatrixValues = nodes.reduce((acc, value) => {
    return { ...acc, [value]: false };
  }, {});

  for (let i = 0; i < nodes.length; i++) {
    const v = nodes[i];
    adjList[v] = [];
    matrix[v] = { ...initMatrixValues };
  });

  for (let proj in dependencies) {
    for (let dep of dependencies[proj]) {
      if (graph.nodes[dep.target]) {
        adjList[proj].push(dep.target);
      }
    }
  }

  const traverse = (s: string, v: string) => {
    matrix[s][v] = true;

    for (let adj of adjList[v]) {
      if (matrix[s][adj] === false) {
        traverse(s, adj);
      }
    }
  };

  nodes.forEach((v) => {
    traverse(v, v);
  });

  return {
    matrix,
    adjList,
  };
}

As you might have learned in this article, using reduce isn't the smartest thing to do when you care about performance, so our first step was to replace reduce with for loop and Array.push. We then focused on the usual suspect - the recursion in the traverse function. But all potential improvements failed to make a difference. So we again added console.time to establish what was going on. It turned out, that the problematic line was:

matrix[v] = { ...initMatrixValues };

So what was going on there?

We first create an initMatrixValues object, a default hash map where the project name points to false. We then loop through the nodes and set the values in the matrix where each project points to a copy of the initMatrixValues. This seemed like a good idea.

Unfortunately, we forgot that spreading the object iterates through all its members, effectively making another inner loop in the forEach. It also creates a temporary object each step and copies all the members to the new object every step of the way. You guessed right - this adds another loop, effectively giving O(n³) complexity.

The first step in the optimization was to generate matrix values for every node in an inner loop instead of spreading the prebuilt one.

for (let i = 0; i < nodes.length; i++) {
  const node = nodes[i];
  adjList[node] = [];
  const initMatrixValues = {};
  for (let j = 0; j < nodes.length; j++) {
    initMatrixValues[nodes[j]] = false;
  });
  matrix[node] = { ...initMatrixValues };
});

And then Stefan, who helped with the performance investigation, had a brilliant revelation - we explicitly use false to mark that there is no connection between two nodes. Still, we could also assume that no value means there is no connection. Instead of generating an object with pre-filled nodes with an explicit false value, we could start with an empty object. This finally brought us to ideal O(n) and gave us 100x improvement in the original run (the function itself is exponentially faster, as seen in this benchmark).
The code now looked like this:

for (let i = 0; i < nodes.length; i++) {
  const node = nodes[i];
  adjList[node] = [];
  matrix[node] = {};
});

The original tweet showed the performance improvement numbers on the client's project:

Miro

@meeroslav

🤫 Coming soon to your @NxDevTools monorepo

More info later...

10:01 AM - 21 Jul 2022

Conclusion

In this article, we showed how small changes could make an enormous impact on your code performance. Even a simple change like slice instead of replace + regex can reduce time. Modern higher-order functions make your code look slicker, but this often comes with a price tag.

If you are looping over a small collection, you will most likely never notice any difference from introducing such micro-optimizations.

However, suppose your code iterates over an extensive collection and does that repeatedly, as our rule does. In that case, you might want to investigate whether you can optimize the code by using loops and lookup tables.

You don't need to know the performance of different native implementations by heart. Just think of all the possible ways you can achieve a specific task and run it through a benchmark to see which one performs best. It's a fun experiment, and your users will be grateful for it.

Credits

Thanks to Stefan Van de Vooren, who helped benchmark the improvements on his company's repository and provided valuable inputs, and whose company brought to our attention the severity of the performance issue on the linter. Thank you goes also to Juri Strumpflohner for the valuable feedback while reviewing this article.

Taming Code Organization with Module Boundaries in Nx

Miroslav Jonas — Wed, 01 Jun 2022 12:13:48 +0000

As your repository grows, it becomes more challenging to organize and name the applications and libraries. This organization, when done right, feels intuitive and allows team members to find projects and understand how they work together easily. When done poorly, it results in a mess we eventually call “the legacy software”. This article will show you different ways how you can prevent your repo from descending into chaos.

Our book Enterprise Angular Monorepo Patterns presents an in-depth guide to assist you with naming and organization. If you still haven’t read this book, we warmly recommend you do. Don’t let the name fool you, though — the architecture guidelines explained in this book apply to any framework.

On large projects, you will most likely find multiple teams working on different parts of the solution. Those projects are usually split into logical domains, where each team focuses on a single domain. Each domain block can have a clear public API which other domains can use to consume the information.

But the code organization is just one piece of the puzzle. The physical organization does not prevent developers from consuming the domains or parts of those domains, that should otherwise be outside of their reach. Nx ships with enforce-module-boundaries ESLint rule that helps restrict that possibility.

Understanding the default configuration

When you generate the first project in your workspace using one of our generators, one of the things you get for free is the full linter setup. The linter is preconfigured with a default ruleset that includes a set of best practices. Alongside the standard set of rules, the initial generated configuration includes a setup for enforce-module-boundaries rule.

Let’s dissect what each of these properties does.

The allow array acts as a whitelist listing the import definitions that should be omitted from further checks. You can read more about it in the Overriding the overrides section below.

The depConstraints section is the one you will be spending most time fine-tuning. It represents an array of constraints, each consisting of sourceTag and onlyDependOnLibsWithTags properties. The default configuration has a wildcard * set as a value for both of them, meaning that any project can import (depend on) any other project.

Note, the wildcard only applies to libraries. Applications and E2E applications cannot be imported. It wouldn’t make any sense. If you want to combine applications, you should use the micro-frontends approach with the module federation.

The circular dependency chains such as lib A -> lib B -> lib C -> lib A are also not allowed. The self circular dependency (when lib imports from a named alias of itself), while not recommended, can be overridden by setting the flag allowCircularSelfDependency to true.

Finally, the flag enforceBuildableLibDependency prevents us from importing a non-buildable library into a buildable one. You can read more on what buildable libraries are used for in our docs.

Using tags to enforce boundaries

To best express the need for the boundaries and assist us through the explanation, we will be using the repository represented by the following graph:

Our repository consists of two applications — Store and Admin. Each of them is composed of several feature libraries — Products (for Store), Sales, and Invoices (for Admin). Also, both of these applications depend on the Core library, and every project in our repo depends on our Shared library. Using common sense, we would like to enforce certain boundaries:

A shared or core library should not be able to depend on a feature library
A feature library can depend on another feature library or a shared library

First, we will use the project configuration to annotate our projects with tags.

Tags used to live in nx.json but were in the recent version moved closer to the project, so you can locate them now in your project.json or workspace.json

Let’s define the types of projects. We will use the following tags:

type:app for application
type:feature for feature library
type:util for utility library

Your changed project configuration should now have the tags section defined.

Your enhanced graph will now look similar to this:

The above list of library types is not complete. You might add specific ones for E2E projects or UI component libraries. Using the naming format type:* is just a suggestion. Consider this being a hashtag on your favorite social app. You can use any prefix or format you feel fitting. The important thing is that it's readable and intuitive to all the members of your team.

Now, that we have marked all of our projects, we can continue to define the rules in the root .eslintrc.json.

Adding a second dimension

We said that a feature library can depend on any other feature library, but there is a small catch. Our two apps could be built with a different framework so mixing feature libraries would not be possible. To avoid any future impediments, we don’t want to allow a feature library used in Store to depend on the feature library from Admin and vice versa. Additionally, only our apps should be able to load the Core library.

Let’s add another dimension to allow such restrictions. We will define the necessary scope tags:

scope:store for store app-related projects
scope:admin for admin app related projects
scope:shared for shared projects
scope:core for core projects

Our diagram should now look like this:

Let us now define our missing rules!

Fine-grained external dependencies

You may want to constrain what external packages a project may import. In our example above, we want to make sure projects in the scope:store does not import any angular packages, and projects from the scope:admin do not import any react library. You can ban these imports using bannedExternalImports property in your dependency constraints configuration.

We can now enhance our rule configuration by providing additional information.

Using the wildcard * to match multiple projects e.g. react* we can save ourselves the effort of manually specifying every single project we want to ban.

Restricting transitive dependencies

Our solution doesn’t contain only internal projects but also depends on various external NPM packages. These external dependencies are explicitly declared in our package.json. Unfortunately, a package is rarely an island. They often consist of a tree of transitive dependencies branching out leading to thousands of packages being installed in your node_modules folder. Although we have control over what version or which direct dependency we install, we have no control over what versions of what packages this dependency depends on. The transitive dependencies are often the source of our app's vulnerabilities. We can also never guarantee those dependencies will be there. Just by simply running npm install parent may get updated to a patch or minor version that would wipe out one of the transitive dependencies or replace it with one with breaking changes.

Therefore it’s wise not to allow developers to import transitive dependencies in their projects. Our ESLint plugin provides a simple flag to turn this restriction on.

If you now try to import a transitive dependency, your linter responds with an error. This flag is disabled by default for now, but we highly recommend you enable it.

Overriding the overrides

Sometimes, we just need to override this configuration for a given project. The scenario for this might be testing or during the development, if we are unsure yet how a certain project will be tagged. While we strongly encourage you to plan your architecture carefully and never override the boundaries configuration, you still have an option to bale out and override it.

Summary

Monorepos are often viewed only from the technical side, but they also bring a shift in human resources organization. Teams that were once isolated, now have to work together on the same solution.

Having a clean separation of concerns and well-defined cohesive units helps us scale our organization more easily and gives us more confidence in our architecture. not only does Nx provide tools to speed up the overall performance, but it provides tooling to enforce the organizational constraints in an automated way.

In this post, we listed several strategies which you can use to restrict your packages from being misused or use unplanned external resources. Use them thoroughly and on time, before things go out of hand.

Prefer a visual presentation over text? Then check out this talk recording by Juri Strumpflohner https://www.youtube.com/watch?v=pER_Ak1yUaA&t=687s

Parsing ISO 8601 duration

Miroslav Jonas — Thu, 29 Oct 2020 09:15:09 +0000

Cover photo by Nick Hillier on Unsplash

ISO 8601 is the international standard document covering date and time-related data. It is commonly used
to represent dates and times in code
(e.g. Date.toISOString). There is one less known specification in this standard related to duration.

What is duration standard?

Duration defines the interval in time and is represented by the following format:

P{n}Y{n}M{n}W{n}DT{n}H{n}M{n}S

Letters P and T represent, respectively, makers for period and time blocks. The capitals letters Y, M, W, D, H, M, S represent the segments in order:
years, months, weeks, days, hours, minutes, and seconds. The {n} represents a number. Each of the duration
segments are optional.

The following are all valid durations:

P3Y - 3 years
P24W6D - 24 weeks, 6 days
P5MT7M - 5 months, 7 minutes
PT3H5S - 3 hours, 5 seconds

Human-readable format

Using the specification it's easy to implement a parser that would parse ISO standard into human-readable form.
First, we need the regex that would extract the necessary segments:

/P(?:(\d+)Y)?(?:(\d+)M)?(?:(\d+)W)?(?:(\d+)D)?(?:T(?:(\d+)H)?(?:(\d+)M)?(?:(\d+)S)?)?$/

Let's dissect this regex to understand what it does:

First character P matches P literally
Group (?:(\d+)Y)? is non-capturing group (due to ?: modifier)
- The group can 0 or 1 appearances (due to ? at the end)
- The inner part (\d+)Y matches 1 to many digits followed by Y
- The digits part (\d+) is a capturing group (due to surrounding brackets)
Same logic applies for (?:(\d+)M)?, (?:(\d+)W)? and (?:(\d+)D)?
Group (?:T(?:(\d+)H)?(?:(\d+)M)?(?:(\d+)S)?)? is also non-capturing group
- Group starts with T literal
- The group is optional (due to ? at the end)
- Group consist on sub-groups (?:(\d+)H)?, (?:(\d+)M)? and (?:(\d+)S)? to which the above mentioned logic applies

If we run this regex on an arbitrary string it will try to match P at the beginning and then extract numbers for
years, months, weeks, days, hours, minutes, and seconds. For those that are not available, it will return undefined.
We can use array destructuring in ES6 to extract those values:

const REGEX = /P(?:(\d+)Y)?(?:(\d+)M)?(?:(\d+)W)?(?:(\d+)D)?(?:T(?:(\d+)H)?(?:(\d+)M)?(?:(\d+)S)?)?$/;

function parseDuration(input: string) {
  const [, years, months, weeks, days, hours, minutes, secs] = input.match(
    REGEX
  );

  // combine the values into output
}

We can use those values to export something like 3 years 5 days 23:11:05. We will first
create an array of parsed segments:

  [3, undefined, undefined, 5, 23, 11, 5] -> ['3 years', '5 days', '23:11:05']

And then simply flatten/join the array using whitespace. Parsing time has an additional logic:

we return the time segment only if at least one of hours, minutes or seconds is specified (and different than 0)
we map every time subsection into two digits signature

Here is the full parser function:

const REGEX = /P(?:(\d+)Y)?(?:(\d+)M)?(?:(\d+)W)?(?:(\d+)D)?(?:T(?:(\d+)H)?(?:(\d+)M)?(?:(\d+)S)?)?$/;

export function parseDuration(input: string) {
  const [, years, months, weeks, days, hours, mins, secs] =
    input.match(REGEX) || [];

  return [
    ...(years ? [`${years} years`] : []),
    ...(months ? [`${months} months`] : []),
    ...(weeks ? [`${weeks} weeks`] : []),
    ...(days ? [`${days} days`] : []),
    ...(hours || mins || secs
      ? [
          [hours || '00', mins || '00', secs || '00']
            .map((num) => (num.length < 2 ? `0${num}` : num))
            .join(':'),
        ]
      : []),
  ].join(' ');
}

// usage
parseDuration('P2Y'); // -> 2 years
parseDuration('PT12H34M'); // -> 12:34:00
parseDuration('P4WT5M'); // -> 4 weeks 00:05:00

Extra: Angular Pipe

Wrapping the above function into an angular pipe is straight forward:

import { Pipe, PipeTransform } from '@angular/core';
import { parseDuration } from './parse-duration'; // our parser function

@Pipe({
  name: 'duration',
  pure: true,
})
export class DurationPipe implements PipeTransform {
  transform(value: string): string {
    return parseDuration(value);
  }
}

We can now use our pipe in the template:

{{ input | duration }}

Understanding the structure of the ISO 8601 standard allowed us to easily parse the segments and then construct the
mapper that would map the segments into the desired format. With minimal changes, it's easy to construct
a parser that would map duration into different output string or add localization and internationalization.

Today I learned - Sentence case in CSS

Miroslav Jonas — Wed, 09 Sep 2020 10:19:12 +0000

Cover photo by Patrick Tomasso on Unsplash

This post was originally published on my website.

CSS provides an amazing toolset for style manipulation, but when it comes to text manipulation you often need to resort to JavaScript for additional help. However, CSS has many tricks up his sleeve you might not be aware of.

One thing that can be irritating is when backend API (over which you might not have full control) responds with different casing than expected. It might not even be your backend's fault - perhaps the user forgot to switch off caps lock before submitting that message.

CSS has two builtin options to change the case:

text-transform: lowercase
text-transform: uppercase

But what if we need to change the casing to sentence case (first letter uppercase, rest lowercase).

Turns out, CSS provides pseudo selectors that can target the first letter.

.sentence-case {
  text-transform: lowercase;
}
.sentence-case::first-letter {
  text-transform: uppercase;
}

Which renders this:

<p>THIS TEXT IS ALL CAPITAL</p>
<p>this one is all lowercase</p>
<p>aNd THiS oNE iS MIXeD</p>

Into this:

This text is all capital
This one is all lowercase
And this one is mixed

One important thing to know is that first-letter pseudo selector ONLY works on block elements (p, div, li, etc.).

If you need to use it on inline elements (e.g. span, b) you will have to set display: inline-block:

span.sentence-case {
  display: inline-block;
}

Immutable arrays and objects

Miroslav Jonas — Tue, 08 Sep 2020 19:57:33 +0000

Cover photo by Dušan Smetana on Unsplash

Recently there has been an explosion of popularity of libraries like Redux and NGRX. One common requirement they have is an immutable state. The state of the application is a result of a list of actions sequentially applied to the initial state. Each state of the application is unchangeable. A new action uses the existing state to calculate a new one. This helps us to avoid accidental state changes via mutable operations. It also allows us to investigate which actions led to our current state.

Normally, we describe states through objects and arrays:

const state = {
  userName: 'jdoe',
  favouriteColours: ['blue', 'orange', 'green'],
  company: 'UltimateCourses',
  skills: ['javascript', 'react', 'vue', 'angular', 'svelte']
};

Even simple state changes, normally done with two-way binding (e.g. v-model in Vue or ngModel in Angular), could benefit from the immutable approach. We do this by making a copy of the component's input, mutating the copy and output the mutated copy to the caller. This largely reduces the potential for side effects.

Common state action is to add or remove items from an array or to add or remove fields from an object. However, the standard operations are mutating the original object. Let's see how we can apply them in an immutable way. Our goal is to create a new object, rather than changing the existing. For simplicity, we will be using rest and spread operators introduced in ES6, but all this is possible (albeit less elegantly) with ES5 functions as well.

Immutable array operations

Array has several mutable operations - push, pop, splice, shift, unshift, reverse and sort. Using them is usually causing side effects and bugs that are hard to track. That's why it's important to use an immutable way.

Push

Push is an operation that adds a new item on top of the array.

const fruits = ['orange', 'apple', 'lemon'];
fruits.push('banana'); // = ['orange', 'apple', 'lemon', 'banana']

The resulting array is a concatenation of the original array and the item. Let's try to accomplish that in an immutable way:

const fruits = ['orange', 'apple', 'lemon'];
const newFruits = [...fruits, 'banana']; // = ['orange', 'apple', 'lemon', 'banana']

The spread operator ... here is 'spreading' the items of the array as arguments.

Unshift

Unshift is an operation similar to push. However, instead of adding the item at the end we will prepend the item at the beginning of the array.

const fruits = ['orange', 'apple', 'lemon'];
fruits.unshift('banana'); // = ['banana', 'orange', 'apple', 'lemon']

Similarly, we will use a spread operation to achieve immutability, but with a slight modification:

const fruits = ['orange', 'apple', 'lemon'];
const newFruits = ['banana', ...fruits]; // = ['banana', 'orange', 'apple', 'lemon']

Pop

Pop is an operation that removes the last item from the end of the array and returns it.

const fruits = ['orange', 'apple', 'lemon', 'banana'];
const lastFruit = fruits.pop(); // = 'banana', fruits = ['orange', 'apple', 'lemon']

To remove the item in an immutable way we will use slice. Note that we are making a copy of the last item before this operation. If the copy is not needed we can skip the second line, of course.

const fruits = ['orange', 'apple', 'lemon', 'banana'];
const lastFruit = fruits[fruits.length - 1]; // = 'banana'
const newFruits = fruits.slice(0, fruits.length - 1); // = ['orange', 'apple', 'lemon']

Shift

The shift is an operation similar to pop, but instead of removing the item from the end we remove the item from the beginning of the array.

const fruits = ['orange', 'apple', 'lemon', 'banana'];
const firstFruit = fruits.shift(); // = 'orange', fruits = ['apple', 'lemon', 'banana']

Our immutable solution is equivalent to the immutable pop. We don't have to specify the end limit of slice operation if we want to take all items until the end.

const fruits = ['orange', 'apple', 'lemon', 'banana'];
const firstFruit = fruits[0]; // = 'orange'
const newFruits = fruits.slice(1); // = ['apple', 'lemon', 'banana']

Removal and inserting of items

To add or remove an item from an array, we usually use splice.

const fruits = ['orange', 'apple', 'lemon', 'banana'];
// remove two items from position 1, and replace it with 'strawberry'
fruits.splice(1, 2, 'strawberry'); // = ['orange', 'strawberry', 'banana']

Combined slice and spread gives us the same result, but in an immutable fashion:

const fruits = ['orange', 'apple', 'lemon', 'banana'];
const newFruits = [...fruits.slice(0, 1), 'strawberry', ...fruits.slice(3)]; // = ['orange', 'strawberry', 'banana']

Sort and reverse

Sort and reverse are operators that, respectively, sort and invert the array's items order.

const fruits = ['orange', 'apple', 'lemon', 'banana'];
fruits.sort(); // = ['apple', 'banana', 'lemon', 'orange'];
fruits.reverse(); // = ['orange', 'lemon', 'banana', 'apple'];

Both, sort and reverse, are mutable in nature. However, using spread, we can make a copy of the array so the mutation happens on the copy, instead of the original array.

const fruits = ['orange', 'apple', 'lemon', 'banana'];
const sorted = [...fruits].sort(); // = ['apple', 'banana', 'lemon', 'orange'];
const inverted = [...fruits].reverse(); // = ['banana', 'lemon', 'apple', 'orange'];
const sortedAndInverted = [...sorted].reverse(); // = ['orange', 'lemon', 'banana', 'apple'];

Thanks to the immutability, we can now separate sorting from inversion. As a result, we have all four variants (including the original array) available.

Immutable object operations

State objects tend to grow in applications. However, for certain functionality of the application, we don't need the full state. Usually, we change a small portion of the object and then merge it back. Let's learn how to split and change the object, without affecting the original.

Modify and/or add property

Let's say we want to change the selected fruit and set the new quantity. The standard way to do it is by mutating the object.

const state = {
  selected: 'apple',
  quantity: 13,
  fruits: ['orange', 'apple', 'lemon', 'banana']
};
state.selected = 'orange';
state.quantity = 5;
state.origin = 'imported from Spain';
/*
state = {
  selected: 'orange',
  quantity: 5,
  fruits: ['orange', 'apple', 'lemon', 'banana'],
  origin: 'imported from Spain'
}
*/

Again, we can leverage the spread operator to create a copy of the object with fields changed. The spread here is, similar to array, spreading the key-value pairs of the original object onto a new one. With the next two lines, we are overriding the values from the original object. The last line is creating a new field called 'origin'.

const state = {
  selected: 'apple',
  quantity: 13,
  fruits: ['orange', 'apple', 'lemon', 'banana']
};
const newState = {
  ...state,
  selected: 'orange',
  quantity: 5,
  origin: 'imported from Spain'
};
/*
newState = {
  fruits: ['orange', 'apple', 'lemon', 'banana'],
  selected: 'orange',
  quantity: 5,
  origin: 'imported from Spain'
}
*/

Remove a property

To remove an object's property in a mutable way, we will simply call delete:

const state = {
  selected: 'apple',
  quantity: 13,
  fruits: ['orange', 'apple', 'lemon', 'banana']
};
delete state.quantity;
/*
state = {
  selected: 'apple',
  fruits: ['orange', 'apple', 'lemon', 'banana']
}
*/

Removing a property in an immutable way requires a little trick provided by the spread's counterpart rest. The rest operator is written in the same way as spread - with .... However, the meaning, in this case, is not to spread all the fields, but rather the remaining ones.

const state = {
  selected: 'apple',
  quantity: 13,
  fruits: ['orange', 'apple', 'lemon', 'banana']
};
const { quantity, ...newState } = state;
/*
quantity = 13
newState = {
  selected: 'apple',
  fruits: ['orange', 'apple', 'lemon', 'banana']
}
*/

This technique is called destructuring assignment as we are unpacking the original state object. We assign the quantity key-value pair to constant quantity and assign the rest of the object to newState.

Complex structures

Complex structures have nested arrays or objects. In the following example, state has nested array gang.

const state = {
  selected: 4,
  gang: [
    'Mike',
    'Dustin',
    'Lucas',
    'Will',
    'Jane'
  ]
};
const newState = { ...state };
newState.selected = 11;
newState.gang.push('Max');
newState.gang.push('Suzie');
/*
state = {
  selected: 4,
  gang: [
    'Mike',
    'Dustin',
    'Lucas',
    'Will',
    'Jane'
    'Max',
    'Suzie'
  ]
}
newState = {
  selected: 11,
  gang: [
    'Mike',
    'Dustin',
    'Lucas',
    'Will',
    'Jane'
    'Max',
    'Suzie'
  ]
}
state.gang === newState.gang
*/

Not what we expected, right? Performing spread operation on the complex structures makes just a shallow (first level) copy of the structure. Here it only copied the reference to the gang array, not the actual array. Adding new elements to the array influenced both state and newState. To solve this we need to spread the array separately.

const newState = {
  ...state,
  gang: [...state.gang]
};

However, gang could also be a complex structure (e.g. array of objects). If we change one of the objects underneath, it will change in both arrays.

const state = {
  selected: 4,
  gang: [
    { id: 1, name: 'Mike' },
    { id: 2, name: 'Dustin' },
    { id: 3, name: 'Lucas' },
    { id: 4, name: 'Will' },
    { id: 11, name: 'Jane' }
  ]
}
const newState = {
  selected: 11,
  gang: [...state.gang]
}
newState.gang[4].name = 'Eleven';
/*
state = {
  selected: 4,
  gang: [
    { id: 1, name: 'Mike' },
    { id: 2, name: 'Dustin' },
    { id: 3, name: 'Lucas' },
    { id: 4, name: 'Will' },
    { id: 11, name: 'Eleven' }
  ]
}
newState = {
  selected: 11,
  gang: [
    { id: 1, name: 'Mike' },
    { id: 2, name: 'Dustin' },
    { id: 3, name: 'Lucas' },
    { id: 4, name: 'Will' },
    { id: 11, name: 'Eleven' }
  ]
}
*/

One solution would be to spread also every gang member object, but this can go on forever. Also, we might not know how many levels there are. Not to worry, as there is a trick that handles all those cases.

Calling JSON.parse(JSON.stringify(obj)) makes a deep clone of an object. It converts an object to a string representation and then parses it back to a new object. All references from the original object remain intact.

In most cases, of course, spread on the first level is enough. But we need to be aware of this peculiar behavior to circumvent the potential problems.

Conclusion

We learned how we can replace mutable operations with their immutable counterparts. Switching to the immutable state helps us to more easily reason about our state of the application and to easily track changes. It also helps us to avoid an unplanned side effect.

Please have in mind that immutable operations are recreating the array or object every time. If you are dealing with large objects or collections, this might not be the ideal way to handle your data. There are some libraries that are specialized in fast immutable operations (e.g. Immutable JS or Immer), so if you hit the performance roadblock with immutable operations, be sure to check them out.

Used materials:

You don’t need a full stack developer

Miroslav Jonas — Tue, 06 Aug 2019 14:11:28 +0000

Almost every company working with the web today is looking for full-stack developers. Developers, as a direct consequence, try to represent themselves as a full stack developer adding a little bit of front-end and a little bit of back-end to their CV. But are they really? And more important — does the company actually need a full stack developer?

Hiring a developer is like scouting a new band member. You are looking for someone with a specific skill, who will match your spirit, doesn’t have a band already or is willing to change a band for the perks you are offering.
If your band is famous, musicians will be queuing to join you. If you’re a small local band, your choices are unfortunately limited. So you better be sure you know who you’re looking for and whether you can afford them.

Every musician starts as a “full-stack”. That’s the only way you can practice and learn. You find one instrument you want to play, and since you don’t have a band, you have to sing, play or hum the melody and give yourself a rhythm. You have to be a one-man-band. It’s not that seldom that musician changes an instrument he/she started with originally at a later point because of the demand or simply because they realized it suits them better.

Some musicians remain “full-stack” throughout their career. If they become composers, they play multiple different instruments while composing a new piece. Every band needs at least one. Some musicians are simply musical geniuses and play with the same degree of virtuosity on multiple instruments.

Prince was one of them. On his first album, he played all 27 instruments. Damon Albarn and Phil Collins also play several instruments.

A full-stack developer is someone who works on front-end, services, database, and devops — the entire application’s stack. A good full stack developer is proficient in all those stacks. A great one has several years of parallel professional experience in all of them.
In my career, I have met a few people who I consider to be good (some even great), full-stack developers. Yet, most of them would say they prefer one tier over others.

I have also met considerably more great front-end and back-end developers who can work across tiers and are not afraid to own the whole vertical — from the database to front-end. You can count on them when you need help in the stack they are not expert in. But their expertise lies somewhere else. Therefore, they do not see themselves as full-stack developers.

Just like probably everyone else out there, at the beginning of my career, I have worked as a full-stack developer working across an entire vertical. At some point, I realized I prefer front-end over other layers and decided I want to focus on it. I still do some work on the back-end or database when needed. I also deploy my apps as part of my daily work. When it comes to my private projects, I’m a one-man show. But if you ask me what kind of a developer I am, I will tell you — I am a front-end developer.

Most of the developers are like me. They can work on every part of the application when needed but they are best in their area of specialty. And just like those musicians, we can learn any new concept, any new framework, any new programming language should we be given enough time and more importantly — a reason.

While most of the companies looking for a full stack developer hope to hire the next Prince or Phil Collins, most of them cannot afford or simply do not have a band that is exciting enough for such artists. Instead, they end up with someone like this guy:

Don’t get me wrong, I’m sure this guy is super talented.
But ask yourself this — if you were a band manager, would you bring a couple of those guys to play in your band, or would you rather find the best singer, best guitar player, best drummer, best (chose your instrument) that you can afford? Because that best guitar player can for sure sing or play drums on a decent level if needed. After all, no one wants a musician who doesn’t care for the rest of the composition.

Do you really need a team of full-stack developers, or do you actually want developers who are amazing in their specialty area, knowledgeable in others and play well together? Because the latter may not necessarily see themselves as full-stack developers.