DEV Community: Raul Melo

A Practical Guide to Layered Configuration for Modern JavaScript Applications

Raul Melo — Tue, 01 Jul 2025 06:32:44 +0000

Writing applications that scale is hard.

To do so, most of the time we try to apply well-known standards and practices to better address the problems we might encounter.

One problem every single application ever written has is managing the application configuration. It could be basic things like the port your app should use, but it can go deeper, like the database host, API URLs, feature-specific settings, etc.

In this post, I want to dive deep into this problem and propose a solution for when you're writing JS/TS applications that will definitely help you better organize your application configs in a sane way.

The Config Problem

Say you're building an app and this app needs 2 configs (at least for now): the port and some db settings.

You could declare this where it's being used, but as a good practice, you're isolating this into a config.ts file:

const config = {
  port: 3000,
  db: {
    host: "localhost",
    port: 5432
  }
} as const

Cool. Now, you have the settings isolated in a place where you and the people who will work on this project know how to change them.

But now we have a problem: while this works locally, we want to deploy our application and use different settings. For now, let's say we'll only have a "local" and "production" environment.

We'll need to expand our config.ts and start considering using an APP_ENV environment variable to determine where the application is running:

interface AppConfig {
  port: number;
  db: {
    host: string;
    port: number;
  };
}

type Environment = "local" | "prod";

const localConfig: AppConfig = {
  port: 3000,
  db: {
    host: "localhost",
    port: 5432,
  },
};

const productionConfig: AppConfig = {
  port: 8080,
  db: {
    host: "db.example.com",
    port: 5432,
  },
};

const configMap: Record<Environment, AppConfig> = {
  prod: productionConfig,
  local: localConfig,
};

// @ts-expect-error - We know it can be undefined
export const config = configMap[process.env.APP_ENV];

if (!config) {
  throw new Error(
    `Configuration for environment "${process.env.APP_ENV}" not found.`,
  );
}

At this point, the code started to get a bit more complex but is still manageable. That's because we only have two environments and our config is small, but what if we want to have more environments such as dev and/or staging?

Well, we'd need to add those values to our Environment type and implement the configuration.

To better organize the code, you could even split it into files, like:

./
└─ ...
└─ src/
   └─ ...
   └─ config/
      ├─ dev.ts
      ├─ index.ts
      ├─ local.ts
      ├─ prod.ts
      ├─ staging.ts
      └─ type.ts

In each "environment config file", you'll declare the values for that environment and we'll end up with a config file like this:

import { devConfig } from "./dev";
import { localConfig } from "./local";
import { productionConfig } from "./prod";
import { stagingConfig } from "./staging";
import type { AppConfig, Environment } from "./types";

const configMap: Record<Environment, AppConfig> = {
  prod: productionConfig,
  local: localConfig,
  dev: devConfig,
  staging: stagingConfig,
};

// @ts-expect-error - We know it can be undefined
export const config = configMap[process.env.APP_ENV];

if (!config) {
  throw new Error(
    `Configuration for environment "${process.env.APP_ENV}" not found.`,
  );
}

Much better to reason about and to organize, but we still have some flaws in this workflow, for example:

... What if, for all environments we have, the configuration is the same? Or, it's the same for all except for live?

Here, you could have a "base config" where you merge (if you have nested objects you must deep merge) the configuration:

// config/base.ts
import type { AppConfig } from "./types";

export const baseConfig: AppConfig = {
    port: 3000,
    db: {
        host: "localhost",
        port: 5432,
    },
};

// config/dev.ts
import { baseConfig } from './base'
import type { AppConfig } from "./types";

export const devConfig: AppConfig = deepMerge({}, baseConfig, {
  port: 8080,
  db: {
    host: "db-dev.example.com",
  },
})

This will generate a config object like:

{
    "port": 8080,
    "db": {
        "host": "db-dev.example.com",
        "port": 5432 // port is from base
    }
}

... What if I want to have runtime validation for my config?

In this case, you can use zod or any schema validation library, define your app config schema and validate it. By doing this you ensure type-safety at both compile and runtime:

// config/types.ts
import { z } from "zod/v4";

export const AppConfig = z.object({
  port: z.number(),
  db: z.object({
    host: z.string(),
    port: z.number(),
  }),
});
export type AppConfig = z.Infer<typeof AppConfig>;

export const Environment = z.enum(["dev", "local", "prod", "staging"]);
export type Environment = z.Infer<typeof Environment>;

// config/index.ts
import { devConfig } from "./dev";
import { localConfig } from "./local";
import { productionConfig } from "./prod";
import { stagingConfig } from "./staging";
import { AppConfig, Environment } from "./types";

const configMap: Record<Environment, AppConfig> = {
  prod: productionConfig,
  local: localConfig,
  dev: devConfig,
  staging: stagingConfig,
};

const env = Environment.parse(process.env.APP_ENV);
export const config = AppConfig.parse(configMap[env]);

... What if I want to use environment variables?

For this, you'd probably have to do something like this:

import { z } from "zod/v4";
import type { AppConfig } from "./types";

const port = z.coerce.number().default(3000).parse(process.env.PORT);

export const localConfig: AppConfig = {
  port,
  db: {
    host: "localhost",
    port: 5432,
  },
};

... What if I don't want to deal with all of that?

It starts to become cumbersome, right? But before we address this with a better solution, I want to briefly talk about a good strategy that other languages promote called Layered Configuration.

Layered Configuration and 12Factor App

Layered Configuration is an abstract concept for managing application or system settings by organizing them as independent layers that are combined to produce a final configuration.

In our context, it's a bit of what we have discussed already: having separate files for each environment so we can easily manage them.

On top of that, we can also bring in the 12Factor App, which is a set of 12 principles for building better and more scalable applications.

The third concept is called "Config" and defines that we should move configurations out of our code. It advocates for using configuration only as environment variables:

"[...] The twelve-factor app stores config in environment variables (often shortened to env vars or env). Env vars are easy to change between deploys without changing any code; unlike config files, there is little chance of them being checked into the code repo accidentally; and unlike custom config files, or other config mechanisms such as Java System Properties, they are a language- and OS-agnostic standard. [...]"

https://12factor.net/config

Personally, I'm not a huge fan of this idea. I mean, environment variables are cool but I'd rather have configuration (when non-sensitive) stored in our repository. This way, we can easily trace the changes and roll back if needed.

But also, I like to be able to override those values via environment variable if I need to, for example, run my application "as production" but maybe change the API URL only, so I can run the app locally.

In that sense, we can mix Layered Configuration and the 12Factor-app Config strategy into a new approach.

Layerfig

To address this problem and come up with a solution that implements these concepts, I've created Layerfig, a library that gives you this implementation out of the box.

It was heavily inspired by the official rust library for dealing with this problem: config-rs.

Here's how it works:

You create your configuration files:

./
├─ config/
│  ├─ base.json
│  ├─ local.json
│  └─ prod.json
└─ src

Now, you install the library

npm install @layerfig/config

Now, you create your config file that will handle and export the config object:

import { ConfigBuilder, z, FileSource, EnvironmentVariableSource } from "@layerfig/config";

const Environment = z.enum(["local", "prod"]);
const env = Environment.parse(process.env.APP_ENV);

const configSchema = z.object({
  appURL: z.url(),
});

export const config = new ConfigBuilder({
  validate: (finalConfig) => configSchema.parse(finalConfig)
})
  .addSource(new FileSource("base.json"))
  .addSource(new FileSource(`${env}.json`))
  .addSource(new EnvironmentVariableSource())
  .build();

... and that's it.

The sources you add will be loaded and merged in the sequence you've defined:

load from base.json
load from <env>.json and override base
load from environment variables and override the two objects

Now, when you import your config object into your application, this object will be type-safe at both compile and runtime.

Key features

It's focused on server-side applications but also works in client side using ObjectSources.
Can be used in Node, Bun, Deno, and any JS engine that supports node:fs and node:path.
You can use the built-in and exported zod/v4 or use your own schema library. All you need to do is return the result of the schema validation in the validate function.
By default, only .json files are supported, but if you want .jsonc, .json5, .yml, or .toml, you can use the official parsers.
You can use slots and define environment variables inside the config:
```
  {
    "port": "${PORT}",
    "appURL": "http://localhost:${self.port}"
  }
```
And if the PORT variable is found (e.g. 3000), it'll replace ${PORT} with its value.
Allows you to override values (including nested and deep properties) via environment variables

Check the documentation for more details and examples.

Conclusion

I hope I was able to show you how we can better manage application settings and how layerfig can help you in this case.

If you have any problems while using it or want a specific feature, feel free to engage in the layerfig github repository

Resources

Deploying on Netlify via GitHub Actions: A Seamless Guide

Raul Melo — Thu, 04 Jan 2024 11:14:35 +0000

Netlify is a fantastic platform.

As developers, sometimes we don't want to spend much time figuring out how to deploy our apps, setting up servers, AWS accounts, etc. Here's where products like Netlify shine by abstracting all this complexity away from us and providing easy-to-use dashboards.

But all these facilities come with a price.

On Netlify, for example, if we use the basic and free plan, we have a few limitations, like the number of contributors we can have, the amount of network bandwidth, how many websites we can host, etc.

One that recently pissed me off a lot is the amount of build minutes.

I have a few projects at the same account (team), plus some automated processes to update my dependencies, which drained my build minutes in two weeks.

What made me angry was that Netlify thought it would be a good idea to charge me 7 dollars automatically (without giving me a chance to say NO) for extra build minutes. Because I didn't have a credit card, my account was almost suspended until I paid it.

So, I've decided to do something about it, which is being away to use their infrastructure to run my builds, and here we are.

Let's dive deep into how to handle the builds through GitHub Actions.

Netlify host, builds, and previews

When we integrate a project directly from our GitHub and use Netlify as a host, we have to specify a bunch of info, including how to build our app and where the dist files are located.

By doing that, on every push in our GitHub repository, our code will be synced with Netlify, which will install dependencies, run the build, and deploy. Depending on your tools and project size, this can go from a few seconds to minutes.

By default, every project has a very nice feature called build preview, which builds your project and hosts it on another URL, making it possible to see a preview of your app with the changes you have made.

This is incredible, but it hits hard on the build times.

Imagine your project takes 1 minute to build, and you push 10 times to your feature branch. For a single feature, we're talking about 3%~ of your monthly build minutes quota.

If you have something like Dependabot or Renovate that creates PRs automatically to update dependencies, then you almost have no control over how fast the quota can go.

There are ways to mitigate that, though. You can turn off the deploy preview, which sucks because this is such a nice feature.

Also, if you're in a monorepo, you can prevent building a preview if the code touched isn't part of the project hosted on Netlify using the ignore builds feature.

Or if you already use GitHub Actions to do the deployment, in case you already use it to run checks, tests, or code quality.

Deploying to Netlify through GitHub Actions

To demonstrate how to do that, I'm going to use the following setup, but you can abstract the concept and adapt it to your tooling:

Node v20.10.0
npm v10.2.3 (to lower the entry bar)
vite + svelte starter project (build static)
netlify-cli v17.10.1

Creating the basic Github Action Workflow

With the project created, let's jump straight to where the magic will happen.

I will create a deploy workflow, but feel free to hook the deployment steps into your existing workflow.

.github/workflows/deploy.yml

name: Deploy website

on: [push]

jobs:
  build-and-deploy:
    runs-on: ubuntu-latest
    steps:
      - name: Repository Checkout
        uses: actions/checkout@v4

      - name: Setup NodeJS
        uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: "npm"

      - name: Install Dependencies
        run: npm ci

Breaking it down:

on every push
We run on an ubuntu machine the following steps:
- do a git checkout to fetch the code
- setup node JS using version 20 and use npm cache strategy
- install our project dependencies

When we push this code, the pipeline should run successfully:

Ok, before we finish this setup, let's create a project on Netlify.

Netlify project

I don't want to create a connection between my Github project and Netlify or drop build files there. So, to cut to the chase, I will use the Netlify CLI to create the project.

The first step is installing the Netlify CLI globally on my machine:

npm install netlify-cli -g

Now, we need to log in with our account:

netlify login

They will ask you to open a link in your browser and log in.

After that, we can create the project:

netlify sites:create

You'll need to fill in some information, like the team and site names.

After finishing, you'll have your site details, such as your Admin URL, Site URL, and Site ID.

$ netlify sites:create
? Team: test
? Site name (leave blank for a random name; you can change it later): netlify-deploy-gha

Site Created

Admin URL: https://app.netlify.com/sites/netlify-deploy-gha
URL:       https://netlify-deploy-gha.netlify.app
Site ID:   87c1c6cb-5eae-4797-8c20-58608f633a98

Adding local .netlify folder to .gitignore file...
Linked to netlify-deploy-gha

If you open your Admin URL, you will see an empty project because we haven't done a deployment yet.

Before going back to the GitHub part, we'll need to save two pieces of information we'll use there: the Site ID and generate an API token.

The Site ID you already have, so now let's create an API Token.

Generate Netlify API Token

Attention
If for some reason you can't follow this section, search on Netlify about "Personal access tokens"

First, access your User Settings, then click on Applications. There, you'll find a section called Personal Access Tokens. Click on New access token:

Now, give your token a name and choose when it expires:

After clicking to create your token, copy the value and save it somewhere:

Now, let's head back to the GitHub actions part:

Deploy setup

To do the deployment, we have to add a few steps to our workflow.

We need to:

Ensure we have Netlify CLI globally in our environment
Run the build command so we can send those files to Netlify

name: Deploy website

on: [push]

jobs:
  build-and-deploy:
    runs-on: ubuntu-latest
    steps:
      - name: Repository Checkout
        uses: actions/checkout@v4

      - name: Setup NodeJS
        uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: "npm"

      - name: Install Netlify
        run: npm install netlify-cli@17.10.1 -g

      - name: Install Dependencies
        run: npm ci

      - name: Build project
        run: npm run build

I always install pinned versions because a major version update e.g., from 17 to 18, could suddenly break my setup.

Before adding the deploy command, we need to add the Site ID and the API Token to our project environment variables.

These values are secrets and can't be hard-coded in our code.

Adding GitHub secrets

In your project, click on Settings, then Secrets and variables, Actions, and New repository secret.

Create two variables:

NETLIFY_SITE_ID => will hold your Netlify Site ID
NETLIFY_API_TOKEN => will hold your Netlify API personal token

Deploy command

Finally, let's add the deploy command.

Before doing that, you must know what folder is produced when you run your build command.

On Vite's projects, usually, this folder is called dist, but in other frameworks, it could be something like build.

name: Deploy website

on: [push]

jobs:
  build-and-deploy:
    runs-on: ubuntu-latest
    steps:
      - name: Repository Checkout
        uses: actions/checkout@v4

      - name: Setup NodeJS
        uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: "npm"

      - name: Install Netlify
        run: npm install netlify-cli@17.10.1 -g

      - name: Install Dependencies
        run: npm ci

      - name: Build project
        run: npm run build

      - name: Deploy to Netlify
        id: netlify_deploy
        run: |
          netlify deploy \
            --dir dist \
            --site ${{ secrets.NETLIFY_SITE_ID }} \
            --auth ${{ secrets.NETLIFY_API_TOKEN }}

If you configured everything correctly, when we push this file, the pipeline will run and deploy a preview of your site.

The field 'Website draft URL' holds the deployment preview URL.

Though it's working, we are not deploying to production yet.

Let's do some tweaks to consider this case.

Production deployment

Deploying previews is useful only when pushing to any branch but our main branch.

Also, if we want to deploy to production, we must pass a --prod flag.

So, we can do some bash code with these two pieces of information.

.github/workflows/deploy.yml

name: Deploy website

on: [push]

env:
  BRANCH_NAME: ${{ github.ref_name }}

jobs:
  build-and-deploy:
    runs-on: ubuntu-latest
    steps:
      - name: Repository Checkout
        uses: actions/checkout@v4

      - name: Setup NodeJS
        uses: actions/setup-node@v4
        with:
          node-version: 20
          cache: "npm"

      - name: Install Netlify
        run: npm install netlify-cli@17.10.1 -g

      - name: Install Dependencies
        run: npm ci

      - name: Build project
        run: npm run build

      - name: Deploy to Netlify
        id: netlify_deploy
        run: |
          prod_flag=""
          if [ "$BRANCH_NAME" = "main" ]; then prod_flag="--prod"; fi
          netlify deploy \
            --dir dist \
            --site ${{ secrets.NETLIFY_SITE_ID }} \
            --auth ${{ secrets.NETLIFY_API_TOKEN }} \
            $prod_flag

Breaking the changes down:

on lines 5 and 6, we declare that, in all steps, we'll have access to an environment variable called BRANCH_NAME. We'll use it to determine if it's the main branch or not
on line 33, we create a bash variable called prod_flag
on line 35, we assert against our BRANCH_NAME. If it's main or whatever your main branch is named, we add the --prod to the prod_flag variable
on line 39, we append the prod_flag variable. It might be --prod or an empty string (for deploy previews)

Our "Deploy to Netlify" step will produce a different out when we push these changes. Instead of "Website draft URL", we'll have "Website URL", the production one.

# ...

Build logs:        https://app.netlify.com/sites/netlify-deploy-gha/deploys/6590b2e90ce0fbbf1abd9ac1
Function logs:     https://app.netlify.com/sites/netlify-deploy-gha/functions
Unique deploy URL: https://6590b2e90ce0fbbf1abd9ac1--netlify-deploy-gha.netlify.app
Website URL:       https://netlify-deploy-gha.netlify.app

Which means we accomplished what we wanted.

Bonus: Pull Request URL comment

When we integrate GitHub + Netlify and open a PR, they have a bot that gets the Deploy preview URL and posts a comment.

This is nice because right now if we want to see the URL of our preview, we need to click to see the details of our runner, expand the "Netlify Deploy" step, search for "Website draft URL", copy the link, and open it.

In other words, we kinda lost a very nice DX by handling the integration, and that means we must do something about it.

Again, there are many ways you can approach this, I'll just give a suggestion by adding a few more steps with some bash code and JavaScript via actions/github-script.

Getting the Website draft URL

We need to get the URL of our draft so we can comment on our PR.

Let's save the output of our deploy command into a file:

.github/workflows/deploy.yml

#... rest of file

 - name: Deploy to Netlify
   id: netlify_deploy
   run: |
     prod_flag=""
     if [ "$BRANCH_NAME" = "main" ]; then prod_flag="--prod"; fi
     netlify deploy \
       --dir dist \
       --site ${{ secrets.NETLIFY_SITE_ID }} \
       --auth ${{ secrets.NETLIFY_API_TOKEN }} \
-      $prod_flag
+      $prod_flag \
+      > deploy_output.txt

With > deploy_output.txt, every output produced will be saved in the deploy_output file.

Now, let's add another step that will read this file, extract the URL, and make it available for the following steps:

.github/workflows/deploy.yml

# ... rest of the file ...

- name: Generate URL Preview
  id: url_preview
  if: ${{ env.BRANCH_NAME != 'main' }}
  run: |
    NETLIFY_PREVIEW_URL=$(cat deploy_output.txt | grep "Website draft URL: " | cut -d' ' -f4)
    echo "NETLIFY_PREVIEW_URL=$NETLIFY_PREVIEW_URL" >> "$GITHUB_OUTPUT"

Breaking it down:

we must define an id so we can get the output in another step
in the if clause, we only run this when it's not main
then, we concat some bash commands where we:
- get the content of the deploy_output.txt file
- extract the content of "Website draft URL: " entirely using grep;
- split the line into fields delimited by spaces -d and then select the fourth field -f4, which is where the URL is expected to be based on your output sample
- save the result of this operation on a variable called NETLIFY_PREVIEW_URL
finally, we say to GitHub Actions that this step will have an output called NETLIFY_PREVIEW_URL with its value

To test if it's working, we can simply add another step after this:

.github/workflows/deploy.yml

# ... rest of the file ...

- name: DEBUG=WILL BE REMOVED
  run: echo ${{ steps.url_preview.outputs.NETLIFY_PREVIEW_URL }}

Posting URL preview as PR comment

Now we have access to the URL, we can use it to create a PR comment.

Here it'll be raw and dirty; feel free to enhance that in a script, a more fancy message, update an existing comment, etc.

To achieve that, I'm going to use a GitHub action called github-script.

In case you don't know, this action gives us access to Github's REST API, the runner context, etc., and lets us script with JavaScript syntax without having to install anything.

The first step is to give our workflow permission to write on pull requests:

.github/workflows/deploy.yml

name: Deploy website

on: [push]

env:
  BRANCH_NAME: ${{ github.ref_name }}

permissions:
  pull-requests: write

# ... rest ...

Then, as the last step, we can add the following code:

.github/workflows/deploy.yml

# ... rest ...

- name: Comment URL Preview on PR
  uses: actions/github-script@v7
  if: ${{ env.BRANCH_NAME != 'main' }}
  env:
    NETLIFY_PREVIEW_URL: ${{ steps.url_preview.outputs.NETLIFY_PREVIEW_URL }}
  with:
    script: |
      async function comment(){
        const result = await github.rest.repos.listPullRequestsAssociatedWithCommit({
          owner: context.repo.owner,
          repo: context.repo.repo,
          commit_sha: context.sha,
        })

        const issueNumber = result.data[0].number

        if(issueNumber){
          await github.rest.issues.createComment({
            issue_number: issueNumber,
            owner: context.repo.owner,
            repo: context.repo.repo,
            body: 'Preview URL: ' + process.env.NETLIFY_PREVIEW_URL
          })
        }else{
          console.log('No PR found for commit ' + context.sha)
        }
      }

      comment()

Breaking it down:

on line 5, we define we only want to run this step on branches different than the main branch
on line 7, we save the URL preview into an environment variable
on line 9, we add a javascript script where:
- on lines 11 to 15, we use GitHub's SDK to fetch the pull requests with the current commit hash
- on line 17, we save this information;
- we save this information.
- if there's an issue number (PR number), we create a comment with the text 'Preview URL'.

By doing this, if the PR number is found, a comment will be made by the GitHub Bot:

And now, we finally have a minimal DX, and we can rest 😅.

Monorepo case

In case you're using a monorepo, there's just one thing you need to tweak, in the deploy command, you have to add a flag called --filter with the project name:

.github/workflows/deploy.yml

#... rest of file

- name: Deploy to Netlify
  id: netlify_deploy
  run: |
    prod_flag=""
    if [ "$BRANCH_NAME" = "main" ]; then prod_flag="--prod"; fi
    netlify deploy \
      --dir dist \
      --site ${{ secrets.NETLIFY_SITE_ID }} \
      --auth ${{ secrets.NETLIFY_API_TOKEN }} \
+    --filter <my-project-name> \
      $prod_flag \
      > deploy_output.txt

Why not...

... use the official action from Netlify?

If you encountered the official Netlify's Github Action that provides the CLI out of the box, you might notice that the last time it was updated (at least when I'm writing this post was a year ago).

I tried, but I got some version conflict problems. It seems I was trying to use an argument that wasn't present in the CLI version of the step.

.... use the "action-netlify-deploy"?

This is an open-source action that abstracts most of the setup we've made here.

I started using it, but I encountered an issue where it didn't output correctly the Preview URL. So I thought: "If I need to use a third-party action that doesn't do what I need, then I can do it by myself".

Conclusion

There is a trade-off between leaving the control of processes with the platform and taking the reins of your deployment. It's up to you to decide what is more worthwhile.

I hope with this guide, you at least can have another option to make the decision that fits you best.

Cheers.

References

Build a JavaScript library with multiple entry points using Vite3

Raul Melo — Fri, 05 Aug 2022 06:25:00 +0000

Original post: https://www.raulmelo.dev/blog/build-javascript-library-with-multiple-entry-points-using-vite-3

Intro

This post probably will get outdated soon.

That's because while I'm writing, Vite 3 (v3.0.4) still does not support an out-of-the-box solution for using it in Library mode with multiple entry points.

Despite proposing two strategies, I'd also point to a problem I encountered while trying to solve this: How to have multiple type definitions?

I wish everything was more straightforward than it actually is, but I'll walk you step-by-step through the problems and solutions.

About Vite

Vite is an opinionated build tool that aims to provide a faster and learner development experience for modern web projects.

https://vitejs.dev/guide/#overview

It's focused on ES Modules structure (although we can use it with Common JS), and it gives us a development server (like when you run npm run dev) with a blazing fast Hot Module Replacement (HMR).

We can define HMR for the situation when you're running a dev server, change a file, this file is processed again (to be supported by the browser), and the server gets refreshed.

Under the hood, Vite combines esbuild for a particular file optimization (due to its outstanding performance) and Rollup (for actually outputting the files), having an excellent performance in general.

If you're new to Vite, I strongly recommend you walk through their docs and try to create a simple app using one of their templates to see how amazing it's with your eyes.

Building a JS library

If you want to build a JS library, you will likely pick up Rollup.

That's because it's a mature tool, isn't as complex as Webpack, and is not that hard to configure.

Even though the configuration isn't too hard, you still have to install a bunch of plugins, care about TypeScript parsing (in case you write your lib in TS), care about transpiling CommonJS code, etc.

Luckily Vite has something called "Library Mode", which allows us to point to an entry file (like a index.ts file that exports everything the package contains) and get it compiled using all the Vite ecosystem.

The documentation around that is great, and I believe it's enough so you can have a lib ready to be published or consumed by an application itself.

The only problem with that is: "What if instead of having a single entry point, I want to have it multiple?"

You might have noticed already, that some libraries allow you to import a few things from the main import:

import { foo } from 'package'

And from the same library, also give us a sub-module:

import { bar } from 'package/another-module'

An example of it is Next.JS. When we install next, we can import a few things from the main package and other things from sub-modules:

import Link from 'next/link';
import Image from 'next/image';
import type { GetServerSideProps } from 'next';

We can't simply point to an index.ts for those cases and have multiple outputs. We need to point to other files.

In the same example used before (next), they most likely would point to multiple files like src/image.tsx, src/link.tsx are compiled to files like dist/image.js, and dist/link.js.

Side note, they don't use Vite for that. Looking at their code base, the ecosystem and building are more complex than we need and for that, they have a different approach and tooling.

Ok, but if Vite doesn't support multiple entries, how do we achieve this?

Strategies

It might have be a lot of ways to solve that, but here, I want to mention two strategies I found very simple.

Single and configurable vite.config

I saw this comment on Vite's thread regarding this topic, and because I'm into scripting, I wanted to enhance that.

The thing is, because we're in a Node environment and we have access to environment variables, we can them inside a JavaScript file for example:

console.log(process.env.MY_ENV_VAR);

Then, when I do this:

MY_ENV_VAR=random-value node index.js

It consoles the value I've given to MY_ENV_VAR:

random-value

Ok. Now, let's imagine I want to build a lib that exports two modules: logger and math (just for the sake of the example):

.
├── src
│ ├── lib.ts
│ └── math.ts
└── package.json

In our vite.config.js for a single entry, we would have something like this:

import { resolve } from 'path'
import { defineConfig } from 'vite'

export default defineConfig({
  build: {
    lib: {
      entry: resolve(__dirname, 'lib/index.ts'),
      fileName: 'my-lib',
      formats: ['cjs', 'es'],
    },
  }
})

On Vite 3, a vite.config file must export default the result of defineConfig.

To have a more dynamic build, we could have an object in our config file that holds the different information between math and logger:

const config = {
  math: {
    entry: resolve(__dirname, "./src/math.ts"),
    fileName: "math.js",
  },
  logger: {
    entry: resolve(__dirname, "./src/logger.ts"),
    fileName: "logger.js",
  },
};

The entry and fileName are the only difference between these two files.

From there, we can use an environment variable to determine which configuration it should be used and which file should be output:

import { resolve } from "path";
import { defineConfig } from "vite";

const config = {
  math: {
    entry: resolve(__dirname, "./src/math.ts"),
    fileName: "math.js",
  },
  logger: {
    entry: resolve(__dirname, "./src/logger.ts"),
    fileName: "logger.js",
  },
};

const currentConfig = config[process.env.LIB_NAME];

if (currentConfig === undefined) {
  throw new Error('LIB_NAME is not defined or is not valid');
}

export default defineConfig({
  build: {
    outDir: "./dist",
    lib: {
      ...currentConfig,
      formats: ["cjs", "es"],
    },
    emptyOutDir: false,
  },
});

Summarizing the actions:

We get the config based on the environment variable we'll specify while running this command;
We add validation to help us identify if we misspelled the environment variable and try to build a lib that is not mapped;
We call defineConfig with the common config and spread the current config

Now, we can run the following command:

$ LIB_NAME=math npx vite build

vite v3.0.4 building for production...
✓ 1 modules transformed.
dist/math.js.cjs 0.15 KiB / gzip: 0.14 KiB
dist/math.js.js 0.06 KiB / gzip: 0.08 KiB

The npx command call the binary of Vite (the CLI itself)

Now, we can also use the same command for the logger lib:

$ LIB_NAME=logger npx vite build

vite v3.0.4 building for production...
✓ 1 modules transformed.
dist/logger.js.cjs 0.16 KiB / gzip: 0.16 KiB
dist/logger.js.js 0.09 KiB / gzip: 0.09 KiB

To make our life easier, we could have these two commands as npm script and a single command that calls both scripts:

{
  "scripts": {
    "build:math": "LIB_NAME=math vite build",
    "build:logger": "LIB_NAME=logger vite build",
    "build": "npm run build:math && npm run build:logger"
  }
}

Importing build from Vite in a custom script

In the same discussions, another user suggested a different strategy for solving this problem: using build from Vite.

In case you don't know, Vite exposes build method so we can do it programmatically:

import { build } from 'vite';

With that in mind, all we need to do is create an array of configurations and iterate over this array calling build:

import { build } from "vite";
import path from "path";
import { fileURLToPath } from 'url';

const __dirname = path.dirname(fileURLToPath(import.meta.url));

const libraries = [
  {
    entry: path.resolve(__dirname, "../src/logger.ts"),
    fileName: "logger",
  },
  {
    entry: path.resolve(__dirname, "../src/math.ts"),
    fileName: "math",
  },
];

libraries.forEach(async (lib) => {
  await build({
    build: {
      outDir: "./dist",
      lib: {
        ...lib,
        formats: ["es", "cjs"],
      },
      emptyOutDir: false,
    },
  });
});

And that would generate the exact same output as the previous strategy.

The difference is, instead of calling vite build, all we need to do is call our script with node:

$ node scripts/build.mjs

vite v3.0.4 building for production...
vite v3.0.4 building for production... (x2)
✓ 1 modules transformed.
✓ 1 modules transformed. (x2)
dist/math.js 0.06 KiB / gzip: 0.08 KiB
dist/logger.js 0.09 KiB / gzip: 0.09 KiB
dist/math.cjs 0.15 KiB / gzip: 0.14 KiB
dist/logger.cjs 0.16 KiB / gzip: 0.16 KiB

Because I've defined this file to be a .mjs extension, I had to do the workaround to have __dirname and I need to use Node 16 or higher.

Configuring the package.json

One of the most crucial steps for having a javascript lib is defining in our package.json how node needs to resolve the files.

On Vite docs, they have a recommendation of how to do that, which is almost what we want:

{
  "name": "my-lib",
  "type": "module",
  "files": ["dist"],
  "main": "./dist/my-lib.umd.cjs",
  "module": "./dist/my-lib.js",
  "exports": {
    ".": {
      "import": "./dist/my-lib.js",
      "require": "./dist/my-lib.umd.cjs"
    }
  }
}

Explaining each field:

files is an array of what files we're going to publish (so you can pack to your lib only the necessary code);
main is the entry point for the common.js (require) statements;
module is the entry for ES Modules (import ? from) statements;
exports is a modern alternative to allow us to have multiple entry files (exactly what we need)
- "." in this case stands for the main entry point (require('my-lib') and import 'my-lib')
- import is actually for ES Modules;
- require is for CommonJS.

main, module, and export has the same goal. The difference is that exports has a more flexible way of mapping the files we want to expose, while main and module is more for a single entry point.

Keep in mind that that exports requires at least Node v12 to work, which I don't think it's a problem nowadays.
Alright, now let's create our own config.

Because the lib we're creating doesn't have a default entry point, we can get rid of main and module and only use exports.

In our exports field, we can now define both math and logger sub-modules and point to the files that the build command will output:

{
  "type": "module",
  "exports": {
    "./math": {
      "import": "./dist/math.js",
      "require": "./dist/math.cjs"
    },
    "./logger": {
      "import": "./dist/logger.js",
      "require": "./dist/logger.cjs"
    }
  },
  "files": [
    "dist/*",
  ]
}

Be aware that the file we're pointing should exist, and it'll when we run npm run build

Another observation, we're defining "type": "module" to define how node should run the file (using .mjs extension or not). Vite will use this information to generate the file with either .mjs or .cjs extension. Read here

And that's it.

Now, we'll be able to use import/require this package by:

// ES Modules environment
import { sum } from 'my-package/math'
import { logger } from 'my-package/logger'

// CommonJS (node)
const { sum } = require('my-package/math')
const { logger } = require('my-package/logger')

Generating type definitions

TypeScript is getting more popular every year.

Because of that, it's a good practice for library maintainers also to provide the type definitions from the library, so our IDEs or Text Editors IntelliSense can give us hints about the function signature, etc.

Because the code of this package is in TypeScript, we can use the tsc compiler to generate it automatically for us.

It's a good practice to have a tsconfig.json config in case you want to compile our code with tsc or even use the compiler as a "linter".

However, for this case, because we only want generate the types, we can skip the config part and straight use the compiler:

tsc src/*.ts --declaration --emitDeclarationOnly --declarationDir dist/

Here we're saying to the compiler to only emit declarations for all .ts files present on src and put them on the dist folder.

When we build our files with Vite and run this command to generate the files, we'll have a dist folder like this:

.
└── dist
    ├── logger.cjs
    ├── logger.d.ts
    ├── logger.js
    ├── math.cjs
    ├── math.d.ts
    └── math.js

Cool.

Now we have to declare our types using the package.json attribute types.

types isn't a standard field from Node but something TypeScript introduced, and Node and the JS Ecosystem have embraced it.

The biggest problem here is that types only accept a string, not an array of strings. How can we actually point to multiple definitions then?

Problem

This is a problem I found very intriguing.

I read some people saying that once you have the type definition (d.ts) in the same folder as we have the built file (e.g. math.cjs), TypeScript would be able to infer that automatically the types.

I tried that, and didn't work out as I expected.

Maybe because in this lib dist folder, I not only have math.cjs but also math.js, and they would be imported differently. Maybe TypeScript gets confused trying to guess if that definition is from the file I'm actually importing.

To be fair, I don't know for sure.

It would be great if we could define the types inside the exports:

{
  "exports": {
    "./math": {
      "import": "./dist/math.js",
      "require": "./dist/math.cjs",
      "type": "./dist/math.d.ts"
    },
    "./logger": {
      "import": "./dist/logger.js",
      "require": "./dist/logger.cjs",
      "type": "./dist/logger.d.ts"
    }
  }
}

But that won't work.

As I said, exports is something official from node while types is just something TypeScript come up.

Solution

I'm not sure the best way to solve that, but I couldn't find any better.

Using Next.js as example again, they do a great job of providing such separate packages and types, so I went there to see how they do.

I realize that despite the source code, they have in the root of the main next package type definition for each package.

In every single .d.ts, they only export * from './dist/<package-name>.js and, the cherry of the cake, they have one file called index.d.ts that uses <reference /> directive from TypeScript and "import" all those types.

The logic behind that is that every single submodule we're going to export will point to its type definition generated by the compiler on the dist folder.

Then, we use the <reference> directive from TypeScript to inform the compiler that those referenced files need to be included in the compilation process.

With this instruction, TypeScript compiler will be able to infer that what's the type definition for that sub-module.

Again, I'm not sure if that's the best way of solving that, but in my tests, that worked as a charm so let's implement this.

Hands-on

First thing, we're going to create in our root folder two .d.ts files, one for the logger module and another for the math module.

In both, we'll only export everything that the dist/*.js has:

export * from "./dist/logger";

export * from "./dist/math";

Now, we create an index.d.ts file, referencing both type definitions:

/// <reference path="./logger.d.ts" />
/// <reference path="./math.d.ts" />

Cool.

You might have noticed that now we have a single entry for our types (index.d.ts).

The last step is to point this entry file to types and listing these 3 new type definition files to our files field:

{
  "types": "./index.d.ts",
  "files": [
    "dist/*",
    "index.d.ts",
    "logger.d.ts",
    "math.d.ts"
  ]
}

And that would do the job.

Demo

To show you this works really well, I've created a demo app so you can see and use it as a reference for building your own library.

The only difference is that in the post I use npm as example, but in the demo, I used pnpm just for having a workspace where I could maintain both build strategies and a vanilla TS app that consumes the packages.

https://github.com/raulfdm/vite-3-lib-multiple-entrypoints

Conclusion

Again, I think this problem might be solved in a future version of Vite but because there's a struggle around that, I thought it worth a post with some explanation.

I hope I could help you somehow.

Peace!

References

How to Manage Multiple Runtime Versions With a Single Tooling

Raul Melo — Thu, 08 Jul 2021 13:10:54 +0000

Photo by Nana Smirnova on Unsplash

Hey, devs.

Today I want to present to you a simpler and (IMO) better way to have installed different versions of the same runtime in your machine with a single tool called asdf.

The tool itself is relatively simple, the docs are well written and you won’t have trouble getting started asap.

Here I will just make a summary and present you some gotchas I had after a while using it.

The Hell of Runtime Versions

Back in 2013 when I was learning development in college, I remember having a lot of trouble running some projects because they rely on a specific Java version.

“Oh, this one uses Java 5, I need to download the runtime. But wait, I already have Java 6 in my machine… damn”

This was very overwhelming, especially because I didn’t have much experience and back then my professors and colleagues didn’t know a better way of easing this problem.

In the first company I started work as a software developer, all backend apps were written in ruby and well… the exact same problem.

To avoid consuming the production/staging environment while I was coding my client-side application, I needed to run a couple of servers in my machine and, again, they required different ruby versions.

Luckily the backend team had a wiki page recommending to use a tool called RVM (Ruby Version Manager), which as the name already explains itself, is a CLI to control multiple ruby versions in our machine.

And it worked pretty well, to be honest.

After a while coding front-end applications, I started to see more often the same problem happening with node js.

This project uses node 7. But I have node 10 installed... Oh no…

When I googled to find a similar solution as rvm, I found NVM (Node Version Manager), which does the same thing as rvm but for the Node runtime.

Ok, now I have 2 CLIs to handle the same problem.

It was then I started to study Go and thought: “oh, here we go again… another CLI”.

After I complained about that to a colleague he said:

Why the heck don't you use asdf for all these languages?

And my mind explodes because of course, if I was bothered dealing with that, someone in the plant was bothered as well, and most importantly: he/she had already created a tool that solves this problem.

The Master CLI: asdf

asdf is a CLI tool that solves the runtime version management in a well-architected and elegant way: by being a single tool for all runtimes.

The idea is that asdf is a core application that does the handling heavy-lifting of providing CLI options, being hooked via terminal, etc. but instead of also solving the version available for every single runtime that exists, it delegates this to third party plugins.

This means that asdf does not care about any runtime like java, go, deno, rust, or whatever, but it provides an abstract interface where someone in the community can simply create and maintain a plugin that provides all the information to download the version X for example.

If you’re familiar with Front-end development, this is pretty much like babel does. It handles our code but you can write your plugin that hooks there and do something else without creating a burden to the Babel’s team to maintain the code itself.

The following sections will show how the tooling works. If you want to try, you need to make sure that you have asdf properly installed and configured in your bash, fish, or zsh terminal configuration.

asdf installation guide

Plugins

As I mentioned, a plugin is a specific runtime manager. In that sense, the asdf-ruby plugin only deals with how to download ruby runtime, which version was released, etc.
Installing a plugin is relatively easy but I strongly recommend you check the plugin’s repository and see if there are more instructions needed before adding it.

Check here all plugins available for asdf

In general, the instruction follows the same basic way of installing, running asdf plugin add <plugin-name> <git-url>. The git-url isn’t required but recommended just to be sure you’re consuming for the correct repository:

asdf plugin add nodejs
asdf plugin add ruby https://github.com/asdf-vm/asdf-ruby.git

Now, my asdf will have 2 plugins: one capable to manage node versions and another to manage ruby versions.

Versions

The plugin itself won’t add any version by default. It’s time to finally install some versions we want to use in some projects.

For this example I want to have in my machine:
Latest 10.2 version of node
Latest 14 version of node
Exact 2.6.5 version of Ruby
Latest 2.7 version of Ruby

For all these versions I'll run asdf install <plugin-name> <version>. So all I need to do is running in the terminal:

asdf install nodejs latest:10.2
asdf install nodejs latest:14
asdf install ruby 2.6.5
asdf install ruby latest:2.7

Obs.: depending on your environment you might need to install some stuff before being able to install a runtime, like ruby for example that requires having libssl-dev, etc, in a Linux machine before installing successfully.

Note that since in some cases we didn't care about the exact version we simply specified a latest: prefix before the version number and this will end up installing the latest version available for that number.

Now if we want to know the versions installed, we can simply run asdf list <plugin-name>:

asdf list nodejs
 10.24.1
 14.17.3

asdf list ruby
 2.6.5
 2.7.4

Defining a runtime version

There are 2 cases where you want to define a version: for global and local scope.

Global

We always need to define a version for the global context. Personally, I like to use the latest LTS version of NodeJS (today 14.17) by default and fallback down depending on the project

To do that all I need to do is running asdf global <plugin-name> <version>:

asdf global nodejs 14.17.3

Now if I check my node version it'll show this specific version:

node -v
v14.17.3

Local

And now, back to the main problem, we're trying to solve: running a specific version for a specific project.

For specifying a local version, all we need to do is to run asdf local <plugin-name> <version>:

asdf local nodejs 10.24.1

By doing that, asdf will create a file in the folder you’re called .tool-versions that contains the following content:

nodejs 10.24.1

Now, every time I run yarn start for example (which will invoke somehow node runtime) because asdf is intercepting the call and finds this file, it’ll use version 10 instead of my global 14.

Tip: if for some reason you don't want to commit the .tool-versions file you can either add it in the .gitignore of your project or if this rule will be applied to all projects, setting this ignore globally like I explain here

Removing a plugin

Let's say now you left the company and will no longer need ruby versions in your machine.

All you need to do is by simply removing the plugin with asdf plugin remove <plugin-name>:

asdf plugin remove ruby

BY doing that, the ruby plugin and all versions we've installed in the previous steps will be completely removed from our machine.

Compatibility

If you already are using another tooling and want to migrate to asdf without having to remove the existing version file, in your $HOME path you can create a file called .asdfrc and add the flag legacy version file:

legacy_version_file = yes

More info: https://asdf-vm.com/#/core-configuration?id=tool-versions

Conclusion

I hope now it’s clear to you how awesome asdf is not only to solve this cumbersome version problem but also in terms of using a very robust plugin mechanism for the correct problem.

Any questions or comments, please reach out on Twitter.

Cheers.

References

https://asdf-vm.com/

Data Structure with JavaScript: Linked List

Raul Melo — Sun, 04 Jul 2021 15:44:11 +0000

Hello, devs.

Today I want to dive deep into a very specific data structure to hold a collection that's called Linked List.

First I'll briefly explain the array problem and how linked lists came to solve those problems and then we'll how to implement that in JavaScript.

I already can say that as a JS dev I don't see big use cases because we have natively a pretty decent way of handling collections. However, as a software engineer, I consider it very important to have a base understanding of it in case I need it in the future.

Knowledge is power and never useless.

About Lists

Arrays are one of the most efficient ways of storing data collections like a list of friends on Instagram for example.

In JavaScript when we want to create a list of something, all we need is a very simple open/close square bracket ([]) and push as many elements you want to it.

However in some other languages, especially the ones which are focus on performance the approach is different. There, if you need a list, you have to specify the size of your list, which sometimes is a problem because we often handle dynamic data.

And it makes sense to have this approach. If you have little memory and need to write some code that compiles a very low machine language (like binary) if you say that your list will have only 3 elements, for example, the compiler can say:

Hey, for this list here can be allocated only a few bytes because we know in advance it'll only contain a max of 3 elements

Also, if you want to insert or remove an element in some specific position, you need to entirely move the list and these operations could be tedious and expensive.

In JS we don't suffer much about that because the language was designed in that way and we also have native array methods (very well optimized I suppose) that remove or add an element and regen the list, like the method Array.prototype.splice().

const months = ['Jan', 'March', 'April', 'June'];

// insert exactly in the index one (1, 0) the string `Feb`
months.splice(1, 0, 'Feb');

console.log(months); // Array ["Jan", "Feb", "March", "April", "June"]

// removes everything from the index 3 til the last el ("April" and "June")
months.splice(3, months.length)

console.log(months); // ["Jan", "Feb", "March"]

Linked List: Concept

Linked list implementation tries to solve the max number of elements we can store in a list and how to easily navigate through a list by change the data structure used from arrays to simple linked objects (node).

Every node will have 2 properties:

element: the data we want to store in our list;
next: a link to another node or the value null (non-existing next node).

Maybe the best way to visualize it is by imagining a train.

In a train we always have the "head" which and from there it's connected the first "wagon", then a second "wagon" is connected to the first until the end of the train.

If we want to remove a defective wagon, for example, all we need to do is find this wagon, link the previous wagon to the next wagon, and done.

The principle is the same when we want to add a new "wagon". All we need is to find where we want to add it, connect the previous and the next wagon into the new one.

In other words, linked lists are all about creating and modifying connections between nodes.

In the next section, we'll step-by-step implement all those additions and removals and you'll that it's a relatively simple concept.

Linked List: Implementation

Before any implementation, let's take a look in the API we'll need for this kind of list:

.append(element) - method used to append a new element to the end of the list;
.indexOf(element) - method used to known where in the index our element was added;
.insertAt(position, element) - method used to add an element in a specific position;
.remove(element) - method used to remove an element from the list;
.removeAt(position) - method used to remove an element in some specific position;
.toString() - method used to have an overview of our list.

Once again, instead using JS classes/prototype, I'm going to use my favorite pattern Factory with some placeholders for our API already:

function LinkedListFactory() {
  return {
    append,
    indexOf,
    insertAt,
    remove,
    removeAt,
    toString,
  };

  function append(element) {}

  function indexOf(element) {}

  function insertAt(position, element) {}

  function remove(element) {}

  function removeAt(position) {}

  function toString() {}
}

"Global" variables

Before implementing the methods, we'll need to create 2 variables that will be used in almost all methods:

head - variable to hold our very first element, where everything will start. It'll start with the value null;
length - a control variable to easily hold the list size. It'll start with the value 0.

function LinkedListFactory() {
  let head = null;
  let length = 0;

  return {
    append,
    indexOf,
    insertAt,
    remove,
    removeAt,
    toString,
  };

  function append(element) {}

  function indexOf(element) {}

  function insertAt(position, element) {}

  function remove(element) {}

  function removeAt(position) {}

  function toString() {}
}

.append(element)

In the append method, we first need to create an internal basic structure which we can call "node".

A node is a simple object which will hold the element we're adding to the list and the next element (the link itself).

Since append will always add an element to the end of the list, next will always be null:

function append(element) {
  const node = {
    element,
    next: null
  }
}

The first scenario is when our list is empty, or, when head is null. For this case, we'll assign our newly created node to the head:

function append(element) {
  const node = {
    element,
    next: null,
  };

  if (head === null) {
    head = node;
  }
}

Now, we have to consider the other cases (if not head or second to last node case).

Since we want to append an element to the end of our list, we have to iterate through all nodes until .next equal to null.

function append(element) {
  const node = {
    element,
    next: null,
  };

  if (head === null) {
    head = node;
  } else {
    let currentNode = head;

    while (currentNode.next !== null) {
      currentNode = currentNode.next;
    }
  }
}

Now that we encountered the last element, all we need to do is link the .next property of this element to our newly created node:

function append(element) {
  const node = {
    element,
    next: null,
  };

  if (head === null) {
    head = node;
  } else {
    let currentNode = head;

    while (currentNode.next !== null) {
      currentNode = currentNode.next;
    }

    currentNode.next = node;
  }
}

Finally, we'll need, for both cases (head or not), increment 1 to our list size (length) so it's important to be outside the condition

function append(element) {
  const node = {
    element,
    next: null,
  };

  if (head === null) {
    head = node;
  } else {
    let currentNode = head;

    while (currentNode.next !== null) {
      currentNode = currentNode.next;
    }

    currentNode.next = node;
  }

  length++;
}

.indexOf(element)

This method is meant to find where a given element is placed in our list.

First, we'll need two controller variables: nodeIndex and currentElement. The first will be used as return value but also to know where we are in the iteration and the second to do the comparison if the element is the one we're looking for:

function indexOf(element) {
  let nodeIndex = 0;
  let currentNode = head;
}

Remember when I told you that head might be null or the .next of the last node will be null? We'll use this condition to loop through all nodes.

function indexOf(element) {
  let nodeIndex = 0;
  let currentNode = head;

  while (currentNode) {
    if (element === currentNode.element) {
      return nodeIndex;
    }

    nodeIndex++;

    currentNode = currentNode.next;
  }
}

Now, until currentNode is not null, we'll first check if the element is the one we're looking for. If so, we can straight return the value of nodeIndex.

If not, then we'll need to increment 1 to nodeIndex and assign currentNode to currentNode.next, or in other words, simply moving to the next node to run the comparison over again.

Finally, if we can't find the element the user is looking for, we have to give an indication we couldn't.

Traditionally, for cases like that such methods return -1 but nothing stops us of returning other value like null for example:

function indexOf(element) {
  let nodeIndex = 0;
  let currentNode = head;

  while (currentNode) {
    if (element === currentNode.element) {
      return nodeIndex;
    }

    nodeIndex++;

    currentNode = currentNode.next;
  }

  return -1
}

.insertAt(position, element)

In this operation, we'll do a similar operation as we did for indexOf (controlling the index) plus we'll have to tweak the node connections.

Imagine the following scenario: we have 4 nodes linked in our list and we want to insert a new element in position 2 (second position because it's a 0-based index).

We basically will need:

Loop through the nodes;
Find who is in the position 2;
make this node .next point to the element we're inserting
make our new node .next point to the element we just found .next

It might seem a lit bit confusing but I'll drive you step-by-step in the implementation itself.

The first validation we need to do is if the position the user is asking to add exists in our list. We need to ensure the if we don't add an element in the position 4 if we only have 1 element in our list:

function insertAt(position, element) {
  const isPositionInTheRange = position > -1 && position <= length;

  if(!isPositionInTheRange){
    return false
  }
}

Tip: the variable which holds this logic is just a semantic way of describing (in English) a condition.

Like in the other methods, we'lll need to iterate over our list to see where we need to add this element. This means we'll need to create a controller variable and our node:

function insertAt(position, element) {
  const isPositionInTheRange = position > -1 && position <= length;

  if(!isPositionInTheRange){
    return false
  }

  // Our brand new node
  const node = {
    element,
    next: null
  }

  // Controller to iterate over the list
  let currentNode = head;
}

Our first case here is that the user wants to add an element at the first position (head). All we need to do is to say that the new node .next will be the current element and the head now will be the new node:

function insertAt(position, element) {
  const isPositionInTheRange = position > -1 && position <= length;

  if (!isPositionInTheRange) {
    return false;
  }

  const node = {
    element,
    next: null,
  };

  let currentNode = head;

  const isHeadPosition = position === 0;

  if (isHeadPosition) {
    // Assign currentNode (head) to `node.next`
    node.next = currentNode;
    // Replace the current head with this node
    head = node;
  } else {
  }
}

Now we need to handle the case where the position is after the head.

First, we'll need 2 controller variables, index (for iterate based on that) and previousNode (for re-create the links when we find the position):

function insertAt(position, element) {
  const isPositionInTheRange = position > -1 && position <= length;

  if (!isPositionInTheRange) {
    return false;
  }

  const node = {
    element,
    next: null,
  };

  let currentNode = head;

  const isHeadPosition = position === 0;

  if (isHeadPosition) {    
    node.next = currentNode;
    head = node;
  } else {
    let previousNode = null;
    let index = 0;
  }
}

Then, we'll iterate using index. While index is less then the desired position, we'll update our controllers previousNode and currentNode:

function insertAt(position, element) {
  const isPositionInTheRange = position > -1 && position <= length;

  if (!isPositionInTheRange) {
    return false;
  }

  const node = {
    element,
    next: null,
  };

  let currentNode = head;

  const isHeadPosition = position === 0;

  if (isHeadPosition) {    
    node.next = currentNode;
    head = node;
  } else {
    let previousNode = null;
    let index = 0;

    while (index++ < position){
      previousNode = currentNode;
      currentNode = currentNode.next;
    }
  }
}

Remember: index++ will first evaluate the number and only then increment 1.

This step is only necessary to walk through our list until the position matches the one we want to change.

When we reach that, all we need to do is re-do the links between the previousNode <-> new node <-> currentNode:

function insertAt(position, element) {
  const isPositionInTheRange = position > -1 && position <= length;

  if (!isPositionInTheRange) {
    return false;
  }

  const node = {
    element,
    next: null,
  };

  let currentNode = head;

  const isHeadPosition = position === 0;

  if (isHeadPosition) {    
    node.next = currentNode;
    head = node;
  } else {
    let previousNode = null;
    let index = 0;

    while (index++ < position){
      previousNode = currentNode;
      currentNode = currentNode.next;
    }

    previousNode.next = node;
    node.next = currentNode;
  }
}

Finally, we need to add +1 in our list length, not matter where in the list it was inserted and return true to inform for the user that the operation succeeded:

function insertAt(position, element) {
  const isPositionInTheRange = position > -1 && position <= length;

  if (!isPositionInTheRange) {
    return false;
  }

  const node = {
    element,
    next: null,
  };

  let currentNode = head;

  const isHeadPosition = position === 0;

  if (isHeadPosition) {    
    node.next = currentNode;
    head = node;
  } else {
    let previousNode = null;
    let index = 0;

    while (index++ < position){
      previousNode = currentNode;
      currentNode = currentNode.next;
    }

    previousNode.next = node;
    node.next = currentNode;
  }

  length++;
  return true;
}

.removeAt(position)

The removeAt method has a very similar implementation as we just saw in the insertAt, we'll need to:

iterate over the list;
find the correspondent element in that position;
connect the previous element into the next;
decrease the list size

Starting, once again let's first validate if the request position contains an element:

function removeAt(position){
  const isPositionInTheRange = position > -1 && position < length;

  if(!isPositionInTheRange){
    return null
  }
}

Then, we need to create controller variable currentNode to iterate through:

function removeAt(position){
  const isPositionInTheRange = position > -1 && position < length;

  if(!isPositionInTheRange){
    return null
  }

  let currentNode = head;
}

Again we'll have 2 situations: head or not head. If head, all we need to do is to reassign head to be the currentNode (in this case the head element itself) to it's .next value:

function removeAt(position){
  const isPositionInTheRange = position > -1 && position < length;

  if(!isPositionInTheRange){
    return null
  }

  let currentNode = head;

  if(position === 0){
    head = currentNode.next;
  }
}

In case you're not aware, by just removing all references from this element the garbage collector will understand that it is no longer needed and prune it from the memory.

Now, we need to remove elements which are not the head. For that, let's create two other controller variables, index and previousNode:

function removeAt(position){
  const isPositionInTheRange = position > -1 && position < length;

  if(!isPositionInTheRange){
    return null
  }

  let currentNode = head;

  if(position === 0){
    head = currentNode.next;
  } else {
    let index = 0;
    let previousNode = null;
  }
}

And once again, iterate over all elements until we reach the position we want to:

function removeAt(position){
  const isPositionInTheRange = position > -1 && position < length;

  if(!isPositionInTheRange){
    return null
  }

  let currentNode = head;

  if(position === 0){
    head = currentNode.next;
  } else {
    let index = 0;
    let previousNode = null;

    while(index++ < position){
      previousNode = currentNode;
      currentNode = currentNode.next
    }
  }
}

Now, we recreate the node links by linking previousNode.next into the currentNode.next:

function removeAt(position){
  const isPositionInTheRange = position > -1 && position < length;

  if(!isPositionInTheRange){
    return null
  }

  let currentNode = head;

  if(position === 0){
    head = currentNode.next;
  } else {
    let index = 0;
    let previousNode = null;

    while(index++ < position){
      previousNode = currentNode;
      currentNode = currentNode.next
    }

    previousNode.next = currentNode.next;

  }
}

And finally, we need to subtract 1 from the list length and return the element we're removing so the user can do something with it:

function removeAt(position){
  const isPositionInTheRange = position > -1 && position < length;

  if(!isPositionInTheRange){
    return null
  }

  let currentNode = head;

  if(position === 0){
    head = currentNode.next;
  } else {
    let index = 0;
    let previousNode = null;

    while(index++ < position){
      previousNode = currentNode;
      currentNode = currentNode.next
    }

    previousNode.next = currentNode.next;
  }

  length--;
  return currentNode.element;
}

.remove(element)

This method will be quite simple to implement. That's because we already have a method which find an index from an element (indexOf) and also have a method to remove an element from a position (removeAt):

function remove(element){
  const elementIndex = indexOf(element);
  return removeAt(elementIndex);
}

.toString()

This method is purely to give to whoever is using this linked list the notion of all elements present in the list.

Once again we'll need to navigate through all nodes and concatenate the element value into a string:

function toString() {
  let result = "";
  let current = head;

  while (current) {
    result += `${current.element}${current.next ? ", " : ""}`;
    current = current.next;
  }

  return result;
}

Obs.: This will work only for primitive values. If you want to support elements as objects, functions, etc. you'll need to enhance the concatenation and data parsing.

Final Result

function LinkedListFactory() {
  let head = null;
  let length = 0;

  return {
    append,
    indexOf,
    insertAt,
    remove,
    removeAt,
    toString,
  };

  function append(element) {
    const node = {
      element,
      next: null,
    };

    if (head === null) {
      head = node
    } else {
      let currentNode = head;

      while (currentNode.next !== null) {
        currentNode = currentNode.next;
      }

      currentNode.next = node;
    }

    length++;
  }

  function indexOf(element) {
    let nodeIndex = 0;
    let currentNode = head;

    while (currentNode) {
      if (element === currentNode.element) {
        return nodeIndex;
      }

      nodeIndex++;

      currentNode = currentNode.next;
    }

    return -1;
  }

  function insertAt(position, element) {
    const isPositionInTheRange = position > -1 && position <= length;

    if (!isPositionInTheRange) {
      return false;
    }

    const node = {
      element,
      next: null,
    };

    let currentNode = head;

    const isHeadPosition = position === 0;

    if (isHeadPosition) {
      node.next = currentNode;
      head = node;
    } else {
      let previousNode = null;
      let index = 0;

      while (index++ < position) {
        previousNode = currentNode;
        currentNode = currentNode.next;
      }

      previousNode.next = node;
      node.next = currentNode;
    }

    length++;
    return true;
  }

  function removeAt(position) {
    const isPositionInTheRange = position > -1 && position < length;

    if (!isPositionInTheRange) {
      return null;
    }

    let currentNode = head;

    if (position === 0) {
      head = currentNode.next;
    } else {
      let index = 0;
      let previousNode = null;

      while (index++ < position) {
        previousNode = currentNode;
        currentNode = currentNode.next;
      }

      previousNode.next = currentNode.next;
    }

    length--;
    return currentNode;
  }

  function removeAt(position) {
    const isPositionInTheRange = position > -1 && position < length;

    if (!isPositionInTheRange) {
      return null;
    }

    let currentNode = head;

    if (position === 0) {
      head = currentNode.next;
    } else {
      let index = 0;
      let previousNode = null;

      while (index++ < position) {
        previousNode = currentNode;
        currentNode = currentNode.next;
      }

      previousNode.next = currentNode.next;
    }

    length--;
    return currentNode.element;
  }

  function remove(element) {
    const elementIndex = indexOf(element);
    return removeAt(elementIndex);
  }

  function toString() {
    let result = "";
    let current = head;

    while (current) {
      result += `${current.element}${current.next ? ", " : ""}`;
      current = current.next;
    }

    return result;
  }
}

const linkedList = LinkedListFactory();

linkedList.append(1);
linkedList.append(10);
linkedList.append(-1);
linkedList.append(40);
linkedList.append(-123);

console.log(linkedList.toString()); // 1, 10, -1, 40, -123

console.log(linkedList.removeAt(3)); // 40

console.log(linkedList.toString()); // 1, 10, -1, -123

console.log(linkedList.indexOf(1)); // 0

console.log(linkedList.remove(1)); // 1

console.log(linkedList.toString()); // 10, -1, -123

Conclusion

I hope I could explain to you what's linked list about and how to simply implement one.

There are also two variants of it: "doubly linked" (next and previous link) and circular, but I think it'll be better in another article.

Again, because we're in a JS environment I don't see a strong usage to it but it's important to know that exists in case we get in contact with it in other languages.

If you have any comments on that, please tweet me so we can build knowledge together!

Cheers.

Data Structure with JavaScript: Queue

Raul Melo — Tue, 22 Jun 2021 05:56:54 +0000

When we talk about queue as a data structure, it's exactly the same as you can imagine in real-world examples: a lot of things a line one in front of the other, waiting for something, like the imagine hero shows.

In this post, we're gonna talk about the basic implementation of a queue (like a supermarket queue), a priority queue (like a hospital queue), and also a circular queue (like a list of things you have to do repetitively).

Basic Queue

The basic queue will give us the base for the other queues. Here, we need to implement a mechanism called FIFO (First In, First Out), which means the first element added will be the first to be removed.

To start, let's see the API interface we need to implement that:

enqueue(element) - add new element(s) to the queue;
dequeue() - remove first element from the queue;
front() - returns the first element from the queue (for checking purposes);
isEmpty() - returns if the queue is empty or not;
size() - returns how many elements the queue contains.

Keep in mind that the API might differ from different languages which have Queue native or other implementations, but the principles are the same.

There are many ways of creating this structure, some people would straight use JS class but personally, I like to use the factory approach.

So let's create a factory function with a placeholder for all those methods:

function QueueFactory() {
  const items = [];

  return {
    enqueue,
    dequeue,
    front,
    isEmpty,
    size,
  };

  function enqueue() {}

  function dequeue() {}

  function front() {}

  function isEmpty() {}

  function size() {}
}

.enqueue(element)

This method will simple take an element as argument and add to the end of the list:

function enqueue(element) {
  items.push(element)
}

.dequeue()

Dequeue will remove the first element of our items and return it. We can simple use Array.prototype.shift for that task:

function dequeue() {
  return items.shift();
}

Friendly reminder that shift mutates items array.

.front()

This method will only return for visibility purposes the first element of the list:

function front() {
  return items[0];
}

.isEmpty()

As the name says, this method will check if our queue is empty or not:

function isEmpty() {
  return items.length === 0;
}

.size()

This will simply return the length of our list:

function size() {
  return items.length;
}

(Optional) .print() / .queue()

It's important that we don't expose our queue as part of the public interface because the whole idea is having a custom implementation for arrays.

Also, arrays are mutable, which means that if the user (we or other devs) push has access to the reference, new elements could be pushed or removed.

So if you want to provide a method to check the entire list, you could return a copy of this list:

function queue() {
  return [...items];
}

Or maybe a method which prints the list:

function print() {
  console.log(items.toString());
  // or console.log(JSON.stringify(items))
}

Final result

function QueueFactory() {
  const items = [];

  return {
    enqueue,
    dequeue,
    front,
    isEmpty,
    size,
    print
  };

  function enqueue(element) {
    items.push(element);
  }

  function dequeue() {
    return items.shift();
  }

  function front() {
    return items[0];
  }

  function isEmpty() {
    return items.length === 0;
  }

  function size() {
    return items.length;
  }

  function print() {
    console.log(items.toString());
  }
}


const myQueue = QueueFactory();

myQueue.enqueue(3);
myQueue.enqueue(2);
myQueue.enqueue(6);

console.log(myQueue.front()); // 3
myQueue.print(); // 3,2,6

console.log(myQueue.dequeue()); // 3
myQueue.print(); // 2,6

Priority Queue

In some cases, only the basic queue isn't enough. We need that behavior but we also want to take into account priorities, like a hospital emergency queue where the worst case has the highest priority no matter when it arrives first.

The good news is that from our previous implementation, only a few changes will be necessary.

Internal data structure

Before we were simply pushing the element we received from our enqueue method into a list.

Now, to keep tracking which element has higher or lower priority we might want to have an internal structure, a simple object where we simply hold the element and the priority:

function createQueueElement(element, priority) {
  return {
    element,
    priority,
  };
}

Now, iniside the enqueue method, we have to also accept a priority, so we create our element with our internal structure:

function enqueue(element, priority) {
  const newEl = createQueueElement(element, priority);

  items.push(newEl)
}

Nothing has changed until here, only our internal data structure.

Priority

To take into account where to add the element we'll need to loop over all items and check if the current element has higher priority than the one we're trying to add.

Don't forget that when the element we're comparing has the same priority as the one we're trying to add, the new one should be added after it (respecting FIFO):

// q.enqueue('Martin', 1);

{element: 'Karen', priority: 1}
{element: 'Caroline', priority: 1}
                                    <- Martin should be added here
{element: 'John', priority: 2}

Since we need the index of the iteration to insert an element in between a list, let's use a simple for loop:

function enqueue(element, priority) {
  const newElement = createQueueElement(element, priority);
  let added = false;

  for (let index = 0; index < items.length; index++) {
    const currentElement = items[index];

    if (newElement.priority < currentElement.priority) {
      items.splice(index, 0, newElement);
      added = true;
      break; // We don't need to keep running the loop
    }
  }

  if (!added) {
    items.push(newElement);
  }
}

Summarizing the operation:

We create a controller variable "added" for the cases where our loop condition is not satisfied (like empty list or first element with that priority);
We loop over all elements in the queue;
If the current element has greater priority than our new element, we set our new element in the current element position using the method Array.prototype.splice;
We set our controller variable to true and break the loop because the operation that matters was completed;
If was not added because was the first element, for example, we just add the element with .push()

.print()

Our previous print method was simple and good enough because we had an array of strings.

Now we have some structure, might be good to enhance the code to better visualize all elements with their priorities.

  function print() {
    for(const item of items){
      console.log(`element: ${item.element} - priority: ${item.priority}`)
    }
  }

Final Result

function PriorityQueueFactory() {
  const items = [];

  return {
    enqueue,
    dequeue,
    front,
    isEmpty,
    size,
    print,
  };

  function createQueueElement(element, priority) {
    return {
      element,
      priority,
    };
  }

  function enqueue(element, priority) {
    const newElement = createQueueElement(element, priority);
    let added = false;

    for (let index = 0; index < items.length; index++) {
      const currentElement = items[index];

      if (newElement.priority < currentElement.priority) {
        items.splice(index, 0, newElement);
        added = true;
        break;
      }
    }

    if (!added) {
      items.push(newElement);
    }
  }

  function dequeue() {
    return items.shift();
  }

  function front() {
    return items[0];
  }

  function isEmpty() {
    return items.length === 0;
  }

  function size() {
    return items.length;
  }

  function print() {
    for(const item of items){
      console.log(`element: ${item.element} - priority: ${item.priority}`)
    }
  }
}

var q = PriorityQueueFactory();

q.enqueue('John', 2);
q.enqueue('Olivia', 1);
q.enqueue('Karmen', 3);
q.enqueue('Oliver', 1);

q.print(); /*
element: Olivia - priority: 1
element: Oliver - priority: 1
element: John - priority: 2
element: Karmen - priority: 3
*/

Since the logic from this method is pretty much the same as the basic Queue, we could break it in a way of using either function composition (my preferred way) or class inheritance but for the sake of the tutorial let's focus on the data structure implementation itself.

Circular Queue

Unfortunately, we don't have as many applications for circular queues as we have for the others, but it's still important to know we have this concept.

A circular queue has the same principles as the regular queue. The only difference is that when it achieves the end of the queue, it returns for the first element and starts over again.

In that sense, we'll need to change our implementation a bit because we can't simply remove elements from the queue but we need to keep them somewhere else.

Usage example

Let's imagine we've created a small application that has a list of tasks to do every 3 hours.

We're going to run this list of tasks till the end and after 3 hours, it'll start over again.

I know we could simply run a function which has these tasks inside and every time has these new 3 elements, but help me here in the exercise 😅

To do that, let's create something on top of our already existing basic queue.

The first step is creating a factory function that adds a queue into its closure and returns an object (API interfaces later)

function SchedulerFactory() {
  const queue = QueueFactory();

  return {};
}

We'll then create 3 methods for this data structure:

.add(element): will add a new task;
.pick(): will return the next task to be executed;
.size(): will return how many tasks it has.

.add(element)

Adding a new task will be very straight forward, we'll just enqueue the task:

function SchedulerFactory() {
  const q = QueueFactory();

  return {
    add
  };

  function add(task){
    q.enqueue(task)
  }
}

.pick()

For picking a new task, we'll need to store the current task which was pick:

function SchedulerFactory() {
  const q = QueueFactory();
  let currentTask;

  return {
    add
  };

  function add(task){
    q.enqueue(task)
  }  
}

Then, we'll need to:

if there is a current task, we have to enqueue that (will move to the end of the queue)
assign the current Task to the result of dequeue (pick the first element of our queue)
return current task

In other words, we'll add the previous element back to the queue and replace it with the first queue element.

function SchedulerFactory() {
  const q = QueueFactory();
  let currentTask;

  return {
    add,
    pick
  };

  function add(task){
    q.enqueue(task)
  }

  function pick(){
    if(currentTask){
      q.enqueue(currentTask); // add the previous task to the end
    }

    currentTask = q.dequeue(); // get next task

    return currentTask;
  }
}

.size()

For the size, we cannot rely on the queue size because it'll always miss an element (the task we're currently executing).

So we can create an internal counter and increment 1 every time a new task is added:

function SchedulerFactory() {
  const q = QueueFactory();
  let currentTask;
  let numberOfTasks = 0;

  return {
    add,
    pick,
    size,
  };

  function add(task) {
    q.enqueue(task);
    numberOfTasks++;
  }

  function pick() {
    if (currentTask) {
      q.enqueue(currentTask);
    }

    currentTask = q.dequeue();

    return currentTask;
  }

  function size() {
    return numberOfTasks;
  }
}

Using SchedulerFactory

Now, we can use use our SchedulerFactory:

var taskScheduler = SchedulerFactor();

taskScheduler.add("Clean up memory");
taskScheduler.add("Check weather");
taskScheduler.add("Check stocks prices");
taskScheduler.add("Scrape website");
taskScheduler.add("Send email with info");

executeAllTasks(taskScheduler);

function executeAllTasks(scheduler) {
  console.log("Starting Tasks...");

  for (
    let taskIndex = 0;
    taskIndex < scheduler.size;
    taskIndex++
  ) {
    const task = scheduler.pick();

    console.log(`Task[${taskIndex}]: ${task}`);
  }

  console.log("Finish Tasks");
}

The function executeAllTasks just loop over all tasks (using the scheduler size) and console them. Of course in real scenarios it'll be more complex tasks and executions but note that everything you call executeAllTasks with the same task scheduler (taskScheduler), it'll execute all tasks and start from the beginning:

executeAllTasks(taskScheduler);
executeAllTasks(taskScheduler);
executeAllTasks(taskScheduler);

// Starting Tasks... debugger eval code:40:11
// Task[0]: Clean up memory debugger eval code:49:13
// Task[1]: Check weather debugger eval code:49:13
// Task[2]: Check stocks prices debugger eval code:49:13
// Task[3]: Scrape website debugger eval code:49:13
// Task[4]: Send email with info debugger eval code:49:13
// Finish Tasks debugger eval code:52:11
// Starting Tasks... debugger eval code:40:11
// Task[0]: Clean up memory debugger eval code:49:13
// Task[1]: Check weather debugger eval code:49:13
// Task[2]: Check stocks prices debugger eval code:49:13
// Task[3]: Scrape website debugger eval code:49:13
// Task[4]: Send email with info debugger eval code:49:13
// Finish Tasks debugger eval code:52:11
// Starting Tasks... debugger eval code:40:11
// Task[0]: Clean up memory debugger eval code:49:13
// Task[1]: Check weather debugger eval code:49:13
// Task[2]: Check stocks prices debugger eval code:49:13
// Task[3]: Scrape website debugger eval code:49:13
// Task[4]: Send email with info debugger eval code:49:13
// Finish Tasks

Conclusion

Well, that's it about queues.

I hope you could understand that the data structure itself isn't that complicated to understand and the implementation isn't rocket science.

This is another tool for your toolbox of problem-solving in development. Every time you realize you're trying to solve a queue problem but you don't have the exact data structure, create your modeling of a queue to help you out.

How to have better NPM Scripts

Raul Melo — Fri, 16 Apr 2021 06:30:18 +0000

As a JavaScript developer (no matter back or front-end), we often rely upon npm scripts to automate common tasks like starting a server, building a project, and even performing tasks before or after certain scripts like postbuild, prebuild, etc.

When those commands are simple like node index.js, having them a single line in our package.json isn't a problem at all. The real problem starts when we need an extensive command, adding environment variables, and concatenating commands:

(Example extracted from Material UI package.json)

{
    "scripts": {
      "proptypes": "cross-env BABEL_ENV=development babel-node --extensions \".tsx,.ts,.js\" ./scripts/generateProptypes.ts",
      "deduplicate": "node scripts/deduplicate.js",
      "benchmark:browser": "yarn workspace benchmark browser",
      "build:codesandbox": "lerna run --parallel --scope \"@material-ui/*\" build",
      "release:version": "lerna version --exact --no-changelog --no-push --no-git-tag-version",
      "release:build": "lerna run --parallel --scope \"@material-ui/*\" build",
      "release:changelog": "node scripts/releaseChangelog",
      "release:publish": "lerna publish from-package --dist-tag next --contents build",
      "release:publish:dry-run": "lerna publish from-package --dist-tag next --contents build --registry=\"http://localhost:4873/\"",
      "release:tag": "node scripts/releaseTag",
      "docs:api": "rimraf ./docs/pages/api-docs && yarn docs:api:build",
      "docs:api:build": "cross-env BABEL_ENV=development __NEXT_EXPORT_TRAILING_SLASH=true babel-node --extensions \".tsx,.ts,.js\" ./docs/scripts/buildApi.ts  ./docs/pages/api-docs ./packages/material-ui-unstyled/src ./packages/material-ui/src ./packages/material-ui-lab/src --apiPagesManifestPath ./docs/src/pagesApi.js",
      "docs:build": "yarn workspace docs build",
      "docs:build-sw": "yarn workspace docs build-sw",
      "docs:build-color-preview": "babel-node scripts/buildColorTypes",
      "docs:deploy": "yarn workspace docs deploy",
      "docs:dev": "yarn workspace docs dev",
      "docs:export": "yarn workspace docs export",
      "docs:icons": "yarn workspace docs icons",
      "docs:size-why": "cross-env DOCS_STATS_ENABLED=true yarn docs:build",
      "docs:start": "yarn workspace docs start",
      //.....
    }
}

But what if I told you could have those commands extracted into a separate file and having a scripts config like this:

{
    "scripts": {
      "proptypes": "scripty",
      "deduplicate": "scripty",
      "benchmark:browser": "scripty",
      "build:codesandbox": "scripty",
      "release:version": "scripty",
      "release:build": "scripty",
      "release:changelog": "scripty",
      "release:publish": "scripty",
      "release:publish:dry-run": "scripty",
      "release:tag": "scripty",
      "docs:api": "scripty",
      "docs:api:build": "scripty",
      "docs:build": "scripty",
      "docs:build-sw": "scripty",
      "docs:build-color-preview": "scripty",
      "docs:deploy": "scripty",
      "docs:dev": "scripty",
      "docs:export": "scripty",
      "docs:icons": "scripty",
      "docs:size-why": "scripty",
      "docs:start": "scripty",
    }
   //.....
}

Scripty

Scripty is an npm package that enables us the ability to have executable files to run npm scripts.

The whole idea is to treat these giant script lines we have as code and keep our package.json clean and simple.

Let's say we have this:

{
  "scripts": {
    "lint": "eslint . --cache --report-unused-disable-directives --ext .js,.ts,.tsx --max-warnings 0"
  }
}

Using scripty it'll look like this:

{
  "scripts": {
    "lint": "scripty"
  }
}

The magic behind

Of course, the command we just removed needs to be somewhere. To make it simple as that, scripty does a pairing of <npm-script-nam>:<executable-file-name>.

In other words, if we have an npm script called lint, we need an executable file called lint, lint.sh, or lint.js.

The default folder is always, at the root level, a folder called scripts. So, to solve the previous migration, we would create a file called lint.sh under the scripts folder, like this:

#!/usr/bin/env bash

yarn eslint . --cache --report-unused-disable-directives --ext .js,.ts,.tsx --max-warnings 0

Executable Bash or .JS

Scripty can handle only handle executable bash or JavaScript executables.

To have one of those, the file needs to:

having the shebang at the top of the file (e.g. #!/bin/bash or #!/bin/node;
having permission to execute (while ls -la, it needs to have x flag);

Quick tip, if you're in a UNIX environment, you can quickly give this permission by running the command chmod u+x <file-path>

Also, file extensions are not necessary. You can write a test.sh, test.js or only test. What will define the syntax highlight and the execution will be one of the shebang instructions I've mentioned before.

#!/bin/node

const fs = require('fs');

fs.copyFileSync('static/base.css', 'dist/base.css');
// ...

#!/usr/bin/env bash

NODE_ENV=production

yarn nest build

For JS executable, keep in mind that it'll be executed by node and you can't use invalid js-node (e.g. import) syntax.

Batching

Another requirement we often have is running a bunch of scripts related. Let's say we have a lot of test script and we want to run all of them, like test:*:

{
  "scripts": {
    "test:unit": "jest",
    "test:e2e": "cypress run --ci",
    "test": "npm-run-all test:*",
  }
}

With scripty, we can create a subfolder called test and declare those 2 types of tests there:

.
├── package.json
├── scripts
│   └── test
│       ├── e2e
│       └── unit
└── yarn.lock

By having those files with those instructions, you can change your package.json to be:

{
  "scripts": {
    "test:unit": "scripty",
    "test:e2e": "scripty",
    "test": "scripty",
  }
}

Note that only the test script will be sufficient for this case. We'll only keep test:unit and test:e2e in case we want to run one of these commands isolated.

When you run test, scripty will understand you have a folder called test with a lot of scripts and it'll run all of them.

Keep in mind that this is a concurrency call and you should not rely on the execution order.

Controlling the batching sequence

If you need them being executed in a certain order, with the same package.json as before, all you need to do is, in our scripts/test folder, creates a script called index witch will be responsible for executing the other scripts in the sequence we want to:

.
├── package.json
├── scripts
│   └── test
│       ├── index
│       ├── integration
│       └── unit
└── yarn.lock

#!/bin/bash

scripts/test/unit
scripts/test/integration

Keep in mind that the current working directory (CWD) will be always where it's being executed, in that case, our root folder.

Parallel watch

Another common scenario is when we have certain scripts we need to run which will stay in watch mode, in other words, lock a section and keep listening for file changes so it can perform something.

{
  "scripts": {
    "watch:css": "sass src/scss/main.scss public/css/main.css -s compressed",
    "watch:js": "webpack --config webpack.config.js --watch --mode=development",
  }
}

A way of booting both commands would be opening two tabs and running each command in a tab. But that's tedious. What if we could somehow have a single terminal tab and run all watch at the same time?

To do that using scripty all we have to do is to create a folder called watch inside scripts, pretty much as we did before for test

.
├── package.json
├── scripts
│   └── watch
│       ├── css
│       └── js
└── yarn.lock

But instead of only passing scripty word to our npm script, we have to specify an environment variable called SCRIPTY_PARALELL with true:

{
  "scripts": {
    "watch": "SCRIPTY_PARALLEL=true scripty"
  }
}

Now both will keep running.

Caveats

The biggest caveat here is for windows users.

If you're one of them or you maintain a project which can be run in a Windows machine, you'll need some special treatment for that and I suggest you take a look at their docs with those instructions.

Conclusion

Scripty allows us to treat or npm scripts as code, having a file containing all instructions to execute some tasks.

It also eases the ability to roll back an incorrect script instruction and provides a great and isolated git history.

So be creative.

References

How to test publishing your JavaScript package locally

Raul Melo — Sun, 28 Feb 2021 16:04:18 +0000

Since I've started using npm as a JS package manager (maybe back in 2015), I always wanted to publish my own packages which could be either a very particular package I'd use in my side projects or a package that attempts to solve a common problem and help the other devs.

Every time I needed to do that I also wanted to test the workflow of publishing and installing locally. Also, I'd like to see a kind of "preview" of how it'll look like when it's published, the "real" npm page, just to see if the README is ok for example.

After a lot of struggle and attempts with various approaches to solving this problem, I think I finally figure out the best (at least for me) of how to solve those 2 problems.

Before diving deep into the final solution, let me tell you about the problems I had with other solutions.

Symlink

Back in 2016, trying to find a way to do that I saw a lot of people talking about Symlink.

In short, Symbolic Link (or Symlink), is when you create a reference link between 2 (or more) files making just a reference to each other.

Imagine you have a library (my-library) and want to use the local files of it in your project (website). A Symlink in this case will be, inside the node_modules, instead of having the production files of my-library, it points to the local folder of it.

But... how to do that?

NPM/Yarn Symlink

Of course, I wasn't the only person in the world desiring a proper way of doing that. And that's why both npm and yarn provide a way of doing symlink out of the box.

I won't explain how to use that in this article, but if you still want to know, you can find a link to how to do that.

In a nutshell, what happens is by doing that, npm or yarn will:

Create a global reference to your package;
Replace the real module with that reference inside your node_modules

Using this approach solves most of the problem of testing packages locally cross any project... until it doesn't.

Symlink problem

My biggest pain-point with global Symlink was with nested node_modules and how the tools (at least back them) resolves what version of a module A (used in both the project and the library) was supposed to be resolved.

The first time I saw that was while writing a React component library. The workflow was:

Go to my library
Run yarn link to create a Symlink
Go to my project
Link my-library
Start my dev server

By only doing that, I started having issues with some React internal rules. That was weird because the error message was really true.

After a couple of hours digging into this problem I finally found an issue on React's repo reporting the exact same problem I had and he pointed about about the Symlink:

The answer from the maintainer was clear:

It's not expected that it would work unless you link react back from your module to your app.

This has actually always been the case (React apps are subtly broken when there are two copies of React module). Hooks surface this immediately which I guess is good.

We do have another issue tracking a better error message for this case.

Of course, it makes a lot of sense. In my component library, React was a peerDependency and I didn't ship that within the bundle. Now, using it via Symlink, React was installed in my library AND my project.

Someone posts a workaround for solving that where you would also need to link the react and react-dom inside the library and use that link in our project. So my flow would be:

Go to my library
Navigate to node_modules/react
Run yarn link to create a react symlink
Navigate to node_modules/react-dom
Run yarn link to create a react symlink
Go back to the root level and run yarn link to symlink my lib
Go to my project
Use the link of my-library, react and react-dom
Start my dev server

By doing that, my problem was gone! But... gosh. Really?

After finishing my tests I had to remove those 3 links from your project and force install the dependencies.

Doing that a couple of times was ok but after 10 times I got really annoyed and created a bash script to execute those steps for me.

Also, now I'm using Docker to run my projects and I've realized that Symlink does not work with my base setup.

Probably because when I run my container, I only create a volume which the current project folder. When the container is up and tries to use that Symlink, it might need to navigate through my file system and I think that's not possible.

It might be possible to do that by adding some extra configs but I just don't want to. I want an easy way of doing something in my lib, push and install it whenever I need to use WITHOUT polluting my real package at npm.

Also, using Symlink you can't tell for sure if you're shipping all files your application will need to work.

Luckily, I found a very simple way to solve that and I want to share it with you.

NPM Proxy Registry

Companies also want to grasp package management in their projects, but maybe some of them need to be private to protect their business and intellectual property.

NPM offers the service of using private packages but as you can imagine, it charges the company for that.

A solution for that would be using an npm proxy.

An npm proxy is just a server that sits in front of the official npm registry server and resolves the dependencies to you.

You can publish an internal package using it and instead of the proxy pushing your package to npm server, it'll store it in its own server.

By running npm install using a proxy server, under-the-hood you'll pass a list of packages you want to install. If the proxy has a package published in its "database", it'll return to you that package. If not, it'll ask for the NPM server that package and return it to you:

Private packages are one of the capabilities of npm proxies.

Imagine you fork axios and publish to your proxy server that modified version. When you run install, the proxy instead of returning axios from NPM, will return to you the version you've published.

You might have been thinking:

Hmm... If we can have this set up in a cloud server... what if we use that system locally?

Yes... that was the conclusion I've made while observing that solution and that's how we'll tackle the problem of testing packages locally.

There are a couple of options to perform this workflow, but in this article, I'm gonna show you how to do that using Verdaccio.

Verdaccio

Verdaccio is:

A lightweight open-source private npm proxy registry

It's tremendously powerful and in version 4 it has NO config to start using it, which does not necessarily mean we won't need to do configs, but not really in the proxy itself.

Running a local server

The easiest way of using Verdaccio locally is installing as a global npm dependency:

yarn global add verdaccio

## Or with npm

npm install -g verdaccio

After that, you can start the server by running:

verdaccio

If you prefer, you can also run this server with docker:

docker run -it --rm --name verdaccio -p 4873:4873 verdaccio/verdaccio

Keep in mind that when the container will be stopped, the packages you've to publish would be removed.

After running the server, you can check the website at http://localhost:4873 (or http://0.0.0.0:4873)

Adding your npm user

To be able to publish a package into your local Verdaccio, you first have to register an npm user there. To do that, run:

npm adduser --registry http://localhost:4873 # OR http://0.0.0.0:4873

The information does not need to be secure or accurate. Remember, it's only a local thing! :)

Publishing and consuming

For both publishing and consuming your local package, you have to always specify what's the registry URL. In other words, what's the server npm must find the packages.

One way of doing that is by creating in the root level of the repo you want to consume/publish a file called .npmrc and specify the registry link there:

# /my-project/.npmrc

registry=http://localhost:4873 # OR http://0.0.0.0:4873

I strongly recommend this method for npm users. The reason is that npm asks you to sets a global registry via npm config set or via publishConfig.registry in your package.json. Both ways are a hassle to rollback when you want to use the regular npmjs.org registry.

With a custom .npmrc per project when you wanted to use from the official registry all you have to do is comment out the registry line in that file.

The other option is for Yarn users which consists in specifying the flag --registry:

# For publishing
yarn publish --registry http://localhost:4873 # OR http://0.0.0.0:4873

# For consuming
yarn add my-private-pkg --registry http://localhost:4873 # OR http://0.0.0.0:4873

By doing that, yarn will resolve the registry without any extra file nor config. If you eventually get annoyed with having to write the registry flag, you can also create a .npmrc file and yarn will also be able to resolve the registry URL from there.

After this configuration, when you publish or install your local package, npm or yarn will first ask your Verdaccio local server for that package, and Verdaccio will do all the job to store or retrieve local packages and resolve public packages at NPM.

Caveats

When we install a dependency, a bunch of information about it is added in package.lock.json or yarn.lock file and one of them is resolved, the URL where we got that dependency:

// package.lock
{
  // ....
 "node_modules/my-package": {
      "version": "1.6.0",
      "resolved": "http://localhost:4873/my-package-1.6.0.tgz",
   // ....
  }
}

This means that if you commit and push the lock file with the local server URL, wherever your project will be built or tested (like a CI/CD), it will fail because there's no localhost:4887 there.

In that sense, we always want to remember to clean this change before pushing any code.

Also, differently from Symlink where you can just turn a watch mode to compile the files and see these changes directly in your project, using this approach, you'll need to publish a new version and install that version every new change you've made.

Conclusion

I hope you enjoy this gotcha.

With a simple solution like that, now you're capable to test a production publishing/consuming a package workflow without too much trouble.

References

Module Resolution or Import Alias: The Final Guide

Raul Melo — Sun, 29 Nov 2020 17:23:16 +0000

Hello, devs.

This is might be an old topic but I think it still can be a bit confused when you try to do this configuration:

How can I add aliases to my imports?

The idea here is to do not stitch in any specific framework/tool but to give you an idea of knowing what and how to do, based on your application setup.

First, let's check what problem we're trying to solve, a solution in a more abstract way and how to configure your project to support that.

Table of Content

Problem
Solution
Configuration
- Editor completion
- Babel
- Rollup
- Webpack
- Jest
- Typescript
- Other Tools
Prefix Strategy
Conclusion

Problem

In well-structured JavaScript applications it's common we organize our codebase in a way that makes explicit what these files do or each domain they belong to.

Notwithstanding we don't have a standard of "how to structure", we always try to organize like components, helpers, models, etc. and sometimes, creating subfolders inside these folders and as consequence, creating a deeply nested structure.

Let's see an example of a tiny and not too nested app folder structure:

.
├── package.json
├── src
│   ├── components
│   │   └── Button
│   │       ├── Button.js
│   │       └── index.js
│   ├── helpers
│   │   └── text.js
│   └── index.js
└── yarn-error.log

This is a very basic setup where:

We have our src (source) which holds our app code;
src/components which holds all our app components;
src/helpers, which holds all our helpers/utilities which can be used anywhere in our code

Now let's say we to import a function called sanitizeText from our helper text inside our Button component. We would do something like:

import React from "react";
import { sanitizeText } from "../../helpers/text";

export const Button = ({ text }) => {
  return <button>{sanitizeText(text)}</button>;
};

It's not so bad, but as soon as you start having more and more imports from different folders and levels, it starts to become more confusing.

Also, you always need to guess how many levels you have to go up and down to import your code. Of course, modern code editors can help you out with that by just showing you which level you are and sometimes even completing it for you, but still.

Another problem is that if you eventually create a subfolder (for whatever reason you might have), you need to fix ALL imports by adding another "go up" level in the path.

That's not too much work but isn't by far optimal. We have a better way to do that and it's via module resolution or import alias

Solution

Module resolution or import alias is a way we can emulate the same way we import node_modules but with our internal code.

We can say to the tool we're using:

Hey, when you encounter "helpers/text", could you consider "./src/helpers/text" please?

In the same example above we would have some code like this:

import React from "react";
import { sanitizeText } from "helpers/text";

export const Button = ({ text }) => {
  return <button>{sanitizeText(text)}</button>;
};

Much cooler, right?

You'll import from helpers and it does not matter how deep you are in your app structure. Even if you move around this file, the imports will be always correct.

At the end of the day it's like import a library in our project. Imagine if you have to every time go up and down inside node_modules until you find your dependency:

import React from "react";
import { sanitizeText } from "helpers/text";
import { camelCase } from '../../../node_modules/lodash'

export const Button = ({ text }) => {
  return <button>{camelCase(sanitizeText(text))}</button>;
};

This would be hell. A lot of waste of energy having to navigate to the code.

Configuration

JavaScript itself does not allow us doing that those imports natively. But luckily we are always using a tool that supports that (e.g. Babel, Webpack, etc).

You might think:

Nice, I only need to do once this configuration!

And the answer is: it depends... but probably not.

The more tooling you introduce in your application, the more complicated it becomes to get it configured. Here some examples where this could be tricky:

If you're working on a JS project only with Babel and add jest to test your app, you'll need to add aliases in both places.
If you're working in a JS project with some built-in Webpack config and you add Storybook, you'll need to add an alias in both webpack.config.js and also customize Storybook babel.config.js.
If you're working on a JS project with all these configurations and want to move to TypeScript, you'll need to keep this configuration everywhere plus configure the tsconfig.json file to let TS aware of how to resolve those aliases.

As you can see this can be puzzling but here I want to give you an understanding of each possible tool. After that, you'll check how your project works and do all configurations needed to enable this feature into your project.

Editor completion

Before we dive deep into the configs, let's talk about Developer Experience (DX).

If you use VSCode, you probably already notice that when you need to import something, because VSCode uses TS, and it does a lot of inference, usually it's possible either to automatically imports the method you're trying to use or have an autocompletion for the imports, right?

When you just configure alias in Babel, for example, you kinda lost that and that's sucks.

If we want to let VSCode aware of how to suggest these modules, we need to create a file called jsconfig.json (in root level), which it's a JavaScript version of tsconfig.json and also declare those alias there:

{
  "compilerOptions": {
    "baseUrl": ".",
    "paths": {
      "*": [
        "./src"
      ],
      // Your custom aliases here
      "helpers/*": [
        "helpers"
      ],
      "models/*": [
        "models"
      ]
    }
  }
}

Every time you add a new alias to your project, all you need to do is come to this file and add your new alias.

If you use Typescript you don't need this file because this configuration will be made there.

Now, let's dive deep into the specifics.

Babel

Description

If your project has a babel.config.js or .babelrc file at the root level, you'll probably need to do this configuration.

Configuration

First, you need to install the plugin babel-plugin-module-resolver:

yarn install -D babel-plugin-module-resolver

Then, add this plugin into your babel config file:

{
  "presets": [
    // ....
  ],
  "plugins": [
    // ....
    [
      "babel-plugin-module-resolver",
      {
        "root": [
          "./src"
        ],
        "alias": {
          // Add your aliases here
          "helpers": "./src/helpers",
          "models": "./src/models"
        }
      }
    ]
  ]
}

After this, you'll be able to import files from helpers/<file> and models/<file>.

Resources

Babel Only Demo: https://github.com/raulfdm/aliases-post-demo/tree/main/babel-only
babel-plugin-module-resolver: https://www.npmjs.com/package/babel-plugin-module-resolver

Rollup

Description

Rollup is one of the most annoying tools to do this configuration. That's because the official plugin to do that isn't smart enough to recognize index.js imports.

If you try to do something like models/User/ the plugin will complain saying that "models/User" is a folder, not a file. In that sense, you'll need to give full import, like models/User/index.

After some tests, I've realized that isn't necessarily the plugin's fault. If we try to enable this feature via @rollup/plugin-babel, we still need to import index at the end of our import. It seems that that's the way Rollup handles import and we cannot do anything about it.

Configuration

The first step is installing @rollup/plugin-alias.

After that, in your rollup.config.js you import it and set it up:

import alias from "@rollup/plugin-alias";
import path from "path";

export default {
  input: "src/index.js",
  output: {
    format: "cjs",
    dir: "dist",
  },
  plugins: [
    alias({
      /**
       * For custom files extension you might want to add "customerResolver"
       * https://github.com/rollup/plugins/tree/master/packages/alias#custom-resolvers
       *
       * By doing that this plugin can read different kind of files.
       */
      entries: [
        {
          find: "models",
          replacement: path.resolve(__dirname, "src/models"),
        },
        {
          find: "helpers",
          replacement: path.resolve(__dirname, "src/helpers"),
        },
      ],
    }),
  ],
};

Remember: imports from index.js files DOES NEED to be full imports

Resources

Rollup Demo: https://github.com/raulfdm/aliases-post-demo/tree/main/rollup
Rollup + Babel Demo: https://github.com/raulfdm/aliases-post-demo/tree/main/rollup-babel
@rollup/plugin-alias: https://github.com/rollup/plugins/tree/master/packages/alias#readme

Webpack

Description

Webpack allows us to do the same as Babel but via its configuration. In case you're working on a project which already had a webpack.config.js somewhere in the code, it's there you'll need to tweak.

Configuration

const path = require('path');

module.exports = {
  // ...
  resolve: {
    alias: {
      // Your custom aliases here
      // __dirname will stands for your root directory level
      // path.resolve will concatenate your project folder location with the aliased folder
      helpers: path.resolve(__dirname, 'src/helpers/'),
      models: path.resolve(__dirname, 'src/models/')
    }
  }
};

If you're working with a framework/platform which works with Webpack under the hood and allows you to extend its configuration, the solution will be slightly different but the same concept.

For example, I'll show you a next.config.js file from NextJS which allow us to extend their configuration:

const path = require('path');

module.exports = {
  webpack: (config) => {
    // Mutate the alias configuration
    config.resolve.alias = {
      // Spread everything to avoid remove any alias they might have
      ...config.resolve.alias,
      // Your custom aliases here
      helpers: path.resolve(__dirname, 'src/helpers/'),
      models: path.resolve(__dirname, 'src/models/')
    }

    // Important: return the modified config
    return config
  },
}

Resources

Demo Webpack only: https://github.com/raulfdm/aliases-post-demo/tree/main/webpack
Extending Webpack NextJS: https://github.com/raulfdm/aliases-post-demo/tree/main/next-webpack
Webpack Docs: https://webpack.js.org/configuration/resolve/#resolvealias

Jest

Description

Now that we already have our aliases working in our regular environment, let's see how can we make Jest aware of how to resolve our aliases

Unfortunately, their API for that is by far one of the most complicated ones. That's because they use for stub out mocks and their test stuff and not for our organized code. But luckily we can grasp it.

To do that, first, you need to understand where your jest config is. It can be inside of package.json or in a jest.config.js file in the root level of your directory.

Configuration

module.exports = {
  // ...
  moduleNameMapper: {
    // Your custom aliases here
    "^helpers/(.*)": "<rootDir>/src/helpers/$1",
    "^modules/(.*)": "<rootDir>/src/modules/$1",
  },
};

Resources

TypeScript

Description

For TS projects, we usually have to attack in two ways:

Configure the transpiler/compiler/bundler tool (e.g. Webpack, Babel)
Configure tsconfig.json

That's because those tools use a lot of internal tools AND Typescript to generate your final files and for Typescript, what matters is your tsconfig.json file.

Also, you might want to add tsc --noEmit as a fancy linter to be sure that at least your code has no errors and will compile right.

Configuration

Open your tsconfig.json file and set a baseUrl and paths:

{
  "compilerOptions": {
    // ...

    /* Base URL is a MUST. */
    "baseUrl": ".",
    /* Your aliases will live here */
    "paths": {
      "models/*": [
        "src/models/*"
      ],
      "helpers/*": [
        "src/helpers/*"
      ],
    }
  }
}

Tip: If you're using VSCode, usually you'll probably need to restart it to be able to navigate these aliased imports.

Just to be clear, baseUrl: "." will tell Typescript to consider the root level of your project to build the paths. In that sense, TS will find "models" and consider something like <root>/src/models/*.

It's also common for people defining baseUrl to be src. If you do so, you can remove src from your paths:

{
  "compilerOptions": {
    // ...

    /* Base URL is a MUST. */
    "baseUrl": "src",
    /* Your aliases will live here */
    "paths": {
      "models/*": [
        "models/*"
      ],
      "helpers/*": [
        "helpers/*"
      ],
    }
  }
}

Resources

Typescript demo: https://github.com/raulfdm/aliases-post-demo/tree/main/typescript
Typescript Docs: https://www.typescriptlang.org/docs/handbook/release-notes/typescript-2-0.html#base-url

Other tools

I tried to focus on the most common tools we use independently a specific framework but each one can have some specificities on how to extend or do this in an easy way.

My suggestion for you to figure this out is: always search at Google for <framework-or-tool-name> import alias. Usually, you'll find the answer in one of the top 3 results.

Prefix Strategy

In all examples, I just use regular names for our aliases but it's also a common practice is adding a prefix (a character before) to them.

Personally, I'm very fan of the prefix @ just because it's really nice to read import something "at" components but some people don't like this strategy because this special character is very popular for orgs, like @babel, @rollup, and it can mislead developers to think that this import is from an external resource.

As you notice, this is optional. Feel free to use or not whatever special char to give more clarity about internal X external imports.

Conclusion

I hope at the end of this article you feel comfortable to tweak your configs and enable this feature if you feel useful somehow.

Data Structure with JavaScript: Stacks

Raul Melo — Wed, 26 Aug 2020 15:02:14 +0000

Hello, devs.

This is the first article of many I want to write to document my studies of algorithms and data structure.

After I failed in an interview because of a sorting algorithm, I've decided to dive deep into some computer science concepts I've learned in the college.

Today's posts will be about the data structure Stack. I hope you can learn what's it and mainly how to implement this data structure in JS.

Table of Content

What is a Stack
- Last In, First Out (LIFO) order
- Methods
Implementation
- Stack
- Stack.push
- Stack.pop
- Stack.peek
- Stack.isEmpty
- Stack.size
- Stack.clear
Usage
- Washing plates program
- Decimal to binary problem
- Real-world use cases
Conclusion
References

What is a Stack

As I already told you before, Stack is a data structure which represents... guess what? a regular stack of things.

Imagine you're working in a kitchen as a kitchen porter and unfortunately, the washing machine just broke. Now you have to wash all plates by hand 😢.

The waiters and waitresses are bringing the client's plates to the kitchen and you have to gather all of them and organize in a way to make it easier to wash.

The best way to do that is stacking one plate on top of each other:

How are you going to start this duty?

Yes, that's correct, with the first plate on the top of the stack.

After finish that, you create another stack of clean plates until your task is done.

Last In, First Out (LIFO) order

The problem you just solved in the kitchen had a well-known sequence called LIFO, or Last In, First Out. Still, in the example, the last plate you stack is the first one you're gonna wash.

In that sense, the data structure Stack can be used in any problem you might solve that you need to create a list of things in a specific sequence and then removing them from the last added to the first.

Later in this article, we'll implement 2 exercises, one script to wash the plates for us and another (a bit more practical) that converts numbers to binary.

Methods

The Stack methods are divided by essential and non-essential:

Essential

These two methods are a must in any Stack implementation, does not matter what programming language you're using:

push - to add an element;
pop - to remove the latest added element.

Non-essential

Also, there are a couple of nice-to-have methods which can be different across other languages, especially in the naming. They are:

peek - to get what is the element on top of our stack (does not remove it though);
isEmpty - to check if our stack is empty;
size - to check how many elements we have there;
clear - to completely clean up the stack.

It does not seem complex, right? And trust me, it isn't. Let's check now how we would implement that.

Implementation

To implement a Stack we'll be using our old friend Array, after all, a Stack is just a vertical list of things, right?

To get some encapsulation, I'll use regular functions but in a Factory way so that any instance of the stack will have direct access to the items.

It can be also be written using class syntax our the old school function + its scope, but again, doing in that way the instances will have access to the items list which isn't the desired behavior unless you're reading this article in the future and private attributes in class are already in the language (or just using a babel preset).

https://github.com/tc39/proposal-class-fields#private-fields

At the end of this article, I'll write those 2 other versions if you're curious about it.

Stack (basic structure)

So let's start by creating our function:

function Stack() {
  let items = [];

  return {};
}

Pretty simple. We:

creates our function Stack (camel case because it represents a class);
creates an array called items where all our data will be stored.
return an (temporary) empty object but which exposes the Stack methods we want to make public.

Stack.push

Let's start one of the required methods Stack.push method.

Since we're using an array to control our stack elements, we can just use the native array method push:

function Stack() {
  let items = [];

  function push(element) {
    items.push(element);
  }

  return {
    push,
  };
}

Very forwarded. We:

create an internal function called push which accepts an element and push it into the items list;
make this function publicly available so we can do myStack.push(<element>).

Stack.pop

Time to implement the other required method: Stack.pop.

Here we'll also be using the native Array.prototype.pop, that removes the last element in a list and return this removed value:

function Stack() {
  let items = [];

  function push(element) {
    items.push(element);
  }

  function pop() {
    return items.pop();
  }

  return {
    push,
    pop,
  };
}

Stack.peek

Now it's time for the nice-to-have-methods. Let's start by implementing the Stack.peek method.

Here we want to return the element on top of our stack, or, the last element in our list WITHOUT removing it. It's just for a matter of knowing what's on the top.

function Stack() {
  let items = [];

  function push(element) {
    items.push(element);
  }

  function pop() {
    return items.pop();
  }

  function peek() {
    return items[items.length - 1];
  }

  return {
    push,
    pop,
    peek,
  };
}

If you are still learning JS, keep in mind that array indexes start at 0. If we have a list ['A', 'B', 'C'], it'll be represented by:

index 0: 'A'
index 1: 'B'
index 2: 'C'

However, list.length will be 3. If we want to pick the latest, we always need to get the length (3) and subtract 1 so then we respect the index 0-base from a JS list.

Stack.isEmpty

Next is the method Stack.isEmpty that will just evaluate if our stack (aka array) has a length equals zero:

function Stack() {
  let items = [];

  function push(element) {
    items.push(element);
  }

  function pop() {
    return items.pop();
  }

  function peek() {
    return items[items.length - 1];
  }

  function isEmpty() {
    return items.length === 0;
  }

  return {
    push,
    pop,
    peek,
    isEmpty,
  };
}

Stack.size

Then we have the Stack.size method that will be returning the length of our array.

The only difference between length and size is the naming convention commonly used in other languages (at least I couldn't find a good explanation, if you know, please leave a comment).

function Stack() {
  let items = [];

  function push(element) {
    items.push(element);
  }

  function pop() {
    return items.pop();
  }

  function peek() {
    return items[items.length - 1];
  }

  function isEmpty() {
    return items.length === 0;
  }

  function size() {
    return items.length;
  }

  return {
    push,
    pop,
    peek,
    isEmpty,
    size,
  };
}

Stack.clear

Next is Stack.clear that will simply throw away the current stack and replace it with an brand-new and empty one:

function Stack() {
  let items = [];

  function push(element) {
    items.push(element);
  }

  function pop() {
    return items.pop();
  }

  function peek() {
    return items[items.length - 1];
  }

  function isEmpty() {
    return items.length === 0;
  }

  function size() {
    return items.length;
  }

  function clear() {
    items = [];
  }

  return {
    clear,
    push,
    pop,
    peek,
    isEmpty,
    size,
  };
}

The reason I created items using let was to make this process easier. We could have some functional approach here but I don't see anything wrong with reassigning values in a controlled scope.

And that's it. Our data structure is done.

If you're curious to see this code using class or function this, check it here:

old school function scope syntax

function Stack() {
  this.items = [];

  this.push = function (element) {
    this.items.push(element);
  };

  this.pop = function () {
    return this.items.pop();
  };

  this.peek = function () {
    return items[this.items.length - 1];
  };

  this.isEmpty = function () {
    return this.items.length === 0;
  };

  this.size = function () {
    return this.items.length;
  };

  this.clear = function () {
    this.items = [];
  };
}

const stack = new Stack();

Be aware that items will not be private in stack instance, which means that doing stack.items will be possible to manipulate the list out of our "predefined rules".

class syntax

class Stack {
  constructor() {
    this.items = [];
  }

  push(element) {
    this.items.push(element);
  }

  pop() {
    return this.items.pop();
  }

  peek() {
    return this.items[this.items.length - 1];
  }

  isEmpty() {
    return this.items.length === 0;
  }

  size() {
    return this.items.length;
  }
}

const stack = new Stack();

It has the same problem described in the old school function scope syntax, items will be available publicly.

There are a couple of ways to try to guarantee that until we don't have private fields natively but I won't dive deep into that in this post.

Usage

Now we have our Stack data implemented, let's try it out:

const stack = Stack(); // create a new stack (new instance of it)

console.log(stack.isEmpty()); // true
console.log(stack.size()); // 0

// Pushing up some values
stack.push("Zilmira");
stack.push("John");
stack.push("Joel");

console.log(stack.isEmpty()); // false
console.log(stack.size()); // 3
console.log(stack.peek()); // Joel

const removedElement = stack.pop();

console.log(removedElement); // Joel

console.log(stack.isEmpty()); // false
console.log(stack.size()); // 2
console.log(stack.peek()); // John

stack.clear();
console.log(stack.isEmpty()); // true
console.log(stack.size()); // 0

Nice, now we have a new type (custom) in our application where we can use it.

Examples

Ok, now we already now what is a Stack and have it implemented. Let's apply it into some problem solution.

Washing plates program

Imagine that now you're tired of washing plates by hand and will create a robot to do this duty for you.

Time to grasp our new data structure to solve that.

First, let's create our barebone function washPlates that receive a list of plates:

function washPlates(plates) {}

Then, we create a variable that hold how long it takes to wash a single plate (to avoid magical numbers) and also a stack of plates:

function washPlates(plates) {
  const timeToWashAPlateInMilliseconds = 2000; // Long but descriptive
  const plateStack = Stack();
}

Now, we have to fill our plateStack with all plates received. So let's iterate through it and add them to the stack:

function washPlates(plates) {
  const timeToWashAPlateInMilliseconds = 2000;
  const plateStack = Stack();

  plates.forEach((plate) => stack.push(plate));
}

Note that this logic could be implemented in many ways, like using the regular for or for...of loop for instance.

Then, let's just add some console messages to make easy to understand what's going on and start an iteration through our stack:

function washPlates(plates) {
  const timeToWashAPlateInMilliseconds = 2000;
  const plateStack = Stack();

  plates.forEach((plate) => stack.push(plate));

  console.log(`I have ${platesStack.size()} plates to wash!`);
  console.log("Starting the duty!");

  while (!platesStack.isEmpty()) {
    // do something
  }
}

Note that in our while will keep doing until the stack has items.

Now, we need to take the plate we're gonna wash and do the job.

To emulate that and make it easier to make this code runnable, I'll create a self-implemented sleep utility that will represent the act of washing the plate. But don't pay much attention to that.

// A code to block the execution after X time
function sleep(timeout) {
  return new Promise((resolve) => setTimeout(resolve, timeout));
}

async function washPlates(plates) {
  const timeToWashAPlateInMilliseconds = 2000;
  const plateStack = Stack();

  plates.forEach((plate) => stack.push(plate));

  console.log(`🤖 says: I have ${platesStack.size()} plates to wash!`);
  console.log("🤖 says: Starting the duty!");

  while (!platesStack.isEmpty()) {
    const currentPlate = platesStack.pop(); // Get the plate on the top
    console.log("🤖 says: Start washing plate:", currentPlate);
    await sleep(TIME_TO_WASH_A_PLATE_IN_MILLISECONDS); // Wash it
    console.log(`🤖 says: Plate ${currentPlate} done.`); // We're done with this plate
  }

  console.log("🤖 says: All plates are cleaned!");
}

To pause this function execution using sleep, I needed to make this function async and explicitly await until it's done.

So here we get the plate on the top of our platesStack to wash it using the pop method.

Now if we run this program passing 5 plates, we'll have:

washPlates([1, 2, 3, 4, 5]);

// 🤖 says: I have 5 to wash!
// 🤖 says: Starting
// 🤖 says: Start washing plate: 5
// 🤖 says: Plate 5 done.
// 🤖 says: Start washing plate: 4
// 🤖 says: Plate 4 done.
// 🤖 says: Start washing plate: 3
// 🤖 says: Plate 3 done.
// 🤖 says: Start washing plate: 2
// 🤖 says: Plate 2 done.
// 🤖 says: Start washing plate: 1
// 🤖 says: Plate 1 done.
// 🤖 says: All plates are cleaned!

Cool, right?

Of course, we could solve this problem in various ways but since our problem fits perfectly the Stack data structure, why not just give it a shot?

Decimal to binary problem

Ok, time to solve a more (not much) realistic problem. Let's implement a function that converts a decimal number and returns a string with the binary representation of it.

There are a few methods to do that and the one we're going to use is by division and it fits perfectly using Stack to solve that because we need to store the result operation in a LIFO sequence (it'll be clearer later).

If you want to learn in-depth how it works you can watch the following video:

In a nutshell, we'll divide the received decimal number by 2 using the Remainder operator (%) and store the rest (0 or 1) in a Stack until the number is zero.

After that we'll compose our binary popping out our stack.

Ok, let's starting by creating the function:

function decimalToBinary(decimal) {}

Then, let's create a new Stack and a few variables of control:

function decimalToBinary(decimal) {
  const binaries = Stack();

  let nextNumber = decimal;
}

Here:

binaries a stack which will hold the binary value from each division;
nextNumber will hold the next number we need to divide.

Then, let's vary a bit and use a do...while loop with the implementation:

function decimalToBinary(decimal) {
  const binaries = Stack();

  let nextNumber = decimal;

  do {
    let remainder = nextNumber % 2;
    binaries.push(remainder);

    nextNumber = Math.floor(nextNumber / 2);
  } while (nextNumber !== 0);
}

Here we:

creates a variable to hold the remainder of this operation (it could be done in a single line within the push);
pushes the remainder to our binary stack;
divides nextNumber by 2 (bi...nary) ignoring floating points with Math.floor

This loop will happen until nextNumber is something but 0, we don't want to divide 0, right?

Last part will be looping through our stack of binaries and create our result:

function decimalToBinary(decimal) {
  const binaries = Stack();

  let binaryResult = "";
  let nextNumber = decimal;

  do {
    let remainder = nextNumber % 2;
    binaries.push(remainder);

    nextNumber = Math.floor(nextNumber / 2);
  } while (nextNumber !== 0);

  while (!binaries.isEmpty()) {
    binaryResult += binaries.pop();
  }

  return binaryResult;
}

Here we:

create the variable binaryResult. I just moved it to the top to put together if all other variables;
loop through our stack until it becomes empty and concatenate all elements using the Assign addition operator (+=);
finally return the result.

Let's test it out:

console.log(decimalToBinary(123)); //> 1111011
console.log(decimalToBinary(332112)); //> 1010001000101010000

Real-world use cases

Both problems still seem a bit vague, I mean, when we need to implement a binary converter or fake software to washing plates, right?

While reading the real examples of Stack use, I found a common problem I believe a lot of people need to solve or already thought how to solve: "Undo" action.

Imagine you have a stack of elements and the user could simply remove them. A possible implementation would be pop the last element and hold it for a couple of sections. If the user clicks in a undo button, you just push this element again on top of your stack.

Another nice and advanced use case is in Redux dev tools. Every single action you dispatch is put into a stack. So if you want to go back and forth in a replay mode is just a matter of pushing and popping elements from the stack.

Conclusion

In this article, we learned what is a Stack, how to implement it in JavaScript, and most importantly using it to solve problems.

Think data structure as tools. As much bigger it's your toolbox, as much easier will be to solve a specific problem.

I hope Stack is in your toolbox now.

Thanks if you read until this point.

References

I failed an interview because of an algorithm

Raul Melo — Sun, 16 Aug 2020 11:57:36 +0000

Photo by Siora Photography on Unsplash

Hello, devs.

For the last couple of weeks, I've been participating in a hiring process for a big company.

It was a very nice experience and it gave me a lot of insights about hiring, code challenges, and how (possibly) to become more prepared for interviews.

As you already saw in the title, I didn't pass, but I went through all the stages of it. So it might be interesting for you if you're trying to get your first job, or even if you're thinking of switching jobs but don't do interviews for a couple of years already.

I'll explain all stages, with some details and thoughts, but my main focus here is to tell where and why I failed in the last part of it.

Hope you like it and I hope my experience can help you somehow!

Table of content

Stage 1: Getting an interview
Stage 2: Human Resources (aka HR)
Stage 3: The managers
Stage 4: the HR (again)
Stage 5 (latest): The devs
- About me
- Technical questions
  - JavaScript questions
  - CSS questions
  - HTTP questions
- Technical Challenge
- After the test
The last HR Call
Thoughts about hiring X our skills
Learnings
Conclusion

Stage 1: Getting an interview

A very common practice in companies, in general, is to motivate people (with money) to recommend professionals to fill opening jobs (that's a good reason why you should build good connections as much as you can with people in and out of the internet).

A friend of mine recommended me to work in his company. Good salary, very well structured and it seemed a very good place to work in, why not try it out?

I just needed to do some adjustments to my online CV and I was ready.

Stage 2: Human Resources (aka HR)

The HR from the company just called me to know a bit about me.

They asked about all my experience, things I've done in my life, whether I had some specific knowledge (like React, testing, Redux, etc) and also questioned about the company I'm working on.

To be honest this call just seemed to be a filter for people who lied on their CV, because everything was very basic and trivial.

I was pretty confident in all the answers I gave to her, and at the end of the call she just gave me a positive feedback and then she'd schedule the next phase with the tech lead and team manager.

Stage 3: The managers

The following week, I had a 1-hour meeting (now with webcam) with the "managers".

Their job was getting to know about my soft skills. A lot of questions like:

What is the proudest moment of my career
What was the worst technical decision I have ever made
How I deal with conflicts
How I deal with top-down decision
How I handle hard discussions

And in between these questions, they constantly asked me to give examples of it, some situations I have been passed that prove I think or solve a problem in the way I was doing.

It was very nice because I could remember some good and bad moments in my career which made me grow up a lot as a professional and also as a person.

When I turned the call off, I was still very confident about the final result.

After a couple of days, the HR called me saying the managers liked me and the way I answered their questions (maybe someday I can share in details some situations I've passed and my current way to deal with people and job).

The only feedback she gave to me was: "I needed to be less of a follower".

And... of course, they thought I was more of a follower than a leader or something because, to be honest, I'm almost 30 and I kinda lost the energy to keep fighting pointless discussions about technical tastes and personal ways to code.

I have my opinions and I always can argue why I have them, but to be honest, when the discussion reaches the moment where it's an ego thingy or personal flavor, I just give up and let people do what they want.

I do not care about:

what's the best JS framework;
what's the best state management library;
if we should use reduce or map + filter
if we should use it or test in jest assertions

I do care about:

what solution fits best for the problem;
good naming;
tested code;
a simple solution when it's possible;
ship the product;
my salary at the end of the month;

In the past, I used to have endless debates about what's best, what's the worst. Nowadays I still have these discussions but if I notice that we're not going anywhere, I just agree and live my life.

But of course, don't be a fool. If I'm convinced that some bad decisions are being made and I can prove that, I'll go until where's needed to defend that.

From a company’s perspective, what's the most valuable personality? We can have a long discussion about this.

Stage 4: the HR (again)

The next phase was again a HR call and she asked me some questions the managers made to me the week before.

She wanted to see for herself my soft skills.

Well, I didn't lie in any of the questions I gave the same answers.

I also tried to explain why the managers thought I was more follower like and it seemed she had got it.

She gave another positive feedback and then told me the next step would be with 2 devs to test my hard skills and a QA manager would be present just to watch the process.

Stage 5 (latest): The devs

This one was the trickiest stage and I want to give you full details about how it went. I want to recommend to you guys some articles that might answer those questions if you don't know.

I received an email with some topics they could ask like JS, CSS, Testing, State management, React, and Http but still wasn't clear how they would approach those concepts.

It was scheduled to be 1h30m short but it started 10 min late.

When I joint the chatroom, as always we did some round presentation and they explained to me how it would work:

20~ min to get know about what I currently do;
30 min to answer and explain technical concepts;
30 min to do a code challenge.

About me

Here again, I had to explain what I do in my current job, but more technically.

Well, it's only me and another guy responsible for the entire web ecosystem so I kinda know what the product is and how things are structured.

I had explained to them in detail about what's the platform, my responsibilities, what’s my daily tasks to give to them an idea and how we're handling them, etc.

Sometimes they asked me to dive deeper in some topics but it was easy at the end.

Technical questions

Here starts to become a bit tricky because they started a certain quiz. I'll try to not dive deep into explanations and solutions but instead, just give my impression, a short idea about it and share an article explaining in detail those questions and concepts.

JavaScript questions

Can you explain the difference between var, let, and const?

It can be indeed tricky, especially if you have started learning JS recently. I believe people just say: always prefer const but if you need to reassign then use let.

Luckily I've passed the transition from var to let and const and remember clearly it's about scope.

To explain that I used the object mutation even when we use const, so they got a second question from here:

Var, let and const- what's the difference?

Sarah Chima ・ Oct 25 '17

#es6 #javascipt #newbie

How can you freeze an object?

I know this concept because I caused some bugs while learning ES6. After a few hours trying to know why my object was being mutated, I've found documentation about the native API Object.freeze.

It's very straight forward. When we freeze an object any operation which will mutate an object does not change the object itself. Be aware that this freeze only happens at a shallow level (1 object-level deep) and does not freeze objects and arrays nested.

const 📦, seal 🤐, freeze ❄️ & immutability 🤓 in JS

Benjamin Mock ・ Jan 11 '20

#javascript #webdev #beginners #tutorial

Can you explain what DOM is?

I think that was the easiest one, despite the fact I forgot what the acronym means (Documment Object Model).

DOM Tutorial

Sukhbir Sekhon ・ Jun 1 '20

#javascript #html #css #dom

Can you explain how events propagate in the DOM?

I knew this question because before I learned React I've created my own JS framework (it was a nice experiment) where I learned a lot about capturing event in different levels and also how bubbling works.

Event Propagation in Three Parts!

Jesse ・ Oct 31 '19

#javascript #event #propagation #capture

What's the advantage of using a single event to capture events?

It was still related to this question and to be honest, I wasn't sure.

I remember when I had implemented that in Vanilla JS where I had a form and instead attached onChange to all inputs, I just did in the form level and had some sort of switch to decide what information to update or what to do but... I've forgotten why this was good. My assumption was about performance.

The dev told me it was exactly about this and gave me a very good example. Imagine a table with thousands of records. If we attach a single event to each row we would have for sure performance issues because the DOM will have thousands of events to dispatch.

I really liked this example.

Event Delegation - What is it and why should I care?

Paul Ryan ・ Apr 17 '20

#javascript #html #event

Do you know what the Request animation frame API is and why we would use that?

I think I've used Request Animation Frame (RAF) API once or twice in 5 years so I was not sure why such API exists, then I tried a guess: “tell the browser that we're gonna do some animations and get some performance.”

They told me it was correct.

Creating animations in Javascript using requestAnimationFrame

Sriram ・ Jul 4 '20

#css #javascript #animation

CSS questions

Do you know about the different displays?

Didn't get much this question. I wasn't sure if I needed to explain each display or just say: "yes, I know".

But my answer was saying I have a few displays we have like inline, inline-block, block, grid, flex, etc.

To my surprise, it's what they wanted to hear.

CSS Display property

Hieu ・ Jun 15 '19

#css #layout

What's the difference between display inline and display inline-block?

This I didn't know by heart.

I knew the difference between inline and block because it's a very common thing to do in CSS but I couldn't remember the difference between inline and inline-block.

The dev explained to me that when we have inline-block we can define margin and padding to the element while in display inline we can't.

CSS Basics Layout Fundamentals: Block, Inline & Inline Block

Smeet J Thakkar ・ May 7 '20

#todayilearned #beginners #codenewbie

How can I center something vertically?

Also another tricky question and maybe because we have several ways to do that.

The way I know by heart is using display: flex combined with justify-content: center and align-items: center.

My answer was good enough.

How to Vertically Align Anything with CSS

Lauralee Flores ・ Apr 25 '18

#css #beginners #webdev #frontend

HTTP questions

Can you tell us what methods are available in HTTP?

Not that difficult since I sometimes play with backend in Node.

GET
POST
DELETE
PUT
PATCH

Then they asked me if I knew the difference between PUT and PATCH and that's tricky.

I remembered one replaced the entire data with the new content and the other just updated the information sent, but I didn't know which one did what.

They told me it was right but who does partial updates is PATCH and a full replacement is PUT.

On top of this question they asked if I knew about other methods, like OPTIONS and to be honest, I didn't.

Often I see OPTIONS requests in my network tab but I have never asked myself what hell was that.

Difference Between Those HTTP Requests

Milecia ・ Jun 18 '19

#beginners #webdev #http #server

How the HTTP status codes are divided?

Basic HTTP things.

I just remember that by heart because since I started building apps that consume APIs, I was curious to know what's the thing about these codes and I found this https status dogs which it's an amazing way to understand each code. 😁

The HTTP Status Codes You Need to Know

Mark Michon for Bearer ・ Feb 20 '20

#beginners #http #webdev

Technical Challenge

I had the feeling I ruled the quiz questions and I was much ready for the technical challenge.

"30 min might be enough to solve something." I thought

Then they sent to me a link to a Code Sandbox with a single JS file:

// index.js
/*
We have lots of deployment dashboards where we have drop-down UI to select an application version to deploy. We want to ensure that software version numbers delivered to our UI from the back-end API are always ordered with the newest (highest) available version at the top.

Eg:
const input = [0.1.0, 3.2.1, 2.2.3, 0.1.1];
sortVersions(input) should return [3.2.1, 2.2.3, 0.1.1, 0.1.0];
*/

const sortVersions = (unsortedVersions) => {
  // code
};

export default sortVersions;

And a test file with some assertions to ensure it was correct

// index.test.js
import sortVersions from './index';

it('should sort correctly numbered versions', () => {
  const versions = ['0.2.0', '3.1.2', '0.1.6', '5.0.0'];
  const output = ['5.0.0', '3.1.2', '0.2.0', '0.1.6'];
  expect(sortVersions(versions)).toEqual(output);
});

// Other assertions

Rules to solve that challenge:

I could use google to search about JS APIs
I couldn't use google to search for the problem specifically.

The first assertion was very easy. My solution was simply:

const sortVersions = (unsortedVersions) => {
  // code
  return unsortedVersions.sort().reverse();
};

And it passed! :)

Then I had to solve the second assertion:

// index.js

it('should sort correctly numbered versions with different lengths', () => {
  const versions = [
    '1.3.0.9',
    '0.2.0',
    '3.1.2',
    '0.1.6',
    '5.0.0',
    '3.3.3.3',
    '3.3.3.3.3',
    '3.10',
    '0.2.0',
  ];

  const output = [
    '5.0.0',
    '3.10',
    '3.3.3.3.3',
    '3.3.3.3',
    '3.1.2',
    '1.3.0.9',
    '0.2.0',
    '0.2.0',
    '0.1.6',
  ];

  expect(sortVersions(versions)).toEqual(output);
});

And here I just got stuck.

My previous solution didn't cover cases like this. It just returned:

[
  '5.0.0',
  '3.3.3.3.3',
  '3.3.3.3',
  '3.10',
  '3.1.2',
  '1.3.0.9',
  '0.2.0',
  '0.2.0',
  '0.1.6',
];

And the reason is kinda obvious for simple string sorting. By default, it goes alphabetically where 3.3 comes after than any 3.1. But in semantic versioning, we know that 3.10 is greater than 3.3.

After playing around with sort methods, I had an idea about making all versions the same size based on the largest version.

To be honest, at that moment I was already nervous because I couldn't think in a way to solve that "simple" problem and consequently I started to get anxious.

I had some tips from them about how I could solve that but it was a bit hard to understand. Not because they didn't explain exactly but because I was trying to figure out many things at the same time.

While I was doing some code to make all versions in the same length, time was over.

The dev asked me if I had more time how would I solve that and I explained that my ideas were making all versions in the same length so I could compare per blocks.

I thank them for their time and we turned off the call.

If you're curious about how to solve this problem I recommend you take a look into this answer in StackOverflow. I just tested it and it solves all specs.

After the test

To be honest with you after I turned the call I felt myself a failure.

In the past 5 years, I've already solved so much harder problems than that and at the end, I was beaten by a sorting problem, what was wrong with me?

But I tried to comfort myself: "Ok... I went well in all previous steps, even when they asked me technical questions.. this shouldn't count too much".

The last HR Call

After a week waiting to see if I was approved I would be accepted or not, the HR person called me saying:

"Unfortunately we won't proceed with you"

I only could feel very frustrated.

Then I asked why and if I could have feedback about what I could improve and she told me:

"The devs really like your communication, your energy while doing the interview. They liked that you answered most of the technical questions correctly but they thought you didn't go well solving the last problem.

hmm... ok, I said.

I could argue in many aways against this decision but let's be honest, the decision was made. It wouldn't change anything.

I turned off the call, stopped for 5 minutes to digest the feeling of failure, and tried to think about all this process.

Thoughts about hiring X our skills

I don't know if you already have had the chance to be the interviewer in your life. I had and I can say for sure: it isn't an easy task. And that's because the hiring process is very hard in general.

We need, somehow, to define steps and things to evaluate a candidate and despite our goal of being transparent and analytical, it isn't by far a perfect method.

People think, learn, solve problems, communicate differently from each other, and in a hiring process, we try to create some guides and filters that we expect about people.

It means that if someone just does not fit in what you expect, it might be a good signal that this person isn't ideal for the job. But... does it mean that this person would be a bad professional?

I don't think so.

I remember when I finally could get my first job as a developer, I made a similar code challenge and I just ruled that.

Do you want to know the difference between me 3 years ago and me now? At that moment I was constantly practicing solving algorithms in the FreecodeCamp platform.

After I got the job I proved myself a valuable employee and I can say for sure that the hard thinking I used to pass in the interview was not by far what made me ship product, because there were rarely moments I could not use Google to look how people solve this problem to create another solution on top of it.

I had some challenging problems to solve and I solved them all in a short or long time, sometimes with others' help, sometimes by myself.

To get my current job I had to create a React APP which consumes a public API, add tests (E2E), infinite scroll, animation between pages and host somewhere, explain my solution, my struggles, and do a pair programming to solve some problems I had.

And it's basically what I do most of the days to earn money.

Does not being able to solve an algorithm mean that I'm not a good developer?

I don't think so. In my mind, it just says I failed into solving a problem in certain conditions. But again, this is their metric of how to evaluate a professional.

My point here isn't to complain about why I was not hired but to make it clear for you that: failing in an interview does not mean you're a failure as a professional.

Learnings

Overall, doing all this process taught me a lot.

For the next interviews, I'll be prepared to easily answer questions about soft skills. I'll never forget the difference between inline and inline-block or what is the HTTP OPTIONS method and the difference between PUT and PATCH.

But my biggest learning was: try to be prepared for everything.

You might be requested to implement some APP using React, Vue, Angular, whatever, or even solving some tricky problems in JS.

For that, I already started spending few hours per day trying to solve problems in platforms like Codewars, FreeCodeCamp Learn and Codility.

While I'm still implementing new things on my personal website and get sharper in how to create a robust front-end application.

Conclusion

Hope my experience could give you some insight into what you could do differently to get your first job or at least to be prepared to rule a hiring process!

Yaml for Web Developers

Raul Melo — Sat, 18 Jul 2020 12:58:39 +0000

More and more, we, web developers need to learn about different areas to become better professionals and less dependent on others for simple things.

If you start a career as FE, you might see a lot of .yml files like .travis.yml (for Travis Build), .gitlab-ci.yml (for git lab CI), etc. but, let's be honest, what the hell?

Why would people use this kind of file? What's the benefit of it? How does this thing work?

So, the goal of this article is to introduce you to the YAML structure and give you more confidence to understand, read, and change a file like this when you need it.

After all, we tend to feel very uncomfortable and frustrated and we need to do something and we can't even understand what's that.

But first, What is YAML?

According to the official website, Yaml is:

"YAML (a recursive acronym for "YAML Ain't Markup Language") is a human-friendly data serialization standard for all programming languages."

Heavily used to write configuration files, which explains A LOT, right?

People were tired to have a bunch of configs nobody could understand until someone just say:

What if we could somehow write our configuration like a "cake recipe"? I mean minimum bare text, very straight forward?

Boom, in May 2001 Yaml was created.

YAML vs JSON

Surprisingly (or not really), Yaml is a superset of our well-known buddy JSON.

"Superset is A programming language that contains all the features of a given language and has been expanded or enhanced to include other features as well." - Font

If I could give you a perspective of what it means I would say:

In a FE world, Yaml would matches for TypeScript while JSON for JavaScript

To better understand how this would be even possible let's see this example:

{
  "compilerOptions": {
    "module": "system",
    "noImplicitAny": true,
    "removeComments": true,
    "preserveConstEnums": true,
    "outFile": "../../built/local/tsc.js",
    "sourceMap": false,
    "types": ["node", "lodash", "express"]
  },
  "include": ["src/**/*"],
  "exclude": ["node_modules", "**/*.spec.ts"]
}

This is a tsconfig.json example. very easy to read, we can easily identify what's what but... it has some limitations, like:

Can't create variables;
Can't use external variables (e.g. environment variables)
Override values;

In JS world, if we can create a .json a configuration file, we almost always also can create a .js (like .eslintrc or .eslint.js) which allows us to mitigate the CONS mentioned before.

But then, if you're using other programing language, JS files isn't an option. And it's at this point Yaml starts to shine.

If we'd have to re-write the tsconfig.json in YAML syntax and having the exactly result, it'd be like:

compilerOptions:
  module: system
  noImplicitAny: true
  removeComments: true
  preserveConstEnums: true
  outFile: '../../built/local/tsc.js'
  sourceMap: false
  types:
    - node
    - lodash
    - express
include:
  - src/**/*
exclude:
  - node_modules
  - '**/*.spec.ts'

Note this is only an example. You cannot write your tsconfig in YAML! 😜

I hope you're starting to get the idea from these files.

Concepts, Types, and Syntax

Now, let's dive deep a bit in the concept of the language.

Indentation

In Yaml, indentation does matter. It uses whitespace indentation to nest information. By whitespace, keep in mind tab is not allowed.

If you're like me and uses tab for everything, install some plugin in your IDE to replace your tabs for spaces (like editorconfig). Doing that, when you hit a tab, it'll automatically replace your tab by space and you don't even need to use your space bar! ;)

Root

Since indentation matters here, if there's no space before the first declaration YAML will understand that's the root (level 0) of your file:

person:
  age: 20

Like we have in JSON with the first { curly brackets:

{
  "person": {
    "age": 20
  }
}

Key/Value

Like in JSON/JS, YAML also uses the key/value syntax and you can use in various ways:

key: value
key_one: value one
key one: value # This works but it's weird
'my key': somekey

Comments

To write a comment you just have to use # followed by your message.

# I'm a comment
person: # I'm also a comment
  age: 20

This is cool to document some decision or make a note. Unfortunately, we can't do this with JSON.

Lists

There're 2 ways to write lists:

JSON way: array of strings

Remember Yaml is a JSON's superset? we can use its syntax:

people: ['Anne', 'John', 'Max']

Hyphen syntax

The most common (and probably recommended)

people:
  - Anne
  - John
  - Max

Strings

There're a few ways to declare a string in Yaml:

company: Google # Single words, no quotes
full_name: John Foo Bar Doe # Full sentence, no quotes
name: 'John' # Using single quotes
surname: "Christian Meyer" # Using double quotes

While in JSON we would have only a way to use double quotes:

{
  "company": "Google",
  "full_name": "John Foo Bar Doe",
  "name": "John",
  "surname": "Christian Meyer"
}

As a suggestion, prefer to use quotes when you want to use any special character like _, @, etc.

Numbers

Like in any programming language, we have 2 types of number: Integer and Float:

year: 2019 # Integer
nodeVersion: 10.8 # Float

Node Anchors (variables-ish)

An anchor is a mechanism to create a group of data (an object) that can be injected or extended from other objects.

Let's imagine you need to create a configuration for your CI. It'll have both production and staging environments. As you can imagine, they share almost the same base settings.

In JSON world, we would have to duplicate these configs:

{
  "production": {
    "node_version": "13.0.0",
    "os": "ubuntu",
    "package_manager": "yarn",
    "run": ["yarn install", "NODE_ENV=${ENVIRONMENT} yarn build"],
    "env": {
      "ENVIRONMENT": "production"
    }
  },
  "staging": {
    "node_version": "13.0.0",
    "os": "ubuntu",
    "package_manager": "yarn",
    "run": ["yarn install", "NODE_ENV=${ENVIRONMENT} yarn build"],
    "env": {
      "ENVIRONMENT": "staging"
    }
  }
}

Copy and paste are also annoying, especially when you have to change something in all places it's been used those Infos.

Anchors came to solve that problem. We can:

First, create our anchor

# I name it as "base-config" but it can be whatever
# &base will be the "variable name" you'll use in the injection
base-config: &base
  node_version: 13.0.0
  os: ubuntu
  package_manager: yarn
  run:
    - yarn install
    - NODE_ENV=${ENVIRONMENT} yarn build

Then, injecting the anchor created in the level we want to have to see these values being injected:

base-config: &base
  node_version: 13.0.0
  os: ubuntu
  package_manager: yarn
  run:
    - yarn install
    - NODE_ENV=${ENVIRONMENT} yarn build

production:
  # I'm injecting all "base" attributes and values inside production
  <<: *base
  env:
    - ENVIRONMENT: production

staging:
  # I'm injecting all "base" attributes and values inside staging
  <<: *base
  env:
    - ENVIRONMENT: staging

Looks simpler, right? And also easier to maintain.

If you copy this code and paste into a "Yaml to JSON converter" online tool you'll see the same code as I mentioned early in the JSON example but within the addition of the base config:

{
  "base-config": {
    "node_version": "13.0.0",
    "os": "ubuntu",
    "package_manager": "yarn",
    "run": ["yarn install", "NODE_ENV=${ENVIRONMENT} yarn build"]
  },
  "production": {
    "node_version": "13.0.0",
    "os": "ubuntu",
    "package_manager": "yarn",
    "run": ["yarn install", "NODE_ENV=${ENVIRONMENT} yarn build"],
    "env": [
      {
        "ENVIRONMENT": "production"
      }
    ]
  },
  "staging": {
    "node_version": "13.0.0",
    "os": "ubuntu",
    "package_manager": "yarn",
    "run": ["yarn install", "NODE_ENV=${ENVIRONMENT} yarn build"],
    "env": [
      {
        "ENVIRONMENT": "staging"
      }
    ]
  }
}

JSON syntax (yes, JSON)

As explained before a superset of a language is the base language PLUS some extra features, which means we could write a Yaml file in JSON way

{
  "details": {
    "company": {
      "name": "Google",
      "year": 2019,
      "active": true
    },
    "employees": [
      "Anne",
      "John",
      "Max"
    ]
  }
}

Doubting? Copy this code and paste it here

If you convert this YAML to JSON, you'll have the same structure:

{
  "details": {
    "company": {
      "name": "Google",
      "year": 2019,
      "active": true
    },
    "employees": ["Anne", "John", "Max"]
  }
}

Shell/Bash environment

As I told at the beginning of this article, it's very common .yml files are used as config files for many things, but especially for CI/CD environment.

For those, you'll have to describe how the machine/docker should work, what should be installed, ran, etc.

Commonly, all those environments are Linux, which means you'll also have access to the environment itself.

On GitLab CI, for instance, you can specify on a global level environment variables you want to have available for the whole process:

variables:
  NODE_IMAGE: node:10

stages:
  - build

test:
  image: $NODE_IMAGE
  stage: build

Note that the syntax to use variables by $ isn't from YAML but shell/bash.

What GitLab CI does is getting everything you'd defined in variables and creates shell variables.

Some other platforms also inject other values like commit ref, branch name, build time, author and also secret keys defined outside the configuration:

variables:
  NODE_IMAGE: node:10

stages:
  - build

test:
  image: $NODE_IMAGE
  stage: build
  artifacts:
    name: $CI_COMMIT_REF_NAME

In the example above, we're using a $CI_COMMIT_REF_NAME external environment variable that GitLab CI platform makes available which describes The branch or tag name for which the project is built.

Conclusion

I hope you now understand a bit more about YAML and at least feel comfortable reading and write your files.

Keep in mind that what you'll have access to or not, the limitations will be determined by the platform you're using. Travis defines a different configuration than GitLab CI or CircleCI for example.

Always check the documentation from the platform you're working on to see what's possible or not to be done! :)

References

YAML Website. (You can find parses for all languages there);
Learn X in Y Minutes: YAML: Here a whole guide/introduction about everything YAML can do for you;
JSON ←→ YAML Online converter: Useful to visualize what generates what and create a better understanding;
YAML by Wikipedia