DEV Community: Ramo Mujagic

How to setup Deep Links for Android applications

Ramo Mujagic — Sun, 09 Jun 2024 12:24:12 +0000

Deep links are special URIs that take users directly to specific content within a mobile app, rather than just launching the app or opening a webpage in a browser.

These links enhance user experience by letting users navigate to precise content or a particular section of the app with just one click.

Types of links

On Android, there are clear distinctions between different types of links: Deep Links, Web Links, and Android App Links. To implement deep link handling on Android, it's essential to understand how each type behaves. The following illustration shows the relationships between these types of links.

In the image, you can see that Web Links and Android App Links are special cases of deep links. Let's explore them a bit further.

Deep Links

Deep links can take various forms depending on the URI scheme and the type of content they point to within the app. Here is an overview of possible schemes available with deep links:

1️⃣ HTTP/HTTPS Scheme

These are URLs that can open web pages in a browser or specific content within the app if the app is set up to handle them. Here is an example:

https://shopapp.com/shop
https://shopapp.com/profile

2️⃣ Custom Scheme

These are custom URI schemes defined by the app. They don't open in a web browser and are designed to be handled only by the app. Here is an example:

shopapp://shop
shopapp://profile

Notice that URI starts with shopap, this is called custom scheme. If there is an app on your Android device configured to handle this custom scheme, Android will delegate handling of the link to the capable app.

In simple terms, deep links are URIs that navigate users directly to specific content within a mobile app and can use both HTTP/HTTPS and custom URI schemes.

Web Links

Web links are deep links that use the HTTP/HTTPS schemes. Starting with Android 12, clicking a Web Link that is not an Android App Link always shows content in a web browser.

On devices running previous versions of Android, if the app or other apps installed on a user's device can also handle the Web Link, users might not go directly to the browser. Instead, they'll see a system dialog letting them choose which app to use to open the link.

Android App Links

Android App Links are a special type of URL designed for Android that looks similar to a regular web link (using HTTP/HTTPS). They allow users to be directed to specific content within an app, instead of a web page. Here are some examples:

https://www.shopapp.com/product/67890
https://www.shopapp.com/profile/12345

When the user clicks on an Android App Link, the app opens immediately if it's installed. The Android system will not show any modals to the user; it just works. If the user doesn't want your app to be the default handler, they can override this behavior in the app's settings.

App Links are required to support deep linking behavior using HTTP/HTTPS scheme on Android 12 and onwards, ensuring a seamless user experience.

Implicit Intent

Before configuring deep links, it's important to understand how the Android system handles such links. This is done using something called an Intent.

In the world of Android apps, intents act as messengers. These messages coordinate actions between different parts of an app, or even between different apps entirely. There are two main types of intents: explicit intents and implicit intents. Today, we'll focus on implicit intents because they play a key role in deep linking and how the Android system directs users to specific app content.

Implicit intents specify a general action to be performed. They do not specify which application to use for performing the action. The system decides which installed application is best suited to handle the implicit intent. To achieve this, the system checks something called intent filters.

How Deep Links use Implicit Intents

Deep links rely on intent filters specified in the app's AndroidManifest.xml to handle particular URL schemes or host/path combinations.

When a deep link is triggered (e.g., a user clicks a link), an implicit intent is created with the action Intent.ACTION_VIEW and a data URI specifying the URL to be handled.

Next, the Android system matches this intent against the intent filters declared by installed applications to find the appropriate activity to handle the link.

When setting up deep links, our goal is to ensure that the system has all the necessary data required to start our application when the designated deep link gets triggered.

Understanding intent filters

When you define an intent filter in your AndroidManifest.xml to handle deep links, the system uses the specified <data> attributes to match incoming intents to the correct activity. The <data> element can specify the scheme, host, port, path, pathPrefix, and pathPattern. If these attributes do not match the incoming URL, the intent filter will not match, and the activity will not be triggered (e.g., the application will not start).

Intent filters are crucial in handling deep links, allowing you to fine-tune how your application responds to specific links. We will discuss this in more detail in the next sections.

Deep Link flow

Deep links can seem complex at first, but the following flowchart will break it down into simpler steps. Let's see how deep links navigate users to specific app content.

The flowchart illustrates how the Android system resolves deep links, which can be in the form of a custom domain or an App Link.

The process begins with the user triggering the deep link. This can happen by pressing a button or clicking a link, for example in an email or messaging app. Upon this action, the Android system creates an implicit intent using the URL associated with the button or link.

The system checks the intent filters declared in AndroidManifest.xml for every installed app to find a matching filter that can handle the URL.

The system then checks if any installed apps have registered an intent filter that can handle the URL from the deep link and analyzes the deep link further to determine its specific type.

If the deep link uses a custom URI scheme but no matching app is found, the link isn't handled by the Android system. In all other cases, the link opens in a web browser.

If the system finds a matching intent filter, it takes further action based on the deep link type. For custom URI schemes, the link is directly opened within the corresponding app. However, for App Links, an additional step verifies domain ownership to ensure security. We'll explore this verification process in detail later. For now, let's focus on what happens after successful verification of an App Link.

Upon successful domain ownership verification, the link opens directly within the app. However, if verification fails, the link will be opened in a web browser for user safety.

I hope this gives you a clearer picture of how deep links are processed and handled by the system. If it's still not clear, consider reading this section once more.

Now, let's dive deeper and explore how to configure App Links to leverage this functionality within your own app!

Using App Links

As mentioned earlier, App Links are a special type of URL designed for Android that looks similar to a regular web link. Since App Links use the HTTP/HTTPS scheme, we need to configure them for designated web URLs.

Setting up App Links involves a few key steps to ensure that your application can handle URLs and open the app directly when those URLs are clicked.

Before we start, let's say we want to configure the app so that when the following link is clicked: https://shopapp.com/shop, it opens the ShopActivity inside the app itself.

There are a couple of important pieces of information that we can extract from the link:

Host: shopapp.com
Scheme: https
Pathname: /shop

We will later use this information to construct the required intent filter needed to open the app when the link is triggered. So, let's get started.

Update Manifest file

First, we need to declare intent filters in the AndroidManifest.xml for the activity that should handle our link.

<activity android:name=".ShopActivity">
  <intent-filter android:autoVerify="true">
    <action android:name="android.intent.action.VIEW" />
    <category android:name="android.intent.category.DEFAULT" />
    <category android:name="android.intent.category.BROWSABLE" />

    <!-- Define the scheme, host and path -->
    <data
    android:scheme="https"
    android:host="shopapp.com"
    android:pathPrefix="/shop"
    />
  </intent-filter>
</activity>

So, what just happened? Let's break it down.

One very important attribute is android:autoVerify="true", and links will not work without it! According to Google, this attribute allows the app to designate itself as the default handler of a given type of link. So, when the user clicks on an Android App Link, your app opens immediately if it's installed.

The intent filter that we just created will only match https://shopapp.com/shop, but what if we want to open the app when https://shopapp.com link is clicked? Well, we can modify the intent filter and remove the android:pathPrefix="/shop" attribute, like this:

<intent-filter android:autoVerify="true">
  <action android:name="android.intent.action.VIEW" />
  <category android:name="android.intent.category.DEFAULT" />
  <category android:name="android.intent.category.BROWSABLE" />

  <!-- Define the scheme and host -->
  <data
    android:scheme="https"
    android:host="shopapp.com"
  />
</intent-filter>

This will match any URL with the scheme https and host shopapp.com, regardless of the path. So, URLs like https://shopapp.com, https://shopapp.com/shop, and https://shopapp.com/profile/1117 would all open the application.

You might wonder, when this is the case, why even use android:pathPrefix attribute? There are a couple of reason to consider, but most important is Selective Handling. This basically means that you can ensure that only specific paths within your domain are handled by your application.

If your website has different parts like /shop, /categories and /profile, and you only want the app to handle links under /shop, you would use android:pathPrefix="/shop" to ensure that the app only opens links that have /shop as a path.

Create the Digital Asset Links File

Great, now we have intent filter that say to the system: “Hey, when you see this link https://shopapp.com/shop being clicked somewhere please open our app”.

But, this is still not enough. The system will not trust our app without proof that we own the link domain. Why, you might wonder?

Well, we could have added an intent filter for https://amazon.com/, but we obviously do not own this domain, and it would be really weird, and a major safety hazard, if the system would use our app to handle the link that points to the amazon.com domain.

So how can we prove to the system that we actually own the domain of the link? This is where the Digital Asset Links File comes into play. This file is used to prove the ownership of the domain (e.g., the association between your website and your app).

This file needs to be named assetlinks.json, and as the file extension implies, it needs to be in JSON format. This is how the file might look:

[
  {
    "relation": ["delegate_permission/common.handle_all_urls"],
    "target": {
      "namespace": "android_app",
      "package_name": "com.shopapp.app",
      "sha256_cert_fingerprints": [
  "XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX:XX"
      ]
    }
  }
]

You can see that the file is an array, and the configuration for our application is contained in an object. This means if you own multiple apps that can open the links from your website, you need to list the configuration for both (or more) apps in this file.

For Android apps, you will have to modify only the package_name and sha256_cert_fingerprints attributes. The package_name is pretty self-explanatory; it is the package name of your app.

In the sha256_cert_fingerprints attribute, you need to put the SHA-256 fingerprint of your app's signing certificate. There can be multiple fingerprints added (e.g., debug and production).

⚠️ Warning
When adding certificate, it is important to use capital letters for HEX values! Not using capital letters might prevent the system from opening your app (errorCode: ERROR_CODE_MALFORMED_CONTENT).

If you do not know your SHA-256 fingerprint, you can use the following keytool command to get it:

keytool -list -v -keystore <path-to-keystore> -alias <key-alias> -storepass <keystore-password> -keypass <key-password>

Make sure to replace the following:

<path-to-keystore> - Path to the keystore file used to sign the app.
<key-alias> - Alias name given to the keystore when it was created.
<keystore-password> - Keystore password given to the keystore when it was created.
<key-password> - Key password given when the keystore was created.

It is important to add the correct SHA-256 fingerprint to the sha256_cert_fingerprints attribute or the link will not work!

If you are debugging the app using the default Android keystore, you can get the fingerprint for it using the following command and add it to the sha256_cert_fingerprints as well.

keytool -list -v -keystore ~/.android/debug.keystore -alias androiddebugkey -storepass android -keypass android

With this done, your assetlinks.json should be ready, and we can move to the next step.

Upload Digital Asset Links File to website host

We've created the assetlinks.json file, but it won't work its magic just sitting on your computer. Let's explore what the Android system needs from us to leverage this file for App Link verification.

When the Android system tries to verify the https://shopapp.com/shop link, it will make a GET request to a specific location on our website. In our example, the request URL will look like this:

https://shopapp.com/.well-known/assetlinks.json

Notice the .well-known/assetlinks.json path in the link. The Android system will try to get our Digital Asset Links File from that location. This means we need to upload/host our previously created assetlinks.json in that specific location.

⚠️ Warning
The assetlinks.json file must be hosted in the .well-known directory at the root of the domain to work correctly.
That means, URL like https://shopapp.com/shop/.well-known/assetlinks.json will not work because /shop path is added.

Here's what the Android system expects for your assetlinks.json file to function correctly:

Directly Accessible: The file must be reachable without any redirects.
Open to Bots: The file needs to be accessible by automated programs (bots).
Correct Content Type: The file's content type should be identified as application/json.
Secure Connection: The file must be served over a secure HTTPS connection for added security.

In most cases, simply uploading your assetlinks.json file as a static file to https://<your_domain_name>/.well-known/assetlinks.json will be good enough to satisfy all the requirements.

Verify that Digital Asset Links File is correct

Let's assume we have our Digital Assets Link File hosted on https://shopapp.com/shop/.well-known/assetlinks.json and ready to be used. How can we know that the configuration is correct and the Android system will be able to use it? Luckily, for this reason, Google provides a Digital Asset Links API to verify the accuracy.

https://digitalassetlinks.googleapis.com/v1/statements:list?source.web.site=https://<domain_name>&relation=delegate_permission/common.handle_all_urls

You can execute this request straight in you browser, make sure to replace <domain_name> with the correct domain name of your website.

If the API encounters any problems processing your assets file, it will return an error code along with additional details about the issue. This information can help you diagnose and fix problems within your asset file.

Test App Links on Device

The best way to know if deep links are working is, obviously, to test them on the device. To test deep links, you can either use a real device or an Android emulator.

Before any testing can happen, let's follow some setup phases:

Make sure that AndroidManifest.xml is properly configured.
Use the correct signing certificate (keystore), the one you added to the assetlinks.json file.
Create new app build.
Install the app on device or emulator.

With the setup out of the way, we can start testing deep links. The first thing we can do is to artificially create test deep links using the adb command like this:

adb shell am start -W -a android.intent.action.VIEW \
  -d "https://shopapp.com/shop"

This command simulates a deep link click on your connected device or emulator. It essentially tells the system to launch an activity capable of handling the provided URL, mimicking how the system would normally create an implicit intent for a deep link. Think of it as manually crafting an intent to test your app's deep link response. Remember to replace https://shopapp.com/shop with your actual deep link.

If our setup was correct, the System will open the link in our app rather than opening it in the browser.

For a more hands-on test, you can leverage the deep link itself! Send yourself an email or message containing the link, or generate a QR code from the link and scan it using your device's camera app. If the app is installed on your device, clicking the link or scanning the QR code should automatically launch the app and navigate to the intended content.

If you have some issues, make sure you followed the guide correctly and that nothing is skipped over.

Using Deep Links with Custom scheme

Deep links with a custom scheme are another way to implement deep linking functionality. While deep links via the HTTP/HTTPS scheme are more commonly used, custom schemes can also come in handy in some cases.

So how to set it up? Luckily, it can be done in just one step compare to App Links that involved quite a lot things to go trough.

Update manifest file

As with App Links, we need to define an intent filter in the AndroidManifest.xml file to specify the custom URI scheme that the app will handle.

<activity android:name=".ShopActivity">
    <intent-filter>
        <action android:name="android.intent.action.VIEW" />
        <category android:name="android.intent.category.DEFAULT" />
        <category android:name="android.intent.category.BROWSABLE" />

        <!-- Define custom scheme -->
        <data android:scheme="shopapp" />
    </intent-filter>
</activity>

The intent filter for a custom scheme deep link resembles the one for App Links, but with a key difference. The crucial attribute here is android:scheme, which specifies the custom scheme your app will respond to. In this example, the shopapp scheme is defined.

It goes without saying, but we also need to add code to the Activity to handle the deep link intent. However, as this is not part of our topic, we will skip it.

Believe it or not, with this setup, the configuration is actually completed. Our app should open when someone triggers a link like this one:

shopapp://shop

The Android system will create an intent, check intent filters, and open the app immediately without any validation or modals in between—it just works.

Test link on Device

To test your custom scheme, similar to App Links, we can use the adb command.

adb shell am start -a android.intent.action.VIEW \
  -d "shopapp://shop"

The command triggers an activity on the connected Android device or emulator to handle the specified URL, testing how the app responds to the deep link. If the app is installed and set up correctly, the link should open in the app.

Another way to test it us by creating a simple HTML file with a link that uses our custom scheme.

<!DOCTYPE html>
<html>
<head>
  <title>Test Custom Scheme</title>
</head>
<body>
  <a href="shopapp://shop">Open App</a>
</body>
</html>

You can use a simple static HTTP server to serve the file. Open the file on the device and click on the "Open App" button. When the link is triggered, the system should open the link in the app.

Wrapping up

Deep links are a powerful tool in today's mobile development landscape. Integrating deep links into your app can significantly improve user interaction by allowing users to access specific features or content directly from external sources.

We explored key deep link concepts and provided instructions for setting up deep links in Android applications. By understanding these concepts and implementing the discussed techniques, developers can leverage deep links to enhance user navigation, streamline user journeys, and ultimately create a more engaging app experience. Whether through custom URI schemes or verified web links (App Links), deep linking offers a robust solution for guiding users to specific content and improving the overall app experience.

Check out other articles on wannabedev.io.

Visualize Bubble Sort Algorithm using SVG

Ramo Mujagic — Mon, 09 Oct 2023 18:01:27 +0000

I was playing around with some SVG (Scalable Vector Graphics) for some time and wanted to make a quick article about it. The problem was that I had no idea what to create using SVG. Fortunately, I stumbled upon one thread about sorting algorithms and thought it would be interesting to animate how sorting algorithms work.

Of course, I decided to go with the simples of the bunch, Bubble Sort, just to give it a try. To cut to the chase, you can check it out on the demo page.

How Bubble Sort works

Most of you already know about sorting algorithms, but let's just give a quick introduction to Bubble Sort so we understand how it works.

If you know how the Bubble Sort algorithm works, feel free to skip to the next section.

Bubble Sort is a straightforward sorting algorithm that involves repeatedly comparing and swapping adjacent elements in a list until the list is sorted.

Let's say we have following list of numbers:

[5, 6, 9, 2, 3]

The Bubble Sort algorithm will loop over this list a total of n - 1 times, where n is the number of elements in the list. This means, for our example, Bubble Sort will loop over the list four times in total.

Let's see what happens in the first pass through the list:

It begins by comparing the first and second elements, starting with the first index (numbers 5 and 6). No swap is needed.
Next, compare 6 and 9. No swap is needed.
Next, compare 9 and 2. Swap the numbers since they are not ordered.
Continue this process until the biggest numbers ”bubble” up to the top.

At the end of the first pass, we will have the following list of numbers:

[5, 6, 2, 3, 9]

For the second pass, exactly the same thing will happen, but in this case we do not have to compare with the last number in the list since we already know that it's the biggest number. At the end of the second pass, we will have the following list of numbers:

[5, 2, 3, 6, 9]

We are almost there; when the third pass starts, we do not have to check for the last two numbers in the list since we know that the list does not have bigger numbers. At the end of the third pass, we will have the following list of numbers:

[2, 3, 5, 6, 9]

Hmmm, it looks like the list is already sorted, and we did not even start the fourth pass. Well, that can happen, but it does not really matter.

There is nothing to prevent the algorithm from starting the final, forth, pass. At least it will be a quick one since we know that the last three numbers are already sorted. And as we expected, at the end of the fourth pass, the list stays the same.

Prepare wireframe

So, there is a simple question: how can we visualize the Bubble Sort algorithm? Let's start with the basics and define what we want to make. Take a look at the following image.

In the image, you can see the basic UI of our Bubble Sort visualization player. We will have two inputs that allow us to tweak player settings a little bit.

Number of items - changes the number of items in the list.
Animation speed - changes the time it takes for animation to finish a single step.

Underneath, we have three buttons that control the player.

“Play” button - starts the visualization.
“Pause” button - pauses the currently running visualization.
“Stop” button - stops the currently running visualization and reset the player.

Finally, under the buttons, we have a player section where the visualization is going to be shown. Initially, the text “Press Play button to start” is shown. As soon as the “Play” button is pressed, we have to remove the text and start the player.

To make the article shorter, only important parts of the code will be shown. You can always find complete code on Github.

The wireframe is complete, and it helps us visualize what the end result should look like, but now it's time to bring it to life.

List and elements

When the “Play” button is clicked, we have to generate the list (array) that needs to be sorted, as well as the elements that have to be animated, and insert them into the player container.

Let's start by creating a random list of numbers that the Bubble Sort algorithm has to sort. A function to quickly generate the list can be created like this:

function createData(length = 15, maxNumber = 200) {
  return Array.from(
    { length },
    () => Math.round(Math.random() * maxNumber),
  );
}

The function is quite simple; it generates a list that has a certain length of elements. To limit the maximum number that can be generated, maxNumber is used.

This generated list of numbers will also be represented using SVG elements. Every list number will have its own corresponding SVG element. By animating these elements, we can visualize how the list itself is being sorted.

You can see different SVG elements used to represent one single number from the list. The exact number value is shown using the text element. This value is then visualized as a bar represented by the rect element, and finally, the g element is used to group both text and rect elements together so they can be easily animated to the correct positions during sorting.

Before we can focus on creating SVG elements, there are a couple of variables that we need to take care of. Consider the following image:

To properly layout elements, we have to define certain variables that describe how the elements are related to each other.

On the image, you can see that we need to define four different variables as follows:

Bar with - width of the single bar.
Bar spacing - spacing between neighboring bars.
Text offset - space between the bar and correlated text.
Maximal bar height - maximum height that a single bar can reach.

Using these variables, we can make sure that our SVG representation of the list looks consistent. This makes things easier when we need to apply animations.

Since SVG is basically a 2D (two-dimensional) graphic, we have to use the x and y axes to position our elements.

On the image, you can see how we can position our SVG elements inside the 2D coordinate system. You might notice that the 2D coordinate system on the image looks upside-down. This is because of the scroll behavior in the browser. As you scroll down, the y value is increasing; the more you scroll, the bigger the value of y.

To make sure that our elements are initially positioned at the same level on the y axis, we want to put them exactly where the maximum bar height is. From there, we can change the height property of the rect element to visualize the value.

To calculate the value for the x axis, we have to take into account the bar width, bar spacing, and position of the element in the list.

function createDOMElements(dataList, settings) {
  const { barWidth, barSpacing, textOffset, maxBarHeight } = settings;
  const svgContainer = document.createElementNS('http://www.w3.org/2000/svg', 'svg');

  /**
   * Create SVG group for each element in the array.
   * SVG group consists of the vertical bar (rect) and text.
   */
  for (let i = 0; i < dataList.length; i++) {
    const barHeight = dataList[i];
    const coordinateX = i * (barWidth + barSpacing);
    const coordinateY = maxBarHeight - barHeight;

    const svgRect = document.createElementNS('http://www.w3.org/2000/svg', 'rect');
    svgRect.setAttribute('y', coordinateY);
    svgRect.setAttribute('rx', '0.3rem');
    svgRect.setAttribute('width', barWidth);
    svgRect.setAttribute('height', barHeight);

    const svgText = document.createElementNS('http://www.w3.org/2000/svg', 'text');
    svgText.setAttribute('x', barWidth/2);
    svgText.setAttribute('y', maxBarHeight + textOffset);
    svgText.textContent = `${barHeight}`;

    const svgGroup = document.createElementNS('http://www.w3.org/2000/svg', 'g');
    svgGroup.setAttribute('class', 'bar')
    svgGroup.setAttribute('data-index', i)
    svgGroup.setAttribute('transform', `translate(${coordinateX} 0)`);
    svgGroup.appendChild(svgRect)
    svgGroup.appendChild(svgText)
    svgContainer.appendChild(svgGroup);
  }

  return svgContainer;
}

First, we create a svg element using the document.createElementNS API and save it into svgContainer. We will need it at the end of the function again.

After that, we loop over the dataList, which contains numbers that we want to visualize. For each item in the list, we have to do a couple of things:

Calculate the x coordinate value using the formula: item index * (bar width + bar spacing), and save the result into coordinateX. We have to use the item index, i, to make sure that the elements are not stacked on top of each other.
Calculate the y coordinate value using the formula: maximal bar height - item value, and save the result into coordinateY.
Create a rect element and attach the required attributes.
Create a text element, attach the required attributes, and set the textContent property. The x attribute is set to half of bar width to keep the text in the center of the bar. And the y attribute is set to maximal bar height + text offset to keep it under the bar.
Create an g element and attach the required attributes. We use the data-index attribute to link the g element with the corresponding item from the list. Also, the transform attribute is initialized, as it will be used to update the element's position later on.
The rect and text elements are then appended to the g element to keep them grouped with each other.
Finally, the grouped elements are then appended to the svgContainer created at the beginning.

Function will return the svgContainer itself, which can be inserted in the player container when the play button is pressed. With preparation done, we are ready to continue with the animation part.

Bubble Sort animation driver

To animate the Bubble Sort algorithm, we cannot just write a function that will do the sorting. We need to be able to control the algorithm and execute side effects at each iteration. To do this, we need to make some kind of “driver”.

For a driver to properly work, it needs to manage its internal state. Here are the things we need to take care of:

Player element - HTML element where the animation player is located.
Timer id - ID of the currently active timer.
List - a list of numbers that is currently being sorted.
Pause - a flag to indicate that the animation is paused.
Loop index - index of the running loop.
Remaining repetitions - the total number of remaining repetitions.

As we already know from the wireframe, we have three buttons: play, pause, and stop. This means that the driver needs to be able to properly respond to these buttons. For that, we can create corresponding functions.

function driver() {
  // Default settings
  let settings = {
    maxBarHeight: 200,
    barWidth: 30,
    barSpacing: 5,
    textOffset: 20,
    animationSpeed: 300,
    listLength: 10,
  }

  // Initial player state
  let playerState = {
    playerElement: document.getElementById('player-wrapper'),
    timerId: null,
  };

  // ...

  return {
    play(options = {}) {
      if (playerState.pause) {
        playerState.pause = false;
      } else {
        settings = { ...settings, ...options };

        // Extend player state before the next animation is started
        playerState = {
          ...playerState,
          list: createData(settings.listLength, settings.maxBarHeight),
          pause: false,
          loopIndex: 0,
          remainingRepetitions: settings.listLength - 1,
        }
      }

      bubbleSort();
    },
    stop() {
      // Clear scheduled timeout
      clearTimeout(playerState.timerId);
      playerState.pause = false;

      // Reset player container
      playerState.playerElement.classList.remove('scroll-x');
      playerState.playerElement.innerHTML = '<p>Press "Play" button to start</p>';
    },
    pause() {
      playerState.pause = true;
    }
  }
}

The driver function returns an object that provides methods for controlling the sorting animation. We can call the specific method when the corresponding button in the UI is clicked.

You might have noticed that in the play method, there is a call to the bubbleSort function. This function implements the Bubble Sort algorithm and controls the sorting animation.

function bubbleSort() {
  const { loopIndex, remainingRepetitions } = playerState;
  const svgContainer = getPlayerSvgContainer();
  adjustHorizontalScroll(svgContainer);

  // ...
}

We first have to create an SVG container element where the sorting visualization is displayed by calling the getPlayerSvgContainer function. This container will hold the bars that represent the data being sorted.

function getPlayerSvgContainer() {
  let svgContainer = playerState.playerElement.querySelector('svg');
  if (svgContainer === null) {
    svgContainer = createDOMElements(playerState.list, settings);
    playerState.playerElement.innerHTML = '';
    playerState.playerElement.append(svgContainer);
  }
  return svgContainer;
}

This function will return the existing svg element if it is found, or create a new one by calling the createDOMElements function.

Next, we have to adjust the horizontal scroll bar in case the svgContainer width is bigger than the wrapping player container. We can do that by calling the adjustHorizontalScroll function.

function adjustHorizontalScroll(svgContainer) {
  // Check content width
  const playerWrapperWidth = playerState.playerElement.clientWidth;
  const svgContainerWidth = svgContainer.clientWidth;

  // Allow horizontal scroll if player wrapper can not show all elements
  if (playerWrapperWidth > svgContainerWidth) {
    playerState.playerElement.classList.remove('scroll-x')
  } else {
    playerState.playerElement.classList.add('scroll-x')
  }
}

So far, so good. Let's continue with our bubbleSort function.

function bubbleSort() {
  // ...

  // Get SVG elements for sorting (SVG g elements)
  const elementsToSort = Array.from(svgContainer.children);

  // Check if list is sorted
  if (loopIndex >= remainingRepetitions) {
    elementsToSort.forEach(element => element.classList.add('swapped'));
    return;
  }

  const [leftElement, rightElement] = getBubbleSortElements(elementsToSort);

  // ...
}

We need to get SVG elements (representing the bars and their labels) from the SVG container. To do so, we can use the Array.from method by passing the svgContainer.children property.

Before we continue, we need to add an exit condition that is only true when the sorting process is complete. We can do that by checking if the current loopIndex is the same or bigger than the remainingRepetitions. If this is the case, we can mark all SVG elements as sorted by providing 'swapped' CSS class. If the exit condition is not satisfied, we continue with sorting.

To know if elements need to swap positions, we have to get two neighboring elements based on the loopIndex and compare them.

To get the needed elements, leftElement and rightElement, we can call the getBubbleSortElements function. This function uses the data-index attribute to match the corresponding element with the item from the list.

function getBubbleSortElements(svgElements) {
  const { loopIndex } = playerState;

  const leftElement = svgElements.find(
    element => Number(element.getAttribute('data-index')) === loopIndex
  );
  const rightElement = svgElements.find(
    element => Number(element.getAttribute('data-index')) === loopIndex + 1
  );
  return [leftElement, rightElement];
}

To visually indicate which elements are being compared in the current iteration, we do so by adding the CSS class 'current' to them.

function bubbleSort() {
  // ...

  // Mark elements as current
  leftElement.classList.add('current');
  rightElement.classList.add('current');

  // Player is paused, skip sorting for the moment
  if (playerState.pause) {
    return;
  }

  const didSwap = performSwapIfNeeded(leftElement, rightElement);

  // ...
}

When the elements are marked, it is a good time to handle the eventual pause event; in other words, if the pause button is clicked. We can do so by checking playerState.pause and exiting the function if the pause property is set.

When the play button is pressed again, the pause property will be set to false, and the bubbleSort function will be called again to resume sorting.

Next, we can finally swap elements if they are out of position. To do so, we call the performSwapIfNeeded function.

function performSwapIfNeeded(leftElement, rightElement) {
  const { list, loopIndex } = playerState;

  if (list[loopIndex] <= list[loopIndex + 1]) {
    return false;
  }

  [list[loopIndex], list[loopIndex + 1]] = [list[loopIndex + 1], list[loopIndex]];

  leftElement.setAttribute('transform', `translate(${(loopIndex + 1) * (settings.barWidth + settings.barSpacing)} 0)`);
  rightElement.setAttribute('transform', `translate(${loopIndex * (settings.barWidth + settings.barSpacing)} 0)`);

  leftElement.setAttribute('data-index', loopIndex + 1);
  rightElement.setAttribute('data-index', loopIndex);

  return true;
}

We pass leftElement and rightElement as arguments to the function. In the function, we can take the list and loopIndex directly from the playerState.

If the item at loopIndex is smaller than the item at loopIndex + 1, the function returns false, indicating that no swap occurred.

If a swap is needed, we have to update the list by swapping the positions of the two items in the list. By updating the transform attribute of leftElement and rightElement, we can adjust horizontal positions to visually reflect the updated order after the swap.

We also have to update the data-index attribute to match the corresponding elements with the items from the list. Finally, the function returns true to indicate that a swap occurred.

We now have to visually update affected elements and prepare the player state for the next invocation of the bubbleSort function.

Continuing, we have to schedule the timer for the next invocation of the bubbleSort function, update the player state, and set the correct CSS classes on the affected elements from the previous invocation.

// Update loop index
function bubbleSort() {
  // ...

  const didSwap = performSwapIfNeeded(leftElement, rightElement);
  playerState.timerId = setTimeout(() => {
    playerState.loopIndex++;

    if (playerState.loopIndex >= playerState.remainingRepetitions) {
      // Current repetition has finished
      if (didSwap) {
        leftElement.setAttribute('class', 'bar swapped');
        rightElement.classList.remove('current');
      } else {
        leftElement.classList.remove('current');
        rightElement.setAttribute('class', 'bar swapped');
      }

      playerState.remainingRepetitions--;
      playerState.loopIndex = 0;
    } else {
      // Current repetition still in progress
      leftElement.classList.remove('current');
      rightElement.classList.remove('current');
    }

    bubbleSort();
  }, settings.animationSpeed);
}

To schedule the next step of the sorting animation we can use setTimeout API. The timeout time will be set to the value of settings.animationSpeed property.

Let's focus on callback function that is triggered by setTimeout. First thing we have to do is to increment the loopIndex to move to the next step in the sorting process.

We can now check if the current repetition has finished and updates the visual representation of the elements.

Depending on the didSwap flag we can update leftElement and rightElement by removing or setting 'current' and 'swapped' CSS classes.
Lastly we can decrement the remainingRepetitions counter and resets loopIndex to 0.

If the current repetition is still in progress, we just have to remove the 'current' class from both leftElement and rightElement.

When the player state and visual elements are properly updated, we can call bubbleSort recursively to continue the animation with the next comparison.

Use the animation driver

To use the driver, we just need to call driver function which will return interface/object that contains play, pause and stop methods.

// Get DOM elements
const playButtonElement = document.getElementById('play-button');
const pauseButtonElement = document.getElementById('pause-button');
const stopButtonElement = document.getElementById('stop-button');
const listLengthInputElement = document.getElementById('list-length');
const animationSpeedInputElement = document.getElementById('animation-speed');

// Initialize driver to run Bubble Sort visualization
const bubbleSortDriver = driver();

// Call driver.play when play button is clicked
playButtonElement.addEventListener('click', (e) => {
  e.currentTarget.disabled = true;
  driver.play({
    listLength: parseInt(listLengthInputElement.value),
    animationSpeed: parseInt(animationSpeedInputElement.value)
  });
});

// Call driver.stop when stop is clicked
stopButtonElement.addEventListener('click', (e) => {
  e.currentTarget.disabled = true;
  driver.stop();
});

// Call driver.pause when pause is clicked
pauseButtonElement.addEventListener('click', (e) => {
  e.currentTarget.disabled = true;
  driver.pause();
});

We just need to make sure to call correct driver method depending the button that was clicked. Driver encapsulates all logic related to managing Bubble Sort visualization under the hood and exposes simple API that can be easily used.

Conclusion

In this article I tried to give a concrete example on how to use SVG elements to actually visualize data and algorithm in action. We used SVG elements to visualize Bubble Sort algorithm in action by creating an animation driver that can be control using buttons.

I hope you have a better idea about how to use SVG graphic and that this article was helpful is one way or another.

Check out other articles on wannabedev.io.

Everything you need to know about JSON Web Token

Ramo Mujagic — Tue, 05 Sep 2023 18:13:00 +0000

Ready to read about JSON Web Token so much that it becomes boring? Well, me neither, but here we are.

Before we start, keep in mind that this is a more conceptual guide and includes almost no code examples. If you are looking for an article that explains how to use JWT for a specific use case with a specific language implementation, you are in the wrong place. With that out of the way, let's get started.

Upss, I almost forgot, here is table of contents so you can jump to different topics quickly.

What is JWT
- Payload
- Header
- Signature
Token Claims
- Registered Claims
- Public Claims
- Private Claims
Formats
- JWS (JSON Web Signature)
- JWE (JSON Web Encryption)
How It Works
- Token Generation
- Token Verification
- Token Expiration
- Token Revocation
HS256 Algorithm
- Pros
- Cons
RS256 Algorithm
- RSA encryption
- JWT signature generation
- JWT signature verification
- Pros
- Cons
Benefits of JWT
JWT and Authentication
- Authentication server
- Application server
JWT and Authorization
Best practices
Conclusion

What is JWT

JWT is an open industry standard (RFC 7519) that defines compact, simple and self-contained way to securely share information between two entities. As the name implies, information contained in JWT is provided in JSON format.

JWT is specifically made to be used in environments like HTTP Authorization headers and URI query parameters. This makes JWT a great option for authentication and authorization purposes as well as information exchange. It consists of three different components: the header, payload, and signature.

Header and payload are formatted using JSON format, while signature usually get generated one way or another. Every component serves a specific purpose, and JWT is not complete if either of these components are missing.

We will talk much more about JWT components, but for now, let's introduce them so we have an idea of what they look like.

Payload

The payload holds the actual data that we want to secure and transmit using JWT. It can literally be anything. Take a look at the following example.

{
  "name": "John Doe",
  "moderator": true
}

The payload is written in JSON format and includes key-values pairs, also known as claims.

Header

The header component of a JWT contains metadata about the token and the cryptographic algorithms used to secure it.

{
  "alg": "HS256",
  "typ": "JWT"
}

It typically consists of two parts:

typ (Type): declare the media type.
alg (Algorithm): signing algorithm used to sign JWT.

There are also more properties that you can specify in the header, but only alg is required. The header is very important, especially for token verification, since it provides information about the algorithm used to sign JWT.

Signature

The signature is the final component of a JWT. It is generated by combining the encoded header, encoded payload, and a secret key known only to the entity responsible for issuing the token.

Wait, slow down. Generated, encoded, secret key? What does it even mean? I know this sounds like a lot, but just keep going, and everything will come together.

That being said, the signature ensures the integrity and authenticity of the token. It is the only thing that can be used to prove that the token can be trusted.

Token Claims

When we talked about the payload, we said that it consists of key-value pairs, also known as claims. Well, this is exactly what token claims are: key-value pairs that contain pieces of information about the subject that the token is issued for.

The payload must not include duplicate keys (claim names).

There are three types of JWT claims: registered claims, public claims, and private claims. None of the mentioned claims are mandatory or required in the payload.

Registered Claims

Registered claims are defined by JWT specification and have standardized meanings. These claims provide useful information for interoperability and common use cases. In total, there are 7 registered claims, as follows:

iss (Issuer): identifies the entity that issued the JWT.
sub (Subject): represents the subject of the JWT, typically the user or client it belongs to.
aud (Audience): identifies the recipients that the JWT is intended for.
exp (Expiration Time): the expiration time on or after which the JWT should not be accepted anymore.
nbf (Not Before): the time before which JWT must not be accepted.
iat (Issued At): indicates the time at which the JWT was issued.
jti (JWT ID): unique identifier for the JWT.

Using registered claims helps ensure consistency and understanding across different systems that consume JWTs. This is important when working with third-party services, for example.

Public Claims

Public claims are custom claims that are defined by developers. They are used to convey application-specific information and can vary based on the requirements of the system.

They should be registered with an appropriate authority or namespace to avoid conflicts. You can see a list of available public claims in the IANA JSON Web Token Claims Registry.

Public claims enable developers to include additional context or data within the JWT to support their application's specific needs.

Private Claims

Private claims are custom claims that are specific to a particular application or organization. They are usually not meant to be used or understood by other parties. Private claims allow developers to include proprietary or internal information within the JWT.

Private claims are useful for carrying application-specific data without worrying about potential conflicts.

This all might seem unnecessary, but keep in mind that there are a lot of big companies and services that rely on these claims and recommendations to be followed. They provide the necessary context for authentication, authorization, and any additional data required by the consumer.

Formats

Before we can continue further, you need to understand that JWT is just an abstraction built on top of already-existing industry standards. These standards are JWS (JSON Web Signature) and JWE (JSON Web Encryption).

This means that JWTs can be represented in two different formats: JWS or JWE. The information about which of these formats is used is presented in the JWT header.

Still remember this alg property in the header that defines which algorithm is used to secure the token? Well, it turns out that JWS and JWE have different algorithms. So, just by specifying the algorithm, you are also selecting the format itself.

JWS (JSON Web Signature)

JWS is by far the most common way to represent JWT. Also, in the rest of this article, we will focus only on JWT represented using JWS.

In JWS format, the token is signed using an encoded header, encoded payload, and secret key (password). This process will generate a JWT signature. The signature provides authenticity and integrity to the token.

Look at the payload that we had before.

{
  "name": "John Doe",
  "moderator": true
}

When a signature is created and a token is shared, this payload will be completely visible to everyone who gets the token.

You might ask yourself, why is this token useful when everyone can see its payload? The important thing is that you should not share any sensitive data in the payload. Even if a potential attacker gets his hands on the token, he will not be able to extract any meaningful information from the payload.

Obviously, JWS has its own advantages and disadvantages, but we will stop here, or this part might get too big very quickly.

JWE (JSON Web Encryption)

The payload of a JWE token is encrypted using a private key (asymmetric encryption), and only entities with the corresponding public key can decrypt and verify the token.

So, in practical terms, the JWT payload will be encrypted when using JWE format and completely visible when using JWS format.

JWE is useful when the payload contains sensitive information that should be kept confidential. But for most use cases, it is not required.

Remember, in this article, when we say JWT, it means JWT in the form of JWS. It is the most commonly used form and the one you will most likely encounter much more often.

How it works

We talked a lot about more theoretical and perhaps boring stuff. Let's switch our focus to what actually matters and explore how JWT actually works.

Like almost any token, it is important to consider how to generate, verify, revoke, and expire JWT. Let's start by exploring token generation.

Token Generation

Token generation describes the process of creating JWT. It usually involves a couple of steps that we will go through.

The first thing we need to do is define the payload for our token. It can literally be anything; depending on specific usage, different claims can be provided. The payload might look something like this:

{
  "name": "John Doe",
  "iat": 1687716785,
  "exp": 1741651200
}

Notice that we added some registered claims like iat and exp, but not all of them. It is up to you to decide which claims to add.

With that out of the way, we need to consider which signing algorithm to use for our JWT. There are many types of algorithms available, but the most common are HS256 and RS256.

We will talk more about the HS256 and RS256 algorithms a little bit later.

For now, let's keep it simple and select HS256 as the signing algorithm for our JWT. With that decision out of the way, we can also create a header like this:

{
  "alg": "HS256",
  "typ": "JWT"
}

Next, we have to generate the signature. Remember, the signature is the part that actually ensures the integrity and authenticity of the token and is the only thing that can be used to prove that the token can be trusted.

To generate signature, we have to combine header and payload. But, there is one more thing we have to do before that. We can not just leave them in plain JSON format. After all, the token will be transferred via HTTP headers or URI query parameters and it has to be safe to use in such environments.

This is where base64url comes into play. It is basically the same encoding as base64, but it uses a URL and filename-safe alphabet making it perfect for applications where sharing via URL is important.

Header and payload first have to be converted to UTF-8 strings, which are basically a stringified version of the JSON format. This string can then be encoded using base64url, which will produce an encoded version of the header and payload.

We can now use the encoded header and payload to generate the signature. Here is where things could change a little bit. Depending on the algorithm specified in the header via the alg property, signature generation might be a little bit different.

Since we decided to use the HS256 algorithm, we have to provide the symmetric secret key. It is obviously important to choose a hard-to-guess sequence of characters. For our arbitrary example, let's just say that the secret key is the string super-secret.

A symmetric secret key is used both to sign the token and to verify it.

Finally, we can use the HS256 (HMAC-SHA256) cryptographic hash function to generate the signature. This function is already implemented for probably every language you can think of.

HMACSHA256(
  encoded_header + "." + encoded_payload,
  "super-secret"
)

The output of this operation is going to be our signature, also known as HMAC (Hash-Based Message Authentication Code), which is a form of digital signature. The generated signature also has to be base64url encoded.

Lastly, we have to create a compact version of the JWT. To do so, we need to combine the encoded header, payload, and signature separated by . (period) character.

// base64url encoded JWT components
header = "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9"
payload = "eyJuYW1lIjoiSm9obiBEb2UiLCJpYXQiOjE2ODc3MTY3ODUsImV4cCI6MTc0MTY1MTIwMH0"
signature = "zrwE8dR_RbbkjKsxul-U8cL9FUcMjeCvOSnk6obd0Xo"

// JWT
token = header + "." + payload + "." + signature

As you can see, the final JWT is a string representation of the encoded header, payload, and signature. This compact form of the JWT is actually what we can use in our applications and APIs.

On the first glance, there might be a lot of moving parts and things we have to consider, but it is not that hard. If we put it all together into a single image, it might look something like this:

We have covered a lot of things just now. Take your time and read it again if something is not clear. Here are the most important things related to token generation:

Prepare the token payload by adding the needed claims. Try to add as many registered claims as possible.
Decide on which algorithm to use.
Populate the token header by providing a minimum alg claim.
Make sure to choose a strong secret key and keep it safe.
Encode the header and payload using base64url and concatenate them using a . period.
Create a token signature by running the HMAC-SHA256 hash function on the concatenated header and payload together with the secret key.
Encode the token signature using base64url.
Generate compact form by concatenating the encoded header, payload, and signature.

Keep in mind that token generation will be slightly different depending on the algorithm you choose to use. Here, we have used HMAC-SHA256 by specifying HS256 in the header alg property. We will talk more about HS256 and RS256 algorithms later in this guide.

Token Verification

It's time to explore the other end of the spectrum: token verification. Before we can verify the token, we need to know which algorithm was used to digitally sign it.

To do so, we can take the header part of the token and inspect the alg property. More often than not, we would know this information ahead of time since we could get it from the entity that generated the token. Anyway, we know that the algorithm used is actually HS256, which means that the token was signed using HMAC-SHA256.

The only way to confirm that the token can be trusted is to generate a control signature and compare it to the signature obtained from the token.

Remember, a token consists of an encoded header, payload, and signature separated by the . period character. As a reference, look at the token that we generated in the previous section:

// Separated with a new line for an easier overview.
eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.
eyJuYW1lIjoiSm9obiBEb2UiLCJpYXQiOjE2ODc3MTY3ODUsImV4cCI6MTc0MTY1MTIwMH0.
zrwE8dR_RbbkjKsxul-U8cL9FUcMjeCvOSnk6obd0Xo

There is one more very important thing that we need to generate the signature: the secret key. The same secret key has to be used to generate and verify the token. Otherwise, the verification step would fail. So, when using HS256, you need to make sure that the secret key is available at the point where the token is generated and at the point where the token is verified.

Now that we have all the required data, we can finally verify the token. First, we split the token into three components: the encoded header, encoded payload, and encoded signature. Next, we concatenate the encoded header and encoded payload using . period.

The resulting string is taken and passed to HMAC-SHA256 together with the secret key to generate the control signature. We also need to make sure that the control signature is encoded using base64url.

Finally, we can compare the control signature with the signature from the token and check if they are an exact match. If they are the same, we know that the token can be trusted and that no one has tried to change its payload. If the signatures do not match, the token is invalid, and someone has tried to compromise it.

It is also important to check the claims provided in the payload. To do so, we first have to decode the payload using the revers action of base64url. Depending on the requirements, token claims can be used to further fine-tune the verification process by ensuring that they meet the required criteria.

Token Expiration

JWTs can have a limited lifespan to enhance security and mitigate the risks associated with long-lived tokens. It is very important to create tokens in such a way that they can expire. If expiration time is not taken into consideration, in theory, tokens could be valid indefinitely.

Every JWT should include an expiration time provided via the exp claim, which indicates the timestamp after which the token is considered invalid.

However, just adding an exp claim is not enough. The token verification strategy also needs to be implemented in such a way that the exp claim is always checked. All expired tokens should be rejected, no matter what.

Token Revocation

We just talked about expiration and the exp claim, but it cannot help us in all situations. What happens if we lose or leak a valid token that has not yet expired? Anyone in token possession could use it to access restricted resources. This is where revocation and revocation strategies come into play.

Token revocation is a big issue for JWT since they are stateless. This means the token itself holds all the data necessary for its verification. The entity that created the token does not need to keep any token-related state. So, whenever a token is lost, it will continue to be valid until it expires.

What can we do to invalidate these tokens? The common way to do it is to create some kind of storage for revoked tokens. This storage should always be checked during token verification. If the token is found in the storage, it should be marked as revoked, and access should be denied.

With this approach, a token does not have to expire to become invalid. As soon as it's added to the list of revoked tokens, it is basically useless since its verification will fail.

There are also different strategies to manage the revocation process, but that is not the topic of this article. If you are curious, I encourage you to explore it on your own.

HS256 Algorithm

As we have said many times before, the security of JWT mainly depends on the type of algorithm we choose to use. So, let's check out HS256 in more detail.

HS256, which stands for HMAC-SHA256, is one of the popular signing algorithms used in JWT. HMAC (Hash-based Message Authentication Code) is a symmetric cryptographic algorithm that uses a shared secret key to generate a hash-based MAC (Message Authentication Code), also known as HMAC. Using a shared key means that the same key has to be used to sign and verify the token.

We have already talked about signature generation and verification using the HS256 algorithm.

I will not go into details about how this algorithm generates a token signature. If you want to know how it works under the hood, read about HMAC, hashing functions, and SHA-256. They all play a role here.

Pros

HS256 is computationally efficient and performs well in terms of generating and verifying signatures. The algorithm is relatively fast and can handle a high volume of token operations.

Implementing HS256 is straightforward and simple. It only requires a shared secret key between the issuer and the recipient.

I would like to add more pros, but realistically, compared to RS256, there are not many left on the table. That being said, HS256 is still a fine algorithm to use if we take special care about the cons.

Cons

Even if it seems like an impossible task, HS256 signatures can still be brute-forced. This mostly happens if the chosen secret key is weak and easily guessable.

Maybe the biggest weakness is the existence of a symmetric secret key that has to be shared between issuer and receiver and any other entity that is using JWT. This comes with a lot of issues related to the management of secret keys and token revocation.

The secret key contributes to the difficult key rotation. Key rotation requires the secret key to be changed on every entity that is using JWT. It becomes increasingly hard and error-prone based on the number of entities relying on the secret key, as you have to update the secret key for each one of them.

Another issue is the lack of separation between token generation and token verification. When using HS256, every entity can both generate and verify a token, and those actions cannot be separated. This increases the chances of a secret key being stolen since there are just more places where it could happen.

I think you get the point. In the long run and in more complex systems, HS256 can cause some really serious security concerns. Luckily for us, RS256 can solve all these problems.

RS256 Algorithm

RS256, which stands for RSA-SHA256, is another widely used algorithm for signing JWTs. RS256 is based on asymmetric cryptography, utilizing a key pair consisting of a private key for signing and a corresponding public key for verification.

I know there are a lot of different things going on: RSA, SHA256, cryptography, private and public keys. What is all of that?

Well, let's say that many smart people have come together to improve the security of the Internet. This is how we got all of these concepts, algorithms, and implementations. They are by no means only used in JWT but rather in many different applications, but that is a topic for another time.

To keep it concise, we will explore how RS256 works at a higher level. We will not go into too many details, but it will be just enough for you to grasp the concept.

At the end, we only need this digital signature for our token, so we can assume that the final product of the RS256 algorithm is just that: a digital signature that we can append to our token.

RSA encryption

As RS256 uses the RSA asymmetric cryptography algorithm and RSA keys, we have to talk about them a little bit. This will be a very short introduction to RSA encryption, and I encourage you to explore more.

RSA uses a key pair consisting of two correlated keys: private and public. A private key should always be kept in a safe place and should only be used for encryption. A public key can be shared with anyone and should only be used for decryption.

So what the private key encrypts, the correlated public key is able to decrypt. This is very important, as it allows the private key to stay safe since it's not required for decryption. Let's examine how it translates to the process of generating JWT signatures.

JWT signature generation

We said before that, depending on the chosen algorithm, the process of generating a signature might be slightly different. It's time to explore what it looks like for RS256.

First, we need an RSA key pair. Since RSA is a very common and widely used algorithm, there is an official implementation of it in every common programming language. So we do not have to write it from scratch but rather use the existing implementation to generate the key pair. For now, let's assume that we have the needed key pair.

We can take an encoded header and encoded payload and hash them using SHA256. The output of this hashing function is by no means secure. It can be easily recreated by the attacker using the same header and payload taken from the token itself.

To make it secure, we have to take the hash output and encrypt it using the RSA private key. This will produce an RS256 digital signature. Before adding it to the token, the generated digital signature needs to be encoded using base64url.

JWT signature verification

When doing verification of the RS256 signature, we cannot generate exactly the same signature as we could for the HS256 algorithm. For that, we would need to have a private key, which should only be available to the entity that generates the token.

Instead, we can use a public key, available at the point of verification, to help us determine if the signature is valid.

We first need to take the encoded header and encoded payload, concatenate them with a . period, and pass the result to the SHA256 hashing function to generate the hash.

Do you still remember what we can do with the public key? That's right, we can take the signature from the token, decode it, and decrypt it using the public key. As a result, we will get the original signature hash.

Now we can compare our generated hash with the hash from the signature. If they are the same, we can conclude that the signature is valid and can be trusted.

Anyone could have calculated that hash, but the decryption process to get the original signature hash from the token would only work if the token signature was encrypted with the matching RSA private key.

Pros

As you might have noticed already, RS256 provides higher security and resistance to brute-force attacks. This makes it probably impossible for attackers to forge signatures.

Compared to HS256, there is an added RSA private and public key pair, one for encrypting and one for decrypting data. This eliminates the need for a shared secret key, which caused a lot of security concerns with the HS256 algorithm.

Relatively simple key rotations can be executed using RS256. If the private key is compromised, a new RSA key pair can be quickly created, and the public key can be distributed immediately to all relevant entities.

One nice side effect of using a public and private key pair is that it provides great separation of concerns. With RS256, the private key should only be available to the entity that handles token generation, while the public key is available to the entity that has to perform token verification. This greatly increases security by reducing the exposure of private keys.

Cons

There is not much bad going on with the RS256 algorithm. However, if we want to be nitpicky, there are some things to consider.

Using RS256 can introduce computational complexity since it involves operations like encryption and decryption, which can be resource-intensive when dealing with long messages. But this is also not a major concern since it is only required to encrypt or decrypt the signature hash, not the whole token.

Some might say that key management could cause some concerns related to the secure generation, storage, and distribution of RSA key pairs. I think it's a valid point, but as with any credentials, you have to proceed with care and be aware of related risks.

As a last note, compared to HS256, we could say that it is marginally harder to implement JWT using RS256, mainly because of asymmetric cryptography and RSA key pairs.

Benefits of JWT

After reading all of this, it might still not be clear what the benefits of JWT are compared to other forms of authentication. JWT has gained widespread popularity as a secure and efficient method for transmitting data between parties because of the benefits it provides, so let's explore some of them.

Industry standard - JWT was created as an industry standard, which further encourages its usage. Being an industry standard ensures interoperability across different platforms, programming languages, and frameworks. Developers can confidently implement JWT-based solutions, knowing that they will work seamlessly across different technologies.
Compact - The less data you send over the Internet, the better. JWT fits nicely because of its compact form. When encoded, it can easily be sent via URL query parameters or HTTP Headers.
Strong security - Tokens are digitally signed, which prevents attackers from modifying token content. Depending on the algorithm used in signature creation, asymmetric keys can be used to further improve security.
Stateless - The token itself contains all the data necessary for it to be verified, which means that the server doesn't need to store any session information. This eliminates the need for server-side session management and improves scalability.
Single Sign-On (SSO) capable - JWT is widely used in SSO systems. Once a user logs in and receives a JWT, they can use the same token to access multiple applications or services without the need for separate authentication. This seamless user experience simplifies the login process and enhances user productivity.
Customizable - JWT supports custom claims, allowing for the inclusion of additional information relevant to the application's specific requirements. These custom claims can be used to convey application-specific data, user preferences, or any other necessary information.

I hope you can grasp why JWT is so popular and widely used by looking at the listed benefits. With all of its benefits, JWT is challenging other forms of authentication and authorization and gaining ever-growing support.

JWT and Authentication

We know by now that JWT can be used for user authentication, but how exactly does that work? Depending on the application requirements, there are different ways to implement user authentication.

This section will include code snippets written in JavaScript. Node, the JavaScript runtime, can be used to run the snippets.

Let's assume that we want to set up user authentication for our application. We will have the authentication server responsible for authenticating users and generating JWTs, and the application server that will verify JWT and handle requests from the application client.

In the image above, you can see the basic authentication flow for our application. The application client represents the user interface for our application. It can be a website or mobile app. The user wants to authenticate with our application and goes to the login page.

The application client will ask for the user's credentials, like email/username, and password. When the user enters the credentials, the following authentication flow will be initialized:

The application client will take the credentials and send them to the authentication server in the body of an HTTP Post request via a secure HTTPS connection.
The authentication server takes the credentials and checks if the user with the provided credentials exists. If the user is found, the authentication server will generate a JWT and return it as a response to the request.
The application client now holds JWT specific to the user and can use it to access protected resources on behalf of the user. It will now redirect the user to the main page of the application and make a new HTTP GET request with the token provided via the HTTP Authorization header on the application server.
The application server will take the token provided via the Authorization header and verify its signature and claims. If the token is valid, the application server will return user-specific data to the application client.

It is very easy to split the authentication server from our application server when using JWT. This is why we can outsource the authentication part to third-party services like Clerk, Auth0, or any other provider. This allows us to focus on the business logic of our application, which is handled by the application server.

Now that we understand how the authentication flow works, we will focus on the authentication server, which is responsible for generating JWT.

Our goal here is to point out important parts related to JWT generation and verification. We are not going to write server or business logic implementations.

Authentication server

We already said that the authentication server is used to authenticate the user and generate JWT, which can be used by the application client to access restricted resources on the application server.

Before handling the requests from the application client, we have to prepare all the necessary data required for token generation. The first thing we need to do is decide which algorithm to use for signing JWT signatures. Based on the listed benefits we saw before, I think it is safe to say that the RS256 algorithm would be a better option than the HS256 algorithm.

I hope you still remember how RS256 works. Since it uses RSA encryption, we have to generate an RSA key pair: a private and public key. There are many ways to do this, but since we will work with Node, it is also fitting to generate RSA keys using Node.

import fs from 'fs';
import crypto from 'crypto';

const KEYS_DIR_PATH = './keys';

function generateKeyPair() {
  const { publicKey, privateKey } = crypto.generateKeyPairSync('rsa', {
    modulusLength: 2048,
    publicKeyEncoding: {
      type: 'spki',
      format: 'pem'
    },
    privateKeyEncoding: {
      type: 'pkcs8',
      format: 'pem'
    }
  });

  return { publicKey, privateKey };
}

const { publicKey, privateKey } = generateKeyPair();

if (!fs.existsSync(KEYS_DIR_PATH)) {
  fs.mkdirSync(KEYS_DIR_PATH)
}

fs.writeFileSync('./keys/private.pem', privateKey);
fs.writeFileSync('./keys/public.pem', publicKey);

This simple script can be used to generate the required RSA keys. It uses Node's native crypto module to generate the keys. When keys are generated, we just save them in the destination directory under the specified name. Private and public keys are saved in keys/private.pem and keys/public.pem files, respectively. With the groundwork done, we can switch to the more exciting topic of generating the JWT.

In our example, the authentication server will receive an HTTP POST request with credentials provided in the request's body. The authentication server needs to extract the credentials (username and password) and query the database to check if the user with the provided credentials exists. At this point, the server has retrieved user data and roles, and we can already define JWT payloads using user data.

const JWT_PAYLOAD = {
  /**
   * Custom claims.
   * This can be anything, depending on the use-case you have.
   */
  name: "John Doe",
  role: "EDITOR",

  /**
   * Registered claims.
   */
  sub: "22334455",
  iat: Math.floor(Date.now() / 1000),
  exp: Math.floor(Date.now() / 1000) + (3 * 24 * 60 * 60),
  iss: "Authentication server (NO.1)",
}

Now we have a JWT_PAYLOAD that holds data related to the user. You can notice that we have provided a couple of registered claims; we have talked about them in previous sections. Probably the most important one is exp, which says that the token will expire in 3 days. For iat and exp, we have to use the Unix timestamp format.

Other than registered claims, we also have a couple of custom claims. Here, we have the option to specify the claim name and claim value. Those claims are application-specific, and we can provide as much as we like. But be careful; those claims are visible to everyone, so they should not contain sensitive data.

With the payload ready, we need to prepare the header. We already know which algorithm to use, so the header for our token will look like this:

const JWT_HEADER = {
  alg: "RS256",
  typ: "JWT"
}

The next step is to create a JWT signature. For this part, we can use already-existing npm packages. You might ask, Why use npm packages? The answer is quite simple. This algorithm is complex and hard to implement; it is much faster and more secure to rely on a well-tested package maintained by a lot of smart people.

In this case, I decided to use the jose npm package. It is well maintained, supports all JWT use-cases, and has a nice growing community of supporters. To use it, we first need to install the package by running the following command:

npm install jose

This module provides a lot of useful functions and utilities, but we will focus only on a couple of them. You can read more about the module itself if you feel like it.

Before using any utilities provided by the module, we have to import the RSA private key. This key will be used for signing the signature hash.

const PRIVATE_KEY = fs.readFileSync('./keys/private.pem', { encoding: 'utf-8' })
const runtimePrivateKey = await jose.importPKCS8(PRIVATE_KEY, JWT_HEADER.alg)

We just have to read the content of the ./keys/private.pem file and save it into PRIVATE_KEY const. Before the jose module can use it, we have to make a runtime-specific private key using the importPKCS8 function and save it into the runtimePrivateKey const.

When the runtime-specific private key is ready, we can generate the final token in its compact form using the sign method on the instance of the SignJWT class.

const generatedToken = await new jose.SignJWT(JWT_PAYLOAD)
  .setProtectedHeader(JWT_HEADER)
  .sign(runtimePrivateKey)

You might think, What just happened? We did not encode the payload and header or take any intermediate steps, as described in previous sections. Well, simply put, the jose module did it for us; in fact, all the following things are covered by the module:

Convert the JWT_PAYLOAD object into the stringified version and encode it using base64url.
Convert the JWT_HEADER object into the stringified version and encode it using base64url.
Generate the signature using the encoded header, encoded payload, and RSA private key.
Encode the JWT signature using base64url.
Concatenate the encoded header, encoded payload, and encoded signature with a . period and produce a compact form of the token.

I hope you can see value in using the jose module here. It took care of all things related to token generation. At the end, we got the final token just by using a simple API. If we log the value of generatedToken, it looks something like this:

// Separated with a new line for an easier overview
eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.
eyJuYW1lIjoiSm9obiBEb2UiLCJyb2xlIjoiRURJVE9SIiwic3ViIjoiMjIzMzQ0NTUiLCJpYXQiOjE2ODk0MDcxMDYsImV4cCI6MTY4OTY2NjMwNiwiaXNzIjoiQXV0aGVudGljYXRpb24gc2VydmVyIChOTy4xKSJ9.
v9LXLIjGPtakpRvGPYYDvPyyQ1avxmJW2zlJ4ND_74YIPbloZBPGtJ3gMCjY40m5Qbf1jy7xv-jIphLOi2EBYNW4EUIGqZb3Lj0Tn4W_hZiL9tq1ASUma4nURVPREeDGpD_B5AojIjk-iiLc_K3Iin8kwG3u5L3RDU9zLOEdW2TUPwR-Q4NNm8kG_9OBoA2z0H6Uknk4Udv6VL0EuKKds1OXgsHAK9vRi-psGISB0W1AGHjegPFJXotcXW2DOBHKAUm-NTwcpPt0YMzNLmTLpfwzsqYyHJxwneQW-KQOTpBNpIUuWVh3qyJumyVk10hHLMlr8lOeHL_KjmH1vupOVA

Now, the authentication server can return the token to the application client as a response to the initial request. The application client can use the token to request protected data from the application server.

Application server

It's time to focus on the application server. This is where we need to handle token verification. Again, let's start with what we already know. First of all, we need access to the public key, and we also know that the algorithm used to sign the token was RS256.

As a side node, we could have gotten the information about the algorithm used from the header itself, as it is visible the same as the payload.

const PUBLIC_KEY = fs.readFileSync('./keys/public.pem', { encoding: 'utf-8' })
const SIGNATURE_ALGORITHM = 'RS256'

We just took those values and saved them into the PUBLIC_KEY and SIGNATURE_ALGORITHM const. You need to make sure that the public key is correlated, part of the same RSA key pair, to the private key used to sign the token.

The application client will make an HTTP GET request to one of the endpoints on our application server. This request will also include the token provided via the HTTP Authorization header.

The application server has to extract the token from the Authorization header so it can be verified. The token received from the header might look like this:

// Separated with a new line for an easier overview
eyJhbGciOiJSUzI1NiIsInR5cCI6IkpXVCJ9.
eyJuYW1lIjoiSm9obiBEb2UiLCJyb2xlIjoiRURJVE9SIiwic3ViIjoiMjIzMzQ0NTUiLCJpYXQiOjE2ODk0MDcxMDYsImV4cCI6MTY4OTY2NjMwNiwiaXNzIjoiQXV0aGVudGljYXRpb24gc2VydmVyIChOTy4xKSJ9.
v9LXLIjGPtakpRvGPYYDvPyyQ1avxmJW2zlJ4ND_74YIPbloZBPGtJ3gMCjY40m5Qbf1jy7xv-jIphLOi2EBYNW4EUIGqZb3Lj0Tn4W_hZiL9tq1ASUma4nURVPREeDGpD_B5AojIjk-iiLc_K3Iin8kwG3u5L3RDU9zLOEdW2TUPwR-Q4NNm8kG_9OBoA2z0H6Uknk4Udv6VL0EuKKds1OXgsHAK9vRi-psGISB0W1AGHjegPFJXotcXW2DOBHKAUm-NTwcpPt0YMzNLmTLpfwzsqYyHJxwneQW-KQOTpBNpIUuWVh3qyJumyVk10hHLMlr8lOeHL_KjmH1vupOVA

It is the same token that we generated previously. When the token is extracted, it has to be verified to confirm that the user is legit. Again, we will use the jose module to help us with this process.

const runtimePublicKey = await jose.importSPKI(PUBLIC_KEY, SIGNATURE_ALGORITHM)

try {
  const { payload, protectedHeader } = await jose.jwtVerify(
    userToken,
    runtimePublicKey,
    {
      algorithms: [SIGNATURE_ALGORITHM],
      issuer: 'Authentication server (NO.1)',
      requiredClaims: ['iss', 'exp', 'role'],
    }
  )

  if (payload.exp < Math.floor(Date.now() / 1000)) {
    const TokenExpiredError = new Error('Token has expired')
    TokenExpiredError.code = 'ERR_TOKEN_EXPIRED'
    throw TokenExpiredError;
  }

  // Token verified & User authenticated
} catch (error) {
  // Verification failed & Access denied
}

First, we have to import the public key and convert it to the runtime-specific key representation required by the module. We can do that using the importSPKI function.

We then use the jwtVerify function, providing the token, public key, and additional options. Via the options object, we can fine-tune verification with some smart defaults. Here is what we provided:

algorithms - specifies all allowed algorithms. Our tokens will be signed with the RS256 algorithm, meaning only this algorithm has to be allowed.
issuer - specifies allowed issuers. In authentication server, we set issuer to Authentication server (NO.1) which means that all tokens, generated by the authentication server, must have this issuer set.
requiredClaims - defines which claims must be available in the payload. We could list all the claims that the authentication server would use, but we have listed only important ones.

At the end, this function will return the payload and header, both in the form of JavaScript object. If any of these requirements are not met, the jwtVerify function will throw an error indicating that verification was not successful. Here is what the jwtVerify function did for us:

Take the token and extract the encoded header, payload, and signature.
Decode and convert the header and payload to plain JavaScript objects.
Check which algorithm is used to sign the token, and throw an error if it is not RS256.
Check that the issuer claim is set in the payload and allow only Authentication servers (NO.1).
Check the required claims provided in the payload and throw an error if a required claim is missing.
Verify the token signature using the RS256 algorithm.
Return the payload and header as a result.

It is obvious that jose module does a lot of heavy lifting for us, and this is why we want to use it anyway. Without the jose module, we would have to implement all these things, done by the module, manually.

Next, in the code above, we check the expiration time provided via exp claim. We just need to compare the exp claim with the current time on the server. If the timestamp provided in the exp claim is less than the current time, the token has expired, and we throw another verification error.

If no error has occurred during the verification process, we can conclude that the token is valid and can be trusted. Application servers can now return protected data requested by the application client.

JWT and Authorization

We've said a lot of times that JWT can also be used for authorization. But what is this authorization, and how is it different from authentication?

In simple terms, authorization is the process of determining whether an entity has the necessary rights and permissions to access a particular resource. It is basically access control.

Authorization usually only kicks in after authentication, which verifies identity. Once a user is authenticated, authorization comes into play to determine what they are allowed to do or access. It usually involves using some kinds of roles and/or permissions.

In this case, the user is already authenticated. We know his name is John Doe, and we know he has the role of editor. However, this user should not have access to all available APIs.

First verify, and then handle authorization logic.

When the user asks the application client to create a new post using the /create REST API endpoint, the server returns an HTTP Response with the status 200 OK. This means that this request has been successfully fulfilled.

On the other hand, when users try to delete a post using the /delete REST API endpoint, the server returns an HTTP Response with the status 403 Forbidden. This means that the user does not have the required permissions to access this API.

This example provides a simple use case for authorization. Depending on the permissions that the user has, different resources can be accessed.

We can easily extend the application server from our previous example to implement this behavior. Remember, we already have a custom role claim in the payload.

// JWT payload
{
  name: 'John Doe',
  role: 'EDITOR',
  sub: '22334455',
  iat: 1689407106,
  exp: 1689666306,
  iss: 'Authentication server (NO.1)'
}

We can just use this role claim to implement simple access control. As you might have noticed, authorization logic will be implemented on the application server since this server handles the business login.

In case JWT does not have any claims to resolve permission level, like role, we would also have to modify the authentication server to generate tokens with the required claim.

Our application server now has two additional endpoints: /create and /delete. When a user makes a request to one of those endpoints, the application server needs to verify the token in exactly the same way as before.

We just have to extend the /delete API handler to make sure that user has one of the allowed roles. It is as simple as adding an if statement.

const ALLOWED_ROLES = ['ADMIN', 'CHIEF_EDITOR']
if (ALLOWED_ROLES.includes(payload.role)) {
  // Can access (Response OK 200)
} else {
  // Can not access (Response 403 Forbidden)
}

With this simple check, we have added authorization to the /delete endpoint. As our user has the role of EDITOR, he would not be able to access this API.

This is just a simple example to demonstrate how easy it can be to add simple authorization using JWT. Because of JWT's flexibility, we can add any claims to the token to fine-tune access control even further.

Best practices

Over the years of usage by a lot of developers and companies, several recommendations and workflows have emerged that are so common and essential that they are rightfully described as best practices.

All though "best practices" are good things to follow and try to implement in our own applications, there are so many of them that it is often not possible. I will mention some of the most important ones, in my opinion, but I encourage you to investigate this segment on your own.

Always use HTTPS when transmitting JWT. It might seem obvious, but there are many websites out there that do not use HTTPS. Remember, JWT is often used as an access token, and it is critical to protect it from network attacks and other security threats.

JWT is flexible enough to allow us to put any kind of claim into the payload. But whatever you do with the token, do not expose sensitive data in the payload. If you use a signed token (JWS), everyone can see what's in the payload. The information provided in the payload is not secure at all.

When possible, use asymmetric algorithms. We already talked about the benefits of asymmetric algorithms over symmetric ones when we compared RS256 and HS256. Always try to use asymmetric algorithms, as they provide much more security.

To improve security, always set ext claim to a reasonably short expiration time. Short-lived tokens reduce the window of vulnerability if a token is stolen or compromised. Short-lived tokens can be used in combination with refresh tokens to renew access without requiring re-authentication.

Make sure that the token is issued by someone you know, so always check for the iss claim. Every token should have this claim, and there is no reason for a token not to include it. Tokens without an iss claim should just be rejected.

This one is obvious, but always verify and validate incoming JWTs. Verify the signature to ensure the token's authenticity and integrity. Validate the token's claims to avoid accepting expired, not properly formatted, or tampered tokens.

Consider implementing token revocation mechanisms, such as blacklisting, to handle scenarios where tokens are compromised.

On client applications, tokens have to be stored somewhere. Most often than not, they are stored in the local storage of the client's browser. Make sure to protect token storage on the client side from common attacks like, for example, cross-site scripting (XSS) attacks.

Include the aud claim to specify the intended audience of the token. This helps ensure the token is used for its intended purpose. The application server should check for the aud claim and verify that the token was issued to an audience that the server is part of.

Always remember that the security of JWT depends mostly on how we use and verify it. There is no magic algorithm that can save us if the underlying use of JWT is flawed. Following best practices is a way to prevent flawed use of JWT. They help us ensure the secure and effective use of JWT in our applications while mitigating potential security risks and vulnerabilities.

Conclusion

This was probably a long read, and you have finally reached the end. I hope you have more theoretical as well as practical knowledge about JWT (JSON Web Token).

As an open standard that provides a secure way to transmit information between parties in a compact and self-contained way, JWT has emerged as a powerful and versatile tool for secure authentication and authorization in modern applications.

Understanding how JWT works is essential for its proper implementation. As applications and networks continue to evolve, so will security standards. What is good today does not mean it will be good tomorrow. We always have to be ready to act and not fall behind in this evolution. I will try to keep this article up-to-date in case there are any changes in the JWT world.

Through this comprehensive guide, I really hope you have gained the knowledge and insights needed to finally say, "Yes, I understand JWT".

Check out other articles on wannabedev.io.

Create text change hover effect with JavaScript

Ramo Mujagic — Tue, 21 Mar 2023 18:25:30 +0000

I stumbled upon one website that had this hover effect where the original text would change to some random characters and start revealing itself over time.

It looked like a fun exercise, so I decided to recreate it. Below you can see how it looks and check out the source code as well. This is a beginner-friendly article and includes a step-by-step guide. If you completely understand the source code, feel free to skip reading so you do not waste your time.

In case you decided to continue reading, you might be interested in what we are going to cover here. We will start with a basic HTML structure and mention some CSS styles. However, the main focus will be on JavaScript. You can learn how to listen for and handle events, a little bit of string manipulation, and how to leverage the power of closure. If you are ready, let's get started.

HTML structure

As for any hover effect, you need to have something that you want to hover over. In this case, let's create a couple of links using <a> tag and put them into a body tag, just for demonstration purposes.

<body>
  <a class="text-hover-effect" href="#">pricing</a>
  <a class="text-hover-effect" href="#">projects</a>
  <a class="text-hover-effect" href="#">about us</a>
  <a class="text-hover-effect" href="#">subscribe!</a>
</body>

Notice that every <a> tag has a class attribute with a value of text-hover-effect. We will need it later so we can target those HTML elements with JavaScript.

CSS styles

Since the HTML structure is ready, let's add some styles. In this case, I only want to do some simple things, like changing the background color and making sure that links are centered on the page. We can also add styles to links so they look a bit nicer.

* {
  box-sizing: border-box;
}

body {
  margin: 0;
  background-color: #18122b;
  display: flex;
  align-items: center;
  justify-content: center;
  flex-direction: column;
  flex: 1;
  height: 100vh;
}

a {
  font-family: sans-serif;
  color: #635985;
  text-transform: uppercase;
  letter-spacing: 0.1rem;
  font-size: 2.6rem;
  font-weight: bold;
  box-shadow: 0 0 2rem 1rem rgba(0, 0, 0, 0.2);
  background: transparent;
  padding: 0.6rem;
  border-radius: 0.4rem;
  min-width: 370px;
  border: 1px solid #635985;
  text-decoration: none;
  display: block;
  margin-bottom: 2rem;
}

I will not go into details about what every CSS declaration does, you can always play around with them in the sample project. However, I want to bring your attention to one thing.

Even though we added some CSS, it is actually not required. Notice that there is not even a :hover pseudo-class. This is because the hover effect will be handled entirely by JavaScript.

Understanding the effect

To implement the desired effect and behavior, we first need to understand it. Take a look at the following image.

In the default state, the original text is visible. When the cursor hovers over the text, it will immediately be populated with random characters. Only the first character from the original text should remain the same.

After some artificial delay, the character in the second position from the original text will replace the randomly generated character in the same position. This delay and replace process will continue until the last character is reached and the original text is completely revealed.

I hope this was clear enough for you to understand what we are trying to create here. If not, do not worry; we will split it into manageable chunks and cover every chunk separately.

JavaScript code

This is the part where we will spend the most time. First of all, we need to somehow generate random characters for our effect. We can do that by creating a simple array of characters.

// 🔡 Characters to cycle trough
let allowedCharacters = ['X', '$', 'Y', '#', '?', '*', '0', '1', '+'];

The variable allowedCharacters holds a list of the allowed characters. Feel free to add or remove them from your version.

We also need a function that can randomly select one of the characters from the previously created list.

// 🎁 Function to return random character
function getRandomCharacter() {
  const randomIndex = Math.floor(Math.random() * allowedCharacters.length);
  return allowedCharacters[randomIndex];
}

Function getRandomCharacter uses Math object to randomly generate a number, index, between 0 and the length of the allowedCharacters array. The generated index is then used to get the character from the allowedCharacters.

To handle hover events, we need to get all <a> tags and attach event listeners to all of them.

// 🛠 Event handler
function handleHoverEvent(e) {
  // TODO: Add implementation
}

// Attach an event listener to elements
document.querySelectorAll('.text-hover-effect').forEach((element) => {
  element.addEventListener('mouseover', handleHoverEvent);
});

Here, document.querySelectorAll is used to get all elements that have a value of text-hover-effect in the class attribute. Since we only added this class to <a> tags, it will return the iterable collection of Anchor, <a>, elements.

Using addEventListener, we can add mouseover event listener to every element and pass the function handleHoverEvent to handle the events. Now, whenever a user hovers over any of <a> tags, handleHoverEvent function will be executed.

So far, so good. The groundwork is complete; it's time to work on the hover effect implementation now. As we already said, we will split the task into multiple chunks. For the beginning, let's focus on the following part:

On hover, we have to replace the original text, except for the first character, with a randomly generated one.

When an event is triggered, the handleHoverEvent function gets called and an Event-based object will be passed as an argument to the function.

function handleHoverEvent(e) {
  const text = e.target.innerHTML;
  const randomizedText = text.split('').map(getRandomCharacter).join('');

  e.target.innerHTML = `${text.substring(0,1)}${randomizedText.substring(1)}`;
}

The function will receive the e, the event object, when the event is triggered. First, we have to save the current text value from the <a> tag that was hovered. To do so, we can use e.target.innerHTML and save the value into the text variable.

Keep in mind that innerHTML gets or sets the HTML markup contained within the element. If your hovered element has other HTML tags as children, they will be serialized and returned as well.

Next, we need to generate randomized text with the same length as the original. This is where we can use the getRandomCharacter function that was created before.

The value saved in the text variable is a string that includes multiple characters. But getRandomCharacter generates only one character per call. To solve this problem, we can convert the original text into an array of characters using the split('') method.

Now that we have an array, we can chain the map method and pass the getRandomCharacter function to it. Now, for every original character, a random one will be generated. Finally, to convert the resulting array back to a string, the join('') method is used.

The "subscribe" string is used as an example to illustrate how the text is being processed. You can see that at the end, we have randomly generated text that is the same length as the original one.

Good, we now have two strings, one original and one randomly generated. The last step is to combine them and update the e.target.innerHTML property. To get the first character from the original text, we can use text.substring(0,1) and to get the rest of the characters from the randomized text, we can use randomizedText.substring(1).

We could have done this in a different way as well, but I think using the substring method is totally fine, and, if interested, you can find more details here.

After some delay, the next character from the original text must be shown instead of the random one, and it needs to be repeated until the last character is revealed. We can achieve this using for loop and the setTimeout web API method.

function handleHoverEvent(e) {
  const text = e.target.innerHTML;
  const randomizedText = text.split('').map(getRandomCharacter).join('');

  for (let i = 0; i < text.length; i++) {
    setTimeout(() => {
      const nextIndex = i + 1;
      e.target.innerHTML = `${text.substring(0, nextIndex)}${randomizedText.substring(nextIndex)}`;
    }, i * 70);
  }
}

String values are iterable, so we can use the for loop on them as well. We will loop through the original text saved in the text variable. In every iteration, we invoke the setTimeout function. Notice that the delay for setTimeout is multiplied by the i value. This allows us to properly schedule updates, one after another, using the same wait time of 70 milliseconds between each update.

Lastly, we just needed to modify the assignment part of e.target.innerHTML to include nextIndex value instead of hardcoded ones. We need the nextIndex to ensure that substring method works properly and produces the expected result.

We used the for loop and setTimeout to create scheduled tasks. When the delay time expires, setTimeout will invoke the task function, which will update the innerHTML value of the hovered element. This will be repeated for every scheduled task, generated by the for loop.

Our effect should work fine now, but there is one more problem. If you hover over the element really fast, the handleHoverEvent function could be triggered multiple times. This is obviously not what we want. We need to make sure that handleHoverEvent cannot be triggered unless the currently running effect is over.

To make this happen, we can rely on the power of closure. Consider the following code:

// 🏭 Creates new event handler with a private variable
function createEventHandler() {
  // 🏃‍♂️ Private variable: Keep track of the event in progress
  let isInProgress = false;

  // 👇 Event handler implementation
  return function handleHoverEvent(e) {
    if (isInProgress) {
      return;
    }

    const text = e.target.innerHTML;
    const randomizedText = text.split('').map(getRandomCharacter).join('');

    for (let i = 0; i < text.length; i++) {
      isInProgress = true;

      setTimeout(() => {
        const nextIndex = i + 1;
        e.target.innerHTML = `${text.substring(0, nextIndex)}${randomizedText.substring(nextIndex)}`;

        if (nextIndex === text.length) {
          isInProgress = false;
        }
      }, i * 70);
    }
  };
}

Most of the code is the same, but there are a couple of important differences. First of all, we have put our handleHoverEvent function inside another function called createEventHandler. Also, we make sure that createEventHandler returns handleHoverEvent. In doing so, we have created the conditions for closure to occur.

Why do we even need closure in the first place? In this particular case, we need to create a variable where we can store information if the effect is in progress. This variable must be private to the handleHoverEvent function since only this function should be able to update it. When it comes to creating private variables, closure is the way to go.

In createEventHandler, we have defined the variable isInProgress and initialized it with false. This variable is used to keep track of whether the current effect is still in progress.

In the function handleHoverEvent, as soon as the loop starts, we will set isInProgress to true, indicating that the effect has started. When the last task is executed, isInProgress is set to false, indicating that the effect has finished.

In the same function, at the beginning, we will check if isInProgress is set to true. In case it's set, we can assume that the effect is still running and return immediately, effectively skipping the handling of the event.

Now we need to modify the code responsible for attaching an event handler to the <a> tag, Anchor elements.

document.querySelectorAll('.text-hover-effect').forEach((element) => {
  const eventHandler = createEventHandler();
  element.addEventListener('mouseover', eventHandler);
});

For every Anchor, <a>, element new handleHoverEvent function will be created using the createEventHandler function. The resulting function is then saved into the eventHandler and passed to the addEventListener. Since we have leveraged the power of closure to keep track of the event in progress, new events will not be handled until the currently running one is finished.

Conclusion

Initially, I did not plan to make this tutorial too big, but it turned out to be quite lengthy.

I wanted to give a detailed explanation of how this hover effect could be achieved and to show that it is not difficult at all if you split it into smaller pieces. You should be able to do this for every task—split and conquer.

This was also a nice opportunity to show just how useful the concept of closure is in JavaScript and how it can be leveraged in a practical example.

Check out other articles on my blog.

Mastering the Mysteries of JavaScript Scopes

Ramo Mujagic — Wed, 22 Feb 2023 07:28:41 +0000

While writing your programs in JavaScript you will quickly get familiar with creating variables and functions, assigning values, using and updating them. But, have you ever asked yourself how the JavaScript knows which variable or function is accessible by the given statement, or how are these things managed internally?

This is where scopes and the scope chain come into play. The concept of scopes and scope chain plays a vital role in determining the accessibility of variables and functions. Mastering their intricacies can make the difference between writing clean, maintainable code and struggling with bugs and unexpected behavior.

In this article, we will dive deeper into the inner workings of scopes and explore how they impact your code. That being said, this is not an article for total beginners and basic understanding of JavaScript is required.

Lexical Scope
- Global Scope
- Function Scope
- Block Scope
Nested Scope
Scope Chain
- Lookup
- Peek under the hood
Conclusion

Lexical Scope

From the perspective of a program, it is very important where you place variable or function declarations, as well as how and when you are using them.

The reason it's important is because of lexical scope and how the program behaves in relation to it. By definition, lexical scope determines the visibility and accessibility of variables and functions based on their position in the source code. It makes code easier to reason about and helps avoid naming conflicts between variables.

In general, JavaScript has different kinds of scopes: global scope, function scope and, more recently, block scope. Some might argue that there is also module scope, but we will not discuss it in this article.

You might have also heard about the term local scope. It is a more general term that can refer to both function scope and block scope, depending on the context.

Global Scope

Let's immediately start with an example. Consider the following code.

var movies = [
  {
    title: "District 9",
    releaseYear: 2009,
    genre: ["Action", "Sci-Fi", "Thriller"]
  },
  {
    title: "Transformers: Age of Extinction",
    releaseYear: 2014,
    genre: ["Action", "Adventure", "Sci-Fi"]
  }
];

console.log(movies);

It is easy to spot that the movies variable is declared, initialized and printed to console using console.log. This is, however, not what I want to point out.

Even now, with a program as simple as this, there is a scope created. This scope is referred to as global scope. It exists in every program and contains all other scopes that we could possibly define.

Think of it as a container where variables and functions can be declared and other scopes created.

When a variable or function is defined in global scope, it can be accessed from any other scope through the program. There are some exceptions to this, but it should not bother you for now.

There is also one more important aspect of global scope, in JavaScript, called global object. Global object always exists in the global scope.

We all know that JavaScript code can be executed in the browser. However, there are different environments that can run JavaScript code like, for example, Node.js runtime.

Depending on the environment where JavaScript is hosted, a different global object might be present. In browsers, the global object is more commonly known as Window, but in Node.js it's referred to as global.

Since the global object is part of JavaScript language, in ES2020 specification, it has finally received a unique identifier globalThis.

Function Scope

Whenever you create a function, function scope is created along with it. No matter if you use function definition, function expression or even arrow function, function scope is always present.

It exists inside the function body. Every variable, or even other functions, created inside the body will be part of the function scope.

var movies = [...]

function printTitle(index) {
  var movie = movies[index]
  var title = movie.title

  console.log(title)
}

printTitle(1)

Look at the body of the function printTitle, space between {...}, it has two variables defined as movie and title. Since variables are defined inside the function body, they are also part of the function scope.

Also, notice that the movies variable is referenced, but it's not defined in the function scope. This variable can still be used by the function, since it's created in the global scope. Remember, every variable created in the global scope is accessible in all other scopes.

We can see how function scope is created inside the global scope when function is defined. In the image, the global object is removed since we will not talk about it anymore, but keep in mind, it is always there in the global scope.

You might have noticed that the identifier index is marked with a blue color. When we defined the function printTitle, we did it in the following way function printTitle(index) {...}. In other words, it has one parameter called index and, by default, every function parameter is added to the function scope.

If you try to access index, movie or title outside function scope, an error will be thrown.

var movies = [...]

function printTitle(index) {
  var movie = movies[index]
  var title = movie.title

  // 🖨 Prints to the console
  console.log(title)
}

printTitle(1)

// 🚫 ReferenceError: movie is not defined
console.log(movie)

Variables defined in function scope can not be accessed outside the function. When referencing a variable that is not defined in the scope, JavaScript will throw ReferenceError.

In the example, we have defined variables using the var keyword, but the same behavior would occur for variables defined via let or const keywords.

Block Scope

Before ES2015, block scoping did not exist in JavaScript. Reason being that the only way to create variables was using the var keyword.

When let and const keywords are introduced, block scoping has become possible. Having block scoped variables helps a lot to prevent name pollution and accidental overrides of variables that are defined outside the block scope.

Not all variables can be block scoped
Variables defined using var keyword can only be function scoped. When used inside a block, such a variable will leak its definition until the first function scope is reached.

Block scope is defined by {} and can be put basically in any part of the program. Consider the following code.

var movies = [...];

{
  var listOfTitles = movies.map((movie) => movie.title);
  let listOfReleaseYears = movies.map((movie) => movie.releaseYear);

  // 🖨 Prints to the console
  console.log(listOfTitles);

  // 🖨 Prints to the console
  console.log(listOfReleaseYears);
}

// 👇 Leaked definition
// 🖨 Prints to the console
console.log(listOfTitles);


// 🚫 ReferenceError: listOfReleaseYears is not defined
console.log(listOfReleaseYears);

Program above is making use of block scope by putting variables inside {}. Inside the block scope, variables listOfTitles and listOfReleaseYears are defined, initialized and printed to console using console.log. This works as expected, the content of both variables is printed to the console.

When we try to print them again, outside the block scope, listOfTitles will be printed but trying to reference listOfReleaseYears will throw a ReferenceError. Any idea why is this happening?

If we look back at the code, notice that listOfReleaseYears is defined using the let keyword. This is the reason why the variable listOfReleaseYears can not be used outside the block scope where it was initially created. On the other hand, listOfTitles is defined using the var keyword and it does not see block scope.

Using {} to create block scope might not be the most common way to do it. More often than not, block scopes will be created using statements like if...else or switch. In fact, for and while loops are also block scoped. It often can catch developers off guard since they might think that the scope inside the loops behaves the same as the function scope, which is not the case.

var movies = [...];

function printTitle(index) {
  var movie = movies[index];
  var title = movie.title;

  if (title.length > 12) {
    const truncatedTitle = `${title.substring(0, 12).trim()}...`;

    // 🖨 Prints to the console
    console.log(truncatedTitle);
  }

  // 🚫 ReferenceError: truncatedTitle is not defined
  console.log(truncatedTitle);
}

printTitle(1);

The variable truncatedTitle is defined inside the block scope created by the if statement, and it will be accessible as long as we reference it from the block scope. However, if we try to reference truncatedTitle outside the block scope, in function scope, ReferenceError will be thrown.

To get a better picture, how block scope fits into our program, let's visualize it to show scope boundaries.

If you look at the image and the source code, can you spot similarities? It should be obvious that scopes are defined by the source code. When you write your program, think about scopes as well. They will always be involved.

Nested Scope

In most non-trivial programs, chances are, many of the so-called nested scopes will probably be created. Nested scope does not represent a new kind of scope, remember, there are only global, function and block scopes in JavaScript.

The term nested scope is used to describe a situation in which one scope is defined inside another scope. With that being said, we have already encountered nested scope in our previous examples.

var movies = [...];

function printTitle(index) {
  var movie = movies[index];
  var title = movie.title;

  if (title.length > 12) {
    const truncatedTitle = `${title.substring(0, 12).trim()}...`;
    console.log(truncatedTitle);
  }

  console.log(title);
}

printTitle(1);

Can you spot which scope would be considered as nested? It is a block scope created by the if statement.

As we already know, the function printTitle will create a function scope. However, it only has global scope as its outer scope. When scope is defined directly inside the global scope, it is usually not considered to be nested.

The global scope is the outermost scope and any other scopes will, by default, be created inside it. For this reason, nesting usually implies a scope to be defined inside another scope that is not the global scope.

In our example, the block scope ticks all the boxes. It is nested inside another scope, created by the printTitle function, and is not directly in the global scope.

Many nested scopes can be defined through the program. This creates a hierarchical relationship between the scopes, where the inner scope has access to its own variables as well as the variables of outer scopes. This nesting of scopes allows for more complex programs to be built, as it guarantees separation of concerns and the creation of reusable code.

Let's get back to our program and shed some light on scope related terminology. If we look from the perspective of the printTitle function, we can say that the block scope, created by the if statement, is a child or inner or nested scope. In the same note, the function itself has the global scope as its parent scope.

Continuing, if we look from the perspective of the block scope, the function scope will be its parent scope or the first outer scope while the global scope will be it's second outer scope.

Another important term is current scope. It refers to the scope where the JavaScript engine is currently executing code. So, hypothetically, if the engine is executing code in the block scope, we can refer to it as the current scope.

If you find it a bit hard to understand nested scope and related terminology, try to read it again at a slower pace and don't give up so easily.

Scope Chain

So far, we have talked a lot about lexical scope and how it works, but there is one more important topic, closely related to lexical scope, that we need to cover.

We have said that scope determines visibility and accessibility of variable and function declarations. You might be thinking, but how does it work? It has something to do with the so-called scope chain.

Scope chain is a mechanism used to determine the accessibility of variables and functions through the program. It defines the order of scope resolution during the runtime/execution phase. To better understand this, let's talk about scope chain lookup first.

Lookup

When a referenced variable or function can not be found in the current scope, the engine has to perform a search in the scope chain. This search is usually called scope chain lookup.

var movies = [...];

function printTitle(index) {
  var movie = movies[index];
  var title = movie.title;

  if (title.length > 12) {
    const truncatedTitle = `${title.substring(0, 12).trim()}...`;
    console.log(truncatedTitle);
  }

  console.log(title);
}

printTitle(1);

Take a look at the function scope, the variable movies is referenced, but it is not defined inside the function scope. The same is for the block scope, the variable title is referenced, but it is not defined in the block scope.

So how does the engine know what is the value of each of those variables? This is where scope chain lookup comes into play.

When a variable or function is referenced, the engine starts at the current scope, looking for definition. Remember, current scope is the scope where the engine is currently executing the code. If the reference is not resolved in the current scope, the engine moves up in the scope chain searching for the reference. This process is repeated until the reference is resolved or the end of the scope chain is reached. In case that the reference could not be resolved, the engine will throw ReferenceError.

The variable movies is referenced in the function scope. Since it is not defined in the same scope, it could not be found. The engine moves upwards in the scope chain and tries to resolve the reference. The next scope in the chain is the global scope. Since the variable movies is defined in the global scope, the engine can resolve the reference and get the value.

Similarly, when the engine tries to access title from block scope, it has to do a lookup, go upwards in the scope chain trying to resolve the reference. In this case, the reference will be resolved in the function scope since the variable title is defined in there.

To recap, scope chain lookup refers to the process of searching through a list of scopes to find a referenced variable or function. The engine will start at the current scope and move upwards the scope chain until the reference is resolved or ReferenceError is thrown.

Peek under the hood

At this point, you should have a better understanding of lexical scope and scope chain. It's time to go one step deeper and look under the hood to see how the engine creates and maintains a scope chain.

Scope chain is constructed and maintained during the execution phase. Just before the engine starts executing a piece of code, it needs to create a structure that holds variables and functions declarations. This structure is known as the lexical environment.

Before you ask, yes, lexical environment is related to lexical scope. It holds information about identifiers and their values that are defined in lexical scope.

Each lexical environment has a reference to its parent environment, creating the scope chain. Do not be concerned about how the engine implements scope chain structure under the hood, after all, it's a job of the engine.

Now, when we have a good picture of how it works, let's use the same code from previous examples and walk through steps that the engine might take to create the scope chain.

This will not be a detailed description on how the engine executes the code. Our goal here is to point out parts that are important for the scope chain creation.

var movies = [
  {
    title: "District 9",
    releaseYear: 2009,
    genre: ["Action", "Sci-Fi", "Thriller"]
  },
  {
    title: "Transformers: Age of Extinction",
    releaseYear: 2014,
    genre: ["Action", "Adventure", "Sci-Fi"]
  }
];

function printTitle(index) {
  var movie = movies[index];
  var title = movie.title;

  if (title.length > 12) {
    const truncatedTitle = `${title.substring(0, 12).trim()}...`;
    console.log(truncatedTitle);
  }

  console.log(title);
}

printTitle(1);

Obviously, code first needs to be parsed and compiled. At this point, the engine already knows about lexical scopes. Just before execution is started, the engine will create the global lexical environment.

One important step, more commonly known as hoisting, will occur during the creation of every lexical environment. The engine will hoist variables and functions to the top of their respective scopes and assign them their default values.

Hoisting behaves differently for variables defined using let and const keywords.

Taking the hosting into account, a freshly created global lexical environment, for our program, might look like this.

{
  movies: undefined,
  printTitle: <ref. to function printTitle>,
  outer: null
}

Finally, the engine starts executing the code, line by line. It will start with var movies = [...]. Since the movies variable is already created, courtesy of hoisting, the engine will create an array of objects in memory and save the reference to that array into the movies variable.

Next, the engine will see the function printTitle part of the code. However, there is nothing that needs to be done here. During hoisting, the engine has already created the printTitle identifier and assigned, to it, a reference to the printTitle function.

Good, the engine has made some progress and the global lexical environment looks like this now.

{
  movies: <ref. to array>,
  printTitle: <ref. to function printTitle>,
  outer: null,
}

The variable movies has been updated, and it now holds the reference to the array of movies saved in the memory.

It is important to point out that the global lexical environment contains movies and printTitle identifiers only because we have put them into the global scope.

Did you notice the outer identifier in the global lexical environment? This identifier is injected by the engine whenever a new lexical environment is created. It has a value of null since the engine is executing code in the global scope.

Continuing, the engine will see that we have called the printTitle function passing number 1 as an argument. Calling a function will trigger function execution. However, before the code inside the function gets executed, the lexical environment for the function itself will be created and hosting, for the function scope, will occur.

{
  index,
  movie: undefined,
  title: undefined,
  outer: <globalLexicalEnvironment>
}

Since the function has received 1 as the first argument, the engine will immediately initialize the parameter index and set its value to 1.

Notice that outer identifier has reference to the global lexical environment. This reference allows the engine to access the global lexical environment. You know what that means? That's right, the engine has just created the scope chain.

When executing var movie = movies[index] the engine will see that the variable movies does not exist in the current scope, and it will use the reference, saved in the outer identifier, to access the global lexical environment. Now, it will start looking for the movies identifier and resolve its value. Does this sound familiar? The engine has just performed a scope chain lookup.

The variable movies holds the reference that points to the movies array saved in memory. According to our code, movies[index], the engine will take the object on the index 1 and save its reference into the movie variable. At the end, reference to the second object in the movies array is saved into the movie variable.

Continuing, statement var title = movie.title is next to execute. To initialize the title variable, the engine will need to resolve the movie identifier. Since it's declared in the local scope, scope chain lookup will not be required. As we already know, the variable movie holds reference to the movies[1] object. The engine will take the value from the property title which is string "Transformers: Age of Extinction" and save it into the title variable.

Reference or value?
It is very important to know when the engine is passing/returning reference and when the actual value. In general, only primitive data types are passed by value, and non-primitive ones are passed as a reference.

The engine has just executed the first two lines inside the function. Here is how the local (function) lexical environment looks like now.

{
  index: 1,
  movie: <ref to movies[1]>,
  title: “Transformers: Age of Extinction”,
  outer: <globalLexicalEnvironment>
}

Both movie and title are updated. Notice that the movie variable now holds reference to the movies[1] object, while title holds the string value "Transformers: Age of Extinction".

Woah, the engine is flying through our program and we are already in the if (title.length > 12) part of the code. When the engine checks title.length > 12, it will get true as a result. Remember, title is still part of local (function) scope, so no scope chain lookup is required.

Because of the truthful condition, the engine will create a new lexical environment for the if statement. Remember from before, this lexical environment is created only because we used const inside the if statement.

After creating a new lexical environment for the if block, taking hoisting into consideration, the engine will give us something like this.

{
  truncatedTitle,
  outer: <functionLexicalEnvironment>,
}

The engine has created yet another lexical environment, and look where does the outer identifier points to? That's right, to the function lexical environment, and now we have scope chain that consists of 3 lexical environments.

Notice that truncatedTitle is not initialized with any value. At the moment, truncatedTitle is in the so-called TDZ (Temporal Dead Zone). The engine knows about the truncatedTitle variable, but it can not be used before it's initialized or ReferenceError error will be thrown. This only happens when using variables declared with let and const.

The engine starts executing the first line of code inside the block const truncatedTitle = `${title.substring(0, 12).trim()}...`. Here is where the engine will initialize the truncatedTitle variable and, after this point in time, TDZ for truncatedTitle ends.

The engine will search the scope chain to resolve the title identifier and get the value. This identifier will be resolved in the function scope. Because of substring and trim string methods, the engine will return the first 12 characters, removing potential empty space along the way. At the end, the value saved into the truncatedTitle will be string "Transformers...".

{
  truncatedTitle: “Transformers...”,
  outer: <functionLexicalEnvironment>,
}

Just before the end of the block scope, console.log(truncatedTitle) will be executed and the value of the truncatedTitle variable will be printed to the console.

Continuing, the engine has to exit the block scope. Usually, when the engine exits a lexical scope, the lexical environment for that scope is typically destroyed and garbage-collected by the engine's memory management system. This means that any variables and functions declared in that scope are no longer accessible and do not occupy memory. This is an automatic process that happens in the background and usually the developer does not have to worry about it.

Taking that into account, when the engine exits the block scope, the block lexical environment will be destroyed, which means that variable truncatedTitle no longer exists and its value "Transformers..." is removed from the memory.

If we take a look at the scope chain, the lexical environment created for the block scope is removed from the scope chain. The engine will manage this automatically when the lexical environment is destroyed.

Next in order of execution is console.log(title). The engine is executing in the function scope again. This means that the variable title is accessible from the local lexical environment and scope chain lookup is not required. When the engine is done executing this line, the value of the variable title will be printed to the console.

Continuing, the engine has to exit the function and garbage-collection will kick in by destroying the lexical environment and releasing used memory. Now, if we look at the scope chain, it has only one lexical environment left.

Finally, there is no code left to be executed, and the engine has reached the end of our program. It terminates the process and releases all the resources that were used during execution time, including memory and CPU time.

Conclusion

Lexical scope and scope chain are fundamental concepts in JavaScript. Lexical scope determines which variables and functions are accessible within a certain code block.

The scope chain, on the other hand, is an internal structure managed and used by the JavaScript engine to resolve variables and functions during runtime, following the hierarchy of lexical scopes. Understanding these concepts is important for writing efficient and effective JavaScript code.

I tried to give more details about lexical scope and related terminology, but also to dive deeper into scope chain management and explain how it works under the hood. I really hope that this was helpful and you have more knowledge about this topic than before reading this article.

Check out other articles on my blog.

Bulk install VS Code extensions

Ramo Mujagic — Mon, 16 Jan 2023 18:41:45 +0000

If you are reading this, you probably already know what VS Code and VS Code extensions are. Usual way to install extensions is through VS Code Interface. It is really easy and intuitive. However, every extension needs to be installed one by one.

This is great if you need to install one extension at the time, which is usually the case, but what if you need to install 20 or more extensions? It can become a time consuming process.

You might wonder, why would I ever need to install 20 or more extensions at the time? Let's answer this question in the next section.

The reason why

Recently, I needed to setup a new working environment and basically replicate everything that I already have on the old machine on the new one.

For some, this can be fun and interesting. It also gives you an opportunity to add some changes to your setup that you didn't want to do before. But when you do it often or when you are happy with your current setup, it can be really tedious to do it again from scratch.

Anyways, eventually, I needed to setup VS Code to be exactly the same as on the old machine. However, I have noticed that I could not install extensions all at once. Since I have more than 20 extensions that I needed to install, I did not want to do it manually, one by one.

Bulk install script

I decided to make a script that can automate the extensions installation process. For me, it would be good enough to loop over a list of extensions and install them.

I spent some time trying out the code command and possible options. Luckily, it is possible to use the code --list-extensions command to list all installed extensions and also code --install-extension <extension_id> to install the extension.

Lastly, I decided to make it a bit more robust and added some logging so it's easy to see what the script is doing. Source code is available here.

How to use it

First, you need to prepare a .txt file that will be used as an input for the script. This file should contain a list of all VS Code extensions, one per line. Here is a sample of how file content should look like.

...
akamud.vscode-theme-onedark
akamud.vscode-theme-onelight
dbaeumer.vscode-eslint
eamodio.gitlens
esbenp.prettier-vscode
formulahendry.auto-rename-tag
mikestead.dotenv
...

Every extension must be in <publisher>.<name> format. Make sure that the file is correctly formatted and that there are no extra spaces or blank lines in the file.

Alternatively, you can use a code command to extract the list of extensions from your current VS Code installation.

code --list-extensions > extensions-list.txt

When the command is executed, it will generate a list of all installed extensions and save them into the extensions-list.txt file. You can change the file name in the command if you want.

Running the script

Next step would be to run the script and use the previously created file as an input. It can all be done in just one command like this.

curl -fsSL https://raw.githubusercontent.com/rmmgc/vscode-extensions-bulk-install/main/bulk-install.sh | sh -s <path_to_input_file>

For the script to work properly, VS Code needs to be installed and the code command must be exported on the PATH.

When the command is executed, it will get text content of the script from the URL and pipe the output to the sh command. The script file itself will not be saved on your machine. Additionally, the path to the input file is passed as an argument to the script.

Make sure to replace path_to_input_file with a path to the input file that has a list of extensions.

After execution, you should see a detailed log in the terminal. It might look something like this.

...

🔧 Working on dbaeumer.vscode-eslint extension.
✅ Extension already installed.
Skipping further steps.

🔧 Working on formulahendry.auto-rename-tag extension.
Running: code --install-extension formulahendry.auto-rename-tag.
Installing extensions...
Installing extension 'formulahendry.auto-rename-tag'...
(node:93018) [DEP0005] DeprecationWarning: Buffer() is deprecated due to security and usability issues. Please use the Buffer.alloc(), Buffer.allocUnsafe(), or Buffer.from() methods instead.
(Use `Electron --trace-deprecation ...` to show where the warning was created)
Extension 'formulahendry.auto-rename-tag' v0.1.10 was successfully installed.
✅ Extension installed successfully.

🔧 Working on mikestead.dotenv extension.
✅ Extension already installed.
Skipping further steps.

💡 Check the logs above for detailed report.
🎉 Successfully finished.

All missing extensions should be installed. In case some extensions were already installed before running the script, they will be skipped.

Conclusion

It became boring always installing VS Code extensions whenever setting up a new working environment. That's why I decided to create and share a script that can help to speed up this process a little bit.

Check out other articles on my blog.

VS Code extensions I always use

Ramo Mujagic — Mon, 19 Dec 2022 18:37:22 +0000

If you are working with JS, chances are that you are already using VS Code. It is the most popular editor for JS development. Some of the reasons why VS Code is so popular is the fact that it costs nothing - completely open source, easy to use, fully customizable, really fast and a joy to work with.

Since VS Code is so popular there are many extensions created to extend its functionality. From so many extensions available to you it might be hard to choose which ones to use. I decided to make a list of extensions I always use on every JS project no matter the requirements.

This list of extensions is not based on a particular JS framework or library like Vue or React, so no snippet extensions, but rather on extensions that every JS developer might find useful.

To make it easier, I have decided to group extensions into two categories, UI Extensions and Utility Extensions.

UI Extensions

Under this category, I will mention extensions that actually change the appearance of User Interface inside VS Code. They will make your editor stand out and look much nicer than the default you get after installing VS Code, even though the default one looks pretty good to begin with. After all, having a nice working environment is really important for productivity.

Atom One Dark Theme

For me, there is no better dark theme than Atom One Dark Theme. It is just perfect in any way. Nice color scheme, good contrast between colors, nice to look at, it basically ticks all the boxes for me. There is a reason why this extension has more that 3.8M downloads, at the time of writing.

This is of course highly opinionated. There are many other good dark themes that could be a better fit for you. I would say, If you don't like this theme, go ahead and try the others until you find something you like. To know what fits you the best you need to try it first.

Atom One Light Theme

I personally do not use light themes often. But there are some situations that I can find myself in where a light theme is better suited than the dark one.

I work for a company that has large open offices with a lot of natural light. When working in a bright environment, like this, it is much easier to read text/code when using a light theme.

When a light theme is needed, I go for Atom One Light Theme just because of the similarities with Atom One Dark Theme.

To quickly switch between themes, hit Command/Ctrl + K + T to open the Theme picker where you can select the desired theme.

VS Code Icons

When it comes to icons I always go for VS Code Icons. It is very popular extension as well with over 12.5M downloads.

This extension has a large collection of images for basically every file type and even some specially named directories will have a dedicated icon.

To quickly switch between icon themes, hit Command/Ctrl + Shift + P to open the Command Palette. Type "icon theme" and select the first option. Icon Theme picker will show up where you can select the desired icon theme.

Utility Extensions

Let's take a look at non-ui extensions. These extensions should actually help to make you faster and write better code. There are a lot of different kinds of extensions under this category, but for sake of simplicity, I just call them utility extensions.

Auto Rename Tag

This extension is really simple, it only does one job. Auto Rename Tag will automatically rename paired HTML/XML tags. Job it does may seem trivial, but this extension is really a time saver.

DotENV

More often than not, your projects will require .env files where Environment Variables can be provided. This is probably more important to people that are working on Node.js applications, but also modern frontend frameworks might require it.

To bring syntax highlighting for .env files, in VS Code, you can use DotENV extension. After installing extensions, all your .env files should have proper syntax highlighting.

ESLint

This is one of the most popular extensions on the VS Code marketplace and for a good reason. ESLint extension integrates ESLint, the most popular JS linter, into VS Code.

For this extension to properly work, you need to have the eslint npm package installed in your workspace or globally on your machine.

I would recommend always installing eslint into workspace rather than doing it globally. Navigate to your project/workspace and run the following command.

npm install -D eslint

Command will install eslint as a dev dependency. You can fine tune how JS linter integrates with your project by creating a .eslintrc.js configuration file. Find out more information about it on the official website.

Prettier

Another very popular extension is Prettier. It is an opinionated code formatter that enforces consistent code style throughout the whole project.

You might wonder, why would I need a code formatter if I write code in a consistent way?

If you work alone on the project and you don't really care if your code is consistently formatted or not, you can get by without it.

Where prettier really shines is when you make your project open-source, to allow more people to contribute, or when you work in a team.

No one writes code in the same way. You can put ; at the end of the line, but someone else will leave it out. This is where prettier comes into play, to fix those kinds of inconsistencies.

This extension requires the prettier npm package to be installed. Although it can be configured to work with global installation of prettier, it is recommended to use local installation. Navigate to your project/workspace and run the following command.

npm install -D prettier

Command will install prettier as a dev dependency. You can fine tune how prettier integrates with your project by creating a .prettierrc.js configuration file. Find out more information about it on the official website.

GitLens

One of the best extensions that supercharges VS Code with amazing git support is GitLens. Feature list is crazy long and it really helps better understand what is going on especially if working in a team.

However, if you do not need most of the features that GitLens provides, I would recommend using different git related extensions that might be simpler to work with.

npm Intellisense

When working on a JS project, chances are that you will have a lot of third-party packages installed. To make it easy to import all those modules, you can use npm Intellisense.

This extension has one job. It will autocomplete npm modules in import statements. You just need to start typing the name of the module you want to import and extension will provide you with best matching suggestions.

Thunder Client

Sooner or later, you will have to test some API and check which kind of data it returns and how it behaves. Before, the most popular tool for this kind of job was Postman, but now you can do it straight from VS Code itself without installing a dedicated application.

Thunder Client is the REST API client that I always use. It is lightweight, has a simple to use interface and really easy to work with.

Tabnine

Let's finish this list with Tabnine. This is an AI code assistant that, essentially, does code completions.

Personally, in the beginning I was a bit skeptical about AI assistants, but this one turned out to be pretty good. I can genuinely say that it does help save some time with code completions that it provides.

There is also a Pro version and this one is, obviously, not free. The Pro version is much more powerful and has more useful features.

Conclusion

VS Code is an amazing editor with a lot of features, but what makes it special, in my eyes, is the extensions marketplace. There are so many third-party extensions out there created for almost anything.

I have listed my favorite extensions that I always use, on every project. However, based on the project, I usually install additional extensions which are, most of the time, project specific. If you do the same, make sure to remove unused extensions after. It can make VS Code a bit faster.

Check out other articles on my blog.

Understanding equality in JavaScript

Ramo Mujagic — Mon, 28 Nov 2022 18:12:10 +0000

We have all been there… You compare two values using ==, receive a totally unexpected result and wonder WT* is going on?! - Just Equality == operator at its finest.

Equality == operator has the following syntax.

x == y

It takes two operands, x and y, converts them into the same type, compares them by value and returns a Boolean result.

Always use Strict Equality === operator. There is literally no benefit in using == instead of ===, except if your goal is to introduce bugs to your codebase.

But you did not come here so I can tell you to start using ===. Just look at the code below. You want to know how it can possibly be explained, don't you? - And yes, you should know it. After all, you are working with JavaScript and this is a JavaScript feature.

'' == '0'             // Result: false
0 == ''               // Result: true
0 == '0'              // Result: true

false == undefined    // Result: false
false == null         // Result: false
null == undefined     // Result: true

[] == 0               // Result: true
[] == '0'             // Result: false
['a', 'b'] == 'a,b'   // Result: true

NaN == NaN            // Result: false
'\n\r\t' == 0         // Result: true

IsLooselyEqual algorithm

When you use ==, under the hood, this algorithm kicks in and gives the result of comparison. So, yes, here lies the cure for our pain.

This algorithm is described in ECMAScript specification and basically every JavaScript engine has to implement it as described.

It is common to represent algorithms with some kind of graph or flow chart, but since I am very bad at drawing, I present to you this magnificent pseudocode.

Take two operands, `x` and `y`, as an input

If type of `x` is the same as type of `y`
  If type of `x` is `Undefined`
    return `true`
  If type of `x` is `Null`
    return `true`

  If type of `x` is `Number`
    If `x` is `NaN`
      return `false`
    If `y` is `NaN`
      return `false`
    If `x` is the same as `y`
      return `true`
    If `x` is `-0` and `y` is `+0`
      return `true`
    if `x` is `+0` and `y` is `-0`
      return `true`
    return `false`

  If the type of `x` is `String`
    If `x` and `y` are exactly the same sequence of characters
      return `true`
    return `false`

  If the type of `x` is `Boolean`
    If `x` and `y` are both `true`
      return `true`
    If `x` and `y` are both `false`
      return `true`
    return `false`

  If `x` and `y` refer to the same object
    return `true`

  return `false`

If `x` is `null` and `y` is `undefined`
  return `true`
If `x` is `undefined` and `y` is `null`
  return `true`.

If type of `x` is `Number` and type of `y` is `String`
  Convert `y` to `Number`
  Return the result of the comparison `x == y`
If type of `x` is `String` and type of `y` is `Number`
  Convert `x` to `Number`
  Return the result of the comparison `x == y`

If type of `x` is `Boolean`
  Convert `x` to `Number`
  Return the result of the comparison `x == y`
If type of `y` is `Boolean`
  Convert `y` to `Number`
  Return the result of the comparison `x == y`

If type of `x` is either `String` or `Number` and type of `y` is `Object`
  Convert `y` to primitive value
  Return the result of the comparison `x == y`
If type of `x` is `Object` and type of `y` is either `String` or `Number`
  Convert `x` to primitive value
  Return the result of the comparison `x == y`

Return `false`

Keep in mind that some parts of the algorithm are missing in pseudocode representation. If you want to see detailed specifications, check it out on ECMAScript website.

Yes, I know, it's just a bunch of if statements and conversions. This conversion part is really what's important. You see, when types are the same, == behaves exactly the same as the === operator. Important difference is that == enforces coercion when types are different.

Coercion is a fancy JavaScript term used when we talk about converting from one value type to another, like from string to number. Coercion plays an important role in == comparison.

If you don't want to remember the whole algorithm, think about it in this way. If operands do not have the same type, they are converted to Number. When types are the same, operands are compared by value.

Object to primitive

One more important thing, before we focus on some examples, is conversion of the Object type values to primitive values.

In JavaScript, there are 7 primitive values: string, number, bigint, boolean, undefined, null and symbol.

Value with the type of Object can automatically be converted to primitive value by == operator. This conversion will happen when the type of one operand is Object while the other operand has one of primitive types.

When converting Object to primitive value, JavaScript will do a couple of things. It will first try to call the built-in prototype method valueOf. If valueOf does not return a primitive value, the built-in prototype method toString will be called next.

As almost anything in JavaScript this behavior has a couple of edge cases and it can be modified and overridden, but you don't have to worry about it for now.

Examples

Still remember that code sample from the beginning? Let's cover it line by line, now when we know how the algorithm works.

I will make one assumption here. Operand on the left side of == will be called x and the operand on the right side will be called y. With that out of the way, let's get started.

Keep in mind, if operands do not have the same type, == will enforce type conversion as long as operands have different types. Only then will operands be compared by value.

Sample 01

'' == '0'             // Result: false

This one is pretty simple. Both operands have the same type, String. This means that operands will be compared by value. Since values are not the same, the result is false.

Sample 02

0 == ''               // Result: true

Type of operand x is Number and type of operand y is String. Types are not the same and y will be converted to Number.

Empty string '' converted to Number, will be 0. Test it by running Number('') in your browser's console.

When conversion is completed operands will be compared by value. Since values are the same, 0 is indeed equal to 0, result will be true.

Sample 03

0 == '0'              // Result: true

Type of operand x is Number and the type of operand y is String. Operand y will be converted to Number.

When '0' is converted to Number, its value is going to be 0. Test it by running Number('0') in your browser's console.

When conversion is completed operands will be compared by value. Since values are the same, 0 is indeed equal to 0, result will be true.

Sample 04

false == undefined    // Result: false
false == null         // Result: false
null == undefined     // Result: true

I grouped these examples together since the reasoning for this behavior, in all of them, is pretty much the same.

According to the algorithm, undefined can only be equal to undefined or null. Same is for null, it can only be equal to null or undefined. Because of this, comparing them to false will always result in false.

Sample 05

[] == 0               // Result: true

Operand x is the type of Object and the type of operand y is String. As we already know, before they can be compared by value, type conversion is going to happen. In this case, operand x will be converted to a primitive value. This means Object to primitive conversion will kick in.

So, let's see how the conversion of [] to primitive value looks like. Method valueOf is going to be called on [] directly. Test it by running [].valueOf() in your browser's console. The result of this operation will be [].

As you can see, we still have [] which is not a primitive value, so method toString() gets called. Test it by running [].toString() in your browser's console. Returned value is '' which is a primitive value. With this, conversion of Object to primitive value is finally completed.

But, look closely, operands still have different types, x is now '' which is type String and y is still 0 which is type Number. Conversion must continue.

Now, x will be converted to Number. We already know, from previous examples, that '' is going to be 0 after conversion. When conversion is completed operands will be compared by value. Since values are the same, 0 is indeed equal to 0, result will be true.

Sample 06

[] == '0'             // Result: false

Operand x is type of Object, but operand y is type of String. Operand x, with value [], will be converted to primitive value. We already know, from Sample 05, the result of this conversion is going to be empty string ''. After conversion, operand x will have value '' and type String.

Since both operands are now the same type, String, conversion is completed. When compared by value, '' is not the same as '0', so this comparison is false.

Sample 07

['a', 'b'] == 'a,b'   // Result: true

Operand x is type of Object and operand y is type of String. In the conversion step, Object will be converted to primitive value.

After conversion, operand x will have value 'a,b' and type String. Test it by running ['a', 'b'].toString() in your browser's console.

Both operands are now types of String. When compared by value, the result is true, since values are the same.

Sample 08

NaN == NaN            // Result: false

This one is quite strange and yet simple. Comparing any value with NaN is always going to result in false. This is the case even if NaN is compared to NaN, which seems quite weird.

Sample 09

'\n\r\t' == 0         // Result: true

Operands have different types, so conversion is required. Since the type of x is String it will be converted to Number. After conversion, '\n\r\t' will be 0. Now, when compared by values, 0 is the same as 0, so the result is true.

If you are wondering how '\n\r\t', after conversion, can become 0 - it's because of how conversion to Number works.

There are some special Single Character Escape Sequences like \n (new line), \t (horizontal tab), \v (vertical tab), \r (carriage return) that are always going to be 0 when converted to Number. It does not matter in which order and how many of them are contained inside the same string value.

Conclusion

We all know JavaScript can be weird, but if you look a bit under the hood, things can become much clearer. The same thing is with the Equality == operator. On the surface it can be really strange, but when you know the algorithm behind it, things can become much easier to understand.

Either way, I strongly recommend always using the Strict Equality === operator. But don't get me wrong, it is still really important to know how == works. It is, after all, a JavaScript feature which is always going to be part of the language.

Check out other articles on my blog.

Quick introduction to SSH

Ramo Mujagic — Fri, 18 Nov 2022 20:26:57 +0000

First of all, you might ask yourself, what is SSH?
Well, in short, SSH stands for Secure Shell. It allows you to gain, cryptographically secured, command-line (shell) access on a remote machine.

Remote machine can be your web or build server, or any other machine that can be accessed via SSH.

On Linux and macOS, command-line SSH client usually comes preinstalled with the system. As such, it can be used directly in the terminal window. For Windows, SSH client is not usually installed with the system, but there are plenty of SSH clients that can be installed manually. In this article, focus will be on macOS.

Password based authentication

Establishing an SSH connection is pretty straightforward. But, before it can be done, you need to know username and hostname, or IP address, of the server.

So basically, you will use the SSH client, on your machine, to connect with the SSH server on the remote machine. Assuming you are on macOS or Linux, you can run the following command to connect via SSH.

ssh <username>@<hostname>

username - server user.
hostname - server host. If not known, IP address can be provided instead.

If you are connecting with the server for the first time, you will be asked to confirm server identity. If you confirm it, the public key of the server will be saved into your local ~/.ssh/known_hosts file. Next time you try to connect, your known_hosts file will be checked and since the server public key is already added there, this step will be skipped.

As a next step, you will be asked for a password to confirm that you are authorized to access the server. If provided password is not correct, connection will be rejected.

As you might have already noticed, every time you want to establish an SSH connection with the remote machine, you will have to enter a password. This approach is good if you don't access the server frequently, but even so, it is still better to use SSH keys instead.

If you are currently connected via SSH, go ahead and run following command.

logout

It will terminate the currently running SSH session. Now, let's check out what are those SSH keys.

SSH keys basics

SSH protocol uses SSH keys as an access credential. This means that, instead of the password that was used before, you can use SSH keys to connect to the server. Other than that, SSH keys are much more robust, longer and complex than any password could ever be. As such, they are a better and more secure method of authentication.

SSH uses a pair of keys, public key and private key. They are generated using public key cryptographic algorithms and most common of them are RSA and DSA. These algorithms are based on a mathematical formula that takes 2 very large prime numbers and outputs public and private key.

Public key can be shared freely with others, while private key must be saved securely and only the key owner should know its value.

Session key

There is one more important key called session key. This key is negotiated by both parties, client and server, during the connection process. Session key is generated using a version of the Diffie-Hellman algorithm. This algorithm works in a way that both parties contribute equally to the generation of the session key. Because of the nature of the algorithm, the session key represents a shared symmetric key. It means that this key can be used for both encryption and decryption.

Authentication flow

Before a key-based connection can be established, something called SSH handshake is going to happen. SSH cryptographic handshake ensures that both parties, client and server, are authenticated. Below is a sample of communication between client and server during SSH handshake.

Client initializes SSH connection with the server and sends ID of its public key.
Server checks ~/.ssh/authorized_keys file of the user that the client is attempting to log in. If a public key with matching ID is found, the server generates a random number and uses the public key to encrypt the message. Server sends the encrypted message back to the client.
Client decrypts the number using its private key.
Client combines decrypted number and shared session key to generate MD5 hash value.
Client sends this MD5 hash back to the server as an answer to the encrypted number message.
Server uses a shared session key and the original number to calculate MD5 hash value.
Server compares its calculation with the one that the client has sent.
If two values are matched, it proves that the client has the correct private key. Client is authenticated and connection is established.

If the SSH handshake was successful, connection between client and server will be established. This process is secure since it relies on cryptographic authentication to establish connection between client and server.

Generating SSH keys

By now, you might ask yourself, how can I generate those SSH keys?
SSH keys can be generated really quickly using following command.

ssh-keygen -t rsa -b 4096 -C <email_address> -f <file_name>

This command will create SSH keys using your the email address as a label. Optionally, you can pass -f flag where you can specify file name for SSH keys.
If -f flag is not provided, you will be prompted to provide file name or you can press enter to accept default. If you press enter, file name id_rsa will be used as a default.

Next, you will be asked to enter a passphrase. If a passphrase is provided, it will be required when ever key is used. This can be useful if someone gains access to the machine where private key is stored, since they will not be able to use private key without correct passphrase.

When command finishes running, you shold see following files in your ~/.ssh directory.

├── <file_name>
├── <file_name>.pub
└── ...

File with the extension .pub contains the data about public key and it can be shared with anyone else. The other file, without .pub extension, contains the data about private key and it needs to stay secure.

Enable key-based authentication

To enable key-based authentication with a server, you need to tell the server about your public key. To do so, you need to add your public key to ~/.ssh/authorized_keys file on the server.

cat ~/.ssh/<file_name>.pub | pbcopy

This command will copy your public key to the clipboard. Now, coppied public key needs to be added into the ~/.ssh/authorized_keys file on the server. To do so, login to the server using password authentication.

Once you gain access to the server, run following command.

pbpaste >> ~/.ssh/authorized_keys

It will paste your, previously copied, SSH key and append it at the end of the content in ~/.ssh/authorized_keys file.

Execute following command to confirm that your public key is saved.

cat ~/.ssh/authorized_keys

You should see content of authorized_keys file printed in terminal window. Your public key should be printed as a last item.

Now, execute logout command to terminate current SSH session.
Next time you start a new SSH connection with the same server, authentication will be made using SSH keys.

This way of authentication is often used on CI/CD to automate processes and authenticate machines without user interaction.

Conclusion

Today, SSH and SSH keys are incredibly important and used everywhere. From online services to automatically authenticate each other to your Github account to push the code changes.

You don't have to be a Secure Shell expert by any means, but knowing the basics can come a long way. I hope this post has provided you with just that - the basics.

This post was first published on my blog.

React hooks: useMemo

Ramo Mujagic — Fri, 11 Nov 2022 10:54:24 +0000

If you are working with React, you have probably heard about the term useMemo.
In a nutshell, useMemo is a built-in React hook that allows you to memoize output of a function to prevent it from re-running on every component re-render.

So, we could say, the goal of useMemo hook is to provide a simple way to achieve gains in performance using memoization. But, there is more to it, so let's dive deeper.

How it works

This hook has the following signature.

const memoizedValue = useMemo(compute, dependencies);

You can see that useMemo accepts 2 arguments.

compute - function used to compute a result.
dependencies - array of dependencies used by compute function.

On the initial render of the component, useMemo will invoke the compute function, memoize the result of the compute function and return the value.

On every component re-render, userMemo will check if dependencies are changed. If change is detected, useMemo will invoke the compute function again, otherwise it will return previously memoized value.

In programming, memoization is usually implemented in a way that memoization function remembers all inputs and results based on those inputs. Next time, when the same inputs are sent to the memoization function, it will return memoized result.

On the other hand, useMemo only remembers previous inputs and results based on those inputs. As soon as next inputs are not the same as previous inputs, useMemo will run the computation again.

When should it be used

Thing is, useMemo does not give free performance gains. Running useMemo will consume CPU time and memory which can hurt performance if used incorrectly.

For this reason, useMemo should not be used everywhere whenever possible, but rather in some situations where memoization would provide performance benefits.

Overall, make sure that your component does not depend on useMemo at all. Only if you notice performance issues, introduce useMemo. Here are some questions you can ask yourself to decide if useMemo should be used.

Is the component running expensive calculations?
Is the component re-rendering often?
Is the component doing data transformation?

There is no guideline which will tell you when exactly to use this hook. You need to make sure that you understand the problem you are having and then, you will know if useMemo can help you.

Show me some code

It is true that usage of useMemo hook may vary and depend on your component, but it is still possible to provide some examples where useMemo would make sense and where not.

Bad case

As the first example, let's say we want to calculate user age based on the year of birth and the current year. While at it, you might want to add useMemo there, you know, just in case.

const userAge = useMemo(
  () => currentYear - yearOfBirth,
  [currentYear, yearOfBirth]
);

Well, there you go, this is a bad usage of the useMemo hook. Computers are fast and doing simple calculations like this will take basically no time. Just do the following instead.

const userAge = currentYear - yearOfBirth;

When doing calculations on primitive values like numbers, strings, booleans, never wrap them with useMemo. It can take longer to execute useMemo than the calculation itself.

Good case

In this example, there is a function that takes an array of users and formats the activities for every user. Additionally, alongside activities, we want to return users' full name and age.

const activitiesPerUser = useMemo(() => {
  return users.map((user) => {
    return {
      fullName: `${user.firstName} ${user.lastName}`,
      age: user.age,
      activities: formatUserActivities(user),
    };
  });
}, [users]);

This function might take a lot of time if formatting of activities is complicated or if the users array is large. In this case, it makes sense to wrap the function in the useMemo hook, especially if the component gets re-rendered multiple times with the same users object.

Memoize results of functions that are doing expensive calculations and data transformations. Even if component is not being re-rendered at all, it is still good to wrap those functions with useMemo.

Conclusion

Oh, congrats, you made it to the end. I hope that the useMemo hook is not as complicated anymore.

From my experience, it is always worth memoizing computed non-primitive values, like arrays or objects, that are passed down to the child components. It is even more important if the parent component hosts state variables that are frequently changed. This can prevent unnecessary re-renders of child components.

In general, do not think about performance optimization too much. If your application is not performant, don't worry, you will notice it.

This post was first published on my blog.

Configuring macOS command line

Ramo Mujagic — Mon, 07 Nov 2022 12:57:44 +0000

If you are serious about doing any kind of development work, it is really important to have an easy to use command line interface. Out of the box, macOS comes with the Terminal app. We will start from there and try to improve overall experience.

Before taking a deeper dive, take a look at the image below to get a glimpse of what is going to be covered here.

There will be plenty of things that can be changed and you can end up with completely different interface, if you choose so.

Shell setup

Before going any further, you need to choose which shell to use. There are many out there and macOS usually comes with two of them pre-installed, bash and zsh.

My choice here is zsh. It is highly customisable and comes with many powerful features. Even if you currently use bash, switching to zsh is quite simple. More so, zsh is an extended version of bash. They basically share a lot of common functionality and features.

Install Zsh

Majority of macOS versions already come with zsh pre-installed. To check if zsh is installed on your system, run the following command.

zsh --version

If you have zsh installed, it's version should be printed out. In this case, you do not need to do any additional changes.

However, if zsh is not installed, you can easily install it using homebrew. Homebrew is a macOS package manager that makes it easy to install missing software. It usually comes pre-installed on many macOS versions. To install zsh using homebrew, run the following command.

brew install zsh

You can confirm if it's installed by running the zsh --version command again.

Zsh as a default shell

Depending on the macOS version, your default shell might be set to bash. You can easily check it by running the following command.

echo $0

If you see that the output is zsh, then your default shell is already set properly. In case you need to set zsh as a default shell, do so by running the following command.

chsh -s $(which zsh)

Finally, start new shell session (close the current and open a new Terminal window), for the changes to take effect.

Migrate from bash

If you are coming from bash, you might have some custom configuration saved in your .bash_profile or .bashrc files. Since those files are bash specific, they will not be used by zsh and all configuration saved in them must be migrated into zsh specific files.

.zshenv is sourced on all shell invocations. Here you should save all exports that are needed for other programs.
.zshrc is sourced when running in interactive shells. Here you can put all configs that make the shell experience more enjoyable, like: aliases, functions, options, etc.

Make sure that all of your $PATH exports are saved in the .zshenv file, since it will be sourced on every shell invocation.

Oh My Zsh framework

When you switch to zsh, installing Oh My Zsh is one single thing that you must do. This will make your shell really awesome!

Oh My Zsh is a community driven framework for managing zsh configuration. Installing it could not be simpler, just run the following command.

sh -c "$(curl -fsSL https://raw.github.com/ohmyzsh/ohmyzsh/master/tools/install.sh)"

Oh My Zsh install command assumes that you already have git installed on your machine. In case you don't, follow this guide to install it.

Oh My Zsh will save it's configuration into the ~/.zshrc file. In case you already had .zshrc created before, it's content will be copied and saved in ~/.zshrc.pre-oh-my-zsh file.

You can move your previous configuration back to the ~/.zshrc by copying it from ~/.zshrc.pre-oh-my-zsh.

It is usually a good idea to create ~/env.sh where you can keep your custom configuration. Then in ~/.zshrc you can use the source command to load it.

...

# Load evn.sh file
source ~/env.sh

This ensures that your custom configuration will always be sourced when running shell in interactive mode. When you need to add or change custom configuration, just do it in the env.sh file.

Themes

Your shell might be powerfull now, but it is not good enough if it looks ugly. Luckily, setting a theme with Oh My Zsh is a quite simple.

Open ~/.zshrc file in your favorite editor, vim of course. In there, you will find a variable with the name ZSH_THEME. By default, it will be set to robbyrussel. To change the theme, just provide a different theme name.

There are tons of already pre-made themes available to pick from.

Personally, I use the arrow theme since it provides simple, minimal and clean interface. To enable the arrow theme just set ZSH_THEME="arrow" in your ~/.zshrc file. Make sure to restart your shell session after.

...

# Set name of the theme to load --- if set to "random", it will
# load a random theme each time oh-my-zsh is loaded, in which case,
# to know which specific one was loaded, run: echo $RANDOM_THEME
# See https://github.com/ohmyzsh/ohmyzsh/wiki/Themes
ZSH_THEME="arrow"

...

You can also use external themes which are created by the community. Using external themes requires more work since they need to be added to the custom theme directory. You can follow this guide for more information about this topic.

Plugins

Great thing about Oh My Zsh is that it comes bundled with all kinds of plugins. This allows you to easily extend the functionality of your shell, just by enabling them. Plugins can be easily enabled in the ~/.zshrc file by changing the plugins variable.

One such bundled plugin is called web-search. It allows you to query popular services, like google, from your terminal window. To enable this plugin, just add web-search to the plugins variable.

...

# Which plugins would you like to load?
# Standard plugins can be found in $ZSH/plugins/
# Custom plugins may be added to $ZSH_CUSTOM/plugins/
# Example format: plugins=(rails git textmate ruby lighthouse)
# Add wisely, as too many plugins slow down shell startup.
plugins=(web-search)

...

Keep in mind that plugins variable is actually an array. If you have multiple values inside, they must be separated with empty space.

Now, let's switch to third party plugins, since there are so many good ones. To use third party plugins you need to install them into the $ZSH_CUSTOM/plugins directory. After that, the only thing that needs to be done is to add the plugin, by it' name, into the plugins variable.

I would recommend installing following plugins since they can increase your shell productivity significantly.

zsh-autosuggestions - suggests commands as you type based on history and completions.
zsh-syntax-highlighting - highlights commands while you type them in the interactive terminal.

Install zsh-autosuggestions running following command.

git clone https://github.com/zsh-users/zsh-autosuggestions ${ZSH_CUSTOM:-~/.oh-my-zsh/custom}/plugins/zsh-autosuggestions

Install zsh-syntax-highlighting running following command.

git clone https://github.com/zsh-users/zsh-syntax-highlighting.git ${ZSH_CUSTOM:-~/.oh-my-zsh/custom}/plugins/zsh-syntax-highlighting

When plugins are installed, enable them via plugins variable and restart your shell for effects to take place.

...

# Which plugins would you like to load?
# Standard plugins can be found in $ZSH/plugins/
# Custom plugins may be added to $ZSH_CUSTOM/plugins/
# Example format: plugins=(rails git textmate ruby lighthouse)
# Add wisely, as too many plugins slow down shell startup.
plugins=(web-search zsh-autosuggestions zsh-syntax-highlighting)

...

Plugins can provide a lot of value and extend your shell in so many ways, but there is also one downside. Every plugin you add will impact shell performance and startup time. So, choose plugins wisely and keep only the ones you are actually using.

Terminal app

This section is all about configuring, default, Terminal app that come preinstalled on every macOS.

You might wonder, why not use more feature rich apps like iTerm2?
The reason for me is really simple, I just do not use much of the provided features and the benefits for me are not that big.

With that out of the way, let's configure the Terminal app.

Install Fira Code font

I have tried a lot of fonts in the terminal environment, but this one just feels the best. It is a completely free, monospaced font, containing ligatures for common programming multi-character combinations.

Fastest way to install the font is to go to the releases page of the Fira Code repository and download the latest version. Once downloaded, you can just install all fonts provided in the ttf directory. With that done, Fira Code is installed onto your system.

Custom profile

In the Terminal app, profiles can define custom configuration and behavior. To modify customization to our liking, a custom profile needs to be created.

It is totally fine to customize it manually, but there are also a lot of third party themes that can be used in the Terminal app. One example is the Dracula theme. Usually, third party themes will be provided via .terminal file which can be imported in the Terminal app.

I personally use the Dracula theme myself. You can download .zip from official website. It should come with the Dracula.terminal file.

To install it, open the Terminal app and go to Preferences > Profiles. From there, click on the More icon and a dropdown will be shown where you need to select Import. When the theme is imported, it will be saved as a new profile with the name Dracula.

Make sure that Dracula profile is selected. On the bottom, you should see the Default button. Click on the button and it will set, currently selected, profile as default. With this, when the new Terminal window is open, it will automatically use selected profile.

On the right side, you can see all the settings that can be changed. Since we use the Dracula theme, most of the settings does not need to be changed, but I like to tweak some of the options to make it more nice.

Let's start with Background settings. Click on the Background icon and a new window should open. Here you can configure Terminal background settings. I like to keep it simple, so for me, settings are set to following.

Background color   black (#000000)
Opacity            90%
Blur               5%

This will give you a Terminal window that is slightly transparent with a blurred background. I like it since it looks clean and opacity is high enough so underlying windows are not visible that much.

Next, the Font needs to be changed. Click on the Change button under the Font section and the new window should open. Select Fira Code as your font and adjust settings to your linking.

You can also tweak other settings as well, but I do not change them since they are defined by the Dracula theme anyways.

Conclusion

Some people find working with terminal and command line tools very hard and often are trying to avoid it completely. But the truth is, it is inevitable.

When things are like that, why don't we make sure that the command line environment looks nice and actually help us to increase our productivity?

By choosing the right terminal app, changing shell and tweaking theme configuration it is possible to enhance command line experience drastically. I really hope that this post has given you a glimpse into what is possible and, at the end, you are able to customize your command line environment to your liking.