DEV Community: Ferenc Almasi

Learn How to Make Colorful Fireworks in JavaScript

Ferenc Almasi — Thu, 30 Dec 2021 16:55:03 +0000

New Year is around the corner and soon, fireworks will fill the sky. As the last tutorial for this year, I've decided to try to replicate fireworks in JavaScript.

In this tutorial - inspired by Haiqing Wang from Codepen - we will take a look at not only firing colorful fireworks with mouse clicks but also on

How to create and manage different layers
How to load and draw images
How to rotate objects around a custom anchor point
How to generate particles affected by gravity

If you would like to skip to any of the parts in this tutorial, you can do so by using the table of contents below. The project is also hosted on GitHub.

The final version of this tutorial

Setting Up the Project

Let's start by setting up the structure of the project. As always, start with an index.html with two canvas and two script elements:

<!DOCTYPE html>
<html lang="en">
    <head>
        <meta charset="UTF-8" />
        <meta name="viewport" content="width=device-width, initial-scale=1.0" />
        <title>✨ Fireworks in JavaScript</title>
        <link rel="stylesheet" href="styles.css" />
    </head>
    <body>
        <canvas id="background"></canvas>
        <canvas id="firework"></canvas>

        <script src="background.js"></script>
        <script src="firework.js"></script>
    </body>
</html>

This is because we will have two separate layers; one for the background where we draw the static assets, and one for the actual fireworks and interactive elements. At this stage, both script files are currently empty. I also referenced a styles.css, that will only have two rules:

body {
    margin: 0;
}

canvas {
    cursor: pointer;
    position: absolute;
}

We will make the canvas take the whole screen, so make sure you reset the margin on the body. It's also important to set canvas elements to absolute positioning, as we want to overlay them on top of each other.

Lastly, I have two images in an assets folder, one for the wand, and one for the wizard. You can download them from the GitHub repository. With this in mind, this is how the project structure looks like:

Drawing the Background

To get some things on the screen, let's start by adding the background first. Open up your background.js file, and set the canvas to take up the whole document with the following:

(() => {
    const canvas = document.getElementById('background');
    const context = canvas.getContext('2d');

    const width = window.innerWidth;
    const height = window.innerHeight;

    // Set canvas to fullscreen
    canvas.width = width;
    canvas.height = height;
})();

I've put the whole file into an IIFE to avoid name collisions and polluting the global scope. While here, also get the rendering context for the canvas with getContext('2d'). To create a gradient background, add the following function:

const drawBackground = () => {
    // starts from x, y to x1, y1
    const background = context.createLinearGradient(0, 0, 0, height);
    background.addColorStop(0, '#000B27');
    background.addColorStop(1, '#6C2484');

    context.fillStyle = background;
    context.fillRect(0, 0, width, height);
};

This will create a nice gradient from top to bottom. The createLinearGradient method takes in the starting and end positions for the gradient. This means you can create a gradient in any direction.

Values are x1, y1, x2, y2 in order

You can also add as many colors with the addColorStop method as you want. Keep in mind, your offset (the first param) needs to be a number between 0 and 1, where 0 is the start and 1 is the end of the gradient. For example, to add a color stop at the middle at 50%, you would need to set the offset to 0.5.

To draw the foreground - represented by a blue line at the bottom - extend the file with the following function:

const drawForeground = () => {
    context.fillStyle = '#0C1D2D';
    context.fillRect(0, height * .95, width, height);

    context.fillStyle = '#182746';
    context.fillRect(0, height * .955, width, height);
};

This will create a platform on the last 5% of the canvas (height * 95%). At this stage, you should have the following on the screen:

Drawing the wizard

To add the wizard to the scene, we need to load in the proper image from the assets folder. To do that, add the below function to background.js:

const drawWizard = () => {
    const image = new Image();
    image.src = './assets/wizard.png';

    image.onload = function () {
        /**
         * this - references the image object
         * draw at 90% of the width of the canvas - the width of the image
         * draw at 95% of the height of the canvas - the height of the image 
         */
        context.drawImage(this, (width * .9) - this.width, (height * .95) - this.height);
    };
};

You need to construct a new Image object, set the source to the image you want to use, and wait for its load event before you draw it on the canvas. Inside the onload event, this references the Image object. This is what you want to draw onto the canvas. The x and y coordinates for the image are decided based on the width and height of the canvas, as well as the dimensions of the image.

Drawing stars

The last thing to draw to the background are the stars. To make them more easily configurable, I've added a new variable at the top of the file, as well as a helper function for generating random numbers between two values:

const numberOfStars = 50;
const random = (min, max) => Math.random() * (max - min) + min;

And to actually draw them, add the following function to the end of your file:

const drawStars = () => {
    let starCount = numberOfStars;

    context.fillStyle = '#FFF';

    while (starCount--) {
        const x = random(25, width - 50);
        const y = random(25, height * .5);
        const size = random(1, 5);

        context.fillRect(x, y, size, size);
    }
};

This will create 50 stars at random positions, with random sizes, but not below the half of the screen. I've also added a 25px padding to avoid getting stars drawn to the edge of the screen.

The area that can be covered by stars

Note that I'm using a while loop. Although this is a small application, drawing to the screen, especially animating things is a computation heavy process. Because of this, I've chosen to use - at the writing of this article - the fastest loop in JavaScript. While this can be considered premature optimization, if you are writing a complete game or a computation heavy application, you want to minimize the amount of used resources.

Adding the Wand

The next step is to add the wand. Open your firework.js and add a couple of variables here as well:

(() => {
    const canvas = document.getElementById('firework');
    const context = canvas.getContext('2d');

    const width = window.innerWidth;
    const height = window.innerHeight;

    const positions = {
        mouseX: 0,
        mouseY: 0,
        wandX: 0,
        wandY: 0
    };

    const image = new Image();

    canvas.width = width;
    canvas.height = height;

    image.src = './assets/wand.png';
    image.onload = () => {
        attachEventListeners();
        loop();
    }
})();

Once again, you want to give the same height and width for this canvas element as for the background. A better way than this would be to have a separate file or function that handles setting up all canvases. That way, you won't have code duplication.

This time, I've also added a positions object that will hold the x and y coordinates both for the mouse as well as for the wand. This is where you also want to create a new Image object. Once the image is loaded, you want to attach the event listeners as well as call a loop function for animating the wand. For the event listener, you want to listen to the mousemove event and set the mouse positions to the correct coordinates.

const attachEventListeners = () => {
    canvas.addEventListener('mousemove', e => {
        positions.mouseX = e.pageX;
        positions.mouseY = e.pageY;
    });
};

As we will have event listeners for the fireworks, we need to add both the wand and the fireworks to the same layer. For the loop function, right now, only add these two lines:

const loop = () => {
    requestAnimationFrame(loop);
    drawWand();
};

This will call the loop function indefinitely and redraw the screen every frame. And where should you put your requestAnimationFrame call? Should it be the first the or the last thing you call?

If you put requestAnimationFrame at the top, it will run even if there's an error in the function.
If you put requestAnimationFrame at the bottom, you can do conditionals to pause the animations.

Either way, the function is asynchronous so it doesn't make much difference. So let's see what's inside the drawWand function:

const drawWand = () => {
    positions.wandX = (width * .91) - image.width;
    positions.wandY = (height * .93) - image.height;

    const rotationInRadians = Math.atan2(positions.mouseY - positions.wandY, positions.mouseX - positions.wandX) - Math.PI;
    const rotationInDegrees = (rotationInRadians * 180 / Math.PI) + 360;

    context.clearRect(0, 0, width, height);

    context.save(); // Save context to remove transformation afterwards
    context.translate(positions.wandX, positions.wandY);

    if (rotationInDegrees > 0 && rotationInDegrees < 90) {
        context.rotate(rotationInDegrees * Math.PI / 180); // Need to convert back to radians
    } else if (rotationInDegrees > 90 && rotationInDegrees < 275) {
        context.rotate(90 * Math.PI / 180); // Cap rotation at 90° if it the cursor goes beyond 90°
    }

    context.drawImage(image, -image.width, -image.height / 2); // Need to position anchor to right-middle part of the image

    // You can draw a stroke around the context to see where the edges are
    // context.strokeRect(0, 0, width, height);
    context.restore();
};

This function might look a little complicated at first, so let's break it down. First, we need to get the position for the wand on the canvas. This will position the wand at 91% / 93%, next to the hand of the wizard.

Based on this position, we want to calculate the amount of rotation between the pointer of the cursor, and the position of the wand. This can be done with Math.atan2 at line:5. To convert this into degrees, you want to use the following equation:

degrees = radians * 180 / Math.PI

Note that since the context is flipped, you need to add +360 to the value to get positive numbers. They are easier to read and work with, but otherwise, you could leave this out and replace the values used in this function with their negative counterparts.

You also want to save the context to later restore it at the end of the function. This is needed, otherwise the translate and rotate calls would add up. After saving the context, you can translate it to the position of the wand.

Translating the context to the position of the wand

Next, you want to rotate the image to make it always point at the cursor. Note that you need to convert degrees back to radians, as rotate also expects radians. The if statements are used for preventing the wand to be fully rotated around its axes.

The wand follows the mouse as long as the rotation of the wand is between 0–90°

Lastly, you can draw the image. As the last step, you need to minus the width and half of the height to put the anchor point at the right-middle part of the image.

Stroke was drawn around the wand to help visualize anchor points

Shooting Fireworks

Now we want to finally shoot some fireworks. To help keep things more configurable, I've set up some variables and helper functions at the top of the file:

const fireworks = [];
const particles = [];
const numberOfParticles = 50; // keep in mind performance degrades with higher number of particles

const random = (min, max) => Math.random() * (max - min) + min;

const getDistance = (x1, y1, x2, y2) => {
    const xDistance = x1 - x2;
    const yDistance = y1 - y2;

    return Math.sqrt(Math.pow(xDistance, 2) + Math.pow(yDistance, 2));
};

let mouseClicked = false;

We have two arrays for holding each firework, and eventually, the particles associated with them. I've also added a variable for the number of particles, so it's easier to tweak them. Keep in mind that performance will degrade fast if you increase the number of particles to high values. I've also added a flag for keeping track of whether the mouse is clicked. And lastly, we also have a function for calculating the distance between two points. For that, you can use the Pythagorean theorem:

d = √x² + y², where x = x1 - x2, and y = y1 - y2

To track mouse click events, add the following two event listeners to the attachEventListeners function:

const attachEventListeners = () => {
    canvas.addEventListener('mousemove', e => {
        positions.mouseX = e.pageX;
        positions.mouseY = e.pageY;
    });

    canvas.addEventListener('mousedown', () => mouseClicked = true);
    canvas.addEventListener('mouseup', () => mouseClicked = false);
};

We will use this variable to decide when to draw a firework. To create new fireworks, we will use a function with an init function inside it:

function Firework() {
    const init = () => {
        // Construct the firework object
    };

    init();
}

This is where we will initialize the default values of each firework object, such as its coordinates, target coordinates, or color.

const init = () => {
    let fireworkLength = 10;

    // Current coordinates
    this.x = positions.wandX;
    this.y = positions.wandY;

    // Target coordinates
    this.tx = positions.mouseX;
    this.ty = positions.mouseY;

    // distance from starting point to target
    this.distanceToTarget = getDistance(positions.wandX, positions.wandY, this.tx, this.ty);
    this.distanceTraveled = 0;

    this.coordinates = [];
    this.angle = Math.atan2(this.ty - positions.wandY, this.tx - positions.wandX);
    this.speed = 20;
    this.friction = .99; // Decelerate speed by 1% every frame
    this.hue = random(0, 360); // A random hue given for the trail

    while (fireworkLength--) {
        this.coordinates.push([this.x, this.y]);
    }
};

First, you have the length of the firework. The higher this value is, the longer the tail will be. The x, y, and tx, ty values will hold the initial and target coordinates. Initially, they will always equal to the position of the wand, and the position where the click occurred. Based on these values, we can use the getDistance function we defined earlier to get the distance between the two points, and we will also need a property to keep track of the traveled distance.

And a couple more things; we need to keep track of the coordinates, its angle and speed to calculate velocities, and a random color defined as hue.

Drawing fireworks

To draw each firework based on the defined values, add a new method to the Firework function called draw:

this.draw = index => {
    context.beginPath();
    context.moveTo(this.coordinates[this.coordinates.length - 1][0],
                   this.coordinates[this.coordinates.length - 1][1]);
    context.lineTo(this.x, this.y);
    context.strokeStyle = `hsl(${this.hue}, 100%, 50%)`;
    context.stroke();

    this.animate(index);
};

// Animating the firework
this.animate = index => { ... }

This will take the index from the fireworks array and pass it down to the animate method. To draw the trails, you want to draw a line from the very last coordinates from the coordinates array, to the current x and y positions. For the color, we can use HSL notation, where we give it a random hue, 100% saturation, and 50% brightness.

Animating fireworks

This alone, won't do much, you also have to animate them. Inside your animate method, add the following:

this.animate = index => {
    this.coordinates.pop();
    this.coordinates.unshift([this.x, this.y]);

    this.speed *= this.friction;

    let vx = Math.cos(this.angle) * this.speed;
    let vy = Math.sin(this.angle) * this.speed;

    this.distanceTraveled = getDistance(positions.wandX, positions.wandY, this.x + vx, this.y + vy);

    if(this.distanceTraveled >= this.distanceToTarget) {
        let i = numberOfParticles;

        while(i--) {
            particles.push(new Particle(this.tx, this.ty));
        }

        fireworks.splice(index, 1);
    } else {
        this.x += vx;
        this.y += vy;
    }
};

In order, this method will get rid of the last item from the coordinates, and creates a new entry at the beginning of the array. By reassigning the speed to friction, it will also slow down the firework (by 1% each frame) as it reaches near its destination.

You also want to get the velocity for both axis based on:

x = cos(angle) * velocity
y = sin(angle) * velocity

These values are used for updating the x and y coordinates of the firework, as long as it didn't reach its final destination. If it did reach - which we can verify, by getting the distance between the wand and its current positions, including the velocities and checking it against the target distance - we want to create as many particles as we have defined at the beginning of the file. Don't forget to remove the firework from the array once it's exploded.

As a very last step, to create these new fireworks, add the following to your loop:

if (mouseClicked) {
    fireworks.push(new Firework());
}

let fireworkIndex = fireworks.length;
while(fireworkIndex--) {
    fireworks[fireworkIndex].draw(fireworkIndex);
}

This will initiate a new Firework, every time the mouse is clicked. As long as the array is not empty, it will draw, and animate them.

Shooting fireworks with random colors, and without particles

Adding Particles

The last thing to add is the particles, once the trail reaches the destination. Just as for the fireworks, create a new function with an init called Particle.

function Particle(x, y) {
    const init = () => { ... };

    init();
}

This will take an x and y coordinates as parameters. For the init, we will have roughly the same properties as for fireworks.

const init = () => {
    let particleLength = 7;

    this.x = x;
    this.y = y;

    this.coordinates = [];

    this.angle = random(0, Math.PI * 2);
    this.speed = random(1, 10);

    this.friction = 0.95;
    this.gravity = 2;

    this.hue = random(0, 360);
    this.alpha = 1;
    this.decay = random(.015, .03);

    while(this.coordinateCount--) {
        this.coordinates.push([this.x, this.y]);
    }
};

First, you can define the length of the particles, create the x and y coordinates and assign a random angle and speed to each individual particle. random(0, Math.PI * 2) will generate a random radian, with every possible direction.

friction and gravity will slow down particles and makes sure they fall downwards. For colors, we can define a random hue, and this time, an alpha for transparency, and a decay value, which is used to tell how fast each particle should fade out.

Drawing the particles

For the draw method, add the following lines:

this.draw = index => {
    context.beginPath();
    context.moveTo(this.coordinates[this.coordinates.length - 1][0],
                   this.coordinates[this.coordinates.length - 1][1]);
    context.lineTo(this.x, this.y);

    context.strokeStyle = `hsla(${this.hue}, 100%, 50%, ${this.alpha})`;
    context.stroke();

    this.animate(index);
}

The same logic applies here, what is used for the trail of the firework. Only this time, the strokeStyle also contains an alpha value to fade out the particles over time.

Animating the particles

For the animate method, you want a similar logic to fireworks. Only this time, you don't need to worry about distances.

this.animate = index => {
    this.coordinates.pop();
    this.coordinates.unshift([this.x, this.y]);

    this.speed *= this.friction;

    this.x += Math.cos(this.angle) * this.speed;
    this.y += Math.sin(this.angle) * this.speed + this.gravity;

    this.alpha -= this.decay;

    if (this.alpha <= this.decay) {
        particles.splice(index, 1);
    }
}

Again, start by getting rid of the last item in the coordinates and adding a new one to the beginning of the array with unshift. Then reassign speed to slow each particle down over time, and don't forget to also apply velocities for the x and y coordinates. Lastly, the alpha value can be decreased each frame until the particle is not visible anymore. Once it's invisible, it can be removed from the array. And to actually draw them, don't forget to add the same while loop to the loop function you have for the fireworks:

let particleIndex = particles.length;
while (particleIndex--) {
    particles[particleIndex].draw(particleIndex);
}

Summary

And you've just created your very first firework effects in JavaScript! As mentioned, the project is hosted on GitHub, so you can clone it in one piece and play with it.

Do you have anything else to add to this tutorial? Let us know in the comments below! Thank you for reading through, this was the last tutorial for this year, but more to come next year. Happy coding and happy holidays! 🎉🎅🎄❄️

Writing Your Very First Neural Network in JavaScript

Ferenc Almasi — Thu, 23 Dec 2021 16:02:14 +0000

In the last couple of years, we’ve seen a number of machine learning libraries popping up in the JavaScript community, allowing front-end developers to bring AI to the web and solve complex problems with easy-to-use APIs.

TensorFlow is one of the most popular open-source platform for machine learning, mainly used with Python. In 2018 Google announced the first version of TensorFlow for JavaScript, called TensorFlow.js. This is what we are going to explore in this tutorial.

We are going to build an image classifier. At the end of the tutorial, our classifier will be able to distinguish images between Santa and an everyday old man. But to get started, we first need to understand what are neural networks.

How Neural Networks Work

Before diving into coding, there are a couple of concepts we need to understand. If you are already familiar with neural networks, then you can skip this part and jump straight into coding. Otherwise, keep on reading as I’m going to refer to things mentioned here throughout the tutorial. Let’s see first what are neural networks?

Neural networks

There are a lot of technicalities involved, but to give you a brief overview, neural networks are machine learning algorithms inspired by biological neural networks found in our brains. They are used for pattern recognition and consist of nodes, called neurons connected together by synaptic weights.

Model

When these machine learning algorithms are trained with data, we get a model as an output. It is a representation of the training process. Models can be used for future predictions.

Features and labels

The data that you feed into the model usually consists of features and labels. Features are attributes associated with each sample in your data set. A classical example is the classification of Iris flowers based on features such as the width of sepals and petals.

Labels represent how would you classify each sample. Staying with the example, based on the features you categorize the entry into one of the free species. It either gets a label of “Iris setosa”, “Iris virginica” or “Iris versicolor”.

Now that we have everything out of the way, let’s see what we will have in the end.

The Final Output of This Tutorial

At the end of the tutorial, we will have an app that you can train on two different sets of images: Santa and the elderly. Once you have enough data, the app will be able to predict what the image is about, without ever seeing it.

I will make the whole project available on CodeSandbox so you can tweak around with it. You’ll find a link to it at the end of the tutorial. And a nice gif about how our app will work:

You can drag images onto a canvas element and train the classifier by clicking on one of the buttons below. Once you have sufficient data, you can drag a new image in and predict what’s on it.

Pulling in Dependencies

Our very first step is to pull in everything we need. We’re going to use TensorFlow.js and a popular pre-trained model called MobileNet for image classification. We will then use a technique called transfer learning, where we will extend the pre-trained model with our own custom training set. For that, we are going to need a classifier. We will be using the K-nearest neighbor module. It will allow us to categorize images and once we use prediction, it will choose the category most suitable for the image.

This means we will have 3 dependencies: TensorFlow, MobileNet and the KNN classifier module. Here is the whole document we’re going to work with:

<!DOCTYPE html>
<html lang="en">
    <head>
        <meta charset="UTF-8" />
        <meta name="viewport" content="width=device-width, initial-scale=1.0" />
        <title>Image classification with Tensorflow.js</title>
        <script src="https://unpkg.com/@tensorflow/tfjs"></script>
        <script src="https://unpkg.com/@tensorflow-models/mobilenet"></script>
        <script src="https://unpkg.com/@tensorflow-models/knn-classifier"></script>
        <link rel="stylesheet" href="styles.css" />
    </head>

    <body>
        <main class="app">
            <span class="loading">Loading Model...</span>

            <canvas width="224" height="224"></canvas>

            <div class="actions">
                <span class="info">Train classifier with examples</span>
                <button class="santa">Santa</button>
                <button class="elderly">Elderly</button>
            </div>

            <button class="predict">Predict</button>
        </main>

        <script src="classifier.js"></script>
        <script src="ui.js"></script>
    </body>
</html>

Before the closing of the body, we will have two different scripts. One for the classifier and one for dealing with UI events.

You may also notice that the canvas element has a really specific width and height. This is because the pre-trained model has been trained with 224x224px images. To match the format of the data, we have to use the same size. You can still work with larger images, you’ll just have to resize the data before feeding it into the classifier.

Before starting to work on the classifier, let’s quickly put the UI together.

Creating the UI

Create a new file called ui.js. We’re going to make the canvas accept images by drag-and-drop. First, get the canvas and its context and attach some event listeners to it:

const canvas = document.querySelector('canvas');
const context = canvas.getContext('2d');

canvas.addEventListener('dragover', e => e.preventDefault(), false);
canvas.addEventListener('drop', onDrop, false);

We need to handle two events: dragover and drop.onDrop will draw the image onto the canvas, while dragover only prevents the default action from happening. This is needed because without it, the image would be opened on the same tab.

Without preventing the default action, the page is reloaded

Without preventing the default action, the page is reloaded
Let’s see what’s inside the onDrop function.

const onDrop = (e) => {
    e.preventDefault();

    const file = e.dataTransfer.files[0];
    const reader = new FileReader();

    reader.onload = file => {
        const img = new Image;

        img.onload = () => {
            context.drawImage(img, 0, 0, 224, 224);
        }

        img.src = file.target.result;
    }

    reader.readAsDataURL(file);
}

The very first thing is to prevent the default action just like we did for dragover. Next, we want to get the data from the file we dropped onto the canvas. We can get it using e.dataTransfer.files[0]. Then we want to create a new FileReader object and read the file as a data URL. We also want to create a function for its onload event. When the reader finished reading the file, we create a new Image object and set its source to file.target.result. This will hold a base64 encoded image data. And once the image has been created, we draw it on the canvas. Remember, we need to scale it to 224x244.

While also here, let’s quickly add the event listeners to the buttons, and then we can start creating the classifier. We have 3 buttons and 3 event listeners:

const santaButton = document.querySelector('.santa');
const elderlyButton = document.querySelector('.elderly');

santaButton.addEventListener('click', () => addExample('santa'));
elderlyButton.addEventListener('click', () => addExample('elderly'));

document.querySelector('.predict').addEventListener('click', predict);

We will have an addExample function that will take in one parameter: a label for the category. And we will have a predict function.

Building the Classifier

We will create both functions inside a new file called classifier.js. But first, we need to create a classifier and load in MobileNet. To do that, add the following function to classifier.js and call it:

let classifier;
let net;

const loadClassifier = async () => {
    classifier = knnClassifier.create();
    net = await mobilenet.load();

    document.querySelector('.loading').classList.add('hidden');
}

loadClassifier();

Both classifiers and the model will be used later on. So I created two variables for them outside the function. It takes some time for the model to load in and it does it asynchronously, hence we use async / await. Once it’s loaded, we can hide the loading message.

To extend the model with our own custom data set, we have to add our own examples. To do that, anytime the user clicks on “Santa” or “Elderly”, we call the addExample function.

Adding examples

The function will do the following:

Get the image data from the canvas
use MobileNet to get the features of the retrieved data
Train the classifier by adding an example using the features and an associated label to it

let trainingSets = [0, 0];

const addExample = label => {
    const image = tf.browser.fromPixels(canvas);
    const feature = net.infer(image, 'conv_preds');

    classifier.addExample(feature, label);

    context.clearRect(0, 0, canvas.width, canvas.height);

    label === 'santa' ? 
        santaButton.innerText = `Santa (${++trainingSets[0]})` :
    elderlyButton.innerText = `Elderly (${++trainingSets[1]})`;

    document.querySelector('.info').innerText = `Trained classifier with ${label}`;

    image.dispose();
}

First we get the pixel values from the canvas using ft.browser.fromPixels. Next, we get the features using the infer method of MobileNet. After that, to add examples, we can simply call addExample on the classifier with the feature and the label.

Then we need to update the DOM elements. We clear the canvas, so we can add a new example. We also need to update the buttons to tell the user how many examples have been added to each set. For this, I’ve created a new array which counts the number of examples.

Lastly, we can call dispose on the image to free up memory.

Predicting

Once we have the necessary data, we can test the model. By clicking on “Predict”, the predict function will be called which is very similar to addExample:

const predict = async () => {
    if (classifier.getNumClasses() > 0) {
        const image = tf.browser.fromPixels(canvas);
        const feature = net.infer(image, 'conv_preds');

        const result = await classifier.predictClass(feature);

        context.clearRect(0, 0, canvas.width, canvas.height);

        document.querySelector('.info').innerText = `Predicted to be ${result.label}`;

        image.dispose();
    }
}

The very first thing is to test whether we have examples in our data set. If we haven’t added any examples, there’s no way it’s going to predict us anything. This is what classifier.getNumClasses checks.

Just like for addExample, we need the features of the image, which we can get the same way. We then need to call classifier.predictClass passing in the features to get a prediction about the image. Once we get a result, we clear the canvas, print out the prediction, and dispose the image object.

Summary

And now our image classifier works as expected. After training the model with a couple of images for both sets, we can predict the next image with confidence.

To get the full source code and tweak around with the project, visit codesandbox.io. You can also clone it from GitHub. This is my last tutorial for this year, but I will be back with new topics in the next decade. Thank you for reading through!

As a last word, I would like to wish you a Merry Christmas and a Happy New Year! 🎄 🎉

Learn how you can build upon classifications and also add emotion detection:

What is the Difference Between A URL And A URI?

Ferenc Almasi — Fri, 09 Jul 2021 05:22:12 +0000

As a developer, you've already heard about URL countless times. You may have also come across URI before and now you want to know what is the difference between the two if there's any. They are often used interchangeably but we have to differentiate between the two. Before doing a comparison, let's see what each of them means by definition.

What is a URL?

A Uniform Resource Locator or URL for short — as the name suggests — is a reference for a resource and a way to access that resource. It is often referred to as the address of a website. The one you find in your address bar.

A URL is made up of a couple of different parts:

A protocol: Usually https or http. A way to tell how to access the resource. You may have also seen other commonly used protocols such ftp, or file.
This is followed by ://
A hostname: A registered name — or an IP address — that represents an IP address — a numeric identifier that is used for identifying a device connected to a network.
Followed by an optional port that is preceded by a colon.
A path: It can refer to a file system path, but is also often used as a slug.
Optional query parameters that are preceded by a question mark, and where multiple parameters are concatenated with an ampersand
Lastly, an optional fragment preceded by a hash. It is used for providing quick links for headings on a page.

To demonstrate it through the above image, this is how the URL would look like in a diagram:

What is a URI?

Now that you fully understand how a URL is made up, let's see what exactly is a URI. Just like a URL, a Uniform Resource Identifier also provides a way to identify resources. But unlike URLs, they don't necessarily provide the means to locate said resources.

An example for a URI - that is not a URL - is an ISBN number that is used for identifying books. It clearly identifies the resource with a unique number, but it does not provide any means to actually reach the resource.

Therefore, we can conclude that a URI is a superset of URLs and that each URL is essentially a URI as well.

The Difference Illustrated

To illustrate how they are connected, take a look at the following Venn diagram:

Really a URL is a type of URI, that also includes information on how to access a resource. If there's only one thing to take from this tutorial, be it this one sentence from RFC3986:

The term "Uniform Resource Locator" (URL) refers to the subset of URIs that, in addition to identifying a resource, provide a means of locating the resource by describing its primary access mechanism.

Summary

As web developers, we have to memorize lots of acronyms and initialisms. The deeper you understand core concepts about how the internet works, the easier you can do your job, the higher the quality of your work will be, and the more knowledgeable you will become.

With this tutorial, now you also know the difference between URI and URL. If you still have any doubts in your head, don't hesitate to ask questions in the comments section. Thank you for reading through, happy coding!

How to Set Up Protected Routes in Your React Application

Ferenc Almasi — Fri, 05 Mar 2021 14:58:21 +0000

Oftentimes, when you’re building a web application, some of the routes may be involved with authentication. You want to restrict user access to certain pages or you have your whole application behind a login.

React Router is a great way to go when it comes to routing, but you don’t really have the option to protect routes from being accessed by anyone. Luckily the solution to this is really simple and straightforward.

In this tutorial, I would like to show you what was my solution to the problem and how I got around it. I’ll start from scratch using create-react-app and only include the absolutely necessary things so you can follow along. Without further ado, let’s jump into coding.

Setting Everything Up

After I bootstrapped the project with create-react-app, I also installed react-router-dom for routing. We won’t need any other dependencies for this project. With not much modification to the initial create-react-app, this is the current state of the package.json file:

{
  "name": "auth",
  "version": "0.1.0",
  "private": true,
  "dependencies": {
    "@testing-library/jest-dom": "^4.2.4",
    "@testing-library/react": "^9.4.0",
    "@testing-library/user-event": "^7.2.1",
    "react": "^16.12.0",
    "react-dom": "^16.12.0",
+   "react-router-dom": "^5.1.2",
    "react-scripts": "3.3.1"
  },
  "scripts": {
    "start": "react-scripts start",
    "build": "react-scripts build",
    "test": "react-scripts test",
    "eject": "react-scripts eject"
  },
  "eslintConfig": {
    "extends": "react-app"
  },
  "browserslist": {
    "production": [
      ">0.2%",
      "not dead",
      "not op_mini all"
    ],
    "development": [
      "last 1 chrome version",
      "last 1 firefox version",
      "last 1 safari version"
    ]
  }
}

Modifying the Index File

We will achieve protection by creating a custom component that will handle incoming requests. We have the plain old Route component in React. This will be used for the sole public route we have, the login page. We also want to have a custom component as well that will handle private routes. Let’s call it ProtectedRoute.

The purpose of the component will be very simple. If the user has been authenticated, render the passed component. Otherwise, redirect them to the login page.

import React from 'react';
import ReactDOM from 'react-dom';
+ import { Route, BrowserRouter, Switch } from 'react-router-dom';
import './index.css';
+ import Login from './Login';
+ import Dashboard from './Dashboard';
+ import Settings from './Settings';
+ import ProtectedRoute from './ProtectedRoute';
import * as serviceWorker from './serviceWorker';

+ ReactDOM.render((
+     <BrowserRouter>
+         <Switch>
+             <Route path="/login" component={Login} />
+             <ProtectedRoute exact={true} path="/" component={Dashboard} />
+             <ProtectedRoute path="/settings" component={Settings} />
+             <ProtectedRoute component={Dashboard} />
+         </Switch>
+     </BrowserRouter>
+ ), document.getElementById('root'));

// If you want your app to work offline and load faster, you can change
// unregister() to register() below. Note this comes with some pitfalls.
// Learn more about service workers: https://bit.ly/CRA-PWA
serviceWorker.unregister();

As you can see, I’ve imported Route, BrowserRoute, and Switch from react-router-dom. I also created some components so we can test our router.

The ProtectedRoute will work in the following way:

It expects a component prop, the one it should render
It also expects a path so it knows what component to render on which URL

Here I defined the Dashboard component two times. This is because we want to land on the Dashboard if no path has been defined. This is what line:15 handles. We also want to load in the Dashboard if the user types in an invalid URL. By omitting the path attribute on line:17, we tell React Router to fallback to the provided component. So let’s see how the ProtectedRoute component works on the inside.

Creating Protected Routes

So what’s inside the custom component we’ve imported? It’s actually really simple, we only have a render function:

import React from 'react'
import { Redirect } from 'react-router-dom'

class ProtectedRoute extends React.Component {

    render() {
        const Component = this.props.component;
        const isAuthenticated = ???;

        return isAuthenticated ? (
            <Component />
        ) : (
            <Redirect to={{ pathname: '/login' }} />
        );
    }
}

export default ProtectedRoute;

We get the component from the props and return it if the user has been authenticated. I’ve also made use of Redirect from react-router-dom. If isAuthenticated turns out to be false, we redirect the user to the login page.

So how do we actually decide if the user is authenticated or not? What should we assign to isAuthenticated?

Authenticating Users

The last step is to authenticate users. This can be done in various ways, your choice of implementation may differ. We can either use cookies, or localStorage, or a combination of both, or maybe something else. Either way, we want to store some information about the user on the client-side so we know when they are logged in.

render() {
    const Component = this.props.component;
    const isAuthenticated = localStorage.getItem('token');

    return isAuthenticated ? (
        <Component />
    ) : (
        <Redirect to={{ pathname: '/login' }} />
    );
}

To prevent forgery, this information may be the presence or absence of a token that you also want to verify on your server-side. That way, you make sure that users cannot log in based on only the presence of the token key. It ensures that it is its value that actually matters.

As you see from the gif above, if I refresh the page, it takes me to the login screen. If I try to access any other route that has restrictions, I get redirected back to the login page. If I set a token — no matter the value for now — I can access those pages. Also note that if I try to access a route that doesn’t exist while I’m logged in, I land on the Dashboard. As soon as I clear the storage, I lose my access.

Summary

To summarise: Even though your React code base will be obfuscated, the way your client application works can be reverse-engineered. This is why it is essential that everything that involves authentication or authorization should be backed up by a server-side implementation.

Thank you for taking the time to read through, let me know your thoughts on this approach in the comments below. What would be your security solution? 🔒

Understanding Regular Expressions in JavaScript

Ferenc Almasi — Fri, 09 Oct 2020 11:18:22 +0000

In a previous article, I talked about how I managed to reduce my CSS bundle size by more than 20%. I had a lot of examples of regex patterns there, and recently I also got questions related to the topic, so I thought it’s time to collect things down in one place.

What are regular expressions?
Let’s start off by first defining what regular expressions actually mean? According to Wikipedia

A regular expression, regex or regexp is a sequence of characters that define a search pattern.

That’s a pretty good definition; regexes are nothing more than a combination of characters that are mostly used to find patterns in text or to validate user input.

Tools of the Trade

To give you a simple example, say we have an input field where we expect the user to type in some numbers in the following format: YYYY/MM/DD
Four numbers followed by a slash, followed by two numbers, a slash, and two numbers again. A date. 🗓️

Now when it comes to writing regex patterns, there are a number of great tools out there that can help you achieve your goals. There are two I’d like to mention and these are:

RegExr helps you with a handy cheat sheet and also lets you test it out right away as the expressions are evaluated in real-time.

This is how I actually “learned” to write regex. Regexper is another great tool that helps you visualize the pattern with a diagram. Back to the example, the right solution is as simple as doing:

/\d{4}\/\d{2}\/\d{2}/g

Above example represented with a diagram

Before starting, I would like to advise you to follow along by copy-pasting the examples into RegExr and play around with the “Text” field.

The Start

Now let’s break it down, starting from the basics. Every regex pattern is denoted by two /, the pattern itself goes between them. We can also have flags after the closing slash. The two most common you are going to come across are g and i or the combination of both: gi. They mean global and case insensitive respectively.

Say you have a paragraph in which the digits appear more than once. In order to select every occurrence, you have to set the global flag. Otherwise, only the first occurrence will be matched.

Say you want to select both javascript and JavaScript in a piece of text. This is where you would use the i flag. In case you want to select all occurrences then you need the global flag as well, making it /javascript/gi. Everything that goes between the slashes will get picked up by regex. So let’s examine what we can have between //g and what do they actually mean.

Character Classes

The regex in the first example starts with \d. This is called a character class. Character classes — also called “Character Sets” — lets you tell the regex engine to match either a single or a set of characters. The \d selects every digit. To select a set of characters you can use brackets. For example, to do the same, you can alternatively use [0-9].

This can also be done with letters. [a-z] will select every letter from a to z. Note that this will only select lowercase letters. To include uppercase as well you need to say [a-zA-Z]. Multiple characters can be stacked by simply writing them one after another. Can you guess what [a-z0-9] will do? That’s right, it will select every letter from a to z including every digit from 0 to 9.

Quantifiers and Alternations

Moving on we have {4} after \d. This is called a quantifier and it tells the regex engine to look for exactly four digits. Therefore /\d{4}/g will match for 2019, but not for 20 19, 20, 201, or anything else that’s not four digits long.

This is what we have done for months and days with \d{2}. We want to get numbers that are exactly two digits long. You can also define a range with two numbers, starting from the minimum: \d{2,4}. This will get numbers that are at least 2 digits long but not longer than 4. You can also omit the max value \d{2,} and it will get every number longer than 2 digits.

There are also four other alternations I would like to cover as they are often used. The | (or) operator lets you define multiple alternatives. Say you have to write a regex for URLs and you need to match both “HTTP” and “WWW”. Piping them together lets you match either one of them: /http|www/g.

The other three are really similar to each other and are used to define quantity. They are in order: \d*, \d+, \d?.

Star is used to match 0 or more of the preceding character.
Plus is used to match 1 or more of the preceding character.
The question mark is used to match 0 or 1 of the preceding character. It can be used if you want to express optionality. Let’s say you want to match both http and https this time. This can be done by /https?/g, which will make the (preceding) letter “s” optional.

Escaped Characters

Next, we have the following: \/. This is an escaped character. We wanted to match for a forward slash, but to do so, we first need to escape it with a backslash, and vice versa. The same goes for other special characters that otherwise would have another meaning.

For example, a dot means any character, except a new line. But if you specifically want to match “…”, you can’t just write /.../g. Instead, you need to escape them with a backlash: /\.\.\./g.

You know that brackets are used to match for character sets. But what if you want to target the [] characters themselves? They also need to be escaped, so instead of [] you would do \[\], and so on.

Groups and Lookarounds

Now say you use this regex in your JavaScript code and whenever you find a match, you want to extract a portion of it. In this case, it would be nice if we could retrieve the year, month, and day separately so we could do different kinds of stuff later with them. This is where capturing groups come into place. See the three examples below:

// Original example
/\d{4}\/\d{2}\/\d{2}/g.exec('2020/01/02'); // Outputs: ["2020/01/02", index: 0, input: "2020/01/02", groups: undefined]

// With capturing groups
/(\d{4})\/(\d{2})\/(\d{2})/g.exec('2020/01/02'); // Outputs: ["2020/01/02", "2020", "01", "02", index: 0, input: "2020/01/02", groups: undefined]

// With named capturing groups (as of writing, currently in stage 4 for ES2018)
/(?<year>\d{4})\/(?<month>\d{2})\/(?<day>\d{2})/g.exec('2020/01/02'); // Outputs: ["2020/01/02", "2020", "01", "02", index: 0, input: "2020/01/02", groups: {…}]

/**
 * Groups will include the following:
 * groups:
 *   day: "02"
 *   month: "01"
 *   year: "2020"
 */

In the original example, when you use the exec method on the regex and pass in a date, you get an array back. (meaning we have a match, otherwise exec would return null). In this case, you would still need to call '2020/01/02'.split('/'); to get what you want.

With the second example, you can get around this by grouping everything together with parentheses. By saying (\d{4}), you group the year which you can later extract with exec. Now in the output, you get back the year, the month, and the day separately and you can access them, starting from the first index of the array: arr[1]. The zero index will always return the whole match itself.

I also included a third example which uses named capturing groups. This will give you a group object on the output array, which will hold your named groups with their value. However, this is not standardized yet and not supported in all browsers so I would advise avoiding it using in production code just yet.

There can also be cases where you need to group part of the pattern together, but you don’t actually want to create a group for it when calling from JavaScript. A non-capturing group will help you in this case. Adding ?: to the beginning of the group will mark it as non-capturing: (?:\d{4}).

Lookarounds

We talked about groups but we also have so-called “lookarounds”. Among them, we have positive and negative lookaheads, which basically tells the regex engine to “Look forward and see if the pattern is followed by a certain pattern!”.

Imagine you have a domain regex and you only want to match domains that are ending with “.net”. You want a positive lookahead because you want to end it with “.net”. You can turn your capturing group into that by adding ?= to the beginning: domainRegex\.(?=net).

The opposite of that is a negative lookahead. You want a negative lookahead when you don’t want to end it with “.net”. The pattern in this case is ?!, so domainRegex\.(?!net) will match every domain, except the ones that have a “.net” ending.

There’s also lookbehinds which do the exact opposite; look back and see if a pattern is preceded by the one specified in the lookbehind. They are ES2018 features, just like named capturing groups, so using them in production is not advised.

It’s important to note, lookarounds will not be part of a match, they only validate or invalidate it!

Practice Time

Let’s say I want to create a regex that matches a URL for webtips and I want it to work with “HTTP”, “HTTPS”, “WWW” or no protocol at all. That means I need to cover four different cases:

Starting from the beginning I can just say:

/https?/g

This will match for both “HTTP” and “HTTPS”. This is followed by a double colon and two forward slashes. Your eyes light up and you say: “We must escape those!” So we can expand the pattern to:

/https?:\/\//g

And now we can finish up the rest with the hostname itself, taking into consideration that we also have to escape the dot, leading us to:

/https?:\/\/webtips\.dev/g

Now, this will definitely work for the first two cases but we can also have “WWW” and no protocol at all. So we “or” it with a pipe:

/https?:\/\/|www\.webtips\.dev/g

And the only thing left to do is to make it optional so we have a match when we don’t provide any protocol. We can do this with a question mark at the end of “WWW”, but to make it effective to HTTP, we have to group them together, so that leaves us with:

/(https?:\/\/|www\.)?webtips\.dev/g

Use Cases in JavaScript

There are a couple of methods that you can use with regular expressions in JavaScript. We have to differentiate between methods attached to the RegExp object and methods on the String object. We already looked at exec, but we also have another common RegExp method which returns either true or false, based on the provided input. With that, you can easily create checks in your code:

if (/graph/g.test('paragraph')) { ... } // Will evaluate to true

We also have a couple of handy functions on the String object. The most common one that you will use is probably match, which returns an array of matches if there’s any, or null if there’s none. The above example can be rewritten in the following way:

'paragraph'.match(/graph/g); // Returns ["graph"]

There’s also matchAll, but instead, it returns either a RegExpStringIterator or an empty array. A RegExpStringIterator is an iterator on which you can loop through. Each item contains an array, similar to the output of exec. You can get the outputs by using spread on the return value of matchAll.

[...'paragraph'.matchAll(/graph/g)];

Last but not least, there’s String.search, which returns the index number for the match, in case there’s any. If there’s none, it will return -1 instead. In the example below, it will find a match, starting from the 5th character of the provided string, hence it returns 4. (As we start the index from 0)

'paragraph'.search(/graph/g); // Returns 4

As a last word, I would like to encourage you to practice and hack the regex used in the subtitle and comment your solution down below. The right answer gets the cookie 🍪. To give you a little bit of help, here’s a diagram of that.

Cheatsheet

To recap everything, here’s a quick reference to things mentioned in this article. I marked ES2018 features with an exclamation mark.

Flags

g — Global
i — Case Insensitive

Character classes

\d — Match for every digit
\w — Match for every word
[a-z] — Match a set of characters inside the brackets (a to z)

Quantifiers, Alternations

a{4} — Match preceding token that is exactly 4 characters long
a{2,4} — Match preceding token that is between 2 and 4 characters long
a{2,} — Match preceding token longer than 2 characters

z* — Match 0 or more of the preceding character
z+ — Match 1 or more of the preceding character
z? — Match 0 or 1 of the preceding character

a|z — Match “a” or “z”

Escaped characters

\/ — Escape a forward slash (char code 47)
\\ — Escape a backslash (char code 92)
\. — Escape a dot (char code 46)

Groups, Lookarounds

(2020) — Capturing group
(?:2020) — Non-capturing group
(?<year>2020) — Named capturing group ⚠️
(?=2020) — Positive lookahead
(?!2020) — Negative lookahead
(?<=2020) — Positive lookbehind ⚠️
(?<!2020) — Negative lookbehind ⚠️

JavaScript functions

regex.exec('string') — Returns null or array containing the match
regex.test('string') — Returns true or false based on the provided string

str.match(/regex/g) — Returns null or array containing matches
str.matchAll(/regex/g) — Returns an empty array or RegExpStringIterator
str.search(/regex/g) — Returns the index, returns -1 if no match is found

What is The Event Loop in JavaScript?

Ferenc Almasi — Fri, 02 Oct 2020 08:47:00 +0000

You may have heard that JavaScript has a thing called the “event loop”. But what is it actually and what does it really mean? Why it’s important to know and what is the benefit of knowing about it? In this tutorial, we will explore the questions and simplify a seemingly complex idea.

In one single sentence, let’s define what is the event loop:

Moving events from the task queue to the call stack is called the “event loop”

So then what is this task queue and call stack? And how do they come into play? Let’s have a look at a simplified version of a browser’s architecture to have a deeper understanding of what happens when your JavaScript code is being executed.

A simplified representation of a browser’s architecture

Inside the browser, we have four main components. The one we are interested in is the event loop. But in order to understand the job of the event loop, we need to clear the fog around the other three: the call stack, the web API thread, and the task queue.

The Call Stack

As you may have already know, JavaScript is a single-threaded language, which means it only has one call stack, unlike other multi-threaded languages.

This also means that it can only execute one code at a time. The order in which this code is being executed is handled by the call stack. It holds information about where we are in the code right now, in terms of execution order.

To fully understand its workings, let’s demonstrate it through an example:

console.log('Learning');
console.log('About');
console.log('The Event Loop');

We all know what will happen. We are going to see Learning About The Event Loop in the console. But what happens inside the call stack? Let’s break it down how it is handled, line by line:

We start off at line:1; We push console.log onto the top of the call stack and pop it off immediately as we return implicitly.
We do it for the other two lines as well. We push console.log onto the stack and pop it off once it’s executed.

Let’s look at another example, what do you think will happen?

const recursion = () => {
    recursion();
};

recursion();

We define a function called recursion and we call it on line:5. Then inside the function, we call it again and again and again… Which will give us an error:

Chrome throws an error when the maximum call stack size is exceeded

We keep pushing recursion onto the stack without ever popping off one item, leading to a RangeError, which prevents the browser from crashing.

You can also see the stack trace — the order in which your code is being executed — below the error message.

That’s all the call stack does. It pushes code onto the stack and pops them off once they are executed. It’s basically a snapshot of the current state of your program; where you are when the JavaScript engine executes your code.

So if JavaScript is single-threaded and we are only able to run one piece of code at a time, then how come we have asynchronous functionality? — Meaning we are non-blocking. This is where browser or web APIs come into play.

The Web API Thread

Let’s take a look at the following example now:

console.log('🐹');

setTimeout(() => console.log('🐹🐹'), 0);

console.log('🐹🐹🐹');

What do you think the output will be in the code example above? If you guessed:

🐹
🐹🐹🐹
🐹🐹

You’ve either learned it on the hard way or you already know how the event loop works. Maybe both. But why not:

🐹
🐹🐹
🐹🐹🐹

Even though we call setTimeout with 0 milliseconds, it is still being executed as the last thing. Let’s quickly go through how the code above is being handled by the call stack once more:

We start at line:1, we push the console.log into the call stack and we pop it off immediately, so we have ‘🐹’ in the console.
We arrive at line:3, we push setTimeout into the stack, but we can’t execute it because setTimeout is not part of the V8 engine. It’s not a core JavaScript function, it’s a browser API. It is added to JavaScript through the browser. This is the reason you need to polyfill the Promise object or the DOM inside node as they are provided by the browser. These API calls are handled in a different thread, so we initiate a call to the web API thread and we pop setTimeout off the stack. This starts a timer in another thread pool, outside of the JavaScript world.
We continue our execution and we push the other console.log on line:5 onto the stack and we pop it off, so now we have ‘🐹🐹🐹’

This is the current state of the browser. We have an empty call stack, but we have one item waiting to be executed in the thread which handles web API calls. This holds our last console.log.

Now whenever the call to setTimeout is finished (which is instantaneous in this case since we passed in 0 milliseconds), we want to execute it. In order to do so, we need to move it back onto the stack. Since this can finish anytime in the future but we don’t know when, we can’t just push it onto the stack as it would mean our code is not deterministic. It would appear randomly at some point in time.

Think of a network request. It can finish in 300ms or never or anything in between. So instead, it is pushed into the task queue.

The Task Queue

The task queue — often called the callback queue or the event queue — is responsible for collecting the return value of each successfully executed web API call.

Whenever our setTimeout finishes, its callback is moved into the task queue. In our case, the last console.log.

So now we have an empty call stack and an empty web API thread. But we have the console.log waiting in the task queue. If we were to have multiple web API calls, we would push them into the task queue one after the other.

So how do we get events from the task queue into the call stack? This is where the event loop comes in, the protagonist of the article.

The Event Loop

From all of the above, the event loop is probably the simplest piece among all: Its job is to look at the task queue and the call stack;

If the call stack is empty and we have events waiting to be executed in the task queue, it pushes the first event from the task queue back into the call stack. And it keeps going until the task queue is empty. That’s all the event loop does:

// If the call stack is empty and the task queue is not
// move the first event from the task queue into the call stack
if (callStack.isEmpty && taskQueue.length) {
    eventLoop();
}

So now our last console.log is being pushed back into the call stack, which is executed again and hence we get:

🐹
🐹🐹🐹
🐹🐹

With a 0 milliseconds setTimeout, we essentially told JavaScript to defer the call until the stack is empty.

So the next time you’re on an interview and the interviewer asks you questions like: What is the event loop? or How can it be that JavaScript is asynchronous and single-threaded at the same time? — hopefully, you will know the answer.

Recommendation

If you’re still in doubt however and this tutorial didn’t make any sense to you, then I’m probably really bad explaining things and sharing knowledge. In any case, if you want to dive deeper into the things mentioned above and you’re more of the visual type, I can highly recommend What the heck is the event loop anyway by Philip Roberts.

He helped me understand the event loop once and for all. This article may heavily reflect his teachings as he explains it in a clear and understandable way. He goes into great detail explaining the inner working of the JavaScript Runtime.

Do you have some additions that should be mentioned? Let us know in the comments below! Thank you for reading through, happy coding!

Why Do You Need to Know About Functional Programming?

Ferenc Almasi — Thu, 24 Sep 2020 09:55:26 +0000

When I first started learning about functional programming, I had a hard time wrapping my head around it. I understood the concept and the main principles but I lacked the practical knowledge.

With this tutorial, I want to cover not just the concepts, but give you examples and show you how you can apply the functional programming paradigm on your own code.

Let’s first start by defining what is functional programming.

Functional programming is a programming paradigm.

Just like object-oriented programming, functional programming has its own concepts. For example, everything revolves around being pure — functions always return the same output given the same input. They have no side effects, meaning they don’t alter or mess with any data outside of their scope.

It also advocates being immutable — once something is created, it cannot be changed. You may also often hear that functional programming uses a declarative approach as opposed to the imperative approach that is also used by the object-oriented paradigm.

These are just some of the concepts that make up functional programming. But why are these principles important? What can they give us?

Why Functional Programming Can Benefit Us?

It’s important to mention that functional programming is not a new paradigm. In fact, Lisp which was developed in the late 1950s was heavily functional. Still, we can benefit from it today for a couple of reasons.

One of them is that it will make your code easier to reason about. It focuses more on the “What is your program doing?” instead of “How does it do its thing?” — meaning you go with a declarative approach opposed to imperative implementations. To demonstrate, take a look at the two examples below.

In the first example, you focus on how the program is doing its thing, while in the second, you focus on what the program is doing:

Imperative

for (let i = 0; i < products.length; i++) {
    products[i].price = Math.floor(product.price);
}

Declarative

products.map(product => {
    product.price = Math.floor(product.price);

    return product;
});

The two implementations are doing the same thing; modifies an array so we have rounded numbers for each product.

For this small example, it may seem like you are writing more code. But behind the scenes, map will also return you a brand new array, meaning your original products will be kept intact. This is immutability in action.

It also makes your code more easily testable as it focuses on small contained functions called pure functions. As mentioned before, these functions are deterministic. you can guarantee that if you keep passing it the same value, you get the same output.

In the end, functional programming makes your code easier to reason about. It makes it easier to read and follow the process you took and makes your application less prone to bugs. In case something still goes wrong, it’s easier to troubleshoot since your code is more concise.

To demonstrate how you can use functional programming in action, I’ve prepared some code examples that show you how to be declarative.

Declaring What You Mean

One of the best ways to start is by looking at array functions. Higher-order array functions are a good example of the functional programming approach.

I have an entire article describing some of the array methods mentioned here, which you can check in the link below:

but let’s quickly go through some of the more important ones and see what they do and how they shorten your code to make it more readable.

Array.prototype.find
Used for finding a specific element that passes the test, returns the first match

// Even if we have multiple products that are on sale, it will only return the first match
products.find(product => product.onSale);

Array.prototype.filter
Used for returning the elements that pass the test, returns every match

// This will return every product that is on sale
products.filter(product => product.onSale);

Array.prototype.every
If every element meets the criteria, it will return true

// Every product should have a name so we get back true
products.every(product => product.name);

Array.prototype.some
If at least one element matches the criteria, it will return true

// If we have at least one product that is on sale, we get back true.
products.some(product => product.onSale);

Array.prototype.map
Used for transforming an array, gives back a new one

// Rounding prices for products
products.map(product => {
    product.price = Math.floor(product.price);

    return product;
});

Array.prototype.reduce
Used for producing a single value from an array

// Sum the prices of each product
products.reduce((accumulated, product) => accumulated + product.price, 0);

You can already see how these array methods can shorten your code instead of using for loops, but we can make them even more powerful by chaining them.

Most of these functions return an array, on which you can call another method and keep going until you get the desired result.

Function Chaining

Function chaining is another great concept. It makes your code more reusable and again, reduces the noise and creates a shorter, more concise code that is both more readable, and in case of any bugs, it’s easier to debug.

In the example below, you’ll see that since each function call returns an array, you can keep calling new functions on them to create a chain.

const round = (num) => Math.floor(num);
const isDivisibleByTwo = (num) => num % 2 === 0;
const add = (accumulated, num) => accumulated + num;

const numbers = [0, 1.2, 2.4, 3.6, 4.8, 5, 6.2, 7.4, 8.6, 9.8];

const sum = numbers.map(round)
                   .filter(isDivisibleByTwo)
                   .reduce(add, 0);

Instead of using three different for loops to get the desired value, you can simply call functions one after another and get it done in 3 lines.

Last but not least, libraries can help you avoid writing down the same things over and over again — and reinventing the wheel — by introducing helper functions for commonly occurring problems.

Libraries

There are many libraries out there that are following the functional programming paradigm. Some of the more well known are Lodash and Ramda.

To give you some visual differences between the two let’s take a look at how you can retrieve nested properties in each — a commonly occurring problem.

If one of the objects does not exist, you will get an error saying:

Let’s say we have a user object where we want to get their email address:

const user = {
  name: 'John Doe',
  dob: '1999.01.01',
  settings: {
    email: 'john@doe.com'
  }
}

Lodash
Lodash uses underscore

// returns "john@doe.com" || undefined
_.get(user, 'settings.email');

Ramda
Ramda uses R

// returns "john@doe.com" || undefined
R.path(['settings', 'email'], user);

In each library, we can avoid getting an error if the parent of email does not exist. Instead it silently fails with an undefined.

Now you have a better understanding of how to be more declarative. What are some other important concepts in functional programming? — It’s in the name, it is functions.

Functions In Functional Programming

Functions are not only an essential part of functional programming but of JavaScript as well. They can help you break up your code to smaller, more digestible pieces. It increases readability and makes your code more easily testable by separating your code into smaller sections, often called components.

There are many concepts of how you can use functions to your own advantage. Let’s see some of the more commonly occurring definitions you can find in functional programming.

Pure functions

As discussed previously, pure functions don’t depend on any data other than what is passed into them. They also don’t alter any data other than what they returned.

To give you a practical example for pure functions, think of the Math object:

// This will return ??? - we don't know
Math.random();

// This will return 10, no matter what.
Math.max(10, 5);

Here, Math.random is impure since it always returns a different value, even if we were to pass it the same input. Math.max however is a pure function since it will return the same output given the same input.

We need to note that in case our function doesn’t have a return value, it is not pure.

First-class functions

In JavaScript and other functional languages, functions can also be assigned to variables and you can pass them around, just like they were variables.

const greet = function () {
    console.log('👋');
}

// The greet variable is now a function, we can invoke it
greet();

Higher-order functions

A higher-order function is nothing more than a simple function that takes in another function as one of its arguments. Functions that return another function are also called higher-order functions.

A great example for higher-order functions are previously discussed array functions such as filter or map.

Function composition

Function composition is all about combining functions to form brand new functions.

For example, Ramda has the compose function which takes in a list of functions as arguments and returns a function. You can call this with the input for which you want to apply the series of functions.

// Produces 7.283185307179586
R.compose(
  R.add(1), 
  R.multiply(2)
)(Math.PI);

Currying

Currying is a technique where you call a sequence of functions with one argument instead of calling one function with multiple arguments. Each function returns another function. The function at the end of the chain returns the actual expected value.

// Instead of
const add = (a, b, c) => a + b + c;

add(2, 2, 2);

// Currying does
const curry = (a) => {
    return (b) => {
        return (c) => {
            return a + b + c;
        }
    }
};

curry(2)(2)(2);

Recursion

Recursion happens when a function keeps calling itself until some condition is met. In the example below, we are counting down from 100.

finalCountdown = (number) => {
    // If we don't specify an exit criteria, the number will continue into minus until the browser crashes
    if (!number) {
      return;
    }

    console.log(`It's the final countdown! - ${number}`);

    finalCountdown(number - 1);
}

// Will print out numbers from 100 till 1
finalCountdown(100);

It’s important to specify an exit condition otherwise you will create an infinite loop that eventually crashes the browser.

Now if you feel like you are starting to become overwhelmed by the amount of information, don’t worry, it’s a good sign that means you are expanding your knowledge. There are only two more important concepts we need to cover. They go hand in hand. They are immutability and side effects.

Immutability

When we talk about immutable variables and objects, we simply mean that once declared, their value can’t be changed. This can reduce the complexity of your code and make your implementation less prone to errors.

To demonstrate immutability through an example, let’s say you have an array where you need to remove the first item. Take a look at the differences below:

const presents = ['🎁', '📦', '🎀', '💝', '🎄'];

// --- Mutable solution ---

// we get back 🎁
// and presents will be equal to ['📦', '🎀', '💝', '🎄'];
presents.shift();

// --- Immutable solution ---

// newPresents will be equal to 📦 🎀 💝 🎄
// and presents will be still equal to ['🎁', '📦', '🎀', '💝', '🎄'];
const newPresents = presents.slice(1);

In the first example, you modify the original array with the shift function. If you want to achieve the same but keep the original array intact, you can use slice instead. This way you can avoid having unforeseen bugs in your application where you unintentionally modify data that should be kept in pristine condition.

One downside of immutability is performance. If you create too many copies you will run into memory issues so in case you operate on a large data set, you need to think about performance.

What Are The Side Effects?

We also need to talk about side effects, not because they are part of the functional programming paradigm but because they happen regardless of what programming pattern you take. They are an important part of any program and you need to know when and why they happen.

So what are side effects? — Side effects can occur when a function is impure, therefore it does not necessarily return the same output given the same input. One commonly occurring example would be a network request. No matter what is the input, you can get back anything from 200 (OK) to 500 (Internal Server Error).

So you can’t avoid having side effects and your goal shouldn’t be to eliminate them entirely, but rather to be deliberate. Deliberate about why and when they happen.

Summary

Functional programming is a great way to organize your code in a better way. There are other programming paradigms out there like object-oriented programming. So what should you use, which is better?

There’s really no answer, it depends on your situation and there’s no one above the other. You can also combine multiple paradigms together so it’s not a “one way or the other”.

Thank you for taking the time to read this article, happy coding!

What Is “this”, After All? — A Look at JavaScript’s this Keyword

Ferenc Almasi — Thu, 17 Sep 2020 05:20:55 +0000

What is this all about? This doesn’t even make any sense… Where is this coming from? — Everyone asked these questions to themselves at some point in their lives, so let’s settle the argument and demystify this once and for all. If you haven’t figured it out yet, this story is going to be about the dreaded this keyword in JavaScript.

First, let’s define what this is.

The “this” keyword in JavaScript refers to the object it belongs to.

Open up your console and write “this”. In this case “this” alone in itself refers to the global object. The global object in a browser is the window itself.

First Example

Now, what if we have our own object? What do you think the output will be in the following case?

const user = {
    name: 'Heisenberg',
    occupation: 'entrepreneur',
    sayMyName() {
        console.log(this.name);
    }
};

const sayMyName = user.sayMyName;

sayMyName();

If you guessed “Heisenberg”, you were wrong. You would actually get an empty string. But why is that? What would happen if you were to just call user.sayMyName() right away? — It would log out Heisenberg. Wait… WHAT??? 😨 Let’s start with the latter before I manage to confuse you even more.

We said that the keyword refers to the object it belongs to. When you call user.sayMyName(), this will point to the user object, hence when you call this.name, sure enough, you get back “Heisenberg”.

So what happens when you assign user.sayMyName to a new variable like we did in the example above? — Simply put, user.sayMyName becomes a plain function, completely unrelated to the user object.

Try to copy the example above to your DevTools and instead of calling sayMyName() write console.log(user.sayMyName) to log out the function itself. You would get back the exact function we defined in the user object. However, this time, the parent object of the function becomes the window.

And by the alignment of the stars, we do have a name property on the window, but by default, it’s value reads “” — an empty string. If we were to change this.name to this.userName, you would get undefined, because there’s no window.userName by default.

How do we fix this?

So we know we don’t get back the expected output because we are referring to the wrong object. Okay, that’s cool, but how do we fix it? Well, you simply bind the context, which you can do with the bind method. Change line:9 to the following:

const sayMyName = user.sayMyName.bind(user);

Bind expects a parameter that sets the this keyword to the provided value’s context. In this case, we want to bind the context to the user object so we pass “user”.

What if you want to use the function in a callback? — Same as before, you only need to bind the context, like we did before, and pass the extracted function as a callback:

document.getElementById('say-my-name').addEventListener('click', sayMyName);

Second Example

Let’s see two more examples. By now, it’s starting to become suspicious whether it will give back the expected value or not. In any case, you’re sitting in an interview and the interviewer writes down a coding exercise on the whiteboard with an evil smile when you suddenly get the question you are anticipating —

“What do you think the output will be for the following?”

const shape = {
    radius: 10,
    diameter() {
        return this.radius * 2;
    },
    perimeter: () => 2 * Math.PI * this.radius
};

shape.diameter();
shape.perimeter();

Of course, they can’t expect you to calculate all this in your head right? — You’re thinking… There must be a catch. There is! Let’s break it down.

First, you call shape.diameter, everything seems to be fine, we return the object’s radius * 2. Nothing fancy is going here, you get back 20. Next, you call shape.perimeter, you get back NaN 🤦‍♂️.

Comparing the two methods, it must have something to do with the way they are written. And you are right. The second one is an arrow function. Arrow functions don’t bind their own context, rather they are referring to the enclosing scope in which the object is defined, which is again, the window. And window.radius is evaluated to undefined. So the above function evaluates to 2 * 3.14 * undefined which in return, gives us NaN.

Note that for one-liners in arrow function, you can omit the return keyword. The above example is equivalent to this:

perimeter: () => {
    return 2 * Math.PI * this.radius;
};

Third Example

Let’s see a last one, this time going back to the very first example with a little twist, because why not.

Imagine you’re investigating a bug and you suspect that the root cause is related to a piece of code where you have an object with a method. You also have an enclosing inner function inside said method for some reason.

const user = {
    name: 'Heisenberg',
    occupation: 'entrepreneur',
    sayMyName() {
        const closure = function() {
            console.log(this.name);
        };

        return closure();
    }
};

const sayMyName = user.sayMyName;

sayMyName();

You quickly realize that this is not what it’s supposed to be. You want this to work, you want this to point to your object, but again nothing seems to work, you get an empty string. Seems like it is pointing to the window once more.

Can’t we just delete window to solve all our problems?

Just like for the previous one, you have a great idea!💡 Bind the user object to the assigned function!

const sayMyName = user.sayMyName.bind(user);

But you are still getting "". Unfortunately, that’s only half of the equation. To understand why, we need to pick it apart. If we are logging out sayMyName again, you get the body of the function which returns the inner function at line:9. If you insert console.log(closure) to line:8, you'll see that we get back the closure’s body with the console.log inside.

We know that we are getting back an empty string because this is pointing to the window object, so we must bind the right context to closure, right? That’s right, so you go ahead and return closure.bind(this) instead, but this time, you are getting back the body of the function 🤔.

That’s because bind only does the binding, but doesn’t actually call the function which we need. So you say we only need to do either

return closure.bind(this)();

user.sayMyName()();

As you probably already guessed, this is kind of a workaround and looks hacky and is not really the proper solution. We have another method that can be used to call a specific function with a given context. It’s the call method.

By changing the return to return closure.call(this), you tell JavaScript to call the function with the given context passed as a parameter. So that leaves us with the final solution being:

const user = {
    name: 'Heisenberg',
    occupation: 'entrepreneur',
    sayMyName() {
        const closure = function() {
        console.log(this.name);
        };

    return closure.call(this)
    }
};

const sayMyName = user.sayMyName.bind(user);

sayMyName();

You first bind the user object to your function assignment on line:13 and inside sayMyName, you also have to use call on the closure function to call it with the proper context.

As you can see, this works according to some rules which after you start to understand, everything else will make more sense… hopefully.

Things to Keep in Mind

By default this refers to the global object, which is the window if you are in a browser.
When you use this inside another object, it refers to the object it belongs to.
When this is used inside an arrow function, it refers to the parent object.
If you use a function call with bind or call, this will refer to the context passed as the first parameter to these methods. (bind will only bind the context while call will call the function as well.)

Why You Should Care About SEO?

Ferenc Almasi — Fri, 11 Sep 2020 18:48:37 +0000

Attracting visitors to your site can be tough considering the fact that there are more than 1.5 billion active websites that want the same. Get a slice of the pie and gain as much traffic to their website as humanly possible.

The number of live websites as shown by internet live

So you want to compete, grow your traffic and gain visitors. You know that displaying your site on the first page of Google could potentially bring in a huge amount of traffic from organic search. You don’t know the how, you just know the why. It can either make or break your business.

You may also happen to know that in order to achieve this, you need to wander into the mysterious land of SEO. There’s a lot of confusion about the exact meaning of SEO, even for developers.

To clear the confusion, let’s see what SEO actually is. Then we will see why you should care about it, and how you can rank your site higher in Google with its help.

The Meaning of SEO

If you haven’t heard about SEO yet, it means Search Engine Optimization. But what does it mean exactly? Why do we need to optimize for search engines? And what do we need to optimize?

You may think the answer is your content. And while this is true, it is only a small part of the whole picture.

The short answer is: basically everything on your site. Your content, your site’s performance. Things that you don’t usually see, like meta-information or even your design.

But why does it even matter for Google how does my site looks like? To understand that, we need to look into how search engines work.

How Search Engines Work?

Imagine you go to Google because you are interested in what SEO is, and how does it work. So you search for the term “SEO” and Google returns you a list of results. SEO can mean different things. The question is: Are you looking for “Social Engine Optimization” or for “Search Engine Optimizer”? Maybe you are looking for “Sociedad Española de Ornitología”?

None of the above. Of course, you are looking for articles on Search Engine Optimization. Google somehow manages to find articles for you on the subject. But how did it know what you were looking for?

Photo by Alfons Morales on Unsplash

Google constantly indexes websites to create a giant digital collection of pages that you can think of as a library. When you search for a keyword, Google loops through these millions of indexed websites, looking for possible matches. It tries to create a page for you, putting the most relevant information up top.

In the early days, Google worked on simple principles. The pages that had the most links to them from authentic and trustworthy websites must be the most relevant page for a given search. If everyone references the same resource, it must be important, right? Things have changed since then. But the basic principle stayed.

There are now hundreds of factors that go into ranking a site. Things that in the early days weren’t taken into account. Like user experience. This is why your design also matters. This is what we are interested in. If we know what are the factors and what to look after, we can rank our site higher. But before we begin, let’s see why you should even care.

Why You Should Care About SEO?

So what are the reasons to care about SEO, apart from the fact that you can rank higher in Google?

One of the most important things is that it builds trust over time and gives your site credibility. Whatever you provide to your audience, if you don’t have authority, they will leave because of the lack of credibility. By establishing your site on the first page of Google, it will indicate that your site is relevant to their search.

It’s also important to mention that — even if you are not a developer — you can still improve your SEO score massively by keeping an eye on the different factors that will rank you higher. But only if you constantly measure and monitor key metrics and resolve any issues related to SEO. Therefore, it is relatively cheap to implement good SEO. You mostly invest time.

In the end, it will mean increased traffic for your site which eventually will mean more conversions.

Factors for Ranking Higher

So what are the factors to rank higher? What should we keep an eye on?

As noted before, there are hundreds of factors that go into ranking a page. So you won’t be able to comply with everything. Yet, there are some key factors which you should take good care of.

Content

The most important is your content. Is it relevant to your visitors? Are they getting what they want? Here you should think about things like:

Using a clear, descriptive heading for your content. This is what users will first see and can decide whether they choose to stay or leave.
Structuring it with subheadings. It may sound obvious but your content should be easily digestible.
Using the right keywords throughout it. If you want to rank for the keyword “custom action figures”, you should use it throughout your site. Avoid spamming keywords. Google is smart and it can rank your page lower if it categorizes it as spammy content.

Document structure

The structure of your HTML document won’t be visible to the end-user. But it is just as important as your content. Keep an eye on:

meta tags: These will hold meta-information about your site, that will be displayed in search results. You should also use open graph tags to properly format your site on social media platforms.
proper, descriptive title tag: Your site must cater for the disabled. Title tags are not only used by search engines — to determine whether a site is relevant to a search — but by assistive technologies as well. Such as screen readers.
alt attributes for images: Often omitted by people, but providing alternate text for images is also a factor that determines your SEO score.
descriptive links: They help search engines better understand your content.

User experience

User experience has started to gain importance. These are some things you should be looking after:

page speed: Speed is starting to become a large factor for ranking your site. The slower it is, the worse the user experience.
mobile-friendly: In 2016, mobile users surpassed desktop. Your site must be responsive to be properly displayed on mobile devices, so users can easily navigate your content.
appropriately sized tap target: For these devices, interactive elements such as buttons should be large enough for the user to be easily tapped without accidentally clicking on other elements.
legible font-size: You should avoid using font sizes smaller than 12px. This helps to keep your content easily readable.

Of course, in order to know if you met the things mentioned above, you need to measure it somehow. There are several tools out there for measuring your SEO score. But you should also think about measuring other key factors.

Measuring SEO

Now that we know what are some factors that can rank our site higher, there’s only one thing left to do. Measure the results. If we can’t measure it, we don’t know if we are on the right track. But what should we measure exactly?

There are some key metrics that you should track. Probably the two most important are organic traffic and bounce rate. Organic traffic simply defines the traffic you receive from Google without paying for placement. The opposite of organic traffic is paid traffic.

Bounce rate on the other hand tells you how many visitors left your site after viewing only one page. The lower the number, the better. Having a high bounce rate can mean several things. Your visitors may didn’t found what they were looking for. Or they left immediately after they found what they were looking for. In either way, you want to keep users on your site by using internal links.

You also need to pay close attention to your click-through rate or CTR for short. There’s no use in appearing on the first page if visitors don’t click through to your site. This is what the click-through rate measures. Say you have 100 impressions, meaning your site has been seen by 100 people on Google. If one of them clicks through to your site, your CTR will be 1%. You can find out your CTR by using the dashboard on Google Search Console.

Photo by Webaroo.com.au on Unsplash

Tools for Monitoring and Improving SEO

So how can you track these? Luckily there are plenty of tools out there — both free and paid — that can help us achieve our goals. The most commonly used that you probably already heard of is Google Analytics.

Goole Analytics

Google Analytics is completely free and is easy to integrate into your site. It gives you a detailed report on your traffic. Their geographic location, demographic data, and much more. You can also implement A/B testing which helps you to improve your site in small iterations.

To get the most out of your SEO, you also need to know where it needs improvement. You need to know what are the weak points of your site. Just as for measuring metrics, there are tools that can identify issues.

Learn more about Google Analytics

Lighthouse

Lighthouse will probably be your best friend. It is built right inside Chrome’s DevTools. You can reach through the Audits tab. Apart from measuring SEO and giving you hints on what to improve, it also tests your site for performance. It is becoming a key factor for achieving higher ranks on Google.

Learn more about Lighthouse

If you take some time to monitor your key metrics and address any issues brought up by the tool, you will most likely increase your site’s ranking and get yourself in a better position on Google.

If you are also looking for extra tools that can help you along the way, make sure to check out my SEO checklist on GitHub. There, I provide additional analytics tools, SEO checkers, and chrome extensions. You can also find information on how to improve technical, on-site, and off-site SEO.

Summary

Although following the tips mentioned above will highly improve your ranking, we have to note that taking care of SEO is a long term investment. You won’t see results overnight. Keep improvements consistent and the invested time and effort will pay off.

What are some SEO tips you always look after? Share us in the comments. Thank you for taking the time to read this article, happy improving!

Turning Your Friend Into a Chatbot

Ferenc Almasi — Mon, 31 Aug 2020 06:59:06 +0000

Ever got angry over a friend for not replying for a long period of time? Maybe you were just waiting for some reaffirmation or just wanted to get your mind off of other things. Whatever the reason may be, thanks to advances in natural language processing or NLP for short, as a developer you can make a change.

Natural Language Processing is a field of AI, which gives computers the ability to understand human language.

By turning your friend into a chatbot, you’ll always have them around no matter what. It won’t be as smart as your friend, but it will be a chatbot anyway. Does that mean you are beyond lonely? Maybe it does. Is it really a good idea? Maybe it isn’t, but we can definitely learn interesting things about NLP along the way.

What Will We Need?

To start off, we are going to need some data to work with, namely your messaging history with the friend you want to turn into a chatbot. Despite the hero image, this article will assume you are using Facebook’s Messenger. But if you are using another platform and you can get the raw data from there, that’s fine as well.

We are also going to make use of natural, a robust natural language processing toolkit in JavaScript. The only other thing we are going to need is the built-in fs module of npm to read and write datasets.

How Does it Work?

So how does it work? First and foremost we will need to retrieve the data from Facebook. The more messages we have, the better the end result will be. After getting the messages, we will need to parse it to remove all unnecessary information, metadata like timestamps, and irrelevant messages like shared stickers or attachments. We are only interested in text.

Once we have a clean dataset in place, we can start the work on it, which is classification. Basically, we are going to classify sentences into different categories so whenever an input is given, a closely matching output is chosen based on previous messages.

This is why it’s so important to have sufficient data — the wider the data range, the better the end results. For reference, I will be working with a messaging history consisting of roughly 18k lines of data.

Retrieving The Dataset

To export your messages, go to your Facebook account settings and click on the third tab on the left-hand side: “Your Facebook information”.

You’ll see an option that says “Download your information”. Clicking on View will take you to a new page. Here you can request a copy of your collected Facebook information. There’s a ton of stuff you can export, but the one we are interested in right now is messages. So click on “Deselect all” to remove the checkboxes from each category and scroll down to select only messages.

It’s also important to change the Format from HTML to JSON as that is easier to work with. Everything else can stay the same. Once you click on “Create File”, it will start creating an archive for you, which may take a while, but once it is ready, you will get a notification and you will be able to download the prepared zip file for you.

Setting Up The Project

Once you have your data ready, we can start setting up the project. I created a project folder and I have a sub-folder inside, called data. This is where we will store all data required for classification.

In your downloaded zip file, you should have an inbox folder containing all your messages with the name of people being the folder names for each conversation. Open the one you want to work with and copy the message.json file to the data folder we’ve just created.

While also here, let's get the single dependency we are going to use which is natural. You can pull it down by running npm i natural, after your npm init.

Parsing The Data

If you are not using data extracted from Messenger, the steps in this section may differ for you, but I would still advise you to read through to get a good understanding of what steps you need to take to format any dataset.

If you open the message.json file, you’ll notice we have a bunch of irrelevant information that can be considered as noise for our training algorithm. Right at the beginning, we have a participants array. You can get rid of it right away, which only leaves us with a messages array.

Inside each message, we still have information that we don’t need. Removing them manually would take ages and developers are known for being lazy anyway. So let’s write a script that loops through them and extract the relevant information.

For that, I’ve created a file called parser.js at the root of my project and I also added a script called parse into the package.json file that runs parser.js with node parser.js.

To fully understand what we are going to implement, let’s take a look at the end result: (The end format of the training data we would like to achieve)

For the training data that will be used to train our classifier, we want to have labels. Inside each label we will have an inputs and outputs array containing messages and their responses from the original Messenger history. This is what our classifier will use to train itself. What response to give to each input. It will even manage to give us good responses to inputs it has never seen before.

Before starting the parsing itself, we need to define some variables and helper functions:

const fs = require('fs');

const messages = {};
const messageObject = {
    inputs: [],
    outputs: []
};

const input = './data/message.json';
const output = './data/trainingData.json';

const messageHistory = JSON.parse(fs.readFileSync(input));

const decodeUtf8 = (string) => decodeURIComponent(escape(string));

const getMessageType = (message) => {
    if (message) {
        return message.sender_name === 'Me' ? 'input' : 'output';
    }

    return '';
}

let hasInput = false;
let hasOutput = false;

In the very first line, we pull in the fs module which will help us reading in the messaging history and writing out the final training data that will be used to train our classifier. Next, we define a message object which will hold all of our inputs and outputs with their corresponding labels and we also define a blueprint for each individual message using the messageObject variable.

We then set the input source for the messages that we read in and the output source for the training data.

I also defined two helper functions. Messages are not stored in UTF-8 encoding so we need a decoder. This is what you see on line:14 and another function which helps to determine whether the message was sent by you (in that case it will be an input) or by your friend. (in that case, it will be an output)

This will help us to put separate consecutive messages into the same array. The same goes for the two variables on line:24 and 25 that will act as flags.

Consecutive messages from the same sender are put into one array.

Before starting the actual parsing we can also reverse the array, to put the latest messages at the top and oldest at the bottom:

// Reversing the order of messages to make most recent messages appear at top
messageHistory.messages = messageHistory.messages.reverse();

This way, we can start creating the training set with more relevant information. Now we can loop through the message history and start populating the training data:

messageHistory.messages.forEach((singleMessage, index) => {
    const currentMessageType = getMessageType(singleMessage);

    if (currentMessageType === 'input') {
        messageObject.inputs.push(decodeUtf8(singleMessage.content));
        hasInput = true;
    }

    if (currentMessageType === 'output') {
        messageObject.outputs.push(decodeUtf8(singleMessage.content));
        hasOutput = true;
    }

    // Only proceed if next message sender is not the same as the current in the loop
    if (hasInput && hasOutput && getMessageType(messageHistory.messages[index + 1]) !== currentMessageType) {
        messages[`label${index}`] = Object.assign({}, messageObject);

        messageObject.inputs = [];
        messageObject.outputs = [];

        hasInput = false;
        hasOutput = false;
    }
});

fs.writeFileSync(output, JSON.stringify(messages, null, 4), 'utf-8');

console.log(`parsed data written to ${output}`);

For each message, we want to determine who was the sender. Me or you? If it’s me, it’s going to be an input. If it’s you, it’s going to be an output. We push the message into the corresponding array and we set the input or output flag to true respectively.

And if we have them both set to true and the next message’s sender is different from the current one, we can create a new label containing both inputs and outputs arrays. Then we set everything back to its initial value and start again. Note that we need to use Object.assign to create copies from the original object. Otherwise, we would start populating the messages object with empty arrays.

When we all finished, we write the created object into the output file using UTF-8 as encoding.

Running npm run parse will execute the JavaScript file and you should see a trainingData.json file generated into the data folder.

Classification

Classification will be much easier than parsing the data. We can do it using just a couple of lines. For the training, I’ve created a separate file called train.js and a new script in package.json again so we can do npm run train to execute the train.js file.

Now let’s take a look at the training algorithm and go through it step by step to see how to train the classifier:

const fs = require('fs');
const nlp = require('natural');

const input = './data/trainingData.json';
const output = './data/classifier.json';

const trainingData = JSON.parse(fs.readFileSync(input));
const classifier = new nlp.LogisticRegressionClassifier();

Object.keys(trainingData).forEach((label, index) => {
    trainingData[label].inputs.forEach(input => classifier.addDocument(input.toLowerCase(), label));

    if (index + 1 === Object.keys(trainingData).length) {
        classifier.train();
        classifier.save(output, (err, classifier) => {
            if (err) {
                console.error(err);
            }

            console.log(`Classifier generated at ${output}`);
        });
    }
});

We start off by importing the FileSystem API and Natural. Since we want to work on the training data we need it as an input and our generated output will be the classifier.json file.

We store the training data inside the trainingData variable and we also define the type of classifier we want to use. In Natural we can choose between various classifiers. The one we are using here is a Logistic regression classifier.

To train the classifier, we loop through the training data. For each label, we loop through the inputs inside them and we call classifier.addDocument, passing in the inputs individually while also making everything lowercase — this will make the classifier more accurate — and the label to use for them.

addDocument also accepts an array so we could simply do: classifier.addDocument(traininData[label].inputs, label);, but this way, we will have more fine-grained samples, which will give us more accuracy.

And if we reach the end of the training data, we call classifier.train to train it. To also make the trained classifier reusable, we save it to the output location we defined on line:5, with classifier.save.

Final Results

To test out everything we’ve done, we can create a final script to handle an input text and generate an output for it. One more time, I’ve created a separate script with a separate command, making the package.json file look like this:

{
  "name": "natural",
  "version": "1.0.0",
  "scripts": {
    "parse": "node parser.js",
    "train": "node train.js",
    "test": "node test.js"
  },
  "author": "",
  "license": "ISC",
  "dependencies": {
    "natural": "0.6.3"
  }
}

I added a test script which runs test.js. And for that, we bring our chatbot to life with roughly 20 lines of code:

const fs = require('fs');
const nlp = require('natural');
const data = JSON.parse(fs.readFileSync('./data/trainingData.json'));

const input = process.argv.slice(2)[0];

nlp.LogisticRegressionClassifier.load('./data/classifier.json', null, (err, classifier) => {
    const guesses = classifier.getClassifications(input.toLowerCase());
    const guess = guesses.reduce((x, y) => x && x.value > y.value ? x : y);

    // if guess probability is less than 90% we can fallback to a generic message
    if (guess.value < 0.9) {
        console.log(`I don't understand what do you mean by "${input}"...`);

        return;
    }

    const replies = data[guess.label].outputs;
    const reply = replies[Math.floor(Math.random() * replies.length)];

    console.log(reply);
});

First, we import all the dependencies and we’re going to make use of the training data we created as well. Then we define the input which can be passed in right from the command line. So we can do run test "Hey". Here the passed string will be the input.

Next, we load in the classifier.json which we generated earlier, and as a callback function, we do classifier.getClassifications to get a classification for the input and store it inside the guesses variable. This will give us an array with labels and probability values. The higher the value, the more sure we can be we have a match. Remember, we set a threshold of 90%.

Logging out guesses gives us the potential categories with the most probable at the top. (98%)

We then select the one with the highest score with reduce and if the probability of that is less than 90%, we fall back to a generic message and return. Otherwise, we read the outputs from our trainingData.json with the label of the most probable guess and we select a random reply from the outputs array.

The used training data

Testing out the dataset with unknown inputs still gives us 98–99% probability.

The classifier can tackle sentences it has never seen before and still manages to produce valid outputs.

No more “seen“ and no more scene, this is how you turn your friend into a chatbot only waiting for your messages. 🤖

Do you already have experience with building chatbots? What are some common pitfalls developers tend to commit? Let us know in the comments below!

Thank you for reading through, happy coding!

Understanding Bind, Call and Apply in JavaScript

Ferenc Almasi — Thu, 27 Aug 2020 06:38:24 +0000

Like applying cold water to a burned area in the real world, we can also apply additional information to our function calls in the digital world.

Recently, I tried to clear the confusion around JavaScript’s this keyword and I briefly touched on bind and call. But this time, I would like to give a more in-depth look on them with some addition about apply.

Let’s go in order, according to the title, and start with bind. But first, we need some code to demonstrate all three of them, so take a look at the following:

const currentYear = new Date().getFullYear();

const spongebob = {
    name: 'Spongebob Squarepants',
    dob: 1986,
    species: 'sea sponge',
    greet(qoute) {
        console.log(`Hey, it's me, ${this.name} and I'm ${currentYear - this.dob} years old`);
        qoute && console.log(qoute); // If we have a quote, log it out
    }
};

const patrick = {
    name: 'Patrick Star',
    dob: 1978,
    species: 'starfish',
    greet(qoute) {
        console.log(`Hey, it's me, ${this.name} and I'm ${currentYear - this.dob} years old`);
        qoute && console.log(qoute);
    }
};

Bind

bind is used in JavaScript to bind certain context to a function. When you have a function called funky and you call it like this: funky.bind(soul), you are actually creating a new function where the context of this is set to the value of soul. Keep in mind that this does not modify the original function nor will it call.

// Since bind doesn't modify the original function, this.name will still be "Spongebob".
spongebob.greet.bind(patrick);
spongebob.greet(); // Hey, it's me, Spongebob...

// Assigning the bound greet to a variable and calling that will give back Patrick's details.
const greetPatrick = spongebob.greet.bind(patrick);
greetPatrick(); // Hey, it's me, Patrick...

The above code example demonstrates that bind does not change the actual function but creates a brand new one. When we greetPatrick() the second time, we get back Patrick’s details because of the bound context, even though we are calling spongbob.greet.

Call

Unlike bind, call will actually call the function immediately with the specified context. Let’s take a look at the following:

// This will immediately calls greet with the context of patrick.
spongebob.greet.call(patrick);

// Since we call the function right away, the value of greetPatrick will be the return value
// When we don't have an explicit return statement eg.: 'return true;', "undefined" is returned implicitly
const greetPatrick = spongebob.greet.call(patrick);
console.log(greetPatrick); // undefined

spongebob.greet.call(spongebob, 'I\'m a good noodle!');

On line:9, we call Spongebob with the spongebob context and for the argument, we are passing in a string. This line is essentially equivalent to the following:

spongebob.greet('I\'m a good noodle!');

Apply

Apply functions as call. The only difference between the two is that while call accepts a list of arguments, apply accepts an array of arguments.

patrick.greet.apply(patrick, ['Is mayonnaise an instrument?']);

Note the difference between call and apply. One is called with an array while the other is not. If we were to have multiple arguments they would look the following:

// Accepts list of arguments
spongebob.greet.call(spongebob, 'Believe it or not', 'I\'m a good noodle!');

// Accepts array of arguments
patrick.greet.apply(patrick, ['Is mayonnaise an instrument?', 'Mayonnaise is not an instrument 😔']);

I think that concludes the differences between the three. Let’s recap everything and draw the conclusion.

Conclusion

Use bind when you want to bind a context to a function you want to call later.
Use call or apply if you want to invoke the function immediately.

And the greatest question of the universe when talking about call and apply —

Which one to choose? It really depends on your choice.

Though if we look at which one performs better, it seems that the winner is call.

How to Lazy Load Images Natively

Ferenc Almasi — Tue, 18 Aug 2020 11:28:35 +0000

Native image lazy loading is starting to come into the web. Images account for the largest static resources we have. Many of these images are off-screen initially, so there’s no point in downloading every single one of them upfront.

Still to this day, lazy loading is mostly done through JavaScript as this feature is not widely supported yet. It works by detecting if the user is scrolled close enough to the image that it’s worth requesting it from the server.

By lazy loading them, we can reduce initial page load and enhance performance. Especially on websites that are content-heavy, like image galleries or social networks based around image posts.

Starting from Chrome version 75, we can now use native lazy loading. To know whether your Chrome supports it, go to chrome://help to check for your version.

How Does It Work?

It simply works by adding the loading attribute to either images or iframes. Yes, you can also do it with iframes. The content should start downloading as soon as the user scrolls near to it. By the time it is scrolled into view, the content should already be ready.

The loading attribute can take three different values.

auto

By default, it is set to auto. In this case, the browser will determine on its own whether to lazy load the image or not.

<img src="http://place-puppy.com/500x500" loading="auto" />

eager

eager tells the browser that the image is not a good fit for lazy loading, instead it should be loaded in right away.

<img src="http://place-puppy.com/500x500" loading="eager" />

lazy

Lastly, this is the attribute we are mostly interested in. It signals to the browser that the image is a good candidate for lazy loading it.

<img src="http://place-puppy.com/500x500" loading="lazy" />

Whenever the image comes close to the viewport, a request will be made to fetch the image.

Browser Support

As stated, this is not widely adopted yet. As of writing this article, the global support is roughly at around 73%, according to caniuse.

In case you specify the attribute but the browser does not support it, nothing will happen so you don’t have to worry about breaking your page. If you still want to start using it today, you can polyfill for browsers that don’t support it using feature detection in JavaScript.

Feature detection

Based on whether you want to support images or iframes, you simply have to check for the loading property on the prototype of the appropriate HTML element.

if ('loading' in HTMLImageElement.prototype) { 
    // This means the browser supports lazy-loading natively, we are good to go
} else {
    // This means the browser does not support lazy-loading natively
    // We have to polyfill it with either our own implementation or with an existing library.
}

If it’s unsupported, you can go with your own lazy loading implementation. To make sure it is enabled in Chrome, go to chrome://flags and search for the word “lazy” then set it to enabled.

Now you can go and give native image lazy loading a try. Thank you for taking the time to read this article, happy optimizing!