DEV Community: Sandor | tutorialhell.dev

Has anyone else noticed an increase in New, Empty Profile Followers recently?

Sandor | tutorialhell.dev — Thu, 22 Feb 2024 14:05:41 +0000

I've been noticing something a bit unusual over the past few days and I'm curious if anyone else has experienced the same. A significant number of new followers have been added to my profile, however, upon closer inspection, I realized that many of these accounts are newly created with no information at all - no posts, comments, or any form of engagement.

This trend has left me feeling a bit puzzled and, honestly, a bit discouraged. It's made me question the authenticity of the engagement on this platform. Are these real people interested in my content, or is there something else at play here?

Must-Have Practical Tutorial on Node.js Modules for Badass Beginners

Sandor | tutorialhell.dev — Tue, 20 Feb 2024 13:00:00 +0000

It's a practical tutorial that will provide you with everything you need to understand how require() works and how to use it efficiently. It includes detailed instructions and code examples that you can run either locally on your computer or in a prepared online Node environment to quickly get the hang of it. And there will be practical exercises waiting for you at the end of this tutorial.

It's part of my tutorialhell.dev project, aimed at helping beginners learn easily without getting stuck.

In Node.js, a module is a self-contained block of code that doesn't affect other code. JavaScript files are treated as modules in Node.js, where you can export functions, objects, or values from one file and use them in another.

Node.js supports two module systems: CommonJS modules and ECMAScript modules.

The CommonJS modules were the original method for packaging JavaScript code in Node.js.
The ECMAScript modules standard is utilized by browsers and other JavaScript runtimes and is generally viewed as the preferred approach for being future-proof. However, to get started, we will focus on CommonJS modules since they are still widely used. As we progress, I will introduce you to ECMAScript modules in the upcoming tutorials.

Let's break down the basics, starting with how to create a module in Node.js, how to export from it, and how to use the exported entities in other files.

If you don't have Node.js set up, you can get started using a ready-made Node.js environment in this sandbox.

Creating a Simple Module

Create a File: Every module is a JavaScript file. Create a file named 'myModule.js'.

Write Some Code: Inside 'myModule.js', define a function that you want to make available outside this file.

function sayHello(name) {
  return `Hello, ${name}!`;
}

Exporting from a Module

To make functions, objects, or values available outside the module where they are defined, you need to export them using module.exports.

module.exports is an object included in every JavaScript file in Node.js by default. It is used to export functions, objects, or values from a module. Here is the full code:

function sayHello(name) {
  return `Hello, ${name}!`;
}
module.exports = sayHello;

Importing a Module

To use the exported entities from a module in another file, you need to require that module.

Create Another File: 'index.js'. (It's already there if you are using the sandbox. Simply replace the contents with the code provided below.)

Require the Module: Use the require() function to import the module, passing the path to the module file as an argument:

const sayHello = require('./myModule');
console.log(sayHello('World')); // Outputs: Hello, World!

Explanation:

require('./myModule'): This line imports the module myModule.js. The ./ indicates that the file is located in the same directory as index.js. Node.js reads myModule.js, executes it, and then module.exports from myModule.js is returned by require() and assigned to the variable sayHello in index.js.

sayHello('World'): This line calls the imported function with the argument 'World', and the function returns the string 'Hello, World!'.

Multiple exports

To perform multiple exports from a Node.js module, you can use either the module.exports object or the exports shorthand. Each approach allows you to export multiple items, such as functions, objects, or values, from a single module. Here's how you can do it:

Using module.exports

You can directly assign an object to module.exports containing all the items you want to export. This approach is useful when you want to structure your exports clearly in one place. Create a 'multipleExport1.js' and paste the following code there:

function func1() {
  console.log('Function 1');
}

function func2() {
  console.log('Function 2');
}

module.exports = {
  func1,
  func2,
};

Using exports

The exports shorthand allows you to attach exports directly to the exports object. This is syntactically more concise for exporting multiple items. Create a 'multipleExport2.js' and paste the following code there:

exports.func3 = function () {
  console.log('Function 3');
};

exports.func4 = function () {
  console.log('Function 4');
};

Multiple imports

Similarly, you would import these in another file like this:

const myModule = require('./multipleExport1');
const myModule2 = require('./multipleExport2');

Or, using destructuring:

const { func1, func2 } = require('./multipleExport1');
const { func3, func4 } = require('./multipleExport2');

Replace the contents of the 'index.js' file with this code:

const sayHello = require('./myModule');
console.log(sayHello('World')); // Outputs: Hello, World!

// Approach 1
console.log('------- Approach 1 -------');
const myModule = require('./multipleExport1');
const myModule2 = require('./multipleExport2');

myModule.func1(); // Calls 'Function 1'
myModule.func2(); // Calls 'Function 2'

myModule2.func3(); // Calls 'Function 3'
myModule2.func4(); // Calls 'Function 4'

// Approach 2
console.log('------- Approach 2 -------');
const { func1, func2 } = require('./multipleExport1');
const { func3, func4 } = require('./multipleExport2');

func1(); // Calls 'Function 1'
func2(); // Calls 'Function 2'
func3(); // Calls 'Function 3'
func4(); // Calls 'Function 4'

And you should see the following logged in your terminal:

You might have noticed that when functions are imported through destructuring, they are assigned unique identifiers like func1, func2, etc. This uniqueness is necessary because two functions with the same name cannot coexist in the same file (module). It requires us to use unique names in different modules, which is not convenient.

However, there is a technique to rename these functions while importing them. This allows functions to have identical names across multiple modules and be renamed when imported into the same file to avoid name collisions. Here is how you do it:

Create a new file named 'multipleExport3.js' and paste the following code there:

exports.func3 = function () {
  console.log('Function 3 with an identical name from a different module');
};

exports.func4 = function () {
  console.log('Function 4 with an identical name from a different module');
};

Then add this code to the bottom of your 'index.js' file:

const {
  func3: renamedFunc3,
  func4: renamedFunc4,
} = require('./multipleExport3');
console.log('------- Destructuring with Renaming -------');
renamedFunc3();
renamedFunc4();

And your updated terminal output will look like this:

👉 Working code you can find here.

Choosing Between `module.exports` and `exports`

Use module.exports when you want to export a single item, like a function or an object, or when you prefer to define all exports at once in an object.

Use exports to attach multiple exports individually. This can be more straightforward when exporting several items but remember that you cannot assign directly to exports as it will break the reference to module.exports. For example, doing exports = { func1, func2 } will not work as expected.

Both methods are commonly used and choosing between them often comes down to preference and the specific needs of your module.

Here’s a breakdown of the key feature that every beginner should grasp:

`require()` is Synchronous and Caches Modules

Synchronous Loading: When you use require() to import a module, Node.js reads and executes the entire module file synchronously. This means that Node.js will halt the execution of your code until the entire module is loaded. This behaviour is particularly important to understand because it affects how and where you use require() in your application. Here's a simple code example to demonstrate that require() is synchronous in Node.js, meaning Node.js waits for the module to be fully loaded and executed before moving on to the next line of code.

Or try it out yourself by pasting the following code in the sandbox.

Create a file named 'greet.js' and paste the following code there:

console.log('Loading greet.js...');

function greet() {
  console.log(
    'Hello and welcome to learning web development with www.tutorialhell.dev!'
  );
}

console.log('greet.js loaded successfully.');

module.exports = greet;

Create or update (it's already there in the sandbox) the 'index.js' file and paste the following code there:

console.log('Starting index.js...');

const greet = require('./greet'); // Synchronously loads and executes "greet.js"

console.log('greet.js has been required.');

greet();

console.log('Ending index.js...');

How it works:
When you run 'index.js' using the command node index.js, you will see the logs in the following order, demonstrating the synchronous nature of require():

Notice how the execution of 'index.js' pauses until 'greet.js' is fully loaded and executed (evidenced by the logs from 'greet.js' appearing before the continuation of 'index.js').

In simple terms, synchronous means doing things one by one in order, where you have to finish one thing before you can start the next thing.

Caching Mechanism: Node.js caches the first instance of each module you require. If you require() the same module multiple times in different parts of your application, Node.js will not reload or re-execute the module file after the first time it's loaded. Instead, it reuses the cached version. This caching mechanism improves performance by avoiding the overhead of loading and compiling the module's code multiple times. However, it also means that any state maintained in the module is shared across all parts of your application that import it.

In simple terms, caching means storing something in a place where it can be quickly accessed later. It's like saving a shortcut to find information faster without having to go through the whole search process again.

Why It Matters

Performance Implications: Knowing that require() is synchronous helps in structuring your code for optimal performance, especially during the startup phase of an application. It encourages the practice of requiring modules at the beginning of files or in sections of the code where blocking the event loop has minimal impact on the application's responsiveness.

Understanding JavaScript's synchronous, single-threaded nature is key in Node.js to avoid blocking code. Node.js uses techniques like callbacks, promises, and async/await to enable asynchronous operations, preventing the event loop from being blocked. These methods allow Node.js applications to remain efficient and responsive, handling tasks in the background without slowing down the main execution flow.

Single-threaded means that a program or environment, like JavaScript in Node.js, can only execute one operation at a time on a single path of execution. Imagine it as a single-lane road where cars (operations) must go one after another, not side by side. This setup ensures that operations are carried out in sequence, but it also means that one operation must complete before the next one starts, preventing simultaneous execution of multiple operations.

Module State Sharing: Awareness of the caching mechanism is important when designing modules. Since the same instance of a module is shared wherever it's required, any state stored in the module is also shared. This can be leveraged to maintain application-wide state but requires careful management to avoid unintended side effects.

In programming, "state" refers to the current condition or data held by an application or a module at any given time. It's like remembering the score while playing a game; just as the score tells you how well you're doing at that moment, the state in programming keeps track of information like what's been clicked, entered into forms, or saved temporarily as the program runs. This state can change based on what actions are taken within the application, just like the game score changes when you score points or lose points.

It's important to distinguish this concept of "state" from a database (DB). While a database can store data that might represent part of an application's state, such as user profiles or application settings, the "state" in the context of programming generally refers to the in-memory condition of an application at a specific moment. This includes data that the application is currently working with but hasn't necessarily saved to a database.

Debugging: Understanding how require() works can aid in debugging issues related to module loading, such as why changes to a module don't seem to take effect (due to caching) or why an application may be blocking at startup (due to synchronous loading of modules).

Practical Exercises

To reinforce your understanding of the require() functionality and the caching mechanism, I recommend practising. Here is the Node.js sandbox, in case you need it..

Objective:

Your task is to create a simple Node.js application that demonstrates how the state is shared and managed across modules due to Node.js's caching mechanism. You will create a shared state module and two additional modules that modify and display the shared state, respectively.

Instructions:

1. Create the Shared State Module:

File Name: 'counterState.js'
Task: This module will maintain a counter. Write two functions within this module: one named incrementCounter to increment the counter and another named getCounter to retrieve the counter value. Export both functions.

The counter code will look something like this:

let count = 0;

function incrementCounter() {
  count++;
}

function getCounter() {
  return count;
}

module.exports = { incrementCounter, getCounter };

2. Create the First Additional Module:

File Name: 'incrementModule.js'
Task: This module will require 'counterState.js' and use its incrementCounter function to increment the counter, and the getCounter function to display the counter's current value. It should then log a message showing the new counter value. Export a function named increment that performs these tasks. You need to write this code yourself.

3. Create the Second Additional Module:

File Name: 'displayModule.js'
Task: This module will also require 'counterState.js' but will only use the getCounter function to display the current value of the counter. Export a function named display that logs the counter's current value. You need to write this code yourself.

4. Create the Entry Point File:

File Name: 'index.js'
Task: In this file, you will require both 'incrementModule.js' and 'displayModule.js'. Use their exported functions to increment the counter and display its value, demonstrating the shared state across modules. You need to write this code yourself.
Instructions: Increment the counter, display its value, and then increment it again to show the shared and persistent state. Then display the value again to see how it updates in the other file (module) that requires the 'counterState.js' module.

The working code should log the following:

By completing this homework, you will understand how Node.js caches imported modules, allowing shared state between different parts of your application. This knowledge is fundamental for efficient Node.js development, especially for applications relying on shared data across modules.

Here are the solutions to the above practical exercises.

Transferable Knowledge and Patterns

Understanding of Synchronous Operations: Recognising that require() operates synchronously, halting execution until a module is fully loaded, teaches the general principle of synchronous execution in programming. This concept is critical for understanding how different parts of a program can affect execution flow and performance, applicable in various programming languages and environments.
Importance of Caching for Performance: The caching mechanism of require() illustrates a fundamental performance optimisation technique used widely in computing. Caching, or temporarily storing data for quick access upon repeated requests, is a concept that can be applied in numerous scenarios beyond module loading, such as web content delivery, database queries, and more.
Module State Sharing and Global State Management: The idea that a module's state is shared across all parts of an application that import it underscores the broader concept of global state management. This is a key consideration in software design, relevant for understanding how state is managed across an application.
Performance Optimisation Practices: The advice to require modules at the beginning of files or in strategic code sections to minimise performance impact teaches a broader lesson on resource management and optimisation in software development. Prioritising resource-intensive operations and understanding their impact on the application's responsiveness and efficiency are valuable skills.
Debugging and Troubleshooting: The emphasis on how understanding require()'s behaviour aids in debugging extends to a general principle in programming: knowing how your tools and frameworks work under the hood can significantly improve your ability to troubleshoot and solve problems. This is a critical skill for any developer, as it applies to debugging in all programming languages and environments.

That said, understanding how to import modules in Node.js opens up a wealth of critical knowledge that you will learn once and use everywhere. It shows you how to avoid getting stuck.

You are more than welcome to explore these topics independently because it’s the best way to develop problem-solving skills. But keep in mind that we will gradually cover all these topics through my tutorials because that’s why tutorialhell.dev was created.

Learning path from here:

The concepts mentioned in the examples, such as function definitions, template literals, and destructuring, are frequently used, and thus I strongly encourage you to engage in some self-learning as well. Yes, to develop problem-solving skills. I cannot emphasize this enough.

Introduction to Objects: Learn about JavaScript objects, including creation, access, methods, property shorthand and computed properties.
Function Definitions: Explore different ways to define functions, including function declarations, function expressions, and arrow functions.
Template Literals: Learn about string interpolation and multi-line strings in ES6, using backticks to define template literals and ${} for embedding expressions.
Destructuring: Understand the destructuring assignment syntax for arrays and objects.
File Paths: Learn about relative and absolute file paths to grasp how to navigate and reference files within your projects effectively.

If you like this content, consider following me and subscribing to receive these cool tutorials as soon as they are ready.

The sunfish on the post cover was sourced from the O'Rly Generator.

Must-Have Practical Tutorial on Node.js Modules for Badass Beginners

Sandor | tutorialhell.dev — Mon, 19 Feb 2024 12:04:30 +0000

It's part of my tutorialhell.dev project, aimed at helping beginners learn easily without getting stuck.

Node.js supports two module systems: CommonJS modules and ECMAScript modules.

The CommonJS modules were the original method for packaging JavaScript code in Node.js.
The ECMAScript modules standard is utilized by browsers and other JavaScript runtimes and is generally viewed as the preferred approach for being future-proof. However, to get started, we will focus on CommonJS modules since they are still widely used. As we progress, I will introduce you to ECMAScript modules in the upcoming tutorials.

Let's break down the basics, starting with how to create a module in Node.js, how to export from it, and how to use the exported entities in other files.

If you don't have Node.js set up, you can get started using a ready-made Node.js environment in this sandbox.

Creating a Simple Module

Create a File: Every module is a JavaScript file. Create a file named 'myModule.js'.

Write Some Code: Inside 'myModule.js', define a function that you want to make available outside this file.

function sayHello(name) {
  return `Hello, ${name}!`;
}

Exporting from a Module

To make functions, objects, or values available outside the module where they are defined, you need to export them using module.exports.

module.exports is an object included in every JavaScript file in Node.js by default. It is used to export functions, objects, or values from a module. Here is the full code:

function sayHello(name) {
  return `Hello, ${name}!`;
}
module.exports = sayHello;

Importing a Module

To use the exported entities from a module in another file, you need to require that module.

Create Another File: 'index.js'. (It's already there if you are using the sandbox. Simply replace the contents with the code provided below.)

Require the Module: Use the require() function to import the module, passing the path to the module file as an argument:

const sayHello = require('./myModule');
console.log(sayHello('World')); // Outputs: Hello, World!

Explanation:

sayHello('World'): This line calls the imported function with the argument 'World', and the function returns the string 'Hello, World!'.

Multiple exports

Using module.exports

function func1() {
  console.log('Function 1');
}

function func2() {
  console.log('Function 2');
}

module.exports = {
  func1,
  func2,
};

Using exports

exports.func3 = function () {
  console.log('Function 3');
};

exports.func4 = function () {
  console.log('Function 4');
};

Multiple imports

Similarly, you would import these in another file like this:

const myModule = require('./multipleExport1');
const myModule2 = require('./multipleExport2');

Or, using destructuring:

const { func1, func2 } = require('./multipleExport1');
const { func3, func4 } = require('./multipleExport2');

Replace the contents of the 'index.js' file with this code:

const sayHello = require('./myModule');
console.log(sayHello('World')); // Outputs: Hello, World!

// Approach 1
console.log('------- Approach 1 -------');
const myModule = require('./multipleExport1');
const myModule2 = require('./multipleExport2');

myModule.func1(); // Calls 'Function 1'
myModule.func2(); // Calls 'Function 2'

myModule2.func3(); // Calls 'Function 3'
myModule2.func4(); // Calls 'Function 4'

// Approach 2
console.log('------- Approach 2 -------');
const { func1, func2 } = require('./multipleExport1');
const { func3, func4 } = require('./multipleExport2');

func1(); // Calls 'Function 1'
func2(); // Calls 'Function 2'
func3(); // Calls 'Function 3'
func4(); // Calls 'Function 4'

And you should see the following logged in your terminal:

Create a new file named 'multipleExport3.js' and paste the following code there:

exports.func3 = function () {
  console.log('Function 3 with an identical name from a different module');
};

exports.func4 = function () {
  console.log('Function 4 with an identical name from a different module');
};

Then add this code to the bottom of your 'index.js' file:

const {
  func3: renamedFunc3,
  func4: renamedFunc4,
} = require('./multipleExport3');
console.log('------- Destructuring with Renaming -------');
renamedFunc3();
renamedFunc4();

And your updated terminal output will look like this:

👉 Working code you can find here.

Choosing Between `module.exports` and `exports`

Use module.exports when you want to export a single item, like a function or an object, or when you prefer to define all exports at once in an object.

Both methods are commonly used and choosing between them often comes down to preference and the specific needs of your module.

Here’s a breakdown of the key feature that every beginner should grasp:

`require()` is Synchronous and Caches Modules

Or try it out yourself by pasting the following code in the sandbox.

Create a file named 'greet.js' and paste the following code there:

console.log('Loading greet.js...');

function greet() {
  console.log(
    'Hello and welcome to learning web development with www.tutorialhell.dev!'
  );
}

console.log('greet.js loaded successfully.');

module.exports = greet;

Create or update (it's already there in the sandbox) the 'index.js' file and paste the following code there:

console.log('Starting index.js...');

const greet = require('./greet'); // Synchronously loads and executes "greet.js"

console.log('greet.js has been required.');

greet();

console.log('Ending index.js...');

How it works:
When you run 'index.js' using the command node index.js, you will see the logs in the following order, demonstrating the synchronous nature of require():

Notice how the execution of 'index.js' pauses until 'greet.js' is fully loaded and executed (evidenced by the logs from 'greet.js' appearing before the continuation of 'index.js').

In simple terms, synchronous means doing things one by one in order, where you have to finish one thing before you can start the next thing.

In simple terms, caching means storing something in a place where it can be quickly accessed later. It's like saving a shortcut to find information faster without having to go through the whole search process again.

Why It Matters

Practical Exercises

To reinforce your understanding of the require() functionality and the caching mechanism, I recommend practising. Here is the Node.js sandbox, in case you need it..

Objective:

Instructions:

1. Create the Shared State Module:

File Name: 'counterState.js'
Task: This module will maintain a counter. Write two functions within this module: one named incrementCounter to increment the counter and another named getCounter to retrieve the counter value. Export both functions.

The counter code will look something like this:

let count = 0;

function incrementCounter() {
  count++;
}

function getCounter() {
  return count;
}

module.exports = { incrementCounter, getCounter };

2. Create the First Additional Module:

File Name: 'incrementModule.js'
Task: This module will require 'counterState.js' and use its incrementCounter function to increment the counter, and the getCounter function to display the counter's current value. It should then log a message showing the new counter value. Export a function named increment that performs these tasks. You need to write this code yourself.

3. Create the Second Additional Module:

File Name: 'displayModule.js'
Task: This module will also require 'counterState.js' but will only use the getCounter function to display the current value of the counter. Export a function named display that logs the counter's current value. You need to write this code yourself.

4. Create the Entry Point File:

File Name: 'index.js'
Task: In this file, you will require both 'incrementModule.js' and 'displayModule.js'. Use their exported functions to increment the counter and display its value, demonstrating the shared state across modules. You need to write this code yourself.
Instructions: Increment the counter, display its value, and then increment it again to show the shared and persistent state. Then display the value again to see how it updates in the other file (module) that requires the 'counterState.js' module.

The working code should log the following:

Here are the solutions to the above practical exercises.

Transferable Knowledge and Patterns

Understanding of Synchronous Operations: Recognising that require() operates synchronously, halting execution until a module is fully loaded, teaches the general principle of synchronous execution in programming. This concept is critical for understanding how different parts of a program can affect execution flow and performance, applicable in various programming languages and environments.
Importance of Caching for Performance: The caching mechanism of require() illustrates a fundamental performance optimisation technique used widely in computing. Caching, or temporarily storing data for quick access upon repeated requests, is a concept that can be applied in numerous scenarios beyond module loading, such as web content delivery, database queries, and more.
Module State Sharing and Global State Management: The idea that a module's state is shared across all parts of an application that import it underscores the broader concept of global state management. This is a key consideration in software design, relevant for understanding how state is managed across an application.
Performance Optimisation Practices: The advice to require modules at the beginning of files or in strategic code sections to minimise performance impact teaches a broader lesson on resource management and optimisation in software development. Prioritising resource-intensive operations and understanding their impact on the application's responsiveness and efficiency are valuable skills.
Debugging and Troubleshooting: The emphasis on how understanding require()'s behaviour aids in debugging extends to a general principle in programming: knowing how your tools and frameworks work under the hood can significantly improve your ability to troubleshoot and solve problems. This is a critical skill for any developer, as it applies to debugging in all programming languages and environments.

That said, understanding how to import modules in Node.js opens up a wealth of critical knowledge that you will learn once and use everywhere. It shows you how to avoid getting stuck.

Learning path from here:

Introduction to Objects: Learn about JavaScript objects, including creation, access, methods, property shorthand and computed properties.
Function Definitions: Explore different ways to define functions, including function declarations, function expressions, and arrow functions.
Template Literals: Learn about string interpolation and multi-line strings in ES6, using backticks to define template literals and ${} for embedding expressions.
Destructuring: Understand the destructuring assignment syntax for arrays and objects.
File Paths: Learn about relative and absolute file paths to grasp how to navigate and reference files within your projects effectively.

If you like this content, consider following me and subscribing to receive these cool tutorials as soon as they are ready.

The sunfish on the post cover was sourced from the O'Rly Generator.

Don't use vanilla JavaScript! I am serious!

Sandor | tutorialhell.dev — Wed, 28 Jul 2021 11:50:30 +0000

People often claim that frameworks are evil. It's a horrible statement.

Using vanilla you will find yourself doing things that frameworks have already done and tested for you.

If you think vanilla is better for simple projects, then you are missing the point of Frontend tools.

Any simple website is easy to build using a framework in a matter of minutes. Sure thing, you can do it with vanilla, but it will take x10+ longer.

Take static site generators. Not only do they address all SEO requirements, but they are also blazing fast out of the box. They have tons of integrations only to save you hours, if not days of work and countless bug fixes.

Easy and free deployment is a great bonus.

A good web developer knows the tools and uses them to be productive.

There is no way you can be productive reinventing the wheel.

The best devs to work with are pips who found the balance between delivering and enjoying coding.

Another crazy statement is to push beginners to learn fundamentals without any context. Will cover this problem in my next post.

Follow for more.

You don't know Redis (Part 2)

Sandor | tutorialhell.dev — Tue, 22 Jun 2021 19:26:32 +0000

In the first part of You don't know Redis, I built an app using Redis as a primary database. For most people, it might sound unusual simply because the key-value data structure seems suboptimal for handling complex data models.

In practice, the choice of a database often depends on the application’s data-access patterns as well as the current and possible future requirements.

Redis was a perfect database for a Q&A board. I described how I took advantage of sorted sets and hashes data types to build features efficiently with less code.

Now I need to extend the Q&A board with registration/login functionality.

I will use Redis again. There are two reasons for that.

Firstly, I want to avoid the extra complexity that comes with adding yet another database.

Secondly, based on the requirements that I have, Redis is suitable for the task.

Important to note, that user registration and login is not always about only email and password handling. Users may have a lot of relations with other data which can grow complex over time.

Despite Redis being suitable for my task, it may not be a good choice for other projects.

Always define what data structure you need now and may need in the future to pick the right database.

Implementation

I use serverless functions, the ioredis library and Upstash Serverless Redis.

I can’t help but talk about serverless all the time because it greatly simplifies development. I love when complexity is removed whenever possible and Upstash is doing just that for me.

I have zero work with setting up Redis. Moreover, I am using Upstash both in development and production.

Registration flow

During registration, we collect the user name, email and password. Before registering a user, we need to make sure that the email has not been registered already (is unique in the system).

Redis does not support constraints. However, we can keep track of all registered emails using a sorted set named emails.

On every new registration, we can use the ZSCORE command to check whether the provided email is already registered.

If the email is taken, we need to notify the user about it.

⚠️ Note, that this isn’t the best option because by telling that a given email is registered we provide a simple way for anyone to check whether someone is registered with a particular service, albeit it’s not a big security issue.

Before we can save a new user, we need to:

Generate a unique user ID.

We can use the INCR command to always get a unique value by incrementing a number stored at a key by one. If the key does not exist, Redis will set it to 0 before performing the operation. This means that the initial value will be 1.

const id = await redis.incr('user_ids') // -> 1

Whenever you need to create a counter, INCR is a great choice. Or you can build a rate-limiter to protect your API from being overwhelmed by using INCR together with EXPIRE.

Hash the password with the bcrypt library.

const hash = await bcrypt.hash(password, 10)

Now that we have the unique user ID (e.g. user ID is 7) and the hashed password, we can:
1. Store user details in a hash under the user:{ID} key.

redis.hmset('user:7', { 7, name, email, hash })

Knowing the ID, we can easily get all user details using the HGETALL command:

redis.hgetall('user:7');

2. Add the user’s email to the emails sorted set.

redis.zadd('emails', -Math.abs(7), email)

This allows us to lookup emails to check if they are registered or get the user's ID by email which is exactly what we need for the login process.

redis.zscore('emails', email) will return the score which is the ID or nil if the email is not found.

Notice how we use this sorted set for two important features, namely ensuring unique emails and looking up users by email.

But we are taking it one step further and set scores (which represent user IDs) as negative numbers to mark emails as unverified: -Math.abs(7). Then, when the email is verified, we simply convert it to a positive number.

redis.zadd('emails', Math.abs(7), email)

If a specified email is already a member of the emails sorted set, Redis will update the score only.

During the login process, we can always check for negative numbers and request users to verify their email instead of logging them in.

Retrieving all unverified emails is a trivial operation done with the ZRANGEBYSCORE command.

redis.zrangebyscore('emails', '-inf', -1, 'WITHSCORES');

Registration function source code

Login flow

Before logging in the user, we check if the provided email exists in our database. As mentioned before, the score is the user ID.

const userId = await redis.zscore('emails', email);

If so, we first check if the email is verified by making sure the ID is a positive number. If not, we ask users to verify their email.

If the email is verified, we get the password hash that we stored for the user:

const hash = await redis.hget('user:7', 'hash');

and check whether the password is correct:

const match = await bcrypt.compare(password, hash);

If the password is correct, we generate a token and return it to the client.

And we are done.

Conclusion

As you can see, we needed four Redis commands for registration and only two for login.

Probably you noticed that while describing the registration and login process with Redis we also revealed two more use cases for Redis, namely counter and rate-limiting.

Redis has a lot more use cases beyond cache and learning about them will only make you even more efficient.

Follow me to read about how I am implementing a secure production-ready registration flow with email verification and password recovery backed by Redis.

Check out my article on how I implemented the LinkedIn-like reactions with Serverless Redis.

You don't know Redis

Sandor | tutorialhell.dev — Wed, 02 Jun 2021 20:09:22 +0000

In my previous post, I touched on the point that Redis is more than just an in-memory cache.

Most people do not even consider Redis as a primary database. There are a lot of use cases where Redis is a perfect choice for non-cache related tasks.

In this article, I will demonstrate how I built a fully functional Q&A board for asking and upvoting the most interesting questions. Redis will be used as a primary database.

I will use Gatsby (React), Netlify serverless functions and Upstash Serverless Redis.

Upstash has been a good choice so far and I decided to try it out in a more serious project. I love everything serverless and how it makes things simpler for me.

Serverless will be a great choice for most tasks however you need to know the pros and cons of the tech you are using. I encourage you to learn more about serverless to get the most out of it.

Q&A board features

As you may know, I run a tech newsletter for recruiters where I explain complex tech in simple terms. I have an idea to collect questions from recruiters using a Q&A board and let them vote for questions.

All questions will eventually be answered in my newsletter, however, the most upvoted questions will be addressed first.

Anyone can upvote a question and registration is not required.

Questions will be listed in three tabs:

Active - questions sorted by votes and available for voting.
Most recent - questions sorted by date (newest first).
Answered - only questions that have answers.

Upvoting will be one of the most frequently used features and Redis has a data type and optimized commands for it.

A Sorted set is ideal for this task because all its members are automatically sorted by the score.

Scores are numeric values that we will associate with votes. It is very easy to increment a score (add a vote) by using the ZINCRBY command.

We will also leverage scores for handling unmoderated questions by setting the score for them to 0. All approved questions will have a score of 1+.

It allows us to fetch all unmoderated questions by simply using the ZRANGEBYSCORE command specifying the min and max arguments as 0.

To fetch all approved questions sorted by the score (highest first) we can use the ZREVRANGEBYSCORE command setting the min score argument to 1.

This is great that by using just a few Redis commands we can also solve logical tasks along the way. Lower complexity is a huge benefit.

We will also use sorted sets for sorting questions by date or filtering questions that have answers. I will explain it in more detail in a moment.

Less frequent operations, namely creating, updating and deleting questions are also easy to accomplish using hashes.

Implementation details

The most interesting part is always the actual implementation. I use serverless functions and the ioredis library and I will link the source code explaining what it does.

This article is dedicated to client-facing functionality. Although I will explain admin-related functions, in the final source code there will be no backend interface. You will need to use Postman or a similar tool to call the admin related endpoints.

Let’s take a look at the API endpoints and what they do.

Add a question

Users can create questions. All questions require moderation before they become visible.

A question is an object and Redis hash is a perfect data type to represent objects.

This is the structure of a questions:
{"datetime":"1633992009", "question":"What are Frontend technologies?", "author":"Alex", "email":"alex@email.com", “score:” “0”, “url”: “www.answer.com” }

We will store questions in hashes using the HMSET command which takes a key and multiple key-value pairs.

The key schema is question:{ID} where ID is the question ID generated using the uuid library.

This is a new question and there is no answer yet. We skip the url property but it will be an easy task to add it later using the HSET command.

The score for a newly created question is 0 by default. By our design, it means that this question needs moderation and will not be listed because we only fetch questions with scores starting from 1.

Since we keep the score value in a hash, we’ll need to update it whenever it changes. There is a HINCRBY command that we can use to easily increment values in hashes.

As you can see, using Redis hashes solves a lot more for us than just storing data.

Now that we know how we’ll store questions, we also need to keep track of questions to be able to fetch them later.

For that, we add the ID of a question to a sorted set with a score of 0 using the ZADD command. A sorted set will allow us to fetch question IDs sorted by scores.

As you can see, we are setting the score to 0 just like we do it for the score property in the hash above. The reason why we duplicate the score in a hash is that we need it when showing the most recent questions or questions that have answers.

For instance, the most recent questions are stored in a separate sorted set with timestamp as a score hence the original score value is not available unless it’s duplicated in a hash.

Since we store the score in two places, we need to make sure that values are updated both in a hash and in a sorted set. We use the MULTI command to execute commands in a manner where either all commands are executed successfully or they are rolled back. Check Redis Transactions for more details.

We will use this approach where applicable. For example, HMSET and ZADD will also be executed in a transaction (see source code below).

ZADD command takes a key and our schema for it is questions:{boardID}

All questions are mapped to a boardID. For now, it’s a hardcoded value because I need one board only. In the future, I may decide to introduce more boards, for example, separately for Frontend, Backend, QA and so on. It’s good to have the needed structure in place.

Endpoint:
POST /api/create_question

Here is the source code for the create_question serverless function.

Approve a question

Before a question becomes available for voting, it needs to be approved. Approving a question means the following:

Update the score value in hash from 0 to 1 using HINCRBY command.
Update the score value in the questions:{boardID} sorted set from 0 to 1 using the ZADD command.
Add the question ID to the questions:{boardID}:time sorted set with the timestamp as the score to fetch questions sorted by date (most recent questions) using the same ZADD command.

We can get the timestamp by looking up the question by its ID using the HGET command.

Once we have it, we can execute the remaining three commands in a transaction. This will ensure that the score value is identical in the hash and the sorted set.

To fetch all unapproved questions the ZRANGEBYSCORE command is used with the min and max values as 0.

ZRANGEBYSCORE returns elements ordered by a score from low to high while ZREVRANGEBYSCORE - from high to low. We’ll use the latter to fetch questions ordered by the number of votes.

Endpoint for fetching all unapproved questions:
GET /api/questions_unapproved

Endpoint for approving a question:
PUT: /api/question_approve

Here is the source code for the questions_unapproved serverless function. For the most part, this code is similar to other GET endpoints and I will explain it in the next section.

Here is the source code for the question_approve serverless function.

Fetch approved questions

To fetch all approved questions we use the ZREVRANGEBYSCORE command setting the min argument to 1 in order to skip all unapproved questions.

As a result, we get a list of IDs only. We will need to iterate over them to fetch question details using the HGETALL command.

Depending on the number of questions fetched, this approach can become expensive and block the event loop in Node (I am using Node.js). There are a few ways to mitigate this potential problem.

For example, we can use ZREVRANGEBYSCORE with the optional LIMIT argument to only get a range of elements. However, if the offset is large, it can add up to O(N) time complexity.

Or we can use a Lua script to extend Redis by adding a custom command to fetch question details based on IDs from a stored set without us doing it manually in the application layer.

In my opinion, it would be overhead in this case. Besides that, one must be very careful with Lua scripts because they block Redis and you can’t do expensive tasks with them without introducing performance degradation. This approach may be cleaner however we would still use the LIMIT to avoid large amounts of data.

Always research the pros and cons before the final implementation. As long as you understand the potential issues and have evaluated ways to mitigate them, you are safe.

In my case, I know that it will take significant time before I will have enough questions to face this issue. No need for premature optimization.

Endpoint:
GET /api/questions

Here is the source code for the questions serverless function.

Vote for a question

The process of upvoting a question consists of two important steps that both need to be executed as a transaction.

However, before manipulating the score, we need to check if this question has no answer (url property). In other words, we do not allow anyone to vote for questions that have been answered.

The vote button is disabled for such questions. But we do not trust anyone on the internet and therefore check on the server if a given ID exists in the questions:{boardID}:answered sorted set using the ZSCORE command. If so, we do nothing.

We use the HINCRBY command to increment the score in the hash by 1 and the ZINCRBY command to increment the score in the sorted set by 1.

Endpoint:
PATCH /api/question_upvote

Here is the source code for the question_upvote serverless function.

Fetch most recent approved questions

It’s very similar to how we fetch all approved questions with the only difference being that we read another sorted set where the key schema is questions:{boardID}:time. Since we used the timestamp as a score, the ZREVRANGEBYSCORE command returns IDs sorted in descending order.

Endpoint:
PATCH /api/questions_recent

Here is the source code for the questions_recent serverless function.

Update a question with an answer

Updating or adding new properties to hashes is simple with the HSET command. However, when we add an answer, we move the question from the questions:{boardID} sorted set to the questions:{boardID}:answered one preserving the score.

To do so, we need to know the score of the question and we obtain it using the ZSCORE command. Answered questions will be sorted by score in descending order.

Then we can:

update the hash with the url property using the HSET command;
add the hash to the questions:{boardID}:answered sorted set using ZADD;
remove the question from the questions:{boardID} sorted set running the ZREM command.
remove the question from the questions:{boardID}:time sorted set running the ZREM command.

All four commands are executed in a transaction.

Endpoint:
PATCH /api/question_add_answer

Here is the source code for the question_add_answer serverless function.

Fetch questions with answers

Again, the process is similar to fetching all approved questions. This time from the questions:{boardID}:answered sorted set.

Endpoint:
PATCH /api/questions_unswered

Here is the source code for the questions_unswered serverless function.

Full source code.
Working DEMO on my website.

Conclusion

Redis has a lot of use-cases going way beyond cache. I’ve demonstrated only one of the multiple applications for Redis that one can consider instead of reaching for an SQL database right away.

Of course, if you already use a database, adding yet another one may be an overhead.

Redis is very fast and scales well. Most commercial projects have Redis in their tech stack and often use them as an auxiliary database, not just in-memory cache.

I strongly recommend learning about Redis data patterns and best practices to realize how powerful it is and benefit from this knowledge in the long run.

Check my previous article where I created LinkedIn-like reactions with Serverless Redis if you haven’t already.

Here is You don't know Redis (Part 2)

Follow for more.

How to create LinkedIn-like reactions with Serverless Redis

Sandor | tutorialhell.dev — Wed, 19 May 2021 11:12:31 +0000

As a side hustle, I teach tech recruiters web and software development technologies using plain English. It helps them with understanding job specs and resumes and it makes all of us, tech people, happier.

I run a weekly newsletter and often get feedback from recruiters via email or LinkedIn DMs.

I thought that I could try to collect feedback using the “Reactions” feature just like LinkedIn or Facebook does. It’s not as informative as personalised messages but is a simple feature that may incentivize more people to provide some general feedback.

Either way, it’s worth trying and as a software developer, I can’t wait to implement it.

This tutorial is about implementing a feature that will be used in real life on my project.

Planning

As with any feature or project, we start with the planning phase.

I am going to stick with LinkedIn-like reactions because they are more appropriate for the type of content I post.

I will use the Like, Insightful and Curious reactions and will substitute them with 👍, 💡 and 🤔 emojis respectively.

My static website is built with Gatsby. I do not use a database but I need one to store reactions. I need to decide what database to use.

I deploy with Netlify and I use functions for backend related functionality. That means that I go serverless. Ideally, I need a serverless database too to not have to deal with deploying my own DB or overpaying for PaaS with fixed plans.

As of writing this article, I am using Netlify's free tier and can easily go for more features with a paid plan at a very good price. Using a reasonably priced database would be a perfect complement to my current tech stack.

These are the tasks:

1. Research database options.

I need to find out what serverless databases exist and choose one.

2. Create a Serverless Backend with functions for:

storing reactions and incrementing/decrementing the count
fetching reactions by a post id

3. Create the "reactions" component.

Gatsby is based on React and I will build a simple “reactions” component.

4. Put it all together to make my static website a little bit dynamic.

Static websites can have dynamic features and it’s what makes them, static site generators so incredibly awesome.

In this tutorial, I will focus on the first two tasks. The “reactions" component implementation you can check in the source code

Research

This second phase in Software Development Life Cycle (SDLC) is named Prototyping but I call it Research in this tutorial because I will skip the proof of concept (POC) part.

Research is always fun because it provides a great opportunity to learn about new technologies. While this process is interesting, it can also take a lot of time if we do not make our research more specific.

SQL and NoSQL are the most common database types. The choice isn’t difficult if we know what data will be stored. Let’s quickly take a look at what data structure we will have.

Every post has a set of reactions and we need to count those reactions.

Since I simply want to get an idea about how people react to my posts, I will not require them to log in or limit the types of reactions.

Based on the above, our data structure could look as follows for a post with 3 likes, 12 insightful and 7 curious reactions: { "like":3, "insightful":12, "curious":7 }

A NoSQL database looks like a good choice for storing and manipulating a data structure like the above.

I immediately thought of MongoDB and Redis. However, based on how we will manipulate data, namely persist it once and then increment or decrement certain values, I decide in favor of Redis.

The reason being is that Redis has built-in and performance-optimized commands to support what we need.

Besides that, I found a serverless Redis database Upstash which looks simple and has reasonable pricing including a free plan. I like starting free and paying as I scale.

Note that we are using Redis as a primary database. Redis can be configured to write data to disk which provides a degree of data safety comparable to what PostgreSQL offers.

Redis solves a much wider range of problems than just in-memory caching and can be used either as a primary database or as an additional database for solving problems that other databases struggle with.

I like that Upstash enables persistence by default keeping data both in memory and disk. This removes the headache of configuring things which would be an overhead for a task like this one. This is why I always use serverless and PaaS whenever possible.

To sum up this part, let me share with you an interesting free e-book called “Redis in Action” packed with valuable information and use-cases for Redis.

Serverless Backend

I will use Netlify’s serverless functions with Node instead of creating my own backend. You are free to use any backend architecture.

The easiest way to connect Redis with Upstash is to use the redis-client as described here.

First things first, we need to create a Serverless Redis database following this Getting Started guide.

Please, note that I leave out the Strong Consistency Mode, because Eventual Consistency is suitable for my task.

This is the schema for naming the keys: post:{id}:{reaction}

post:{id}:like -> post:856f9d0a:like
post:{id}:insightful -> post:856f9d0a:insightful
post:{id}:curious -> post:856f9d0a:curious

We could go with the most basic kind of Redis value known as Strings. But we’ll go with hashes because we want to store objects as value and it is advisable to use hashes when possible.

This is how we do it for a given post ID:

To increment a reaction by 1, for example, “Like”, we will use the HINCRBY command. If the key does not exist, it will create the key and set its value to 0 before incrementing. If the key does exist, it will simply increment by the value we provide. It allows us to reuse this function both for creating and updating reactions.

HINCRBY post:856f9d0a:reactions like 1

Here is the implementation

If we had to decrement reactions for cases when a reaction is removed or changed, we would simply pass the value to be incremented by a negative number: -1

To read reactions for a post ID we’ll use the HGETALL command which returns key-value pairs:

HGETALL post:856f9d0a:reactions -> // will return {"like":"3","insightful":"1","curious":"2"}

Here is the implementation

With this approach, knowing the post ID, we can efficiently set, update and retrieve reactions for a post with a ridiculously small amount of code.

Post IDs are not short but we will use them for naming keys because they will not cause any noticeable memory usage increase in our case.

But you should always keep in mind that along with having a readable key naming schema, you need to control the length of keys. Long keys can use more memory and even cause performance implications as described in the Redis keys section.

Off-topic

Even though this is not directly related to this tutorial, I know that later I will want to show the most popular posts based on reactions.

To get posts with the most Likes, Insightful and Curious reactions, we need to track them using a sorted set.

For instance, this is how we store a post that received a Like reaction:
ZINCRBY reaction:like 1 post:856f9d0a

And this is how we get the most liked 5 posts:
ZREVRANGEBYSCORE reaction:like +INF -INF withscores LIMIT 0 5

I will provide more details and a working implementation in one of my next posts.

It’s always a good idea to design a system taking into account all known future requirements and choose technologies that will support them in the future.

Here is a working implementation on my website. Pick any post and you will find reactions at the bottom. The source code you can find here

Conclusion

If you still perceive Redis as an in-memory cache, get ready because I have more posts coming up that cover powerful features backed by this amazing database.

The next post will be about how I built a Q&A board for asking and upvoting the most interesting questions using Redis.

Follow for more!

How to get out of tutorial hell and avoid the imposter syndrome

Sandor | tutorialhell.dev — Thu, 28 Jan 2021 12:23:10 +0000

Taking tutorials may be exciting at the beginning. But it doesn't take long to realize that what you've learned in a tutorial is hard to put into practice.

The problem with most tutorials is that they teach you how to solve a specific small task, often a tiny one. It's impossible to translate such knowledge onto a larger task, let alone a project.

Imagine if your driving instructor would tell you to look at a smashed insect on the windshield instead of looking through it to see the road ahead. You would end up in trouble in a matter of seconds.

The same happens with tutorials. You can't learn how to build a project by taking tutorials that teach you how to create an HTML form and submit it using an API endpoint, which is magically available...

It just doesn't work.

Most of us have a vague idea about how exactly cars function, but we know about the engine, about the steering wheel, accelerator, breaks, gearshift, etc and what those things are for.

If someone told you to repair a flat tyre, you'd either do it yourself or call a service to do it for you. Either way, you will get it done. Simply because you know what this part is and where to look for a solution.

Similarly, in software development, you can figure out how to build a feature if you understand its role in the system. You can ask the right questions if you hesitate. You can develop it yourself or decide to go for a third-party solution.

Learning web development should start with a general overview of all the building blocks. For example, Frontend, Backend, Databases, testing and deployment as well as the Software Development Life Cycle (SDLC).

Knowing how software is developed gives a very good idea about the necessary steps to take to build reliable software and unlock Continuous Integration and Delivery (CI/CD) that all companies mention in job specs.

With this knowledge, it will be so easy to develop the right mindset - the DevOps mindset.

Why would a beginner need it? Those things are relatively easy to explain. They open up a whole different perspective. It's like seeing an ocean instead of a drop.

Once you have a general understanding of where you are heading, you can start learning every bit in more detail.

Another crucial point is to learn in a reverse order. First, you define what you want to build and then learn what is needed to build it. This way you can see how code maps to either what you see on the screen or the result you expect.

The logical first step for a web developer is to learn Frontend technologies: HTML, CSS and JavaScript.

Let's take HTML and CSS as an example. It's not only about the tags and styles. It's also about structures, performance, CSS preprocessors and CSS frameworks.

Beginners tend to keep all styles in one file until it grows huge and becomes hard to maintain.

Then they start to realize they are missing something. But what is it?
If they were told about CSS methodologies that help with writing modular, reusable and scalable code, beginners would quickly understand the idea behind them. Instead of struggling, they would know what to learn next to fix the problem.

Nobody explains to beginners, that animations can be done both with CSS and JavaScript and that depending on the task, one can be faster than the other.

Just like with CSS methodologies, knowing that animation performance depends on the implementation will help beginners with finding a solution when they run into performance issues. Simply, by googling "CSS Versus JavaScript Animations" one can find useful resources that explain this topic in depth.

The same is true with understanding reflow and repaint in the browser and how it's affecting web performance. When solving a performance problem, beginners will find the solution because they know that it can be one of the reasons causing the issue.

Knowing the limitations of CSS, beginners will quickly get the idea of CSS preprocessors and appreciate the benefits they bring. This will naturally lead to learning the build tools and task runners.

CSS frameworks are great but only when you understand when and why using them.

It is so much easier to absorb knowledge when you see how it applies in practice by solving a real problem.

When it comes to JavaScript, not knowing about security issues is the same as driving a car without learning the driving code. Many beginners think security is only Backend related...

Algorithms are a must because they are closely related to the performance and reliability of the system. And they are easy to understand when used in practice.

Design patterns are a battle-tested solution to common problems and beginners need to know what they are for to avoid reinventing the wheel.

Beginners need to have a rough idea about all of that I mentioned above. When they need to deal with security, they will know about the OWASP top 10 and will explore it. Then put new knowledge into practice.

When it comes to solving performance issues in javascript, they will doubt data structures they use and reach for resources on that topic.

And instead of reinventing the wheel, they will first check if there are existing patterns that solve what they need.

The bottom line is that it's not about how much you learn. It's about knowing what's out there that you may need to explore to solve the problem.

The scope of web development is the critical knowledge the most beginners do not have, unfortunately.

It is exactly what I am solving with my free beginner-friendly tutorial on creating Trello clone.

I explain every step by providing details on what is needed to build large-scale apps. I let beginners run into issues and help them with finding a solution.

I teach beginners web development concepts, best practices and patterns that they will apply in any project.

I help to avoid or get out of tutorial hell as well as avoid the imposter syndrome.

Comment if you'd like to take my tutorial. It's free.

What is one of the most common mistakes beginner developers make

Sandor | tutorialhell.dev — Mon, 18 Jan 2021 08:40:08 +0000

It may seem that when you are a beginner, you'll do simple things only. No need to learn data structures and algorithms. No need to understand Big O notation, complexity and stuff like that.

This couldn't be further away from the truth!

In 2008, when I just started learning to program, I spent a lot of time reading books on PHP and MySQL. Months later, when I felt confident, I took my first freelance project. It was a real estate website. A simple one. I used a custom-made ORM and everything worked just fine!

When I released it, the search feature quickly became sluggish and made the website unusable.

I was wondering what the heck had happened. I figured out that database queries became very slow when there were over 200 real estate objects added to it.

This is it. What worked fine during testing did not work in real life.

I was a self-taught developer. I did not know how to measure if my project scaled well. I didn't even know that I had to do it.

I thought algorithms mattered only for launching a spaceship.

If I had some basic understanding of algorithms, I would have known that the more the input, the longer it takes.

I am not saying I would have come up with a robust solution as a junior, but I would have looked for a solution because I knew there would be a problem.

Please, don't make the same mistake!

Of course, data structures and algorithms are much more than that and they apply differently depending on what you work on.

But a basic understanding of data structures and algorithms is a must for every software developer.

I am writing about my experience of what I wish I knew when I was a beginner and I review programming courses to find those that are worth learning.

If you are a junior or a beginner and you want to know what you need to learn, I can help!

I am sharing my knowledge in a newsletter here https://ns.comparecourses.dev

Please, don't make the same mistake

Sandor | tutorialhell.dev — Thu, 14 Jan 2021 13:47:22 +0000

It may seem that when you are a beginner, you'll do simple things only. No need to learn data structures and algorithms. No need to understand Big O notation, complexity and stuff like that.

This couldn't be further away from the truth!

When I released it, the search feature quickly became sluggish and made the website unusable.

I was wondering what the heck had happened. I figured out that database queries became very slow when there were over 200 real estate objects added to it.

This is it. What worked fine during testing did not work in real life.

I was a self-taught developer. I did not know how to measure if my project scaled well. I didn't even know that I had to do it.

I thought algorithms mattered only for launching a spaceship.

If I had some basic understanding of algorithms, I would have known that the more the input, the longer it takes.

I am not saying I would have come up with a robust solution as a junior, but I would have looked for a solution because I knew there would be a problem.

Please, don't make the same mistake!

Of course, data structures and algorithms are much more than that and they apply differently depending on what you work on.

But a basic understanding of data structures and algorithms is a must for every software developer.

I am writing about my experience of what I wish I knew when I was a beginner and I review programming courses to find those that are worth learning.

If you are a junior or a beginner and you want to know what you need to learn, you might find my newsletter useful https://ns.comparecourses.dev

How to track change to any content with FaunaDB's Temporality feature

Sandor | tutorialhell.dev — Wed, 19 Aug 2020 18:36:48 +0000

This is the third article from the series of articles on how I build an MVP with Gatsby and FaunaDB. This time we will look at yet another FaunaDB’s built-in feature called Temporality. In simple terms, it helps you track how exactly data has changed over time.

The simplest use-cases are content moderation or versioning. In our MVP, authors can create and update courses and we want to see what changed and approve course content before it goes live. Luckily, we don't need to build our own implementation because FaunaDB offers this feature on a database level out of the box.

This article is based on the previous two articles. If you missed them, it’s best to start with:

the first article where we connect our starter project to FaunaDB and fetch data at build time
and the second article where we implement user authentication with FaunaDB's built-in Login function, ABAC and also explore FQL together with user-defined functions (UDF).

Schema update

Our schema has changed again. Here it is:

type Course {
 title: String!
 description: String
 visible: Boolean!
 author: User!
 bookmarks: [Bookmark!] @relation
}

type Bookmark {
 title: String
 private: Boolean!
 user: User!
 course: Course!
}

type User {
 name: String!
 email: String!
 role: Role!
 courses: [Course!] @relation
 bookmarks: [Bookmark!] @relation
}

input CreateUserInput {
 name: String!
 email: String!
 password: String!
 role: Role!
}

input LoginUserInput {
 email: String!
 password: String!
}

input LogoutInput {
 allTokens: Boolean
}

type AuthPayload {
 token: String!
 user: User!
}

type CourseUpdates @embedded {
 title: String
 description: String
 visible: Boolean
}

type HistoryUpdate @embedded {
 ts: Long!
 action: String!
 data: CourseUpdates
}

type HistoryPage @embedded {
 data: [HistoryUpdate]
}

type Query {
 allCourses: [Course!]
 allBookmarks: [Bookmark!]
 allUsers(role: Role): [User!]
 allCoursesInReview(visible: Boolean = false): [Course!]
 courseUpdateHistory(id: ID!): HistoryPage
   @resolver(name: "course_update_history")
}

type Mutation {
 createUser(data: CreateUserInput): User! @resolver(name: "create_user")
 loginUser(data: LoginUserInput): AuthPayload! @resolver(name: "login_user")
 logoutUser(data: LogoutInput): Boolean! @resolver(name: "logout_user")
}

enum Role {
 AUTHOR
 DEVELOPER
 MANAGER
}

We added:

the visible property to Course type to make sure that only visible (approved) courses will be listed;
the allCoursesInReview query to list all courses that require approval based on the value of the visible property;
the courseUpdateHistory query to list all changes made to a particular course that we can look up by its _id;
The MANAGER role

We also use the @embedded directive on the return type for the courseUpdateHistory query so that FaunaDB does not create collections for those types.

Copy the above schema into a file named schema.gql and apply it with FaunaDB as shown on the screenshot below:

Since we added the visible property as non-nullable (visible: Boolean! - the exclamation mark means that the field is non-nullable, meaning that there must be always a value) and none of our courses has it, we need to set it for all courses to avoid getting the "Cannot return null for non-nullable type" GraphQL error. We need to make a small change to our existing data to accommodate the schema update which we can do in pure FQL. Copy and paste the following code into the Shell and run it:

Map(
  Paginate(
    Match(Index("allCourses"))
  ),
  Lambda("X", 
    Update(
      Select("ref",  Get(Var("X"))),
      { data: { visible: true }})
    )
)

Author role

Before we see how FaunaB’s temporal feature works, we need a way to create and update courses as authors would. While we could do it via the GraphQL playground, it’s always better to see a real-life example.

Authors can log in, but we do not have an Author role yet. Let’s create it and define what privileges we will give to authors.

Here is our FQL for the AUTHOR role:

CreateRole({
  name: "Author",
  membership: [
    {
      resource: Collection("User"),
      predicate: Query(
        Lambda(
          "userRef",
          Equals(Select(["data", "role"], Get(Var("userRef"))), "AUTHOR")
        )
      )
    }
  ],
  privileges: [
    {
      resource: Collection("User"),
      actions: {
        read: true
      }
    },
        {
          resource: Collection("Bookmark"),
          actions: {
            read: Query(
              Lambda(
                "bookmarkRef",
                Let(
                  {
                    bookmark: Get(Var("bookmarkRef")),
                    private: Select(["data", "private"], Var("bookmark"))
                  },
                  Equals(Var("private"), false)
                )
              )
            )
          }
        },
    {
      resource: Collection("Course"),
      actions: {
        read: true,
        write: Query(
          Lambda(
            ["oldData", "newData"],
            And(
              Equals(Identity(), Select(["data", "author"], Var("oldData"))),
              Equals(Select(["data", "visible"], Var("newData")), false),
              Equals(
                Select(["data", "author"], Var("oldData")),
                Select(["data", "author"], Var("newData"))
              )
            )
          )
        ),
        create: Query(
          Lambda(
            "data",
            And(
              Equals(Identity(), Select(["data", "author"], Var("data"))),
              Equals(Select(["data", "visible"], Var("data")), false)
            )
          )
        )
      }
    },
    {
      resource: Index("allUsers"),
      actions: {
        read: true
      }
    },
    {
      resource: Function("logout_user"),
      actions: {
        call: true
      }
    },
    {
      resource: Index("allCourses"),
      actions: {
        read: true
      }
    },
    {
      resource: Index("course_author_by_user"),
      actions: {
        read: true
      }
    },
    {
      resource: Index("bookmark_user_by_user"),
      actions: {
        read: true
      }
    }
  ]
})

We allow the AUTHOR role to create and edit own courses only, see all courses and public bookmarks and log out.

What’s interesting to note is that the visible property in the Course type is false by default for new and updated courses which means that the course is “In review”. The author can’t override this value. We are checking for it in the predicate function in these lines:

for “Create” action: Equals(Select(["data", "visible"], Var("data")), false)
and for the “Write” action: Equals(Select(["data", "visible"], Var("newData")), false)

We can safely set the visible property to false on the client-side because we know that even if the value is changed intentionally, it will not pass the ABAC. This means that we do not need to create a custom resolver or a backend function to set this value manually.

We will test it in a minute.

Copy and paste the above FQL into the Shell and run it.

Now we need to clone the repository with the latest changes for this article.
Note that you will need the .env.development and .env.production files containing the bootstrap key and other variables that we added in the previous article

Follow these steps:

git clone --single-branch --branch article-3/temporality git@github.com:sandorTuranszky/Gatsby-FaunaDB-GraphQL.git gatsby-fauna-db
cd gatsby-fauna-db
copy the .env.development and .env.production files
npm install
gatsby develop to start the project in development mode

You should see the MVP up and running.

If you see an error, make sure that the token cookie and the user_data in local storage have been removed or remove them manually.

And you will be redirected to the following page /app/courses:

Now we can create new courses or update an existing one.

Let’s create a new course:
Title: “Node.js Masterclass”
Description: “We’ll cover some of the topics including integrating Node.js with Express and asynchronous code”

We can see that the newly created course is marked “In review”. As it was mentioned before, all new or updated courses have a default false value for the visible property. This means that the course needs to be reviewed.

The default false value is set on the client and is controlled by ABAC. You can test it by changing the default value to true in the mutation in /src/components/updateCourse.js file:

and try to update the “React for beginners” course. You will get an error:

Now, revert the default value of the visible prop to false and click “Update”

You can see that the updated course is now marked as (In review) too.

Great, we have created and updated courses using the author account. This is what we needed to be able to test out the temporal feature.

Temporality

FaunaDB’s Temporality has two features: Snapshots and Events.

Snapshots

Snapshots allow us to see the state of our data at a particular point in time. We have just added two new courses and updated one course above. If you run the following FQL in the Shell, you will see all the 7 courses in their latest state including the newly created “Node.js Masterclass” course:

Map(
  Paginate(
    Match(Index("allCourses"))
  ),
  Lambda("X", 
    Let({
      ts: Select("ts", Get(Var("X"))),
      data: Select("data", Get(Var("X")))
    },
    {
      ts: Var("ts"),
      data: Var("data")
    }
    )
  )
)

How can we learn which articles we just created and updated? We will use the Snapshots feature to travel back in time to see our data before the updates that we made.

To use the At function we need a timestamp. We can get it from the above query which returns a ts property for each course (see the first arrow on the screenshot above).

We know that we did not update the course with the title “NodeJS Tips & Tricks” (we added it when following the previous article) hence we can assume that its timestamp represents a state before the changes that we just made. Let’s check.

Run the following FQL in the Shell.

Note that you need to copy the timestamp from your list in the Shell above (ts property is above the data property for each course)

At(
  1558455784100000,
  Map(
    Paginate(
      Match(Index("allCourses"))
    ),
    Lambda("X", Get(Var("X")))
  )
)

And you will see a list of courses without the newly added “Node.js Masterclass” course.

We remember that we created the “Node.js Masterclass” course first and then updated the “React for beginners” course. This means that with the timestamp of the “Node.js Masterclass” course, we can get the state for the “React for beginners” course, where the visible property is true and without the description property.

Copy the timestamp for the “Node.js Masterclass” course from the query result that we made to list all courses and run the following FQL in the Shell:

At(
  1594566013530000,
  Map(
    Paginate(
      Match(Index("allCourses"))
    ),
    Lambda("X", Get(Var("X")))
  )
)

You will see that the visible property is true and the description property is nowhere to be seen for the “React for beginners” course. The “Node.js Masterclass” course is also listed.

Cool, right?

Events

FaunaDB creates a new copy of the document containing all the changes we’ve made. The original document is never changed. It means that FaunaDB has at least two copies of the “React for beginners” course, one where the visible property is true and one where it’s false and the description property is available.

Since we ran a script to update the visible property for all courses, we will have more copies. To see them for the “React for beginners” course, run the following FQL:

Paginate(
  Events(
    Select(
      "ref",
      Get(Match(Index("course_by_title"), "React for beginners"))
    )
  )
)

The above query will list all copies created as a result of changes that were made to the above-mentioned course. I have four copies. The first change has an action “create” when the document was created and the other three have action “update”.

You should see the first item in the list without the visible property. Then, one copy with the visible property set to true and the last copy with the visible property set to false and the description property as well. This is exactly how we changed it.

I have one more copy which does not represent any meaningful change and the timing of the change tells me that the copy was created when we tried to manually set the default value for the visible property to true and got an error response.

Note that here we looked up the course by the name for simplicity. In our UDF we will look up courses by their _id field.

Now, when we’ve seen how the temporal features work, let’s use it for our MVP.

UDF for listing changes for a particular course

We need a custom resolver to return all changes that have been made to any given course. Copy and paste the following FQL and run it in the Shell:

Update(
  Function("course_update_history"),
  {
    "body": Query(
      Lambda(["id"],
        Let({
          page: Paginate(
            Events(
              Select(
                "ref",
                Get(Ref(Collection("Course"), Var("id")))
              )
            )
          )
        },
          Var("page")
        )
      )
     )
  }
)

Now the courseUpdateHistory mutation has a revolver. To test it out in our MVP, we need to create a MANAGER role and a few manager users.

The MANAGER role will have the same rights as the GUEST role with a few extra privileges, namely to read the history for courses, update courses and call the logout_user UDF. Run the following FQL in the Shell:

CreateRole({
  name: "Manager",
  privileges: [
    {
      resource: Collection("Course"),
      actions: {
        read: true,
        write: true,
        history_read: true
      }
    },
    {
      resource: Index("allCourses"),
      actions: {
        read: true
      }
    },
    {
      resource: Function("course_update_history"),
      actions: {
        call: true
      }
    },
    {
      resource: Function("logout_user"),
      actions: {
        call: true
      }
    },
    {
      resource: Index("allCoursesInReview"),
      actions: {
        read: true
      }
    },
    {
      resource: Collection("Bookmark"),
      actions: {
        read: Query(
          Lambda(
            "bookmarkRef",
            Let(
              {
                bookmark: Get(Var("bookmarkRef")),
                private: Select(["data", "private"], Var("bookmark"))
              },
              Equals(Var("private"), false)
            )
          )
        )
      }
    },
    {
      resource: Index("allBookmarks"),
      actions: {
        read: true
      }
    },
    {
      resource: Index("bookmark_user_by_user"),
      actions: {
        read: true
      }
    },
    {
      resource: Collection("User"),
      actions: {
        read: true,
      }
    },
    {
      resource: Index("allUsers"),
      actions: {
        read: true
      }
    }
  ],
  membership: [
    {
      resource: Collection("User"),
      predicate: Query(
        Lambda(
          "userRef",
          Equals(Select(["data", "role"], Get(Var("userRef"))), "MANAGER")
        )
      )
    }
  ]
})

Run the following FQL in the Shell to create a couple of manager users:

Map(
  [
    {
      name: "Manager 1",
      email: "manager1@email.com",
      password: "password",
      role: "MANAGER"
    },
    {
      name: "Manager 2",
      email: "manager2@email.com",
      password: "password",
      role: "MANAGER"
    },
  ],
  Lambda(
    "data",
      Let(
      {
        userRef: Select(
          "ref",
          Call(Function("create_user"), [
            {
              name: Select("name", Var("data")),
              email: Select("email", Var("data")),
              password: Select("password", Var("data")),
              role: Select("role", Var("data"))
            }
          ])
        )
      },
      {
        result: "Success"
      }
    )
    )
)

Now we are ready to test how a course review process might look like in real life. This is simply an example showing what data is available.

Note that mixing manager (admin) functionality with the client-facing app is a bad idea and we do it for simplicity reasons only. It’s much better to have a dedicated app for administration purposes.

Head over to our MVP and log in using the following credentials:
Email: manager1@email.com
Password: password

You will be redirected to the following page /app/courses/review where you will see courses for review.

Click on the ”See details” link for the first course.

In the first box, you can see the current state of the course.
In the second box - how the course was updated. First, it was created, then all other changes are listed (you may see a different list of changes depending on what you changed and in what order).

If you like the changes, you can click ”Approve”

Conclusion

This is how you can leverage the FaunaDB's Temporality feature to track changes to any document.

We could create a fancy UI to allow an interactive time travel but this sounds like another project!

With this article finished, we have addressed all the challenges I listed in the first one. We have proved that using FaunaDB built-in features combined with Gatsby removes the need to reinvent the wheel and allows us to concentrate on the idea.

This is the approach I am taking with my project. I am using FaunaDB and will share more insights once I release it. Stay tuned.

How to add user authentication to your MVP using FaunaDB

Sandor | tutorialhell.dev — Tue, 04 Aug 2020 13:27:20 +0000

It is the second article in the series of articles covering FaunaDB with Jamstack. The first article is about How to launch an MVP with Jamstack, FaunaDB and GraphQL with zero operational costs

As the title suggests, we will be talking about an important feature that most MVPs need - user authentication. Building your own reliable and secure solution is time-consuming. Using a third-party solution with a free tier is often expensive to scale.

In the first article where I share how I build my MVP with Jamstack and FaunaDB, I mentioned that FaunaDB offers secure user authentication and attribute-based access control (ABAC) out of the box.

What makes FaunaDB’s ABAC even more powerful is that changes to access rights are reflected immediately because ABAC is evaluated for every query. This means that access can be granted or revoked without requiring a user to re-login.

In this article, we will see how easy it is to add user authentication features to a website using FaunaDB. We will also allow authenticated users to bookmark courses. All bookmarks are public by default so anyone can see them. To see FaunaDB’s ABAC in action, we will add a feature to make bookmarks private. Here I'd like to stress that nothing is public in FaunaDB. All data that unauthenticated users will see still needs to be made accessible using either a Role or a Key system. FaunaDB is secure by default!

I assume that you have followed the first article and have the starter project with data coming from FaunaDB. If not, the best is to start here. Alternatively, checkout the branch with the finished implementation of what was covered in the first article running the following code:

git clone --single-branch --branch article-1/source-data-from-FaunaDB git@github.com:sandorTuranszky/Gatsby-FaunaDB-GraphQL.git gatsby-fauna-db

FaunaDB provides Login and Logout built-in functions that can be used to create and invalidate user authentication tokens.

For example, we can pass an email and password to the Login function to get an authentication token or an error, if credentials are invalid.

In the first article we didn't have to create resolvers, as FaunaDB did it for us and we were fine with the logic. However, the login process might vary depending on the requirements, For example, we might want to use a username instead of an email. This is why FaunaDB gives us the flexibility to create our own resolvers.

Let’s take a look at our current schema:

It allows us to create authors with the name field only (see 3 - AuthorInput). To log in, an author will need to have an email and password. Not a big deal that we did not do it right away, because modifying schema with FaunaDB is very easy. All we need is to upload an updated schema.

Here it is:

type Course {
 title: String!
 description: String
 author: User!
 bookmarks: [Bookmark!] @relation
}

type Bookmark {
 title: String
 private: Boolean!
 user: User!
 course: Course!
}

type User {
 name: String!
 email: String!
 role: Role!
 courses: [Course!] @relation
 bookmarks: [Bookmark!] @relation
}

input CreateUserInput {
 name: String!
 email: String!
 password: String!
 role: Role!
}

input LoginUserInput {
 email: String!
 password: String!
}

input LogoutInput {
 allTokens: Boolean
}

type AuthPayload {
 token: String!
 user: User!
}

type Query {
 allCourses: [Course!]
 allBookmarks: [Bookmark!]
 allUsers(role: Role): [User!]
}

type Mutation {
 createUser(data: CreateUserInput): User! @resolver(name: "create_user")
 loginUser(data: LoginUserInput): AuthPayload! @resolver(name: "login_user")
 logoutUser(data: LogoutInput): Boolean! @resolver(name: "logout_user")
}

enum Role {
 AUTHOR
 DEVELOPER
}

Notice that the Author type was renamed to User. The project requirements have changed and we want to have different types of users depending on their roles. This is why we added the role field and the Role enum.

We also added the bookmarks relation for the “Bookmarks” feature and the email and password fields to the User type for login purposes.

The allAuthors query was renamed to a generic allUsers.

There is also a relation added between Courses and Bookmarks types that will allow us to highlight which courses have been bookmarked by an authenticated user.

The createUser, loginUser and the logoutUser mutations are self-explanatory. Notice the @resolver directive. It allows us to define our custom logic for resolvers. The provided create_user, login_user and logout_user names are how we will name our custom resolver functions. In FaunaDB’s documentation, they are referred to as User-defined functions (UDF)

Schema update

We have two options to update the existing schema:

Option 1: UPDATE SCHEMA creates any missing collections, indexes, and functions and overrides existing ones. All other elements of the existing schema remain the same.

Option 2: OVERRIDE SCHEMA removes all database elements, such as collections, indexes and functions and creates new once. This may result in loss of data this is why it’s not suitable for production apps. The GraphQL schema evolution is a much better and safer approach.

You can read about updating schema in more details here

Since we are at an early stage in our project and changes to our schema are quite significant, we will go with option 2 and override our existing schema. This will require us to repopulate the database with test data and it’s acceptable for us.

Copy the above schema and paste inside a schema-courses.gql file (you can name it whatever you want)

Click the OVERRIDE SCHEMA link, choose the file, click Open and wait 1 min. As explained here, there is a 60-second pause to allow all cluster nodes to process the schema changes.

Note that fields in the database collections that are no longer declared in the schema are not accessible via GraphQL queries. It might make sense to clean them up before overriding the schema. You can do it with delete* mutations in the GraphQL playground.

Before the override, we had two collections populated with some data.
The Author collection:

And the Course collection:

After the schema update, we have two newly created Course and User collections, that are both empty.

As expected, everything was removed and we will need to repopulate our collections with test data, but this time users will have emails, passwords and roles.

UDF to create a user

Before we can create users, we need to define the create_user resolver function and use FaunaDB’s built-in authentication feature to ensure passwords are hashed.

Although the create_user (UDF) has been created by FaunaDB automatically as a "template" UDF based on the @resolver directive, we will get an error if we try to run the createUser mutation now. This is because no logic has been implemented yet.

We will use Fauna Query Language (FQL) to update the createUser UDF. If you prefer, you can create and modify UDF via the dashboard under the FUNCTIONS menu.

Here is the FQL code:

Update(
  Function("create_user"),
  {
    "body": Query(
      Lambda(["data"],
        Create(
          Collection("User"),
          {
            credentials: { password: Select("password", Var("data")) },
            data: {
              name: Select("name", Var("data")),
              email: Select("email", Var("data")),
              role: Select("role", Var("data")),
            },
          }
        )
      )
     )
  }
)

What’s happening in this FQL code.
As mentioned before, the create_user UDF has already been created by FaunaDB. It looks like this:

Query(
  Lambda(
    "_",
    Abort(
      "Function create_user was not implemented yet. Please access your database and provide an implementation for the create_user function."
    )
  )
)

To add custom logic to an existing UDF, we use the Update function. It accepts two arguments: ref and param_object.

As the first argument, we pass in the reference to our existing create_user UDF with the help of the Function function.

As the second argument, we provide an object with a single key body, that holds a query to be run when the function is executed. This query must be wrapped in a Query function, which takes a Lambda function and defers its execution because we want this Lambda function to only run when the mutation is called.

The Lambda function is used to execute custom code - in our case, the custom code is how we create a new user.

The UDF accepts an array of arguments (Lambda(["data"],...), the same arguments as defined in the GraphQL schema. We have an argument data in createUser(data: CreateUserInput) mutation with values defined in CreateUserInput.

The Create function is used to create a document in a collection. It takes two arguments: collection and param_object.

As the first argument, we pass in the references to the User collection with the help of the Collection function.

As the second argument, we provide and object with two keys:

data key is an object that holds fields to be stored in the document. In our case, those fields are name, email and role. The password will NOT be stored here!
credentials key is an object that encrypts values and this is why we use it to store the password. Once created, this value can’t be retrieved anymore. This means that passwords or other sensitive data can’t be leaked accidentally.

The Var statement returns a value stored in a named variable - in the data object in our case and the Select function extracts a single value by the key name. For example, the following Select("name", Var("data")) is same as Select("name", {name: "Johns Austin", email: "johns.austin@email.com", password: "password", role: AUTHOR}) that will return the value of the name key: "Johns Austin"

Now that it’s clear how exactly our custom createUser UDF works, navigate to the Shell (1) page, copy and paste (2) the FQL code and click Run Query (3) as shown on the following screenshot:

Now, go to the Functions page, and you should see the create_user function there:

Create test users

Now we can repopulate our database running two simple FQL queries.
The first one is an index that we will need to set up relations between bookmarks and courses. Copy, paste it into the Shell and run it (similarly like we did above)

CreateIndex({
  name: "course_by_title",
  source: Collection("Course"),
  terms: [{ field: ["data", "title"] }],
  values: [{ field: ["data", "title"] }]
})

The next script will add all the test data we need. It will create authors with courses, developers and bookmarks. Copy, paste it into the Shell and run it.

Map(
  [
    {
      name: "Johns Austin",
      email: "johns.austin@email.com",
      password: "password",
      role: "AUTHOR",
      courses: [
        {
          title: "React for beginners"
        }
      ]
    },
    {
      name: "Andrews Winters",
      email: "andrews.winters@email.com",
      password: "password",
      role: "AUTHOR",
      courses: [
        {
          title: "Advanced React"
        }
      ]
    },
    {
      name: "Wiley Cardenas",
      email: "wiley.cardenas@email.com",
      password: "password",
      role: "AUTHOR",
      courses: [
        {
          title: "NodeJS Tips & Tricks"
        },
        {
          title: "Build your first JAMstack site with FaunaDB"
        },
        {
          title: "VueJS best practices"
        }
      ]
    },
    {
      name: "Blake Fletcher",
      email: "blake.fletcher@email.com",
      password: "password",
      role: "AUTHOR",
      courses: [
        {
          title: "Mastering Vue 3"
        }
      ]
    },
    {
      name: "Hamilton Lowe",
      email: "hamilton.lowe@email.com",
      password: "password",
      role: "DEVELOPER",
      bookmarks: [
        {
          title: "React for beginners",
          private: false
        },
        {
          title: "VueJS best practices",
          private: true
        }
      ]
    },
    {
      name: "Melinda Haynes",
      email: "melinda.haynes@email.com",
      password: "password",
      role: "DEVELOPER",
      bookmarks: [
        {
          title: "Advanced React",
          private: false
        },
        {
          title: "Build your first JAMstack site with FaunaDB",
          private: false
        }
      ]
    }
  ],
  Lambda(
    "data",
    Let(
      {
        userRef: Select(
          "ref",
          Call(Function("create_user"), [
            {
              name: Select("name", Var("data")),
              email: Select("email", Var("data")),
              password: Select("password", Var("data")),
              role: Select("role", Var("data"))
            }
          ])
        ),
        courses: Map(
          Select("courses", Var("data"), []),
          Lambda(
            "course",
            Create(Collection("Course"), {
              data: {
                title: Select("title", Var("course")),
                author: Var("userRef")
              }
            })
          )
        ),
        bookmarks: Map(
          Select("bookmarks", Var("data"), []),
          Lambda(
            "bookmark",
            Create(Collection("Bookmark"), {
              data: {
                title: Select("title", Var("bookmark")),
                private: Select("private", Var("bookmark")),
                user: Var("userRef"),
                course: Select("ref", Get(Match(Index("course_by_title"), Select("title", Var("bookmark")))))
              }
            })
          )
        )
      },
      {
        result: "Success"
      }
    )
  )
)

Rebuilding the app

If you rebuild our project with gatsby develop, you will see an error:

"There was an error in your GraphQL query: Insufficient privileges to perform the action"

Because we replaced the Author collection with the new User collection, we need to update our Guest role privileges that we created in the previous article. We have seen how to manage roles and privileges using the UI - it’s very comfy and easy to understand. In this article, I will use FQL for brevity reasons and will explain what changed.

This is what our Guest role privileges look like now.

The Author collection and allAuthors index do not exist anymore - we need to fix it.

Run the following FQL query from the Shell to update the Guest role privileges.

Update(Role("Guest"), {
  privileges: [
    {
      resource: Collection("User"),
      actions: {
        read: true
      }
    },
    {
      resource: Collection("Course"),
      actions: {
        read: true
      }
    },
    {
      resource: Collection("Bookmark"),
      actions: {
        read: true
      }
    },
    {
      resource: Index("allUsers"),
      actions: {
        read: true
      }
    },
    {
      resource: Index("allCourses"),
      actions: {
        read: true
      }
    },
    {
      resource: Index("allBookmarks"),
      actions: {
        read: true
      }
    },
    {
      resource: Index("bookmark_user_by_user"),
      actions: {
        read: true
      }
    }
  ]
})

Head over to the role management section and select the Guest role. You should see this:

The Author collection is gone and the Guest role can read the new User collection and allUsers index.

Since bookmarks can be public, unauthenticated users can see them. We need to allow the Guest role to read the Bookmark collection, the allBookmarks and the bookmark_user_by_user indexes.

If you rebuild our project with gatsby develop now, you will see that the list of courses and authors looks exactly as it was before we updated the schema.

UDF for user login

To manage the Bookmarks feature, we need to allow users to log in. Let’s create the login_user UDF. Here is the code:

Update(
  Function("login_user"),
  {
    "body": Query(
      Lambda(
        ["data"],
        Let(
          {
            response: Login(
              Match(Index("user_by_email"), Select("email", Var("data"))),
              { password: Select("password", Var("data")) }
            )
          },
          {
            data: {
              token: Select("secret", Var("response")),
              user: Select("instance", Var("response"))
            }
          }
        )
      )
    )
  }
)

It uses FaunaDB’s built-in Login function, which takes two arguments: identity and param_object.

For the identity argument, we pass in a reference to a User document which we look up by the provided email. To find a user by an email the user_by_email index is used (we will create it soon).

The Match function finds the exact match in the given index for the provided search terms.

As the second argument, we pass in the provided password.

The Login function will return an object containing the secret under the token key, the reference to the document (user) and some other data.

However, we need to have a custom response structure with the token and user keys. To restructure the response, we first store the Login function response in the response variable using the Let statement and then shape the response object to match what we need and wrap it in the data object because a UDF must return a GraphQL-compatible type.

When authentication fails, the Login function returns an error.

Before we add the login_user UDF, we need to create the user_by_email index. Click on the Indexes menu item and then click NEW INDEX link as shown on the next screenshot:

Fill in the form as shown in the following screenshot:

Choose the User collection
Add index name
Type email in the “Terms” input and click the “+” icon on the right - it will automatically prefix the data. part -> data.email
Check "Unique" checkbox (we need an exact match)
Save it

Now we can test the newly created index on the Index page (Indexes menu item)

Paste the following email wiley.cardenas@email.com
Click the "Search" button
Get the result

One more thing is left - we need to allow the Guest role to read the user_by_email index and call the login_user UDF.

Navigate to the Security page and click MANAGE ROLES, then choose the Guest role.
Select the user_by_email index from the dropdown
Add “Read” action
Select the login_user UDF from the dropdown
Add “Call” action and save

Now we are ready to update the login_user UDF the same way as we added the create_user one. Copy the login_user UDF code, paste it into the Shell and run the query:

Head over to the GraphQL Playground to test the Login mutation:
Run the following mutation and you will receive the token and the user details:

mutation Login {
  loginUser(data: {
    email: "wiley.cardenas@email.com"
    password: "password"
  }) {
    token
    user {
      name
      email
      role
    }
  }
}

Try to provide invalid credentials and you will get an error: authentication failed

UDF for user logout

We also need to create a logout_user UDF to allow users to invalidate their login sessions. Here is the UDF code:

Update(
  Function("logout_user"),
  {
    "body": Query(Lambda(["data"], Logout(Select("allTokens", Var("data"), false))))
  }
)

Let’s quickly add it the same way as we did with the login_user UDF - via the Shell. See the screenshot:

Testing how it works on a real app

The best way to test out the login and logout, as well as the ABAC features, is to see how it works on a real app. I’ve added features to our starter and you can clone the source code from a branch in the same repo. This way, we will save time on copy and pasting a lot of code.

Before you clone, note that you will need the .env.development and .env.production files from the first article containing the bootstrap key and other variables.

Follow these steps:

git clone --single-branch --branch article-2/authentication-ABAC git@github.com:sandorTuranszky/Gatsby-FaunaDB-GraphQL.git gatsby-fauna-db
cd gatsby-fauna-db
npm install
copy the .env.development and .env.production files into the root of the project
gatsby develop

You should see our updated app with a few menu options including the Login.

Note if you get any TypeErrors, delete the token cookie and the user_data object from the localStorage.

The main page contains the same static data as before. If you navigate to the Developers page, you will see all public bookmarks being loaded dynamically.

If you reload this page multiple times, you will see the Loading... message for a while before the bookmarks are listed. And if you navigate to the devs tools, you’ll notice the /graphql request that is made to load bookmarks dynamically.

This is how easy it is to have a mix of static and dynamic pages with Gatsby.

There is one issue with bookmarks though. If you take a closer look, you will see that one private bookmark is listed together with the public once for unauthenticated users:

This is wrong and we will fix it using FaunaDB’s ABAC.

Navigate to the role management page and select the Guest role.
Add the following FQL code as shown on the screenshot and save it.

Lambda(
  "bookmarkRef",
  Let(
    {
      bookmark: Get(Var("bookmarkRef")),
      private: Select(["data", "private"], Var("bookmark"))
    },
    Equals(Var("private"), false)
  )
)

The FQL code above makes sure that only public bookmarks are accessible for the Guest role by checking whether the private property on a bookmark is false.

Reload the /developers page, and you should see no private bookmarks listed. No need to rebuild the app, because bookmarks are loaded dynamically.

This is how easy it is to manage what users can access and what not using FaunaDB's ABAC.

Testing the login feature

Security notice: In our example app, we will call a FaunaDB’s GraphQL endpoint right from the client-side. I strongly discourage you from taking this approach if you will manage sensitive data in your application.

The right way is to call an endpoint on your server under the same domain which would then communicate with third-party APIs. It will also allow you to implement security techniques and best practices to mitigate common client-side vulnerabilities.

In our example app, we deal with bookmarks which aren't sensitive information at all and this is why we can go with this simplified approach without a backend although it would be easy to implement thanks to Netlify functions.

Token invalidation notice: Tokens created by FaunaDB’s Login function do not have a default Time-To-Live (TTL) value. You can set TTL optionally, however, at the time of the writing of this article, the TTL does not guarantee the token removal. The good news is that a reliable token invalidation is being developed by FaunaDB’s dev team and will become available soon. I will update this article once the feature is released.

It is not an issue for our MVP (users can stay logged-in indefinitely) therefore we will not address token invalidation in this tutorial.

More on authentication-related security you can read here

To test out how the login/logout works, we need to create a new, Developer role and define privileges for it.

Run the following FQL query using the Shell to create the Developer role with required privileges.

CreateRole({
  name: "Developer",
  membership: [
    {
      resource: Collection("User"),
      predicate: Query(
        Lambda("userRef", 
          Equals(Select(["data", "role"], Get(Var("userRef"))), "DEVELOPER")
        )
      )
    }
  ], 
  privileges: [
    {
      resource: Collection("User"),
      actions: {
        read: true,
      }
    },
    {
      resource: Collection("Bookmark"),
      actions: {
        read: Query(
          Lambda(
            "bookmarkRef",
            Let(
              {
                bookmark: Get(Var("bookmarkRef")),
                userRef: Select(["data", "user"], Var("bookmark")),
                private: Select(["data", "private"], Var("bookmark"))
              },
              Or(
                Equals(Var("userRef"), Identity()),
                Equals(Var("private"), false)
              )
            )
          )
        ),
        write: Query(
          Lambda(
            ["oldData", "newData"],
            And(
              Equals(Identity(), Select(["data", "user"], Var("oldData"))),
              Equals(
                Select(["data", "user"], Var("oldData")),
                Select(["data", "user"], Var("newData"))
              )
            )
          )
        ),
        create: Query(
          Lambda(
            "data",
            Equals(Identity(), Select(["data", "user"], Var("data")))
          )
        ),
        delete: Query(
          Lambda(
            "ref",
            Equals(Identity(), Select(["data", "user"], Get(Var("ref"))))
          )
        )
      }
    },
    {
      resource: Index("allUsers"),
      actions: {
        read: true
      }
    },
    {
      resource: Index("allBookmarks"),
      actions: {
        read: true
      }
    },
    {
      resource: Index("bookmark_user_by_user"),
      actions: {
        read: true
      }
    },
    {
      resource: Function("logout_user"),
      actions: {
        call: true
      }
    },
    {
      resource: Collection("Course"),
      actions: {
        read: true,
      }
    },
    {
      resource: Index("allCourses"),
      actions: {
        read: true
      }
    },
    {
      resource: Index("bookmark_course_by_course"),
      actions: {
        read: true
      }
    },
    {
      resource: Index("course_author_by_user"),
      actions: {
        read: true
      }
    }
  ]
})

Let’s inspect the Developer role:

Just like the Guest role, Developer role needs to read the User, Course and Bookmark collections
Privileges to read indexes are needed to query data on collections
Logged-in users need to call the logout_user function to invalidate the auth token.
For authenticated users, ABAC gets more complex. We need to control not only what users can read, but also what resources they can change and in what way. Our logged-in user can read, create, update and delete bookmarks.

Let’s look at the “Read” action:

Unlike the Guest role, the Developer role can see its own private bookmarks. The code above (the predicate function) makes sure that if the bookmark is private, the user defined on the bookmark is the same user that is logged-in.

We can see it in action. Navigate to the /app/login page and log in with the following credentials:
email: hamilton.lowe@email.com
password: password

You will be redirected to the following page:

Now navigate to the /developers page where the logged-in user can still see his own bookmarks including private once. However, for the developer Melinda Haynes only public bookmarks are listed.

Now log out and log in with melinda.haynes@email.com email and password password. Note that password is the same for all users to keep things simple.

It turns out, Melinda has no private bookmarks. Navigate to /developers page:

We know that Hamilton has a private bookmark, but Melinda can’t see it.
You can add bookmarks for Melinda and then log in as Hamilton to see that it will work as expected. Or you can mark all bookmarks as private and they will not be visible for other logged-in or unauthenticated users.

You can also remove the FQL code that controls the “Read” action for bookmarks to see how things will get messed up.

While we were testing, we logged out users successfully that proves that logged-in users are allowed to call the logout_user function.

The “Create” action is also controlled by a predicate function.

The idea here is to control that bookmarks are created by users for themselves only. This is more a business logic rather than a security consideration in our case. We can easily imagine a content manager or an admin creating content for others.

The “Delete” action is similar by logic to the “Create” action - allowing you to delete only your own bookmarks.

The “Write” action is controlled by a predicate function to make sure users can only update their own bookmarks and that the original bookmark owner does not change during an update.

Now let’s look at the membership. By adding the User collection, we state that all users who are members for the User collection will be granted the privileges we’ve defined for this role, once they obtain a valid token using the Login function.

Conclusion

As you can see, it is really easy to create user authentication flow using FaunaDB built-in Auth and ABAC features. Although we have barely scratched the surface, it was still sufficient enough to demonstrate how powerful and flexible the FaunaDB ABAC is.

I mentioned in the first article, that ABAC rules can also check for date or time for each transaction offering simple solutions to common problems most web apps have - for example, the need to control free trials periods or manage access to paid content.

With that said, the second part of the challenge is done:

Users can authenticate and access data depending on the privileges they have.
Bookmarks are loaded dynamically on pages where some of the data is static - a mix of static and dynamic content.
We reflect what courses have been bookmarked by the logged-in user on static courses data so they can’t bookmark them twice.

Now we have a half static and half dynamic website powered by FaunaDB. We still have a lot of room for optimization, and I will cover it in future posts.

But now, the last thing that is left to do is to explore the Temporality feature and see how it can be applied in real-life examples. I will do so in the next article.

UPDATE: Here is the third article.

The live demo of the app is available here

DEV Community: Sandor | tutorialhell.dev

Has anyone else noticed an increase in New, Empty Profile Followers recently?

Must-Have Practical Tutorial on Node.js Modules for Badass Beginners

Creating a Simple Module

Exporting from a Module

Importing a Module

Multiple exports

Multiple imports

Choosing Between module.exports and exports

Here’s a breakdown of the key feature that every beginner should grasp:

require() is Synchronous and Caches Modules

Why It Matters

Practical Exercises

Transferable Knowledge and Patterns

Learning path from here:

Must-Have Practical Tutorial on Node.js Modules for Badass Beginners

Creating a Simple Module

Exporting from a Module

Importing a Module

Multiple exports

Multiple imports

Choosing Between module.exports and exports

Here’s a breakdown of the key feature that every beginner should grasp:

require() is Synchronous and Caches Modules

Why It Matters

Practical Exercises

Transferable Knowledge and Patterns

Learning path from here:

Don't use vanilla JavaScript! I am serious!

You don't know Redis (Part 2)

Implementation

Registration flow

Login flow

Conclusion

You don't know Redis

Q&A board features

Implementation details

Add a question

Approve a question

Fetch approved questions

Vote for a question

Fetch most recent approved questions

Update a question with an answer

Fetch questions with answers

Conclusion

How to create LinkedIn-like reactions with Serverless Redis

Planning

1. Research database options.

2. Create a Serverless Backend with functions for:

3. Create the "reactions" component.

4. Put it all together to make my static website a little bit dynamic.

Research

Serverless Backend

Off-topic

Conclusion

How to get out of tutorial hell and avoid the imposter syndrome

What is one of the most common mistakes beginner developers make

Please, don't make the same mistake

How to track change to any content with FaunaDB's Temporality feature

Schema update

Author role

Temporality

Snapshots

Events

UDF for listing changes for a particular course

Conclusion

How to add user authentication to your MVP using FaunaDB

Schema update

UDF to create a user

Create test users

Rebuilding the app

UDF for user login

UDF for user logout

Testing how it works on a real app

Testing the login feature

Conclusion

Choosing Between `module.exports` and `exports`

`require()` is Synchronous and Caches Modules

Choosing Between `module.exports` and `exports`

`require()` is Synchronous and Caches Modules