DEV Community: Michael Otu

In Introduction to SQL using SQLite: Data Manipulation, we discussed creating, reading, updating and deleting records from a SQLite database using SQL. In this session, we will be doing the data manipulation with node:sqlite, which is a native (built-in) module for SQLite.

node:sqlite APIs

To use the sqlite module, you would have to import it as:

const { DatabaseSync } = require("node:sqlite");

const { DatabaseSync } = require('sqlite'); will throw an error, Error: Cannot find module 'sqlite'

Connect database

Using DatabaseSync, we can connect to an existing database or create a new one.

const database = new DatabaseSync("<DATABASE NAME>");

exec

The database object's exec method executes one or more SQL statements without returning results. It's useful for running SQL statements from a file.

database.exec(sql);

This is usually useful when creating and inserting at the same time.

database.exec(`
 CREATE TABLE IF NOT EXISTS TABLE_NAME (
 id INTEGER PRIMARY KEY AUTOINCREMENT,
 name TEXT NOT NULL,
 ...
 );

 INSERT INTO TABLE_NAME (name, category, quantity) 
 VALUES 
 ('Bam', ...),
 ('Cain', ...),
 ('Dan', ...);
`);

prepare

Now, to manipulate data, we can use the prepare method. This brings us to the point where we discuss prepared statements. Prepared statements are a way to pass data to the SQL safely for execution.

Instead of

INSERT INTO TABLE_NAME (name, ...)  VALUES ('Bam', ...);

We can pass place holders for the values.

INSERT INTO TABLE_NAME (name, ...)  VALUES (?, ...);

The prepare method returns a prepared statement object, StatementSync

run

For queries such as INSERT, UPDATE and DELETE, for there to be a change, a row or several rows would be affected. For this run returns an object of StatementResultingChanges

{
    "lastInsertRowid": number,
    "changes": number
}

In the case of INSERT, the last inserted record's ID is returned and the number of rows that were affected by the query. If there are changes, then some rows were affected, and that is one way that you'd verify if the query was successful or not.

const preparedInsertSQL = database.prepare(
    "INSERT INTO TABLE_NAME (name, ...) VALUES (?, ...)",
);

const response = preparedInsertSQL.run("Fushiguro", ...)

console.log(
    JSON.stringify(response, null, 4),
);

The order of the place holders of the prepared statement matters. It dictates the order of the values passed.

// Delete with prepared statement
const preparedDeleteSQL = database.prepare(
    "DELETE FROM TABLE_NAME WHERE id = ?",
);

const response = preparedDeleteSQL.run(4);

console.log(JSON.stringify(response, null, 4));

// Update with prepared statement
const preparedInsertSQL = database.prepare(
    "UPDATE TABLE_NAME set name = ? WHERE id = ?",
);

console.log(JSON.stringify(preparedInsertSQL.run("Panda", 1), null, 4));

all & get

For SELECT statements, we can use the all or get method on the StatementSync object. The all method returns an array of objects parsed from the result of the query. get returns one object. If you are expecting a list, then use all. If you want the one object from the response, then use get. get is like all indexed 0.

const preparedSelectAllSQL = database.prepare("SELECT * FROM TABLE_NAME");
const rows = preparedSelectAllSQL.all();
// returns an array

const preparedSelectOneSQL = database.prepare(
    "SELECT * FROM TABLE_NAME WHERE id = ?",
);
const rows = preparedSelectAllSQL.get(1);
// returns an object

CRUD with Node SQLite

First of all, for one to execute a program running with the built-in SQLite, we use the experimental flag, node --experimental-sqlite FILE-PATH

Let's create a program for tracking expenses. We have done a similar mini-project before. Check out, Expense tracker project 0 1 2 3.

Connect Database

Let's create a database for this project called expense-tracker.sqlite.

const { DatabaseSync } = require("node:sqlite");

const database = new DatabaseSync("expense-tracker.sqlite");

Create Table

This table will consist of an ID, the name of the expense, the amount and the date of expenditure.

database.exec(`
 CREATE TABLE IF NOT EXISTS expenses (
 id INTEGER PRIMARY KEY AUTOINCREMENT,
 name TEXT NOT NULL,
 amount INTEGER NOT NULL,
 date TEXT
 );
`);

Insert

We can insert with exec

database.exec(
    `INSERT INTO 
 expenses (name, amount, date) 
 VALUES 
 ('Lenovo ThinkPad', 1040.39, '2025-12-24'),
 ('Ergonomic Office Chair', 139.55, '2026-01-05'),
 ('Samsung 27" Essential S3', 140.06, '2026-01-15'),
 ('Samsung 27" Essential S3', 139.06, '2026-01-20'),
 ('Portable External Hard Drive', 60.00, '2026-02-01'),
 ('Logitech M185 Wireless Mouse', 13.00, '2026-02-01'),
 ('Logitech H390 Wired Headset', 20.91, '2026-02-01'),
 ('Dual Monitor Stand', 47.82, '2026-03-11');
 `,
);

We can also insert using a prepare

const prepareInsertSql = database.prepare(
    "INSERT INTO expenses (name, amount, date) VALUES (?, ?, ?)",
);

const insertChanges = prepareInsertSql.run("USB C Cable", 9.91, "2026-02-10");

if (insertChanges.changes > 0) {
    console.log(
        "Expense recorded successfully - ref: ",
        insertChanges.lastInsertRowid,
 );
}

// Expense recorded successfully - ref:  9

Read

Let's select records with amount above 100.

const prepareSelectSql = database.prepare(
    "SELECT * FROM expenses WHERE amount > ?",
);

const rowsWithAmountAboveHundred = prepareSelectSql.all(100);

console.log(rowsWithAmountAboveHundred);
/* 
[
 [Object: null prototype] {
 id: 1,
 name: 'Lenovo ThinkPad',
 amount: 1040.39,
 date: '2025-12-24'
 },
 [Object: null prototype] {
 id: 2,
 name: 'Ergonomic Office Chair',
 amount: 139.55,
 date: '2026-01-05'
 },
 [Object: null prototype] {
 id: 3,
 name: 'Samsung 27" Essential S3',
 amount: 140.06,
 date: '2026-01-15'
 },
 [Object: null prototype] {
 id: 4,
 name: 'Samsung 27" Essential S3',
 amount: 139.06,
 date: '2026-01-20'
 }
]
*/

From this, we can select one.

const rowWithAmountAboveHundred = prepareSelectSql.get(100);

console.log(rowWithAmountAboveHundred);

/* 
[Object: null prototype] {
 id: 1,
 name: 'Lenovo ThinkPad',
 amount: 1040.39,
 date: '2025-12-24'
} 
*/

Update

Try this. Make a price adjustment of 5% on items bought on '2026-02-01'.

Delete

It has been over 3 months yet, so let's return the 'Lenovo ThinkPad' and buy 'Apple 2025 MacBook Pro Laptop with M5 chip' at 1363.43 today's date is'2026-02-10'`.

Conclusion

With node:sqlite, we can integrate SQLite; however, there are limitations to SQLite itself. For learning purposes and cases like this, it's alright to use SQLite. In fact, the limitations of SQLite become obvious when there are multiple writes at the same time. Again, the knowledge from using SQLite can be transferred to another relational database like MySQL and PostgreSQL.

Try these:

What happens when you use run for a select statement?
Add a script to find the max and min items
Add a script to compute the total of all items
Integration of this into an express application - preferably the expense tracker API

In the next section, we will be doing that. Get the experience by doing it yourself.

Resources

Introduction to SQL using SQLite: Data Manipulation

Michael Otu — Mon, 24 Nov 2025 19:53:11 +0000

Objectives

DROP TABLE
INSERT
SELECT
FILTERS
FUNCTIONS
UPDATE
DELETE

Introduction

We discussed that, to create a table for the object,

const profile = {
    name: "John Doe",
    "date of birth": "2000-12-25",
    profession: "Software Engineer",
    "number of pets": 2,
    "weight of protein in grams": 12.5,
    "has a job": true,
};

We would have to analyse it and design a table that meets the requirements of our system, building on the knowledge we already have in JavaScript.

property	JS	SQL	Constraint
`name`	`string`	`TEXT`	`NOT NULL`
`date of birth`	`Date`	`TEXT`
`profession`	`string`	`TEXT`	`DEFAULT ''`
`number of pets`	`number`	`INTEGER`	`DEFAULT 0`
`weight of protein in grams`	`number`	`INTEGER`	`DEFAULT 0`
`has a job`	`boolean`	`INTEGER`	`DEFAULT 0`

This would logically boil down to something like:

CREATE TABLE "profile" (
    "id" INTEGER PRIMARY KEY AUTOINCREMENT,
    "name" TEXT NOT NULL,
    "dateOfBirth" TEXT,
    "profession" TEXT DEFAULT '',
    "numberOfPets" INTEGER DEFAULT 0,
    "weightOfProteinInGrams" INTEGER DEFAULT 0,
    "hasJob" INTEGER DEFAULT 0
);

In this section, we are going to focus on data manipulation, CRUD. Check out Request And Response, where we discussed CRUD and Basic Crud Api With Express, where we started building a CRUD application.

We will borrow a table here.

CRUD	HTTP VERBS	SQL
`CREATE`	`POST`	`INSERT`
`READ`	`READ`	`SELECT`
`UPDATE`	`PUT`	`UPDATE`
`DELETE`	`DELETE`	`DELETE`

DROP TABLE

First things first, if you can create, you should "un"-create. To "un"-create a table, we have to "drop" it. General syntax is:

DROP TABLE <TABLE_NAME>;

For the profile table or human table, we can drop it by executing the script, DROP TABLE "profile"; OR DROP TABLE "human";.

; signifies the end of the script (that line or snippet).

Know that this action is not reversible in most situations.

INSERT

Syntax for INSERT is generally,

INSERT INTO <TABLE_NAME> (<column1>,<column2>,...,<columnN>,) VALUES (<value1>, <value2>, ...,<valueN>);

This is the snippet for creating the "human" table.

CREATE TABLE "human" (
    "id" INTEGER PRIMARY KEY AUTOINCREMENT,
    "firstName" TEXT NOT NULL,
    "lastName" TEXT NOT NULL,
    "socialSecurityNumber" TEXT NOT NULL UNIQUE,
    "email" TEXT NOT NULL UNIQUE,
    "dateOfBirth" TEXT,
    "hasPet" INTEGER DEFAULT 0,
    "numberOfCars" INTEGER DEFAULT 0
);

Let's create a row for John Doe with SSN, 123-45-6789, who has no pet and no car, born on 14th of March, 1990, with johndoe1990@gmail.com as email.

INSERT INTO "human"
 ("firstName", "lastName", "socialSecurityNumber", "email", "dateOfBirth", "hasPet", "numberOfCars")
VALUES
 ('John', 'Doe', '123-45-6789', 'johndoe1990@gmail.com', '1990-03-14', 0, 0);

This will create a unique row for John Doe with 123-45-6789 as SSN.

Do you think we can create another row with the same details? Try it.

You should get UNIQUE constraint failed: human.email.

Create two more "human" records.

We can create more rows at once, bulk insert, by

INSERT INTO <TABLE_NAME>
 (<column1>,<column2>,...,<columnN>,)
VALUES
 (<value1>, <value2>, ...,<valueN>),
 (<value1>, <value2>, ...,<valueN>),
 (<value1>, <value2>, ...,<valueN>),
 (<value1>, <value2>, ...,<valueN>),
 ...
 (<value1>, <value2>, ...,<valueN>);

SELECT

The syntax for a SELECT statement is:

SELECT <FIELDS TO SELECT> FROM <TABLE NAME>;

So let's say we want the firstName and lastName columns from the human table, then will do

SELECT "firstName", "lastName" FROM "human";

This is how my output looks from the terminal.

user$ sqlite3 app.sqlite
SQLite version 3.43.2 2023-10-10 13:08:14
Enter ".help" for usage hints.
sqlite> SELECT "firstName", "lastName" FROM "human";
John|Doe
Anne|Ali
Gifty|Doe
Idris|Johnson
sqlite>

In beekeeper-studio, it looks "pretty"

For the above output, these are the inserts.

INSERT INTO "human"
 ("firstName", "lastName", "socialSecurityNumber", "email", "dateOfBirth", "hasPet", "numberOfCars")
VALUES
 ('Anne', 'Ali', '123-45-6790', 'alianne@gmail.com', '1995-12-21', 0, 1),
 ('Gifty', 'Doe', '123-45-6791', 'giftydoe@gmail.com', '1995-01-11', 1, 0),
 ('Idris', 'Johnson', '123-45-6792', 'johnidrisson@gmail.com', '1992-06-30', 0, 2);

When selecting a column for display, you have to account for the use of the column. When the client doesn't have that column, don't send it.

There is another way to select all the columns from a table, which is useful at times. This can be done by replacing all the with *, an asterisk.

SELECT * FROM "human";

The response is quite similar, but with all the fields this time.

SQLite can be configured to look "pretty". Check out sqlite-dot-commands

We can use the column mode

sqlite> .mode column
sqlite> SELECT * FROM "human";
id  firstName  lastName  socialSecurityNumber  email                   dateOfBirth  hasPet  numberOfCars
--  ---------  --------  --------------------  ----------------------  -----------  ------  ------------
1 John       Doe       123-45-6789 johndoe1990@gmail.com   1990-03-14   0       0
2 Anne       Ali       123-45-6790 alianne@gmail.com       1995-12-21   0       1
3 Gifty      Doe       123-45-6791 giftydoe@gmail.com      1995-01-11   1       0
4 Idris      Johnson   123-45-6792 johnidrisson@gmail.com  1992-06-30   0       2
sqlite>

We also have the box mode

sqlite> .mode box
sqlite> SELECT * from "human";
┌────┬───────────┬──────────┬──────────────────────┬────────────────────────┬─────────────┬────────┬──────────────┐
│ id │ firstName │ lastName │ socialSecurityNumber │         email          │ dateOfBirth │ hasPet │ numberOfCars │
├────┼───────────┼──────────┼──────────────────────┼────────────────────────┼─────────────┼────────┼──────────────┤
│ 1  │ John      │ Doe      │ 123-45-6789          │ johndoe1990@gmail.com  │ 1990-03-14  │ 0      │ 0            │
│ 2  │ Anne      │ Ali      │ 123-45-6790          │ alianne@gmail.com      │ 1995-12-21  │ 0      │ 1            │
│ 3  │ Gifty     │ Doe      │ 123-45-6791          │ giftydoe@gmail.com     │ 1995-01-11  │ 1      │ 0            │
│ 4  │ Idris     │ Johnson  │ 123-45-6792          │ johnidrisson@gmail.com │ 1992-06-30  │ 0      │ 2            │
└────┴───────────┴──────────┴──────────────────────┴────────────────────────┴─────────────┴────────┴──────────────┘

With this we have .exit to exit the SQLite runtime-environment (on the cli).

FILTERS

The idea about filters here, is to not unnecessarily select all the roles that we. We want to be able to select based on conditions and some particular amount at a time.

In the SELECT section, we know how to select only the fields we by passing the column's name.

We will need more rows here. You can download or access more data for insert. You can copy the content and paste it into any of the SQLite clients.

When the more data SQL file is download, we can insert its content by running this commend on SQLite

sqlite> .read more-data.sql
sqlite>

Put more-data.sql in the same folder as app.sqlite or pass the path to more-data.sql

WHERE CLAUSE

The WHERE clause can be used to filter (conditional selection). This takes on the same syntax as SELECT with WHERE.

SELECT <FIELDS TO SELECT> FROM <TABLE NAME> WHERE <CONDITION FOR THE SELECTION>;

Let's find humans that have pets. This means that we are interested in humans with the hasPet column to be 1. To do this we use =.

   SELECT * FROM "human" WHERE "hasPet" = 1;

So the = means is or equal-to. Unlike Javascript, =, single equal-to operator is used for assignment, whilst == or === is for equality check.
In javascript-part-four, we also discussed these operators
Is not equal to !=, so we can have <, >, <=, >=
Try selecting humans that have at least 2 cars
We have && as AND in Javascript, in SQL, it is AND. Same as || which is OR. With these we can have compound conditions.
Let's find humans that have a pet and at least two cars.

   SELECT * FROM "human" WHERE "hasPet" = 1  AND "numberOfCars" >= 2;

There is BETWEEN is used to find rows in ranges, "<COLUMN NAME> BETWEEN <START VALUE> AND <END VALUE>. Let's find humans whose IDs are between ranges of 3 and 10.

   SELECT * FROM "human" WHERE "id" BETWEEN 3 AND 10;

We can find humans who don't have pets but have between 2 to 4 cars.

   SELECT * FROM "human" WHERE  "hasPet" = 0 AND ("numberOfCars" BETWEEN 2 AND 4);

When "things" are getting complicated, wrap "things" in brackets and you will be fine.
We can find the rows of human whose IDs are in the array, [2,5,8,9] by using the IN operator. For this, we do, "<COLUMN NAME> IN (value1, value2, ..., valueN).

   SELECT * FROM "human" WHERE "id" IN (1,2,5,8,9);

We can also have a group of conditions and then negate them by using the NOT operator.

   -- negate IN
   SELECT * FROM "human" WHERE "id" NOT IN (1,2,5,8,9);

   -- negate BETWEEN
   SELECT * FROM "human" WHERE "numberOfCars" NOT BETWEEN 1 AND 3;

   --negate compound conditions
   SELECT * FROM "human" WHERE NOT ("hasPet" = 0 AND ("numberOfCars" BETWEEN 2 AND 4));

ORDER BY

By default, the rows are returned in a ascending order of ID. The rows are return in the order of the first created record to the latest. From the lowest to to the highest This is done by ORDER BY <COLUMN NAME> [DESC|ASC].

Let's see the default order, SELECT * FROM "human" WHERE "id" IN (1,2,5,8,9); Check the order order of the IDs

   sqlite> SELECT * FROM "human" WHERE "id" IN (1,2,5,8,9);
   ┌────┬───────────┬──────────┬──────────────────────┬──────────────────────────┬─────────────┬────────┬──────────────┐
   │ id │ firstName │ lastName │ socialSecurityNumber │          email           │ dateOfBirth │ hasPet │ numberOfCars │
   ├────┼───────────┼──────────┼──────────────────────┼──────────────────────────┼─────────────┼────────┼──────────────┤
   │ 1  │ John      │ Doe      │ 123-45-6789          │ johndoe1990@gmail.com    │ 1990-03-14  │ 0      │ 0            │
   │ 2  │ Anne      │ Ali      │ 123-45-6790          │ alianne@gmail.com        │ 1995-12-21  │ 0      │ 1            │
   │ 5  │ Elarais   │ Kaelon   │ 123-45-6797          │ elarais.kaelon@gmail.com │ 1991-07-22  │ 1      │ 1            │
   │ 8  │ Peridot   │ Valor    │ 123-45-6800          │ peridot.valor@gmail.com  │ 1975-03-10  │ 1      │ 3            │
   │ 9  │ Aelis     │ Caspian  │ 123-45-6801          │ aelis.caspian@gmail.com  │ 1996-12-28  │ 0      │ 1            │
   └────┴───────────┴──────────┴──────────────────────┴──────────────────────────┴─────────────┴────────┴──────────────┘

We can switch the order to DESC by passing, ORDER BY "id" DESC. SELECT * FROM "human" WHERE "id" IN (1,2,5,8,9) ORDER BY "id" DESC;

   sqlite> SELECT * FROM "human" WHERE "id" IN (1,2,5,8,9) ORDER BY "id" DESC;
   ┌────┬───────────┬──────────┬──────────────────────┬──────────────────────────┬─────────────┬────────┬──────────────┐
   │ id │ firstName │ lastName │ socialSecurityNumber │          email           │ dateOfBirth │ hasPet │ numberOfCars │
   ├────┼───────────┼──────────┼──────────────────────┼──────────────────────────┼─────────────┼────────┼──────────────┤
   │ 9 │ Aelis     │ Caspian  │ 123-45-6801 │ aelis.caspian@gmail.com  │ 1996-12-28 │ 0 │ 1 │
   │ 8 │ Peridot   │ Valor    │ 123-45-6800 │ peridot.valor@gmail.com  │ 1975-03-10 │ 1 │ 3 │
   │ 5 │ Elarais   │ Kaelon   │ 123-45-6797 │ elarais.kaelon@gmail.com │ 1991-07-22 │ 1 │ 1 │
   │ 2 │ Anne      │ Ali      │ 123-45-6790 │ alianne@gmail.com        │ 1995-12-21 │ 0 │ 1 │
   │ 1 │ John      │ Doe      │ 123-45-6789 │ johndoe1990@gmail.com    │ 1990-03-14 │ 0 │ 0 │
   └────┴───────────┴──────────┴──────────────────────┴──────────────────────────┴─────────────┴────────┴──────────────┘

FUNCTIONS

SQLite (SQL) supports functions for computing sum, SUM, count, COUNT, average, AVG, to find maximum, MAX, and minimum, MIN.

Let's sum the number of cars humans with pet have.

   SELECT SUM("numberOfCars") FROM "human" WHERE "hasPet" = 1;

The response to this is a little different from what we have seen so far

   sqlite> SELECT SUM("numberOfCars") FROM "human" WHERE "hasPet" = 1;
   ┌─────────────────────┐
   │ SUM("numberOfCars") │
   ├─────────────────────┤
   │ 15                  │
   └─────────────────────┘

Does SUM on non-numeric fields?
Let's count the number of humans with a pet. SELECT * FROM "human" WHERE "hasPet" = 1; returns rows of humans with pet.

   sqlite> SELECT COUNT(*) FROM "human" WHERE "hasPet" = 1;
   ┌──────────┐
   │ COUNT(*) │
   ├──────────┤
   │ 14       │
   └──────────┘

COUNT("id") will return the same response as COUNT(*) since it is the number of rows returned that countered.
We can find the average, AVG, in the same way as we did for SUM

   sqlite> SELECT AVG("numberOfCars") FROM "human" WHERE "hasPet" = 1;
   ┌─────────────────────┐
   │ AVG("numberOfCars") │
   ├─────────────────────┤
   │ 1.07142857142857    │
   └─────────────────────┘

MIN and MAX basically returns the minimum and maximum.
The default ordering is in ASC using the row's id.

   sqlite> SELECT * FROM "human" WHERE "hasPet" = 1;
   ┌────┬───────────┬──────────┬──────────────────────┬──────────────────────────┬─────────────┬────────┬──────────────┐
   │ id │ firstName │ lastName │ socialSecurityNumber │          email           │ dateOfBirth │ hasPet │ numberOfCars │
   ├────┼───────────┼──────────┼──────────────────────┼──────────────────────────┼─────────────┼────────┼──────────────┤
   │ 3  │ Gifty     │ Doe      │ 123-45-6791          │ giftydoe@gmail.com       │ 1995-01-11  │ 1      │ 0            │
   │ 5  │ Elarais   │ Kaelon   │ 123-45-6797          │ elarais.kaelon@gmail.com │ 1991-07-22  │ 1      │ 1            │
   │ 7  │ Indigo    │ Rylanx   │ 123-45-6799          │ indigo.rylanx@gmail.com  │ 2001-09-01  │ 1      │ 0            │
   │ 8  │ Peridot   │ Valor    │ 123-45-6800          │ peridot.valor@gmail.com  │ 1975-03-10  │ 1      │ 3            │
   │ 10 │ Nyxus     │ Joveus   │ 123-45-6802          │ nyxus.joveus@gmail.com   │ 1984-06-03  │ 1      │ 2            │
   │ 12 │ Daxus     │ Onyx     │ 123-45-6804          │ daxus.onyx@gmail.com     │ 1970-02-20  │ 1      │ 1            │
   │ 14 │ Quillon   │ Lior     │ 123-45-6806          │ quillon.lior@gmail.com   │ 1997-04-25  │ 1      │ 1            │
   │ 15 │ Soren     │ Wren     │ 123-45-6807          │ soren.wren@gmail.com     │ 2003-01-07  │ 1      │ 0            │
   │ 17 │ Faelan    │ Zenith   │ 123-45-6809          │ faelan.zenith@gmail.com  │ 1986-05-30  │ 1      │ 1            │
   │ 19 │ Lumina    │ Saga     │ 123-45-6811          │ lumina.saga@gmail.com    │ 1972-03-29  │ 1      │ 2            │
   │ 21 │ Eira      │ Myrddin  │ 123-45-6813          │ eira.myrddin@gmail.com   │ 1998-05-19  │ 1      │ 1            │
   │ 23 │ Kestrel   │ Garnet   │ 123-45-6815          │ kestrel.garnet@gmail.com │ 2000-08-23  │ 1      │ 0            │
   │ 25 │ Solara    │ Roux     │ 123-45-6817          │ solara.roux@gmail.com    │ 1990-09-11  │ 1      │ 1            │
   │ 27 │ Aether    │ Phaedra  │ 123-45-6819          │ aether.phaedra@gmail.com │ 1995-01-05  │ 1      │ 2            │
   └────┴───────────┴──────────┴──────────────────────┴──────────────────────────┴─────────────┴────────┴──────────────┘

If we want to find the minimum or maximum, we'd have to sort the rows.

   sqlite> SELECT * FROM "human" WHERE "hasPet" = 1 ORDER BY "numberOfCars" DESC;
   ┌────┬───────────┬──────────┬──────────────────────┬──────────────────────────┬─────────────┬────────┬──────────────┐
   │ id │ firstName │ lastName │ socialSecurityNumber │          email           │ dateOfBirth │ hasPet │ numberOfCars │
   ├────┼───────────┼──────────┼──────────────────────┼──────────────────────────┼─────────────┼────────┼──────────────┤
   │ 8  │ Peridot   │ Valor    │ 123-45-6800          │ peridot.valor@gmail.com  │ 1975-03-10  │ 1      │ 3            │
   │ 10 │ Nyxus     │ Joveus   │ 123-45-6802          │ nyxus.joveus@gmail.com   │ 1984-06-03  │ 1      │ 2            │
   │ 19 │ Lumina    │ Saga     │ 123-45-6811          │ lumina.saga@gmail.com    │ 1972-03-29  │ 1      │ 2            │
   │ 27 │ Aether    │ Phaedra  │ 123-45-6819          │ aether.phaedra@gmail.com │ 1995-01-05  │ 1      │ 2            │
   │ 5  │ Elarais   │ Kaelon   │ 123-45-6797          │ elarais.kaelon@gmail.com │ 1991-07-22  │ 1      │ 1            │
   │ 12 │ Daxus     │ Onyx     │ 123-45-6804          │ daxus.onyx@gmail.com     │ 1970-02-20  │ 1      │ 1            │
   │ 14 │ Quillon   │ Lior     │ 123-45-6806          │ quillon.lior@gmail.com   │ 1997-04-25  │ 1      │ 1            │
   │ 17 │ Faelan    │ Zenith   │ 123-45-6809          │ faelan.zenith@gmail.com  │ 1986-05-30  │ 1      │ 1            │
   │ 21 │ Eira      │ Myrddin  │ 123-45-6813          │ eira.myrddin@gmail.com   │ 1998-05-19  │ 1      │ 1            │
   │ 25 │ Solara    │ Roux     │ 123-45-6817          │ solara.roux@gmail.com    │ 1990-09-11  │ 1      │ 1            │
   │ 3  │ Gifty     │ Doe      │ 123-45-6791          │ giftydoe@gmail.com       │ 1995-01-11  │ 1      │ 0            │
   │ 7  │ Indigo    │ Rylanx   │ 123-45-6799          │ indigo.rylanx@gmail.com  │ 2001-09-01  │ 1      │ 0            │
   │ 15 │ Soren     │ Wren     │ 123-45-6807          │ soren.wren@gmail.com     │ 2003-01-07  │ 1      │ 0            │
   │ 23 │ Kestrel   │ Garnet   │ 123-45-6815          │ kestrel.garnet@gmail.com │ 2000-08-23  │ 1      │ 0            │
   └────┴───────────┴──────────┴──────────────────────┴──────────────────────────┴─────────────┴────────┴──────────────┘

By eye inspection or comparing the order of "numberOfCars" column, record with ID, 8, has highest number of cars and 23 has the lowest.
For MIN and MAX

   sqlite> SELECT MIN("numberOfCars") FROM "human" WHERE "hasPet" = 1;
   ┌─────────────────────┐
   │ MIN("numberOfCars") │
   ├─────────────────────┤
   │ 0                   │
   └─────────────────────┘
   sqlite> SELECT MAX("numberOfCars") FROM "human" WHERE "hasPet" = 1;
   ┌─────────────────────┐
   │ MAX("numberOfCars") │
   ├─────────────────────┤
   │ 3                   │
   └─────────────────────┘

How do we get the maximum number of cars among all the records?

UPDATE

The general syntax for an update looks like

UPDATE <TABLE NAME> SET <COLUMN 1> = <VALUE 1>, <COLUMN 2> = <VALUE 2>,..., <COLUMN N> = <VALUE N> WHERE <CONDITIONS>;

The WHERE clause is optional.

There are two siblings, John and Gifty Doe. Gifty has a pet but John doesn't. John must have something. Let's updates John's row to increment the number of cars he has to 1. John's ID is 1. You can do this for any user.

UPDATE "human" SET "numberOfCars" = 1 WHERE "id" = 1;

When human with ID 1 is selected, the numberOfCars column would have been updated to 1.

Try updating the email of human with email, giftydoe@gmail.com to giftydoe1995@gmail.com.

What do you think will happen when no WHERE clause is passed?

= is used as assignment operator here.

DELETE

The general syntax of DELETE is similar that of UPDATE.

DELETE FROM <TABLE NAME> WHERE <CONDITIONS>;

The rows which match the condition are deleted. When there are no conditions or WHERE clauses, this action will affect the entire rows. As such use with caution.

SELECT humans with no pet and no cars.
How many are they? Use the COUNT function.
Let's DELETE these records.

   DELETE FROM "human" WHERE "numberOfCars" = 0 AND "hasPet" = 0;

Are there any records with no cars and no pet? Use the COUNT function.

SUMMARY

We can delete a table with DROP <TABLE NAME>;
We can insert with INSERT INTO <TABLE NAME> (...cols) VALUES (...vals);
Using the SELECT and WHERE, we can list records that meets a certain criteria
We can use SQL functions to reduce the number data manipulation we have to execute
We can update a record by, UPDATE <TABLE NAME> SET <col> = <val> WHERE <CONDITION>
We can delete a record by, DELETE FROM <TABLE NAME> WHERE <CONDITION>
When we use UPDATE and DELETE, where no WHERE clause is passed, these action will affect all the records

Introduction to SQL using SQLite: Create Table

Michael Otu — Wed, 05 Nov 2025 19:08:12 +0000

Objectives

Create A SQLite Database
Import Database Into an SQL Client
SQLite Data Types
Constraints
Create Table

Introduction

Previously, we discussed Databases briefly, and in this excerpt, we will look into SQL.

In this excerpt, we will discuss creating tables.

Create A SQLite Database

Creating a SQLite database is quite simple and straightforward. Create a file with a .sqlite extension. I will make one called app.sqlite. Open the newly created database with Beekeeper Studio or SQLite Browser.

In Beekeeper Studio, under the New Connection text, click on the drop-down, Select a connection type.... Choose SQLite. Now, click on the Choose File button and navigate to where your new SQLite database was created and choose it.

Then click on Connect, and this should be what you will see. The database name will appear in the very lower left corner.

In SQLite Browser, you can create a new database with New Database or Open Database. We will open a new database. Click on the Open Database button, then navigate to where your new SQLite database was created and choose it.

After choosing it, the path to the selected database will be displayed in the title bar.

SQLite Data Types

In JavaScript Essentials: Part 1, we discussed data types. In JavaScript, we have Number, Boolean, and String. In SQLite we have INTEGER, REAL, TEXT, BLOB and NULL.

INTEGER and REAL are numbers.
SQLite doesn't have a direct boolean. As we are all aware, a boolean value is either true or false, which can be represented as 1 or 0. Hence, we can use an INTEGER for a boolean.
TEXT for string.
BLOB - "Binary Large Object" is used to store raw binary data such as images, files, etc.
NULL mean no data. When you fill a form and you omit or leave an input blank, it is null.

Constraints

These are limitations, guards or characters of a field.

NOT NULL: NULL means no data, so NOT NULL means no NULL or data is expected or required.
UNIQUE KEY: A field declared unique will have a unique key; as such, any other values will have a different value. A common use case is for emails or usernames. You can't have two users on your platform with the same emails. Imagine you send a verification code to one, and the other also receives it.
PRIMARY KEY: A primary key uniquely identifies a row (record) in a table. Generally, as soon as a field is declared as a primary key, it becomes the primary key, regardless of its type. However, in most cases, the primary key is the row ID.
DEFAULT: It is the value to be used when no value is passed when creating the row. When a field is not NOT NULL constrained, the default value will be NULL.
CHECK: It is a condition that the value must satisfy.
FOREIGN KEY: It is a field used to reference another table. This is usually the primary key from the other table.

Create Table

In Introduction to Databases, we talk about tables and what they are.

To create a table in SQLite, we can either use the client (Beekeeper studio, SQLite, etc as a GUI or script, a script that you will have to execute or directly on the terminal (with SQLite).

Generally, the format of creating a table is:

CREATE TABLE <TABLE_NAME> (
 field1 type [constraints],
 field2 type [constraints],
 ...,
 fieldN type [constraints]
);

Human tables

Let's create a table for a human. Boring the knowledge from JavaScript - knowledge transfer, a human has some characteristics (properties), which will turn into fields (columns) in SQL. Let's say we have a human with a name, date of birth, social security number, email, cars and has a pet, just to keep it short.

Now, what will be the types of the following properties: firstName, lastName, middleName, dateOfBirth, socialSecurityNumber, email, numberOfCars, and hasPet?.

In a real application or API, we don't save SSNs because it is very sensitive information.

The same types would translate to the field type in SQL(ite).

firstName, lastName, middleName, socialSecurityNumber and email are all string, as such they become TEXT
dateOfBirth is a Date object, but we don't have a direct date type in SQLite, so we can parse the dateOfBirth as a string, which means dateOfBirth can be saved as a TEXT. The Date object has a method, valueOf() or getTime(), with this, we can convert the date to a number, which we can either choose to store as an INTEGER or TEXT
numberOfCars is a number and so we will use an INTEGER type
hasPet is a boolean. Since there is no boolean type in SQLite, we can use 1 for true and 0 for false.

Since the subject of this is a human, we can name our table human.

We can choose another format for the way we write out our field names. We will be using Camel Casing
Double quote fields and single quote string/text values

CREATE TABLE "human" (
    "firstName" TEXT,
    "lastName" TEXT,
    "socialSecurityNumber" TEXT,
    "email" TEXT,
    "dateOfBirth" TEXT,
    "hasPet" INTEGER,
    "numberOfCars" INTEGER,
);

This table has no constraints. Apart from the email and SSN, which can be used to identify a unique human, we can also introduce a numeric field for record ID, id. This id will be a primary key.

CREATE TABLE "human" (
    "id" INTEGER PRIMARY KEY,
 ...
);

Now we have a primary key that will be used to identify a row. In practice, this id, which is an integer, must be different for each row to be a primary key. We can add one to the current id to get the next valid id. This will mean we will have to manually or programmatically take care of this and maintain it. However, the database has this constraint called AUTOINCREMENT, which will "automatically" increment the "ids" for uniqueness.

CREATE TABLE "human" (
    "id" INTEGER PRIMARY KEY AUTOINCREMENT,
 ...
);

Primary keys are not null and unique, so we don't have to specify that
Comment in Javascript is // or /* */. In SQL, it is -- for a single-line comment

For the table we have above, by default, all the fields are nullable. This means they are not required. When a field is required (needed not to be null), then we have to specify that it is not null. firstName, lastName, middleName, socialSecurityNumber and email are fields whose values are required, so we will have to set them to NOT NULL. This way, the database will require that values be explicitly passed for these fields.

CREATE TABLE "human" (
    "id" INTEGER PRIMARY KEY AUTOINCREMENT,
    "firstName" TEXT NOT NULL,
    "lastName" TEXT NOT NULL,
    "socialSecurityNumber" TEXT NOT NULL,
    "email" TEXT NOT NULL,
    "dateOfBirth" TEXT,
    "hasPet" INTEGER,
    "numberOfCars" INTEGER,
);

dateOfBirth in our case here will be nullable; as such, we have to handle it appropriately. Not every human has a pet. In a case like this, we set the default value for hasPet to false. The same applies to numberOfCars but to 0.

Why won't null be necessarily a "good" value?

CREATE TABLE "human" (
    ...
    "hasPet" INTEGER DEFAULT 0,
    "numberOfCars" INTEGER DEFAULT 0,
);

socialSecurityNumber and email are supposed to be unique values. We can programmatically handle this as well via code before the data reaches the database.

CREATE TABLE "human" (
 ...
    "socialSecurityNumber" TEXT NOT NULL UNIQUE,
    "email" TEXT NOT NULL UNIQUE,
 ...
);

You can ride the updated script via Beekeeper studio or SQLite. We can also run this via the terminal.

$ sqlite3 app.sqlite
SQLite version 3.43.2 2023-10-10 13:08:14
Enter ".help" for usage hints.
sqlite>

Even when not create, sqlite3 <DATABASE NAME.sqlite> will create a new SQLite database with name <DATABASE NAME.sqlite>
Create a file with a .sqlite extension. I will create one called app.sqlite.

sqlite> CREATE TABLE "human" (
(x1...> "id" INTEGER PRIMARY KEY AUTOINCREMENT,
(x1...> "firstName" TEXT NOT NULL,
(x1...> "lastName" TEXT NOT NULL,
(x1...> "socialSecurityNumber" TEXT NOT NULL UNIQUE,
(x1...> "email" TEXT NOT NULL UNIQUE,
(x1...> "dateOfBirth" TEXT,
(x1...> "hasPet" INTEGER DEFAULT 0,
(x1...> "numberOfCars" INTEGER DEFAULT 0
(x1...> );
sqlite> .tables
human
sqlite>

CREATE, TABLE, INTEGER, DEFAULT, etc can all be in lower case

Analyze and create a table for the profile object below.

const profile = {
    name: "John Doe",
    "date of birth": "2000-12-25",
    profession: "Software Engineer",
    "number of pets": 2,
    "weight of protein in grams": 12.5,
    "has a job": true,
};

Share what you experience

Resources

Introduction to Databases

Michael Otu — Sat, 03 May 2025 07:20:00 +0000

Great... If you are here then we have:

discussed some backend jargon
learnt some JavaScript and wrote some projects
created an api and gradually added to it

At this point what we need is persistence. Also known as storage. The means of storage that we want, in this case, is a database. We want to save the user data that we create and we want to also save the expenses that we create and update. For that to occur, there are a couple of ways to do that. These include but are not limited to:

Memory: The data is assigned to a (global) variable and manipulated to achieve our storage goal. However, it is that the data stored in memory are cleared when the server restarts. We will face what is termed as Data loss. If you have heard of the phrase, "Time is money" then here, "Data is business".
File: Saving (writing) data into a file is quite better than in-memory. The content of the file isn't lost when the server restarts. However, that storage and its management via file becomes inefficient when there are multiple write operations. This means that when we have several users, not that the users are accessing the same data but they are managing their data, simultaneously, then the is a case of we facing data loss due to simultaneously writing to the same file. Can't we use several files (with a schema - structure)? There is an issue with creating relations. We want to map or associate some set of data with another. With file, as a means of persistence, this may become inefficient.
Database: A database is better than a file. We can create and manage complex relationships.

Database Concepts

Database: A database is used to organize and manage (store, retrieve, update and delete) the data our applications depend on. A database can be seen (imagined) as a folder or a book. Our application will use a database to organize (in a well-structured manner) our users' data and expenses. We can create a simple relationship between a user and an expense.
Tables: A table is a (structured) collection of data organized in rows (values) and columns (keys/properties/fields). If a database is a book, each table is like a chapter or page containing related information.. If a database is a folder, a table will be a file. We will have tables that will represent users and expenses. A table has a name. In a table, there are rows and columns.
Column: A column is (or represents) a property/key/field in a given table. For example, for the user we had in the expense api, it had an id, email and password properties. These become columns in a table. Practically a column is the same as a property or field.

Row: A row in a table represents an instance of a record of the said table, having values that map to the columns they fall under. For example, for the user table, each row represents a "user" record.

id	email	password
c96b08b8-f92f-4e27-8f04-c4f3604907d6	john@gmail.com	passwordhash1
decec694-6d97-4199-8544-f4bbe7ea753b	mark@gmail.com	passwordhash2
b58bceeb-4041-4a50-b204-85caa2ad6c20	whatisthis@gmail.com	passwordhash3

Primary Key: In a database, there is a column which is used to uniquely identify a row. This column or columns are referred to as a primary key.
Relational Database: This is a database that is designed to favour creating and manipulating data with relations in mind. Examples of these are, but not limited to PostgreSQL, MySQL, SQLite, etc (etc is not a database). We might end up using SQLite and PostgreSQL. These databases favour or are defined using rows as records and columns as properties.
Document Database: Also known as No-SQL database, is a database designed with data format flexibility in mind. The data format is non-structured unlike in a relational database, for which it is called a No-SQL database. Practically, it more or less semi-structured. Examples of these are but are not limited to MongoDB, Redis, CouchDB, Cassandra, etc. We might end up using MongoDB and Redis.

SQL

SQL stands for Structure Query Language. It is the language of Relational databases. We will start with SQLlite and whatever we would learn here is "transferable". So, do not stress and enjoy the journey. You would want to check out this article, 7-best-sql-books-for-beginners-54gi. There are some great books and resources. Check them out and finally, there is Beekeeper Studio, a database client. We will use it to view and sometimes manage and interact with the databases we create. There is a community edition so read the docs and install whatever version of the applications you want.

In the next section, we are going to create a cheat sheet that we can reference when we are developing. Hold on tight.

Error handling and Middlewares

Michael Otu — Sat, 08 Mar 2025 17:30:00 +0000

Environmental variables
Middlewares
Validation Middleware
Error Handling middleware

In the previous excerpt, Validation, Authentication and Authorization with Libraries, we used Joi for request validation and JsonWebtoken to generate the auth tokens.

Environmental variables

When we were generating the JWT for our auth token, we passed a secret. Usually, as recommended, the secret is passed as an environmental variable. An environmental variable is a configuration value (key value) passed to or accessed from the OS. With this, we can pass data to our api without having to embed the configuration value into the api (as an exposed literal).

Open your console/terminal and enter, node. On the node REPL, enter, process.env. This is what I see:

Users-MacBook-Pro:expense-tracker-simple-api user$ node
Welcome to Node.js v22.12.0.
Type ".help" for more information.
> process.env
{
  NVM_INC: '/Users/user/.nvm/versions/node/v22.12.0/include/node',
  TERM_PROGRAM: 'vscode',
  NVM_CD_FLAGS: '',
  TERM: 'xterm-256color',
  SHELL: '/bin/bash',
  HOMEBREW_REPOSITORY: '/opt/homebrew',
  TMPDIR: '/var/folders/p8/60cwd37n1ds64r381rjkt8g80000gn/T/',
  TERM_PROGRAM_VERSION: '1.97.2',
  ORIGINAL_XDG_CURRENT_DESKTOP: 'undefined',
  MallocNanoZone: '0',
  NVM_DIR: '/Users/user/.nvm',
 ...
}
>

We can pass some configuration value to our node api by doing KEY_1=VALUE_1 KEY_2=VALUE_2 node. When we ran this command, we saw that these newly added key values appeared as part of our response this time.

NB: you have to exit the node's REPL by control + C

Users-MacBook-Pro:expense-tracker-simple-api user$ KEY_1=VALUE_1 KEY_2=VALUE_2 node
Welcome to Node.js v22.12.0.
Type ".help" for more information.
> process.env
{
  KEY_1: 'VALUE_1',
  KEY_2: 'VALUE_2',
  NVM_INC: '/Users/user/.nvm/versions/node/v22.12.0/include/node',
  NVM_CD_FLAGS: '',
  TERM: 'xterm-256color',
  SHELL: '/bin/bash',
  HOMEBREW_REPOSITORY: '/opt/homebrew',
  TMPDIR: '/var/folders/p8/60cwd37n1ds64r381rjkt8g80000gn/T/',
  TERM_PROGRAM_VERSION: '1.97.2',
  ORIGINAL_XDG_CURRENT_DESKTOP: 'undefined',
  MallocNanoZone: '0',
  NVM_DIR: '/Users/user/.nvm',
  ...
}
>

You should see KEY_1: 'VALUE_1' and KEY_2: 'VALUE_2'.

To see how this will work in our api, we can create a temporary file and print out the content of process.env

We can access the values passed using the key. So if we passed a secret, SECRET="some secret" node temp_file.js and we enter, console.log(process.env.SECRET). We would see that the output matches the value passed.

SECRET="some secret" node temp_file.js
some secret

This approach becomes impractical as such we use a .env file. env = environment. Create a file at the root of your project and call it .env. Pass all your key-value configs in there.

Before nodejs 20, we needed a library to help us access .env values. We will discuss it later. However, for all versions above, all we have to do is pass the --env-file file and set its value to the path to the .env file path: node --env-file=.env temp_file.js

node --env-file=.env temp_file.js
some secret

If you are getting anything other than the actual values passed, we can resolve this with a library called dotenv. You should know how to install packages in nodejs by now. The package name is dotenv.

npm i dotenv

In our file, we can add, require("dotenv/config"), at the top of your file. This time we just run, node temp_file.js

require("dotenv").config();
// if the above doesn't work for you, try the approach below
// require("dotenv/config")

console.log(process.env.SECRET);

If you are familiar with git, make sure you create and pass the path to the .env file into .gitignore
For you who are not familiar with git, git is a tool for managing your source code.
.env holds import configuration values and exposing them leads to a potential exploitation. Be careful but don't panic.

With this information, it is up to you now to create a .env file pass any configuration value and access it in the app as it should be.

Just to be safe, check if the value is undefined and respond appropriately - read about the difference between null and ndefined

Since we don't have to make our .env open, we create a sample.env file and pass the keys, only, so that developers can know what the .env file expects.

Middlewares

We have discussed what a controller is. This is a reminder, and perhaps if we haven't discussed it, A controller is a function (method) that handles requests, It is of the forms:

[async] function FunctionName(requestObject, responseObject) {
    // do something on the request
    // return a response
}

The point I want us to understand is that controllers are request handlers.

Request handlers are usually in two forms:

Controllers
Middlewares

Briefly, a middleware is a request handler that seats between a route and a controller.

router.method("/some route", middlewares, controller);

A middleware looks like this:

[async] function FunctionName(requestObject, responseObject, nextMiddleware) {
    // do something on the request
    // return a response if there is an error
    // else go to the next middleware
}

The only difference here is that it has a third parameter which is dubbed, next, most often. next here is the next middleware in the pipeline (controller is like a middleware).

A middleware has access to the request and response object and as such can intercept a request to alter (remove, add or update) the payload. In our index.js file, we have the following:

// parse request body as json
app.use(express.json());

// register routers
app.use(userEndpoints);
app.use(expendituresEndpoints);

app.use(express.json()); is a middleware that parses the request body into json
app.use(userEndpoints); that exposes the user endpoints to the main app
app.use(expendituresEndpoints); that exposes the expenditure endpoints to the main app

Let's create a middleware that logs the following about a request:

time
http method
and route the client visited

Most of this information can be found on the request object. req.method and req.originalUrl.

// add console logging middleware
app.use((req, res, next) => {
    const log = `${req.method} :: ${
        req.originalUrl
    } - ${new Date().toISOString()}`;

    console.log(log);

    return next();
});

Put this middleware after (or below) app.use(express.json());.

One thing to know is that middlewares are executed in the order they have been set.
app.use(express.json()); comes before any endpoint is hit, so data is parsed before it gets to that route.

With this in mind, we can now put the middleware above the registered routers (or routes), but below app.use(express.json());.

On protected routes, routes that required jwt auth, we can have a middleware that checks if the incoming request has a jwt token or not.

If you are wondering what would happen or what to do when there is no jwt, then, you should return an appropriate message or error response. The response object, res, is at your exposure.

Apparently, none of the user routes requires jwt auth however, the expenditures do. Let's create another middleware that checks if a request jwt.

Check the authorize function on how to access the jwt

Have you seen this code snippet and how many have you seen? About five?

const authReponse = authorize(req.headers.authorization);
if (!authReponse.isAuthorized) {
    return res.status(200).json({
        success: false,
        message: "Unauthorized, please log in",
    });
}

There could be more when we are more routes that will require jwt auth.

This time we are going to create a function for this middleware. Try your hands on it.

/* checks the header of the incoming request if there is a jwt */
function hasJwt(req, res, next) {
    // implementation before calling the next function
}

And create another middleware to do the authentication.

/* authenticates and authorizes user */
function isAuthorized(req, res, next) {
    // implement user authentication here
}

Create a file called middlewares.js and name the first middleware for logging the request (details) as logRequest.

(req, res, next) can have any names as far as the order (positions) of the parameters are respected.

Here is what I have or what you should look like:

// middlewares.js;
const jwt = require("jsonwebtoken");

/* Logs the request method, route and the time the request was made */
function logRequest(req, res, next) {
    const log = `${req.method} :: ${
        req.originalUrl
    } - ${new Date().toISOString()}`;

    console.log(log);

    return next();
}

/* checks the header of the incoming request if there is a jwt */
function hasJwt(req, res, next) {
    const authorization = req.headers.authorization;
    if (!authorization) {
        return res.status(200).json({
            success: false,
            message: "Unauthorized, No auth token found",
        });
    }

    return next();
}

function isAuthorized(req, res, next) {
    const authorization = req.headers.authorization;

    const JWT_SECRET = process.env.SECRET;
    if (!JWT_SECRET) {
        return res.status(200).json({
            success: false,
            message: "Something went wrong",
        });
    }

    const { userId, email } = jwt.verify(authorization, JWT_SECRET);

    // now we can fetch the user with email and userId
    const isAuthenticUser = users.find(
        (user) => user.email === email && user.id === userId
    );

    if (!isAuthenticUser) {
        return res.status(200).json({
            success: false,
            message: "Unauthorized, No user matched",
        });
    }

    req.user = { userId, email };

    return next();
}

module.exports = { logRequest, hasJwt, isAuthorized };

req.someProperty = someValue, where someProperty doesn't overwrite some existing key value.
we send a message to the next request handler and access it in the same manner. In the life cycle of the request, the request object is shared throughout.

Now we can update the list expenditures route to add the hasJwt and isAuthorized.

// list expenditures
app.get("/expenditures", (req, res) => {
    // extra auth token from headers
    const authReponse = authorize(req.headers.authorization);
    if (!authReponse.isAuthorized) {
        return res.status(200).json({
            success: false,
            message: "Unauthorized, please log in",
        });
    }

    // parse the auth user id
    const { userId } = authReponse;

    // ====================
    // get the query string and check if it is not a number or something
    // that can be a number else set a default filter value of 0
    let amountMoreThan = Number(req.query.amountMoreThan);
    if (isNaN(amountMoreThan) || amountMoreThan < 0) {
        amountMoreThan = 0;
    }

    return res.json({
        success: true,
        data: expenditures.filter(
            (row) => row.userId === userId && row.amount > amountMoreThan
        ),
    });
});

becomes

// list expenditures
app.get("/expenditures", hasJwt, isAuthorized, (req, res) => {
    // parse the userId and email from the req.user
    const { userId /*,  email */ } = req.user;

    // ====================
    // get the query string and check if it is not a number or something
    // that can be a number else set a default filter value of 0
    let amountMoreThan = Number(req.query.amountMoreThan);
    if (isNaN(amountMoreThan) || amountMoreThan < 0) {
        amountMoreThan = 0;
    }

    return res.json({
        success: true,
        data: expenditures.filter(
            (row) => row.userId === userId && row.amount > amountMoreThan
        ),
    });
});

The point here is that we can add a middleware(s) between the route and the request handler (controller).

Another exercise here is to do the same for the rest.

add hasJwt, isAuthorized,
then remove

// extra auth token from headers
const authReponse = authorize(req.headers.authorization);
if (!authReponse.isAuthorized) {
    return res.status(200).json({
        success: false,
        message: "Unauthorized, please log in",
    });
}

update

// parse the auth user id
const { userId } = authReponse;

// parse the auth user id
const { userId } = req.user;

if you are wondering where req.user is from, then it is from (or was set in) isAuthorized middleware.

Try logging in, and you will see a log, POST :: /users/login - 2025-03-02T14:23:59.310Z. The date may differ but you should have a similar output.

Using the jwt obtained on the expenditures endpoints logs the following:

POST :: /expenditures - 2025-03-02T14:28:17.173Z
GET :: /expenditures - 2025-03-02T14:28:41.074Z

At this point, your expenditure routes should look more or less like

router.get("/route", hasJwt, isAuthorized, (req, res) => {
    /*  */
});

One thing you'd notice is that hasJwt, isAuthorized, is appearing on all the routes and we might as well put it at the root of our expenditures endpoint.
For starters this is alright however since it is the same for all the routes we can go ahead on put it on the root endpoint.
Were there to be unique situations where the middleware will be an issue then we can decide to put or remove them on the affected route.

So we can either have it as,

app.use(hasJwt, isAuthorized, expendituresEndpoints);

router.get("/route", hasJwt, isAuthorized, (req, res) => {
    /* some expenditure request handling */
});

The former is more welcoming but any of them works.

We can update our index file to update our index file from

// register routers
app.use(userEndpoints);
app.use(hasJwt, isAuthorized, expendituresEndpoints);

// register routers
app.use("/users", userEndpoints);
app.use("/expenditures", hasJwt, isAuthorized, expendituresEndpoints);

And we will go into userEndpoints and expendituresEndpoints and then remove the base endpoints (routes). app.delete("/expenditures/:id",...) becomes, app.delete("/:id",...).

Validation Middleware

We are already familiar with the concept of validation, be it, by writing it ourselves or using a library such as Joi. We also know that we can pass data to our api via the headers, query strings, request parameters and body. So here, we can tell where to expect the validation to be done (as in, take its data from).

Let's use signup for a case study.

const { email, password } = req.body;

const validationResponse = authValidationSchema.validate({
    email,
    password,
});
if (validationResponse.error) {
    return res.status(200).json({
        success: false,
        message: validationResponse.error.message,
        // message: validationResponse.error.details[0].message,
    });
}

the data for the validation came from the body: const { email, password } = req.body;.
we call the validate method of a joi schema, const validationResponse = authValidationSchema.validate({ ...})
taking on some data

A normal middleware looks like this:

[async] function FunctionName(requestObject, responseObject, next) {
    // do something on the request
    // return a response
}

Which we should all know at this point. The point here is that we can pass data down to the middleware. We will create a function that returns the request handler.

function FunctionName(parameters) {
    return function (req, res, next) {
        /* do something */

        return next();
    };
}

In our case, the function parameters will be the validation schema of interest, followed by where we want to extract the data from (for the validation). Here request property can be: body, params, query or headers.

we can call this function any name you want, validation, validationMiddleware, etc

/* validation middleware: takes schema and a request property*/
function validation(schema, requestProperty) {
    return function (req, res, next) {
        const validationResponse = schema.validate(req[requestProperty]);

        if (validationResponse.error) {
            return res.status(200).json({
                success: false,
                message: validationResponse.error.message,
                // message: validationResponse.error.details[0].message,
            });
        }

        return next();
    };
}

This should look familiar.

We take the validation schema and pass it as an argument and also provide where the data will be extracted from via requestProperty.
req[requestProperty], if requestProperty was body will result in req["body"] which is req.body.

The sign up route looks like app.post("/signup", (req, res) => {...}). With the new information we have now we will add the validation middleware pass the schema and update the controller to remove the validation (done in the controller so that the controller will just be used for the logic needed).

The sign up route should look like:

app.post("/signup", validation(authValidationSchema, "body"), (req, res) => {
    /*  */
});

We can apply the same refactor to the login.

app.post("/login", validation(authValidationSchema, "body"), (req, res) => {
    /*  */
});

As an exercise, refactor the expenditure routes to make use of the general validation middleware and pass the appropriate schema to it. We can even go the extra mile by also validating the request parameters and string queries.

This is how our endpoint is supposed to look:

/* /users */
app.post("/signup", validation(authValidationSchema, "body"), (req, res) => {
    /*  */
});

app.post("/login", validation(authValidationSchema, "body"), (req, res) => {
    /*  */
});

/* /expenditures*/
app.get("/", validation(expenditureQuerySchema, "query"), (req, res) => {
    /*  */
});

app.get("/:id", validation(IdValidationSchema, "params"), (req, res) => {
    /*  */
});

app.post("/", validation(createExpenseSchema, "body"), (req, res) => {
    /*  */
});

app.put(
    "/:id",
    validation(IdValidationSchema, "params"),
    validation(updateExpenseSchema, "body"),
    (req, res) => {
        /*  */
    }
);

app.delete("/:id", validation(IdValidationSchema, "params"), (req, res) => {
    /*  */
});

Error Handling middleware

I hope maybe, you noticed, that we have been getting this kind of error:

TokenExpiredError: jwt expired
JsonWebTokenError: invalid token

It is bound to occur at some point. Anyways, we have to generally handle errors that may occur in our api consumption (integration). One way to handle errors is to use try-and-catch. We discussed about try-and-catch in JavaScript Essentials: Part 5.

We wrap the code we know might generate an error in a try-and-catch block and then return some default message when there is an error. Example:

function someRequestHandler(req, res, next) {
    try {
        /* Do something */
    } catch (error) {
        return res.status(200).json({
            success: false,
            message: "Something went wrong please, try again later.",
        });
    }
}

We have about six routes and for starts we can handle the errors simply just as shown above. There are cases where the above is okay. However, in the case of a commercial application, when an error occurs, we would want to know and resolve it right? The downside with the above implementation is that we may lose the error trace/stack, informing us what and where the error occurred. The point is, we won't know what caused the error. Express has what we call the Error-handling middleware, just like any middleware or request handler, has the (req, res, next)=> {}. What makes this special is it has four parameters instead of three: (error, req, res, next)=> {}. It is just like any order middleware but to get to it (in the pipeline or sequence of middleware) we have to pass the error that we caught in the catch (error) { /* */ } to the next function. Example:

function someRequestHandler(req, res, next) {
    try {
        /* Do something */
    } catch (error) {
        return next(error);
    }
}

So now, instead of returning a general message that, "Something went wrong please, try again later.", without even knowing what went wrong, now we have a devoted mechanism in place to parse and return an appropriate error response.

The signature of our error-handling middleware will look like this:

/* error-handling middleware */
function errorHandler(error, req, res, next) {
    /*  */
}

We will make use of the instanceof operator. The instanceof operator check is an object is an instance of some class. So far we are aware of TokenExpiredError and JsonWebTokenError, which is a result of using the jwt library.

Now we can return an appropriate response based on the error instance and a global message when we are not sure what this error is. We might be able to write the error into a log file or some remote server, etc.

// require TokenExpiredError and JsonWebTokenError from jwt
const { TokenExpiredError, JsonWebTokenError } = require("jsonwebtoken");
...


/* error-handling middleware */
function errorHandler(error, req, res, next) {
    if (error instanceof TokenExpiredError) {
        return res.status(401).json({
            success: false,
            message: "Unauthorized: please log in",
 });
 }

    if (error instanceof JsonWebTokenError) {
        return res.status(401).json({
            success: false,
            message: "Unauthorized: please format your auth token",
 });
 }

    // anything else
    return res.status(500).json({
        success: false,
        message: "Internal server error, something went wrong please try again",
        // as if the same error would not occur again when the user tries again 😂
 });
}

I am not a fan of relying on status codes. I believe my APIs are a "third party" and as such sending back these status codes that uniquely identify these specific errors is too much. What do you want the client consuming your api to do when there is a 400 error response or a 500 error response? Some http clients throw errors when the status code returned is not 200 or 201. So looking at myself as a "third party", I would an error code, status code or another key, and immediately after that key is present, then something went wrong. I have consumed a 3rd party api where there is a success and error section.

{
    "success": true,
    "statusCode": "000001",
    "error": {},
    "data": {}
}

Anyways, as we know, we can put this error-handling middleware anywhere but it is best below our registered endpoints. As an exercise, wrap your controllers all your request handlers must be wrapped in a try and catch and pass the error to the next function. Don't do this to the error-handling middleware. Frankly do it for those that need it, those that you expect that something could go wrong. There are some functions that I don't try and catch. Why? The controller has a try and catch.

So ever we are going to update will look like this:

function SomeFunction(req, res, next) {
    try {
        /*  */
    } catch (error) {
        return next(error);
    }
}

When all is done, there is a likelihood that we'd get an error response such as:

{
    "success": false,
    "message": "Unauthorized: please format your auth token"
}

Another case of an error is when the user hits an endpoint that doesn't exist. How do we handle that one and would our error-handling middleware account for it? No, the error-handling middleware won't account for it.

Well, the idea is this, we hit an endpoint say, GET http://localhost:3000/audit. We never defined an endpoint/route for audit. So in the chain of middlewares and requests handlers, in the pipeline, nothing will touch this request so express will handle it say:

<!DOCTYPE html>
<html lang="en">
    <head>
        <meta charset="utf-8" />
        <title>Error</title>
    </head>
    <body>
        <pre>Cannot GET /auditing</pre>
    </body>
</html>

Express handled it but we want to return a JSON not html. So we add one final middleware, which will just return something like endpoint not found.

Again, will put this after our error-handling middleware.

/* not found error-handling middleware */
function notFoundHandler(req, res, next) {
    // anything else
    return res.status(404).json({
        success: false,
        message: "Endpoint not found, kindly read our documentation",
    });
}

The function is used after the error handling middleware.

...
/* global error handling */
app.use(errorHandler);

/* handle cannot [METHOD] some endpoint */
app.use(notFoundHandler);
...

And we get a json message.

HTTP/1.1 404 Not Found
X-Powered-By: Express
Content-Type: application/json; charset=utf-8
Content-Length: 79
ETag: W/"4f-ZeAuDFW/On8o/z3qFxGFjcV9e4E"
Date: Mon, 03 Mar 2025 09:01:58 GMT
Connection: close

{
  "success": false,
  "message": "Endpoint not found, kindly read our documentation"
}

Resources

[Boost]

Michael Otu — Thu, 06 Feb 2025 21:47:36 +0000

Building a Polling System with Node / TypeScript using Encore.ts

Marcus Kohlberg for Encore ・ Feb 4 '25

#typescript #javascript #programming #tutorial

Validation, Authentication and Authorization with Libraries

Michael Otu — Fri, 31 Jan 2025 10:53:00 +0000

In the previous excerpt, we discussed validation, authentication and authorization. We mentioned that validation is a mechanism to ensure that data passed to the api is in an accepted format. Authentication is verifying the legitimacy of a user's claim and authorization is what a user can and cannot do. We went ahead and implemented these concepts. In this excerpt, we will look at how to use libraries/packages for validation and auth.

Content

Separation of concerns
Validation with Joi
Auth with JSON Web Tokens

Separation of concerns

We started sectioning or moving some of our code into different files. We have the data we are manipulating in the data.js file, the validations in validations.js, and so on. This process can get complicated and out of hand, so don't push it. We are building a simple application.

As usual, we are building on top of the previous codebase which can be found on GitHub and read more on it on Validation, Authentication and Authorization.

The api we are building is titled Expense Tracker. We have users and their expenses. So we are dealing with two entities. As the project grows and the content of the index.js grows, we have to pull certain codes out or group them to be precise. This is not a must, we can write all our code in one file and it will work fine. There are advantages and disadvantages to it, however, it is better to know when to group your functionalities.

In the index.js, which serves as the entry point into our application, there are:

application configurations, which will include top-level middlewares, etc
routes for expenses
routes for users
there are some snippets we can have as functions and some literals as constants
app, listen on port 3000 (server running here), may do database connections when the server starts, etc

Consider, app.SOME_METHOD(route, request handlers), SOME_METHOD is the http method, is the definition of how routes are passed with some middlewares and the request handlers are passed or handled. We have app.get("/expenditures", (req, res) => {...}) where "/expenditures" is the routes and (req, res) => {...} is a request handler. We can separate the request handler as a standalone function. We already know how to create functions (and even async functions). When we create a function outof (req, res) => {...}, we can pass it as, app.get("/expenditures", functionName). That is it. We can choose to keep the routes and the request handlers (controllers) in the same file or separate them.

We can have a file, just because our project is small else we would use folders, for the list above.

Taking actions

These are some of the changes we can make (follow along):

create a file for user and expenditures endpoints. So we'd have users.js and expenditures.js.
- move the login and sign up endpoints into users.js
- move CRUD expenditure into expenditures.js
- there are imports that we have to take care of such as the validation functions, the data consumed and others.
- remove unused imports from the index.js
import and create router from express. We can name this route app so that it will be easier to maintain the current implementation as is. Or we can give it a custom name and replace app with the custom name. I will keep to app and export it at the very bottom of users.js and expenditures.js.

// import at the very top
const app = require("express").Router();

// there are other code

// very bottom
module.exports = app;

in users.js, we have to import the validation functions and also the crypto package for creating the user ID (UUID). In all we will have an import like this:

const app = require("express").Router();
const crypto = require("crypto");
let { users } = require("./data");
const { isValidEmail, isValidPassword } = require("./validations");

// the endpoints below

// the export

in expenditures.js

const app = require("express").Router();
let { users, expenditures } = require("./data");
const { isValidName, isValidAmount, isValidDate } = require("./validations");
// the endpoints below

// the export

at this point, the index.js file should look more or less

// import the express lib
const express = require("express");

// import lib for generating uuid
const crypto = require("crypto");

// data source imports
let { expenditures, users } = require("./data");
const { isValidEmail, isValidPassword } = require("./validations");

// validation

// create an express application
const app = express();

// parse request body as json
app.use(express.json());

// create a server that listens to requests on port 3000
app.listen(3000, () =>
    console.log(`Api running on ${"http://localhost:3000"}`)
);

we have to clean it since some of these imports are unused and we will import the routers we exported from users.js and expenditures.js.

// import the express lib
const express = require("express");

// import the routers exported from users and expenditures
const userEndpoints = require("./users");
const expendituresEndpoints = require("./expenditures");

// create an express application
const app = express();

// parse request body as json
app.use(express.json());

// register routers
app.use(userEndpoints);
app.use(expendituresEndpoints);

// create a server that listens to requests on port 3000
app.listen(3000, () =>
    console.log(`Api running on ${"http://localhost:3000"}`)
);

the signup and login work fine, you should test them too
we can create a new user (copy the newly created user's log into the data file)
update the http request for expenditures endpoints by adding Authorization to each request
the code for authorizing the user to browse their expenditures is only found in the list expenditures endpoint. Let's "copy" and "paste" it... 🥳🥳🥳 (that is what programmers do but software engineers will create a reusable block of code from it, isn't that right?)
let's a function for authorize in a file called auth.js. This function will take in the auth token and a boolean indicating if the token is authorized and an object which can be used to identify the user, { userId, email }.
we want to change

// extra auth token from headers
const authorization = req.headers.authorization;
if (!authorization) {
    return res.status(200).json({
        success: false,
        message: "Unauthorized, please log in",
    });
}

// decode auth token
const decodeToken = Buffer.from(authorization, "base64").toString();

// parse the decoded token into the email and userId
const [email, userId] = decodeToken.split(":");

// now we can fetch the user with email and userId
const isAuthenticUser = users.find(
    (user) => user.email === email && user.id === userId
);

if (!isAuthenticUser) {
    return res.status(200).json({
        success: false,
        message: "Unauthorized, please log in",
    });
}

into

// auth.js;

const { users } = require("./data");

// pass the req.headers.authorization as argument
function authorize(authToken) {
    const response = {
        isAuthorized: false,
        userId: "",
        email: "",
    };

    if (!authToken) {
        return response;
    }

    // decode auth token
    const decodeToken = Buffer.from(authToken, "base64").toString();

    // parse the decoded token into the email and userId
    const [email, userId] = decodeToken.split(":");

    // now we can fetch the user with email and userId
    const isAuthenticUser = users.find(
        (user) => user.email === email && user.id === userId
    );

    if (!isAuthenticUser) {
        return response;
    }

    // here the token is valid
    response.isAuthorized = true;
    response.email = email;
    response.userId = userId;

    return response;
}

module.exports = { authorize };

let's add this guard into each expenditures endpoint.

// we have to import the authorize function from auth.js

// extra auth token from headers
const authReponse = authorize(req.headers.authorization);
if (!authReponse.isAuthorized) {
    return res.status(200).json({
        success: false,
        message: "Unauthorized, please log in",
    });
}

// parse the auth user id
const { userId } = authReponse;

Try running some of these endpoints... we have passed the auth token and we have a guard in place. Don't believe everything I say 🙈

back to "copy and paste"... We will learn about middlewares later and update our to use middleware.

Running list expenditures is successful
we have to update read expenditure to account for the userId from the auth response and also the resource ID is not a number but a uuid. we will remove the checking if the id is a number and use expenditures.find (or filter if you want to but handle the rest) instead of expenditures[id]

// read expenditure
app.get("/expenditures/:id", (req, res) => {
    // extra auth token from headers
    const authReponse = authorize(req.headers.authorization);
    if (!authReponse.isAuthorized) {
        return res.status(200).json({
            success: false,
            message: "Unauthorized, please log in",
        });
    }

    // parse the auth user id
    const { userId } = authReponse;

    // get the id of the requested resource from the params
    const id = req.params.id;

    const row = expenditures.find(
        (row) => row.userId === userId && row.id === id
    );
    if (!row) {
        // what status code do this should be passed here? 404 not not found? why??
        return res.status(200).json({
            success: false,
            message: `Resource with ID, '${id}' not found`,
        });
    }

    return res.status(200).json({
        success: true,
        data: row,
    });
});

When we run create, the record is created however is not assigned to the user who created it. Did you notice that? Create a very small request to fetch all expenditures or console.log the expenditures variable you are fetching the user's expenditures. This was because we added the authorize, from which we got userId but we didn't use it. All we have to do is add the userId to the expense record and generate a UUID (sample code is in the users.js, sign up).

// generate uuid unique to this expense
const uuid = crypto.randomUUID();

// insert the new record
expenditures.push({
    id: uuid,
    userId,
    name: payload.name,
    amount: payload.amount,
    date: payload.date,
});

for update expenditure:
- to update the expenditure, we can find the index of this (expenditure of interest) and if we get the index, we can update it. There is the map and match approach too where we map the expenditures and match the resource ID to a record and update that record, reassigning the expenditures with the newly created array.
- remove the number checking for resource ID (did similar in read expenditure)
- replace const row = expenditures[id]; with const rowIndex = expenditures.findIndex(...) accounting for the userId. We will use this rowIndex to assign value to row and also use it to index the row we wanted to be updated from the expenditures.
- we will replace, expenditures[id].PROPERTY = PROPERTY ?? row.PROPERTY; with expenditures[rowIndex].PROPERTY = PROPERTY ?? row.PROPERTY;
- update the http request to use the uuid instead of a number.

// update expenditure
app.put("/expenditures/:id", (req, res) => {
    // extra auth token from headers
    const authReponse = authorize(req.headers.authorization);
    if (!authReponse.isAuthorized) {
        return res.status(200).json({
            success: false,
            message: "Unauthorized, please log in",
        });
    }

    // parse the auth user id
    const { userId } = authReponse;

    // get the id of the requested resource from the params
    const id = Number(req.params.id);

    // if the record is not found, rowIndex becomes -1
    const rowIndex = expenditures.findIndex(
        (row) => row.userId === userId && row,
        id === id
    );
    if (rowIndex === -1) {
        // what status code do this should be passed here? 404 not not found? why??
        return res.status(200).json({
            success: false,
            message: `Resource with ID, '${id}' not found`,
        });
    }

    const row = expenditures[rowIndex];

    // we have to validate the name, maybe, their name must have some number of characters
    // we have to make sure that amount is a number
    // we have to also make sure that the date is of the format, yyyy-MM-dd
    const { name, amount, date } = req.body;

    if (!name || !amount || !date) {
        return res.status(200).json({
            success: false,
            message: "name, amount and date for expense are required",
        });
    }

    // validate name if it is passed
    if (name && !isValidName(name)) {
        return res.status(200).json({
            success: false,
            message: "Name is invalid",
        });
    }

    // validate amount if it is passed
    if (amount && !isValidAmount(amount)) {
        return res.status(200).json({
            success: false,
            message: "Amount is invalid",
        });
    }

    // validate date if it is passed
    if (date && !isValidDate(date)) {
        return res.status(200).json({
            success: false,
            message: "Expense date is invalid",
        });
    }

    expenditures[rowIndex].name = name ?? row.name;
    expenditures[rowIndex].amount = amount ?? row.amount;
    expenditures[rowIndex].date = date ?? row.date;

    // there was a time I saw a 204 status - No content
    // since there was no data to return however we'll return the usual
    return res.status(200).json({
        success: true,
        message: "expenditure updated successfully",
    });
});

for delete expenditure:
- most of the changes to be done are similar to the update
- remove the number checking for the resource ID
- find the rowIndex, if it is -1 then there is no row found
- pass the rowIndex in place of id in expenditures = expenditures.filter((_, index) => index !== id);
- Give it a try

At this point, we have :

rewritten most parts of this api
separated the endpoints into their various files
fixed file paths and imports and exports
fixed the resource ID validations
fixed or patched the auth mechanism
fixed managing resources, accounting for userId

Make sure you have a working api by testing (hitting) each endpoint and that you get the appropriate response. Use console logs if you have to, read the error logs displayed in the console.

Validation

I want to point out that even though we wrote most of our validations, we can use a library for that. As we discussed some time ago, a library is a piece of code that does something specific and can be reused, and bundled as one (as in whatever functionalities the library provides). Usually, libraries are well-versioned, maintained and tested. Using a library removes some sort of responsibility in maintaining some functionality. This would save time and reduce the chunks of code "we" write. There are also downsides to this. Some libraries may be outdated and be vulnerable to some attacks as a result of not being maintained and up to date with the current (nodejs) runtime. As such be careful with using third-party libraries.

For validation libraries, there are several and the one we will use today is Joi. At the time of writing this excerpt, the current version was 17.13.3. Spend some time on this this page just to see how to use Joi.

By the way, run, npm i joi, to install Joi

Replicating validations.js

This is something we'd be doing on the side then integrate later.

From the general usage page, validation with Joi is done in two simple steps.

Create a schema
Use the schema to validate your data
Handle the response, error and value. Error is the data that failed the validation criteria. Value is the validated data (passed as argument).

We can validate the literal types: number, string, bool, and an array and an object as well.

Validate Sign up and Login Credentials

There is no difference between the sigu up validation and log in validation, both have email and password.

Just to be clear, create a new file for the validation with Joi. (joi_validations.js)
We will be referencing the old validations rewritten

We want to create a schema that has email and password properties.

const authValidationSchema = Joi.object().keys({
    email: Joi.string().email().required(),
    password: Joi.string().min(6).required(),
});

Assuming we have, this below, as credentials.

const credentials = {
    email: "johndoe1@gmail.com",
    password: "JohnPwd12_",
};

Then we can validate the credentials as follows:

const { error, value } = authValidationSchema.validate(credentials);

console.log({ error, value });

/* console log
{
 error: undefined,
 value: { email: 'johndoe1@gmail.com', password: 'JohnPwd12_' }
}
*/

Let's say we passed, intentionally for demo purposes, invalid credentials:

const inValidCredentials = {
    email: "johndoe1gmail.com",
    password: "John",
};

The schema.validate method, takes an argument of objects, options. The second argument is nullable or rather has some default values and among these, there are:

abortEarly: By default is true. When true, stops validation on the first error, otherwise returns all the errors found. So if the email is invalid, it will stop the validation, ignoring the password validation, and return the validation response.
allowUnknown: By default is false. When true, allows the object to contain unknown keys which are ignored. Let's say we are expecting email and password but the user is passing ide as part of the request, we can ignore this noise by setting allowUnknown to true.
convert: by default is true. When true, attempts to cast values to the required types (e.g. a string to a number).

You have to think about these because these options can affect the user interface and the user experience

Aborting early means, I will have one error field and UI will have just one error component (div or something) else UI would either have to join the list of error messages together and display them as one or have different alerts for each error. Depending on what decision is made, this will affect the User experience.

const invalidResult = authValidationSchema.validate(inValidCredentials, {
    abortEarly: false,
});

console.log(invalidResult.value);
// console log
// { email: 'johndoe1gmail.com', password: 'John' }

console.log(invalidResult.error);

/* console log
[Error [ValidationError]: "email" must be a valid email. "password" length must be at least 6 characters long] {
 _original: { email: 'johndoe1gmail.com', password: 'John' },
 details: [
 {
 message: '"email" must be a valid email',
 path: [Array],
 type: 'string.email',
 context: [Object]
 },
 {
 message: '"password" length must be at least 6 characters long',
 path: [Array],
 type: 'string.min',
 context: [Object]
 }
 ]
}
*/

We can write a custom validation method.

This is what we have for the previous password validation.

function isValidPassword(password) {
    if (!password || !isString(password) || password.length < 6) {
        return false;
    }

    const UPPER_CASE = LETTERS.map((char) => char.toUpperCase());
    if (!hasAnyOf(password, UPPER_CASE)) {
        return false;
    }

    if (!hasAnyOf(password, NUMBERS)) {
        return false;
    }

    return hasAnyOf(password, SPECIAL_SYMBOLS);
}

We can make this function a method to be used in the current validation schema by passing it a method to the custom method.

the reference to constants are inferred (we already have them, 🙈 copy them over or export them from validations.js) and here we either return the value or throw an error when it is not valid.

We define a function that takes in the password (or the value to validate) and a second parameter, called, helper. (We will not use it but just pass it) - Read on helpers here.

So that there would be no naming conflicts, the validation method will be named, customPasswordValidation

const customPasswordValidation = (value, helpers) => {
    if (!value || typeof value !== "string") {
        throw new Error("Password invalid");
    }

    if (value.length < 6) {
        throw new Error("Password must be at least 6 characters");
    }

    const UPPER_CASE = LETTERS.map((char) => char.toUpperCase());
    if (!hasAnyOf(value, UPPER_CASE)) {
        throw new Error(
            "Password must include at least one uppercase character"
        );
    }

    if (!hasAnyOf(value, NUMBERS)) {
        throw new Error("Password must inlcude at least one numeric character");
    }

    if (!hasAnyOf(value, SPECIAL_SYMBOLS)) {
        throw new Error(
            `Password must include at least one of these special characters: [${SPECIAL_SYMBOLS.join(
                ","
            )}]`
        );
    }

    return value;
};

Let's try out this password validation.

const passwords = ["", "John", "john123", "John123", "John123_"];

const customPasswordSchema = Joi.string().custom(
    customPasswordValidation,
    "custom password validation"
);

const errorList = passwords.reduce((errors, password) => {
    const { error } = customPasswordSchema.validate(password);

    errors.push({
        [password]:
            error?.details?.map((log) => log.message).join("and") ?? "No error",
    });

    return errors;
}, []);

console.log(errorList);
/* console log
[
 { "": '"value" is not allowed to be empty' },
 {
 John: '"value" failed custom validation because Password must be at least 6 characters',
 },
 {
 john123:
 '"value" failed custom validation because Password must include at least one uppercase character',
 },
 {
 John123:
 '"value" failed custom validation because Password must include at least one of these special characters: [$,_,-]',
 },
 { John123_: "No error" },
]; 
*/

At this point, we can write some custom validation with Joi. Now we want to make use of this validation scheme where needed .i.e in the sign up and log in. We can comment out or remove (you decide) the unwanted code from the joi_validations.js and export the authValidationSchema accounting for the customPasswordValidation.

// joi_validations.js;
const Joi = require("joi");
const {
    hasAnyOf,
    LETTERS,
    NUMBERS,
    SPECIAL_SYMBOLS,
} = require("./validations");

const customPasswordValidation = (value, helpers) => {
    if (!value || typeof value !== "string") {
        throw new Error("Password invalid");
    }

    if (value.length < 6) {
        throw new Error("Password must be at least 6 characters");
    }

    const UPPER_CASE = LETTERS.map((char) => char.toUpperCase());
    if (!hasAnyOf(value, UPPER_CASE)) {
        throw new Error(
            "Password must include at least one uppercase character"
        );
    }

    if (!hasAnyOf(value, NUMBERS)) {
        throw new Error("Password must inlcude at least one numeric character");
    }

    if (!hasAnyOf(value, SPECIAL_SYMBOLS)) {
        throw new Error(
            `Password must include at least one of these special characters: [${SPECIAL_SYMBOLS.join(
                ","
            )}]`
        );
    }

    return value;
};

const authValidationSchema = Joi.object().keys({
    email: Joi.string().email().required(),
    password: Joi.string()
        .custom(customPasswordValidation, "custom password validation")
        .required(),
});

module.exports = { authValidationSchema };

In user.js, we will import authValidationSchema from joi_validations.js and replace:

if (!isValidEmail(email)) {
    return res.status(200).json({
        success: false,
        message: "Invalid email",
    });
}

if (!isValidPassword(password)) {
    return res.status(200).json({
        success: false,
        message: "Invalid password",
    });
}

with

const validationResponse = authValidationSchema.validate({
    email,
    password,
});
if (validationResponse.error) {
    return res.status(200).json({
        success: false,
        message: validationResponse.error.message,
        // message: validationResponse.error.details[0].message,
    });
}

We will do the same for login and clean up unused imports.

Validate expense

From the http requests that we have we are to validate the expense on creation and update. During creation, all the fields are required but optional during update.

const createExpenseSchema = Joi.object().keys({
    name: Joi.string().min(10).max(255).required(),
    amount: Joi.number().positive().required(),
    date: Joi.date().iso().required(),
    // date: Joi.string().isoDate().required(),
});

const updateExpenseSchema = Joi.object().keys({
    name: Joi.string().min(10).max(255).optional(),
    amount: Joi.number().positive().optional(),
    date: Joi.date().iso().optional(),
    // date: Joi.string().isoDate().optional(),
});

module.exports = {
    authValidationSchema,
    createExpenseSchema,
    updateExpenseSchema,
};

In create expenditures, we will replace the individual validates with:

...
// const { name, amount, date } = req.body
const payload = req.body;

const validationResponse = createExpenseSchema.validate(payload);
if (validationResponse.error) {
    return res.status(201).json({
        success: false,
        message: validationResponse.error.message,
 });
}

In update,

...
const { name, amount, date } = req.body;

if (!name && !amount && !date) {
    return res.status(200).json({
        success: false,
        message: "name, amount or date for expense are required",
    });
}

const validationResponse = updateExpenseSchema.validate({
    name,
    amount,
    date,
});

if (validationResponse.error) {
    return res.status(200).json({
        success: false,
        message: validationResponse.error.message,
     });
}

Auth with JSON Web Tokens

For our auth token we combined the user's email and ID (uuid) and then encoded it. It is not safe, it is not secure and THAT IS CRIMINAL. Check out this platform, BASE64 Decode and Encode. This is a platform that allows you to decode and encode some stringified payloads. Copy and paste any of your auth tokens and you'll realize that an attacker wouldn't need nodejs or some programming runtime to know how to generate an auth token. All they have to do, to get a new auth token, is to get a user ID (uuid) and the rest comes and goes into the past, history. For this case, we will use JSON WebTokens. Our initial auth token doesn't expire but JWTs could.

JSON Web Tokens is a ... No! No definitions. Well A JSON Web Token, is a digitally signed token of an encoded payload in a JSON format, that is shared between and used for communication between two parties. JWT is encrypted and its data can be decoded. Head over to jwt.io, a platform that allows you to decode, verify and generate jwts.

A JWT has three parts, the header, payload, and signature.

So this is the jwt I modified on jwt.io, it is of the format, HEADER.PAYLOAD.SIGNATURE.

eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzdWIiOiIxMjM0NTY3ODkwIiwibmFtZSI6IkpvaG4gRG9lIiwiaWF0IjoxNTE2MjM5MDIyLCJkb2IiOiIyMDA0LTEyLTAyIn0.NjrWV_CeOphs6WR676B-oyVk5S4mafNZW88SAVrzY0I

The header, eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9, indicates which hash algorithm was used and what type of token we are creating, in our case, jwt.

{
    "alg": "HS256",
    "typ": "JWT"
}

Visit BASE64 Decode and Encode or toggle the decode part, and decode the header and payload.
The payload, indicates or contains what is in the encoded data or the data we want to associate with the user: a way to identify the user.

{
    "sub": "1234567890",
    "name": "John Doe",
    "iat": 1516239022,
    "dob": "2004-12-02"
}

The signature is a cryptographic hash of the of the headers and payloads. From the header, we use HS256 to hash the header and payload.
On jwt.io, what happens when you add another key value to the payload? Does the signature change? If it changed then it means when an attacker gets an auth token from our platform, they can not replicate it. They have to sign it. Their secret for hashing will not be the same as ours as such our system will not accept it as a valid auth token. For a jwt with a longer TTL, (Time To Live, expiration time), an attacker can use that compromised token to wreak havoc until the token expires. So usually one is advised to use short TTLs. We whitelist valid tokens or blacklist invalidated tokens (the user logs out but the token is still invalid). We have to be creative.

Using JWT

install jsonwebtoken, npm install jsonwebtoken

Read about the jwt lib from here and a more technical version from, rfc7519.

As usual, just like the Joi library, the usage of the library speeds up our work. This is another reason to use the code another developer has written.

There are three methods to:

sign (create a new) token: pass the payload, the secret and some options like the expiresIn property which defines the TTL for the token
decode jwt: decodes the payload of the jwt, however, does not verify the jwt
verify jwt: verifies the jwt. Has it expired? Is it malformed and all that?

This was how we generated the auth token:

const token = Buffer.from(`${email}:${uuid}`).toString("base64");

But now we are doing it differently, we are going to use the sign(payload, secret, options) method from the jwt lib. We will use the email and uuid as payload. Generally, the sign format is of the form:

const jwt = require("jsonwebtoken");

const authToken = jwt.sign(
    {
        id: "c96b08b8-f92f-4e27-8f04-c4f3604907d6",
        email: "john@gmail.com",
    },
    "SOME SERECT",
    {
        expiresIn: "1h",
    }
);

console.log(authToken);
// eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6ImM5NmIwOGI4LWY5MmYtNGUyNy04ZjA0LWM0ZjM2MDQ5MDdkNiIsImVtYWlsIjoiam9obkBnbWFpbC5jb20iLCJpYXQiOjE3MzgyMTM0NTAsImV4cCI6MTczODIxNzA1MH0.3edLKTeSlgo8vThEAmDJ1uC5GU58AxeLzNpSgVAOYvc

The sign method takes in the payload, which is part of the claims. Refer to the documentation for more details.

For demo purposes, I am passing the secret as "SOME SERECT". In a production application, we'd hide these from the code itself. We will put it into an environmental variable path or file.

DO NOT ADD YOUR SECRET TO YOUR APPLICATION. YOU WILL BE IN A LOT OF TROUBLE, I think 😒.

We will look into how to handle environmental variables later.

In the options object parameter, we have the option to pass an algorithm but the default is HS256. Check the docs for the other options. What we can do is assign the algorithm property to HS384 to set it to HS384.

Now on the expiresIn: "1h", it indicates that the token becomes invalid or expires in one hour. This is directly from the docs:

Eg: 1000, "2 days", "10h", "7d". A numeric value is interpreted as a second count. If you use a string be sure you provide the time units (days, hours, etc), otherwise, the milliseconds unit is used by default ("120" is equal to "120ms").

When you set the TTL to a number, 3600, it is understood that you want the TTL to be in seconds. 3600 one hour.
When you set the TTL to a number string, "3600", it is assumed that you want the TTL to be in milliseconds. S0 "3600" in parsed as 3.6 seconds.
There are other formats as well, specified by the docs on, vercel/ms and I will share a snippet describing some time span:

ms("2 days"); // 172800000
ms("1d"); // 86400000
ms("10h"); // 36000000
ms("2.5 hrs"); // 9000000
ms("2h"); // 7200000
ms("1m"); // 60000
ms("5s"); // 5000
ms("1y"); // 31557600000
ms("100"); // 100
ms("-3 days"); // -259200000
ms("-1h"); // -3600000
ms("-200"); // -200

What we are interested in here is "2 days" for two days, "1d" for one day, etc. We used "1h" for one hour. I think it is clear how to use the ms time span description to define the time span of the ttl.

So the sign.(PAYLOAD, SECRET, { expiresIn: X }), generates a jwt with payload, PAYLOAD signed with the secret, SECRET, which will become invalid in X specified time span.

The decode(authToken) method decodes the payload without verifying it.

console.log(
    jwt.decode(
        "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6ImM5NmIwOGI4LWY5MmYtNGUyNy04ZjA0LWM0ZjM2MDQ5MDdkNiIsImVtYWlsIjoiam9obkBnbWFpbC5jb20iLCJpYXQiOjE3MzgyMTM0NTAsImV4cCI6MTczODIxNzA1MH0.3edLKTeSlgo8vThEAmDJ1uC5GU58AxeLzNpSgVAOYvc"
    )
);
/* 
{
 id: 'c96b08b8-f92f-4e27-8f04-c4f3604907d6',
 email: 'john@gmail.com',
 iat: 1738213450,// seconds
 exp: 1738217050 // seconds
}
*/

Try removing or adding some characters to the auth token, and then decode. What value do you get?

The verify(authToken, secret), verifies then decodes the auth token on success, else throws an error.

console.log(
    jwt.verify(
        "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpZCI6ImM5NmIwOGI4LWY5MmYtNGUyNy04ZjA0LWM0ZjM2MDQ5MDdkNiIsImVtYWlsIjoiam9obkBnbWFpbC5jb20iLCJpYXQiOjE3MzgyMTM0NTAsImV4cCI6MTczODIxNzA1MH0.3edLKTeSlgo8vThEAmDJ1uC5GU58AxeLzNpSgVAOYvc",
        "SOME SERECT"
    )
);
/* 
{
 id: 'c96b08b8-f92f-4e27-8f04-c4f3604907d6',
 email: 'john@gmail.com',
 iat: 1738213450,// seconds
 exp: 1738217050 // seconds
}
*/

Now when the auth token is faulty or has expired, it is likely that you may encounter one of these errors:

TokenExpiredError: when the token has expired

   TokenExpiredError: Thrown error if the token is expired.
       if (err) throw err;
               ^
   TokenExpiredError: jwt expired
    at /Users/user/expense-tracker-simple-api/node_modules/jsonwebtoken/verify.js:190:21
    at getSecret (/Users/user/expense-tracker-simple-api/node_modules/jsonwebtoken/verify.js:97:14)
    at module.exports [as verify] (/Users/user/expense-tracker-simple-api/node_modules/jsonwebtoken/verify.js:101:10)
    at Object.<anonymous> (/Users/user/expense-tracker-simple-api/auth.js:82:9)
    at Module._compile (node:internal/modules/cjs/loader:1565:14)
    at Object..js (node:internal/modules/cjs/loader:1708:10)
    at Module.load (node:internal/modules/cjs/loader:1318:32)
    at Function._load (node:internal/modules/cjs/loader:1128:12)
    at TracingChannel.traceSync (node:diagnostics_channel:322:14)
    at wrapModuleLoad (node:internal/modules/cjs/loader:219:24) {
   expiredAt: 2025-01-30T06:04:10.000Z
   }

JsonWebTokenError: when there is an issue with the token or the secret

   JsonWebTokenError
       if (err) throw err;
               ^
   JsonWebTokenError: invalid signature
    at /Users/user/expense-tracker-simple-api/node_modules/jsonwebtoken/verify.js:171:19
    at getSecret (/Users/user/expense-tracker-simple-api/node_modules/jsonwebtoken/verify.js:97:14)
    at module.exports [as verify] (/Users/user/expense-tracker-simple-api/node_modules/jsonwebtoken/verify.js:101:10)
    at Object.<anonymous> (/Users/user/expense-tracker-simple-api/auth.js:82:9)
    at Module._compile (node:internal/modules/cjs/loader:1565:14)
    at Object..js (node:internal/modules/cjs/loader:1708:10)
    at Module.load (node:internal/modules/cjs/loader:1318:32)
    at Function._load (node:internal/modules/cjs/loader:1128:12)
    at TracingChannel.traceSync (node:diagnostics_channel:322:14)
    at wrapModuleLoad (node:internal/modules/cjs/loader:219:24)

NotBeforeError: Thrown if the current time is before the nbf claim.

We will be looking into how to handle these errors later on

Integrating JWT

To integrate the use of jwt in our application, we need to modify a few parts of api such as the sign up and login in users.js and authorize(authToken) function in auth.js.

In users.js, let's import the jwt library as, const jwt = require("jsonwebtoken");. Then replace all instances of:

const token = Buffer.from(`${email}:${uuid}`).toString("base64");

with the snippet below for sign-up

const token = jwt.sign(
    {
        userId: uuid,
        email,
    },
    "SOME SERECT",
    {
        expiresIn: "1h",
    }
);

and for login

const token = jwt.sign(
    {
        userId: user.id,
        email: user.email,
    },
    "SOME SERECT",
    {
        expiresIn: "1h",
    }
);

Since this looks similar, we can wrap in a function and call the function passing in the payload. The secret and ttl, are always the same.

As an exercise, create a function in the auth.js file, called generateJwt({ userId, email}) and return the token generated.

When we try the log in, we should get a response similar to

{
    "success": true,
    "message": "Login successful",
    "data": {
        "id": "b58bceeb-4041-4a50-b204-85caa2ad6c20",
        "email": "johndoe1@gmail.com",
        "token": "eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJ1c2VySWQiOiJiNThiY2VlYi00MDQxLTRhNTAtYjIwNC04NWNhYTJhZDZjMjAiLCJlbWFpbCI6ImpvaG5kb2UxQGdtYWlsLmNvbSIsImlhdCI6MTczODIzMTE5MiwiZXhwIjoxNzM4MjM0NzkyfQ.GYXR3qhJ_NdXwbjjrHbel-i7cKSBOX2izMBoO6hOhK4"
    }
}

You can copy and paste the auth token into jwt.io to inspect the token.

Do the same for sign-up.
Replace the authorization tokens with the new ones from the login or sign-up response

In auth.js, we will replace

// decode auth token
const decodeToken = Buffer.from(authToken, "base64").toString();

// parse the decoded token into the email and userId
const [email, userId] = decodeToken.split(":");

with

const { userId, email } = jwt.verify(authToken, "SOME SERECT");

Remember to import jwt.

Conclusion

We have come a long way here and gradually we will plough through, surging forward. I could argue that this excerpt is about the importance of using libraries. We can reduce the number of codes by using libraries. The number of code we "reduce" by not writing is written by the library authors. Maybe some code but less writing on our part.

Joi and jsonwebtoken are one of the few implementations of these concepts. For our use case, this is enough and we meet shortly.

Validation, Authentication and Authorization

Michael Otu — Thu, 16 Jan 2025 17:53:54 +0000

Content

Validation
Authentication
Authorization
Adding validation
Adding Authentication and Authorization

In the previous excerpt, Basic CRUD API with express, we discussed creating a CRUD API with express. We installed extensions for our development environment. We also installed express, an npm package, to create our server. We discussed how to create POST, GET UPDATE and DELETE routes, and how to parse (access) data from the request body, query string and request parameter.

This excerpt will discuss validation, authentication and authorization. We will be building on top of the previous excerpt. Refer to the previous material for the code snippet. You can also reference this material from here.

We are using the snippet from the previous excerpt. You can follow that to have the starter code.
Since we always start on a fresh environment, I will have to follow the process of creating a node app and copy the content of the index.js file and package.json. Remember to run npm i or yarn to install the npm packages (if you cloned the project)
If you are having any issues, the project can be found here on GitHub, expense-tracker-api-articles/01-basic-crud-api-with-express/expense-tracker-simple-api or drop a comment. I or any capable individual may attempt to help you, as they see fit.

Validation

Before we jump into definitions of what is what and all that, let me draw your attention to why we have to validate data received from users by referencing some articles written on DEV. Check out these articles to have a different voice on why validations is importance. I think it will spare us the definition of validation.

Why Form Validation Is Important > You need form validation as a security measure. Forms are an easy target for hackers because we all know they're connected to a database somehow. Letting people put whatever they want into your form opens you up to SQL injection attacks as a start and it can get way more advanced than that.
Input Validation: Client-side or Server-side? > There is a rule of thumb to never trust the user's input
Data Validation in Your Backend: A Practical Guide > Failing to validate data can lead to security vulnerabilities, bugs, and data integrity issues
Data validation is a vital step in creating reliable and secure applications. > Data validation is a vital step in creating reliable and secure applications.
In Defense of D > The function has inputs. And the function shouldn't be aware of where those inputs originated. Therefore, from the perspective of the function, the inputs are all potentially dangerous.

From these very short portions about validations from these articles, it should be clear why validation is important and why it should matter. They all point out that it will be a security disaster if proper validations aren't done and this obviously will/may lead to a loss, a financial loss, primarily. One's business may lose data, users, integrity, money, etc as a result of a security breach. As such I, and the community as a whole, encourage you to validate data before inserting or updating existing records.

There are several instances where some client (app) communicates with the backend and these moments are mostly submission of forms.

In the previous excerpt where we were creating and updating expenses, what should we "consider" as an appropriate or acceptable value for name, amount and date?

In this case, the name should specify what object, element or event the expense was for. It could be a word or phrase that describes the expense.
For the amount, usually, the question should be, do we consider a negative value or even zero in this case as an acceptable value for the amount? What is the limit of the value that the user can pass? This is in a sense that, can a user pass 1 million or 1 billion?
For the date value, we should consider the format of the expected expense date. We can have it that the date should start with the year followed by the month and then the day. The year, the month and the day can be separated by some specified character. We used a hyphen, -. So here you can specify whether the date should be separated by a forward slash, /, or even a colon, :. You decide.

If you have been following this series, then you know that we have written some validations in the past. From, JavaScript Essentials: Part 6 (Mastermind in Javascript), this is an example of a validation:

function isValidRound(rounds) {
    return MIN_ROUNDS <= rounds && rounds <= MAX_ROUNDS && rounds % 2 == ZERO;
}

Let's define some validation rules:

Name
- Must be a string
- Must not be empty or null (it is required)
- Minimum characters of 10
- Maximum characters of 255
Amount
- Must be a number or numeric string
- Must not be empty (it is required)
- Must be a positive number
Date
- Must be a string
- Could be empty or null (it is an option, current date will be used)
- Must be in ISO format

As an exercise, implement three functions that take in the specified data, validate it and return a boolean.

function isValidName(name) {
    // returns a bool
}

function isValidAmount(amount) {
    // returns a bool
}

function isValidDate(date) {
    // returns a bool
}

It will be cool to have them exported from another file called validation or some awesome name that hints at what these functions are doing.
It is also a good practice to separate concerns. If this sounds weird or strange then we can simply say that it is better to reduce the number of code in the index.js file.

At this point even if we don't define what validation is, with what we have discussed so far, one can have a solid idea of what validation is and its importance.

So, what is validation? Care to share with us what you believe it is, at this point?

Authentication

Earlier on with discussed that validation is a mechanism that ensures that the values or input taken from a user is an appropriate or an acceptable value that conforms to certain requirements or standards. The idea of authentication in some way is similar to validation.

In authentication, where we make sure that the claims of a user, passing their credentials to have access to some account, is legitimate. This means that we check or verify that a user is who they say they are, based on the credentials they pass. Mostly, these credentials are made up of a public identifier which could be an email, username or some string token (an alias which can be shared with others), and a secret which is dubbed as a password (🙈 not safe to share). We first check if there exists a user with such an identifier, we then cross-check the password or the secret passed alongside the identifier against the password or the secret of the record obtained concerning the identifier. So in authentication, the identity of a user is verified.

Authorization

Authorization on the other hand has to do with what the user can and cannot do on a platform. So a user that claims to be John "cannot" assess a record that doesn't belong to John. John can not create, read, update or delete expense records that belong to Mark. John is not authorized access nor modify Mark's records. However, John "can" create, read, update and delete expense records that belong to John. Another form that authorization can appear is that you have an admin and these admin may have some sort of access levels or permissions. This access level determines what the admin can and cannot do on the platform. Usually, an admin that is a moderator, can only flag and report some accounts and articles, or downvote them (during moderation). Surely there is this admin with enough privileges to even block an active user and even delete or hide an article or a comment.

Adding validation

I implemented my validations for the name, amount and date. Some comments explain some decisions made.

// file name: validations.js

// typeof operator returns the type of a value

// check that a value is a string
const isString = (arg) => typeof arg === "string";

// check that a value is a number
// const isNumber = (arg) => typeof arg === "number";
// if we use this approach then, "2.99" fails as a number even though it is a numeric string
// as such we are going with this and later casting the values to numbers
const isNumber = (arg) => !isNaN(Number(arg));
// when arg is non-numeric or a numeric string, Number(arg) returns NaN
// isNaN checks if a value is NaN and returns true if so else false
// which is why we negate the results from isNaN(...)

/**
 *
 * @param {*} name
 * -   Must be a string
 * -   Must not be empty or null (it is required)
 * -   minimum characters of 10
 * -   maximum characters of 255
 * @returns boolean
 */
function isValidName(name) {
    if (!name || !isString(name)) {
        return false;
    }

    return name.length >= 10 && name.length <= 255;
}

/**
 *
 * @param {*} amount
 * -   Must be a number or numeric string
 * -   Must not be empty (it is required)
 * -   Must be a positive number
 * @returns boolean
 */
function isValidAmount(amount) {
    if (!amount || !isNumber(amount)) {
        return false;
    }

    return amount > 0;
}

/**
 *
 * @param {*} date
 * -   Must be a string
 * -   Could be empty or null (it is an option, the current date will be used)
 * -   Must be in YYYY-MM-DD format
 * @returns boolean
 */
function isValidDate(date) {
    if (!date || !isString(date)) {
        return false;
    }

    const parsedDate = date.split("-");

    if (parsedDate.length !== 3) {
        return false;
    }

    const [year, month, day] = parsedDate;

    // there are more validations to be done on the years, month and day
    // there are 12 months, the max day is 31 and others
    return isNumber(year) && isNumber(month) && isNumber(day);
}

module.exports = { isValidName, isValidAmount, isValidDate };

Adding Authentication and Authorization

For the sake of demonstration, we will create a user record for the authentication and since we have not done database integration yet, we will use an array of objects instead (we did the same for expenditures). Now, let's do some cleanup. We are going to move the expenditures records (array) into a file and export it. Since it is acting as our data source, maybe we can name the file, data.js. After that, we'd add another variable for users.

Make sure that the expenditures and users are exported and imported appropriately.

We will add these lines to data.js

let expenditures = [
 ...
]

let users = [
 {
        id: "c96b08b8-f92f-4e27-8f04-c4f3604907d6",
        email: "john@gmail.com",
        password: "John123$",
 },
];

module.exports = { expenditures, users };

And import expenditures and users in index.js

...
// data source imports
const { expenditures, users } = require("./data");
...

"...", the three dots indicate that there is some code there. Perhaps the original

We will implement an endpoint to create a user and another, to log in users. There are a few things we have to consider:

The data that we are using is saved in memory. This means that when the server is restarted, the whole data will be lost.
For the User ID, we are returning or using a UUID.
The password will be stored as the raw password (NOT BEST PRACTICE, JUST FOR DEMO)

Sign up

To create a user, we first have to check the user data (array) and make sure that the email that the user is providing does not exist - there is no element in the array whose email matches the provided email. When there is no user with such an email, we then hash the password (for now just the raw password) and generate a UUID to uniquely identify the user.

These steps should help with the signup process:

In expense-tracker-api.http, add a post request for user signup

### Sign up
POST http://localhost:3000/users/signup
Content-Type: application/json

{
"email": "johndoe@gmail.com",
"password": "JohnPwd12_"
}

create a post request to handle user signup requests. The route should match the one above, expecting email and password from the request body
validate the email and password
check the users array (from the data file) that there is no record with an email that matches the email provided by the user. If such a record exists, return an appropriate response
we are supposed to hash the password but for demo reasons, we'd pass or save the raw password as is
generate a UUID for the new user
save the email, password and UUID by updating the users array
as part of the successful response, add the email and UUID and a message that says sign up successful or whatever appropriate message
it will be nice to log the new user record so that you can copy and manually add it to the user data (just to make sure that you have something static you are working with)

// sign up
app.post("/users/signup", (req, res) => {
    const { email, password } = req.body;

    if (!isValidEmail(email)) {
        return res.status(200).json({
            success: false,
            message: "Invalid email",
        });
    }

    if (!isValidPassword(password)) {
        return res.status(200).json({
            success: false,
            message: "Invalid password",
        });
    }

    // making sure that there is no record with the same email as this user's
    const existingUsers = users.filter((user) => user.email === email);

    if (existingUsers.length > 0) {
        return res.status(200).json({
            success: false,
            message: "Email already taken",
        });
    }

    // we are supposed to do password hashing here (recommended practice)

    // generate uuid unique to this user
    const uuid = crypto.randomUUID();

    // save this record
    const newUser = {
        id: uuid,
        email,
        password,
    };

    console.log(newUser);

    users = [...users, newUser];

    return res.status(200).json({
        success: true,
        message: "Sign up successful",
        data: {
            id: uuid,
            email,
        },
    });
});

This is the response that should be expected assuming we are on the same page.

HTTP/1.1 200 OK
X-Powered-By: Express
Content-Type: application/json; charset=utf-8
Content-Length: 128
ETag: W/"80-1wtZS+F4SzQ3LG+Q5EXmhzMEFaA"
Date: Wed, 15 Jan 2025 12:00:26 GMT
Connection: close

{
  "success": true,
  "message": "Sign up successful",
  "data": {
    "id": "decec694-6d97-4199-8544-f4bbe7ea753b",
    "email": "johndoe@gmail.com"
 }
}

And the console log should look similar to

{
  id: 'decec694-6d97-4199-8544-f4bbe7ea753b',
  email: 'johndoe@gmail.com',
  password: 'JohnPwd12_'
}

Did you get any other response where the success was false? What was it and how did you fix it?

Log in

To implement login, will check that there is a user with the provided email and that the User password matches the password (hash) of the retrieved record. The login will be very similar to the signup.

Try implementing the login.

When a user logs in to our platform, we have to find a way to allow the user to browse through the resources available to them without them having to log in all the time. What we are going to look at is how to authorize a user to CRUD their data (i.e. expenses and user data). Usually, we provide the user a means for them to access their data without having to log in all the time. There are several to do this. Some of them are:

Basic authorization: we send the user's credentials in every request they make, i.e. their email in this case with the password. We would encode (base64) these credentials for security's sake.
API key (Or token): this is usually a string issued by the platform. It can be used to uniquely identify each user.
Bearer token: an example of a bearer token is JSON Web tokens.
etc

There are several complicated authorization schemas out there. However, at this point, we will discuss each of these above forms of authorization.

Basic auth: The downside with this approach is that the user has to pass these encoded credentials all the time in every request. It is practically logging in all the time. And the user's account can be compromised easily as well. The only way to resolve this is to ... (no I am not talking about this... too much work - don't do this, I hope we understand). Use any of the ones mentioned below.
API key: Just like the basic auth, api keys can be compromised. The advantage of an api key is that we can generate another api when it is compromised. We can have it that the user has to request new api keys every week or within some time frame. We set up a mechanism to expire api keys so that the user will have to generate a new api key. Another downside of this is that we have to save the api key somewhere so that we can match it against any request. (Use a cache)
Bearer token (jwt): jwt is like an api token but the token itself can be time-bound. The token can expire and as such will become invalid after that period of expiry onwards. The token can store a payload which when parsed would be useful for identifying the owner of the token.

In most cases after login, an authorisation token is added as part of the response to the user. This token is then used by the user in every request thereafter to access the api resources. So Just to make this simple and to demonstrate how we can pass an api key for authorization, let's base64 encode the email and uuid passed as the auth token.

It is simple to encode a string to base64. Do, Buffer.from(STRING TO ENCODE).toString("base64"). So for our case, we want to use, email:uuid as the string to encode.

So before the return statement, let's add the encoding and update the response to include the auth token.

// sign up
app.post("/users/signup", (req, res) => {
    // some codes here were redacted

    const token = Buffer.from(`${email}:${uuid}`).toString("base64"); // newly added

    return res.status(200).json({
        success: true,
        message: "Sign up successful",
        data: {
            id: uuid,
            email,
            token, // newly added
        },
    });
});

Run the signup request again with another email and compare the difference in the response. Anything different? What was it?

Now we can do a proper login. The login request will not be that different from the sign-up. It is just the route that will be different.

// login
app.post("/users/login", (req, res) => {
    const { email, password } = req.body;

    if (!isValidEmail(email)) {
        return res.status(200).json({
            success: false,
            message: "Invalid email",
        });
    }

    if (!isValidPassword(password)) {
        return res.status(200).json({
            success: false,
            message: "Invalid password",
        });
    }

    // find a user with the same email and password: authentication taking place here
    const authUser = users.filter(
        (user) => user.email === email && user.password === password
    );

    // the number of records expected is one, anything else is invalid
    if (authUser.length !== 1) {
        return res.status(200).json({
            success: false,
            message: "Invalid credentials",
        });
    }

    // we are supposed to compare the password with the hash normally
    const user = authUser[0];

    const token = Buffer.from(`${user.email}:${user.id}`).toString("base64");

    return res.status(200).json({
        success: true,
        message: "Login successful",
        data: {
            id: user.id,
            email: user.email,
            token,
        },
    });
});

This is the login request:


### Log in
POST http://localhost:3000/users/login
Content-Type: application/json

{
 "email": "johndoe@gmail.com",
 "password": "JohnPwd12_"
}

And this is a sample response on my end.

HTTP/1.1 200 OK
X-Powered-By: Express
Content-Type: application/json; charset=utf-8
Content-Length: 216
ETag: W/"d8-aHIA5gYn4IKu8QQsFeyfBgNuK/I"
Date: Wed, 15 Jan 2025 18:20:17 GMT
Connection: close

{
  "success": true,
  "message": "Sign up successful",
  "data": {
    "id": "5991be49-d396-4f3e-ae61-89f38207db6c",
    "email": "johndoe1@gmail.com",
    "token": "am9obmRvZTFAZ21haWwuY29tOjU5OTFiZTQ5LWQzOTYtNGYzZS1hZTYxLTg5ZjM4MjA3ZGI2Yw=="
 }
}

Now we have an auth token. We will use the auth token to access resources only available to the owner of that token. To do that we have to modify and add (or update) the existing records to account for the userId. To make it simpler, I'd suggest we create multiple users and update several or add more expenses with userIds.

I have two users and three expenses (statically) and as such I will update them as is:

// data.js
// dummy data
let expenditures = [
    {
        id: "0fb7f7b7-9ed4-41af-858c-840258788479",
        name: "Legion Tower 7i Gen 8 (Intel) Gaming Desktop",
        userId: "c96b08b8-f92f-4e27-8f04-c4f3604907d6",
        amount: 2099.99,
        date: "2024-12-31",
    },
    {
        id: "2ea3786a-7391-46e8-a130-4de79504bd29",
        userId: "c96b08b8-f92f-4e27-8f04-c4f3604907d6",
        name: "Apple MacBook Pro 16-inch",
        amount: 2499.99,
        date: "2024-12-15",
    },
    {
        id: "a29e3057-c07d-4009-be19-f8da705bd5b8",
        userId: "decec694-6d97-4199-8544-f4bbe7ea753b",
        name: "Samsung Galaxy S24 Ultra",
        amount: 1199.99,
        date: "2024-12-10",
    },
];

let users = [
    {
        id: "c96b08b8-f92f-4e27-8f04-c4f3604907d6",
        email: "john@gmail.com",
        password: "John123$",
    },
    {
        id: "decec694-6d97-4199-8544-f4bbe7ea753b",
        email: "johndoe@gmail.com",
        password: "JohnPwd12_",
    },
];

module.exports = { expenditures, users };

As presented, now an expense also has an id (a uuid).

Protected endpoints

A protected endpoint or route requires some sort of "authorization" to access resources on that route. In this case, to protect an endpoint, we expect the request to have an auth token and that we can trace the auth token to an authentic user.

The auth token can be passed in any of the means, discussed so far, that data can be passed in the request: body, param, query and headers. We would pass the auth token in the headers.

Let's modify the List Expenditures endpoint to account for the auth token passed in the headers.

When the auth token is added, the request will look like this, below. Or rather, this is how we'd pass the auth token in the headers.

### List Expenditures
GET http://localhost:3000/expenditures
Authorization: am9obmRvZTFAZ21haWwuY29tOjU5OTFiZTQ5LWQzOTYtNGYzZS1hZTYxLTg5ZjM4MjA3ZGI2Yw==

We can get and log the content of the headers by doing so.

app.get("/", (req, res) => {
    return res.send(req.headers);
});

And we'd get something similar to

HTTP/1.1 200 OK
X-Powered-By: Express
Content-Type: application/json; charset=utf-8
Content-Length: 208
ETag: W/"d0-H4HndowWbSuUVoTjck9QS9+XNXo"
Date: Thu, 16 Jan 2025 08:27:35 GMT
Connection: close

{
  "user-agent": "vscode-restclient",
  "authorization": "am9obmRvZTFAZ21haWwuY29tOjU5OTFiZTQ5LWQzOTYtNGYzZS1hZTYxLTg5ZjM4MjA3ZGI2Yw==",
  "accept-encoding": "gzip, deflate",
  "host": "localhost:3000",
  "connection": "close"
}

When we run

GET http://localhost:3000/
Authorization: am9obmRvZTFAZ21haWwuY29tOjU5OTFiZTQ5LWQzOTYtNGYzZS1hZTYxLTg5ZjM4MjA3ZGI2Yw==

From the content in the headers, we can see a key-value pair of values and as present, we have "authorization": "am9obmRvZTFAZ21haWwuY29tOjU5OTFiZTQ5LWQzOTYtNGYzZS1hZTYxLTg5ZjM4MjA3ZGI2Yw==".

Great. At this point, I think we know what to do when we want to pass other values in the header.

authorization is the standard used when we have/want to pass auth tokens. We can use an arbitrary key and it should still work fine.

Now, we have to make sure the user making the request is authorized (using the auth token). We do that by decoding the auth token, in our case (because we encoded it), extracting the payload in there and comparing it against the users and if it's right, we can use the id (uuid) to fetch the expenses based on the userId.

When we were encoding, we did, Buffer.from(STRING TO ENCODE).toString("base64"). Now to decode is fairly similar. We'd do, Buffer.from(STRING TO DECODE, "base64").toString().

This is what we did to generate the auth token.

const token = Buffer.from(`${user.email}:${user.id}`).toString("base64");

And we can decode it by

// decode the auth token
const credential = Buffer.from(authTokenFromHeader, "base64").toString();

// credentials will be EMAIL:UUID and we want just the uuid
// we can get both and user both to fetch a user
const [email, id] = credential.split(":");
// how is that?

console.log(email, id);

When we put all these together, we will have the update endpoint to be:

// list expenditures
app.get("/expenditures", (req, res) => {
    // extra auth token from headers
    const authorization = req.headers.authorization;
    if (!authorization) {
        return res.status(200).json({
            success: false,
            message: "Unathorized, please login",
        });
    }

    // decode auth token
    const decodeToken = Buffer.from(authorization, "base64").toString();

    // parse the decoded token into the email and userId
    const [email, userId] = decodeToken.split(":");

    // now we can fetch the user with email and userId
    const isAuthenticUser = users.find(
        (user) => user.email === email && user.id === userId
    );

    if (!isAuthenticUser) {
        return res.status(200).json({
            success: false,
            message: "Unathorized, please login",
        });
    }

    // get the query string and check if it is not a number or something
    // that can be a number else set a default filter value of 0
    let amountMoreThan = Number(req.query.amountMoreThan);
    if (isNaN(amountMoreThan) || amountMoreThan < 0) {
        amountMoreThan = 0;
    }

    return res.json({
        success: true,
        data: expenditures.filter(
            (row) => row.userId === userId && row.amount > amountMoreThan
        ),
    });
});

HTTP/1.1 200 OK
X-Powered-By: Express
Content-Type: application/json; charset=utf-8
Content-Length: 190
ETag: W/"be-LOBEpYsscV5twPwFE50Va2UM9d4"
Date: Thu, 16 Jan 2025 09:23:13 GMT
Connection: close

{
  "success": true,
  "data": [
 {
      "id": "a29e3057-c07d-4009-be19-f8da705bd5b8",
      "userId": "decec694-6d97-4199-8544-f4bbe7ea753b",
      "name": "Samsung Galaxy S24 Ultra",
      "amount": 1199.99,
      "date": "2024-12-10"
 }
 ]
}

We are good for now. Now users can only access information that strictly belongs to them. How is that?

Conclusion

We have looked at how to do validations, authentication and authorization. What we have discussed here is roughly what these concepts are and some "rough" ways to implement them. For learning purposes, I believe these should be done.

A few things to consider:

Never save a raw password
Never use the auth mechanism employed here in the production environment i.e encoding the email and ID
There are cases where you you want to pass a different status code so that you can drop the success property in the json response. Yeah, do that. When you do that, instead of { data: [ ...😩 ]}, just return [ ...😩 ]

In the next excerpt, we are going to look at how to use a library to validate data and also hash passwords and break our application down.

Let me know what you think... 🥳

Resources

Basic CRUD API with express

Michael Otu — Wed, 01 Jan 2025 23:31:00 +0000

Content

Introduction
Project creation and initialization
Create a simple server and a GET route
Routes and Request handlers
Request and Response
Watch for changes
Create POST, GET, UPDATE, and DELETE routes
API Clients
Request body, params, query, header, ...
Manipulating memory data
Conclusion

Introduction

JavaScript Essentials: Part 7

Michael Otu ・ Nov 2 '24

#javascript #api #beginners #node

What is an API

Michael Otu ・ Feb 24 '24

#javascript #typescript #node #api

What is REST API

Michael Otu ・ Mar 1 '24

#javascript #typescript #node #api

JavaScript Essentials: Part 6 (Mastermind in Javascript)

Michael Otu ・ Oct 28 '24

#javascript #beginners #api #node

What is Nodejs

Michael Otu ・ Jul 31 '24

#javascript #typescript #node #api

Request and Response

Michael Otu ・ Mar 14 '24

#javascript #node #programming #tutorial

What is JSON

Michael Otu ・ Mar 9 '24

#javascript #typescript #node #api

In JavaScript Essentials: Part 7, I hinted that we'd look into developing APIs ([0] [1]) and this is where we start. We will have a taste of what it takes to develop a simple API to track expenditures.

Project Description

For this expense tracker, we will need to keep track of the item purchased, the amount and the date the item was purchased. An expense will look like this:

{
    "name": "Legion Tower 7i Gen 8 (Intel) Gaming Desktop",
    "amount": 2099.99,
    "date": "2024-31-12"
}

At this point, since a real database has not been added yet, we can use a list (an array) to hold the data that we will create. In this excerpt, we will go through each major concept surrounding creating APIs and add some ways to improve this app later.

Know that we will be building on top of this project, so, keep it clean and explore, experiment, fidget, etc as much as you can.

Project creation and initialization

As usual, we need a fresh working environment for each project, so we will begin by creating and initializing a Node JS project. If you are not sure, check out JavaScript Essentials: Part 6 (Mastermind in Javascript).

Now we have to create the parent folder for our API by doing:

# create a folder for the project at the path of your choice
mkdir expense-tracker-simple-api

# open the project with vscode
# code expense-tracker-simple-api
code -r expense-tracker-simple-api

# open the built-in terminal and init node
npm init -y
# this should create the package.json file

# create the entry file, index.js
echo "console.log(\"expense-tracker-simple-api\");" > index.js

# run the index.js file
node index.js

All we are doing with this script is quite direct.

We create a folder for our project
We opened the folder in vscode
We initialized a nodejs project
We added a console log into the index.js file. This creates the file and adds some content
We execute the index.js file

An alternative is to go to wherever you want to create this folder and create it over there then open the folder in vscode and init node project - check out, JavaScript Essentials: Part 6 (Mastermind in Javascript).

At this point, we need to install a nodejs package called express. Express is a library that will help us create our APIs.

We can install this package by running, npm i express. This should modify the package.json file, and create the package-lock.json file and node_modules folder. Consult the excerpt, What is Nodejs, for further understanding and information regarding how to use npm to install packages.

Create a simple server and a GET route

In our index.js file, we can add this code.

console.log("expense-tracker-simple-api");

// import the express lib
const express = require("express");

// create an express application
const app = express();

// create a GET request on the base endpoint
app.get("/", (req, res) => res.send("Hello world"));

// create a server that listens to requests on port 3000
app.listen(3000, () =>
  console.log(`Api running on ${"http://localhost:3000"}`)
);

All that we did was to create an express application, use the application to create a GET request to send a message and let the application listen in on requests coming from port 3000.

const app = express();

Creates an express application (🥳 that how to create an express application)

app.get("/", (req, res) => res.send("Hello world"));

We used the express application instance to create a GET request. app has several methods and properties that include the HTTP methods. Check out the except about http methods here.

app.listen(3000, () =>
  console.log(`Api running on ${"http://localhost:3000"}`)
);

We used the express application to listen on a port and used an arrow function to tell us, tell developers, that our application is running. For the port, we can alter it to another port of our choice. However, some special ports are already meant or used for some particular task and they are well known in the community and as such servers as default when such applications or programs are running on our PC. Check these out - 0 1 2

Note: There are some that are no-go because your system comes with them and there are some that come with applications we install such as some servers and databases and others. Fret not, when we use a port that is already in use, our application will let us know. All we have to do is add one or subtract one. No sweat.

Routes and Request handlers

To create a GET request, use app.get(...), for POST use app.post(...) and so on. For each route we want to create, the app.SomeMethod(...), will take a route or path, indicating which resource the user client desires or action they want to execute. As part of the route, it can take at least one request handler. This means we can have app.SomeMethod(path, hanlder1, handler2, ..., handlern);.

For the GET request above, the path or route is / (a string) and the single request handler we have is (req, res) => res.send("Hello world"), a handler function (a callback or a simple arrow function). The request handlers can be middleware and controllers.

Request and Response

A request handler normally takes in two arguments, the request and the response which are shortened as req and res respectively (you don't have to call them that but the first is always request and the second is the response). The request holds the data (or some information) about who made the request and what they want. The response is a means to provide feedback to the user who made the request. In this case, we send "Hello world" when the client makes a GET request to the / path.

Here you notice that the client and user are interchangeable, in the sense of which device makes the HTTP request to our api server and not a user, as in a user account.

Usually, the request handler will take a third argument which will point to the next handler after the previous handler has done its job. We call this next. It looks more or less like:

app.get("/", (req, res, next) => ...);

The next argument is useful, it points to the request handler and at times it takes an argument, an error. We will implement an error handler to generally handle errors we did not handle or errors that we "pass" to the next request handler.

Now let's cancel the running nodejs process that was running (in the terminal), using control + c. Then run it again. This time with the updated content from Create a simple server and a GET route section, we should see an output in the console (terminal) similar to,

expense-tracker-simple-api
Api running on http://localhost:3000

Open http://localhost:3000 in the browser. What do you see? A text saying, Hello world ?

Watch for changes

Rome was not built in one day as the saying goes. The same applies to software development. Maybe here what we mean is that we will gradually add more features as we develop and in this continuous process, it becomes irritating to start and stop the server all the time.

Go on, add another GET request (route) with /hello path and a request handler that says something you would want to say. Be happy.

You'd have to restart the server (the running nodejs process) and visit, http://localhost:3000/hello in the browser to see the response from that endpoint.

This,GET http://localhost:3000/hello, is an endpoint. You share this with api consumers. Among ourselves, we say route, because we don't have to know the whole URL (including protocol - http, domain - localhost, port - 3000, and path - /hello). Route is METHOD + PATH, more or less, GET /hello.

On macOS or Windows, we can do node --watch index.js or we can look out for changes not just in our entry file but in the whole folder path by, node --watch-path=./ index.js to watch for changes in the file path and also the file itself.

Currently, this is the content of my package.json file:

{
  "name": "expense-tracker-simple-api",
  "version": "1.0.0",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "description": "",
  "dependencies": {
    "express": "^4.21.2"
  }
}

We can add a script called dev under the script section.

"dev" : "node --watch-path=./ index.js"

We can stop the running server with control + c and now execute npm run dev. This will monitor saved changes in our files and reload the server.

So if this is not working for you, we have an alternative. We will install nodemone, npm i nodemon -g. We'd use it everywhere as a utility tool so we don't have to install it as part of our packages. We can watch changes by executing, nodemon index.js. There are cases where this won't work and when it doesn't, dom nodemon --exec node index.js

We can modify our dev script to use nodemon by,

"dev" : "nodemon --exec node index.js"

At this point, you can freely modify your .js files and on save, the server will restart to reload the load changes applied.

Create POST, GET, UPDATE, DELETE routes

We have already created a GET request. In this section, we will look into what each method means briefly since we have discussed them at length in Request and Response.

In this application, we are only serving one kind of resource to our clients, and that is expenditures. Assuming we are serving several resources, then we'd group requests under each resource.

So let's say we have user and expenditure, we have GET, POST, PUT, DELETE, etc for both users and expenditures.

For now, we'd use the /expenditures route to represent the expenditure resource.

GET: This means we will create a route to list, get all, fetch all, etc the records we have on expenditures. We can have a GET request that fetches one of the records. We have created a similar GET
POST: The post method is often used for creating resources
PUT: The put method is used to update recourses
DELETE: The delete method is used to delete resource

Now we can add the following lines of code to the index.js file but above app.listen(3000,...).

// --- expenditure routes ---

// list expenditures
app.get("/expenditures", (req, res) => res.send("list expenditures"));

// create expenditure
app.post("/expenditures", (req, res) => res.send("create expenditure"));

// update expenditure
app.put("/expenditures", (req, res) => res.send("update expenditure"));

// delete expenditure
app.delete("/expenditures", (req, res) => res.send("delete expenditure"));

When you save your file, do you notice the logs in the terminal? Test the GET route for the expenditure in the browser.

We can only run the GET requests in the browser. We will discuss api clients next.

API Clients

An API client in this context is a tool, a program or an application that is used to interact (consume, integrate, or test) APIs. Most commonly used are Postman, Insomnia, curl, etc...

In vscode and alongside some other IDEs, there is an extension that provides extensions to api clients. vscode has some of these extensions for that matter. However, we will be considering an api client known as REST Client. For our use case, it will be simpler to use Rest Client as such don't fret. We are covered.

Note: postman or any other api client of your choice is okay to use

How to use REST Client

First, install, REST Client.
We are creating an HTTP request as such we can create a file with .http or .rest extension - touch expense-tracker-api.http
In expense-tracker-api.http we can define our request
To create a GET request, add the following to the .http file

  GET http://localhost:3000

The endpoint is passed as seen above. For a post, put or delete a request to update the endpoint. Remember the difference between an endpoint and a route?
For a request that requires that data be passed to the api, we can pass the data as part of the route as a parameter or a string query, or we can pass it in the body.

  POST http://localhost:3000
  Content-Type: application/json

  {
      "property1": "value1",
      "property2": "value1",
      ...
      "propertyN": "valueN",
  }

Content-Type: application/json is a header key-value. This means that's how you pass headers using rest-client.
For the request body, we pass it as a json object - a newline is expected between the headers and the body though
Each request can be separated by three pound or ash signs, ###. A text can be added at the end of ### to make it look as if it is a title.

  ### test base GET endpoint
  GET http://localhost:3000

  ### Create a dummy POST request
  POST http://localhost:3000
  Content-Type: application/json

  {
      "property1": "value1",
      "property2": "value1",
      ...
      "propertyN": "valueN",
  }

As an exercise, create the request for the expenditure endpoints. Refer to when you are having any difficulties. We have more to do.

Request body, params, query, header

At this point, I don't have to stress that we'd be using JSON to communicate with our api using the API client.

As mentioned earlier, we can pass data to our api using the body, header, or URL of our request. We have seen how to pass data via the request body and the header (We will look into passing some specific data at another time). Check the POST request created. What we have not looked at is how to pass data as part of the URL.

Let's say we want to read an expenditure which has an id of 4, we can pass the add a parameter (as part of the URL) as, /expenditures/2. For the request which will be handling this requirement, we do, /expenditures/:id, where :id refers to the id of the expenditure. Assuming it is something else other than an id, let's say a name, then we'd do :name. Express will process this a provide us with a means to extract this value without sweat.

Now, for a query string, the idea is similar to request parameters however, it comes after a question, followed by a key1=value1&key2=value2...&keyN=valueN, where the key is the identifier of the value you want to pass. A very direct example is the REST-Client URL, https://marketplace.visualstudio.com/items?itemName=humao.rest-client. The question mark marks the beginning of the query string and whatever follows it maps a key to a value. Eg: ?itemName=humao.rest-client.

It will be a good time to test all your api endpoints and experience playing with it.

Request Body

Now we are going to process a request that passes data using the request body - The POST endpoint.

// create expenditure
app.post("/expenditures", (req, res) => res.send("create expenditure"));

The request object has a property, body, and on this property, are the values that we passed in the request body of our request - req.body.

This is the request that will be running.

### Create Expenditures
POST http://localhost:3000/expenditures
Content-Type: application/json

{
    "name": "Legion Tower 7i Gen 8 (Intel) Gaming Desktop",
    "amount": 2099.99,
    "date": "2024-31-12"
}

This is our endpoint implementation that will just log the request body to the console.

// create expenditure
app.post("/expenditures", (req, res) => {
  console.log(req.body);
  return res.send("create expenditure");
});

What happened? We had the usual response but... undefined was logged in the console. Well, it means everything is alright however, our api server doesn't know that it should parse the incoming as json. Remember json?

Let's add this one line below the const app = express(); which should solve parse the incoming data as json.

// parse request body as json
app.use(express.json());

Now, let's test the POST endpoint again. What did you get this time? Did you get something similar to this?

{
  "name": "Legion Tower 7i Gen 8 (Intel) Gaming Desktop",
  "amount": 2099.99,
  "date": "2024-31-12"
}

Now you know how to get the data from the body. Now as exercise, destructure the body and pass an object in the response. So instead of logging it, return it as the response.

Request parameter

We will create a new GET endpoint to read an expenditure by id.

This will be our API request:

### Read Expenditure
GET http://localhost:3000/expenditures/1

The request object has a property, params and on this property, are the values that we passed in the request param of our request - req.params.

Now the implementation will be similar to what we have done so far but a little different.

// read expenditure
app.get("/expenditures/:id", (req, res) => {
  /* const id = req.params.id;
  console.log(id) */
  console.log(req.params);
  return res.send("read expenditure");
});

We can access the id directly too. I hope you noticed that the :id key or name passed as part of the route matches the key in the logged object. Try renaming the params key in the route and see the output logged.

Request query (query string)

For the query strings, there is a property on the request object, query, which exposes the query strings passed.

To demonstrate this will pass a query string to filter the records to return. This endpoint should do enough.

### List Expenditures
GET http://localhost:3000/expenditures?amountMoreThan=1000&nameStartsWith=Legion

Now the implementation will be something similar to:

// list expenditures
app.get("/expenditures", (req, res) => {
  console.log(req.query);
  return res.send("list expenditures");
});

Now run your api and check your logs. Experiment with this.

Manipulating memory data

At this point, we are not connecting to a database yet so we have to manipulate data from memory. What we intend to do is create an array, add to the array, update an element in it, and remove an element. It sounds feasible as such that's what we are going to do.

We will be doing some modifications to some previously written code as such, feel free to alter your too. The final excerpt will be shared at the end.

Initialize in-memory data

Let's create an array of expenditures (dummy data) below const express = require("express");

// dummy data
let expenditures = [
  {
    name: "Legion Tower 7i Gen 8 (Intel) Gaming Desktop",
    amount: 2099.99,
    date: "2024-12-31",
  },
  {
    name: "Apple MacBook Pro 16-inch",
    amount: 2499.99,
    date: "2024-12-15",
  },
  {
    name: "Samsung Galaxy S24 Ultra",
    amount: 1199.99,
    date: "2024-12-10",
  },
  {
    name: "Sony WH-1000XM5 Noise Cancelling Headphones",
    amount: 349.99,
    date: "2024-12-20",
  },
  {
    name: "Dell UltraSharp 27 4K USB-C Monitor",
    amount: 699.99,
    date: "2024-11-25",
  },
  {
    name: "Logitech MX Keys Wireless Keyboard",
    amount: 119.99,
    date: "2024-10-05",
  },
  {
    name: "HP Envy x360 Convertible Laptop",
    amount: 1349.99,
    date: "2024-09-30",
  },
  {
    name: "NVIDIA GeForce RTX 4080 Graphics Card",
    amount: 1199.99,
    date: "2024-08-15",
  },
  {
    name: "ASUS ROG Zephyrus G14 Gaming Laptop",
    amount: 1599.99,
    date: "2024-07-18",
  },
  {
    name: "Canon EOS R6 Mark II Mirrorless Camera",
    amount: 2499.99,
    date: "2024-07-12",
  },
];

List expenditures

The current endpoint returns just a message using the res.send(message) but what we want to return is json. Though the .send can also take in an object or json, we will use res.json(obj).

I didn't mention but the default status code returned is 200. Have you noticed that? Except that another occurs or there is an issue with the request, the status code remains the same. There is a section under status codes, glance through.

We can alter the status code by passing the desired status code in res.status(desireStatusCode).json(obj). I will maintain the 200 status code throughout.

Make sure the server is still running

We can pass the list of expenditures directly.

// list expenditures
app.get("/expenditures", (req, res) => {
  return res.json(expenditures);
});

What was the response received? Check the status code as well as the response payload.

From experience and also to avoid ambiguity, I prefer to return status code 200 by default and have a either success property, message or data property to return a message or requested resource. By default, when status is false, message will be passed else, message or data may be passed.

// list expenditures
app.get("/expenditures", (req, res) => {
  return res.json({
    success: true,
    data: expenditures,
  });
});

We need to display the id (index of each row)

// list expenditures
app.get("/expenditures", (req, res) => {
  return res.json({
    success: true,
    data: expenditures.map((row, index) => ({ id: index, ...row })),
  });
});

Apply filtering with

// list expenditures
app.get("/expenditures", (req, res) => {
  // get the query string and check if it is not a number or something
  // that can be a number else set a default filter value of 0
  let amountMoreThan = Number(req.query.amountMoreThan);
  if (isNaN(amountMoreThan) || amountMoreThan < 0) {
    amountMoreThan = 0;
  }

  return res.json({
    success: true,
    data: expenditures
      .map((row, index) => ({ id: index, ...row }))
      .filter((row) => row.amount > amountMoreThan),
  });
});

Why was the filter done after the mapping?

Read expenditure

// read expenditure
app.get("/expenditures/:id", (req, res) => {
  // get the id of the request resource from the params
  const id = Number(req.params.id);
  if (isNaN(id) || id < 0) {
    return res.status(200).json({
      success: false,
      message: "Resource ID invalid",
    });
  }

  // we don't take negative ids, why?
  const row = expenditures[id];
  if (!row) {
    // what status code do this should be passed here? 404 not not found? why??
    return res.status(200).json({
      success: false,
      message: `Resource with ID, '${id}' not found`,
    });
  }

  return res.status(200).json({
    success: true,
    data: row,
  });
});

Does this implementation hint to you about, Why was the filter done after the mapping? ?

Create expenditure

// create expenditure
app.post("/expenditures", (req, res) => {
  // const { name, amount, date } = req.body
  const payload = req.body;

  if (!payload?.name || !payload.amount || !payload.date) {
    return res.status(200).json({
      success: false,
      message: "name, amount and date for expense are required",
    });
  }

  // we have to validate the name, maybe, their name must have some number of characters
  // we have to make sure that amount is a number
  // we have to also make sure that the date is of the format, yyyy-MM-dd

  // insert the new record
  expenditures.push({
    name: payload.name,
    amount: payload.amount,
    date: payload.date,
  });

  return res.status(201).json({
    success: true,
    message: "expenditure created successfully",
  });

  // there situations that it is best that you pass the new record created
  /* return res.status(201).json({
    success: true,
    message: "expenditure created successfully",
    data: expenditures[expenditures.length - 1],
  }); */

  // pass a route to fetch the new record created
  /* return res.status(201).json({
    success: true,
    message: "expenditure created successfully",
    route: `/expenditures/${expenditures[expenditures.length - 1]}`,
  }); */
});

Update expenditure

// update expenditure
app.put("/expenditures/:id", (req, res) => {
  // get the id of the request resource from the params
  const id = Number(req.params.id);
  if (isNaN(id) || id < 0) {
    return res.status(200).json({
      success: false,
      message: "Resource ID invalid",
    });
  }

  // we don't take negative ids, why?
  const row = expenditures[id];
  if (!row) {
    // what status code do this should be passed here? 404 not not found? why??
    return res.status(200).json({
      success: false,
      message: `Resource with ID, '${id}' not found`,
    });
  }

  // we have to validate the name, maybe, their name must have some number of characters
  // we have to make sure that amount is a number
  // we have to also make sure that the date is of the format, yyyy-MM-dd
  const { name, amount, date } = req.body;

  expenditures[id].name = name ?? row.name;
  expenditures[id].amount = amount ?? row.amount;
  expenditures[id].date = date ?? row.date;

  // there was a time I saw a 204 status - No content
  // since there was no data to return however we'll return the usual
  return res.status(200).json({
    success: true,
    message: "expenditure updated successfully",
  });
});

Delete expenditure

// delete expenditure
app.delete("/expenditures/:id", (req, res) => {
  // get the id of the request resource from the params
  const id = Number(req.params.id);
  if (isNaN(id) || id < 0) {
    return res.status(200).json({
      success: false,
      message: "Resource ID invalid",
    });
  }

  // we don't take negative ids, why?
  const row = expenditures[id];
  if (!row) {
    // what status code do this should be passed here? 404 not not found? why??
    return res.status(200).json({
      success: false,
      message: `Resource with ID, '${id}' not found`,
    });
  }

  expenditures = expenditures.filter((_, index) => index !== id);

  return res.status(200).json({
    success: true,
    message: "expenditure deleted successfully",
  });
});

Conclusion

We have covered the root of most API developments. This project is as basic as it comes. Relax and glance through again. There is more to look into such as

validation
authentication and authorization
middleware
error handling
SQL
database integration

Practice project

crud api = create, list, read, update, and delete. It is how you approach these problems.

⁠To-Do List

todo object: { id: int, task: string, status: boolean }
crud api
add an endpoint to mark all tasks as completed, success is true or not completed

⁠Calculator⁠

you have to decide, whether you'd create an endpoint for all the operations (addition, subtraction, multiplication, division)
or you would create a single endpoint with different functions that correspond to each operation. The user should be able to pass the operator and the two operands

⁠Currency Converter⁠
You are converting from one currency to another. Do for as many currencies as you can (3 is enough)

⁠Unit Converter⁠ ⁠- Notes App⁠ ⁠- Personal Blog⁠ ⁠- Quiz App⁠

Snippets

Know that the excess was removed.

// import the express lib
const express = require("express");

// dummy data
let expenditures = [
  {
    name: "Legion Tower 7i Gen 8 (Intel) Gaming Desktop",
    amount: 2099.99,
    date: "2024-12-31",
  },
  {
    name: "Apple MacBook Pro 16-inch",
    amount: 2499.99,
    date: "2024-12-15",
  },
  {
    name: "Samsung Galaxy S24 Ultra",
    amount: 1199.99,
    date: "2024-12-10",
  },
];

// create an express application
const app = express();

// parse request body as json
app.use(express.json());

// list expenditures
app.get("/expenditures", (req, res) => {
  // get the query string and check if it is not a number or something
  // that can be a number else set a default filter value of 0
  let amountMoreThan = Number(req.query.amountMoreThan);
  if (isNaN(amountMoreThan) || amountMoreThan < 0) {
    amountMoreThan = 0;
  }

  return res.json({
    success: true,
    data: expenditures
      .map((row, index) => ({ id: index, ...row }))
      .filter((row) => row.amount > amountMoreThan),
  });
});

// read expenditure
app.get("/expenditures/:id", (req, res) => {
  // get the id of the request resource from the params
  const id = Number(req.params.id);
  if (isNaN(id) || id < 0) {
    return res.status(200).json({
      success: false,
      message: "Resource ID invalid",
    });
  }

  // we don't take negative ids, why?
  const row = expenditures[id];
  if (!row) {
    // what status code do this should be passed here? 404 not not found? why??
    return res.status(200).json({
      success: false,
      message: `Resource with ID, '${id}' not found`,
    });
  }

  return res.status(200).json({
    success: true,
    data: row,
  });
});

// create expenditure
app.post("/expenditures", (req, res) => {
  // const { name, amount, date } = req.body
  const payload = req.body;

  if (!payload?.name || !payload.amount || !payload.date) {
    return res.status(200).json({
      success: false,
      message: "name, amount and date for expense are required",
    });
  }

  // we have to validate the name, maybe, their name must have some number of characters
  // we have to make sure that amount is a number
  // we have to also make sure that the date is of the format, yyyy-MM-dd

  // insert the new record
  expenditures.push({
    name: payload.name,
    amount: payload.amount,
    date: payload.date,
  });

  return res.status(201).json({
    success: true,
    message: "expenditure created successfully",
  });

  // there situations that it is best that you pass the new record created
  /* return res.status(201).json({
    success: true,
    message: "expenditure created successfully",
    data: expenditures[expenditures.length - 1],
  }); */

  // pass a route to fetch the new record created
  /* return res.status(201).json({
    success: true,
    message: "expenditure created successfully",
    route: `/expenditures/${expenditures[expenditures.length - 1]}`,
  }); */
});

// update expenditure
app.put("/expenditures/:id", (req, res) => {
  // get the id of the request resource from the params
  const id = Number(req.params.id);
  if (isNaN(id) || id < 0) {
    return res.status(200).json({
      success: false,
      message: "Resource ID invalid",
    });
  }

  // we don't take negative ids, why?
  const row = expenditures[id];
  if (!row) {
    // what status code do this should be passed here? 404 not not found? why??
    return res.status(200).json({
      success: false,
      message: `Resource with ID, '${id}' not found`,
    });
  }

  // we have to validate the name, maybe, their name must have some number of characters
  // we have to make sure that amount is a number
  // we have to also make sure that the date is of the format, yyyy-MM-dd
  const { name, amount, date } = req.body;

  expenditures[id].name = name ?? row.name;
  expenditures[id].amount = amount ?? row.amount;
  expenditures[id].date = date ?? row.date;

  // there was a time I saw a 204 status - No content
  // since there was no data to return however we'll return the usual
  return res.status(200).json({
    success: true,
    message: "expenditure updated successfully",
  });
});

// delete expenditure
app.delete("/expenditures/:id", (req, res) => {
  // get the id of the request resource from the params
  const id = Number(req.params.id);
  if (isNaN(id) || id < 0) {
    return res.status(200).json({
      success: false,
      message: "Resource ID invalid",
    });
  }

  // we don't take negative ids, why?
  const row = expenditures[id];
  if (!row) {
    // what status code do this should be passed here? 404 not not found? why??
    return res.status(200).json({
      success: false,
      message: `Resource with ID, '${id}' not found`,
    });
  }

  expenditures = expenditures.filter((_, index) => index !== id);

  return res.status(200).json({
    success: true,
    message: "expenditure deleted successfully",
  });
});

// create a server that listens to requests on port 3000
app.listen(3000, () =>
  console.log(`Api running on ${"http://localhost:3000"}`)
);

API requests

# Expenditure endpoints

### Create Expenditures
POST http://localhost:3000/expenditures
Content-Type: application/json

{
    "name": "Legion Tower 7i Gen 8 (Intel) Gaming Desktop",
    "amount": 2099.99,
    "date": "2024-31-12"
}

### List Expenditures
GET http://localhost:3000/expenditures?

### Read Expenditure
GET http://localhost:3000/expenditures/1

### Update Expenditure
PUT http://localhost:3000/expenditures/11
Content-Type: application/json

{
    "name": "MacBook pro laptop",
    "amount": 5099.99,
    "date": "2025-01-01"
}

### Delete Expenditure
DELETE http://localhost:3000/expenditures/0

JavaScript Essentials: Part 7

Michael Otu — Sat, 02 Nov 2024 09:08:31 +0000

This is the 7th part of this JavaScript Series (as a part of the whole) and in this part, we will look at how to break down our projects into small pieces so they are manageable. We will create some sort of separation of concerns, making our project appealing and easy to navigate. In all things, there are a beautiful part and of course the ugly part. So, don't overdo it. Do it when it needs to be done.

As mentioned earlier, our focus here is to break some part of our project into a separate file, export it, and then import it into our "main app". There are at the moment two ways to do this in JavaScript. Using the commonjs approach and also the ES6's modular approach. They are all great and we will look at both.

CommonJs

The import and export with commonjs is the default when not specified. That is how we could do, const readline = require("readline");. readline is a built-in package. We can use this approach on the third-party packages or modules written in our project.

Import with commonjs is done with const someVarNameForTheModule = require("modNameOrPath");.
We export by doing, module.exports = thingToExportOrStructuredObjectToExport.

Project

Let's kick off with a project to perform some math. We will create functions to add and subtract. Just these two.

Create a project folder, cmodule: cd ~/Projects && mkdir cmodule && cd cmodule
Initial the node project by doing, npm init -y
You can choose to add, "type": "commonjs" to the package.json file. I am saying you can choose because that is the default.

  {
    "name": "cmodule",
    "version": "1.0.0",
    "main": "index.js",
    "type": "commonjs",
    "scripts": {
      "test": "echo \"Error: no test specified\" && exit 1"
    },
    "keywords": [],
    "author": "",
    "license": "ISC",
    "description": ""
  }

Create two files, lib.js and main.js: touch lib.js main.js
Implement the body for the add function in the lib.js

  function add(x, y) {
    // return the sum of x and y
  }

Now that we have the function implemented, we have to export it to be used in our main.js. To export, we use module.exports = functionName. In our case, we do module.exports = add.

  module.exports = add;

Here the entirety of lib.js is just the add function. We exported lib.js as the add function. So we can import it as, const someName = require("./lib");
In the main.js, we will import the lib.js file and make use of the add function.

  const lib = require("./lib");
  // we did, "./lib", "dot slash lib", because main.js and lib.js are in the same folder.

  console.log(lib(1, 2));

Let's add the subtraction function

  function sub(x, y) {
    // returns the difference x and y
  }

You are supposed to implement these functions yourself 🙂
The question is, how do we export sub? Try it and access it inside main.js
Know that, when we do, module.exports = X, X is exported as a whole module so when we import const moduleName = require("moduleName");, we directly get access to X. So we can not export another value with this same approach.
In a case such as this, we can export both add and sub by exporting them as a group (object).

  module.exports = { add, sub };

Now when we import in main.js we can do

  const lib = require("./lib");
  // lib is an object so we can do lib dot someThing

  console.log(lib.add(1, 2));
  console.log(lib.sub(1, 2));

The lib module is exported as an object so we can do, moduleName.add and moduleName.sub.
We can also import by doing by destructuring, const { add, sub } = require("./lib");

  const { add, sub } = require("./lib");

  console.log(add(1, 2));
  console.log(sub(1, 2));

There is another way to do multiple exports

  exports.add = function add(x, y) {
    // return the sum of x and y
  };

  exports.sub = function sub(x, y) {
    // return the difference of x and y
  };

  exports.add = function (x, y) {
    // return the sum of x and y
  };

  exports.sub = function (x, y) {
    // return the difference of x and y
  };

exports.alias = someThing and exports.someThing= someThing or also works like modules.exports = { someThing }. I'd usually choose the exports.alias = someThing because, the module.exports = { ... } could extra lines.

ES Module

The import and export with the ES module style is not the default currently and as such must be specified explicitly by setting the type property to "module" in the package.json file. In this case, we would be able to do, import readline from "readline"; instead of const readline = require("readline");. We replaced the const with import, the = and require with from.

ES module import is done with import someVarNameForTheModule from "modNameOrPath";.
We export by doing, export default thingToExportOrStructuredObjectToExport or export thingToExportOrStructuredObjectToExport.

Project

We will build a similar project using the ES module style of import and export. We will create functions to add and subtract just as we did previously. So you can copy and paste it this time.

Create a project folder, emodule: cd ~/Projects && mkdir emodule && cd emodule
Initial the node project: npm init -y
Add, "type": "module" to the package.json file.

  {
    "name": "emodule",
    "version": "1.0.0",
    "main": "index.js",
    "type": "module",
    "scripts": {
      "test": "echo \"Error: no test specified\" && exit 1"
    },
    "keywords": [],
    "author": "",
    "license": "ISC",
    "description": ""
  }

Create two files, lib.js and main.js: touch lib.js main.js
Implement the body for the add in the lib.js

  function add(x, y) {
    // return the sum of x and y
  }

Now that we have the add function implemented, we have to export it to be used in our main.js. To export, we can use export default functionName. In our case, we do export default add.

  export default add;

We could have also done

  export default function add(x, y) {}

Here the entirety of lib.js is just the add function. We exported lib.js as the add function. So we can import it as, import someName from "./lib";
In the main.js, we will import the lib.js file and make use of add function.

  import lib from "./lib";
  // we did, "./lib" because main.js and lib.js are in the same folder.

  console.log(lib(1, 2));

Let's add the subtraction function

  function sub(x, y) {
    // returns the difference x and y
  }

The question is, how do we export sub?
In a case such as this, we can export both add and sub by exporting them as a group (object).

  export default { add, sub };

Now when we import in main.js we can do

  import lib from "./lib.js";

  console.log(lib.add(1, 2));
  console.log(lib.sub(1, 2));

We can also import by doing, import { add, sub } from "./lib";

  import { add, sub } from "./lib";

  console.log(add(1, 2));
  console.log(sub(1, 2));

There is another way to do multiple exports

  export function add(x, y) {
    // return the sum of x and y
  }

  export function sub(x, y) {
    // return the difference of x and y
  }

  export const add = function (x, y) {
    // return the sum of x and y
  };

  export const sub = function (x, y) {
    // return the difference of x and y
  };

With this sort of approach, it is either, we bundle the whole exports as one import or access individual imports one by one

  import * as lib from "./lib.js";

  console.log(lib.add(1, 2));
  console.log(lib.sub(1, 2));

  import { add, sub } from "./lib.js";

  console.log(lib.add(1, 2));
  console.log(lib.sub(1, 2));

Summary

To use commonjs or es module import and export style is relative. commonjs comes with no configurations, so one would ask why not use it as is? module.exports = someObject is the same as export default someObject. We can import with const someObject = require("pathToModule"); and import someObject from "pathToModule";. I like said, whichever you choose is okay. You can have a module/default export and also individual exports in the same file.

These are some rules that I try to stick to when I am developing my backend projects:

I avoid default export or module exports as much as possible and use the individual object export
If I have a controller, I use a default/module export without exporting anything else from that same file. So whenever I use module.exports or export default, I don't do any other export from the same file
I either use an object to group my constants and default export it or I would do individual exports of all the constants.
We can export objects without naming them and it works fine with the alias (name you give it) but I prefer to name my exports

Can you guess what is next? Well, we'd start doing some backend magics. We will start backend development.

Side project

If it challenges you, rewrite the mastermind program using multiple files. While on the topic I will challenge you. Complete this project. Either rewrite it for it to work or do whatever you have got to do to make it work and apply today's lesson.

/**
 * Simple User Authentication Logic
 *
 * Concepts Covered:
 * -> Variables
 * -> functions
 * -> conditionals
 *
 * Description
 * -> Create a login system where users can sign up with an email and password, and then attempt to log in.
 * -> Store user data in memory (using an array or object), and compare the entered credentials with the stored ones during login.
 *
 * Challenge
 * -> Implement password validation with the following rules. Password must:
 *  - not be null or empty, hence, required
 *  - be at least six characters
 *  - must have at least one of the special characters: !, @, $, _, -
 *  - have at least one uppercase
 *  - have at least one lowercase
 *  - have at least one number
 * -> Implement email validation with the following rules. Email must:
 *  - not be null or empty, hence, required
 *  - not exceed 256 characters in length
 *  - not contain prohibited characters (spaces, quotes, parentheses, brackets, comma, semicolon, colon, exclamation)
 *  - have a valid syntax (local part + "@" + domain + "." + tld)
 *  - not have 'email' in it
 *  - Local Part (Username):
 *      - Maximum length: 64 characters
 *  - Domain:
 *      - Maximum length: 253 characters
 *  - TLD (Top-Level Domain):
 *      - Must be one of the recognized TLDs (e.g., .com, .org, .net, etc.)
 *      - Maximum length: 6 characters
 */

const readline = require("readline");
const crypto = require("node:crypto");

// users will an object with the key as the email because we intend to make the email unique
// another approach is to generate some random string for the key (the key must be unique)
// so that in the USERS object we can have { keys: {email, password}, ...}
// another approach is to use an array where we'd have [{email, password}, ...]
// try both approaches and let us know what is best
const USERS = {};

// expected actions are either login or signup
const AUTH_ACTIONS = {
  LOGIN: "login",
  SIGNUP: "signup",
};

// We are expecting that the user input will indicate what action to be performed
// and the data to be used.
// Eg: [action] [email] [password]
async function getUserInput(question) {
  const ReadLine = readline.createInterface({
    input: process.stdin,
    output: process.stdout,
  });

  return await new Promise((resolve) => {
    ReadLine.question(question, (answer) => {
      resolve(answer);
      ReadLine.close();
    });
  });
}

// We expect the user to enter-> [action] [email] [password]
function isValidInputFormat(input = "") {
  return input.split(" ").length === 3;
}

// This function should return an object with the property, action, email and password
// We could also have done, { action, data: { email, password } }
function destructureInput(input = "") {
  const [action, email, password] = input.split(" ");
  return {
    action,
    email,
    password,
  };
}

function isValidationAction(action = "") {
  return action && Object.values(AUTH_ACTIONS).includes(action);
}

async function sleep(ms) {
  return new Promise((resolve) => setTimeout(resolve, ms));
}

// will return undefine or the user object
function findUserByEmail(email) {
  return USERS[email];
}

// hashes the password using sha256, we could use something better
// usually something like bcrypt, argon, etc
function hashPassword(password) {
  return crypto.hash("sha256", password);
}

function isValidHash(password, passwordHash) {
  return hashPassword(password) === passwordHash;
}

// usually, password validators will return a boolean, whether the password is valid or not
// however, we should consider adding an error message too
function isValidPassword(password = "") {
  // not be null or empty, hence, required
  if (!password || password === "") {
    return {
      isValid: false,
      message: "Password is required",
    };
  }

  // TODO: Complete the body

  // at this point, we've met all the rules we've set however there are some values
  // that will pass through because of the approach we took
  return {
    isValid: true,
    message: "",
  };
}

function isValidEmail(email = "") {
  //  - not be null or empty, hence, required
  if (!email || email === "") {
    return {
      isValid: false,
      message: "Email is required",
    };
  }

  // TODO: Complete the body

  return {
    isValid: true,
    message: "",
  };
}

// When signing up, we have to make sure that the email doesn't exist
// The email and password are valid
// Then generate a key that hasn't been used and create an object
// and assign an object of the credentials as a value
// Log that signup is successful and log the key and email of the user
function signUpLogic({ email, password }) {
  const user = findUserByEmail(email);
  if (user) {
    return {
      success: false,
      message: "User with such email already exists",
    };
  }

  const passwordHash = hashPassword(password);

  USERS[email] = { email, password: passwordHash };

  return {
    success: true,
    message: "User created successfully, you can now login",
  };
}

// When logging in, we have to make sure that the email exists
// We have to validate the email and password
// Opinion here: I learned from one of the devs that there is no need to
// validate data that you are not going to write to the DB
// What do you think
function loginLogic({ email, password }) {
  const user = findUserByEmail(email);
  if (!user) {
    // here the message could just be invalid credentials
    return {
      success: false,
      message: "User with not found",
    };
  }

  if (!isValidHash(password, user.password)) {
    return {
      success: false,
      message: "Invalid credentials",
    };
  }

  return {
    success: true,
    message: `${user.email} has logged in`,
  };
}

async function App() {
  console.clear();
  console.log("Running authentication app");

  const userInput = await getUserInput("\n$ ");

  if (["help", "h", "about"].includes(userInput)) {
    console.log(
      "About: Simple User Authentication Logic" +
        "\n- Expected format->[action] [email] [password]" +
        "\n\t[action] can be 'signup' or 'login' followed by email and password" +
        "\n" +
        "\n- Expected format->[action]" +
        "\n\t[action] can be 'exit', 'quit', 'list', 'l, 'about', 'help', or 'h'" +
        "\n\t\t- [exit|quit]: exits or quits the program" +
        "\n\t\t- [list|l]: list the user emails available" +
        "\n\t\t- [about|help|h]: displays the about page"
    );
    return;
  }

  if (["list", "l"].includes(userInput)) {
    console.clear();
    console.log("Users\n-----");
    for (const user of Object.values(USERS)) {
      console.log(`${user.email}`);
    }
    return;
  }

  if (["exit", "quit", "\n"].includes(userInput)) {
    console.clear();
    console.log("Application ended");
    process.exit();
  }

  if (!isValidInputFormat(userInput)) {
    console.log(
      "FormatError: Invalid date format. Expected format->[action] [email] [password]"
    );
    return;
  }

  const { action, email, password } = destructureInput(userInput);

  if (!isValidationAction(action)) {
    console.log("FormatError: Invalid action. Expected->login|signup");
    return;
  }

  const emailValidation = isValidEmail(email);
  if (!emailValidation.isValid) {
    console.log(emailValidation.message);
    return;
  }

  const passwordValidation = isValidPassword(password);
  if (!passwordValidation.isValid) {
    // It is not a good idea to trust what API (message) you are using, especially if the
    // message is from a vendor (a 3rd party and the error messages aren't predefined)
    // Usually, I'd just say invalid credentials or throw some error that the client can catch
    // and knowing the error type, they'd be certain of what to do based on the rules
    // the client has set (just print everything out, all the rules regarding a valid password)
    // And most vendors will just return true or false but in our case, we'd add a message
    console.log(passwordValidation.message);
    return;
  }

  // had we several actions, we could use the switch instead
  if (action === AUTH_ACTIONS.LOGIN) {
    // console.log("We are doing a login");
    const { success, message } = loginLogic({ email, password });
    if (!success) {
      console.log(message);
      return;
    }

    console.log(message);
  } else if (action === AUTH_ACTIONS.SIGNUP) {
    // console.log("We are doing a signup");
    const { success, message } = signUpLogic({ email, password });
    if (!success) {
      console.log(message);
      return;
    }

    console.log(message);
  } else {
    console.log(
      `FormatError: Action must be one of ${Object.values(AUTH_ACTIONS)}`
    );
  }
}

(async () => {
  while (true) {
    // Run the App function
    await App();
    await sleep(5000);
  }
})();

JavaScript Essentials: Part 6 (Mastermind in Javascript)

Michael Otu — Mon, 28 Oct 2024 17:46:00 +0000

In this section, we will implement a game called Mastermind in JavaScript. This game development would cover a lot of the concepts that we have discussed so far. We will define functions, pass arguments to them, make use of variables, and make use of loops and if statements. We would briefly look at another concept around functions, known as IIFE, Immediately Invoked Function Expression. We will also look at how to take user input via the command line. At this point, it is just console applications.

You can reference a similar implementation here, Master mind in python

Mastermind is a simple board game that uses colours but We'd use numbers instead.

Summary: Behind a bar are four colours put up by one player. The other player can not see the first player's colours. The first player's colours are called the code maker and the other player's colours are the code breaker. The code breaker has, inclusively, between 2 to 12 attempts at guessing the code makers'. The number of attempts must be even.

Implementation

Create a folder called mastermind on your pc (or where you put your projects) and in mastermind, initialize a node project using npm init -y (on the command line). I am on a Linux machine so this is how I will set up my project.
- Open my terminal, run, cd to move me to the user folder.
- Then, cd ~/projects. projects is where I keep my projects.
- Then mkdir mastermind and cd mastermind to create the mastermind folder and change into that folder.
- Initialize a node project with npm init -y. A package.json file will be created.
- Create app.js with touch app.js.
- Write console.log("Mastermind") into app.js and run it with node app.js. I expect to see Mastermind else I have an issue with my setup.
The starting (entry) point of this game will be in App, a function. Let's create a function called App and add console.log("App"). We can then call App() and execute the code with node app.js. I won't be telling you to run your code but it is something you should be doing as you code along. This is the current content of my app.js file.

console.log("Mastermind");

function App() {
  console.log("App");
}

App();

When the game starts
- user enters the number of rounds they want to play and the value entered must be validated
- user chooses whether to allow duplicates or not
- somewhere the code maker is randomly generated
- user has entered the code breaker
- the code breaker is compared to the code maker and a hint is given if it doesn't match
- in the process, we make the number of rounds
- and to make this more game-like we put the whole App into an infinite loop
Let's implement a function to generate random numbers for the code make, thereby setting random values to the code maker.
First, we need a way to generate random numbers. Not to interfere with the code in the app.js, let's create another file called scratch_pad.js and in this file we experiment.
JavaScript has a simple way to generate random numbers calling Math.random(). In the scratch pad, let's log 4 random numbers using a looping construct.

for (let i = 0; i < 4; i++) {
  console.log(Math.random());
}
// 0.10037268097853191
// 0.20981624777230534
// 0.47828165742292583
// 0.8160883929470153

what we want are integers (numbers like) 0, 1, 2, ..., 9 not decimals. We can multiply the value returned from the Math.random() by 10 and we would have x.something where x will now be in 1,2,3,...,9. Remember these experiments are all done on the scratch pad. Give it a try.
What we want is a number before the dot, the whole number part. We can write code to convert the number to a string and then split it by the "." and get the first element. However, there is a functionality for that called floor which we can use.

for (let i = 0; i < 4; i++) {
  console.log(Math.floor(Math.random() * 10));
}
// 4
// 7
// 3
// 4

The way this works is that, if we want to get random numbers between some number min and max, where max is greater than min, then we can do, min + Math.floor(Math.random() * (max - min + 1)). min is the minimum expected value and max is the maximum expected value. In our case, we have our minimum value to be 0 and maximum to be 9.
This is my snippet for generating the random number. I added parameters to the function because I don't want the function to have an internal state.

function generateRandomNumbersBetween(min, max) {
  return min + Math.floor(Math.random() * (max - min + 1));
}

for (let i = 0; i < 4; i++) {
  console.log(generateRandomNumbersBetween(0, 9));
}

At this point, we can now go back into our app.js and add the function above to generate the random numbers for the code maker. Put it above the App function.
From the summary the number of colours used is 4. So we need to generate 4 numbers for the code maker. We also have to handle if duplicates are allowed. Back to the scratchpad.
We have functions, if and else statements, the for and while loops, etc. These constructs all have a block or a body. Variables initialized in these blocks can be used within the block and not outside of it. This is known as the scope of a variable. So a variable can exist in the global scope, which means that that variable can be used or assessed everywhere. When we declare a variable in a block. The variable becomes internal or limited in that scope. Run this in the scratchpad.

const HP = 100;

if (true) {
  console.log("IF BLOCK::", HP);
}

console.log("End::", HP);

// IF BLOCK:: 100
// End:: 100

Now update this by initializing a variable,x, in the if statement, console.log(x) outside the if block and run your scratch pad. You should get an error similar to this.

 IF BLOCK:: 100
 /home/Projects/mastermind/scratch_pad.js:8
 console.log(x)
    ^

 ReferenceError: x is not defined
    at Object.<anonymous> (/home/Projects/mastermind/scratch_pad.js:8:13)
    at Module._compile (node:internal/modules/cjs/loader:1469:14)
    at Module._extensions..js (node:internal/modules/cjs/loader:1548:10)
    at Module.load (node:internal/modules/cjs/loader:1288:32)
    at Module._load (node:internal/modules/cjs/loader:1104:12)
    at Function.executeUserEntryPoint [as runMain] (node:internal/modules/run_main:174:12)
    at node:internal/main/run_main_module:28:49

 Node.js v20.17.0

At this point I want to bring to your attention the idea about scopes.

When generating the code maker, we want to know if duplicates are allowed and at this point, we know that the code maker is an array of numbers (or numeric strings). Let's start with the scratchpad. We want to implement a function that takes in a boolean argument indicating if duplicates are allowed. The function will add (push) four numbers into the code maker but before that, we have to check if duplicates are allowed and handle when not.

// a global code maker that is accessible inside any other scope
let CODE_MAKER = [];

function generateRandomNumbersBetween(min, max) {
  return min + Math.floor(Math.random() * (max - min + 1));
}

function generateCodeMaker(isDuplicatesAllowed = false) {
  let counter = 0;

  while (counter < 4) {
    let code = generateRandomNumbersBetween(0, 9);

    if (isDuplicatesAllowed) {
      CODE_MAKER.push(code);
      counter += 1;
    } else if (!CODE_MAKER.includes(code)) {
      CODE_MAKER.push(code);
      counter += 1;
    }
  }
}

console.log(CODE_MAKER);
generateCodeMaker(true);
console.log(CODE_MAKER);

// reset the code maker
CODE_MAKER = [];
generateCodeMaker(false);
console.log(CODE_MAKER);
// []
// [ 6, 6, 0, 9 ]
// [ 2, 5, 0, 8 ]

we also have written our code in such a way that the code maker isn't accessed globally in the code maker function. So will return the code maker instead.

// a global code maker that is accessible inside any other scope
let CODE_MAKER = [];

function generateRandomNumbersBetween(min, max) {
  return min + Math.floor(Math.random() * (max - min + 1));
}

function generateCodeMaker(isDuplicatesAllowed = false) {
  let counter = 0;
  let codeMaker = [];

  while (counter < 4) {
    let code = generateRandomNumbersBetween(0, 9);

    if (isDuplicatesAllowed) {
      codeMaker.push(code);
      counter += 1;
    } else if (!codeMaker.includes(code)) {
      codeMaker.push(code);
      counter += 1;
    }
  }

  return codeMaker;
}

console.log(CODE_MAKER);
CODE_MAKER = generateCodeMaker(true);
console.log(CODE_MAKER);

CODE_MAKER = generateCodeMaker(false);
console.log(CODE_MAKER);

// []
// [ 6, 6, 0, 9 ]
// [ 2, 5, 0, 8 ]

In app.js we can now add the code maker function and a variable for the code make.
Now back to the scratchpad. We want to take input from the user from the terminal. Javascript has a way to do that too. Try this snippet.

const readline = require("readline");

const readlineOInstance = readline.createInterface({
  input: process.stdin,
  output: process.stdout,
});

readlineOInstance.question("Enter code maker: ", (userInput) => {
  console.clear();
  console.log(`INPUT: ${userInput}`);
  readlineOInstance.close();
});

There is no issue with this approach of taking user input. It is just that we have to use a callback function and there is no way to pass the entered input to the outer scope of the callback function of readlineOInstance.question.
What are you thinking? Try it out in the "scratch pad". If you are thinking about declaring a variable in the outer scope of readlineOInstance.question the assigning the input entered to it, then it is a good approach but ... Still try it.
Do you remember the concept of Promises? We can use promise here and resolve the input. However, we have to wrap the whole process in function. There are a few parts of the readlineOInstance.question has a header similar to question(query: string, callback: (answer: string) => void. The query is the query (or prompt) to the user and the callback is how we handle the input collection. Since we might reuse the same function somewhere later, we'd pass the query as an argument.

const readline = require("readline");

async function getInput(query) {
  const readlineOInstance = readline.createInterface({
    input: process.stdin,
    output: process.stdout,
  });

  return await new Promise((resolve) => {
    readlineOInstance.question(query, (userInput) => {
      resolve(userInput);
      readlineOInstance.close();
    });
  });
}

(async () => {
  const userInput = await getInput("Enter code maker: ");
  console.clear();
  console.log(`INPUT: ${userInput}`);
})();

Now we can add the getInput function to the app.js. Do not forget the import, const readline = require("readline"). The content of the app.js should be similar to the snippet below.

const readline = require("readline");

console.log("Mastermind");

let CODE_MAKER = [];

function generateRandomNumbersBetween(min, max) {
  return min + Math.floor(Math.random() * (max - min + 1));
}

function generateCodeMaker(isDuplicatesAllowed = false) {
  let counter = 0;
  let codeMaker = [];

  while (counter < 4) {
    let code = generateRandomNumbersBetween(0, 9);

    if (isDuplicatesAllowed || !codeMaker.includes(code)) {
      codeMaker.push(code);
      counter += 1;
    }
  }

  return codeMaker;
}

async function getInput(query) {
  const readlineOInstance = readline.createInterface({
    input: process.stdin,
    output: process.stdout,
  });

  return await new Promise((resolve) => {
    readlineOInstance.question(query, (userInput) => {
      resolve(userInput);
      readlineOInstance.close();
    });
  });
}

function App() {
  console.log("App");
}

App();

Now we ask the user to enter the number of rounds and if a duplicate is allowed. We know that the number of rounds must be even and between 2 to 12. We will implement a function to validate a value (number) to be even and between 2 and 12. It will return a boolean. A number is even when the number modulo 2 is zero. (ie. number % 2 == 0).

function isValidRound(rounds) {
  return 2 <= rounds && rounds <= 12 && rounds % 2 == 0;
}

In the body of the App function, we can ask for the inputs and validate them. We will continuously ask for the proper input for the number of rounds. For the duplicate values in the code, when the user enters anything other than the expected we'd assume that the user doesn't want duplicates. We will use a while loop and set the condition to be true and only break when the rounds are valid however using a try and a catch (for error handling), when the user enters an invalid value we log a message indicating that the value entered is invalid. Try it out.

let rounds = 0;
let isDuplicateAllowed = false;

// The number of times to play must be even between 2 to 12 rounds
while (true) {
  try {
    const roundsInput = await getInput(
      "Enter number of rounds [must be even between 2 to 12]: "
    );

    // Number(some value) -> converts some value to a number
    // if possible else returns NaN. we could directly check
    // that 'rounds' is not NaN but any error will be caught
    // in the catch
    rounds = Number(roundsInput);

    if (isValidRound(rounds)) {
      break;
    }
  } catch (error) {
    console.log(INVALID_ROUNDS_PROMPT);
  }
}

// should there be duplicates
try {
  const duplicateInput = await getInput("Allow duplicates [1/0]: ");
  isDuplicateAllowed = Number(duplicateInput) === 1;
} catch (error) {
  isDuplicateAllowed = false;
}

console.log(
  `Number of rounds: (${rounds}) | Duplicate allowed: (${
    isDuplicateAllowed === 1 ? "Yes" : "No"
  })`
);

Run the app.js and interact with it. This is a similar output during the interaction.

 Mastermind
 App
 Enter number of rounds [must be even between 2 to 12]: 1
 Enter number of rounds [must be even between 2 to 12]: 5
 Enter number of rounds [must be even between 2 to 12]: 7
 Enter number of rounds [must be even between 2 to 12]: 100
 Enter number of rounds [must be even between 2 to 12]: 0
 Enter number of rounds [must be even between 2 to 12]: -2
 Enter number of rounds [must be even between 2 to 12]: 2
 Allow duplicates [1/0]: 1
 Number of rounds: (2) | Duplicate allowed: (Yes)

We have taken the number of rounds and the value for duplication. Now we can generate the code maker. To do this we can call the generateCodeMaker function and pass the duplication option value to it (or leave it since it's by default false).

console.clear();

CODE_MAKER = generateCodeMaker();

console.log(CODE_MAKER);

Now we can ask the user for the code breaker and compare it to the code maker. The code breaker is also an array of numbers. We will also add a hint for the user to know how far they are from a particular code. So if the code for the code breaker is greater than the code of the code maker, we say more. We say equal when they are equal and else we say less when the code from the code breaker is less than the code of the code breaker. Let's head into the scratchpad.
We will create a function that will take a numeric array of 4 elements and then compare the user's input (code breaker).

const readline = require("readline");

// generated by the code maker function
let CODE_MAKER = [3, 6, 7, 0];

async function getInput(query) {
  const readlineOInstance = readline.createInterface({
    input: process.stdin,
    output: process.stdout,
  });

  return await new Promise((resolve) => {
    readlineOInstance.question(query, (userInput) => {
      resolve(userInput);
      readlineOInstance.close();
    });
  });
}

// new code

let HINTS = [0, 0, 0, 0];

async function compareCode(codeMaker) {
  // enter guess with spaces
  const input = await getInput("Enter code breaker with spaces: ");

  // we are converting the code breaker into numbers
  const codeBreaker = input.split(" ").map((value) => Number(value));

  // compare each code at the same index and provide a hint
  for (let codeIndex = 0; codeIndex < 4; codeIndex++) {
    if (codeBreaker[codeIndex] > codeMaker[codeIndex]) {
      HINTS[codeIndex] = 1;
    } else if (codeBreaker[codeIndex] === codeMaker[codeIndex]) {
      HINTS[codeIndex] = 0;
    } else {
      HINTS[codeIndex] = -1;
    }
  }
}

(async () => {
  console.log(`INITIAL Hint: ${HINTS}`);

  // await because of the input
  await compareCode(CODE_MAKER);

  console.log(`INITIAL Hint: ${HINTS}`);
})();

We have a variable to handle the hints and a value for each code related to the code maker and breaker.
We pass the code maker to the function to compare it against the input from the user.
We update the hints to let the user know how to update the values in the code breaker
Now we can add the HINTS and compareCode function to the app.js. It is a great time to run your app.js, above the App function.
Now that we implemented a function to compare the code maker and the code breaker, we can now put this in a loop to account for the rounds (rounds = number of times to play the game). So if the number of rounds is 6, then the game would be played 6 times but we'd have to terminate the game when the user guesses all the codes correctly, that is when the values in the HINTS are all 0s. So when we count the number of 0s in HINTS and it is 4, we can terminate the game and say the user won.

// code breaker guesses the code by the code maker
while (rounds > 0) {
  console.log(`Number of rounds left: ${rounds}`);
  await compareCode(CODE_MAKER);

  // because of the values that we used to hint to the user
  // we have to find some dicey way to break the program
  // when the user guesses the code (all hints go to 0)
  if (HINTS.filter((value) => 0 === value).length === 4) {
    break;
  }

  console.log(HINTS);

  rounds -= 1;
}

The number of rounds is reduced and we'd know whether the user won or not if the number of rounds is not 0.

if (rounds > 0) {
  console.log("You won... Congratulations...");
} else {
  console.log("You lost bitterly to a computer");
}

Some outputs when you run the program

 Mastermind
 App
 Enter number of rounds [must be even between 2 to 12]: 6
 Allow duplicates [1/0]: 0

When I hit enter

 Number of rounds left: 6
 Enter code breaker with spaces: 7 7 7 7
 [ 1, 1, 1, 1 ]
 Number of rounds left: 5
 Enter code breaker with spaces: 1 2 3 5
 [ -1, -1, 1, 1 ]
 Number of rounds left: 4
 Enter code breaker with spaces: 4 5 1 3
 [ -1, -1, 1, 1 ]
 Number of rounds left: 3
 Enter code breaker with spaces: 5 6 0 2
 [ 0, 0, 0, 1 ]
 Number of rounds left: 2
 Enter code breaker with spaces: 5 6 0 1
 You won... Congratulations...

I guess we can enjoy our hard work so far. I have about 130 lines. How many do you have?
This is the full code

const readline = require("readline");

console.log("Mastermind");

let CODE_MAKER = [];
let HINTS = [0, 0, 0, 0];

function generateRandomNumbersBetween(min, max) {
  return min + Math.floor(Math.random() * (max - min + 1));
}

function generateCodeMaker(isDuplicatesAllowed = false) {
  let counter = 0;
  let codeMaker = [];

  while (counter < 4) {
    let code = generateRandomNumbersBetween(0, 9);

    if (isDuplicatesAllowed || !codeMaker.includes(code)) {
      codeMaker.push(code);
      counter += 1;
    }
  }

  return codeMaker;
}

async function getInput(query) {
  const readlineOInstance = readline.createInterface({
    input: process.stdin,
    output: process.stdout,
  });

  return await new Promise((resolve) => {
    readlineOInstance.question(query, (userInput) => {
      resolve(userInput);
      readlineOInstance.close();
    });
  });
}

function isValidRound(rounds) {
  return 2 <= rounds && rounds <= 12 && rounds % 2 == 0;
}

async function compareCode(codeMaker) {
  // enter guess with spaces
  const input = await getInput("Enter code breaker with spaces: ");

  // we are converting the code breaker into numbers
  const codeBreaker = input.split(" ").map((value) => Number(value));

  // compare each code at the same index and provide a hint
  for (let codeIndex = 0; codeIndex < 4; codeIndex++) {
    if (codeBreaker[codeIndex] > codeMaker[codeIndex]) {
      HINTS[codeIndex] = 1;
    } else if (codeBreaker[codeIndex] === codeMaker[codeIndex]) {
      HINTS[codeIndex] = 0;
    } else {
      HINTS[codeIndex] = -1;
    }
  }
}

async function App() {
  console.log("App");

  let rounds = 0;
  let isDuplicateAllowed = 0;

  // The number of times to play must be even between 2 to 12 rounds
  while (true) {
    try {
      const roundsInput = await getInput(
        "Enter number of rounds [must be even between 2 to 12]: "
      );

      // Number(some value) -> converts some value to a number
      // if possible else returns NaN. we could directly check
      // that 'rounds' is not NaN but any error will be caught
      // in the catch
      rounds = Number(roundsInput);

      if (isValidRound(rounds)) {
        break;
      }
    } catch (error) {
      console.log(INVALID_ROUNDS_PROMPT);
    }
  }

  // should there be duplicates
  try {
    const duplicateInput = await getInput("Allow duplicates [1/0]: ");
    isDuplicateAllowed = Number(duplicateInput) === 1;
  } catch (error) {
    isDuplicateAllowed = false;
  }

  console.clear();

  CODE_MAKER = generateCodeMaker(isDuplicateAllowed);

  // code breaker guesses the code by the code maker
  while (rounds > 0) {
    console.log(`Number of rounds left: ${rounds}`);
    await compareCode(CODE_MAKER);

    // because of the values that we used to hint to the user
    // we have to find some dicey way to break the program
    // when the user guesses the code (all hints go to 0)
    if (HINTS.filter((value) => 0 === value).length === 4) {
      break;
    }

    console.log(HINTS);

    rounds -= 1;
  }

  if (rounds > 0) {
    console.log("You won... Congratulations...");
  } else {
    console.log("You lost bitterly to a computer");
  }
}

App();

Is there room for improvement?

Even though this is a simple console/terminal/text-based app, there is more we can do about it.

We can replace all constants such as strings and numbers.
We could pull out (refactor) the code breaker input and splitting of it, out of the compare code and then pass the code breaker and code maker as arguments. We could even let the function return the hints rather than accessing the hints globally. We will create a new hints variable and return it. So compareCode will return hints assigned to the hints variable.

async function compareCode(codeMaker) {
  // enter guess with spaces
  const input = await getInput("Enter code breaker with spaces: ");

  // we are converting the code breaker into numbers
  const codeBreaker = input.split(" ").map((value) => Number(value));

  // compare each code at the same index and provide a hint
  for (let codeIndex = 0; codeIndex < 4; codeIndex++) {
    if (codeBreaker[codeIndex] > codeMaker[codeIndex]) {
      HINTS[codeIndex] = 1;
    } else if (codeBreaker[codeIndex] === codeMaker[codeIndex]) {
      HINTS[codeIndex] = 0;
    } else {
      HINTS[codeIndex] = -1;
    }
  }
}

we can also wrap the console.clear() into a function.
we can let the program slow down before the next game
we can pull out HINTS.filter((value) => 0 === value).length === 4 as a function. The purpose of it is to check if the code breaker has guessed correctly the code maker.
we can also do the same for declaring who won the game

if (rounds > 0) {
  console.log("You won... Congratulations...");
} else {
  console.log("You lost bitterly to a computer");
}

Put all functions that can stand alone into their own file, functions.js and export them. We can then refactor standalone functions that depend on a global variable and then pass that data as an argument to the function using a parameter.
We can even have a separate file for

(async () => {
  while (true) {
    // Run the App function
    await App();

    // Wait for a specified time before the next run
    await sleep(WAITING_TIME);

    // Clear the screen
    clearScreen();

    // Reset the game state for replay
    CODE_MAKER = [0, 0, 0, 0];
    HINTS = [0, 0, 0, 0];
  }
})();

Conclusion

We have used all that we have learnt in this project and there is more. I mentioned that we could group some functions and export them. For this, we will discuss how to import and export in Javascript. I will provide another project that I think will be useful to you. This is the end of the mastermind game and I hope you will also do some refactoring since there are a lot of places that need to be refactored. Best of luck...

const readline = require("readline");

let CODE_MAKER = [];
let HINTS = [0, 0, 0, 0];

// string constants
const ROUNDS_PROMPT = "Enter number of rounds (Even in [2, 12]) 🤗️: ";
const INVALID_ROUNDS_PROMPT =
  "Round must be an even number from 2 to 12 includes 😩️";
const DUPLICATE_PROMPT = "Duplicates allowed? (1/0) 🤤️: ";
const CODE_BREAKER_PROMPT = "Enter codes separated by space: ";
const WIN_PROMPT = "You won the rounds 👏️";
const LOSS_PROMPT = "You lost bitterly to a computer 😏️";

// int constants
const ZERO = 0;
const ONE = 1;

const NUMBER_CODE = 4;
const TERMINATING_VALUE = 0;

const MORE = 1;
const EQUAL = 0;
const LESS = -1;

const MIN_ROUNDS = 2;
const MAX_ROUNDS = 12;

const RAND_INT_MIN = 0;
const RAND_INT_MAX = 9;
const WAITING_TIME = 5000; // in milliseconds

function generateRandomNumbersBetween(min, max) {
  return min + Math.floor(Math.random() * (max - min + 1));
}

function generateCodeMaker(isDuplicatesAllowed = false) {
  let counter = ZERO;
  let codeMaker = [];

  while (counter < NUMBER_CODE) {
    let code = generateRandomNumbersBetween(RAND_INT_MIN, RAND_INT_MAX);

    if (isDuplicatesAllowed || !codeMaker.includes(code)) {
      codeMaker.push(code);
      counter += ONE;
    }
  }

  return codeMaker;
}

async function getInput(query) {
  const readlineOInstance = readline.createInterface({
    input: process.stdin,
    output: process.stdout,
  });

  return await new Promise((resolve) => {
    readlineOInstance.question(query, (userInput) => {
      resolve(userInput);
      readlineOInstance.close();
    });
  });
}

function isValidRound(rounds) {
  return MIN_ROUNDS <= rounds && rounds <= MAX_ROUNDS && rounds % 2 == ZERO;
}

function compareCode(codeMaker, codeBreaker) {
  let hints = [0, 0, 0, 0];

  // compare each code at the same index and provide a hint
  for (let codeIndex = 0; codeIndex < NUMBER_CODE; codeIndex++) {
    if (codeBreaker[codeIndex] > codeMaker[codeIndex]) {
      hints[codeIndex] = MORE;
    } else if (codeBreaker[codeIndex] === codeMaker[codeIndex]) {
      hints[codeIndex] = EQUAL;
    } else {
      hints[codeIndex] = LESS;
    }
  }

  return hints;
}

function clearScreen() {
  console.clear();
}

function hasGuessedCodeMaker(hints) {
  return hints.filter((code) => ZERO === code).length === NUMBER_CODE;
}

function declareResult(rounds) {
  if (rounds > TERMINATING_VALUE) {
    console.log(WIN_PROMPT);
  } else {
    console.log(LOSS_PROMPT);
  }
}

async function sleep(ms) {
  return new Promise((resolve) => setTimeout(resolve, ms));
}

// This is where the whole game is at
async function App() {
  let rounds = ZERO;
  let isDuplicateAllowed = false;

  // The number of times to play must be even between 2 to 12 rounds
  while (true) {
    try {
      const roundsInput = await getInput(ROUNDS_PROMPT);

      // Number(some value) -> converts some value to a number
      // if possible else returns NaN. we could directly check
      // that 'rounds' is not NaN but any error will be caught
      // in the catch
      rounds = Number(roundsInput);

      if (isValidRound(rounds)) {
        break;
      }
    } catch (error) {
      console.log(INVALID_ROUNDS_PROMPT);
    }
  }

  // should there be duplicates
  try {
    const duplicateInput = await getInput(DUPLICATE_PROMPT);
    isDuplicateAllowed = Number(duplicateInput) === ONE;
  } catch (error) {
    isDuplicateAllowed = false;
  }

  clearScreen();

  CODE_MAKER = generateCodeMaker(isDuplicateAllowed);

  // code breaker guesses the code by the code maker
  while (rounds > TERMINATING_VALUE) {
    console.log(`Number of rounds left: ${rounds}`);

    // enter guess with spaces
    const codeBreakerInput = await getInput(CODE_BREAKER_PROMPT);

    // we are converting the code breaker into numbers
    const codeBreaker = codeBreakerInput
      .split(" ")
      .map((value) => Number(value));

    HINTS = compareCode(CODE_MAKER, codeBreaker);

    // because of the values that we used to hint to the user
    // we have to find some dicey way to break the program
    // when the user guesses the code (all hints go to 0)
    if (hasGuessedCodeMaker(HINTS)) {
      break;
    }

    console.log(HINTS);

    rounds -= ONE;
  }

  declareResult(rounds);

  console.log(`CODE_MAKER: ${CODE_MAKER}`);
}

/**
 * Infinitely keeps playing the game.
 *
 * Runs the App function in an infinite loop, with a delay between each run.
 * Resets the game state after each run.
 */
(async () => {
  while (true) {
    // Run the App function
    await App();

    // Wait for a specified time before the next run
    await sleep(WAITING_TIME);

    // Clear the screen
    clearScreen();

    // Reset the game state for replay
    CODE_MAKER = [0, 0, 0, 0];
    HINTS = [0, 0, 0, 0];
  }
})();