DEV Community: Gurpreet Singh

Day 6 (last) - Your first week in NodeJS

Gurpreet Singh — Thu, 12 Aug 2021 12:59:39 +0000

Overview

During the course of this series, we have already developed a Node.js application that can be deployed on the cloud.

There are some providers which provide cloud hosting for Node.js applications, we will be using one of those called Heroku.

Heroku is one of the good options I found in the market which comes with a free DEV account, which you can leverage to start working on small projects or POCs.

Start with account creation

To use Heroku, you defiantly need an account which can be done from here.

Once you have your Heroku account up and running, please create a project after login into your account.

All you need is a unique name here which will become a sub-domain of Heroku, in my case, it will be https://not-failing.herokuapp.com/

Start with Heroku git setup

Heroku-hosted projects are also based on a git revisioning system, you can simply take checkout of your project and commit your changes.

Since we already have our project developed, we can simply convert it into a Heroku project.

Install the Heroku CLI

Download and install the Heroku CLI.

If you haven't already, log in to your Heroku account and follow the prompts to create a new SSH public key.

$ heroku login

Create a new Git repository

Initialize a git repository in a new or existing directory

$ cd my-project/
$ git init
$ heroku git:remote -a not-failing

Deploy your application

Commit your code to the repository and deploy it to Heroku using Git.

$ git add .
$ git commit -am "make it better"
$ git push heroku master

Existing Git repository

For existing repositories, simply add the heroku remote

$ heroku git:remote -a not-failing

Since we have the .env file on our system and node_modules, don't forget to create a .gitignore file with the following content.

You would also need to migrate your .env file variables and their values to Heroku environment variables.

There are two ways to set them, let's check the command line.

$ heroku config:set MONGO_HOST=my_hostname
$ heroku config:set MONGO_PASS=my_password
$ heroku config:set MONGO_USER=my_username
$ heroku config:set MONGO_DB=my_db_name

You can later check your config variables by typing $ heroku config:get MONGO_HOST

Before pushing your code you need to make the following changes in the app.js file.

// app.js

// Add a condition to render "dotenv" package only on local
if (process.env.NODE_ENV !== "production") {
  require("dotenv").config();
}

// Get PORT from process variables (to get Heroku PORT)
const PORT = process.env.PORT || 3000;

// Replace hardcoded PORT with variable.
app.listen(PORT);

I believe your App is up and running on Heroku now, like mine :)

Hope you liked the series, feel free to drop a comment in case you want me to create another series on another topic.

Useful Links

Day 5 - Your first week in NodeJS

Gurpreet Singh — Thu, 12 Aug 2021 12:58:10 +0000

Quick recap

In my previous article, I explained how to implement a template engine to get dynamic variables into your HTML file.

We used static data to show and save users in memory. In this article, we will now connect our application to MongoDB for better features and sustainability.

What is NoSQL

Big data changed how we used to think of databases and storage, we used to have relational databases like MYSQL, SQLServer, etc. but now we have another technique to store data called NoSQL.

It stores data in documents in the form of JSON, instead of tables with rows and columns and it works very well with JavaScript (Node.js). This is how data looks like in the MongoDB database.

[
  {
    "_id": "610ec10c78f7665bfa2cbf11",
    "age": 28,
    "name": "Bobbi Cannon",
    "email": "bobbicannon@dragbot.com"
  }
]

When we are not sure about the data structure or what kind of data is expected we can blindly choose the NoSQL approach and the industry-leading option is to go with MongoDB.

MongoDB Integration

We will be using MongoLabs for our MongoDB database setup which comes with the free tier option.

You can go to https://account.mongodb.com/account/login and claim your free account. You can follow the given steps to create a new database.

Create a new project:

Save project with members (default is logged-in user)

Create your first database

Select free plan

Once you create your cluster, it will take some time for the initial deployment.

Now you need to create a user for your database:

MongoDB Integration

Once you have your database and username/password created, we need to install the mongoose npm package on our local. $ npm install mongoose

To save your MongoDB details, you need to have a file with the name .env, which allows you to send these values in process.env or Node.js environment variables.

MONGO_HOST=my_hostname
MONGO_PASS=my_password
MONGO_USER=my_username
MONGO_DB=my_db_name

You will get above information in MongoDB connect option like mongodb+srv://my_username:my_password@my_hostname/my_db_name?retryWrites=true&w=majority

After creating the .env file you need to install a couple of packages, please refer to the package.json file below:

{
  "name": "node",
  "version": "1.0.0",
  "description": "",
  "main": "app.js",
  "scripts": {
    "start": "nodemon app.js"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "dependencies": {
    "body-parser": "^1.19.0",
    "dotenv": "^10.0.0",
    "ejs": "^3.1.6",
    "express": "^4.17.1",
    "mongoose": "^5.13.5",
    "nodemon": "^2.0.12"
  }
}

Now we are ready to change our app.js file with MongoDB changes.

First, you need to require the dotenv package to get MongoDB credentials from .env or process.env.

require("dotenv").config();

After requiring the package you will get access to all environment variables like this:
const { MONGO_USER, MONGO_HOST, MONGO_PASS, MONGO_DB } = process.env;

Now we will connect the application with MongoDB, it requires the below steps:

require("dotenv").config();
const mongoose = require("mongoose");

// connect to the database
async function connectDB() {
  const { MONGO_USER, MONGO_HOST, MONGO_PASS, MONGO_DB } = process.env;
  await mongoose.connect(
    `mongodb+srv://${MONGO_USER}:${MONGO_PASS}@${MONGO_HOST}/${MONGO_DB}`,
    {
      useNewUrlParser: true,
      useUnifiedTopology: true,
      useFindAndModify: false,
      useCreateIndex: true,
    }
  );
}

connectDB();

We need to require the mongoose package, and then followed by mongoose.connect() we will connect our current to our MongoDB instance.

It's an async operation, you can use promise or async/await functions.

Once you are done with the connection, we will define our Schema and Modal.

Schema is required to let MongoDB know what kind of collections you are going to store into documents.

// Create Schema for users
const usersSchema = new mongoose.Schema({
  age: Number,
  name: String,
  email: String,
});

// Create Modal for users
const UsersModal = new mongoose.model("users", usersSchema);

As I explained in my previous article, _id will be autogenerated by MongoDB.

Please refer to the following updated app.js file (reset of the project setup will remain the same):

require("dotenv").config();
const express = require("express");
const bodyParser = require("body-parser");
const mongoose = require("mongoose");
const app = express();

// connect to the database
async function connectDB() {
  const { MONGO_USER, MONGO_HOST, MONGO_PASS, MONGO_DB } = process.env;
  await mongoose.connect(
    `mongodb+srv://${MONGO_USER}:${MONGO_PASS}@${MONGO_HOST}/${MONGO_DB}`,
    {
      useNewUrlParser: true,
      useUnifiedTopology: true,
      useFindAndModify: false,
      useCreateIndex: true,
    }
  );
}

connectDB();

// Create Schema for users
const usersSchema = new mongoose.Schema({
  age: Number,
  name: String,
  email: String,
});

// Create Modal for users
const UsersModal = new mongoose.model("users", usersSchema);

app.set("view engine", "ejs");

app.use((req, res, next) => {
  res.append("Access-Control-Allow-Origin", ["*"]);
  res.append("Access-Control-Allow-Methods", "GET,PUT,POST,DELETE");
  res.append("Access-Control-Allow-Headers", "Content-Type");
  if (req.url.includes("/api")) {
    res.append("Content-Type", "text/json");
  }
  next();
});

// parse application/x-www-form-urlencoded
app.use(bodyParser.urlencoded({ extended: false }));
app.use(bodyParser.json());
app.use("/assets", express.static("assets"));

const getUserInfo = (id) => {
  return USER_DATA.filter((obj) => obj._id === id);
};

// Get list of users, and set data
app.get("/", (req, res) => {
  UsersModal.find({}, function (err, docs) {
    if (err) {
      res.status(500).send(err);
    }
    res.render("index", {
      data: docs,
    });
  });
});

// Get single user by id, and set data
app.get("/user/:id", (req, res) => {
  UsersModal.findOne({ _id: req.params.id }, function (err, doc) {
    if (err) {
      res.status(500).send(err);
    }
    res.render("user", {
      data: doc,
    });
  });
});

// Loading view of contact page
app.get("/contact", (req, res) => {
  res.sendFile(`${__dirname}/contact.html`);
});

// RESTFul API Services

// get all users
app.get("/api/users", (req, res) => {
  UsersModal.find({}, function (err, docs) {
    if (err) {
      res.status(500).send(err);
    }
    res.send(docs);
  });
});

// get single user by id
app.get("/api/user/:id", (req, res) => {
  UsersModal.findOne({ _id: req.params.id }, function (err, doc) {
    if (err) {
      res.status(500).send(err);
    }
    res.send(doc);
  });
});

// delete single user by id
app.delete("/api/user/:id", (req, res) => {
  UsersModal.findOneAndDelete({ _id: req.params.id }, function (err, doc) {
    if (err) {
      res.status(500).send(err);
    }
    res.send(doc);
  });
});

// update single user by id
app.put("/api/user/:id", (req, res) => {
  const { name, age, email } = req.body;
  if (req.params.id) {
    UsersModal.updateOne(
      { _id: req.params.id },
      { name, age, email },
      function (err, doc) {
        if (err) {
          res.status(500).send(err);
        }
        res.send(doc);
      }
    );
  } else {
    res.status(500).send("id is missing in params");
  }
});

// add new user
app.post("/api/user", (req, res) => {
  const { name, age, email } = req.body;

  if (name && age && email) {
    UsersModal.create(
      {
        age,
        name,
        email,
      },
      function (err, doc) {
        if (err) {
          res.status(500).send(err);
        }
        res.send(doc);
      }
    );
  } else {
    res.status(500).send("error, user not added.");
  }
});

app.listen(3000);

Useful Links

Day 4 - Your first week in NodeJS

Gurpreet Singh — Thu, 12 Aug 2021 12:47:20 +0000

Quick recap

I believe by now you are already comfortable with core modules of Node.js and have created your own RESTFul API service.

It's time to create a front-end now, which you can use to show users information which we developed in the last article.

Template engines

Node.js has by default support for template engines, you can add these template engines as an npm package and start using them. It helps to send dynamic data on a template file which you can further process (server-side) and render as an HTML page.

Serve `html` file as response

// app.js

const app = require("express")();

app.get("/", (req, res) => {
  res.sendFile(`${__dirname}/index.html`);
});

app.get("/contact.html", (req, res) => {
  res.sendFile(`${__dirname}/contact.html`);
});

app.listen(3000);

We are serving .html files based on user requests. But you can't send anything dynamic in these files, and for that, you need a template engine.

To serve dynamic content, we will be using the ejs templating engine which is recommended because it is very fast.

You can install it by using $ npm install ejs. Now we have ejs installed, let's quickly make some changes to our app.js file to integrate it.

You have to do a couple of things here since the EJS template engine looks for the views folder, we will create the views folder under the root folder of our application.

// package.json

{
  "name": "node",
  "version": "1.0.0",
  "description": "",
  "main": "app.js",
  "scripts": {
    "start": "nodemon app.js"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "dependencies": {
    "body-parser": "^1.19.0",
    "ejs": "^3.1.6",
    "express": "^4.17.1",
    "nodemon": "^2.0.12"
  }
}

// users.json

[
  {
    "_id": "610ec10c78f7665bfa2cbf11",
    "age": 28,
    "name": "Bobbi Cannon",
    "email": "bobbicannon@dragbot.com"
  },
  {
    "_id": "610ec10ca916e321b1363301",
    "age": 33,
    "name": "Downs Burns",
    "email": "downsburns@dragbot.com"
  }
]

Partials

<!-- views/user.ejs -->

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <meta http-equiv="X-UA-Compatible" content="IE=edge">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <title>Welcome | <%= data.name %></title>
  <style>
    body {
      background-color: cornflowerblue;
      text-align: center;
      font-size: 16px;
      font-family: Verdana, Geneva, Tahoma, sans-serif;
    }
    h1 {
      font-size: 3rem;
    }
    ul li {
      list-style: none;
      display: inline-block;
      margin: 0 10px;
    }
    a {
      font-size: 1.2rem;
      color: white;
      text-decoration: none;
    }
    a:hover {
      text-decoration: underline;
    }
    ul.users {
      margin: 0;
      padding: 0;
    }
    ul.users li {
      display: block;
      margin-bottom: 1rem;
    }
    .container {
      max-width: 600px;
      margin: 0 auto;
      text-align: left;
    }
  </style>
</head>
<body>
  <h1>Welcome | <%= data.name %> </h1>
  <ul>
    <li><a href="/">Home</a></li>
    <li><a href="/contact">Contact</a></li>
  </ul>
  <div class="container">
    <h2>User details</h2>
    <p><strong>Age: </strong><%= data.age || "N/A" %> </p>
    <p><strong>Email: </strong><%= data.email || "N/A" %> </p>
    <p><a href="/">&LessLess; Back</a></p>
  </div>
</body>
</html>

<!-- views/index.ejs -->

<!-- index.html -->
<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <meta http-equiv="X-UA-Compatible" content="IE=edge">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <title>Welcome</title>
  <style>
    body {
      background-color: cornflowerblue;
      text-align: center;
      font-size: 16px;
      font-family: Verdana, Geneva, Tahoma, sans-serif;
    }
    h1 {
      font-size: 3rem;
    }
    ul li {
      list-style: none;
      display: inline-block;
      margin: 0 10px;
    }
    a {
      font-size: 1.2rem;
      color: white;
      text-decoration: none;
    }
    a:hover {
      text-decoration: underline;
    }
    ul.users {
      margin: 0;
      padding: 0;
    }
    ul.users li {
      display: block;
      margin-bottom: 1rem;
    }
    .container {
      max-width: 600px;
      margin: 0 auto;
      text-align: left;
    }
  </style>
</head>
<body>
  <h1>Welcome to my webpage</h1>
  <ul>
    <li><a href="/">Home</a></li>
    <li><a href="/contact">Contact</a></li>
  </ul>
  <div class="container">
    <h2>Active Users</h2>
    <ul class="users">
      <% data.forEach(function(user){ %>
        <li><a href="/user/<%= user._id %>"><%= user.name %> (<%= user.age %>)</a></li>
      <% }); %>
    </ul>
  </div>
</body>
</html>

<!-- contact.html -->

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <meta http-equiv="X-UA-Compatible" content="IE=edge">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <title>Contact</title>
  <style>
    body {
      background-color: cornflowerblue;
      text-align: center;
      font-size: 16px;
      font-family: Verdana, Geneva, Tahoma, sans-serif;
    }
    h1 {
      font-size: 3rem;
    }
    ul li {
      list-style: none;
      display: inline-block;
      margin: 0 10px;
    }
    a {
      font-size: 1.2rem;
      color: white;
      text-decoration: none;
    }
    a:hover {
      text-decoration: underline;
    }
  </style>
</head>
<body>
  <h1>Contact me on Twitter @gsin11</h1>
  <ul>
    <li><a href="/">Home</a></li>
    <li><a href="/contact">Contact</a></li>
  </ul>
  <p>This page is running on awesome Node.js sever!</p>
</body>
</html>

Server file `app.js`

// app.js

const express = require("express");
const app = express();
const bodyParser = require("body-parser");
let USER_DATA = require("./users.json");

app.set("view engine", "ejs");  // setting up template engine to ejs

app.use((req, res, next) => {
  res.append("Access-Control-Allow-Origin", ["*"]);
  res.append("Access-Control-Allow-Methods", "GET,PUT,POST,DELETE");
  res.append("Access-Control-Allow-Headers", "Content-Type");
  if (req.url.includes("/api")) {
    res.append("Content-Type", "text/json");
  }
  next();
});

// parse application/x-www-form-urlencoded
app.use(bodyParser.urlencoded({ extended: false }));
app.use(bodyParser.json());

function makeId(length) {
  var result = "";
  var characters =
    "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789";
  var charactersLength = characters.length;
  for (var i = 0; i < length; i++) {
    result += characters.charAt(Math.floor(Math.random() * charactersLength));
  }
  return result;
}

const getUserInfo = (id) => {
  return USER_DATA.filter((obj) => obj._id === id);
};

app.get("/", (req, res) => {
  res.render("index", {
    data: USER_DATA,
  });
});

app.get("/user/:id", (req, res) => {
  res.render("user", {
    data: getUserInfo(req.params.id)[0] || {},
  });
});

app.get("/contact", (req, res) => {
  res.sendFile(`${__dirname}/contact.html`);
});

// RESTFul API Services

// get all users
app.get("/api/users", (req, res) => {
  res.send(USER_DATA);
});

// get single user by id
app.get("/api/user/:id", (req, res) => {
  res.send(getUserInfo(req.params.id));
});

// delete single user by id
app.delete("/api/user/:id", (req, res) => {
  const index = USER_DATA.findIndex((obj) => obj._id === req.params.id);
  res.send(USER_DATA.splice(index, 1));
});

// add new user
app.post("/api/user", (req, res) => {
  const { name, age, email } = req.body;
  if (name && age && email) {
    USER_DATA = [
      ...USER_DATA,
      {
        _id: makeId(24),
        age,
        name,
        email,
      },
    ];
    res.send(USER_DATA);
  } else {
    res.status(500).send("error, user not added.");
  }
});

app.listen(3000);

Snapshots

Homepage

Contact page

User details

Middleware and static assets

You have seen in the above templates we are using CSS as inline, in this segment we will include CSS files as static assets as well as images.

First, we need to move inline CSS styles to an external stylesheet.

/* assets/styles.css */

body {
  background-color: cornflowerblue;
  text-align: center;
  font-size: 16px;
  font-family: Verdana, Geneva, Tahoma, sans-serif;
}
h1 {
  font-size: 3rem;
}
ul li {
  list-style: none;
  display: inline-block;
  margin: 0 10px;
}
a {
  font-size: 1.2rem;
  color: white;
  text-decoration: none;
}
a:hover {
  text-decoration: underline;
}
ul.users {
  margin: 0;
  padding: 0;
}
ul.users li {
  display: block;
  margin-bottom: 1rem;
}
.container {
  max-width: 600px;
  margin: 0 auto;
  text-align: left;
}

Now we can include styles.css file with <link href="/assets/styles.css" rel="stylesheet" type="text/css" /> in all template and HTML files.

Now we need to inform our server that we are going to load the static files. By using the app.use function we can call express.static middleware to include static files.

// add this in your app.js

app.use("/assets", express.static("assets"));

Now you can restart your server and will see another request in networks for the CSS file. Similarly, you can include images/videos, etc.

You can create more routes with add/delete user functionality as follow:

Create a new route /user/add followed by a form to add new users.
Provide a button to delete an existing user by invoking API user/:id with the DELETE method.

Useful Links

ejs template engine

Day 3 - Your first week in NodeJS

Gurpreet Singh — Thu, 12 Aug 2021 12:41:03 +0000

Prerequisite

I believe you have already read my previous article where I explained, how to spin up a Node.js server instance, serving HTML/JSON data via an endpoint, basic routing, npm, and nodemon from creating custom modules, event handling, and file read/write operations. Let's jump on today's topic and write the code together.

Start with express.js

It's one of the most popular routing system. It can be integrated with many template engines. It also contains a middleware framework, which helps to enhance security features.

Just install express with $ npm i express:

// app.js

const express = require("express");
const app = express();

app.listen(3000);

With the above code, your express.js server is ready to serve requests on the 3000 port. But it won't serve any request alone unless you provide proper routing to it.

express.js routes

Routes are a very important part of the express.js setup. It helps you to make your entire architecture modularize. See below express, where you will see how we gonna server HTML and JSON requests together from a single server instance.

// app.js

const express = require("express");
const app = express();

app.get("/", (req, res) => {
  res.send("This is homepage");
});

app.get("/contact", (req, res) => {
  res.send("This is contact page");
});

app.get("/api/users", (req, res) => {
  res.setHeader("Content-Type", "text/json");
  res.send(
    JSON.stringify([
      {
        id: 3,
        username: "demo_user",
        email: "demo@demo.com",
        phone_number: "+49 123 4567 890",
      },
      {
        id: 2,
        username: "james",
        email: "james@demo.com",
        phone_number: "+49 765 111 999",
      },
    ])
  );
});

app.listen(3000);

Creating REST API

Similarly, we can now create our rest API to handle user-specific requests:
Since we are not using any database, we can create a static JSON file for initial set of users users.json.

[
  {
    "_id": "610ec10c78f7665bfa2cbf11",
    "age": 28,
    "name": "Bobbi Cannon",
    "email": "bobbicannon@dragbot.com"
  },
  {
    "_id": "610ec10ca916e321b1363301",
    "age": 33,
    "name": "Downs Burns",
    "email": "downsburns@dragbot.com"
  }
]

Now we have users.json in place we need two new packages.

body-parser - To get form-data from POST request.
objectid - To create dynamic unique _id (You can skip, when you have a database in place. eg: MongoDB, Firebase, etc.)

// app.js

const express = require("express");
const bodyParser = require("body-parser");
const objectid = require("objectid");

const app = express();

let USER_DATA = require("./users.json");

app.use((req, res, next) => {
  res.append("Access-Control-Allow-Origin", ["*"]);
  res.append("Access-Control-Allow-Methods", "GET,PUT,POST,DELETE");
  res.append("Access-Control-Allow-Headers", "Content-Type");
  res.append("Content-Type", "text/json");
  next();
});

// parse application/x-www-form-urlencoded
app.use(bodyParser.urlencoded({ extended: false }));
app.use(bodyParser.json());

// get all users
app.get("/api/users", (req, res) => {
  res.send(USER_DATA);
});

// get single user by id
app.get("/api/user/:id", (req, res) => {
  res.send(USER_DATA.filter((obj) => obj._id === req.params.id));
});

// delete single user by id
app.delete("/api/user/:id", (req, res) => {
  const index = USER_DATA.findIndex((obj) => obj._id === req.params.id);
  res.send(USER_DATA.splice(index, 1));
});

// add new user
app.post("/api/user", (req, res) => {
  const { name, age, email } = req.body;
  if (name && age && email) {
    USER_DATA = [
      ...USER_DATA,
      {
        _id: objectid(),
        age,
        name,
        email,
      },
    ];
    res.send(USER_DATA);
  } else {
    res.status(500).send("error, user not added.");
  }
});

app.listen(3000);

The initial set of users will be loaded from the file then you can add/remove users based on API requests.

`GET` request

You can use postman software to make these requests to check your API:

http://localhost:3000/api/users

[
  {
    "_id": "610ec10c78f7665bfa2cbf11",
    "age": 28,
    "name": "Bobbi Cannon",
    "email": "bobbicannon@dragbot.com"
  },
  {
    "_id": "610ec10ca916e321b1363301",
    "age": 33,
    "name": "Downs Burns",
    "email": "downsburns@dragbot.com"
  }
]

`POST` request (Add user)

http://localhost:3000/api/user

// data

{
  name: "Gurpreet Singh",
  age: "21",
  email: "yes@demo.com"
}

// response

[
  {
    "_id": "610ec10c78f7665bfa2cbf11",
    "age": 28,
    "name": "Bobbi Cannon",
    "email": "bobbicannon@dragbot.com"
  },
  {
    "_id": "610ec10ca916e321b1363301",
    "age": 33,
    "name": "Downs Burns",
    "email": "downsburns@dragbot.com"
  },
  {
    "_id": "610eca22f75a370000000001",
    "age": "121",
    "name": "Gurpreet Singh",
    "email": "yes@demo.com"
  }
]

`DELETE` request (delete user)

http://localhost:3000/api/user/<id>

http://localhost:3000/api/user/610eca22f75a370000000001

// response

[
    {
        "_id": "610eca22f75a370000000001",
        "age": "121",
        "name": "Gurpreet Singh",
        "email": "yes@demo.com"
    }
]

Note: Since we are not using any real database, hence this server only keeps data unless you restart.

You can practice the last method of REST PUT in order to update any user record.

Happy coding!

Day 2 - Your first week in NodeJS

Gurpreet Singh — Thu, 12 Aug 2021 12:38:31 +0000

Prerequisite

I believe you have already read my previous article where I explained, basics of Node.js, from creating custom modules, event handling, and file read/write operations. Let's jump on today's topic and write the code together.

Creating a Server

In the current world, people are looking for de-coupled systems/architectures solutions where each and every module can run independently and be responsible. These systems can be stateless or stateful (we will read about it later). These servers can be categories into two segments, Client and Server.

Client (front-end)

The client is where we invite end users, it can be a Mobile App or a Website. When someone is hitting https://google.com they are looking at the front-end of Google. The client platform is then responsible to get data from Server (backend APIs) or sometimes the backend renders the frontend code (like WordPress does) which we also call Traditional or Monolith structure.

Server (backend)

Node.js brought a big change in backend systems wherein people are not heading towards micro-service architecture. Servers/APIs are getting powerful and solving a single problem at a time. Headless architecture is one of its kind which can run without and state and gives expected output.

Let's quickly jump on client-side server creation. We will use the core Node.js module for this server.

We will divide this into two sections, request and response.

JSON response example

// app.js

const http = require("http");

const server = http.createServer((request, response) => {
  response.writeHead(200, { "Content-Type": "application/JSON" });
  response.end(JSON.stringify({ name: "John", employee_id: "DS123" }));
});

server.listen(3000, "127.0.0.1", () =>
  console.log("Server is started on http://127.0.0.1:3000")
);

To run the above file you need to run $ node app.js in your console and you will see a response in JSON format.

It is important to understand response headers here, If you check I am sending the request back with the "application/JSON" header but in string. http module only supports string responses, but can be distinguished by headers.

http://127.0.0.1 equivalent to http://localhost, IP is required when you want to run your application on Local Networks.

HTML response example

<!-- index.html -->

<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <meta http-equiv="X-UA-Compatible" content="IE=edge">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <title>Welcome</title>
  <style>
    body {
      background-color: cornflowerblue;
      text-align: center;
      font-size: 16px;
      font-family: Verdana, Geneva, Tahoma, sans-serif;
    }
    h1 {
      font-size: 3rem;
    }
  </style>
</head>
<body>
  <h1>Welcome to my webpage</h1>
  <p>This page is running on awesome Node.js sever!</p>
</body>
</html>

// app.js

const http = require("http");
const fs = require("fs");

const server = http.createServer((request, response) => {
  console.log(`request made: ${request.url}`);
  response.writeHead(200, { "Content-Type": "text/html" });
  const readStream = fs.createReadStream(`${__dirname}/index.html`, "utf8");
  readStream.pipe(response);
});

server.listen(3000, "127.0.0.1", () =>
  console.log("Server is started on http://127.0.0.1:3000")
);

Output

You can see in the above example, we had to add inline styles for the page. We will cover external styles in the next article.

Don't panic it's just a start, we will go through more robust and better ways of doing this.

Routing

I'll quickly touch upon core http module routing since we will read these routes in detail when we get on the express.js topic.

Routing is nothing but letting users redirect from one page to another. It also helps to divide code into authorized vs. non-authorized pages.

We will quickly create another HTML page so users can navigate through these two different pages.

<!-- index.html -->
<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <meta http-equiv="X-UA-Compatible" content="IE=edge">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <title>Welcome</title>
  <style>
    body {
      background-color: cornflowerblue;
      text-align: center;
      font-size: 16px;
      font-family: Verdana, Geneva, Tahoma, sans-serif;
    }
    h1 {
      font-size: 3rem;
    }
    ul li {
      list-style: none;
      display: inline-block;
      margin: 0 10px;
    }
    a {
      font-size: 1.2rem;
      color: white;
      text-decoration: none;
    }
    a:hover {
      text-decoration: underline;
    }
  </style>
</head>
<body>
  <h1>Welcome to my webpage</h1>
  <ul>
    <li><a href="/">Home</a></li>
    <li><a href="/contact">Contact</a></li>
  </ul>
  <p>This page is running on awesome Node.js sever!</p>
</body>
</html>

<!-- contact.html -->
<!DOCTYPE html>
<html lang="en">
<head>
  <meta charset="UTF-8">
  <meta http-equiv="X-UA-Compatible" content="IE=edge">
  <meta name="viewport" content="width=device-width, initial-scale=1.0">
  <title>Contact</title>
  <style>
    body {
      background-color: cornflowerblue;
      text-align: center;
      font-size: 16px;
      font-family: Verdana, Geneva, Tahoma, sans-serif;
    }
    h1 {
      font-size: 3rem;
    }
    ul li {
      list-style: none;
      display: inline-block;
      margin: 0 10px;
    }
    a {
      font-size: 1.2rem;
      color: white;
      text-decoration: none;
    }
    a:hover {
      text-decoration: underline;
    }
  </style>
</head>
<body>
  <h1>Contact me on Twitter @gsin11</h1>
  <ul>
    <li><a href="/">Home</a></li>
    <li><a href="/contact">Contact</a></li>
  </ul>
  <p>This page is running on awesome Node.js sever!</p>
</body>
</html>

// app.js

const http = require("http");
const fs = require("fs");

const server = http.createServer((request, response) => {
  let readStream;
  const url = request.url;

  console.log(`request made: ${url}`);
  response.writeHead(200, { "Content-Type": "text/html" });

  if (url === "/contact") {
    readStream = fs.createReadStream(`${__dirname}/contact.html`, "utf8");
  } else {
    readStream = fs.createReadStream(`${__dirname}/index.html`, "utf8");
  }

  readStream.pipe(response);
});

server.listen(3000, "127.0.0.1", () =>
  console.log("Server is started on http://127.0.0.1:3000")
);

Home page

Contact page

Node Package Manager (NPM)

NPM is the world's largest software registry. Open-source developers from every continent use npm to share and borrow packages, and many organizations use npm to manage private development as well.

NPM comes as the default package manager with Node.js and gives access to install third-party packages.

We will be using an express package in order to make our Node.js server better from a routing perspective and well as to keep the code more managed and modular.

To start with npm you need to run the $ npm init -y command, it will create a file called package.json which will keep track of all the packages we install later.

Installing NodeMon

You have noticed before whenever we were making any changes to our app.js file we had to restart our node.js server. Well, this new package NodeMon will be able to resolve this problem.

You can install this package with npm install nodemon and you will see another directory created with some packages called node_modules.

We will now quickly open our package.json file and make the following changes:

{
  "name": "node",
  "version": "1.0.0",
  "description": "",
  "main": "app.js",
  "scripts": {
    "start": "nodemon app.js"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "dependencies": {
    "nodemon": "^2.0.12"
  }
}

Instead of running your server with $ node app.js you can now run $ npm start, it will use the start command mentioned under scripts object in package.json.

Now if you make any changes to your app.js file it will automatically restart your Node.js server.

Useful Links

Create Server

Node Package Manager

Nodemon

Day 1 - Your first week in NodeJS

Gurpreet Singh — Thu, 12 Aug 2021 12:35:48 +0000

I believe you know something about Node.js before starting on this course, just to refresh your memory. Node.js is a JavaScript runtime, built on Chrome's V8 JavaScript engine. Node.js uses an event-driven, non-blocking model that makes it lightweight and efficient.

Node.js package ecosystem, npm, is the largest ecosystem of open source libraries in the world.

Let's jump on Day 1 agenda now.

1. Installation

You can simply go to official Node.js website and download the latest stable version (LTS) based on your OS.

After the installation, you can simply open your terminal/command prompt, to check if it's running or not.

$ node --version
// v14.16.1 (in my case)

In addition to the above, you can also go to any editor and create a file app.js (or any name).

// app.js
console.log("node is working");

Now run this file using Node.js

$ node app.js
// node is working

You have installed Node.js on your machine 🏆

2. Global Objects

After installing Node.js, let's quickly touch upon Global objects which we can use without any objection and third-party imports. These objects/functions are nothing but built-in into Node.js objects. If you are already working on JavaScript you defiantly know about some of these objects like (console, setInterval, setTimeout, etc.).

The main difference between JavaScript (window) vs. Node.js global objects is, you can't get access to the screen-related function example: scroll, style, etc.

You can try following code examples in Node.js

// app.js
setTimeout(() => {
  console.log("5 seconds have passed");
}, 5000);

// Run this code again
$ node app.js

Another useful global object is process, we will use this a lot in further lessons. process holds the latest state of your Node.js running process. It gives you access to read which environment your server is running or you can set custom environment variables while starting your server.

The common use case is to store credentials inside process to later use them in your program.

// Setting process variable
$ export DB_NAME=demo_db
$ export DB_PASSWORD=yes_you_are_right

// Run your app
$ node app.js

// app.js
const {DB_NAME, DB_PASSWORD} = process.env
console.log(DB_NAME); // demo_db
console.log(DB_PASSWORD); // yes_you_are_right

3. Modules + `require()`

So far (above), we wrote everything in a single file, which is good for practice programs and all. If you want to work on a real project where you might have other teams/people surrounding you and working on the same project, then recommended approach is to work with modules.

Modules are nothing but an approach to split your code into different-2 re-usable pieces.

It can be a file or a function within the same file, you must have heard of the philosophy of pure-functional programming where every function is responsible to do a single job.

Let's take an example, you want to count the `length` of an array.

// Create a new file next to your app.js
// count.js

const count = (arr) => {
  return arr.length
}

Now you have count.js in place, it's time to make it re-usable for the application.

Before calling it a module, we need to make some changes to the count.js file.

// count.js

const count = (arr) => {
  return arr.length
}

module.exports = count;  // It will export this component as a module

Since we have the count module ready to include, let's call it in app.js.

// app.js

const count = require('./count');  // including `count` function as a module
cosole.log(count([1, 98, 22, 41]))  // return: 4

You can send multiple modules from your single file.

// utils.js

const add = (a, b) => {
  return a + b;
}

const sub = (a, b) => {
  return a - b;
}

const getDatabaseName = () => process.env.DB_NAME;

module.exports = {
  add,
  sub,
  getDatabaseName,
};

// You also change your public function names
module.exports = {
  plus: add,
  minus: sub,
  DBName: getDatabaseName,
};

// app.js

const utils = require("./utils");

console.log(utils.add(1, 2));  // result: 3
console.log(utils.sub(2, 1));  // result: 1
console.log(utils.getDatabaseName());  // result: demo_db

4. Event emitting

Like the above-defined module, we have some built-in core Node.js modules available to use. One of the examples is the events module.

Similar to JavaScript click/onChange events Node.js has the capability to define your own events which can be used when needed.

// custom-events.js

const events = require('events');
const utils = require("./utils");

const eventEmitter = new events.EventEmitter();

eventEmitter.on("showSum", (a, b) => {
  console.log(`Sum is: ${utils.add(a, b)}`);
});

module.exports = eventEmitter;

Our event emitter module is now ready, let's use it in our application.

// app.js

const customEvents = require("./custom-events");
customEvents.emit("showSum", {a: 1, b: 2});  // Sum is: 3

5. Read/Write/Steam files

After learning event handling in Node.js let's do a quick check on file handling with Node.js core module fs.

Let's read the file first in order to try the fs module. But first, we need a file to read. Let's create a file first.

Read operation ()

// Inside the same directory
$ touch read.txt
$ vim ./read.text  // Add some content and save the file.

// app.js

const fs = require("fs");

const readMeFile = fs.readFileSync("./read.txt", "utf-8");
console.log(readMeFile);  // File contents

Write operation

// app.js

const fs = require("fs");

const readMeFile = fs.readFileSync("./read.txt", "utf-8");
fs.writeFileSync("./read-new.txt", readMeFile);

The above statement will simply read the contents from the read.txt file write another file read-new.txt file with its content.

There are two ways to read/write files, synchronous/asynchronous, above we are using sync. method to read the file. It prevents the program execute unless it reads the entire file.

On the other hand fs.readFile can read files asynchronously, sometime we use this method too.

Let's quick touch upon async. file read operation:

const fs = require("fs");

fs.readFile("./read.txt", "utf-8", (err, data) => {
  if(err) {
    console.error(err);
  }
  console.log(data);
});

console.log("Am I first?");

// Result:
// file content
// Am I first?

In the example above you can see, text outside the readFile operation logged first. Async. operation is beneficial when your further operations are independent of the file read/write operations.

Hope you like the starting of the series, stay tuned for further posts in this series.

Happy reading!

Useful Links

PiHole on Docker

Gurpreet Singh — Tue, 22 Jun 2021 22:11:38 +0000

This article is my follow-up article to Up and running with RaspberryPi with Docker. In the previous article, I showed how you can configure docker and docker-compose on your RaspberryPi.

In this article, I will show how you can restrict external traffic from your RaspberryPi with PiHole using docker and docker-compose.

Installation

The first step to defining a place where you want to install PiHole. In my case, I am going to keep it under the ~ path.

$ cd ~
$ mkdir pihole
$ cd pihole
$ touch docker-compose.yml

Once you create the docker-compose.yml file inside the pihole directory, add the following code into it.

$ nano docker-compose.yml

version: "3"

services:
  pihole:
    container_name: pihole
    image: pihole/pihole:latest
    ports:
      - "53:53/tcp"
      - "53:53/udp"
      - "67:67/udp"
      - "80:80/tcp"
    environment:
      TZ: "America/Chicago"
      WEBPASSWORD: "strongPassword"
    volumes:
      - "./etc-pihole/:/etc/pihole/"
      - "./etc-dnsmasq.d/:/etc/dnsmasq.d/"
    cap_add:
      - NET_ADMIN
    restart: unless-stopped

After saving the docker-compose.yml file, you can run the following command to bring your PiHole server up and running.

$ docker-compose up --detach

Initial run will take some time since it will download the PiHole image from docker.

After a successful run, you can check PiHole instance is running on RaspberryPi as a docker service.

# Try following commands to check PiHole service

$ docker ps
# To check the list of running containers

$ docker images
# To check PiHole image

If you have static IP defined configure in my previous article then you can access PiHole, at http://192.168.0.4/admin

Configure devices

After the installation of PiHole now we need to map other devices to use PiHole as the default gateway. For that, you can go to the network/Wifi settings of your individual devices and add 192.168.0.4 IP address to the DNS list (should be on top). See the example below:

Thanks for following the tutorial, happy coding!

Are you logging your NodeJS code?

Gurpreet Singh — Tue, 17 Nov 2020 12:21:23 +0000

Logs are essential part of an application, it provides you deep level access of output comes from user inputs. When application is on production, logs are important to identify the problem if something goes wrong. If you are developer, have you asked yourself a question, “Am I logging or not?”

Why Logs are important?

No matter how careful we are while developing application, it is quite difficult to make it 100% bug free. Even after spending time to find defects in test cycle, we still won’t be able to catch all of them.

These remaining error may cause unexpected exceptions on production environment. In some case your application might crash in certain user journeys. It is always a good idea to keep an extra eye on application flow.

In order to achieve success, we use application logs to check why the application is behaving differently. For this, we have to set up our application to record information about its events and errors. This is what we call logger, it helps us to identify problems with an application running in production.

Best Practices

1. Avoid using `console.log()`

There are some important things we need to consider while configuring logs in application. Adding logs with console.log() won’t stay for longer period of time. They are available till the time user refresh the page.

console.log() can be use for temporary logging since it provides in memory logs which can be easily read through terminal since it uses stdout. Similarly console.error or console.warn can also be used but you can’t store these logs into any file or database.

Hence, console.log doesn’t provide us enough options to use it as primary logging system. You should consider proper library to use as logging system.

2. Third party libraries

Dedicated libraries unlike console.log provides us adequate options to define and configure logging system.

Levels: It offers different levels of logging, example you can use several levels of logging like, info, warn, debug, and error. These helps to filter the problems.
Appearance: You can use different colors and appearance to distinguish your logs.
Data types: Not only appearance, you can also make your logs different in type. Some libraries support JSON format as well.

Winston and Bunyan are two of the most popular logging libraries available for Node applications.

3. Source, Timestamps, Context

Logging is not only to log text when exception or success comes, there are 3 important aspects which makes them more useful when it comes to debugging.

Source: While debugging the application through logs it is important to know what is the source of particular log. Hence, it is important to keep, hostname, method, module name.
Timestamps: Timestamps plays very important role while debugging an application, because it helps you to identify at what timeframe error happened. In the world of micro services it is important to keep timestamps because requests are not sequential but async.
Context: These types of errors/exceptions comes based on users inputs. Example, when user is trying to register in an application but registration failed due to he/she is already registered user or provided email address is wrong etc. So, application behavior was expected but user was not able to register.

4. Logging levels

Use different levels of logs to distinguish them, so developer can easily understand while debugging the application.

Emergency: the system is unusable
Alert: action must be taken immediately
Critical: critical conditions
Error: error conditions
Warning: warning conditions
Notice: normal but significant conditions
Informational: informational messages
Debug: debug-level messages

You can still alter these levels upon your needs.

5. What shouldn’t be part of logs

There are some obvious mistakes developers does while configuring loggings. One shouldn’t log any PII data while logging requests or exceptions. I would like to share some bad and good examples of logging.

Bad Example

const express = require('express');
const winston = require('winston');
const app = express();

// configuring logger
const logger = winston.createLogger({
  transports: [
    new winston.transports.Console(),
    new winston.transports.File({ filename: 'combined.log' })
  ]
});

app.post('/user/add', (req, res) => {
  try {
    modal.addUser({
      email: req.email,
      password: req.pwd,
    }).then(() => {
      logger.log({
        level: 'info',
        message: `${req.email} with password ${pwd} has been successfully registered`
      });
      res.send(200);
    });
  } catch (err) {
    logger.log({
      level: 'error',
      message: `${req.email} with password ${pwd} wasn't registered`
    });
  }
});

Good Example

const express = require('express');
const winston = require('winston');
const app = express();

// configuring logger
const logger = winston.createLogger({
  transports: [
    new winston.transports.Console(),
    new winston.transports.File({ filename: 'combined.log' })
  ]
});

app.post('/user/add', (req, res) => {
  try {
    modal.addUser({
      email: req.email,
      password: req.pwd,
    }).then((response) => {
      logger.log({
        level: 'info',
        message: `Success: ${response.data.id} user has been successfully registered`
      });
      res.send(200);
    });
  } catch (err) {
    logger.log({
      level: 'error',
      message: `An exception occurred while registering new user: ${err}`
    });
  }
});

Summary

All I can say that, it is important to have proper logging system setup in application. This can make developers life easier to debug any problem on production environment with less turnaround time.

Arduino vs. Raspberry Pi

Gurpreet Singh — Thu, 23 Jul 2020 06:13:54 +0000

As a new starter to IOT world, I would like to ask you which device or platform is better to start with. I never write any python code before but I am sound in JavaScript and NodeJS.

DEV Community: Gurpreet Singh

Day 6 (last) - Your first week in NodeJS

Overview

Start with account creation

Start with Heroku git setup

Install the Heroku CLI

Create a new Git repository

Deploy your application

Existing Git repository

Useful Links

Day 5 - Your first week in NodeJS

Quick recap

What is NoSQL

MongoDB Integration

MongoDB Integration

Useful Links

Day 4 - Your first week in NodeJS

Quick recap

Template engines

Serve html file as response

Partials

Server file app.js

Snapshots

Homepage

Contact page

User details

Middleware and static assets

Useful Links

Day 3 - Your first week in NodeJS

Prerequisite

Start with express.js

express.js routes

Creating REST API

GET request

POST request (Add user)

DELETE request (delete user)

Day 2 - Your first week in NodeJS

Prerequisite

Creating a Server

Client (front-end)

Server (backend)

JSON response example

HTML response example

Output

Routing

Node Package Manager (NPM)

Installing NodeMon

Useful Links

Day 1 - Your first week in NodeJS

1. Installation

2. Global Objects

3. Modules + require()

Let's take an example, you want to count the length of an array.

4. Event emitting

5. Read/Write/Steam files

Read operation ()

Write operation

Useful Links

PiHole on Docker

Installation

Configure devices

Are you logging your NodeJS code?

Why Logs are important?

Best Practices

1. Avoid using console.log()

2. Third party libraries

3. Source, Timestamps, Context

4. Logging levels

5. What shouldn’t be part of logs

Bad Example

Good Example

Summary

Arduino vs. Raspberry Pi

Serve `html` file as response

Server file `app.js`

`GET` request

`POST` request (Add user)

`DELETE` request (delete user)

3. Modules + `require()`

Let's take an example, you want to count the `length` of an array.

1. Avoid using `console.log()`