DEV Community: Tirtha Guha

Express Middlewares and Higher Order Functions, part 2

Tirtha Guha — Tue, 29 Jun 2021 12:16:59 +0000

Warning! We're going to see some Functional Programming concepts like pure functions, side effects etc. and obviously higher order functions.

Recap

In the last post, we saw how we can create a middleware wrapper that can wrap any function and make it an express middleware.

//withMiddleware.js

//Higher Order Function
const withMiddleware = (func) => {
  return (req, res, next) => {
    func(); //func has the EXECUTION LOGIC
    next();
  }    
}

module.export = withMiddleware;

And we can use it in this way

const withMiddleware = require('./withMiddleware');

const funcOne = () => {
  //YOUR BUSINESS LOGIC
}

const funcTwo = () => {
  //YOUR ANOTHER BUSINESS LOGIC
}

const middlewareOne = withMiddleware(funcOne);
const middlewareTwo = withMiddleware(funcTwo);

//use this middlewares in a standard way.
app.get("/path", middlewareOne, middlewareTwo, (req, res)=>{
  //further execution, if any
  res.send({});
})

We had a few unanswered questions

Passing data between middlewares? Although we solved it in the first post, see the PS: section, but how can we make it work effortlessly.
What if I'm writing auth or validation middlewares, that needs access to the request object?
How will we handle the async middlewares?
Logging. The generic activities of the middlewares needs to be logged. Like, in case of async middleware, that fetches data over another API, or fetches data from a DB, the time taken for the async operation needs to be logged.

Let's get started...

1. Passing data between middlewares

The most effortless way of passing data between middlewares is using the res.locals object. We're going to use the same.

//withMiddleware.js

//Higher Order Function
const withMiddleware = (func) => {
  return (req, res, next) => {
    // changed to this
    const response = func();
    if (response) {
      res.locals[`${func.name}Response`] = response; //Magic
    }
    next();
  }    
}

module.export = withMiddleware;

and when we finally use it

const withMiddleware = require('./withMiddleware');

const funcOne = () => {
  //YOUR BUSINESS LOGIC
  return true; //Functions returning now
}

const funcTwo = () => {
  //YOUR ANOTHER BUSINESS LOGIC
  return true; // Functions returning now
}

const middlewareOne = withMiddleware(funcOne);
const middlewareTwo = withMiddleware(funcTwo);

//use this middlewares in a standard way.
app.get("/path", middlewareOne, middlewareTwo, (req, res)=>{
  //further execution, if any
  const {funcOneResponse, funcTwoResponse} = res.locals;
  if(funcOneResponse && funcTwoResponse){
    res.send("Aal izz well");
  } else {
    res.status(400).send('Bad Request')
  }
})

2. Access to request and response object

Ok, now we need to give access to the request object. Lets make one more modification to our Higher order function.

//withMiddleware.js

//Higher Order Function
const withMiddleware = (func) => {
  return (req, res, next) => {
    // changed to this
    const response = func(req, res); //Pass the req, res, but not next
    if (response) {
      res.locals[`${func.name}Response`] = response;
    }
    next();
  }    
}

module.export = withMiddleware;

And how are we going to use it?

const withMiddleware = require('./withMiddleware');

//Function now has access to req and res
const funcOne = (req, res) => {
  if(req.body.username && req.body.password){
    return true; 
  } else {
    return false;  
  }
}

// Function may chose to ignore req and res
const funcTwo = () => {
  //YOUR ANOTHER BUSINESS LOGIC
  return true; // Functions returning now
}

const middlewareOne = withMiddleware(funcOne);
const middlewareTwo = withMiddleware(funcTwo);

//use this middlewares in a standard way.
app.post("/path", middlewareOne, middlewareTwo, (req, res)=>{
  //further execution, if any
  const {funcOneResponse, funcTwoResponse} = res.locals;
  if(funcOneResponse && funcTwoResponse){
    res.send("Aal izz well");
  } else {
    res.status(400).send('Bad Request')
  }
})

What if my function is an async function

Well, there is no easy answer to this. We need a different Higher Order Function to handle such cases

//withMiddleware.js

//Higher Order Function
const withMiddleware = (func) => {
  return (req, res, next) => {
    const response = func(req, res); 
    if (response) {
      res.locals[`${func.name}Response`] = response;
    }
    next();
  }    
}

//NEW Higher Order Function
const withAsyncMiddleware = (func) = {
  // Return an async middleware
  return async (req, res, next) => {
    const response = await func(req, res); 
    if (response) {
      res.locals[`${func.name}Response`] = response; // the response will be available as res.locals.${func.name}Response
    }
    next();
  } 
}

//We have to export both the functions now.
module.export = { withMiddleware, withAsyncMiddleware };

And we can use this in the following manner

// Have to change the import first
const { withMiddleware, withAsyncMiddleware } = require('./withMiddleware');

const funcOne = (req, res) => {
  if(req.body.username && req.body.password){
    return true; 
  } else {
    return false;  
  }
}

// FuncTwo is async, as it is going to make an API request.
const funcTwo = async () => {
  const data = await apiResponse(); // Here is the side-effect, and its making the API request.
  return data; // the async function returned the data;
}

const middlewareOne = withMiddleware(funcOne);
const middlewareTwo = withAsyncMiddleware(funcTwo); // wrapping this with async middleware

//use this middlewares in a standard way.
app.post("/path", middlewareOne, middlewareTwo, (req, res)=>{
  //further execution, if any
  const {funcOneResponse, funcTwoResponse} = res.locals;
  if(funcOneResponse && funcTwoResponse){
    res.send(funcTwoResponse); // Pure Magic
  } else {
    res.status(400).send('Bad Request')
  }
})

4. Now the easiest part. how we log the time for the async function?

Simple

//withMiddleware.js

//Higher Order Function
const withMiddleware = (func) => {
  return (req, res, next) => {
    const response = func(req, res); 
    if (response) {
      res.locals[`${func.name}Response`] = response;
    }
    next();
  }    
}

const withAsyncMiddleware = (func) = {
  return async (req, res, next) => {
    const t1 = Date.now();
    const response = await func(req, res); 
    const t2 = Date.now();
    console.log(`Time Taken to execute async ${func.name} is ${t2 - t1}`);
    if (response) {
      res.locals[`${func.name}Response`] = response; // the response will be available as res.locals.${func.name}Response
    }
    next();
  } 
}

module.export = { withMiddleware, withAsyncMiddleware };

That's all for now,
Thanks

Express middlewares and Higher Order Functions

Tirtha Guha — Mon, 28 Jun 2021 05:15:06 +0000

Like I said in the first post of the series, Express is all about middlewares.
So, here's the problem statement

Problem Statement

Can we create a middleware out of any function, so that developers don't have to worry about the tricky syntax and testing of middlewares

Well, the hardest part is over, defining the problem. Let's get our hands dirty now.

The generic syntax of a middleware is

//generic express middleware syntax
const middleware = (req, res, next) => {
  // EXECUTION LOGIC
  next(); // This is where the control leaves this middleware and moves to next item in the queue, which could be another middleware, or the final controller etc
}

Our intention is make it work in a way, that we should be able to inject the EXECUTION LOGIC dynamically.

For this, we slightly move away from OOPS of JS and take help of a functional programming concept called Higher Order Function.

How does it help, you ask? Well, we're going to create a function, that returns a middleware. and through that function, we're going to inject the execution logic.

We're going to create a file, withMiddleware.js

//withMiddleware.js

//Higher Order Function
const withMiddleware = (func) => {
  return (req, res, next) => {
    func(); //func has the EXECUTION LOGIC
    next();
  }    
}

module.export = withMiddleware;

Let's see how we can make this work for us.

const withMiddleware = require('./withMiddleware');

const funcOne = () => {
  //YOUR BUSINESS LOGIC
}

const funcTwo = () => {
  //YOUR ANOTHER BUSINESS LOGIC
}

const middlewareOne = withMiddleware(funcOne);
const middlewareTwo = withMiddleware(funcTwo);

//use this middlewares in a standard way.
app.get("/path", middlewareOne, middlewareTwo, (req, res)=>{
  //further execution, if any
  res.send({});
})

The Magic ends here for now, but there are a few problems yet to be solved for this. In the next post we're going to take this a little higher and address the following

How will we handle the async middlewares?
What if I'm writing auth or validation middlewares, that needs access to the request object?
Passing data between middlewares? Although we solved it in the first post, see the PS: section, but how can we make it work effortlessly
Logging. The generic activities of the middlewares needs to be logged. Like, in case of async middleware, that fetches data over another API, or fetches data from a DB, the time taken for the async operation needs to be logged.

Express routes with Builder pattern

Tirtha Guha — Sat, 26 Jun 2021 12:48:26 +0000

In the last post, we were creating a common interface for creating express routes, thereby improving reusability.

We now want to take it up a notch. How about having some fun with Builder Pattern, using plain old Javascript Functions.

Creating express routes with Builder Pattern

So, I created a file RouteBuilder.js and it looks like this.

// RouteBuilder.js
const express = require("express");

const RouteBuilder = function () {
  this.path = undefined; //string
  this.method = undefined; //string
  this.middlewares = undefined; //array of middlewares
  this.controller = undefined; //final request handler
  this.router = express.Router();

  this.setPath = function (path) {
    this.path = path;
    return this;
  };
  this.setMethod = function (method) {
    this.method = method;
    return this;
  };
  this.setMiddlewares = function (middlewares) {
    this.middlewares = middlewares;
    return this;
  };
  this.setController = function (controller) {
    this.controller = controller;
    return this;
  };
  this.build = function () {
    this.router
      .route(this.path)
      [this.method.toLowerCase()](...this.middlewares, this.controller);
    return this.router;
  };
};

module.exports = RouteBuilder;

and you can invoke this RouteBuilder like this

const RouteBuilder  = require('./RouteBuilder');
const authMiddleware = require("../authMiddleware")
const validateMiddleware = require("../validateMiddleware")
const actionMiddleware = require("actionMiddleware")
const serviceMiddleware= require("serviceMiddleware")

const routeBuilder = new RouteBuilder();

const userController = (req, res, next) => {
  res.send({});
};

//build the route
let route = routeBuilder
  .setPath('/')
  .setMethod('GET')
  .setMiddlewares([
    authMiddleware,
    validateMiddleware,
    actionMiddleware,
    serviceMiddleware,
  ])
  .setController(userController)
  .build();

//Finally set the route to the app
app.use("/users", route); // here app is the express app.

Routes in Express

Tirtha Guha — Thu, 24 Jun 2021 16:54:00 +0000

It all started with a piece of code that I saw while reviewing an express based application.
... and it wasn't something that I was seeing for at the first time. I've seen applications like this since long back, almost all applications generated with express-generator.

var express = require('express');
var router = express.Router();
const authorizationMiddleware = require("../authorizationMiddleware")
const payloadValidationMiddleware = require("../payloadValidationMiddleware")
const createAction = require("createAction")
const serviceRequest = require("serviceRequest")

/* POST request to filter users listing. */
router.post('/', authorizationMiddleware, payloadValidationMiddleware, async function(req, res, next) {
  //Some code
  const action = createAction(req);
  const response = await serviceRequest(action);
  //Some code to modify the response
  res.send(response);
});

module.exports = router;

My obvious questions were, why can't createAction and serviceRequest also be middlewares?

And how can this be made prettier?

For the first question obviously, the answer was, the developer didn't know what is the standard way to pass on data from one middleware to another middleware. That's a knowledge issue.

The second one obviously, kept me up in the night for sometime. What could be the abstractions that can be done, so that there is less code to be written for the developers.

In a standard express app (which exposes restful APIs, not the kind that spits out a UI) , its all about middlewares. The whole application can(and should) be composed with middlewares. and the routes are

So, after some fiddling around, this is what I came up with.

Create an abstraction for router

// router.js
const express = require("express");

const createRouteConfig = ({
  path,// /filter or /filter/:paramOne/:paramTwo etc.
  method, //"GET"
  middlewares, // an array of route middlewares [middleware1, middleware2, ..., middlewareN],
  controller, // the final request handler
}) => {
  const router = express.Router()
  if (middlewares.length) {
    router.route(path)[method.toLowerCase()](...middlewares, controller);
  } else {
    router.route(path)[method.toLowerCase()](controller);
  }

  return router;
};

module.exports = createRouteConfig;

And then in My Route Files, what I can do is

//User Route
const createRouteConfig = require("../router");
const authMiddleware = require("../authMiddleware")
const validateMiddleware = require("../validateMiddleware")
const actionMiddleware = require("actionMiddleware")
const serviceMiddleware= require("serviceMiddleware")


const userController = (req, res, next) => {
  res.send({});
};
const route = createRouteConfig({
  path: "/",
  middlewares: [authMiddleware, validateMiddleware, actionMiddleware, serviceMiddleware],
  method: "post",
  controller: userController,
});

//Finally set the route to the app
app.use("/users", route); // here app is the express app.

Hmmmm.. Code was already looking pretty. I didn't do much, all I did was move things around, and create an api for creating routes.

PS:

I almost published this without answering the first question. The express-approved way of passing data between middlewares is using res.locals object. In one middleware, you set

const authMiddleware = (req, res, next) => {
  // your logic
  res.locals.authenticated = true;
  next();
}

and then in the next middleware you can read them as

const validateMiddleware = (req, res, next) => {
  const {authenticated} = res.locals;
  if(authenticated){
    // your business logic
   next();
  } else {
    // throw error, or respond with 401
  }
}

Reusing Images: Publishing to a registry

Tirtha Guha — Tue, 24 Nov 2020 04:24:31 +0000

Ok, so here's we're going to learn how to reuse images. We often follow the principle of build once deploy anywhere and for that we need to maintain our images in a repository for later reuse.

For this tutorial, we'll use amazon ECR. I've logged into Amazon AWS console and searched for ECR.

Create a repository

The best way to create a repository is probably to follow this https://docs.aws.amazon.com/AmazonECR/latest/userguide/what-is-ecr.html. It even describes how to create an IAM user and use that to manage the ECR interactions, instead of using the root account.

The repository I've created is called blogrepo

Creating and Pushing an image.

For this, we're going to create a bare minimum express application and a Dockerfile

user@host:~/ecr-push-demo-app$ touch Dockerfile
user@host:~/ecr-push-demo-app$ touch app.js
user@host:~/ecr-push-demo-app$ touch package.json
user@host:~/ecr-push-demo-app$ npm install express --save

Put the following in these files

In app.js

//app.js
var express = require("express");
var app = express();

app.use(express.json());
app.use(express.urlencoded({ extended: false }));

app.get("/", function (req, res, next) {
  res.send({ title: "Express", application: "Application 1" });
});

app.listen(3000, () => console.log("Running on http://localhost:3000"));

In package.json

{
  "name": "ecr-push-demo-app",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "start": "node app.js"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "dependencies": {
    "express": "^4.17.1"
  }
}

and finally, in the Dockerfile

FROM node:12-slim
WORKDIR /app
COPY ./package*.json ./
RUN npm install
COPY ./ ./
EXPOSE 3000

# Run the code
CMD [ "npm", "start" ]

Build the image

Lets build the image and tag it as blogrepo20201120

$ sudo docker build -t blogrepo20201120 .

Lets tag this image for pushing to ecr and then push the image

Just a reminder, the repository I've created on ECR console is called blogrepo and I created this in ap-south-1 region

$ docker tag blogrepo20201120:latest your_aws_account_id.dkr.ecr.ap-south-1.amazonaws.com/blogrepo:latest
$ docker push your_aws_account_id.dkr.ecr.ap-south-1.amazonaws.com/blogrepo:latest

Lets see if we can pull the image and run it

$ docker pull your_aws_account_id.dkr.ecr.ap-south-1.amazonaws.com/blogrepo:latest
$ sudo docker run -p 3000:3000 -d your_aws_account_id.dkr.ecr.ap-south-1.amazonaws.com/blogrepo:latest

That's all. Just open your browser to http://localhost:3000

Build images and run them with docker compose

Tirtha Guha — Mon, 21 Sep 2020 19:30:18 +0000

In the previous post, we saw how we can run more than one containers and how to route requests to them using nginx.

Of course, we had to build 3 images individually, and then run the three images in sequence to get the final ecosystem.

In this post, we'll try to combine all the steps in just one command using docker-compose.

We'll start with the same repo, where the previous post's code is checked in, clone it locally, and then setup the docker-compose yml file, and run it.

Step 1. Clone the codebase.

Clone the repo https://github.com/tirthaguha/container-reverse-proxy.

Step 2. Create the YML file

Network Drivers

Okay, so before we create the docker-compose.yml file, its important to understand how networking works in docker-compose.
Docker can create a virtual network for you to abstract the complexities of networking, and all your containers run and interact in that network. One interesting thing about this virtual network is the network driver. Based on your container ecosystem setup, the drivers can be bridge, overlay and macvlan.
In our case, since we're going to run all the containers in the same host, we're going to chose bridge network driver in our docker compose file. You may have a look into this article and learn more about docker networks

Create the docker compose file

docker-compose.yml is a file that contains all the information about the applications, their Dockerfiles and the network they are going to run in.

Here's our annotated docker-compose.yml that you need to put in your container-reverse-proxy directory.

version: "3"

# start by defining the services
services:
  # this is the first nodeJS application
  nodejs_1:
    # lets keep name of the image as first-app
    image: first-app
    # this is where the Dockerfile is compiled into an image
    build:
      # where is the Dockerfile kept, relative to this directory
      context: ./first-app
      # obviously, the Dockerfile in that directory
      dockerfile: Dockerfile
    # port mapping
    ports:
      - 3000:3000
    # name of the container
    container_name: first-app
    # if the container crashes, it would be restarted automatically
    restart: unless-stopped
    # the virtual network of which this container will be a part of
    networks:
      - app-network

  # this is the second nodeJS application
  nodejs_2:
    # lets keep name of the image as first-app
    image: second-app
    build:
      # where is the Dockerfile kept, relative to this directory
      context: ./second-app
      dockerfile: Dockerfile
    ports:
      - 4000:4000
    container_name: second-app
    restart: unless-stopped
    networks:
      - app-network

  # Finally, the nginx, that controlls the route
  nginx-server:
    image: nginx-load-balancer
    build:
      context: ./nginx
      dockerfile: Dockerfile
    ports:
      - 8080:80
    container_name: nginx-load-balancer
    restart: unless-stopped
    networks:
      - app-network

# the mysterious item, network
networks:
  app-network:
    driver: bridge

Step 3. Run the docker-compose file

Just run

$ sudo docker-compose up -d

then go to your browser and try opening the following URLs

Routing requests to containers with nginx

Tirtha Guha — Wed, 09 Sep 2020 14:30:55 +0000

Often in large application ecosystems a common nginx is used as a loadbalancer and reverse-proxy for many applications. In a previous post of this series, we have seen how nginx can be used to load balance traffic multiple containers.
In this post, we're going to see how we can use nginx as a reverse-proxy which would route requests to different containers based on path.

What we are going to do

create an express application, with path /app-first path and launch it in a container
create an express application, with path app-second path and launch it in a container
create an nginx container, and have 2 paths configured in it. path-1 forwards the request to first container, and path-2 forwards the request to second container

Create the directories

mkdir first-app
mkdir second-app
mkdir nginx-docker

Create the first application

I just created a very basic nodeJs application. The app.js looks like the following

var express = require("express");
var app = express();

app.use(express.json());
app.use(express.urlencoded({ extended: false }));

app.get("/app-first", function (req, res, next) {
  res.send({ title: "Express", application: "Application 1" });
});

app.listen(3000, () => console.log("Running on http://localhost:3000"));

The package.json looks like the following

{
  "name": "express-with-env",
  "version": "0.0.0",
  "private": true,
  "scripts": {
    "start": "node app.js"
  },
  "dependencies": {
    "express": "~4.16.1"
  }
}

and in parallel to these, the Dockerfile looks like the following

FROM node:12-slim
WORKDIR /app
COPY ./package*.json ./
RUN npm install
COPY ./ ./
EXPOSE 3000

# Run the code
CMD [ "npm", "start" ]

Create the second application

The second application is exactly like the first application, except, it runs on PORT 4000, and exposes path /app-second. the app.js looks like the following

var express = require("express");
var app = express();

app.use(express.json());
app.use(express.urlencoded({ extended: false }));

app.get("/app-second", function (req, res, next) {
  res.send({ title: "Express", application: "Application 2" });
});
app.listen(4000, () => console.log("Running on http://localhost:4000"));

and the Dockerfile looks like the following

FROM node:12-slim
WORKDIR /app
COPY ./package*.json ./
RUN npm install
COPY ./ ./
EXPOSE 4000

# Run the code
CMD [ "npm", "start" ]

The packageJSON looks exactly like the first application

Create the nginx config

Go to the nginx-docker directory and create nginx.conf file

upstream first-app {
    server 172.17.0.1:3000 weight=1;
}

upstream second-app {
    server 172.17.0.1:4000 weight=1;
}

server {

  location /app-first {
    proxy_set_header X-Forwarded-Host $host:$server_port;
    proxy_set_header X-Forwarded-Server $host;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_pass http://first-app;
  }

  location /app-second {
    proxy_set_header X-Forwarded-Host $host:$server_port;
    proxy_set_header X-Forwarded-Server $host;
    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
    proxy_pass http://second-app;
  }

}

Create a dockerfile for the nginx and here's what it looks like

FROM nginx
RUN rm /etc/nginx/conf.d/default.conf
COPY nginx.conf /etc/nginx/conf.d/default.conf

Create the images and run the containers

Go to first-app directory and build the image

sudo docker build -t first-app .

Go to the second-app directory and build the image

sudo docker build -t second-app .

Run the images now

sudo docker run -p 3000:3000 -d first-app
sudo docker run -p 4000:4000 -d second-app

Now, if you open http://localhost:3000/app-first in browser, you should get the following JSON

{
"title": "Express",
"application": "Application 1"
}

and by opening http://localhost:4000/app-second, you should get

{
"title": "Express",
"application": "Application 2"
}

You see, the applications are running in different ports. Your target is to expose them both in the same port.

Go to the nginx-docker directory and build the nginx image and run it.

sudo docker build -t nginx-proxy .
sudo docker run -p 8000:80 -d nginx-proxy

Now you'll be able to see both the applications running in port 8000, as

PS:
We wrote code for 2 applications(first-app and second-app) here, and wrote configuration for the third(nginx). We created their docker images, and we ran those images. Of course, these are too many commands to handle. In our next post, we'll look into docker-compose to simplify these image building and running.
The code for this tutorial is checked in at https://github.com/tirthaguha/container-reverse-proxy

Setting environment variables to your container

Tirtha Guha — Sat, 05 Sep 2020 17:07:57 +0000

In this post, we'll learn how to set environment variables in docker containers. Often things like password, environment configurations, API credentials etc are not checked in the repositories as a security measure. These are set in environment variables, from where the application reads it during the runtime.

Create a sample application

For this, we'll create a simple express application which will read the environment variable and show it on the browser

run npm init to create a package json

$ mkdir test-app
$ cd test-app
$ npm init --y
$ npm install express --save
$ touch app.js

Open app.js and type in the following

// app.js
var express = require("express");
var app = express();

app.use(express.json());
app.use(express.urlencoded({ extended: false }));

app.get("/", function (req, res, next) {
  res.send({ title: "Express", envVal: process.env.TESTVAL });
});

app.listen(3000, () => console.log("Running on http://localhost:3000"));

add the start script in package.json

{
  "name": "express-with-env",
  "version": "0.0.0",
  "private": true,
  "scripts": {
    "start": "node app.js"
  },
  "dependencies": {
    "express": "~4.16.1"
  }
}

Run the application

I'm using ubuntu v18 for development. If you're on windows, you may need to set the env variable in a slightly different way.

$ TESTVAL=QWERTY npm start

Now if you open http://localhost:3000 in the browser, you would see

{
"title": "Express",
"envVal": "QWERTY"
}

Create the docker image

Let's create a Dockerfile, parallel to your packageJSON

$ touch Dockerfile
$ touch .dockerignore

Lets now open Dockerfile and add these entries into it.

FROM node:12-slim
WORKDIR /app
COPY ./package*.json ./
RUN npm install
COPY ./ ./
EXPOSE 3000

# Run the code
CMD [ "npm", "start" ]

Build the docker image from the Dockerfile

$ sudo docker build -t test-image .

Finally run the image with an entrypoint

$ sudo docker run -e "TESTVAL=Docker with env variable" -p 3000:3000 -d test-image

if you now open http://localhost:3000 in a browser, you'll get

{
"title": "Express",
"envVal": "Docker with env variable"
}

Let spin up another container from the same image

$ sudo docker run -e "TESTVAL=Another Docker Container from the same image" -p 4000:3000 -d test-image

if you now open http://localhost:4000 in a browser, you'll get

{
"title": "Express",
"envVal": "Another Docker Container from the same image"
}

PS:

Often enterprises have multiple prod and non-prod environments to support. Each environment often have different configurations, but your code must follow the strategy build once deploy anywhere. With this tutorial, you'll be able to do that effectively.

Load Balancing containers

Tirtha Guha — Thu, 06 Aug 2020 11:23:47 +0000

In me previous post, I demonstrated a simple scenario of how we can run express based web applications in docker containers.

Often, for performance or resiliency, we run more than one instances of Web Applications. When you containerised your Web Application, you should be able to distribute the traffic evenly/weighted between multiple containers too.

In this exercise, I'm going to demonstrate how we can load balance between two containers.

What we need

A system with docker installed in it.
Complete the exercise in my previous post. If your system has docker, it won't take more that 15 mins.
Basic knowledge of nginx

Lets get started

Quick recap: I have a repo in github with a basic express application. we're going to run this in a docker container.

Clone, Build and Run Container

$ git clone https://github.com/tirthaguha/card-deck-test.git
$ cd card-deck-test
$ touch Dockerfile

Open Dockerfile in a editor and paste the following

FROM node:12-slim

WORKDIR /app
COPY ./package*.json ./
RUN npm ci --only-production
COPY ./ ./

EXPOSE 4000
CMD [ "node", "./bin/www" ]

Build the Image

docker build -t card-deck-test .

Run Multiple Instances of the same container image

docker run -p 4000:4000 -d card-deck-test
docker run -p 4001:4000 -d card-deck-test

Open the following URLs in your browser to check if they are running fine on their individual ports
http://localhost:4000/card-deck/shuffle
http://localhost:4001/card-deck/shuffle

Load Balance them

Enter nginx. nginx is a popular simple open-sourced webserver, often used as a loadbalancer and reverseproxy.

Create the nginx.conf

In a different directory, outside the card-deck-test, create a directory

$ mkdir nginx-docker
$ cd nginx-docker
$ touch nginx.conf
$ touch Dockerfile

Open nginx.conf in your editor and enter the following code

upstream card-app {
    server 172.17.0.1:4000 weight=1;
    server 172.17.0.1:4001 weight=1;
}

server {
    location / {
        proxy_pass http://card-app;
    }
}

Here, you're redirecting web traffic between 2 docker containers running on 172.17.0.1:4000 and 172.17.0.1:4001 respectively. 172.17.0.1 is default docker gateway IP address. You're giving them the weight=1 for both, so traffic is going to be evenly routed, in a round-robin fashion.

Create the Dockerfile for nginx

Open the Dockerfile in the editor and lets put this code in it

FROM nginx

# remove default nginx conf file
RUN rm /etc/nginx/conf.d/default.conf

# Copy the nginx.conf file as default conf
COPY nginx.conf /etc/nginx/conf.d/default.conf

Build and run the nginx docker container

Build the nginx image

$ docker build -t nginx-load-balancer .

Run the nginx image

docker run -p 8080:80 -d nginx-load-balancer

You now can run the nginx URL on the browser
http://localhost:8080/card-deck/shuffle

Happy Load Balancing!

Run Express in Docker

Tirtha Guha — Wed, 05 Aug 2020 18:48:38 +0000

Docker has emerged to be a standard of building and deploying applications. The best part of docker is it gives a complete abstraction over the environment or OS. There are plenty of material on what docker is, or how it works, however, what we are going to discuss over here is to how to put your nodeJS express app in docker container.

First things first

What we need

A computer with docker installed on it. Preferably linux/mac.

Windows has its challenges to run docker desktop, especially win10 Home editions, so you need to really go through the documentation in docker site before installing docker.
WSL too is dependent on Win10, so go as per the documentation given in docker website.

A Express Application to play with.

I have an express app, created out of express generator hosted on github which i'm going to be using for this exercise.
https://github.com/tirthaguha/card-deck-test
The code on this repo runs on PORT 4000, and exposes API
http://localhost:4000/card-deck/shuffle. This repo makes a call to external APIs, so your system should be enabled to make API calls.

Enough talk, run me the docker

If you are using my repository, don't run npm install. :-O. Lets not waste time here running a simple express app, lets just go straight into using docker.

Create the Dockerfile

Within the repository, create a Dockerfile, and a .dockerignore file, next to package.json

touch Dockerfile
touch .dockerignore

Lets open Dockerfile and add these entries into it.

FROM node:12-slim

# create a directory in your docker image
WORKDIR /app

# install dependencies
COPY ./package*.json ./

# run npm install. you just need the runtime dependencies here
RUN npm ci --only-production

# Copy the rest of the code
COPY ./ ./

# Expose the port, on which your application is running
EXPOSE 4000

# Run the code
CMD [ "node", "./bin/www" ]

Save this Dockerfile.

As you would have guessed already, .dockerignore contains the entries which will be ignored when you're copying all your code on the docker image. Enter the following in the .dockerignore file and save it.

node_modules
npm-debug.log

Build the image

Build your first docker image

docker build -t card-deck-test .

Check the image in your local registry

docker images

It should give you an output like

REPOSITORY       TAG      IMAGE ID
card-deck-test   latest   aaaa1111dddd1111

Run the image

docker run -p 4000:4000 -d card-deck-test

The express application is now running on port 4000 in the container, and the container is exposing port 4000. So to summarise, you can now see your application running by opening
http://localhost:4000/card-deck/shuffle on your browser.

What next

Play around with docker

Run multiple containers simultaneously

docker run -p 4000:4000 -d card-deck-test
docker run -p 4001:4000 -d card-deck-test
docker run -p 4002:4000 -d card-deck-test

then open all of them on browser
http://localhost:4000/card-deck/shuffle
http://localhost:4001/card-deck/shuffle
http://localhost:4002/card-deck/shuffle

See the containers running

docker ps -a

Stop the docker container

docker stop [CONTAINER_ID]

Remove the docker container

docker rm [CONTAINER_ID]

Happy Containerising!

What to review in a code review

Tirtha Guha — Wed, 19 Feb 2020 16:30:47 +0000

In a code review, I look for the following

To Prevent Language Specific Issues
To Prevent Framework Specific Issues
To Ensure code quality is maintained
To ensure unit tests validate the right scenarios

I don't think you're required to know the whole codebase when you're reviewing code.

Please share your thoughts in comments.

discuss

Gitflow and NPM versions

Tirtha Guha — Fri, 03 Aug 2018 12:51:35 +0000

I am following standard gitflow, and I have different environments for testing the dev builds, and release builds. master goes to production.

I also have my JS app divided into multiple private npm modules which goes into private npm repository.

Q1

Is there any way I can version my npm packages, against the branches they are built on in a standard way?

What I have tried is, I have prerelease pre-ids added to the versions. like
1.0.0-rc.0 //for master
1.0.0-beta.0 //for release
1.0.0-alpha.0 //for dev

But if I create a feature branch from master, it contains the master's version. When I try to raise a PR from it to dev, then it shows conflict, since dev has -alpha.x in its version. To resolve the conflict, I'll have to consume the target branch's versioning. Same issue when it goes for merging on release branch too.

And when it comes to merging to master, the release version (one with -beta.0) completely replaces the master.
So it becomes like this: on master,

It was	After Merge	After version bump
1.0.0-rc.0	~~1.0.0-rc.0~~ 1.0.0-beta.0	1.0.0-rc.0

Ideally after the version bump i would have wanted it to be 1.0.0-rc.1

Is it possible to keep package JSONs out of versioning.

Q2

How do I control the versioning in the package JSON of the application where these NPM modules are consumed? It too is on gitflow and feature branching model, and I would want that the App, when it is building on dev branch, it builds with artifacts that are published from their respective dev branches.

Honestly, I might be misusing gitflow too, but as of now, too confused to figure out where I'm going wrong.

Thanks in Advance