DEV Community: Paul Arah

Container Networking - The Docker Way

Paul Arah — Sat, 04 Feb 2023 23:02:24 +0000

Containers put together a couple of features of the Linux kernel to achieve isolation. Since containers are inherently isolated from their host and other containers on the same host, there needs to be some networking mechanism that facilitates communication between containers and the host network interface. At the core of it, that's essentially what container networking is. This post focuses on how docker does it; the networking parts of docker we tend to run into daily as developers.

N/B: I am using a Linux machine. The demo might have slight differences if you use a Mac or Windows. This is mostly because docker containers on windows and mac run inside a Linux VM. Most container networking concepts directly map to regular networks, so basic familiarity with networking concepts would be helpful.

Containers

When you create a docker container, it lives in an isolated network namespace and is unreachable from the outside world. We can test this out by creating a container running Apache web server.

// creates a container with Apache httpd running in it
$ docker run -dit --rm --network none  --name my-container httpd

// inspects the container and formats for the networks settings
$ docker inspect -f '{{json .NetworkSettings}}' my-container

When we inspect the container for its networking details, we see it has no IP address or way to reach it. This is because containers are designed to work exactly like this. Containers isolate resources like the file system, networking, processes and memory at the OS level.
Notice we passed a --network flag to none when creating the container. This disables docker networking features. As much as we want isolation, In many cases, we wouldn't want our containers to be unreachable, and docker provides a couple of networking options called drivers to cater to this. We'll focus on bridge and host drivers, as these are the two most common drivers.

//list the available docker networks
$ docker network ls 
  NETWORK ID     NAME      DRIVER    SCOPE
26fb9b15153b   bridge    bridge    local
19a78c13e726   host      host      local
705da40f3ea2   none      null      local

The Default Bridge Network

We create a container, docker uses the default bridge network. The default bridge network allows containers within the same network and host to communicate. We can try this by stopping our container, restarting it without a network option and inspecting its networking settings.

Sidenote: I recently figured out the docker inspect command can be, for lack of a better description "overloaded or variadic". Meaning docker inspect command works without explicitly specifying what docker object we're inspecting. So we don't have explicitly say something like docker container inspect my-container. Docker automatically figures out if the object we're inspecting is a container, network or volume. Cool developer experience, innit?

$ docker run -dit --rm  --name my-container httpd
$ docker inspect my-container 

"Networks": {
                "bridge": {
                    "IPAMConfig": null,
                    "Links": null,
                    "Aliases": null,
                    "NetworkID": "26fb9b15153b91f9ed246b24f400f614f90e6f8f954c2b8f3682ada020a6f55a",
                    "EndpointID": "d1ed5ca3766b6abe731d43cdaf6bf085c35a84021b6adef6d1b7e47f6c3e1a5f",
                    "Gateway": "172.17.0.1",
                    "IPAddress": "172.17.0.2",
                    "IPPrefixLen": 16,
                    "IPv6Gateway": "",
                    "GlobalIPv6Address": "",
                    "GlobalIPv6PrefixLen": 0,
                    "MacAddress": "02:42:ac:11:00:02",
                    "DriverOpts": null
                }
            }
        }

We can see that the container is now attached to the default bridge network. The container is assigned an IPv4 address by default, this can be configured to use IPv6 instead. We can reach the container directly by its IP address from our host(I haven't tried this out, but it's very likely that by default, you wouldn't be able to reach the container by its IP address directly from the host if you're on Mac or Windows).

// apache runs by default on port 80, no need to specify the port.
$ curl 172.17.0.2
<html><body><h1>It works!</h1></body></html>

User-defined Bridge Networks

As the name implies, user-defined bridge networks are bridge networks we create and can configure ourselves. The default bridge network is great and provides a finer level of network isolation, but we still lose out on a couple of great features of docker bridge networks when we stick to the default bridge network. Docker recommends we always create our own bridge networks. Working with user-defined bridge networks is quite intuitive. We can create, configure and delete networks. We can also connect and disconnect containers from a network.

// creates a brige network - by default docker uses the brigde driver 
$ docker network create my-net

// delete a network 
docker network rm my-net

If we inspect the network, we see that there are currently no containers attached to it. We can connect and disconnect containers to the network on the fly.
To try this out, we will create a busybox container in addition to the Apache httpd container running and connect them to the same network.

// creates a container from busybox image 
$ docker run -dit --rm --network my-net  --name busybox busybox

//connects the Apache web server to the my-net network
$ docker network connect my-net my-container

If we inspect the my-net network now, both containers are listed. Since both containers are on the same bridge network, we can easily reach one container from the other.

$ docker exec -it busybox /bin/sh
$ ping my-container

Notice we're pinging the container by its name, my-container? A DNS entry is created for the container name and IP address, allowing easy service discovery. Containers are somewhat ephemeral, and the IP addresses are bound to change. This saves us the hassle of dealing with dynamic IP addresses.

We can map a port from the container to the host. This is called port forwarding. ThIS way, we can reach the container from the host.

//  -p flag maps port 80 on the container to port 3000
$ docker run -dit --rm --network none -p 3000:80  --name my-container httpd
$ curl localhost:3000

The Host Network

The host network driver removes the isolation between the host and the container. The container now directly relies on the host network. This means it doesn't get a network stack or IP address.

// creates a container using the host network driver docker run -dit --rm --network host --name my-container httpd
docker run -dit --rm --network host --name my-container httpd

We can reach our containers directly on the localhost without port forwarding.

$ curl localhost
<html><body><h1>It works!</h1></body></

Overlay Network

An honourable mention here is the overlay network driver. This network type plays an important role in how multi-host systems like our favourite container orchestration tools works. Docker swarm leverages this to connect multiple hosts, and Kubernetes infamously leverages the same concept for a VLAN that spans the cluster nodes.

Bare-bones Container Networking Internals
Docker networks create a great abstraction over the underlying network stack, If you're interested in the low-level Linux details independent of a container runtime, Ivan Velichko has a great blog post on bare-bones container networking.

Deploy and automatically provision SSL certs on a Node.js server with Traefik and Docker.

Paul Arah — Wed, 07 Jul 2021 00:31:33 +0000

In this tutorial, we would learn to deploy Node.js servers with docker on a Linux-based VPS and automatically provision SSL certificates with Traefik.

Prerequisites:

Basic familiarity with Node.js, docker and docker-compose.
A virtual private server with a public IP address from any cloud service provider of your choice.
A domain or subdomain and a corresponding DNS record pointing the public IP address of your VPS.

Introduction

Traefik is an open-source, cloud-native reverse proxy. A reverse proxy essentially sits in front your servers and handles incoming clients request. So instead your client request directly going to your Node.js server, the request first goes through Traefik and Traefik then forwards it to your server. This allows us do things like SSL encryption, canary deploys, load balancing among others.

Now lets get started!

Spin Up a Linux VPS

This could be an Amazon EC2 instance, A digital ocean droplet, Linode VMs, etc or even an on-prem Linux machine with publicly accessible IP address. For this demo, I am using an ubuntu digital ocean droplet.

Install docker and docker compose.

This tutorial focuses on the deployment phase. You can read the docker docs on how to install docker and docker compose for your respective platforms.

Firewall restrictions

Depending your VPS and setup ensure that both port 80 and port 443 are accessible from outside. This could mean adjusting the inbound rules of your security group in your VPC on AWS or opening the ports on ufw.

DNS records

If you haven't done so already, create a DNS record for your domain or sub-domain and point it to the public IP address of your VPS. You can confirm the DNS propagation by pinging your domain and seeing that it resolves to the IP address of your VPS. If you use a DNS provider like cloudfare that supports proxying, you might want to turn off this feature while testing.



ping mydomian.com // should resolve to your VPS IP address

Node.js Server

In this example, I will be demonstrating using a simple fastify server.




//server.js file 

const fastify = require("fastify")({ logger: true });
fastify.get("/", async (request, reply) => {
  return { message: "Hello world! I'm using fastify" };
});
const start = async () => {
  try {
    await fastify.listen(3000, "0.0.0.0");
  } catch (err) {
    fastify.log.error(err);
    process.exit(1);
  }
};
start();

Dockerise Node.js server

We containerise our Node.js server with docker using the Dockerfile below.



FROM node:12-alpine
RUN mkdir home/node-traefik
WORKDIR /home/node-traefik
COPY . .
RUN npm install
EXPOSE 3000
CMD [ "node", "server.js" ]

N/B: This is a basic example on how to dockerise a Node.js App. For production use cases, you should probably read the Node.js and Docker best practices guide here.

Docker Compose

Now we create a docker-compose file and reference our Dockerfile. At this stage we can start our Node.js server with the docker-compose up command.



services:
  node-server:
    build:
      context: ./
      dockerfile: Dockerfile
    ports:
      - "3000:3000"

Configuring Traefik

To introduce Traefik into our flow, we create a new service for Traefik in our docker-compose file.



services:
  reverse-proxy:
    image: traefik:v2.4
    container_name: "traefik"
    command:
      - "--api.insecure=true"
      - "--api.dashboard=true"
      - "--api.debug=true"
      - "--providers.docker=true"
      - "--log.LEVEL=DEBUG"
      - "--entryPoints.web.address=:80"
      - "--entryPoints.websecure.address=:443"
      - "--providers.docker.exposedbydefault=false"
      - "--certificatesresolvers.myresolver.acme.httpchallenge=true"
      - "--certificatesresolvers.myresolver.acme.httpchallenge.entrypoint=web"
      - "--certificatesresolvers.myresolver.acme.email=hello@paularah.com"
      - "--certificatesresolvers.myresolver.acme.storage=/letsencrypt/acme.json"
    ports:
      - "443:443"
      - "80:80"
      - "8080:8080"
    volumes:
      - "./letsencrypt:/letsencrypt"
      - "/var/run/docker.sock:/var/run/docker.sock:ro"

Traefik has the concept static and dynamic configurations. Static configuration are needed by Traefik at startup time and if changes are made to the static configurations, Traefik has to be restarted for these changes to take effect. When using Traefik with docker-compose, we define static configurations as commands in the docker-compose file.

Lets go through each command in the static configuration individually.

- "--providers.docker=true" tells traefik that docker is our key infrastructure components and thus traefik queries the docker API for the relevant information it needs.
--api.insecure enables the traefik dashboard in insecure mode. For production uses cases, you want to use basic authentication and TLS on the dashboard.
- "--providers.docker.exposedbydefault=false" tells traefik not to expose a service unless being explicitly to do so.
The mounted volume with- "/var/run/docker.sock:/var/run/docker.sock:ro" allows Traefik to communicate with docker.
The - "--entryPoints.web.address=:80" and - "--entryPoints.websecure.address=:443" line declare a network and corresponding port entry points into Traefik.
The "--certificatesresolvers.myresolver.acme.email=hello@paularah.com creates a certificate resolver named myresolver. The certificate resolver is responsible for generating, renewing and disposing certificates.
- "--certificatesresolvers.myresolver.acme.storage=/letsencrypt/acme.json" Tells our certificate resolver to save our certificates in acme.json file in the letsencrypt volume.
- "--certificatesresolvers.myresolver.acme.httpchallenge=true" Tells the certificate resolver to use the HTTP challenge.

At the this stage, if we start up our containers, the Traefik dashboard will be accessible on port 8080http://<IP>:8080.

Our Node.js server services has not been linked to Traefik yet. This is where the concept of dynamic configuration comes in. Unlike static configurations, dynamic configurations can be updated after Traefik has started. Traefik watches for changes in the dynamic configurations and applies it with no need to restart Traefik. When using Traefik with docker, we add dynamic configurations by using labels. Traefik reads these meta-data and applies it to the respective service.



 node-server:
    build:
      context: .
      dockerfile: Dockerfile
    container_name: node-server
    labels:
      - "traefik.enable=true"
      - "traefik.http.routers.node-server.rule=Host(`play.paularah.com`)"
      - "traefik.http.routers.node-server.entrypoints=websecure"
      - "traefik.http.routers.node-server.tls.certresolver=myresolver"
      - "traefik.http.middlewares.redirect-to-https.redirectscheme.scheme=https"
      - "traefik.http.routers.redirs.rule=hostregexp(`{host:.+}`)"
      - "traefik.http.routers.redirs.entrypoints=web"
      - "traefik.http.routers.redirs.middlewares=redirect-to-https"

Since we configured Traefik not to expose services except being explictly told to do so, The - "traefik.enable=true" label now exposes our Node.js server container to Traefik.
- "traefik.http.routers.node-server.rule=Host(play.paularah.com)" creates a router that routes network request from the domian play.paularah.com to the Node.js server container.
- "traefik.http.routers.node-server.tls.certresolver=myresolver" tells the router to use the certificate resolver we created earlier.
- "traefik.http.middlewares.redirect-to-https.redirectscheme.scheme=https" creates a middleware to force-redirect HTTP network request to HTTPS.

The next three labels creates a router that matches every request to the host on port 80 and then uses the redirect to https middleware we created previously.

Restarting the containers now and voila! our Node.js server is now available from play.paularah.com, uses SSL and force redirects to HTTPS.

Summary

Traefik makes deploying HTTP servers with docker really easy. We can deploy multiple projects on the same host, by simply adding more services to our docker-compose file. One major advantage of this setup of is having all our configurations in one place and in a single command docker-compose up everything is up and running. This also makes our entire setup easily reproducible and allowing us move a project easily from one cloud service provider to another.

The source code for this blogpost can be found at https://github.com/paularah/node-traefik

Easy JavaScript Toolchain Management with Volta

Paul Arah — Tue, 20 Apr 2021 04:20:58 +0000

Summary: Use Volta for managing everything JavaScript. It's easy, flexible and convenient.

A common problem JavaScript developers face is managing multiple versions of the toolchain we use. Toolchain here refers to Node.js, NPM, TypeScript and everything in between you'd use for working with JavaScript.

I personally have had scenarios where I'd install the needed dependencies for a project and then discover the project isn't compatible with the version of my global installation of Node.js. I'd then have to delete the dependencies, flush my cache, install the right version of Node.js, then install the dependencies for the project all over again.

This begs the question, how do we seamlessly manage different versions of your toolchain across different projects?

Volta to the rescue! Volta is a hassle-Free JavaScript tool manager. Volta allows you easily switch between multiple versions of your toolchain depending on the project requirement. Your team can define a specific version of the toolchain for a project and this stays consistent for everyone making your environment easily reproducible.

To install volta on UNIX-based systems(macOS & Linux).

curl https://get.volta.sh | bash

Close your current shell session and open a new one for the changes Volta made to fully take place.

For Windows, download and run the Windows installer and follow the instructions.

To install Node.js & NPM

volta install node

Now we have Node.js globally available. You can specify the version of Node else it automatically defaults to the LTS version.

Managing your Toolchain Across Projects

To pin a specific version of Node.js to a project, run the command below in the root directory of the project.

volta pin node@12

Checking for the node version outside the project directory would give you the version of your global installation when you originally installed node with volta. Navigating into the project directory and checking the node version would give the node version specific for this project.

$cd home/some-js-project 
$volta pin node@12
$node -v #gives you node version 12 
$cd ../
node -v #gives you node version 14

Assuming your global installation of the typescript compiler is version 4.2 and your project uses version 3.2. Volta automatically detects manages this for you.

$npm i typescript -g
tsc --version #4.2
$cd some-js-project
$tsc --version #3.4

Under the hood, volta works by replacing your global node command with a shim that downloads the right version of your tools for you. Volta is written in rust and shipped as a single binary, thus it's very lightweight with a small overhead. There are also Github actions for it, making it easy to use in your CI. If you need to extend the functionality of Volta, volta has some advanced features like hooks that allow you do this.

Volta works out of the box with no stress. Now we can go back to doing more important things like frying eggs and cooking Jollof Rice.

Dependency Injection In JavaScript

Paul Arah — Fri, 26 Mar 2021 02:03:49 +0000

Writing code that is resilient in the face of changing requirements needs an intentional application of techniques that achieve this goal. In this article, we'll explore dependency injection as one of those techniques.
Take a look at the code snippet below.

const getData = async (url) => {
  const response = await fetch(url);
  const data = await response.json();
  return data;
};

This function retrieves a resource across the network using the Fetch API and returns it. While this works, from a clean and maintainable code perspective, there are quite a number of things that could go wrong here.

If our requirements change in the future and we decide to replace the Fetch API with say another HTTP client like Axios, we would have to modify the whole function to work with Axios.
The Fetch API is a global object in the browser and isn't available or might not work exactly as intended in an environment like Node.js where we would be running our test.
When testing we might not want to actually retrieve the resource from across the network, but there's currently no way to do that.

This is where dependency injection comes into play. At the core of it, dependency injection is giving the dependency(s) our code needs from the outside rather than allow our code to directly construct and resolve the dependencies as we have done in the example above. We pass in the dependencies our code needs as a parameter to the getData function.

const getData = async (fetch, url) => {
  const response = await fetch(url);
  const data = await response.json();
  return data;
};

(async => {
  const resourceData = await getData(window.fetch, "https://myresourcepath");
  //do something with resourceData
})()

The intent behind dependency injection is to achieve separation of concerns. This makes our code more modular, reusable, extensible and testable.

At the core of javascript are objects and prototypes, so we can do dependency injection the functional or object-oriented way. Functional programming features of javascript like higher-order functions and closures allow us implement dependency injection elegantly.

const fetchResource = (httpClient) => (url) =>
  httpClient(url)
    .then((data) => data.json)
    .catch((error) => console.log(error));

The fetchResource function takes an instance of our HTTP client and returns a function that accepts the URL parameter and makes the actual request for the resource.

import axios from "axios";

const httpClient = axios.create({
  baseURL: "https://mybasepath",
  method: "POST",
  headers: { "Access-Control-Allow-Origin": "*"}
});

const getData = fetchResource(httpClient);
getData("/resourcepath").then((response) => console.log(response.data));

We replaced the native fetch with Axios, and everything still works without meddling with the internal implementation. In this case, our code doesn't directly depend on any specific library or implementation. As we can easily substitute for another library.

The object(function in this case) receiving the dependency is often referred to as the client, and the object being injected is referred to as the service.

A service might require different configurations across the codebase. Since our client doesn't care about the internal implementation or configuration of a service, we can preconfigure a service differently as we've done above.

Dependency injection enables us to isolate our code(business logic) from changes in external components like libraries, frameworks, databases, ORMs, etc. With proper separation of concerns, testing becomes easy and straightforward. We can stub out the dependencies and test our code for multiple ideal and edge cases independent of external components.

In more complex use cases, usually bigger projects, doing dependency injection by hand is simply not scalable and introduces a whole new level of complexity. We can leverage the power of dependency injection containers to address this. Loosely speaking, dependency injection containers contain the dependencies and the logic to create these dependencies. You ask the container for a new instance of a service, it resolves the dependencies, constructs the object and returns it back.

There are a number of Javascript dependency injection container libraries out there. Some of my personal favourites are TypeDI and InversifyJS. Here is an example demonstrating basic usage of Typedi with JavaScript.

import { Container } from "typedi";

class ExampleClass {
  print() {
    console.log("I am alive!");
  }
}

/** Request an instance of ExampleClass from TypeDI. */
const classInstance = Container.get(ExampleClass);

/** We received an instance of ExampleClass and ready to work with it. */
classInstance.print();

The technique of dependency injection cuts across different programming languages. As a general rule of thumb, dependency injection can be done with languages that allow the passing of functions and objects as parameters. Some popular HTTP frameworks like NestJs and FastAPI come with an in-built dependency injection system.