DEV Community: Shabd Saran

Contextualising identifiers in a system

Shabd Saran — Wed, 02 Aug 2023 06:42:06 +0000

Introduction

At least two unique identifiers exist even in the most rudimentary authentication systems: user ID and session ID. It is hard to keep track of different identifiers in a system. These identifiers are logged to files, reported to Sentry, sent to the frontend, passed around to developers in Slack in bug reports, and, most importantly, shared with non-programmers in the team.

In no time, the user.Id is propagated as just 507f1f77bcf86cd799439011, which is then, for example, shared on Slack by customer support to report a bug. The developers then have to make sense out of this random ID and try to map it to an actual database record. The backtracking becomes a nuisance when the context is unclear.

The missing jigsaw piece

The identifier may propagate to a log file, Slack message, Sentry report, CMS dashboard or Google Analytics. The root cause of the problem is the loss of context from the identifier. The problem seems trivial if the system is not large enough. In systems with myriad unique identifiers, each representing an essential entity, issues related to backtracking or context retention can escalate rapidly.

But what if the context is attached to the identifier itself? That will solve the problem. The identifier may reach the end of the universe, but the relevant context will travel with it.

That is where identifier prefixing comes in.

Expounding the argument

Most modern systems have identifiers like TXN_507f1f77bcf86cd799439011. Here, the prefix “TX_” contextualises the identifier. For example, the given identifier can be guessed as a transaction ID without additional information.

The idea is to attach meaningful prefixes to critical identifiers. That will ensure these identifiers easily differentiate from the noise outside the system. The prefix is carrying relevant information about the identifier itself.

Getting down to business

Here is an example to demonstrate the implementation.

A program uses UUID strings to assign unique IDs to its database records. The program MUST respect the UUID formatting during retrieval and modification of the database documents. But it may ignore the formatting elsewhere.

In such a system, all the entity modules can implement toUUID and fromUUID functions. Role of these functions, implemented internally, will be to convert a (prefix)_ID to ID and vice-versa, respectively. These functions can translate prefixed identifiers to UUID formatted strings before the database query construction. Attachment of prefix to the returning value of identifiers from the entity modules can also be guaranteed.

The type of the IDs returned from the modules may only be asserted as a string (as opposed to a valid UUID). It is one of the caveats of the implementation: only the entity module can assert the UUID formatting. Everyone else may treat the ID as any other string.

The prefixed identifier may now be sent to Sentry or logged to AWS S3, but it will always carry the context of the identifier with it. Even the non-programmers will recognise a transaction ID from the CMS dashboard. What a relief!

Conclusion

The idea is simple enough to explain without code. It just need two functions to convert an ID to a prefixed ID and vice-versa. But the value addition is massive.

Identifier contextualising is not a new concept. People have been doing it for years now.

Python w/ strict typechecker

Shabd Saran — Fri, 23 Dec 2022 23:26:22 +0000

I’m embarrassed that I didn’t know Python has support for strict type-checking. It was tricky to embed type checking with Python2.x but that was changed with the introduction of Python3.x.

If you didn’t know about this either then it's going to blow your mind!

Setting up pretext

I will assume that you are already familiar with the concept of “typing” in Python3.x. I’d highly recommend you do some research on the topic before moving forward.

Here is a good starting point: https://docs.python.org/3/library/typing.html

Typing isn’t helpful… on it’s own

“typing” in Python3.x is a suggester rather than a strict type checker. For example, you will not run into any problems before * actually * executing the following piece of code:



# file: example.py

def add_numbers(n1: int, n2: str) -> float:
        return n1 + n2

add_numbers(True, {"something": "somethingElse"})

This is enough to make a grown man cry 🥲

You ideally want these types of errors to be raised * while * you are writing code instead of finding them in production logs.

Static type checkers

Google, Facebook and Microsoft have published their static type checkers for Python. The one I will cover here is mypy; it’s one of the most famous ones out there.

Fun fact: mypy has been around even before Python3.x. Programmers would add special comments to annotate types to their functions in Python2.x; similar to how types can be defined in JavaScript using JSDocs.

mypy will go through all the files (that you specify) and check the type definitions of all the functions, methods and variables. For example, you will get the following output when running the above example.py file through mypy:



mypy example.py

# --- Output ---
example.py:2: error: Unsupported operand types for + ("int" and "str")  [operator]
example.py:5: error: Argument 2 to "add_numbers" has incompatible type "Dict[str, str]"; expected "str"  [arg-type]
Found 2 errors in 1 file (checked 1 source file)

That’s what we’re talking about! 🎉

Type checking during development

This is great but what if one of my teammates forgot to use mypy before committing the code to production? That's a solvable problem.

Firstly, we need strict type-checking during the development process.; running a command after writing each function is inconvenient. You can create a mypy.ini file at the root directory of your project with the following content:



[mypy]
strict = true # run type checker through all the included files
exclude = .venv/

This will set global strict type-checking configurations for your project.

VSCode setup

Open the command palette using Command + P (or, Control + P for Windows/Linux users). Search for “Python: Select linter” and choose “mypy”.

Now, your VSCode will inform you of the errors while you are writing code. Here’s an example:

Enforcing type checking

A programmer will still be able to push type-unsafe code to production, even after all these configurations. Because remember, these restrictions are only set at mypy-level; Python3.x interpreter itself allows type unsafe code to be executed.

Our job as a code-maintainer is to ensure that all of our code is type-safe before committing a change, and ensure the type-safety at the remote repository-level as well i.e. GitHub/GitLab.

Pre-commit scripts

We want to run mypy before the code is committed to Git. This can easily be configured using pre-commit. It’s a Python package that can be used with any language to run Git-specific scripts.

We can use the following command to install pre-commit to our project:



pip install pre-commit && pip freeze > requirements.txt

Create a .pre-commit-config.yaml file at the root directory to run scripts before committing the code:



repos:
  - repo: local
    hooks:
      - id: typechecker
        name: typechecker # description to be displayed in STDOUT
        entry: mypy . --strict
        language: system
        always_run: true
        pass_filenames: false

This file is pretty self-explanatory so I don’t think there’s anything to add here.

Run the following command to install pre-commit to the project Git setup:



pre-commit install

You can now edit the .pre-commit-config.yaml file without having to run any more commands. I know that’s a relief 🤌

Here’s what you will see when you try to commit the example.py file we defined earlier:

The changes were not committed to Git. Pretty sweet, huh?

Quality control at remote-repository level

You still want all of these changes at the remote-repository level i.e. GitHub/GitLab. I will demonstrate the setup on GitHub; setup on GitLab wouldn’t be much different.

The aim is to run mypy when a commit is pushed to one of the “important” branches or when a pull request is created to be merged in one of these “important” branches. We can easily configure this using GitHub actions.



# file: .github/workflows/code-quality-checks.yaml

name: Code quality checks

on:
  push:
    branches:
      - main
      - production
  pull_request:
    branches:
      - main
      - production

jobs:
  build:
    runs-on: ubuntu-latest
    strategy:
      matrix:
        python-version: ["3.10.6"]
    steps:
      - uses: actions/checkout@v3
      - name: Set up Python ${{ matrix.python-version }}
        uses: actions/setup-python@v3
        with:
          python-version: ${{ matrix.python-version }}
      - name: Install dependencies
        run: pip install -r requirements.txt
      - name: Type checker
        run: mypy . --strict

You will be able to view the results of these actions on GitHub itself:

Conclusion

We made sure that our code is type-safe when we are committing it to VCS (Version Control System) i.e. Git, and we made sure that all the code is correctly merged at the remote-repository level i.e. GitHub.

That’s all to take care of. You can take it from here to build a Docker image or push it directly to a deployment service. You can be ABSOLUTELY SURE that you will not be surprised by any of the bugs on production unless something is wrong with the business logic itself.

Go on now, comrades! Write quality and type-safe code with Python3.x.

Drop a comment if you think I left something or if you have any suggestions for future posts.

Go migrate from-express

Shabd Saran — Sun, 30 Oct 2022 17:31:05 +0000

Background: why the switch?

NodeJS made backend programming simpler for programmers as it enables you to write services in JavaScript. While NodeJS is still relevant today, JavaScript as a programming language isn’t ideal to write. Microsoft helped the JavaScript community with the introduction of TypeScript but that didn’t solve the underlying problem: the code is still transpiled to JavaScript.

I can write an entire article on JavaScript and TypeScript. Drop a comment if you’d like me to do that. Moving on…

Source code

You can find all the code from this article here: GitHub

I highly suggest that you go through the code to better understand things. Use this article just for understanding the reasoning behind the decisions made in the repository.

Picking the right framework

The focus will be to write an HTTP server. ExpressJS is the most popular framework to build HTTP servers using JavaScript. The base assumption from here will be that you are already familiar with it, and are now trying to switch to Go. I am not going to focus on Go itself. The focus will be how to replicate the same ExpressJS structure in Go. Some familiarity with Go will be helpful.

Gin is one of the most popular frameworks in Golang to build HTTP servers. My general advice would be to go for the most popular framework and libraries when picking a language you’re not comfortable with yet. Once you have mastered the default configurations and setup then you can focus on optimization and picking the right framework for your use case.

Basic project setup and directory structure

A good project structure is always going to help you down the road as the project scales. Code management is dependent on the programming language itself. Here are a few must-read articles on Golang project structure standard conventions:

Standard Package Layout: https://www.gobeyond.dev/standard-package-layout
Golang standards - project layout: https://github.com/golang-standards/project-layout

Standard practices to structure Go code

Note that it is not a standard practice to put all the code inside the src directory in Go. The following directories are a must in almost all the projects:

`cmd`

This is the application's entrypoint. It’s the Go equivalent of index.js in JavaScript projects. You don’t want to put a lot of code here; just project-level structs/interfaces and basic initialization setup.

The cmd directory must not include any implementation logic. Note that the root directory is not allowed to have any external dependencies. It must only call/initiate the respective application packages.

`pkg`

This is where you define the public packages in your application. This is where you write the actual implementation logic. Packages inside the pkg directory are meant to be shared outside of your application; keep that in mind.

`internal`

This directory is similar to the pkg directory except it is only meant to be used by your application. This layout pattern is enforced by the Go compiler itself.

The internal directory may also have a pkg sub-directory to better segregate the code. The pkg sub-directory will contain packages that are shared amongst the internal packages.

`vendor`

This directory contains your application dependencies. It is an automatically managed directory so you don’t have to manually change anything here. All you need to do is execute go mod vendor to create and maintain this directory. The base assumption here is that you are using Go modules for managing dependencies.

Note that you will need to add -mod=vendor to the go build command for building the project.

`configs`

Your application configurations will go here. You are probably going to read most of the configurations from the environment variables. You may use this directory to extract configurations from the environment variables and store them in a struct.

Project structure example

cmd/
    __app_name__/
        init.go # entrypoint
pkg/
    __app_name__/
        users/
        auth/
internal/
    __app_name__/
        razorpay/
vendor/
    github.com/
    golang.org/
    google.golang.org/
configs/
    configs.go
build/
    deploy/
        Dockerfile
go.mod
go.sum

Route structure and middleware

The main.go file inside the cmd/__app_name__ directory will be simple. Its sole purpose will be to initiate the project and setup things in order.

package main

import (
    "fmt"
    "github.com/saranshabd/migrate-from-express/cmd/mfe/http"
    "os"

    "github.com/gin-gonic/gin"
    "github.com/saranshabd/migrate-from-express/cmd/mfe/logger"
    "github.com/saranshabd/migrate-from-express/configs"
)

func main() {

    // Create a Gin server application with default configurations & middlewares
    app := gin.Default()

    // Load all the application HTTP endpoints
    http.InitRoutes(app)

    // Start listening to HTTP requests from the client
    logger.App.Info().Msgf("Firing it up on %d port number.", configs.Configs.Port)
    if err := app.Run(fmt.Sprintf(":%d", configs.Configs.Port)); err != nil {
        logger.App.Error().Err(err).Msg("Could not fire up Gin :/")
        os.Exit(1) // Kill the application
    }
}

All the route handlers will go inside the internal/__app_name__/ directory, based on the directory structure discussed above.

internal/
    middlewares/
    __route_name__/
        routes.go
        handler1.go
        handler2.go

Firstly, we must ensure that all the HTTP responses we send are by default JSON. We can create a simple middleware for this purpose.

package middlewares

import "github.com/gin-gonic/gin"

func JSONResponse() gin.HandlerFunc {
    return func(c *gin.Context) {
        c.Writer.Header().Set("Content-Type", "application/json")
        c.Next()
    }
}

Secondly, we need a place to import all the HTTP route handlers and map them to specific endpoints. We can place this piece of code inside the cmd directory since all we will do is important a bunch of handlers and map them to their respective endpoints.

package http

import (
    "github.com/gin-gonic/gin"
    "github.com/saranshabd/migrate-from-express/internal/mfe/middlewares"
)

func InitRoutes(app *gin.Engine) {
    // Ensure that all the HTTP responses are by-default in the JSON format
    app.Use(middlewares.JSONResponse())

    // Load all the application HTTP routes
    // ...
}

That’s it! The router and middleware setup in Gin is similar to that in ExpressJS. We can call this InitRoutes function from the main.go file.

Extract request parameters from different sources

We can move on to creating a bunch of HTTP endpoints with the current project setup. The endpoints will primarily focus on extracting request parameters from different sources e.g. request body, query and URI. The request params will all be returned in the response object as is.

I have created a bunch of simple HTTP utility functions for better code readability. These functions don’t have anything to do with the tutorial so I won’t go into the detail. You can find all the source code on GitHub.

Go treats functions as first-class citizens so your HTTP setup is going to be just a bunch of functions. An HTTP handler in Gin takes a single argument: Gin context. Context is one of the fundamentals of Go. If you’re not already familiar with it then I’d suggest you take some time in understanding them before you continue.

Extract from request body

Go provides a beautiful “binding” package out of the box for basic validation. The binding module can be loosely compared with “Joi” package in NodeJS. The primary difference between the two is that binding is part of Go language itself so we don’t have to define our validation logic separately.

This is how a typical Go HTTP handler looks like:

type CreateExtractParamsArgs struct {
    Arg1 string `json:"arg1" binding:"required"`
    Arg2 int64  `json:"arg2" binding:"required"`
    Arg3 bool   `json:"arg3,omitempty"` // optional
}

func CreateExtractParams(c *gin.Context) {
    // Extract the parameters from the request.body & validate the object received
    var args CreateExtractParamsArgs
    if err := c.BindJSON(&args); err != nil {
        // Return HTTP 400 if invalid params are passed
        http.InvalidRequestParams(c)
        return
    }

    // Return the params object in response if everything is fine
    http.RespondOkay(c, "Request parameters extracted successfully", args)
}

Note that I am using the custom http package, which is just a utility package I created. You can refer to the RespondOkay function below:

func RespondOkay(c *gin.Context, message string, data any) {
    Respond(c, http.StatusOK, true, message, data)
}

func Respond(c *gin.Context, statusCode int, success bool, message string, data any) {
    c.JSON(statusCode, &gin.H{
        "success": success,
        "message": message,
        "data":    data,
    })
}

The omitempty property will remove the field while deserialising, to let’s say JSON. The struct value of the field will be the default value of the type. E.g. if you access args.Arg3 then you will receive false in the response. Beware of this behaviour. This is not JavaScript!

Extract from query parameters

We can use the same pattern to validate query parameters using the binding package. We will use the BindQuery function instead of BindJSON. Everything else will look the same.

type GetExtractParamsArgs struct {
    Arg1 string `form:"arg1" json:"arg1" binding:"required"`
    Arg2 int64  `form:"arg2" json:"arg2" binding:"required"`
    Arg3 bool   `form:"arg3" json:"arg3,omitempty"` // optional
}

func GetExtractParams(c *gin.Context) {
    // Extract the parameters from the request.query & validate the object received
    var args GetExtractParamsArgs
    if err := c.BindQuery(&args); err != nil {
        // Return HTTP 400 if invalid params are passed
        http.InvalidRequestParams(c)
        return
    }

    // Return the params object in response if everything if fine
    http.RespondOkay(c, "Request parameres extracted successfully", args)
}

Extract from the URI

Again, we can just extend the binding package to validate path parameters i.e. /path/:arg/. You define the path parameters in Gin like this:

r.GET("/:specificPath/", GetSpecificExtractParams)

We can swap the BindQuery function with BindUri and keep everything the same. By now, you must be getting the hang of Go.

type GetSpecificExtractParamsArgs struct {
    SpecificPath string `uri:"specificPath" json:"specificPath" binding:"required"`
}

func GetSpecificExtractParams(c *gin.Context) {
    // Extract the parameters from the request.params & validate the object received
    var args GetSpecificExtractParamsArgs
    if err := c.BindUri(&args); err != nil {
        // Return HTTP 400 if invalid params are passed
        http.InvalidRequestParams(c)
        return
    }

    // Return the params object in response if everything is fine
    http.RespondOkay(c, "Request parameters extracted successfully", args)
}

Header-based authorization

Authentication and authorization are the base of almost all the REST APIs out there. Auth is usually built on top of HTTP headers. We can use the binding package to validate headers received in the request and write middlewares to add custom logic on top of that.

Here’s a simple example of a auth header to check the source of the incoming request:

type VerifiedSourcesArgs struct {
    VerifiedSource string `header:"verified-source" binding:"required"`
}

const (
    AcceptedSource = "accepted-source"
)

func IsVerifiedSource() gin.HandlerFunc {
    return func(c *gin.Context) {
        // Extract the header value from the HTTP request
        var args VerifiedSourcesArgs
        if err := c.ShouldBindHeader(&args); err != nil {
            http.InvalidHeaders(c)
            return
        }

        // Check if the source of the incoming request is accepted
        if args.VerifiedSource != AcceptedSource {
            http.InvalidHeaders(c)
            return
        }

        // Continue with the chain of middlewares/handlers
        c.Next()
    }
}

The `Next` function in Gin middlewares

Gin middlewares look awfully similar to ExpressJS middlewares. But, there is a nasty distinction between the two; the Next function. You do not need to necessarily call the Next function in a Gin middleware.

This is an important distinction because you cannot expect a request to terminate just because you returned from the middleware without calling the Next function. The next in-chain function will still be called when your middleware terminates… unless you abort the request.

func AbortAndRespond(c *gin.Context, statusCode int, message string, data any) {
    c.AbortWithStatusJSON(statusCode, &gin.H{
        "success": false,
        "message": message,
        "data":    data,
    })
}

Instead of calling c.JSON we need to call c.AbortWithStatusJSON function to set the status code and JSON response before aborting the request. This is a nasty distinction especially for programmers coming from the NodeJS background. Beware!

That's it! We setup HTTP routes, validated incoming request data from different sources, and created auth middlewares for the application.

This is just the beginning. The application didn't interact with a database or a third-party service; both are fundamentals to most of the services. That's for the next article in the series :)

Drop a comment if there's anything I missed or could've explained better.

Cheers! 👋

Simple Cookies with Node.js and any frontend JavaScript framework

Shabd Saran — Sat, 26 Mar 2022 08:17:21 +0000

Fact: Elite programmers do simple things. It's the idiots who complicate things just to look smart.

Why another one?

We all have come across blogs to set up HTTP cookies with Node.js and React.js (or any other frontend framework) and let’s be honest it’s hard to find an article with all the information we need to set up our project.

Out there, you find a lot of ifs and buts in configurations so I’ll just tell you what you need to make it work - that’s it. I’ll keep this one short and to the point.

Agenda

Let’s start with what our setup will look like:

Separate configurations for development and production environments; that’s something missing from all the blogs out there.
Configure the frontend axios library to allow the backend to set cookies on the browser.
Configure the backend API to set a cookie with the right configurations; just the ones you need and care about.

Assumptions

I am assuming you will be hosting the frontend and the backend API on separate domains. Just to clarify, api.example.com and example.com are also counted as two separate domains.

Frontend configurations

On the frontend, I assume you will use a third-party library like axios to make requests to your server. I will advise you to create a new instance of Axios, with all the customised default configurations, and use this instance everywhere else in your project.

Create Axios instance with custom default configurations

The .create function allows you to set different configurations for different types of requests that you want to make to the backend. For example, you could have an Axios instance for making authenticated requests, and another one for making unauthenticated requests.

// utils/axios.js

import axios from "axios";

const axiosInstance = axios.create({
  baseURL: "http://api.example.com/api", // optional but recommended
  withCredentials: true, // to allow your API to set cookies on the browser
});

export default axiosInstance;

It’s just a better way of doing things 🤷

You’re done with frontend configurations. You can now use this newly created Axios instance to make requests to the backend like you normally would.

For example:

// utils/api.js

import axios from "utils/axios";

function fetchProducts() {
  return axios.get("/products/all/");
}

Notice, how you don’t have to set the base URL every single time now. You’re welcome :)

Backend configurations

You will just need to install a single library to your project - CORS.

Environment variables

Create an environment variable to store the frontend URL. If you are running the project in the development environment then you might set it to http://localhost:8000, or if you are running it in production then it might be https://example.com.

# .env

FRONTEND_URL=http://localhost:8000

You can use a third-party library like dotenv to load environment variables from the .env file.

Set up Express app

Configure CORS and default response headers to be able to set cookies on the frontend browser.

// index.js

import express from "express";
import cors from "cors";

async function main() {
  const app = express();
  // ...your Express app configurations

  // allow the frontend to make requests to your API
  app.use(cors({
    origin: process.env.FRONTEND_URL,
    credentials: true
  }));

  // set headers for you to be able to set cookies on the browser
  app.use((_, res, next) => {
    res.header("Access-Control-Allow-Origin", process.env.FRONTEND_URL);
    res.header("Access-Control-Allow-Headers", "*");
    res.header("Access-Control-Allow-Credentials", "true");
    next();
  });

  // ...your rest of the configurations

  app.listen(process.env.PORT, () => {
    console.log("App is up and running");
  });
}

main() // again, just a better way of doing things

And, that’s all you need to set up cookies on your backend project. You can now start setting cookies in your endpoint responses.

Set cookies with working configurations

You can use the given format to set cookies in both development and production environments, using the automatically set environment variable NODE_ENV.

// routes/auth.js

const isInDevelopment = process.env.NODE_ENV === "development";
const cookieConfigs = {
  httpOnly: true,
  sameSite: isInDevelopment ? false : "none",
  secure: isInDevelopment ? false : true,
  maxAge: 365 * 24 * 60 * 60 * 1000, // one year
};

router.post("/signIn/", (req, res) => {
  // ...your own login
  res.cookie("cookie-name", "cookie-value", cookieConfigs);
  res.status(204).send(); // read about HTTP status 204
});

Conclusion

That’s it! No more wandering from one Stackoverflow answer to another in search of workable cookies configurations with Node.js.

Do like the article if you found this helpful, and/or drop a comment if there’s anything I missed or anything you’d like me to cover in another article.

Signing off! 👋

How and why to use Docker as a package manager?

Shabd Saran — Mon, 09 Nov 2020 20:31:43 +0000

Introduction

Hello everyone, hope you're coding well.

Prerequisites

In the last article of this DevOps with Docker series, I explained how you can use Docker to make your life easy. I discussed how most of the developers are hesitant to try Docker and there are several reasons for that, so please check out that article if you've not read it already.

In that article, I also mentioned a few use cases of Docker. Now, I'm going to write individual articles for each of the use cases I mentioned there, explaining in-depth and providing code snippets so that even amateurs can pick up and use Docker in their daily development life.

Today's topic of discussion

In this article, I am going to explain how one can use Docker as a package manager for their development machine. Most of the developers use brew and apt as their package managers in macOS and Ubuntu respectively. Although there's nothing wrong with both of them, there might be scenarios where these guys will fail you.

What's wrong with `brew` and `apt`?

Imagine you're a freelancer who's using MongoDB v3.6 for personal projects. You just signed a new client and the client asks you to use the latest version of MongoDB, which is v4.4. Now, how are you going to manage two different versions of MongoDB on your development machine? I'm not saying there's no way of doing this, I'm just saying that there's an easier way, which is to use Docker.

Docker at your service!

Docker is a container management service, and containers by definition provide an isolated environment to whatever is running inside. Isn't that exactly what we need? We need two MongoDB services to run independently, without affecting each other.

You can simply create two Docker containers and run MongoDB v3.6 on one and MongoDB v4.4 on the other. Problem solved!

Solution Implementation

Docker comes with an amazing CLI for developer's to take advantage of its services and features. Docker commands generally tends to be very long and scary, but if you understand the meaning of each and every command and sub-command then it becomes a piece of cake.

Here's a command to run MongoDB v4.2.5 inside a Docker container:

docker run --rm -d \
  -p 27017:27017 \
  --env MONGO_INITDB_DATABASE=test_api_v4 \
  -v $(pwd)/volumes/mongo-4.2.5:/data/db \
  --name test_mongodb \
  mongo:4.2.5

Using this command, you can create a Docker container running MongoDB v4.2.5 on port 27017. So, you can run multiple such instances on your development machine, with each of them hosted on a separate port number.

Command breakdown

As I said, once you fully understand Docker commands, it will take no time for you to write one on your own (maybe for a different use case). I've broken down the above Docker command into meaningful chunks so that it will be easier for anyone to understand what is happening.

Create a Docker container

This part of the command is creating a new Docker container. The --rm flag tells Docker to delete the container once we stop it, and the -d flag tells Docker to run the container in detached mode (which means the container will not terminate once we close the terminal).

docker run --rm -d

Map port numbers

This part of the command maps host port number with container port number. The mapping format is (host-port):(container-port).

-p 27017:27017

Provide environment variables

This part of the command provides any environment variables (if needed) to the container. This is an ideal use case if you want to create a container to run database inside it because you would want to set user credentials and other required database properties at the time of its creation. Here, we are setting the name of the database MongoDB creates by default.

--env MONGO_INITDB_DATABASE=test_api_v4

Create docker volume

This part of the command creates Docker volume. Docker volume is nothing but a storage bridge between the host and the container. In this case, we would like to persist database files to our development machine (which is the host) so we create a Docker volume which maps our current directory to the directory in the container where MongoDB stores its database files.

-v $(pwd)/volumes/mongo-4.2.5:/data/db

Name the container

I think this is self-explanatory. By default, Docker assigns a random string to it, if you don't provide a name explicitly.

--name test_mongodb

Define Docker image

This part of the command defines what image you want to run inside the container. The string is of the format: (image-name):(image-version). By default, if you don't provide a version then Docker assumes it to be latest.

mongo:4.2.5

Support

How Docker can help you sleep well at night?

Shabd Saran — Tue, 27 Oct 2020 16:02:43 +0000

Introduction

Hello everyone, hope you're coding well.

I was very surprised to realize that a lot of junior developers, college students and freelancers were not using Docker and some of them were not even interested to learn it. Unbeknownst to them most of the senior developers, DevOps engineers, product managers and the whole software industry is functioning on this technology.

This article is dedicated to these people, to enlighten them a whole new world of containerization. In this article, we will only focus on Docker and how we can use it to increase our productivity and save our resources.

Terminologies

Before we dig deep into the use cases of Docker, let me brief you on a few terminologies:

Containerization

It is a concept of running software inside containers. The benefit you get by doing so is that all of your applications will run in isolation and if (for some reason) one of the applications crashes then all the other will still keep on running.

Container

A container basically provides an isolated environment to whatever is running inside it. As the name suggests, you can put whatever you want inside a container without worrying about the outer world.

Image

This is the piece of software (along with all of its configurations) which runs inside a container. An image interacts with the container, which in turn interacts with the outer world. This gives developers a benefit to not to worry about the environment in which their code might run. They only have to build the software and configure it to run inside a container.

Docker

At last, our beloved software. This is a container management service, which takes care of all the containers and their interactions with the outer world. For us, it is a magic box which we don't have to worry about.

What all can I do with Docker?

There are a lot of things one can use Docker for, and the best part is that developers can use their creativity along with Docker to increase their productivity. At the end, its just a container management service and it is up to you to use those containers for your own benefit.

People who are not familiar with Docker thinks that it has very specific use cases or that it's just too advanced for them to use. For them Docker is used to:

Either, build software in an environment close to production
Or, deploy code in an isolated environment using containers

Although both of these uses cases are valid, that is not what all Docker can do. Here are a few uses cases that I personally came across with and would like to share with all of you:

Install and manage multiple versions of the same software

You can use Docker to install software on your development machine, rather than using apt or brew. The main benefit you get here by using Docker is that you can easily manage this software and its configurations. For example, if you're installing MongoDB using Docker than you can run multiple MongoDB containers (one for each of your project) and that too on different ports (just in case one of them is already in use). You can install multiple versions of PostgreSQL on your development machine without those containers affecting each other. How cool is that?

Share development and staging infrastructure with the team

Share development infrastructure with your team easily using Docker Compose and GitHub. Imagine a new software developer joined your team, then you will have to share your development infrastructure (test database files, how to run your application on stage, etc). You can do all of that using Docker Compose and push all the files to GitHub, then this new developer will only have to clone this repository and read the README.md file to understand everything. Some of the startups use services on AWS to do the same task, but here you won't have to spend a dime.

Use Linux shell on Mac or Windows

Easily install and use Linux shell on your Mac or Windows machine. A lot of developers either dual boot their system or install some virtual machine software to use a Linux shell, whereas you can do the same thing using Docker as well. And since containers are lighter than virtual machines, you won't notice the difference between your system shell and the Linux shell running inside a container.

One disclaimer I would like to add here is that you will have to create appropriate Docker volumes so that you are saving the work on your computer.

Deploy code to production with just a single command

Pushing your Docker images to a container registry can be really beneficial because you will not have to move your code manually to the production instance (assuming you're working on a small project). With just a simple Docker command you can build and push your code to a container registry (as a Docker image) and then either create a pipeline to automatically deploy the pushed code or SSH into the production instance and simply pull the latest Docker images from the container registry.

This is something which I think could be very useful to freelancers because I noticed that most of the time they're busy either completing the projects, finding new clients or attending meetings with their existing clients so they don't have a lot of time pushing code to production or solving minor production issues which could've easily been avoided using Docker.

Conclusion

Notice that I've not mentioned what all features does Docker provide, instead, I've just mentioned some of the use cases where Docker has proven to be very effective. That is how I use Docker and your use cases might be different from mine, but the tool will remain the same.

Support

Please do leave a comment if you feel something could've been improved here. I can only continue writing such articles if I get support from you guys. Share it with your friends or colleagues if you feel like this could really help them, and do let me know if you want me to write an article explaining a particular topic or use-case from this one.

DEV Community: Shabd Saran

Contextualising identifiers in a system

Introduction

The missing jigsaw piece

Expounding the argument

Getting down to business

Conclusion

Python w/ strict typechecker

Setting up pretext

Typing isn’t helpful… on it’s own

Static type checkers

Type checking during development

VSCode setup

Enforcing type checking

Pre-commit scripts

Quality control at remote-repository level

Conclusion

Go migrate from-express

Background: why the switch?

Source code

Picking the right framework

Basic project setup and directory structure

Standard practices to structure Go code

cmd

pkg

internal

vendor

configs

Project structure example

Route structure and middleware

Extract request parameters from different sources

Extract from request body

Extract from query parameters

Extract from the URI

Header-based authorization

The Next function in Gin middlewares

Simple Cookies with Node.js and any frontend JavaScript framework

Why another one?

Agenda

Assumptions

Frontend configurations

Create Axios instance with custom default configurations

Backend configurations

Environment variables

Set up Express app

Set cookies with working configurations

Conclusion

How and why to use Docker as a package manager?

Introduction

Prerequisites

Today's topic of discussion

What's wrong with brew and apt?

Docker at your service!

Solution Implementation

Command breakdown

Create a Docker container

Map port numbers

Provide environment variables

Create docker volume

Name the container

Define Docker image

Support

How Docker can help you sleep well at night?

Introduction

Terminologies

Containerization

Container

Image

Docker

What all can I do with Docker?

Install and manage multiple versions of the same software

Share development and staging infrastructure with the team

Use Linux shell on Mac or Windows

Deploy code to production with just a single command

Conclusion

Support

`cmd`

`pkg`

`internal`

`vendor`

`configs`

The `Next` function in Gin middlewares

What's wrong with `brew` and `apt`?