DEV Community: Shubham Tiwary

deploying a express.js + react web-app on vercel

Shubham Tiwary — Fri, 14 Nov 2025 16:27:39 +0000

Services like Vercel and Render is pretty damn good for deploying your web-app especially if you have a low usage or for prototyping, and I recently had to get this done, so might as well document it here.

First, let's start with the express.js backend.

Starting with the express.js backend

Let's say you have a GitHub repo with backend/ folder having a npm package with your express app,
where the server.js is the main file to handle that, which should look like this:

import express from 'express'

const app = express()
//all your middlewares, routes, controllers, etc...


export default app

Instead of defining the port, just export the express app object, vercel handles running that.

Then define a vercel.json file to pinpoint the config that the server.js file is the main source (same dir as package.json in backend/):

{
  "version": 2,
  "builds": [
    {
      "src": "server.js",
      "use": "@vercel/node"
    }
  ],
  "routes": [
    {
      "src": "/(.*)",
      "dest": "server.js" 
    }
  ]
}

Although vercel by default looks for index(.ts/.js) or server(.ts/.js), its better to be explicit about it.

That's all the config you need, now just create a new project in your vercel account (connect with GitHub first):

Then:

There's three main config:

framework preset: choose express
build & output: leave default
env: pass your .env file details, that can be access in app as process.env.<ENV>

And then test your application works!

React.js frontend deployment

There's relatively little config needed for react, I'm using vite here, but it's the for create react app

Back on the vercel new project:

Use the framework preset as react app/vite, it will handle the build output by itself (vite build or react build accordingly)

Now for env, it's usually best practice to have the API endpoint address to backend in a env, so let's assume you have API_URL as the API path to backend.

From the express deployment earlier, get the address:

And then pass inside "Environment Variables" the API_URL while making the frontend deployment.

Now test your apps and see if they both work well.. (mention below if they don't)

Some good practices to just have for this:

use CORS(cross origin) blocking in your express server (in deployment mode), and whitelist apps that need access like your react app, so user can't direct make API calls. Maybe have a "MODE" env value for this logic handling.
like mentioned, decouple variables like configs, addresses, ports, etc into envs and pass em' later
use typescript instead, will save you a lot of headache on issue you'll see in prod but not in dev, better to get way more possible errors than way less.

Running a lightweight Kubernetes (K8s) cluster locally with K3s

Shubham Tiwary — Mon, 27 Oct 2025 17:18:48 +0000

NOTE: This articles expects you already know bits about K8s and wanna setup local cluster with 1 or more node, but even if you aren't familiar with K8s, you can still follow along..

Context on K3s

Kubernetes is amazing to run with managed services like AWS EKS or GCP GKE in production, but making changes in production cluster directly might not be a good idea since:

Testing in a dev environment is much faster and better practice
When learning K8s initially, it's better to avoid unnecessary cloud bills

So that's where K3s comes in, it can let you run local K8s clusters with >=1 node and give you consistent production like environment!
There's also other tools like minikube, kinD that you'll find in the offical K8s tool guide
but I'd still recommend this since:

kinD runs on your system's docker's daemon, not with K8s containerd runtime, it's better to have that seperate and not mixup your docker containers here.
minikube has experimental support for multi node cluster

So how is K3s different from K8s
I built a project here recently to deploy to GKE and tested with K3s locally first, so here's what I can say:

KubeAPI metrics for K8s are all supported.
All objects like pods,namespaces,service, everything is the same!
Only difference is etcd store --> uses sqlite instead (which can also be configured to etcd, MySQL, etc)

It's at the end of the day, a K8s binary for running even in low end devices like ARM based Pi, or normal x86.64 systems!

Setup K3s in single node

K3s already installs dependencies like kubectl so you can follow along even if you may or may not already have that

Only if you have firewall rules enabled on local machine
To setup, we'll first need to allow some firewall rules to allow the socket for Kubernetes API server.

I'm assuming you have ufw or some other firewall tool, so here are the rules you need to enable:

6443/tcp                   ALLOW       Anywhere                   # K8s API server
10250/tcp                  ALLOW       Anywhere                   # K8s worker node kubelet metrics

Here's the install script for K3s from the official k3s quick start guide:

curl -sfL https://get.k3s.io | sh -

That's it, you should see a k3s daemon service running if you check with:

systemctl status k3s

What you should see (or something similar):

● k3s.service - Lightweight Kubernetes
     Loaded: loaded (/etc/systemd/system/k3s.service; enabled; preset: disabled)
     Active: active (running) since Mon 2025-10-27 21:43:25 IST; 32min ago
 Invocation: bb923ad8337d473da638169bbfe6867f
       Docs: https://k3s.io
    Process: 309939 ExecStartPre=/sbin/modprobe br_netfilter (code=exited, status=0/SUCCESS)
    Process: 309940 ExecStartPre=/sbin/modprobe overlay (code=exited, status=0/SUCCESS)
   Main PID: 309942 (k3s-server)

K3s also creates a kube-config file in /etc/rancher/k3s/k3s.yaml. To use this cluster you have to make this path as KUBECONFIG:

sudo chown $USER:$USER /etc/rancher/k3s/k3s.yaml  #(if user doesn't have permission)

export KUBECONFIG=/etc/rancher/k3s/k3s.yaml

Then check if it is working:

kubectl get ns

kubectl cluster-info

#Expected log:
#Kubernetes control plane is running at https://127.0.0.1:6443
#CoreDNS is running at https://127.0.0.1:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
#Metrics-server is running at https://127.0.0.1:6443/api/v1/namespaces/kube-system/services/https:metrics-server:https/proxy

Testing with running a sample deployment

Run a sample nginx deployment with 3 replicas:

kubectl create deployment nginx --image=nginx --replicas=3

kubectl get deploy
#expected logs:
#NAME    READY   UP-TO-DATE   AVAILABLE   AGE
#nginx   3/3     3            3           16s

Then expose the service with a ClusterIP:

kubectl expose deployment nginx --port=80 --target-port=80 --type=ClusterIP

kubectl get svc

#expected
#NAME         TYPE        CLUSTER-IP   EXTERNAL-IP   PORT(S)   AGE
#kubernetes   ClusterIP   10.43.0.1    <none>        443/TCP   41m
#nginx        ClusterIP   10.43.2.40   <none>        80/TCP    93s

Here, the clusterIP is 10.43.2.40 (could be different in yours).

Let's try calling HTTP with curl onto that port 80:

curl 10.43.2.40

#<!DOCTYPE html>
#<html>
#<head>
#<title>Welcome to nginx!</title>
#<style>
#html { color-scheme: light dark; }
#body { width: 35em; margin: 0 auto; font-family: Tahoma, Verdana, Arial, sans-serif; }
#</style>
#</head>
#<body>
#<h1>Welcome to nginx!</h1>
#<p>If you see this page, the nginx web server is successfully installed and working. Further configuration is required.</p>

#<p>For online documentation and support please refer to
#<a href="http://nginx.org/">nginx.org</a>.<br/>
#Commercial support is available at
#<a href="http://nginx.com/">nginx.com</a>.</p>

#<p><em>Thank you for using nginx.</em></p>
#</body>
#</html>

Setup more nodes to connect to cluster (optional) - agent node

So for this, setup any other machine you may have - VMs, physical machine that share a network interface.

For example, let's say I have another machine connected in same wifi in wlan0 interface,

And I get this machine's private IP as 10.158.66.194/24 when I check with:

ip -4 addr show

K3s exposes a "node token" to connect to clusters, and needs the URL where the host node is, so:

export K3S_URL="https://10.158.66.194:6443"
export K3S_TOKEN=$(sudo cat /var/lib/rancher/k3s/server/node-token)

On the other machine you run for connecting:

curl -sfL https://get.k3s.io | K3S_URL=https://10.158.66.194:6443 K3S_TOKEN=mynodetoken sh -

NOTE: 10.158.66.194 is just a dummy IP for example, check and use what was there for your interface.

Then check the nodes with:

k8s get nodes

While in terms of setup, yes kinD and minikube might be easier to configure initially, but once this setup is done, its rather easy to manage and relieves you of all the headaches you may have of "it works locally but not in production".

If you liked this, give feedback since I write a lot of stuff on my obsidian that I translate and write here, so I can go on more about K8s, devops and other stuff..

Shrinking docker image size over 100x in just 2 steps

Shubham Tiwary — Mon, 02 Jun 2025 17:47:29 +0000

Smaller image size helps you:

make faster builds on your CI
testing containers becomes faster
less cost for build jobs / container space

. . .

Here's some quick hacks I found after playing around for a while with docker images

In this example, I'm shrinking an image of about ~900 MB to around ~8 MB (as tested on my docker)

For reference, I'll take this simple sample Dockerfile:

FROM golang
WORKDIR /app
COPY main.go .
RUN go build -v -o server
ENTRYPOINT [ "/app/server" ]

Which simply builds a go program, and runs it.

This sizes around 946 MB after building

1 - Base Image as Alpine

Alpine is a very small Linux distro used for containers

So most images you want to run, will have an alpine base equivalent of them. Where to find them?

For example, for the golang image from dockerhub:

You can choose the tags here to choose an alpine image.

So the Dockerfile with alpine base will be:

FROM golang:1.24-alpine  #just any alpine tag from the tag list
WORKDIR /app
COPY go.mod .
COPY main.go .
RUN go build -v -o server -buildvcs=false 
ENTRYPOINT [ "/app/server" ]

With this, the image build takes around 353MB size (3x smaller already)

2 - Multi Stage builds

I like to think of this like a:

multistage rocket, where there's multiple sections and after certain intervals, the sections push the rest, cutting itself off, one by one, until there's one final portion left.

Similarly, suppose in our go-program,

we need the go compiler to build it first of all
once its build, only the executable is needed (compiler or dependencies not needed during runtime may be dropped off)

Let's look at this with a Dockerfile:


# Stage 1: build
FROM golang:1.24-alpine AS builder-stage
WORKDIR /app
COPY go.mod .
COPY main.go .
RUN go build -v -o server -buildvcs=false 


# Stage-2: run
FROM scratch
WORKDIR /app
COPY --from=builder-stage /app/server /app
ENTRYPOINT [ "/app/server" ]

In stage-1 -> we build the "server" executable and it will be placed at /app/server (call this "builder" stage
Then in 2nd stage -> take that executable from builder stage and simply run it.

In the 2nd stage, a base image called "scratch" is used, it is even more lightweight than alpine. But only used for cases like this, where there's literally no dependencies needed, just to run.

After this build, only the 2nd stage remains as final image.

Once we build this, the image size shrinks to 8.15MB! (almost 300x)

Limitations

This seems too good to be true, and it is yeah. Not for all cases can we optimize up to this extent.

Some cases, we simply need all the dependencies in run time as in compile time too (ex: interpreted languages like Python - although there's build tools for that, but limited), so it's tricky to use multi stage builds.

Repositories like Dockerhub also have compression on their end to optimize the size during deployments.

Even in the case of base image as alpine, some tools might not have great support in alpine

Trying it yourself

To replicate the following on your own,

Here's command to build the Dockerfile:

docker build -t <image-name:tag> .

And the go program I used for this (a very simple HTTP server):

package main

import (
    "fmt"
    "log"
    "net/http"
)

func greet(w http.ResponseWriter, req *http.Request) {
    fmt.Fprintf(w, "hello world\n")
}

func main() {
    http.HandleFunc("/", greet)

    fmt.Printf("Starting server at port 8090\n")
    if err := http.ListenAndServe(":8090", nil); err != nil {
        log.Fatal(err)
    }
}

Deploying a Web service with GCP compute & Terraform

Shubham Tiwary — Thu, 29 May 2025 20:44:49 +0000

Let's start off with something simple:
GCP Compute is an on-cloud VM to run everything like your own machine.

We'll deploy a simple http web server container and make it accessible from anywhere.

Pre-Requisites:

GCP account (with Compute API enabled + get your projectID)
Gcloud CLI
Terraform

Provision a GCP Compute Virtual Machine

We'll use terraform to quickly set this up. Terraform configs are written with "*.tf"
It contains information such as what resources (services) to run and what provider (service provider) to use.

First, make a "config.tf" file (or whatever you want, doesn't matter), and write these:

Define GCP to be the provider:

terraform {
  required_providers {
    google = {
      source  = "hashicorp/google"
      version = "6.8.0"
    }
  }
}

Terraform acts as a middleman, where its: us <-> terraform <-> aws/gcp/etc

So we ask it to use the terraform provider SDK for GCP services to be setup, this is the terraform block

Then pass some configuration for the GCP project:

provider "google" {
  project = "<your-gcp-project-id>" 
  region  = "asia-south2"   #delhi
  zone    = "asia-south2-b"
}

I'm using my project ID and closer region, use whatever is closer to you. You can check regions and zones for GCP here

Then make another "main.tf" file in same dir (any name works just like before although).

Now, this is the main thing, the resource block, every service is defined in this order:

resource "google_compute_instance" "vm-instance" {
  name         = "vm-instance-tf"
  machine_type = "f1-micro"
  zone         = "asia-south2-b"

  boot_disk {
    initialize_params {
      image = "ubuntu-os-cloud/ubuntu-2204-lts"
    }
  } 

  tags = ["allow-web-ssh"]
}

This defines a resource block for making a f1-micro (the smallest VM) with ubuntu-22 OS

How to actually format these reources blocks in terraform can be referenced from here

The format for a resource block is:

resource "<type>" "<name-for-terraform>" {
   name = "<name-for-gcp>"
}

Just don't get confused between the names here (will come handy later)

Now, VMs are like our local machines, the only difference being the Network configuration we have to setup to make it accessible. So let's do that:

resource "google_compute_network" "vpc_network" {
  name = "terraform-network"
}

resource "google_compute_firewall" "vm-firewall" {
  name    = "vm-firewall"
  network = google_compute_network.vpc_network.name

  allow {
    protocol = "tcp"
    ports    = ["22", "80", "443", "8080"]
  }

  #access from anywhere (dev mode)
  source_ranges = ["0.0.0.0/0"]
  target_tags   = ["allow-web-ssh"]
}

Define a network firewall resource (just like compute), similarly, any roles, properties to be added are also made into individual resources.

And add this to update the VM resource block for access in these ports:

resource "google_compute_instance" "vm-instance" {
  name         = "vm-instance-tf"
  machine_type = "f1-micro"
  zone         = "asia-south2-b"

  boot_disk {
    initialize_params {
      image = "ubuntu-os-cloud/ubuntu-2204-lts"
    }
  }

  network_interface {
    network       = google_compute_network.vpc_network.name
    access_config {}
  }

  tags = ["allow-web-ssh"]
}

In Terraform, to provision, we have the init -> plan -> apply process.

Init: setup dependencies, providers, etc (run first time)
Plan: make changes (after any change in config)
Apply: apply those changes planned.

Now, regardless of what filenames you have, as long as they end in "*.tf", it'll be counted in this.

So perform these commands:

terraform init

terraform plan  #check and confirm the changes

terraform apply  #see whether it passes

Setup Web Server on this GCP VM

After setup is successful, ssh into the machine with gcloud (will work after doing gcloud auth, if failing, do that first):

gcloud compute ssh vm-instance-tf --zone=<your-zone>

Then, quickly setup Docker (go checkout my series on docker for more detail) or by following the guide in this link

And run this custom container for testing a web app:

docker run -d --name test-go-web-app -p 8080:8080 sardinesszsz/go-hello:v2

Now the web app should run, test it with a HTTP GET request:

curl -X GET http://localhost:8080/  #returns Hello-World!

Each VM has a ephemeral IP which is public for access, since we made port 8080 accessible, we can use this IP for HTTP calls on port 8080 to call this web server.

Get the public IP of this VM with gcloud again:

gcloud compute instances describe vm-instance-tf --zone=<your-zone> --format='get(networkInterfaces[0].accessConfigs[0].natIP)'

Suppose you got 10.10.10.10 as the IP, then make a call on this IP:

curl -X GET http://10.10.10.10:8080/

That should successfully retrieve the response, if not follow this through, and try seeing what the issue could be or doubts in the comments are welcome!

So with Terraform, you didn't even have to go to GCP cloud console! Everything is done here in terminal with very less friction once one gets used to it.

This was a lot to take it (if you're new to Terraform or GCP), but slowly, I'll try to cover more so it can sink in well.

P.S: for destroying the service, make sure to go into the dir with the terraform files and run:

terraform destroy

5 developer tools to really step up your workflow

Shubham Tiwary — Thu, 15 May 2025 19:45:01 +0000

Here's some few developer tools I've used quite a lot these past few months that has made my workflow way more fast & smooth.

Lazygit

Most people choose the git cli over GitHub Desktop.
But this is a TUI (Terminal UI), so you can easily add changes, make commits, deal with branches, revert & rollbacks, etc.

And if you have a server running Git (server with TTY - no GUI), you can easily set lazygit up and use it on the go.

Checkout lazygit github for some more idea

LazyDocker

Similar to lazygit, a TUI manager for docker.
This is my favourite TUI so far, I have it open in my other workspace most of the time.
.
Checking which containers are running, images available, logs, metrics, volumes, etc, there's everything.

Vim Keybindings / Neovim

After using VScode for years now, I tried using Neovim.
Since I was really familiar with VScode's environment, I did revert back to that.
.
But one thing that I took back with me was Vim keybindings, its amazing!

If you've heard about Vim, it is notorious for being overly complicated for no reason.
That's what I thought too, until I just took it slowly, first learning basics like only operations like: write, quit, copy, paste.
Then moving onto things like macros.
.
Once you get the hang of it, its really fun to navigate and play around with.

If you wanna test out vim, would recommend Vim Tutor tool

There's extensions in Vscode and other IDEs for vim support, or you can go with Neovim (which provides IDE features like linters, syntax highlight, LSP, etc)

Obsidian

While this is not technically a developer tool, anybody can use it.

After jumping from one note to google docs to Notion, I settled into obsidian.

Its free, has amazing plugin support, and you get this graph of all the notes you have, that's really cool to watch.

If you're familiar with Notion's syntax of markdown or GitHub readme docs, its a good fit for you. (markdown is really easy even in case you don't know, pretty easy to learn)

Tmux

Tmux is a "Terminal Multiplexer", so suppose you have ssh-ed into a machine, and want to run two foreground process.

How do you do that, both in the same terminal?
So you gotta make another terminal with ssh, but instead with tmux, you can make a "pseudo terminal".

That's just one example.
But in general, if you need >1 terminal, just run a tmux command to split it up!

Then switch between these terminals with keybindings, which you can customize.

Which one of these do you already use? or suggest more under-rated tools in your arsenal.

GCP + Terraform: Intro

Shubham Tiwary — Tue, 13 May 2025 18:24:54 +0000

Google Cloud Platform (GCP) provides tons of services like storage, compute, databases, etc.. and it's a mess to go through all of them.

Terraform is a IAC (infrastructure as code) tool, so you write config (similar to Dockerfile for example), and define what services you need, so it creates that service on your account.

Using them together is a superpower, Terraform works with major providers like Azure, AWS & GCP so anything works.

So let's start with understanding some basic services, later I'll show how to use them in detail, but fundamentals are important.

There's hundreds of services, but I'll go through only the main ones, after that the rest is similar and easy to get used to when needed.

GCP Services

1. Compute Engine

Every server, client or any type of machine is a computer. This is the most basic thing you can get, a machine you get access to, where you can run any type of service, just like your local machine.

There's different family such as:

E2 – cost-effective, good for general workloads.
N2 / N2D – better performance than E2, with N2D offering AMD CPUs.
N1 – older generation, still widely used.
C3 – Newest, highest performance

You don't really need to remember this by the way, just an overview

Depending on your task, you can choose the physical requirements, the memory, CPU cores, disk, etc.. you're in full control.

2. Storage Bucket

Think of it as Google Drive for your backend. Used to store any kind of static assets – images, videos, logs, backups, etc. Fully managed, cheap, and scalable.

You can set permissions, make files public, and connect it to a CDN.

Great for serving assets or storing backups/logs.

3. Cloud Run -> serverless execution

Let’s you run containers without managing any servers. You just give it a Docker image, and it handles the rest – scaling, routing, and more.

Pay-per-use. No traffic? No cost.

Good for APIs, webhooks, or any small backend service.

4.. Firestore -> NoSQL DB

A scalable NoSQL document database. Stores data in collections of documents (like JSON).

Best for quick-moving apps where you don’t need SQL.

What Terraform does?

Initially, you have the cloud console, but using the web console when a lot of services comes to be, its hard to manage.

With terraform you have three steps only:

Init: Install the provider support (in our case, GCP)
Plan: Similar to commit in git, mark changes.
Apply: pretty self-explanatory, no?

Simply by writing config such as:

resource "google_compute_instance" "vm-instance" {
    name = "vm-instance-tf"
    machine_type = "f1-micro"
    zone = "asia-south2-a"

    boot_disk {
      initialize_params {
        image = "ubuntu-os-cloud/ubuntu-2204-lts"  
      }
    }

    network_interface {
      network = google_compute_network.vpc_network.name
      access_config {}
    }

    tags = ["allow-web-ssh"]
}

and using this, you can just create a service easily (this is for a compute engine).
Updating, deleting, making changes also becomes very easy.

For now, let's do setup:

Install terraform tool for your system from here: Install
Sign-up on GCP console - you can choose to have a 90 days trial where you get $300 credits free.

This was to just get familiar with what is what, I'll show a hands on with compute engine + terraform in the next one!

Dockerize mutliple containers with compose

Shubham Tiwary — Fri, 09 May 2025 20:00:38 +0000

Use case - what it solves

So If you're familiar with docker containers so far (if not, read my previous two articles on this),

When there's more than 1 container, and they have to interact, there's some things you have to repeat, like:

pulling image & creating a container.
allocate volumes, pass ENV args, etc to container
have all containers running in same network

Now if there's 3,4,5,.. or more, it gets more hectic, see where this is going? Docker compose solves that, by "composing" up your containers config in a file type called "yaml" (its similar to json - a data format, usually for configs).

You still have things like Dockerfile and concepts of containers are the same, its just the process that's simpler.

Let's compose it up

A quick example of 3 containers rolling up using docker compose.

So here's what we wanna setup:

A node js server that writes random key, values to Redis DB
Then we check the changes from Redis Insight.

Let's start by writing code for this node.js server:

Initialize a npm package:

npm init -y
npm install express redis

Code for node.js server:

const express = require("express");
const redis = require("redis");

const app = express();
const client = redis.createClient({ url: process.env.REDIS_URL });
client.connect();

app.get("/", async (req, res) => {
  const randomKey = `key_${Math.floor(Math.random() * 1000)}`;
  const randomValue = `value_${Math.floor(Math.random() * 1000)}`;

  await client.set(randomKey, randomValue);

  res.send(`stored random K-V pair: ${randomKey} = ${randomValue}`);
});

app.listen(3000, () => console.log("Server running on port 3000"));

Then writing up a Dockerfile to make this into a container:

FROM node:18
WORKDIR /app
COPY . .
RUN npm install
CMD ["node", "server.js"]

Now finally, make a "docker-compose.yaml" file:

version: "3.8"

services:
  api:
    build: .
    ports:
      - "3000:3000"
    depends_on:
      - redis
    environment:
      REDIS_URL: redis://redis:6379

  redis:
    image: redis:latest
    ports:
      - "6379:6379"

  redisinsight:
    image: redis/redisinsight:latest
    ports:
      - "5540:5540"
    restart: unless-stopped

Read through this, you'll notice that you still have the same concepts, such as a Dockerfile, ports and environment.

And what about Network between these containers? Since we had to do that manually earlier, docker-compose puts all of these in a same network, so you don't have to deal with that.

The way one container refers to another is still the same, for example, redis-insight will refer to redis contaner as redis://6379

Then, run this with the command:

docker compose up -d  #d means run in background (detached mode)

Now let's try testing this:

First, make call to http://localhost:3000/ (node.js):

curl http://localhost:3000 #via curl
wget http://localhost:3000 #via wget

Or just use your browser

Then check in redis-insight:

Make connection to the redis instance:

See the random key-value pairs being added:

For more reference on the docker-compose yaml config, check this

Try making more containers with docker compose, to get some hands on experience, you can try making:

A CRUD App: make a node.js + react app with some DB(PG, Mysql, Redis)
A Db monitor: PG DB, use PG exporter for prometheus, with prometheus & grafana. So PG -> exporter -> prometheus -> grafana (and some server to make writes to PG)

Anyways, that's about it.
Maybe I'll cover more things with docker, with some examples, or talk about docker swarm in the next one!

2 years of using Linux: why you may consider the switch

Shubham Tiwary — Thu, 08 May 2025 21:21:19 +0000

As a 2003 kid, I loved playing World of Warcraft, Age of Empires, PVZ and all the old school games in Windows.

Back then, gaming in linux was.. not a good experience. 20+ years into the future, companies like Steam are actively advocating for Stream Deck (which uses Arch Linux), over 90% of servers & embedded systems run linux of some sort.

Here, I want to convince you why you may want to consider switching to linux even as a tech or non tech person.

Gaming experience

Okay let's get this out of the way first. Linux gaming isn't perfect.
The main reason? driver support for Nvidia based GPUs aren't the best, while for AMD that's not really an issue.

However, if you're a gamer and would like to give linux a shot, I'd recommend going with linux mint

Dev experience

This is a middle ground, I've never had any tools that worked in windows that didn't on linux here.
I mainly use:

VScode/Vim
Docker
VirtualBox

for tools like Docker & VBox -- you usually need WSL (for context: its like a linux emulator for windows)

I hate that setup, its way easier to setup tools like this on linux.
And with things like work-spaces (windows also has this) easily accessible by keybindings (like Ctrl + Alt + left/right) to have multiple windows!

Customization

This is the best part! You can literally change everything, on your own need.

This is my setup (Arch Linux + Hyprland):

There's key bindings like:

Super + R: app drawer
Super + num: workspace tab
Super + Shift + L: lock screen

You can add key bindings (3rd party) and customize (limited) on windows, but linux takes it to the next level.

Not sold? Here's some other "ricing" setup (ref: r/unixporn subreddit):

Downsides

Its not all sunshine in the penguin kingdom. Some problems you'll find are:

Mainly Designers: tools like adobe suite, or windows office suite won't work.
Nvidia GPUs support: even though support has improved, there might still be some issues in some softwares.
Learning curve: most people see OS as a tool to work, not play around with, and that's fine, It also may be more difficult to transition coming from a non-tech or depending on distro you choose.

You should switch if

If you fall into these categories, I'd recommend first dual booting a shot, or run on Virtual Box accordingly:

for devs: ubuntu, debian, nixos (mostly used in servers)
for gamers: mint, popOS.
for hackers: kali, parrot
for customization: any (depends more on Window Manager than distro

There's a lot more I can cover like:

networking linux machines to setup a home-lab
ricing (customizing) linux guide
linux commands for getting started.

Do share what more I should get more into!

Everything to get started with Docker! - I've compiled this into a 2 part series, so check it out..

Shubham Tiwary — Fri, 07 Mar 2025 20:04:09 +0000

Use Docker, No more setup Hell! (Part-1)

Shubham Tiwary ・ Mar 7

#docker #devops #programming #learning

Docker continued - Volumes & Network (part-2)

Shubham Tiwary — Fri, 07 Mar 2025 20:02:36 +0000

In case you haven't followed up, first go through the Docker Part-1, this is a continued series from that previous iteration..

Volumes in Docker

Previously, we see that docker containers are "stateless", they can be exposed to outside world, but don't have state persisting, or interaction between containers.

With volumes, you have a directory bind between one in docker container to your system.
So you can do things like:

make code changes in your machine /dir and it appears on the container where that's bind to. (no need to re-build image+container on code changes)
delete container and the data in your system still exists.

Assigning volumes to our container

Back to our express image from previous part, we attach a volume

docker container run -d --name express-app-vol -v /home/dev/work/tests/demo:/app -p 3000:3000 express-dummy:v1

I have my contents of express app in /home/dev/work/tests/demo, which i attach to the /app in container (yours might be different)

Now let's make a simple change in our code (/home/dev/work/tests/demo/index.js in my case) :

const express = require('express')
const app = express();
require('dotenv').config()

const port = 3000;
const num = process.env.num;

app.get('/', (req,res)=>{
  res.send(`Hello ${num}!!!!`);  ## <-- added some "!!!!" at end
})

app.listen(port, ()=>{
  console.log(`Server connected at port: ${port}`)
})

With these code changes, lets stop & restart container:

docker stop express-app-vol
docker start express-app-vol

And response should be something like:

Container Networks.

Suppose there's two containers:

Redis container running on port 6379.
Redis-Insight container running on port 5540 (connect to redis & test on a GUI in browser)

Run them as follows:

docker run --name redis-container -p 6379:6379 -d redis
docker run -d --name redisinsight -p 5540:5540 redis/redisinsight:latest

With this setup, let's try testing redis from redis-insight:

So this is redis insight, and it seems to not find any redis DB on port 6379, why is that?
Since the network environment is different for these containers.

Try running the following command:

docker network ls

It lists down all network present, you'll see these by default:

bridge: default network driver.
host: between container and the docker host.
none: completely isolated

Checking the containers inside the bridge network (default):

docker network inspect bridge

Go down to the "containers" in the response:

you'll notice the redis & redis-insight container, they can talk with your machine but not with each other!

Our Custom Network Group.

Let's create our own custom network (we'll place our containers here afterwards):

#network called "redis-line"
docker network create redis-line

Then, let's add the two containers here:

#add redis-container & redisinsight (the containers) into redis-line network
docker network connect redis-line redis-container
docker network connect redis-line redisinsight

Now, let's inspect the network with:

docker network inspect redis-line

You'll notice the containers added inside the "Containers" parameter.

Now see the "Name", i.e: "redisinsight" & "redis-container". So now when you have to refer to other container, instead of putting an ip, you use these names, example:

So here in the "host", we put "redis-container" which is like its domain in that network.

And from that, it is connected and works!

Just reading through these isn't enough, what might work here might not for you, so its best you try these on your own..

Use Docker, No more setup Hell! (Part-1)

Shubham Tiwary — Fri, 07 Mar 2025 07:53:55 +0000

A case scenario - setup hell

This are the docker containers I'm currently running on my PC.
Earlier, I'd have to setup manually which'd look something like:

Go to the official site, get hit with the register, verify account & login (forgot you password? another mess to recover that)
After that, turns out it doesn't work: go to reddit, find some solution from 7 years ago, a 50/50 hit or miss it does work (bless those guys though)

Another case scenario - "But It works on my machine"

You got a windows PC, your friend got a linux/mac, and you wanna deploy to AWS, so there's something like AMI.
Code is simple, use git. Wanna share & deploy your local dependencies? good luck testing on each platform & writing README files for "how to run" (realize nobody actually reads that)

Creating a container

First, let's see how easy it is to create a container

Let's go to Docker Hub, a repository for docker "images". Think of images like blueprints to make a "container".

We'll try to setup a simple nginx container:

There's instructions on how to use this in the same page:

Running the command to start the container:

docker run --name nginx-dummy -d -p 8080:80 some-content-nginx

Then open your web-browser at localhost:8080, and you'll see the following:

See? It's pretty easy once you get the hang of it, you can do the same for things like:

setting up a database locally like redis, postgres, mongoDB
setting up services like nginx, grafana, selenium, etc.

Now let's set this up on your machine, come back and try running this nginx container later after docker installation!

Docker Installation

Skip if you already have this done.

What you wanna install is "Docker Desktop", docker in itself is a collection of tools such as:

Docker engine.
Docker CLI.
Docker daemon.

Docker desktop has all these packaged, so install only that and you're set to go.

For Windows:

Go here and follow the steps
Note that for windows, you have to setup WSL (windows support for a linux like environment) - I'll like up to this really great tutorial here for this one.

For Linux:

For a GUI based environment:
- Go here, same as windows, setup docker desktop based on your distro.
- Pre-requisite: setup KVM (virtualization) on your system (how to do so present in the same link above)
For a TTY environment:
- In systems like AMI, ubuntu server, you may not have a GUI, and it's more suited to setup docker engine (comes with docker cli)
- Go here and check your distro.

So far, I've tested it in these environments:

Windows (with WSL)
Ubuntu (gnome)
Arch (hyprland - wayland)
Ubuntu Server.
AMI (amazon machine image)

After setup:

Try using the command:

docker --help

To start creating and using docker containers, note that you'll have to turn on the "docker daemon", how to do that? Since docker desktop already has everything, just open the desktop app, and you'll see a whale icon on top right screen, then try again.

The basis: Images & Containers

Gotta get a little bit theoretical on this part

Docker has two fundamental components:

Containers are services that are isolated, like virtual machines, they provide a isolated environment where you can run, test, do anything like its a whole new system inside it. These containers don't persist data in your disk like a normal service, delete a container and all it's data goes with it, it is therefore: stateless
- Advantage? You can create/destroy containers at ease.
- Problem? The isolation can cut itself off from others (we'll learn about controlling isolation with volumes & networks in next part!)

Images are blueprints to creating these container. An example of image can be set of instructions like:
- Clone a repository, install its dependencies.
- Run the main service (ex: express) on port 3000. Containers replicate this formula, so you can create any service into a image --> and then make a container from that image!

So basically, docker-hub (from nginx example) is just a central registry to host these images!

Writing our own custom Docker Image

Create some random directory, initialize node-js project & setup express:

mkdir test && cd test
npm init -y && npm install express dotenv
touch index.js

Make up a simple index.js script for the express server:

const express = require('express')
const app = express();
require('dotenv').config()

const port = 3000;
const num = process.env.num;

app.get('/', (req,res)=>{
  res.send(`Hello ${num}`);
})

app.listen(port, ()=>{
  console.log(`Server connected at port: ${port}`)
})

We write the image's recipe in a "Dockerfile", which looks something like this:

#needs a "base image", we have an express app, so base will be nodejs image. 
FROM node:18   

#inside container, make a /app folder to store our project files
WORKDIR /app  
#indicate that port 3000 is running the service(doesn't actually expose it, just a indicator)
EXPOSE 3000
#env variable used in the script    
ENV num=10   

#now copy our index.js + package.json to the /app folder for use
COPY index.js /app
COPY package.json /app

#run command in workdir(/app in this case), install packages from package.json (i.e: express & dotenv)
RUN npm install 
#start the application from
ENTRYPOINT ["node", "index.js"]

Now let's create the image from this dockerfile:

#run the dockerfile residing in this directory (assuming you are in same directory as dockerfile)
docker build -t express-image:v1 .

Confirm this image is created:

docker image ls

You should notice an "express-image:v1" image in the list.

Now, image is set, create container from that:

docker container run --name express-app -d -p 3000:3000 express-dummy:v1

Notice the -p 3000:3000? Remember the EXPOSE 3000 line back when we wrote the dockerfile, here -p <my-port>:<container-port> maps our PC's port 3000 to container port 3000.
So example: if you do -p 8080:3030, our port 8080 will act like container's port 3000!

Since we set port 3000:3000 mapping, make a request in localhost:3000:

wget localhost:3000

You should get an "index.html" file with contents: "Hello 10"

We used an environment variable "num" earlier but that was static, let's try passing a new value for the env variable:

docker container run --name express-app-2 -d -e num=50 -p 8080:3000 express-dummy:v1

In this new container from same image, we pass the num variable from -e key=value flag.
And this should run in your localhost:8080:

I honestly don't remember all the syntax for docker-file, so using something like the dockerfile reference is recommended

Summary

This is just the basics, but you should be able to use these as basis to start using containers, easily setting them up or even making your own containers, but there's many more things remaining:

Containers are isolated, so how do we let multiple containers connect? So far we've connected our PC to the container. - docker network
These containers drop everything when removed, what about - containers running DBs? we would want some to persist state right. - docker volumes
Is there any simpler way to handle multiple containers? - docker compose

Next Part: Docker continued - Volumes & Network (part-2)

Building a custom HTTP server from scratch

Shubham Tiwary — Thu, 27 Feb 2025 20:52:45 +0000

I've built APIs and played around with virtual machines for a long time now, and if you've done similar things
You know It's always followed by some protocol: http, web-sockets, ssh, ...

So today I wanna breakdown some mysteries on how things work behind the scenes just a little bit. Covering one of the most used protocols, HTTP

For context: a "protocol" is just like a dialect between computers, so different systems can talk in a common manner
.

References (sources you may wanna check after reading this):

High Performace browser networking (book)

HTTP 1.1 RFC (documentation on how HTTP is implemented)

Github source code (implemtation in golang)

Inside the engine - TCP

Now networks are unreliable, data may be lost in between, or take too much time to do so.
TCP was built for this, to provide a reliable connection on a unreliable network

Suppose there's two devices on a network, these devices have some address (their IP), and a process running (on a port), that's all you need for TCP.

It let's the devices:

send data in order (in chunks called "segments") --> like a queue.

if any data is load, retransmission is performed (ensures data reaches)

How do devices know if the other has received or not? Simple, just send an "ACK" (acknowledgement) to confirm, if there's no response after some time, assume it is lost.
Anyways, this in itself is a large topic, so we'll get to HTTP

Now, like HTTP, TCP also has underlying client-server model

And here's how the process kind of turns out to be:

So let's say server has an address (2.2.2.2) & running a TCP service on port (3000).
Client sends a TCP request here, first these devices has to do a "handshake" process (before they start actually sending data)
The handshake process goes as follows: client --> server(ACK) --> back to client (they exchange info needed during actual transmission of data later).
Once client receives data back from server, it starts sending data to server.
Every chunk of data after that, is a client --> server and followed by an ACK received back.

A look at HTTP

Now we've understood TCP a bit, let's look at the data being sent after the handshake.

HTTP is nothing more than just a format on how data should be present (same goes for most protocols).

Example:

When you use something like express, fastapi, flask to make a backend service, their implementation may be different but all of them expect the request in this format!

So the idea is to parse them in order that they are supposed to be and process them, then send a response, which also is in the same format.

Try using the following command in your terminal/command prompt (check if you have wget or not, most systems always has it beforehand):

 wget --server-response --debug --post-data="test" --header="Content-Type: text/plain" https://jsonplaceholder.typicode.com/posts

This will retrieve some json responses.

Note: In the example, it's a simple "Hello world" (a string), but you can send a lot more like: html, json, assets, blobs, etc

Honestly, these are more than enough to start building a HTTP server.
Like I said, the implementation can be different, but the protocol is the same, so here's some guide I'll give you to follow:

First try establishing a TCP connection, send some dummy data back & forth (you'll need a TCP server & a TCP client - check your language's docs - ex: go - net & nodejs - Net)'
Seperate into modules & test indivually:
- "Parser" --> breaks down request into info, header & body.
- "Router" --> like express, fastapi, or any server side framework handles "routing" i.e: check type of request (POST, GET) & path to assign a callback function
- "Composer" --> after processing the callback function, get response from the handler (callback function), and respond to client.

.
This was just a basic overview, this is definitely not something you wanna be using in prod.

But making things from ground up does give you a really good perspective on how things are, and change the way you look at problems, so you know not only how to do it, but also why.

You can follow my implementation at Github (it's in golang, but still you can use as reference to implement in your favourite language).

(There's more to be added in the future in this implementation probably, so I'll share updates here if I do make some changes)