DEV Community: Great-Victor Anjorin

How Blocking Port 80 Made My Time API Project More Secure (And More Annoying)

Great-Victor Anjorin — Tue, 30 Sep 2025 16:39:12 +0000

At some point, when the deadline for the assessment passed, it turned into a project for me. Trying to accomplish each of the tasks helped me to learn how to implement ideas and practices I only knew in theory.

For this article, we'll talk about using a domain for my Time API project. My implementation can be found in the namecom_domain branch of the project's repository. It was actually the main branch before the domain I was using expired. I wanted my API to be accessible from a simple URL.

Before my access to the domain expired, I was able to get a working implementation. To grasp how I achieved that, you'll have to understand a couple of the decisions I made for the project first.

Ingress Routing and Network Security Decisions Shaping Version of the Project

The first decision was to deploy an ingress controller to help with exposing the API, using a Terraform module that deploys an NGINX Ingress Controller Helm chart.

module "nginx-controller" {
 source  = "terraform-iaac/nginx-controller/helm"
 version = ">=2.3.0"

 create_namespace = true
 namespace        = "nginx-ingress"

 depends_on = [azurerm_kubernetes_cluster.time_api_cluster]
}

What is an Ingress Controller? It acts as a manager for all external traffic to microservices intended to be exposed outside the cluster. Its core function is to implement the routing rules defined in Ingress resources.

It handles advanced networking tasks, such as distributing user requests across instances of an application (Layer 7 load balancing), securing and decrypting data connections between users and servers (SSL/TLS termination), and directing traffic to different services using separate unique URL paths on the same website domain (host/path-based routing).

For my specific setup, I created an ingress Resource that defined the following routing rule: when external traffic arrives at the domain api.mywonder.works and the path /time, it should be routed to the API.

resource "kubernetes_ingress_v1" "time_api" {
 metadata {
   name      = "time-api-ingress"
   namespace = "time-api"
   annotations = {
     "cert-manager.io/cluster-issuer" = "certmanager"
   }
 }

 spec {
   ingress_class_name = "nginx"

   tls {
     hosts       = ["api.mywonder.works"]
     secret_name = "time-api-tls"
   }

   rule {
     host = "api.mywonder.works"
     http {
       path {
         path      = "/time"
         path_type = "Prefix"
         backend {
           service {
             name = kubernetes_service_v1.time_api.metadata[0].name
             port {
               number = kubernetes_service_v1.time_api.spec[0].port[0].port
             }
           }
         }
       }
     }
   }

   # Added a Default rule (no host) because my domain expired and I need to use the public IP for now

   rule {
     http {

       path {
         path      = "/time"
         path_type = "Prefix"
         backend {
           service {
             name = kubernetes_service_v1.time_api.metadata[0].name
             port {
               number = kubernetes_service_v1.time_api.spec[0].port[0].port
             }

           }
         }
       }

     }

   }

 }

 depends_on = [kubernetes_service_v1.time_api, time_sleep.wait_for_nginx]
}

Without a domain, the default access method would have been directly through the IP address assigned by the Cloud Service Provider as an endpoint for ingress.

My second decision was to restrict access to the cluster through Port 80 at the Virtual Private Cloud Network level by denying all inbound traffic except on Port 443 and from within the Virtual Network itself, using Network Security Group (NSG) security rules. This prevented unencrypted traffic to the API and minimized the overall attack surface of my setup.

resource "azurerm_network_security_group" "time_api_nsg" {
 name                = "nsg-${azurerm_resource_group.time_api_rg.name}"
 resource_group_name = azurerm_resource_group.time_api_rg.name
 location            = azurerm_resource_group.time_api_rg.location

 security_rule {
   name                       = "allow-https-access"
   priority                   = 100
   direction                  = "Inbound"
   access                     = "Allow"
   protocol                   = "Tcp"
   source_port_range          = "*"
   destination_port_range     = "443"
   source_address_prefix      = "*"
   destination_address_prefix = "*"
 }

 security_rule {
   name                       = "allow-vnet-inbound"
   priority                   = 102
   direction                  = "Inbound"
   access                     = "Allow"
   protocol                   = "*"
   source_port_range          = "*"
   destination_port_range     = "*"
   source_address_prefix      = "VirtualNetwork"
   destination_address_prefix = "*"

 }

 security_rule {
   name                       = "deny-all-inbound"
   priority                   = 4096
   direction                  = "Inbound"
   access                     = "Deny"
   protocol                   = "*"
   source_port_range          = "*"
   destination_port_range     = "*"
   source_address_prefix      = "*"
   destination_address_prefix = "*"
 }
}

This decision alone made the project a little more complex—in other words, I made it much harder for myself.
How hard? Well, now I couldn't use the http-01 validation method to get an SSL/TLS Certificate from Let's Encrypt.
For clarity, let me explain what an SSL certificate and a certificate authority is.

An SSL/TLS Certificate authenticates that a user's browser is truly connecting to your API and not a malicious third party, and it enables the use of the HTTPS protocol — which encrypts all data transmitted between the user's browser and my API's host.

It also reassures users that the connection is secure, since major browsers display a padlock icon in the address bar for websites that have a certificate from a trusted Certificate Authority like Let's Encrypt.

A Certificate Authority is a trusted third party organisation that issues SSL/TLS Certificates. The different ways to get a certificate issued are called validation methods.

Validation Challenge

So why couldn’t I use the http-01 validation method? Http-01 requires access to port 80 on the host server. Since I restricted that, I had to use a different validation method. I opted for dns-01 and that required a few things:

A DNS Provider: The DNS-01 method works by automatically adding a specific TXT record to a DNS Provider's records. To do this programmatically, I needed a DNS provider that Cert-Manager could interact with. I used Name.com since they also had an API that I could use to interact with their services.
Permissions to Modify DNS Records: This is where the Name.com API came in. Cert-Manager needed the authority to create and delete the temporary DNS records required for the challenge. I had to securely store and provide these credentials for accessing the Name.com API to the cluster for the validation to work.
The Cert-Manager Webhook: The dns-01 challenge is not natively supported for all DNS providers. To enable this functionality for my Name.com domain, I had to deploy a custom webhook—an external component that extends Cert-Manager to support my specific DNS provider.

Thankfully, I didn't have to create the webhook by myself. The credit for that goes to someone named Ian Grant. There was a Helm chart available in a GitHub repository of theirs. There is always a risk of it not being maintained, and I intend to learn how to create my own, but it was a huge help.

Implementation with Terraform

Now that you understand my decisions a little, I can better explain my implementation.

Since I am relying on Infrastructure as Code (IaC) with Terraform, I was able to define and automate the entire setup in a reproducible way. In the namecom_domain branch, the core infrastructure provisioning is separated from the application deployment — there's a dedicated microservices/ folder holding the deployment-specific configs like deploy.tf.

This let me provision the AKS cluster, networking, and base add-ons first, then layer on the API deployment and cert management in a second Terraform apply step. I structured it this way to ensure the infrastructure was provisioned correctly before the application deployed—minimizing errors in the first workflow run.

Helm, as a package manager for Kubernetes, made deploying Cert-Manager Controller straightforward. The Cert-Manager Controller manages the issuance and renewal of SSL/TLS certificates in the cluster. I used Helm to install the Cert-Manager Controller and a ClusterIssuer resource, as defined in the provision.tf file.

resource "helm_release" "cert_manager" {
 name       = "cert-manager"
 repository = "https://charts.jetstack.io"
 chart      = "cert-manager"
 version    = "v1.5.4"
 create_namespace = true
 namespace        = "cert-manager"

 set {
   name  = "installCRDs"
   value = "true"
 }

 timeout = 600

 depends_on = [module.nginx-controller]

}

resource "helm_release" "cert_manager_issuers" {
 chart      = "cert-manager-issuers"
 name       = "cert-manager-issuers"
 version    = "0.3.0"
 repository = "https://charts.adfinis.com"
 namespace  = "cert-manager"

 # https://acme-staging-v02.api.letsencrypt.org/directory
 values = [
   <<-EOT

 clusterIssuers:
  - name: certmanager
    spec:
      acme:
        email: "greatvictor.anjorin@gmail.com"
        server: "https://acme-v02.api.letsencrypt.org/directory"
        privateKeySecretRef:
          name: certmanager
        solvers:
          - dns01:
              webhook:
                groupName: acme.name.com
                solverName: namedotcom
                config:
                  username: "${var.namecom_username}"
                  apitokensecret:
                    name: namedotcom-credentials
                    key: api-token               

EOT

 ]

 depends_on = [helm_release.cert_manager, kubernetes_secret_v1.namecom_api_token]
}

I configured the ClusterIssuer to use Let's Encrypt's ACME server. The issuer is cluster-wide—meaning it can issue certificates for any namespace. Key configs include setting the ACME server URL to the production endpoint (https://acme-v02.api.letsencrypt.org/directory) and defining the dns-01 solver.

For the solver, I pointed it to the custom Name.com webhook, which handles the actual Name.com API calls to add and remove TXT records during validation. I installed both the controller and the issuer via Helm release resources in the provision.tf file.

As for the webhook, I deployed it as another Helm release in the same provision.tf file. In the GitHub Actions workflow, I used the actions/checkout@v4 action to clone Ian Grant's Name.com webhook GitHub repository into a local webhook/ directory. The Helm release then references the chart using a relative path (../webhook/deploy) to the locally cloned repository during the workflow run.

- name: Checkout Name.com webhook Github repository
     uses: actions/checkout@v4

     with:
       repository: imgrant/cert-manager-webhook-namecom
       path: webhook

resource "helm_release" "namecom_webhook" {
 name       = "namecom-webhook"
 repository = "../webhook/deploy"
 chart      = "cert-manager-webhook-namecom"
 namespace  = "cert-manager"

 depends_on = [helm_release.cert_manager]

}

The webhook runs in the cert-manager namespace and needs access to Name.com API credentials. I provided these securely by creating a Kubernetes Secret for the sensitive API token, populated with a Terraform variable that gets its value from DOMAIN_API_TOKEN in GitHub Secrets during the Actions workflow.

- name: Apply Terraform configuration
     env:
       ARM_CLIENT_ID: ${{ secrets.ARM_CLIENT_ID }}
       ARM_CLIENT_SECRET: ${{ secrets.ARM_CLIENT_SECRET }}
       ARM_SUBSCRIPTION_ID: ${{ secrets.ARM_SUBSCRIPTION_ID }}
       ARM_TENANT_ID: ${{ secrets.ARM_TENANT_ID }}

       USER_OBJECT_ID: ${{ secrets.MY_USER_OBJECT_ID }}
       USERNAME: ${{ secrets.DOMAIN_API_USERNAME }}
       TOKEN: ${{ secrets.DOMAIN_API_TOKEN }}
     run: |
       cat <<EOF > terraform.tfvars.json
       {
         "my_user_object_id": "$USER_OBJECT_ID",
         "namecom_username": "$USERNAME",
         "namecom_token": "$TOKEN"
       }

       EOF
       terraform apply --auto-approve
     working-directory: ./terraform


resource "kubernetes_secret_v1" "namecom_api_token" {

 metadata {
   name      = "namedotcom-credentials"
   namespace = "cert-manager"
 }

 data = {
   api-token = var.namecom_token
 }

 type = "Opaque"

depends_on = [helm_release.namecom_webhook]

}

For the username, I took a slightly different approach. It is still stored in GitHub Secrets and passed to Terraform as a variable, but instead of creating a Kubernetes Secret, I injected it directly into the values block of the ClusterIssuer Helm release.

This approach ensures the Cert-Manager Controller can authenticate with Name.com via the webhook, without exposing credentials in code.

Once Cert-Manager and the webhook are in place, the magic happens in the Ingress resource defined in microservices/deploy.tf. This file deploys the actual Time API as a Kubernetes Deployment and Service, then creates an Ingress to route traffic.

The Ingress spec includes a host rule for api.mywonder.works, routing the /time path to the backend service on port 5000. To trigger automatic certificate issuance, the annotation cert-manager.io/cluster-issuer: certmanager ensures that Cert-Manager:

watches the time-api Ingress resource for changes,
requests certificates for hosts listed in the tls block (in our case, api.mywonder.works) via the certmanager ClusterIssuer using the challenge type specified in the ClusterIssuer resource (dns-01),
stores the issued certificate in the specified secret (time-api-tls),
and enables TLS termination at the ingress controller level.

Under the hood, when you apply this, Cert-Manager creates a Certificate resource, initiates the ACME challenge with Let's Encrypt, and uses the webhook to temporarily add a TXT record to your Name.com DNS zone (something like, in my case, _acme-challenge.api.mywonder.works). Let's Encrypt verifies the record to confirm domain ownership, issues the certificate, and Cert-Manager stores it as a Secret mounted to the Ingress. Renewals happen automatically before the 90-day expiration, with the same process repeating seamlessly.

This implementation is carefully scripted out in the GitHub Actions workflow defined in build.yaml. The workflow first builds and pushes the Docker image of the Flask app to Docker Hub, then uses Terraform to provision the infrastructure. It populates a terraform.tfvars.json file with GitHub Secrets (domain and Name.com credentials) that I created before triggering the workflow, to avoid hardcoding sensitive info.

After the base infrastructure is up, it moves the microservices/deploy.tf into the root and runs another terraform apply to deploy the app and Ingress.

Like I explained earlier, this staged approach ensures dependencies like the cluster and Cert-Manager are ready before certificate issuance kicks in.

If you want to try this with your own Name.com domain, once deployed, you can hit https://api.YOUR_DOMAIN_HERE/time (note the HTTPS) in your browser or via curl, and you'll get the current UTC time securely. If things go wrong—say, DNS propagation delays or webhook issues—you can troubleshoot with kubectl get certificates to check certificate status, or dive into Cert-Manager logs with kubectl logs -n cert-manager.

Conclusion

This setup not only made my API accessible via a clean, branded URL but also enforced HTTPS-only access without manual cert management. It was a bit of extra work upfront due to the Port 80 restriction, but the automation pays off for scalability and security.

If you're adapting this, remember to update details like the domain in the time-api Ingress resource and email in the cluster issuer resource, and ensure your Name.com API token has the right permissions.

The branch's README has more on prerequisites and manual deployment if you want to tinker locally before going full CI/CD.

Overall, I was forced to delve a little deeper into some cloud-native practices and how to make them happen, and I'm glad I pushed through the complexities—domain expirations aside!

Confessions of a DevOps Noob Who Tried to pick a K8S Service CIDR from an Azure Subnet 🤦‍♂️

Great-Victor Anjorin — Sun, 10 Aug 2025 15:16:19 +0000

Let's kickoff with the first issue I remember that made me feel very dumb - configuring VPC networking for my cluster.

The assessment required that I setup "a NAT gateway to manage egress traffic from the cluster, VPC networking, subnets, and firewall rules for secure communication.

Like I said in my little introduction to this article series, I didn't have to think about or do any of those for my final project at AltSchool. I let Azure Kubernetess Service (AKS) handle it by default. My knowledge on setting up address spaces was weak and I had no experience with NAT gateways, so I got to googling.

My Rookie Mistake

I came across this article on provisioning AKS and a NAT gateway with Terraform. Luckily, they also detailed their Vnet and Subnet configurations, so I tried to replicate it - which is a fancy way of saying I copied and pasted. But I am not that much of a degenerate; I tried to make it my own. Ironically, that where the problems started. I must have thought "why is bro not picking an address space for service_cidr from the VNet/Subnet he provisioned?". As you may have already guessed, I f***ed around and found out. I kept getting errors at terraform apply that I didn't have the patience to decipher.

Since I was basically vibe-coding with knowledge of VPC and cluster networking that was next to zero, it took me quite a while to get out of this hole.

What a `service_cidr` is

service_cidr isn’t part of your physical network. It's a virtual IP space that Kubernetes manages on its own. It’s meant for internal cluster service discovery, separate from your VNet and subnets.

The subnet (attached to your node pool) on the other hand is meant for the nodes (virtual machines) and pods (containers or groups of containers).

To help drive the point home, let's say you have a VNet with an address space of 10.240.0.0/16, a subnet carved out for your cluster node pool at 10.240.0.0/22 and a service_cidr of 172.16.0.0/16.


[VNet: 10.240.0.0/16]
└── [Cluster Subnet: 10.240.1.0/22]
├── Node 1 (10.240.1.10)
│   └── Deployment (3 Replicas) ──────────────────┐
│       ├── Pod (Replica 1) (10.240.1.11) ←┐    Kubernetes Service (svc) ←─ Other deployment/pods or ingress
│       ├── Pod (Replica 2) (10.240.1.12) ←┤←────┘
│       └── Pod (Replica 3) (10.240.1.13) ←┘
└── Node 2 (10.240.1.20)
└── Pod (10.240.1.21)  # Unrelated to the service/deployment above

[service_cidr: 172.16.0.0/16] (Virtual)
└── Service (svc) IP: 172.16.0.50   # Virtual IP for load balancing. Distributes traffic to pods with selector: app=my-app
    └── Load Balancer (kube-proxy)  # Traffic to the Virtual IP is dynamically routed to one available pod endpoint based on kube-proxy rules
        │                           # (iptables/nftables/IPVS) and pod readiness (e.g., round-robin, random, or least connections)
        └──→ Pod IPs: 10.240.1.11, 10.240.1.12, 10.240.1.13

Notes:

- Kubernetes Service (svc): A Kubernetes abstraction that allows pods to communicate with each other using a stable virtual IP address.
- service_cidr: A range of IP addresses used by Kubernetes to assign IPs to services.
- Kube-proxy: A component that manages network rules on nodes to route traffic to the correct pod IPs.
- Pod IP: The IP address assigned to a pod within the provided Cluster Subnet.

- Service (svc) IPs (e.g. 172.16.0.50) are virtual IPs that do not correspond to a physical device.
- The svc uses a selector (e.g., app=my-app) to dynamically target all pods matching the label.
- kube-proxy (running on each node) handles load balancing (e.g., round-robin, random) to distribute traffic to available pods.
- Pods can be on any node (Node 1 or Node 2) and are dynamically added/removed based on deployment scaling or failures.

When a pod calls a service, kube-proxy steps in, maps the service IP (say, 172.16.0.50) to a pod IP (like 10.240.1.13), and routes the traffic over the real network. The service_cidr is just an abstraction — it doesn’t “live” on the Azure infrastructure.

If you're relatively new to Kubernetes, I know what you are thinking. Why not connect directly to a pod’s IP? Well, here are are few more things to note:

Pod IPs are ephemeral (can change at any time); Services provide a stable VIP/DNS.
Services load balance across pod replicas and only route to the ones that are available.
Zero-downtime rollouts are made possible by the abstraction; Upgrades and scaling can change backends without affecting access.

The Code That Saved the Day

After almost pulling my hair out, I finally got it right. Here’s the Terraform config that worked:

resource "azurerm_virtual_network" "time_api_vnet" {
  name                = "vnet-${azurerm_resource_group.time_api_rg.name}"
  address_space       = ["10.240.0.0/16"]
  location            = azurerm_resource_group.time_api_rg.location
  resource_group_name = azurerm_resource_group.time_api_rg.name
}

resource "azurerm_subnet" "time_api_subnet" {
  name                 = "subnet-${azurerm_resource_group.time_api_rg.name}"
  resource_group_name  = azurerm_resource_group.time_api_rg.name
  virtual_network_name = azurerm_virtual_network.time_api_vnet.name
  address_prefixes     = ["10.240.0.0/22"]
}

# .....NSG rules and association configuration would go here, but is omitted for brevity.

resource "azurerm_nat_gateway" "time_api_nat_gateway" {
  name                    = "natgw-${azurerm_resource_group.time_api_rg.name}"
  location                = azurerm_resource_group.time_api_rg.location
  resource_group_name     = azurerm_resource_group.time_api_rg.name
  sku_name                = "Standard"
  idle_timeout_in_minutes = 10

  tags = {
    Environment = "test"
  }
}

resource "azurerm_public_ip" "time_api_public_ip" {
  name                = "public-ip-${azurerm_resource_group.time_api_rg.name}"
  location            = azurerm_resource_group.time_api_rg.location
  resource_group_name = azurerm_resource_group.time_api_rg.name
  allocation_method   = "Static"
  sku                 = "Standard"
}

resource "azurerm_subnet_nat_gateway_association" "time_api_natgw_subnet_association" {
  nat_gateway_id = azurerm_nat_gateway.time_api_nat_gateway.id
  subnet_id      = azurerm_subnet.time_api_subnet.id
}

resource "azurerm_nat_gateway_public_ip_association" "time_api_natgw_public_ip_association" {
  nat_gateway_id       = azurerm_nat_gateway.time_api_nat_gateway.id
  public_ip_address_id = azurerm_public_ip.time_api_public_ip.id
}

resource "azurerm_kubernetes_cluster" "time_api_cluster" {
  name                = "aks-${azurerm_resource_group.time_api_rg.name}-cluster"
  resource_group_name = azurerm_resource_group.time_api_rg.name
  location            = azurerm_resource_group.time_api_rg.location
  dns_prefix          = "dns-${azurerm_resource_group.time_api_rg.name}-cluster"
  kubernetes_version  = data.azurerm_kubernetes_service_versions.current.default_version
  node_resource_group = "nrg-aks-${azurerm_resource_group.time_api_rg.name}-cluster"

  default_node_pool {
    name                 = "default"
    vm_size              = "Standard_D2_v2"
    auto_scaling_enabled = true
    max_count            = 2
    min_count            = 1
    os_disk_size_gb      = 30
    type                 = "VirtualMachineScaleSets"
    vnet_subnet_id       = azurerm_subnet.time_api_subnet.id

    # ...other node pool configurations
  }

  identity {
    type = "SystemAssigned"
  }

  azure_active_directory_role_based_access_control {
    azure_rbac_enabled     = true
    admin_group_object_ids = [azuread_group.time_api_admins.object_id]
  }

  network_profile {
    network_plugin    = "azure"
    network_policy    = "azure"
    load_balancer_sku = "standard"
    dns_service_ip    = "172.16.0.10"
    service_cidr      = "172.16.0.0/16"
    outbound_type     = "userAssignedNATGateway"
    nat_gateway_profile {
      idle_timeout_in_minutes = 4
    }
  }

  # ....rest of the AKS cluster configuration
}

See that service_cidr set to 172.16.0.0/16? It’s safely outside the VNet’s 10.240.0.0/16 range — no overlap, no drama.

Other Relevant VPC Networking Points

Alright, so I survived the service_cidr fiasco, but there were a few more networking nuggets that made me go, “Oh, that’s how it works!” Here’s the stuff I wish I knew before crashing and burning my way through AKS networking. These are the bits that tie the VNet, subnets, and that fancy NAT Gateway together, so you don’t end up in a similar hole like mine.

Your Subnet Is the Real Deal for Nodes and Pods

The subnet you define (like 10.240.0.0/22 in my setup) is where the real network lives. It’s carved out of your VNet (e.g. 10.240.0.0/16) and assigned to your AKS node pool via vnet_subnet_id in the Terraform config. Nodes (VMs) and pods (containers or individual groups of containers) grab IPs from this range—like 10.240.1.10 for a node or 10.240.1.11 for a pod.

Not so Pro Tip: make sure your subnet is big enough (e.g., /22 gives ~1,000 IPs) to handle all your nodes and pods. So that AKS doesn't start throwing “no IPs left” errors when scaling.
Also, some Network Security Group (NSG) rules on that subnet can help to control traffic on the cloud level. Isolation and least privilege access is the name of the game.

`service_cidr` Is Just Kubernetes Playing Pretend

That 172.16.0.0/16 service_cidr? It’s not touching your Azure network — it’s like a figment of Kubernetes' imagination (you know, make-believe). It’s where service IPs (like 172.16.0.50 in the diagram) and the DNS IP (172.16.0.10 for CoreDNS) live. When a pod or ingress pings a service IP, kube-proxy (the traffic cop) translates it to a real pod IP (like 10.240.1.11, 10.240.1.12, or 10.240.1.13) and sends it over the subnet. The diagram shows this load balancing in action — traffic hits the service IP, and kube-proxy picks a healthy pod to route to, using tricks like round-robin. No physical device has that 172.16.0.50 IP; it’s all software magic.

Pick Your `service_cidr` Carefully (or Pay the Price)

You can let AKS pick a service_cidr (it defaults to 10.0.0.0/16), but if it overlaps with your VNet or peered networks, you’re in for a bad time. Trust me, I got my fair share of Terraform errors before I learned this. Stick to RFC 1918 private ranges (10.0.0.0/8, 172.16.0.0/12, or 192.168.0.0/16) — Public IP ranges? Forget it. And whatever you choose, make sure it doesn't clash with your VNet, peered networks, or anything else in your setup. Here's a quick cheat sheet I wish I'd had:

| Your VNet Range   | Good `service_cidr` Options         |
|-------------------|-------------------------------------|
| `10.x.x.x/16`     | `172.16.0.0/16` or `192.168.0.0/16` |
| `172.16.x.x/12`   | `10.96.0.0/12` or `192.168.0.0/16`  |
| `192.168.x.x/16`  | `10.96.0.0/12` or `172.16.0.0/16`   |

Oh, and that dns_service_ip? Avoid the first IP in your range (like 172.16.0.1) — Kubernetes reserves it for internals. As you must have noticed, I went with .10 myself.

Load Balancing Ties It All Together

Back to that diagram: the Kubernetes Service (172.16.0.50) is your VIP (Very Important IP) that load-balances across pods. Kube-proxy decides which pod gets the traffic based on readiness and rules (e.g., round-robin). This is why you don’t hardcode pod IPs — they’re temporary! The service_cidr and kube-proxy make sure your app stays reachable, even if pods move or die. It’s like a traffic director who doesn’t care which car (pod) gets you there, as long as you arrive.

NAT Gateway: The Unsung Hero for Outbound Traffic

Since I used outbound_type = "userAssignedNATGateway", my cluster’s outbound traffic (like pods hitting external APIs) goes through the NAT Gateway. The idle_timeout_in_minutes = 4 in my AKS config (and 10 in the azurerm_nat_gateway resource — oops, my bad for the mismatch!) controls how long idle connections stay open. Four minutes is fine for most apps, but if you’re running something long-lived like streaming, you might want to bump it up to avoid dropped connections. Also, ensure your NAT Gateway is tied to your subnet (via azurerm_subnet_nat_gateway_association) and has a public IP for internet access.

Key Takeaway

In a nutshell, plan your VNet and subnet sizes, keep your service_cidr separate, and double-check for overlaps. AKS can auto-pick some settings, but explicit configs (like in my Terraform) save you from surprises. Oh, and don’t wing it like I did — read or, at least, use any appropriate AI tool of your choice to try to understand the relevant docs first!

Here is the link to the project repo again if you want to try it out for yourself.

See you on the next one.

0 to K8s: How a Take-Home Assessment Took Over My Life 😭

Great-Victor Anjorin — Tue, 05 Aug 2025 14:07:17 +0000

It was September 2024 and I was in way over my head.

I had just completed a DevOps Engineering program (offered by AltSchool Africa) with "moderate" confidence in the skills I had learned during the 12-month stint.

I heard of a job opening and thought "why not" and applied. I was given a take-home assessment that looked very similar to my final project at AltSchool. I was so excited. I thought I would crush it and move on to the interview stage.

I was SO WRONG.

🚧 The Task That Took Me Nearly a Year

Deploy a simple API on Google Kubernetes Engine (GKE) using Terraform to provision both the infrastructure and define the Kubernetes (K8s) deployments.

At AltSchool, I had defined K8s deployments using YAML manifests — like a normal person. I didn’t even know Terraform could handle K8s manifests. But I figured, how hard could it be? With some help from ChatGPT and a lot of grit, I could get the assessment done in a weekend.

Well… I did. After 10 months.

Important to note: I couldn't create a Google Cloud account, so I asked if I could use Microsoft Azure and I was permitted - so that's what I used.

🧠 What I Had to Learn (the Hard Way)

This assessment snowballed into one of the most difficult and rewarding technical challenges I’ve ever faced. Here’s are some of the rabbit holes I fell down:

Configuring a virtual network (something Azure Kubernetes Service usually handles by default) and creating a NAT gateway for the cluster.
Securing the K8s API server without using authorized_ip_ranges (this forced me to rethink access for my chosen CI/CD tool that would run terraform apply and destroy - GitHub Actions)
Learning what Microsoft Azure services could help with monitoring and how to configure them.
Learning what K8s Network Policies were, what they protect, and why they matter.
Learning how to issue a TLS certificate for my name.com domain (that I intended to use to make the API accessible publicly, abstracting its IP address) using Let’s Encrypt with DNS challenge — a mostly unpleasant experience.
Figuring out how to keep any sensitive data (tokens and credentials) secure but accessible to my CI/CD pipeline using GitHub secrets, GitHub Actions workflow environment variables and a here-doc command?! (I just found out that was what it was called while writing the draft for this article)
Writing reusable GitHub Actions workflow scripts to automate deployment and destruction — and learned just how fragile CI/CD can be.
I had clashes with NGINX Ingress on the cluster way too often - network policy misconfigurations, ingress creation blocked by absent admission controller endpoints, among other issues.
My name.com domain expired {frustrated screams} - so I had to pivot to using the ingress IP directly.
And bash scripting. Quite a bit of bash scripting.

🎯 What Changed?

This project broke me down and rebuilt me. I came in thinking I had a decent handle on DevOps. I came out realizing how deep the field is — and how much more I still have to learn.

But here's the key: I did learn. Slowly. Painfully. Iteratively. And now, I know how to deploy a secure, observable, automated microservice stack from scratch — using tools I’d never even heard of when I started.

📝 Why I’m Writing This

I'm not fully done with the project (I'm kind of a perfectionist and there are bonus points in the assessment), but I have decided to write a collection of articles that try to detail the issues I encountered that I can remember and how I worked through them. I don't intend to win like a Pulitzer Prize or anything (though I wouldn't mind) but I genuinely hope my insights can help others battle similar challenges.

In the coming weeks, I’ll publish a series of articles breaking down:

How to deploy Kubernetes workload using Terraform,
My solution to securing an AKS cluster,
- Azure Networking and NAT Gateway configuration
- Securing the K8S API server (without using authorized_ip_ranges block)
- Managing a Private Cluster
- K8S network policies
How to automate deployments,
- Using GitHub Actions (for Continuous Integration and Deloyment) and Secrets (to securely pass sensitive tokens and credentials into Actions workflows),
What not to do with NGINX Ingress (please learn from my pain),
Monitoring with Azure Managed Prometheus & Grafana,
Issuing TLS certificates via Let’s Encrypt DNS-01 + name.com’s API,
Remote Backend Management for Terraform,
And many other topics.

Here is the GitHub repository link for those that don't mind spoilers

🙌 Final Thoughts

No, I didn’t “crush” the take-home assessment. I never even submitted it. But I learned more in these 10 months than in 12 months at AltSchool — and I came out stronger, more capable, and more confident in my abilities.

If you're in over your head right now — keep going. You’re not alone. And you might be a lot closer than you think.

Stay tuned — the war stories (and lessons) are just beginning.

Automating User Creation and Management with Bash: A Step-by-Step Guide

Great-Victor Anjorin — Tue, 02 Jul 2024 21:45:58 +0000

Automating user creation and management can save time, reduce errors, and enhance security for SysOps engineers. In this article, we will go over a Bash script that automates the creation of users and groups, sets up home directories, and manages passwords securely. The script reads from a text file where each line consists of a username and their corresponding groups, logs every action in a log file, and saves the randomly generated passwords for each user in a secure .csv file accessible only to the owner.

Let's dive into the bash script and break it down step by step.

Line-by-Line Explanation

1.) First off, we have our shebang.

#!/bin/bash

This specifies the type of interpreter script will be run with. Since it is a "bash" script, it should be run with the Bourne Again Shell (Bash) interpreter. Also, some commands in the script may not be interpreted correctly outside of Bash.

2.) The paths for the log file and the password file are set to avoid unnecessary repetition in the script.

# Define log and password storage files
LOG_FILE="/var/log/user_management.log"
PASSWORD_FILE="/var/secure/user_passwords.csv"

3.) To ensure that the bash script runs with root privileges, an if statement checks if the Effective User ID (EUID) is equal to zero. The EUID determines the permissions the script will use to run, and 0 represents the root user ID in Linux systems. Only users with administrative privileges (users who can use sudo or the root user itself) can run the script. If someone attempts to run it without such privileges, an error message and the script's run process will be terminated.

# Check if the script is run with root privileges
if [[ $EUID -ne 0 ]]; then
  echo "This script must be run with root privileges." >&2
  exit 1
fi

4.) To ensure that an input file is provided as an argument when running the script, this if statement will terminate the script if no argument is provided. In this statement, $# represents the argument provided when running the script. If it is equal to zero (no argument is provided) or if it is greater than or equal to 2, an error message will be printed and the script's execution is halted.

# Check if the input file is provided
if [[ $# -eq 0 || $# -ge 2 ]]; then
  echo "Usage: $0 <user_file>" >&2
  exit 1
fi

5.) Next is the log_action function that records logs using bold lettering (formatted with ANSI escape codes: \033[1m and \033[0m) and a timestamp (using the date command to get the current date and the specified date format: '%Y-%m-%d %H:%M:%S'). This function is used to log important steps, success messages, and error messages in the script.

# Log function
log_action() {
  echo "--------------------------------------------------" | tee -a "$LOG_FILE"
  echo -e "$(date +'%Y-%m-%d %H:%M:%S') - \033[1m$1\033[0m" | tee -a "$LOG_FILE"
  echo "--------------------------------------------------" | tee -a "$LOG_FILE"
}

6.) Next is the create_user_account function that manages the entire process of creating a user, setting up their home directories with appropriate permissions and ownership, adding them to specified groups, and assigning randomly generated passwords. Every important step is logged.

create_user_account function

create_user_account() {
  local username="$1"
  local groups="$2"

  log_action "Creating user account '$username'..."

  # Check if user already exists
  if id "$username" &> /dev/null; then
    echo "User '$username' already exists. Skipping..." | tee -a "$LOG_FILE"
    return 1
  fi

  # Create user with home directory and set shell
  if useradd -m -s /bin/bash "$username"; then
    echo "User $username created successfully." | tee -a "$LOG_FILE"
  else
    echo "Error creating user $username." | tee -a "$LOG_FILE"
    return 1
  fi

  # Create user group if it does not exist (in case the script is run in other linux distributions that do not create user groups by default)
  if ! getent group "$username" >/dev/null; then
    groupadd "$username"
    usermod -g "$username" "$username"
    log_action "Group $username created."
  fi

  # Set up home directory permissions
  echo "Setting permissions for /home/$username..." | tee -a "$LOG_FILE"
  chmod 700 "/home/$username" && chown "$username:$username" "/home/$username"
  if [[ $? -eq 0 ]]; then
    echo "Permissions set for /home/$username." | tee -a "$LOG_FILE"
  else
    echo "Error setting permissions for /home/$username." | tee -a "$LOG_FILE"
    return 1
  fi

  # Add user to additional groups (comma separated)
  echo "Adding user $username to specified additional groups..." | tee -a "$LOG_FILE"
  IFS=',' read -ra group_array <<< "$groups"
  for group in "${group_array[@]}"; do
    group=$(echo "$group" | xargs)

    # Check if group exists, if not create it
    if ! getent group "$group" &>/dev/null; then
      if groupadd "$group"; then
        echo "Group $group did not exist. Now created." | tee -a "$LOG_FILE"
      else
        echo "Error creating group $group." | tee -a "$LOG_FILE"
        continue
      fi
    fi

    # Add user to group
    if gpasswd -a "$username" "$group"; then
      echo "User $username added to group $group." | tee -a "$LOG_FILE"
    else
      echo "Error adding user $username to group $group." | tee -a "$LOG_FILE"
    fi
  done

  # Log if no additional groups are specified
  if [[ -z "$groups" ]]; then
    echo "No additional groups specified." | tee -a "$LOG_FILE"
  fi

  # Generate random password, set it for the user, and store it in a file
  echo "Setting password for user $username..." | tee -a "$LOG_FILE"
  password=$(head /dev/urandom | tr -dc A-Za-z0-9 | head -c 12)
  echo "$username:$password" | chpasswd
  if [[ $? -eq 0 ]]; then
    echo "Password set for user $username." | tee -a "$LOG_FILE"
    echo "$username,$password" >> "$PASSWORD_FILE"
  else
    echo "Error setting password for user $username. Deleting $username user account" | tee -a "$LOG_FILE"
    userdel -r "$username"
    return 1
  fi
}

In the function, I initially set local variables to hold the values of the specified username and groups, to avoid repetition.

local username="$1"
local groups="$2"

In this next section, I use the log_action function to record the start of each user account creation. Additionally, I verify whether the user already exists. If the user does exist, an error message is displayed and the script's execution is stopped.

log_action "Creating user account '$username'..."

  # Check if user already exists
  if id "$username" &> /dev/null; then
    echo "User '$username' already exists. Skipping..." | tee -a "$LOG_FILE"
    return 1
  fi

Next, there is an if statement that uses the useradd command with the -m and -s flags to create a user with a login shell (In this case, the login shell is set to be /bin/bash. If you want, you can modify or remove the -s /bin/bash part entirely) and assigns a home directory to the user. It stops the script's run process if an error occurs during the execution of the command.

# Create user with home directory and set shell
  if useradd -m -s /bin/bash "$username"; then
    echo "User $username created successfully." | tee -a "$LOG_FILE"
  else
    echo "Error creating user $username." | tee -a "$LOG_FILE"
    return 1
  fi

Next, in the case that the script is run on other Linux distributions that do not create and assign a primary group with the same name as the newly created user, the if statement here will create a group with the same name as the user and add it as the user's primary group.

# Create user group if it does not exist (in case the script is run in other linux distributions that do not create user groups by default)
  if ! getent group "$username" >/dev/null; then
    groupadd "$username"
    usermod -g "$username" "$username"
    log_action "Group $username created."
  fi

In this section of the function, the newly created user is designated as the owner of the newly created home directory. The owner is also granted all possible permissions for the directory. This is all done using the chmod and chown commands. If this process is unsuccessful, an error message is printed and the execution is halted.

# Set up home directory permissions
  echo "Setting permissions for /home/$username..." | tee -a "$LOG_FILE"
  chmod 700 "/home/$username" && chown "$username:$username" "/home/$username"
  if [[ $? -eq 0 ]]; then
    echo "Permissions set for /home/$username." | tee -a "$LOG_FILE"
  else
    echo "Error setting permissions for /home/$username." | tee -a "$LOG_FILE"
    return 1
  fi

In this section, the newly created user is added to additional groups specified in the input file. By setting the Internal Field Separator (IFS) to expect comma-separated values and using the read command with the -ra flags, the groups are individually placed inside an array called group_array to be used in the subsequent for loop. Within the loop, for every value in the group_array, the xargs command removes any whitespace, creates the group if it does not exist, and finally adds the user to the group using the gpasswd command with the -a flag. In the case where no group is specified for the user in the input file, a message will be printed.

# Add user to additional groups (comma separated)
  echo "Adding user $username to specified additional groups..." | tee -a "$LOG_FILE"
  IFS=',' read -ra group_array <<< "$groups"
  for group in "${group_array[@]}"; do
    group=$(echo "$group" | xargs)

    # Check if group exists, if not create it
    if ! getent group "$group" &>/dev/null; then
      if groupadd "$group"; then
        echo "Group $group did not exist. Now created." | tee -a "$LOG_FILE"
      else
        echo "Error creating group $group." | tee -a "$LOG_FILE"
        continue
      fi
    fi

    # Add user to group
    if gpasswd -a "$username" "$group"; then
      echo "User $username added to group $group." | tee -a "$LOG_FILE"
    else
      echo "Error adding user $username to group $group." | tee -a "$LOG_FILE"
    fi
  done

  # Log if no additional groups are specified
  if [[ -z "$groups" ]]; then
    echo "No additional groups specified." | tee -a "$LOG_FILE"
  fi

For the final section of the function, a random 12-character password is generated and set for the user. The head command collects a stream of random bytes from the /dev/urandom file. This stream is piped to the tr command, which filters the bytes to include only alphanumeric characters (A-Z, a-z, 0-9) using the -dc flag. The filtered result is then piped to another head command, which selects only the first 12 characters from the edited stream. The password is then set by piping the user's name and the randomly generated password to the chpasswd command. The user and the generated password are saved in the designated password .csv file. If setting the password fails, the script deletes the user account and logs the error, to avoid any possible security risk.

# Generate random password, set it for the user, and store it in a file
  echo "Setting password for user $username..." | tee -a "$LOG_FILE"
  password=$(head /dev/urandom | tr -dc A-Za-z0-9 | head -c 12)
  echo "$username:$password" | chpasswd
  if [[ $? -eq 0 ]]; then
    echo "Password set for user $username." | tee -a "$LOG_FILE"
    echo "$username,$password" >> "$PASSWORD_FILE"
  else
    echo "Error setting password for user $username. Deleting $username user account" | tee -a "$LOG_FILE"
    userdel -r "$username"
    return 1
  fi

7.) After creating the create_user_account function, the script processes a file containing user information and creates user accounts accordingly.

# Process the user file
user_file="$1"
while IFS=';' read -r username groups; do
  if create_user_account "$username" "${groups%%[ ;]}"; then
    log_action "User account '$username' created successfully."
  else
    log_action "Error creating user account '$username'."
  fi
done < "$user_file"

The script takes the user file as its argument and assigns it to the variable user_file.
A while loop reads each line of the user file. The IFS=';' part sets the Internal Field Separator to a semicolon (;), splitting each line at the semicolon. The read -r username groups part reads the split parts into the username and groups variables.
For each line in the file, the script calls the create_user_account function with the username and the groups (with trailing spaces removed using ${groups%%[ ;]}). The script also logs a message if the result of the create_user_account was a success or failure.

8.) After all that is done, the script gives only the owner (root) and those with root privileges access to the password file, logs the completion of the script's execution and prints the log file and password file location.

# Keep password file accessible only to those with root privileges
chmod 600 "$PASSWORD_FILE"

# Log completion
log_action "User creation script completed."

# Print log file and password file location
echo "Check $LOG_FILE for details."
echo "Check $PASSWORD_FILE for user passwords."

Prerequisites for running the script

A Linux system
A bash terminal (Optional. You can use any available shell terminal on the Linux system).
Root privileges on your system.
The text file containing the usernames and groups, must be formatted as username;group1,group2.

To run the script:
1.) Copy the file from or clone the repository HNG_Stage1

2.) Copy the text file containing the usernames and groups to the folder where the script is located.

3.) Then, in the directory where both the script and the text file are now located, run sudo ./create_users.sh <text file>

Conclusion

This script simplifies some administrative tasks. With such a script, SysOps engineers can automate user and group creation and management, allowing them to focus on more critical work while ensuring efficient and secure user management.

This is one of the project assignments in the HNG Internship program designed to enhance your resume and deepen your knowledge of bash scripting. For the best experience, visit HNG Premium.

DEV Community: Great-Victor Anjorin

How Blocking Port 80 Made My Time API Project More Secure (And More Annoying)

Ingress Routing and Network Security Decisions Shaping Version of the Project

Validation Challenge

Implementation with Terraform

Conclusion

Confessions of a DevOps Noob Who Tried to pick a K8S Service CIDR from an Azure Subnet 🤦‍♂️

My Rookie Mistake

What a service_cidr is

The Code That Saved the Day

Other Relevant VPC Networking Points

Your Subnet Is the Real Deal for Nodes and Pods

service_cidr Is Just Kubernetes Playing Pretend

Pick Your service_cidr Carefully (or Pay the Price)

Load Balancing Ties It All Together

NAT Gateway: The Unsung Hero for Outbound Traffic

Key Takeaway

0 to K8s: How a Take-Home Assessment Took Over My Life 😭

🚧 The Task That Took Me Nearly a Year

🧠 What I Had to Learn (the Hard Way)

🎯 What Changed?

📝 Why I’m Writing This

🙌 Final Thoughts

Automating User Creation and Management with Bash: A Step-by-Step Guide

Line-by-Line Explanation

Conclusion

What a `service_cidr` is

`service_cidr` Is Just Kubernetes Playing Pretend

Pick Your `service_cidr` Carefully (or Pay the Price)