DEV Community: Desmond Goldsmith

Building a Production-Style CI/CD Pipeline with Azure DevOps, ACR, Helm, AKS and PostgreSQL

Desmond Goldsmith — Sun, 13 Sep 2026 23:19:58 +0000

Introduction

After learning Docker, Kubernetes, Helm, Azure and Azure DevOps separately, I wanted to put everything together into one practical project.

The goal was not to build a complicated application.

The goal was to understand what happens when application code moves from source control into a real deployment environment.

The final flow I wanted to understand was:

Developer
    |
    v
GitHub Repository
    |
    v
Azure DevOps
    |
    v
CI/CD Pipeline
    |
    +----------------------+
    |                      |
    v                      v
Build + Test          Docker Build
                           |
                           v
                  Azure Container Registry
                           |
                           v
                         Helm
                           |
                           v
                          AKS
                           |
                           v
                         DEV
                           |
                           v
                    Running Application

The application itself was intentionally simple:

Frontend
Backend
PostgreSQL database

The infrastructure around the application was where most of the learning happened.

I wanted to understand:

Azure Resource Groups
Azure Container Registry
Azure Kubernetes Service
Azure networking
Private networking
PostgreSQL Flexible Server
Private DNS
Docker
Kubernetes namespaces
Deployments
Services
ConfigMaps
Secrets
Helm
Azure DevOps
Azure service connections
Workload Identity Federation
Azure RBAC
Kubernetes RBAC
CI/CD
Troubleshooting

For this project I focused on DEV first.

UAT, PROD and approval gates will be added later.

1. The Final Architecture

The DEV environment ended up looking roughly like this:

                         GitHub
                           |
                           v
                    Azure DevOps
                           |
                           v
                  Build + Test + Docker
                           |
                           v
                   Azure Container
                      Registry
                           |
                           v
                         Helm
                           |
                           v
                  Azure Kubernetes
                       Service
                           |
              +------------+------------+
              |                         |
              v                         v
        frontend namespace        backend namespace
              |                         |
              v                         v
       Frontend Deployment       Backend Deployment
              |                         |
              v                         v
       LoadBalancer Service       ClusterIP Service
              |                         |
              v                         |
        Public IP / Browser          |
                                      v
                            PostgreSQL Flexible
                                 Server
                                      |
                                      v
                              Private Network
                                      |
                                      v
                              Private DNS Zone

The important design decision was that PostgreSQL was not installed inside AKS.

Instead, I used Azure Database for PostgreSQL Flexible Server.

That means AKS is responsible for running application workloads while Azure manages the database service.

2. The Project

The project repository is:

k8app-azure-devops

Repository:

https://github.com/Desmondgoldsmith/k8app-azure-devops

The repository contains the frontend, backend and Kubernetes/Helm configuration.

A simplified structure is:

k8app-azure-devops/
│
├── backend/
│   ├── app/
│   └── backend.dockerfile
│
├── frontend/
│   └── frontend.dockerfile
│
├── helm/
│   └── k8app/
│       ├── Chart.yaml
│       ├── values.yaml
│       └── templates/
│           ├── frontend-deployment.yaml
│           ├── frontend-service.yaml
│           ├── backend-deployment.yaml
│           ├── backend-service.yaml
│           ├── backend-configmap.yaml
│           └── backend-secret.yaml
│
└── azure-pipelines.yml

The exact application is less important than the infrastructure around it.

3. Azure Subscription and Region

I used an Azure for Students subscription.

The region I used was:

South Africa North

I chose the region partly because my student subscription had restrictions on which Azure regions I could use.

For example, some regions were unavailable under the subscription's policy.

Before creating resources, it is worth checking which regions your subscription allows.

You can do that from the Azure Portal when creating a resource, or with:

az account list-locations -o table

4. Create the Azure Resource Group

The first Azure resource I created was the Resource Group.

I used:

Resource Group:
devops-aks-lab-rg-sa

Region:
South Africa North

Why create a Resource Group?

A Resource Group is a logical container for related Azure resources.

Instead of having the project's resources scattered across Azure, I could put the main resources into one place:

devops-aks-lab-rg-sa
    |
    +-- ACR
    +-- AKS
    +-- PostgreSQL
    +-- Networking resources

This also makes permissions, management and cleanup easier.

Azure Portal

Go to:

Azure Portal
    ↓
Resource groups
    ↓
Create

Then select:

Subscription:
Azure for Students

Resource group:
devops-aks-lab-rg-sa

Region:
South Africa North

Click:

Review + create
    ↓
Create

Azure CLI

I could also create it using:

az login

Then:

az group create \
  --name devops-aks-lab-rg-sa \
  --location southafricanorth

Verify:

az group show \
  --name devops-aks-lab-rg-sa \
  -o table

5. Create Azure Container Registry

The next component was Azure Container Registry.

I created:

ACR name:
dessydevopsacr

The registry login server is:

dessydevopsacr.azurecr.io

Why do we need ACR?

The CI pipeline builds Docker images.

Those images need somewhere to live before Kubernetes can deploy them.

So ACR becomes the central image repository:

Azure DevOps
     |
     | Docker image
     v
Azure Container Registry
     |
     | Docker image
     v
AKS

For this project I had two images:

dessydevopsacr.azurecr.io/k8app-backend
dessydevopsacr.azurecr.io/k8app-frontend

Azure Portal

Go to:

Azure Portal
    ↓
Create a resource
    ↓
Search:
Container Registry
    ↓
Create

Under Basics, select:

Subscription:
Azure for Students

Resource group:
devops-aks-lab-rg-sa

Registry name:
dessydevopsacr

Location:
South Africa North

SKU:
Basic

For this project I kept the ACR admin account disabled.

Then:

Review + create
    ↓
Create

Azure CLI

Equivalent CLI:

az acr create \
  --resource-group devops-aks-lab-rg-sa \
  --name dessydevopsacr \
  --location southafricanorth \
  --sku Basic \
  --admin-enabled false

Verify:

az acr show \
  --name dessydevopsacr \
  --resource-group devops-aks-lab-rg-sa \
  --query "{name:name,loginServer:loginServer,sku:sku.name}" \
  -o table

Expected login server:

dessydevopsacr.azurecr.io

Why disable the ACR admin account?

ACR can provide a static username and password through its admin account.

I did not want the CI/CD pipeline to depend on that type of long-lived credential.

Instead, Azure DevOps uses a service connection.

That gives us a cleaner authentication model:

Azure DevOps
     |
     v
Service Connection
     |
     v
Azure / ACR

6. Create the Virtual Network

Because PostgreSQL was going to use private networking, networking needed to be considered before creating the database.

The basic idea was to have separate network areas for different workloads.

Conceptually:

Virtual Network
|
+-- AKS subnet
|
+-- PostgreSQL subnet

The PostgreSQL subnet we used was:

postgres-subnet
10.0.2.0/24

A /24 gives a defined block of private addresses for the database subnet.

The important reason for separating the database into its own subnet is that the database has a different purpose and security boundary from the application workloads.

Important note about the IP plan

The project definitely used:

postgres-subnet:
10.0.2.0/24

The AKS cluster also used:

Pod CIDR:
10.244.0.0/16

Service CIDR:
10.240.0.0/16

These are not the same thing as the Azure VNet subnet.

The exact Azure VNet/AKS subnet CIDR is not preserved in my project notes, so I am intentionally not inventing it here.

When reproducing the project, the network should be designed so the Azure VNet contains non-overlapping subnet ranges, for example:

VNet
|
+-- AKS subnet
|
+-- PostgreSQL subnet
       |
       +-- 10.0.2.0/24

The important rule is:

Do not create overlapping CIDR ranges.

For example, your VNet cannot have two different subnets that both contain the same addresses.

7. Create the PostgreSQL Subnet

In the Azure Portal, I created the subnet from the Virtual Network.

Go to:

Azure Portal
    ↓
Virtual networks
    ↓
Select your project VNet
    ↓
Subnets
    ↓
+ Subnet

Create:

Subnet name:
postgres-subnet

Subnet address range:
10.0.2.0/24

Save the subnet.

The purpose is to reserve a dedicated private network range for PostgreSQL.

8. Why PostgreSQL Was Private

I deliberately did not expose PostgreSQL to the public internet.

Instead:

AKS
 |
 | Private network
 v
PostgreSQL

rather than:

Internet
   |
   v
Public PostgreSQL endpoint

This is closer to how a production architecture would normally be designed.

The application workloads communicate with the database through private networking.

9. Create the Private DNS Zone

This was one of the concepts that took some time to understand.

Creating a private IP for PostgreSQL is not enough.

Applications normally connect to PostgreSQL using a hostname such as:

dessy-k8app-postgres.postgres.database.azure.com

The hostname needs to resolve to the private IP address.

That is where Private DNS comes in.

I created:

private.postgres.database.azure.com

What does Private DNS do?

Think of DNS as a phonebook.

The application knows:

dessy-k8app-postgres.postgres.database.azure.com

but the network ultimately needs an IP address.

Private DNS provides the mapping:

Hostname
    |
    v
Private DNS
    |
    v
Private IP

In this project, the PostgreSQL private IP resolved to:

10.0.2.4

So the flow became:

AKS Pod
   |
   | asks DNS:
   | "Where is
   | dessy-k8app-postgres.postgres.database.azure.com?"
   |
   v
Private DNS Zone
   |
   v
10.0.2.4
   |
   v
PostgreSQL

Without DNS resolution, an application could know the hostname but would not know where to send the network traffic.

10. Create the Private DNS Zone in Azure Portal

Go to:

Azure Portal
    ↓
Create a resource
    ↓
Search:
Private DNS zones
    ↓
Create

Set:

Resource group:
devops-aks-lab-rg-sa

Name:
private.postgres.database.azure.com

Click:

Review + create
    ↓
Create

After creating it, the DNS zone needs to be linked to the VNet.

Go to:

Private DNS zone
    ↓
Virtual network links
    ↓
+ Add

Select the VNet used by the application/database.

This allows resources in the VNet to use that private DNS zone.

11. Create Azure Database for PostgreSQL Flexible Server

I created:

PostgreSQL server:
dessy-k8app-postgres

Region:
South Africa North

PostgreSQL:
16

SKU:
Standard_B1ms

Database:
app

The hostname is:

dessy-k8app-postgres.postgres.database.azure.com

The server uses private networking.

Why use PostgreSQL Flexible Server?

Instead of running PostgreSQL inside Kubernetes:

AKS
 |
 +-- PostgreSQL Pod

I used:

AKS
 |
 | private network
 v
Azure PostgreSQL Flexible Server

This separates application compute from database infrastructure.

AKS can concentrate on running:

frontend
backend

while Azure manages the PostgreSQL service.

12. Create PostgreSQL in the Azure Portal

Go to:

Azure Portal
    ↓
Create a resource
    ↓
Search:
Azure Database for PostgreSQL Flexible Server
    ↓
Create

Under Basics, configure:

Subscription:
Azure for Students

Resource group:
devops-aks-lab-rg-sa

Server name:
dessy-k8app-postgres

Region:
South Africa North

PostgreSQL version:
16

Workload type:
Development / appropriate low-cost option

Compute:
Standard_B1ms

Configure the administrator:

Administrator username:
k8appadmin

Administrator password:
<your password>

Do not publish the password.

13. Configure PostgreSQL Networking

During PostgreSQL creation, select private access rather than public access.

The database needs to use the VNet and the dedicated subnet:

Virtual network:
<project VNet>

Subnet:
postgres-subnet

The private DNS configuration should use:

private.postgres.database.azure.com

The important relationship is:

PostgreSQL
    |
    v
postgres-subnet
10.0.2.0/24
    |
    v
Private DNS
private.postgres.database.azure.com

Azure then creates the private networking relationship.

14. Create the Application Database

After creating the server, I created the database:

app

From the PostgreSQL resource, go to:

PostgreSQL Flexible Server
    ↓
Databases
    ↓
+ Add

Create:

Database name:
app

The application will later connect using:

Server:
dessy-k8app-postgres.postgres.database.azure.com

Database:
app

User:
k8appadmin

The password is kept secret.

15. Verify PostgreSQL Networking From AKS

Creating the database is not enough.

I wanted to answer a more important question:

Can a workload running inside AKS actually reach PostgreSQL?

I tested this from inside Kubernetes.

First I checked DNS resolution.

kubectl run psql-test \
  --rm -it \
  --image=postgres:16 \
  --restart=Never \
  -- \
  getent hosts dessy-k8app-postgres.postgres.database.azure.com

The hostname resolved to the private PostgreSQL IP.

In our case:

10.0.2.4

This proved that the private DNS path was working.

16. Test the PostgreSQL Connection

I then used a temporary PostgreSQL client pod.

kubectl run psql-test \
  --rm -it \
  --image=postgres:16 \
  --restart=Never \
  -- \
  psql "host=dessy-k8app-postgres.postgres.database.azure.com port=5432 dbname=app user=k8appadmin sslmode=require"

When prompted, I entered the database password.

The connection succeeded over TLS.

That proved this path was working:

AKS Pod
   |
   v
Private DNS
   |
   v
Private IP
   |
   v
PostgreSQL Flexible Server

This was an important test because it separated database/networking problems from application problems.

17. Create the AKS Cluster

The next major component was Azure Kubernetes Service.

I created:

AKS:
dessy-aks-cluster

Resource Group:
devops-aks-lab-rg-sa

Region:
South Africa North

The cluster used:

Azure CNI Overlay

Pod CIDR:
10.244.0.0/16

Service CIDR:
10.240.0.0/16

Load Balancer:
Standard

I also enabled Microsoft Entra integration and Kubernetes RBAC.

18. Why Azure CNI Overlay?

AKS networking can be configured in different ways.

For this project I used Azure CNI Overlay.

One important distinction is that these CIDRs:

10.244.0.0/16
10.240.0.0/16

are Kubernetes networking ranges.

They are different from the Azure VNet subnet ranges.

I found this distinction important when learning AKS networking because it is easy to look at an IP such as:

10.244.x.x

and assume it is an Azure VNet IP.

It is not.

The simplified model is:

Azure VNet
    |
    +-- Azure subnet
          |
          +-- AKS nodes
                |
                +-- Kubernetes pod networking
                |
                +-- Kubernetes service networking

19. Create AKS in Azure Portal

Go to:

Azure Portal
    ↓
Create a resource
    ↓
Search:
Kubernetes Service
    ↓
Create

Under Basics, configure:

Subscription:
Azure for Students

Resource group:
devops-aks-lab-rg-sa

Cluster preset:
appropriate development option

Kubernetes cluster name:
dessy-aks-cluster

Region:
South Africa North

Configure the node pool with one node for this learning environment.

20. Configure AKS Identity

Under the identity/security configuration, use:

Managed identity

rather than manually managing long-lived credentials.

This allows Azure resources to work with managed identities.

21. Configure Microsoft Entra ID and Kubernetes RBAC

I enabled Microsoft Entra integration.

The cluster uses Microsoft Entra ID for identity and Kubernetes RBAC for authorization.

This became an important lesson later.

Authentication and authorization are not the same thing.

Think about them as:

Microsoft Entra ID
       |
       | Who are you?
       v
Authentication
       |
       v
Kubernetes RBAC
       |
       | What are you allowed to do?
       v
Authorization

22. Configure AKS Networking

For the cluster networking, I used:

Network plugin:
Azure CNI

Network plugin mode:
Overlay

Pod CIDR:
10.244.0.0/16

Service CIDR:
10.240.0.0/16

I also used a Standard Load Balancer.

This is important because the frontend Kubernetes Service was later configured as:

type: LoadBalancer

Azure then provisions an Azure Load Balancer/public IP for that service.

23. Enable OIDC and Workload Identity

I enabled:

OIDC issuer
Workload identity

These capabilities are useful for modern identity-based authentication between Kubernetes workloads and Azure services.

They also make it possible to avoid embedding long-lived Azure credentials inside applications.

24. Connect kubectl to AKS

After the cluster was created, I retrieved its credentials:

az aks get-credentials \
  --resource-group devops-aks-lab-rg-sa \
  --name dessy-aks-cluster \
  --overwrite-existing

Then:

kubectl get nodes

I expected the node to show:

Ready

This confirmed that my local machine could communicate with the cluster.

25. AKS Identity and ACR

There are several identities involved in this project.

This was one of the most important things I learned.

The AKS cluster has a kubelet managed identity.

I checked it with:

az aks show \
  --resource-group devops-aks-lab-rg-sa \
  --name dessy-aks-cluster \
  --query identityProfile.kubeletidentity.clientId \
  -o tsv

The kubelet identity needs permission to pull images from ACR.

The permission required is:

AcrPull

The flow is:

AKS Kubelet Identity
        |
        | AcrPull
        v
Azure Container Registry
        |
        v
Docker Image

26. Give AKS AcrPull Permission

In Azure Portal, go to:

Container Registry
    ↓
dessydevopsacr
    ↓
Access control (IAM)
    ↓
Role assignments

Add a role assignment.

Select:

Role:
AcrPull

Assign it to the AKS kubelet managed identity.

After this:

AKS
 |
 | AcrPull
 v
ACR
 |
 v
k8app-backend image
k8app-frontend image

The important distinction is:

Azure DevOps pushes the image. AKS pulls the image.

These are two different operations and can use two different identities.

27. Understanding the Three Identity Layers

This project involved three different concepts.

OIDC / Workload Identity

This answers:

Who are you?

For Azure DevOps:

Azure DevOps
      |
      v
OIDC / Workload Identity Federation
      |
      v
Microsoft Entra ID

Azure RBAC

This answers:

What can you do in Azure?

For example:

Azure DevOps identity
       |
       v
Contributor
       |
       v
Resource Group

Kubernetes RBAC

This answers:

What can you do inside Kubernetes?

For example:

Azure DevOps identity
       |
       v
Kubernetes RoleBinding
       |
       v
edit
       |
       +-- frontend namespace
       |
       +-- backend namespace

The simplified mental model is:

OIDC
 ↓
Who are you?

Azure RBAC
 ↓
What can you do in Azure?

Kubernetes RBAC
 ↓
What can you do in Kubernetes?

28. Create the Azure DevOps Project

I created an Azure DevOps project:

DessyTest-1

Go to:

Azure DevOps
    ↓
New Project

Create:

Project name:
DessyTest-1

The GitHub repository remains the source repository.

Azure DevOps is being used for the CI/CD pipeline.

The conceptual relationship is:

GitHub
   |
   v
Azure DevOps
   |
   v
Pipeline

29. Connect GitHub to Azure DevOps

Inside Azure DevOps, create/configure the GitHub connection.

Go to:

Azure DevOps
    ↓
Project Settings
    ↓
Service connections

Create the GitHub connection and authorize the repository.

The purpose is to allow Azure DevOps to retrieve the repository source code.

30. Create the ACR Service Connection

Azure DevOps needs a way to authenticate to ACR.

Go to:

Azure DevOps
    ↓
Project Settings
    ↓
Service connections
    ↓
New service connection

Select:

Docker Registry

Then configure it for Azure Container Registry.

I created the service connection:

dessydevopsacr

This connection is used by the Docker task in the pipeline.

Conceptually:

Azure DevOps
     |
     v
ACR Service Connection
     |
     v
Azure Container Registry

31. Create the AKS Azure Service Connection

The deployment stage needs Azure access so it can obtain AKS credentials.

I created:

dessy-aks-deploy

Go to:

Azure DevOps
    ↓
Project Settings
    ↓
Service connections
    ↓
New service connection

Select the Azure Resource Manager connection type.

Use:

Workload Identity Federation

rather than a long-lived client secret.

This gives:

Azure DevOps
      |
      v
OIDC / Workload Identity Federation
      |
      v
Microsoft Entra ID
      |
      v
Azure

32. Give the Azure DevOps Identity Azure Permissions

The service principal used by the deployment service connection needs permission to work with the Azure resources.

For this learning environment, I assigned:

Contributor

at the resource group scope:

devops-aks-lab-rg-sa

In Azure Portal:

Resource Group
    ↓
devops-aks-lab-rg-sa
    ↓
Access control (IAM)
    ↓
Add role assignment

Select:

Contributor

Then select the service principal used by:

dessy-aks-deploy

This was intentionally broad for the learning environment.

A production environment should use more restrictive permissions where practical.

33. Kubernetes RBAC for Azure DevOps

Getting Azure permissions was not enough.

At one point the pipeline could authenticate to Azure but Kubernetes still rejected some operations.

That taught me an important lesson:

Azure authentication and Kubernetes authorization are separate problems.

The Azure DevOps service principal was:

<AZURE_DEVOPS_SERVICE_PRINCIPAL_OBJECT_ID>

I created a RoleBinding in the:

frontend

namespace and another in:

backend

The binding gives the identity the Kubernetes:

edit

ClusterRole.

Example:

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: azure-devops-deployer
  namespace: backend
subjects:
  - kind: User
    name: <AZURE_DEVOPS_SERVICE_PRINCIPAL_OBJECT_ID>
    apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: ClusterRole
  name: edit
  apiGroup: rbac.authorization.k8s.io

The frontend binding is the same except for:

namespace: frontend

34. Test Kubernetes RBAC

I tested whether the Azure DevOps identity could create deployments.

For backend:

kubectl auth can-i create deployments \
  --as=<AZURE_DEVOPS_SERVICE_PRINCIPAL_OBJECT_ID> \
  -n backend

For frontend:

kubectl auth can-i create deployments \
  --as=<AZURE_DEVOPS_SERVICE_PRINCIPAL_OBJECT_ID> \
  -n frontend

Both returned:

yes

I deliberately did not give the pipeline:

cluster-admin

because that would give it unnecessarily broad access to the whole cluster.

35. Create the Kubernetes Namespaces

I separated the frontend and backend workloads.

The namespaces were:

frontend
backend

Create them:

kubectl create namespace frontend
kubectl create namespace backend

Verify:

kubectl get namespaces

The reason for separate namespaces is organization and isolation.

Instead of:

default
 |
 +-- frontend
 +-- backend
 +-- everything else

we have:

frontend namespace
 |
 +-- frontend Deployment
 +-- frontend Service

backend namespace
 |
 +-- backend Deployment
 +-- backend Service
 +-- ConfigMap
 +-- Secret

It also gives us a clean foundation for namespace-level RBAC.

36. Why Kubernetes Deployment?

A Deployment describes how Kubernetes should run an application.

For example:

kind: Deployment

tells Kubernetes:

I want this application running.

It controls things such as:

Number of replicas
Container image
Container ports
Pod labels
Updates

For example:

replicas: 1

means we currently want one pod.

37. Why Kubernetes Service?

Pods are temporary.

Their IP addresses can change.

A Kubernetes Service gives applications a stable way to communicate with a group of Pods.

The model is:

Pod
 |
 | temporary IP
 v
Service
 |
 | stable name/IP
 v
Other application

There are two different Services in this project.

Frontend:

LoadBalancer

Backend:

ClusterIP

38. Frontend Service

The frontend Service is:

apiVersion: v1

kind: Service
metadata:
  name: k8app-frontend
  namespace: frontend
spec:
  type: LoadBalancer
  selector:
    app: k8app-frontend
  ports:
    - port: 80
      targetPort: 80

The important part is:

type: LoadBalancer

This tells AKS that the service should be externally reachable.

The flow is:

Browser
   |
   v
Azure Load Balancer
   |
   v
k8app-frontend Service
   |
   v
Frontend Pod

39. Backend Service

The backend Service is:

apiVersion: v1
kind: Service
metadata:
  name: k8app-backend
  namespace: backend
spec:
  type: ClusterIP
  selector:
    app: k8app-backend
  ports:
    - port: 80
      targetPort: 80

The important part is:

type: ClusterIP

ClusterIP means the service is intended to be reachable internally within Kubernetes.

Conceptually:

Frontend / internal caller
        |
        v
Backend ClusterIP
        |
        v
Backend Pod

40. Create the Helm Chart

Instead of maintaining raw Kubernetes YAML for every environment, I used Helm.

The chart is:

helm/k8app

Create the directory structure:

helm/
└── k8app/
    ├── Chart.yaml
    ├── values.yaml
    └── templates/

41. Create Chart.yaml

The chart definition is:

apiVersion: v2
name: k8app
description: Helm chart for the K8App frontend and backend
type: application
version: 0.1.0
appVersion: "1.0.0"

The important concept is that Helm packages our Kubernetes configuration into one deployable unit.

Instead of manually doing:

kubectl apply -f ...
kubectl apply -f ...
kubectl apply -f ...

we can do:

helm upgrade --install ...

42. Create values.yaml

The values file contains environment-specific values.

frontend:
  replicas: 1

  image:
    repository: dessydevopsacr.azurecr.io/k8app-frontend
    tag: ""

backend:
  replicas: 1

  image:
    repository: dessydevopsacr.io/k8app-backend
    tag: ""

  config:
    projectName: "k8app"
    postgresServer: "dessy-k8app-postgres.postgres.database.azure.com"
    postgresDb: "app"

  secret:
    postgresUser: "k8appadmin"
    postgresPassword: ""

The correct backend registry is:

dessydevopsacr.azurecr.io

so the backend repository should be:

repository: dessydevopsacr.azurecr.io/k8app-backend

The final version is therefore:

frontend:
  replicas: 1

  image:
    repository: dessydevopsacr.azurecr.io/k8app-frontend
    tag: ""

backend:
  replicas: 1

  image:
    repository: dessydevopsacr.azurecr.io/k8app-backend
    tag: ""

  config:
    projectName: "k8app"
    postgresServer: "dessy-k8app-postgres.postgres.database.azure.com"
    postgresDb: "app"

  secret:
    postgresUser: "k8appadmin"
    postgresPassword: ""

Notice that the password is blank.

That is intentional.

The pipeline supplies it securely during deployment.

43. Why Helm Values Exist

The values file lets us separate:

Application template

from:

Environment values

For example:

image:
  repository: ...
  tag: ""

The chart doesn't need to know which pipeline build is being deployed.

The pipeline can provide:

--set backend.image.tag=$(Build.BuildId)

and:

--set frontend.image.tag=$(Build.BuildId)

So:

Pipeline Build ID
       |
       v
Docker image tag
       |
       v
Helm values
       |
       v
Kubernetes Deployment

This creates a traceable relationship between a pipeline run and the deployed image.

44. Frontend Deployment

The frontend Deployment is:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: k8app-frontend
  namespace: frontend
spec:
  replicas: {{ .Values.frontend.replicas }}
  selector:
    matchLabels:
      app: k8app-frontend
  template:
    metadata:
      labels:
        app: k8app-frontend
    spec:
      containers:
        - name: frontend
          image: "{{ .Values.frontend.image.repository }}:{{ .Values.frontend.image.tag }}"
          ports:
            - containerPort: 80

The important Helm expression is:

{{ .Values.frontend.image.tag }}

Helm replaces this with the actual value during deployment.

For example, if the pipeline passes:

--set frontend.image.tag=21

the resulting Kubernetes image becomes:

dessydevopsacr.azurecr.io/k8app-frontend:21

45. Backend Deployment

The backend Deployment contains the container and environment configuration.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: k8app-backend
  namespace: backend
spec:
  replicas: {{ .Values.backend.replicas }}
  selector:
    matchLabels:
      app: k8app-backend
  template:
    metadata:
      labels:
        app: k8app-backend
    spec:
      containers:
        - name: backend
          image: "{{ .Values.backend.image.repository }}:{{ .Values.backend.image.tag }}"
          ports:
            - containerPort: 80
          env:
            - name: PROJECT_NAME
              valueFrom:
                configMapKeyRef:
                  name: k8app-backend-config
                  key: PROJECT_NAME

            - name: POSTGRES_SERVER
              valueFrom:
                configMapKeyRef:
                  name: k8app-backend-config
                  key: POSTGRES_SERVER

            - name: POSTGRES_DB
              valueFrom:
                configMapKeyRef:
                  name: k8app-backend-config
                  key: POSTGRES_DB

            - name: POSTGRES_USER
              valueFrom:
                secretKeyRef:
                  name: k8app-backend-secret
                  key: POSTGRES_USER

            - name: POSTGRES_PASSWORD
              valueFrom:
                secretKeyRef:
                  name: k8app-backend-secret
                  key: POSTGRES_PASSWORD

This is where ConfigMaps and Secrets become important.

46. What Is a ConfigMap?

A ConfigMap stores non-sensitive configuration.

For this application, things such as:

PROJECT_NAME
POSTGRES_SERVER
POSTGRES_DB

do not need to be treated as passwords.

So I created:

k8app-backend-config

47. Create the ConfigMap

The Helm template is:

apiVersion: v1
kind: ConfigMap
metadata:
  name: k8app-backend-config
  namespace: backend
data:
  PROJECT_NAME: {{ .Values.backend.config.projectName | quote }}
  POSTGRES_SERVER: {{ .Values.backend.config.postgresServer | quote }}
  POSTGRES_DB: {{ .Values.backend.config.postgresDb | quote }}

The resulting Kubernetes object looks conceptually like:

k8app-backend-config
|
+-- PROJECT_NAME=k8app
+-- POSTGRES_SERVER=dessy-k8app-postgres.postgres.database.azure.com
+-- POSTGRES_DB=app

You can inspect it with:

kubectl get configmap k8app-backend-config -n backend

Or:

kubectl describe configmap k8app-backend-config -n backend

48. What Is a Kubernetes Secret?

A Secret is intended for sensitive configuration.

In our case:

POSTGRES_USER
POSTGRES_PASSWORD

The password should not be stored directly in:

Deployment YAML
values.yaml
Git

Instead, I created:

k8app-backend-secret

49. Create the Kubernetes Secret

The Helm template is:

apiVersion: v1
kind: Secret
metadata:
  name: k8app-backend-secret
  namespace: backend
type: Opaque
stringData:
  POSTGRES_USER: {{ .Values.backend.secret.postgresUser | quote }}
  POSTGRES_PASSWORD: {{ .Values.backend.secret.postgresPassword | quote }}

The password is supplied at deployment time.

I did not commit the actual password to Git.

50. How configMapKeyRef Works

This part initially looked confusing:

valueFrom:
  configMapKeyRef:
    name: k8app-backend-config
    key: POSTGRES_SERVER

But it is actually simple.

It means:

Go to the ConfigMap named k8app-backend-config, find the key called POSTGRES_SERVER, and inject its value into the container as an environment variable named POSTGRES_SERVER.

So:

ConfigMap
k8app-backend-config
       |
       +-- POSTGRES_SERVER
                |
                v
        Backend container
                |
                v
        POSTGRES_SERVER

The same thing happens for:

PROJECT_NAME
POSTGRES_DB

51. How secretKeyRef Works

This:

valueFrom:
  secretKeyRef:
    name: k8app-backend-secret
    key: POSTGRES_PASSWORD

means:

Go to the Secret named k8app-backend-secret, find the POSTGRES_PASSWORD key, and inject its value into the container as the POSTGRES_PASSWORD environment variable.

So:

Kubernetes Secret
       |
       +-- POSTGRES_PASSWORD
                |
                v
        Backend container
                |
                v
        POSTGRES_PASSWORD

This keeps sensitive configuration separate from normal configuration.

52. Why the Backend Initially Failed

Before the ConfigMap and Secret were configured correctly, the backend container crashed.

The backend expected environment variables such as:

PROJECT_NAME
POSTGRES_SERVER
POSTGRES_USER
POSTGRES_PASSWORD
POSTGRES_DB

The application's configuration code used environment-based configuration.

Without those variables, the application could not construct its database configuration.

This is an example of why logs are important.

Instead of guessing, I checked:

kubectl logs <backend-pod> -n backend

The logs showed that required configuration values were missing.

That led to:

Backend Crash
     |
     v
Check Pod logs
     |
     v
Missing environment variables
     |
     v
Create ConfigMap + Secret
     |
     v
Reference them from Deployment
     |
     v
Backend starts

53. Verify the Backend Configuration

Check the ConfigMap:

kubectl get configmap -n backend

Check the Secret:

kubectl get secret -n backend

Check the Deployment:

kubectl get deployment k8app-backend -n backend

Check the Pod:

kubectl get pods -n backend

If the pod is running:

1/1 Running

then the application container has successfully started.

54. Verify the Backend Logs

Run:

kubectl logs deployment/k8app-backend -n backend

If something fails, I normally investigate in this order:

kubectl get pods
        |
        v
kubectl describe pod
        |
        v
kubectl logs
        |
        v
Check ConfigMap / Secret
        |
        v
Check Service
        |
        v
Check database/network

This is much better than randomly changing YAML.

55. Verify the Database From the Backend Namespace

Because the backend needs PostgreSQL, I verified that the backend environment contained the correct configuration.

The database host is:

dessy-k8app-postgres.postgres.database.azure.com

The database is:

app

The username is:

k8appadmin

The password remains secret.

56. Validate the Helm Chart Before Deploying

One of the most useful Helm commands is:

helm template test-release ./helm/k8app

This does not deploy anything.

It renders the Helm templates into Kubernetes YAML.

This allows us to catch template problems before sending anything to AKS.

For example:

helm template test-release ./helm/k8app

If there is a problem such as:

nil pointer evaluating interface {}.create

it usually means the template is trying to access a value that doesn't exist in values.yaml.

This is one reason I use:

helm template

before:

helm upgrade

57. Deploy With Helm

The main deployment command is:

helm upgrade --install k8app helm/k8app \
  --namespace backend \
  --set backend.image.tag=21 \
  --set frontend.image.tag=21 \
  --wait \
  --timeout 5m

The number:

was the image tag for that pipeline/build.

In the real pipeline, we don't manually type this.

Azure DevOps provides:

$(Build.BuildId)

So the pipeline can automatically deploy the image produced by that build.

58. Why Helm Release Is in the Backend Namespace

One slightly unusual part of this design is that the Helm release is installed using:

--namespace backend

while the chart itself contains resources in both:

frontend
backend

Therefore:

helm list -n backend

shows:

k8app

while:

helm list -n frontend

does not.

That does not mean the frontend wasn't deployed.

It means the Helm release itself is stored/tracked in the backend namespace while the chart creates resources in both namespaces.

59. Verify the Helm Release

Run:

helm list -n backend

Expected:

NAME    NAMESPACE  REVISION  STATUS    CHART
k8app   backend    3         deployed  k8app-0.1.0

You can also inspect:

helm status k8app -n backend

60. Verify Kubernetes Deployments

Check frontend:

kubectl get deployments -n frontend

Check backend:

kubectl get deployments -n backend

Expected conceptually:

frontend
k8app-frontend

backend
k8app-backend

61. Verify Pods

Run:

kubectl get pods -n frontend

and:

kubectl get pods -n backend

Healthy pods should show:

1/1 Running

For example:

k8app-frontend-xxxxxxxxxx   1/1   Running

and:

k8app-backend-xxxxxxxxxx    1/1   Running

62. Verify Services

Frontend:

kubectl get service -n frontend

Backend:

kubectl get service -n backend

The frontend should be:

LoadBalancer

while the backend should be:

ClusterIP

63. The LoadBalancer Problem

Initially, the frontend Service showed:

EXTERNAL-IP   <pending>

At first it was tempting to think Kubernetes was broken.

Instead, I used:

kubectl describe service k8app-frontend -n frontend

The important event was:

PublicIPCountLimitReached

The subscription had reached the regional public IP limit.

The message indicated that the subscription could not create more than three public IP addresses in that region.

This was a good example of why:

kubectl describe

is useful.

kubectl get tells us what is happening.

kubectl describe often helps explain why.

64. Troubleshooting the Public IP Limit

I checked existing AKS resources.

One older AKS cluster was:

rest-go-aks

I checked its state:

az aks show \
  --resource-group devops-aks-lab-rg-sa \
  --name rest-go-aks \
  --query "{name:name,powerState:powerState.code}" \
  -o table

It showed:

Name         PowerState
------------  ----------
rest-go-aks  Stopped

The old cluster was no longer needed for this project.

I removed it:

az aks delete \
  --resource-group devops-aks-lab-rg-sa \
  --name rest-go-aks \
  --yes \
  --no-wait

Then I checked the public IP resources:

az network public-ip list \
  --query "[?location=='southafricanorth'].{Name:name,ResourceGroup:resourceGroup,IP:ipAddress}" \
  -o table

The old resources were gone.

The frontend LoadBalancer was then able to obtain a public IP.

65. Final Frontend Public IP

The frontend Service eventually received:

4.253.76.190

The browser application became accessible through:

http://4.253.76.190

The final flow was therefore:

Browser
   |
   v
4.253.76.190
   |
   v
Azure Load Balancer
   |
   v
k8app-frontend Service
   |
   v
Frontend Pod

66. Important Lesson From the LoadBalancer Failure

The Kubernetes Service itself was not necessarily the problem.

The chain was:

Service
   |
   v
Azure Load Balancer
   |
   v
Azure Public IP

The Azure subscription had reached its public IP quota.

So the correct troubleshooting process was:

EXTERNAL-IP pending
        |
        v
kubectl describe service
        |
        v
PublicIPCountLimitReached
        |
        v
Check Azure resources
        |
        v
Find unused AKS cluster
        |
        v
Delete unused cluster
        |
        v
LoadBalancer gets public IP

This was one of the most useful troubleshooting lessons in the project.

67. Azure DevOps Variable Group

The database password should not be placed directly in the pipeline YAML.

So I created a variable group:

k8app-dev-secrets

Inside it:

POSTGRES_PASSWORD

The variable was marked as secret.

68. Create the Variable Group in Azure DevOps

Go to:

Azure DevOps
    ↓
Pipelines
    ↓
Library
    ↓
+ Variable group

Create:

Variable group name:
k8app-dev-secrets

Add:

Name:
POSTGRES_PASSWORD

Value:
<your PostgreSQL password>

Enable:

Keep this value secret

Save the variable group.

69. Why the Pipeline Uses `- group`

The pipeline contains:

variables:
- group: k8app-dev-secrets

This means:

Load the variables stored inside the Azure DevOps variable group named k8app-dev-secrets into this pipeline.

Later, the pipeline can reference:

$(POSTGRES_PASSWORD)

So the flow is:

Azure DevOps Library
       |
       v
k8app-dev-secrets
       |
       +-- POSTGRES_PASSWORD
       |
       v
Azure DevOps Pipeline
       |
       v
Helm
       |
       v
Kubernetes Secret
       |
       v
Backend Pod

The actual password should never be committed to Git.

70. The Azure DevOps Pipeline

The pipeline builds both images and then deploys them to DEV.

The important variables are:

variables:
- group: k8app-dev-secrets

- name: dockerRegistryServiceConnection
  value: 'a41b9e21-7348-4152-b6ca-249e8c78aae9'

- name: containerRegistry
  value: 'dessydevopsacr.azurecr.io'

- name: vmImageName
  value: 'ubuntu-latest'

- name: tag
  value: '$(Build.BuildId)'

The important part is:

tag: '$(Build.BuildId)'

This means every pipeline run can produce a unique image tag.

For example:

Build ID = 21

produces:

dessydevopsacr.azurecr.io/k8app-backend:21
dessydevopsacr.io/k8app-frontend:21

The correct registry hostname is:

dessydevopsacr.azurecr.io

71. Build Backend Image

The backend job uses Docker@2.

Conceptually:

- task: Docker@2
  inputs:
    command: buildAndPush
    containerRegistry: $(dockerRegistryServiceConnection)
    repository: k8app-backend
    dockerfile: backend/backend.dockerfile
    buildContext: backend
    tags: |
      $(tag)

The important pieces are:

Dockerfile:
backend/backend.dockerfile

Build context:
backend

Repository:
k8app-backend

Tag:
Build.BuildId

The result goes to:

dessydevopsacr.azurecr.io/k8app-backend:<BuildID>

72. Build Frontend Image

The frontend job does the same thing.

- task: Docker@2
  inputs:
    command: buildAndPush
    containerRegistry: $(dockerRegistryServiceConnection)
    repository: k8app-frontend
    dockerfile: frontend/frontend.dockerfile
    buildContext: frontend
    tags: |
      $(tag)

The result is:

dessydevopsacr.azurecr.io/k8app-frontend:<BuildID>

73. Deploy Stage

After the Build stage succeeds, the pipeline runs:

DeployDEV

The deployment uses:

dessy-aks-deploy

as its Azure service connection.

The pipeline obtains AKS credentials:

az aks get-credentials \
  --resource-group devops-aks-lab-rg-sa \
  --name dessy-aks-cluster \
  --overwrite-existing

Then Helm performs the deployment.

74. Helm Deployment From Azure DevOps

The deployment command is:

helm upgrade --install k8app helm/k8app \
  --namespace backend \
  --set frontend.image.tag=$(Build.BuildId) \
  --set backend.image.tag=$(Build.BuildId) \
  --set backend.secret.postgresPassword="$(POSTGRES_PASSWORD)" \
  --wait \
  --timeout 5m

Let's break this down.

`helm upgrade --install`

helm upgrade --install

means:

If the release exists, upgrade it. If it doesn't exist, install it.

This makes the pipeline reusable.

Release name

k8app

This is the Helm release name.

Chart

helm/k8app

This tells Helm where the chart is.

Namespace

--namespace backend

This is where the Helm release is tracked.

Frontend image

--set frontend.image.tag=$(Build.BuildId)

The pipeline automatically passes the current build ID into Helm.

Backend image

--set backend.image.tag=$(Build.BuildId)

The backend receives the same build ID.

That means one pipeline run corresponds to the same application version:

Build 21
 |
 +-- frontend:21
 |
 +-- backend:21

PostgreSQL password

--set backend.secret.postgresPassword="$(POSTGRES_PASSWORD)"

The password comes from the Azure DevOps variable group.

It is then passed into the Helm chart and rendered into the Kubernetes Secret.

75. The Complete Image Flow

This is the relationship I wanted to understand:

Azure DevOps Build
       |
       | Build.BuildId = 21
       |
       +--------------------+
       |                    |
       v                    v
frontend:21            backend:21
       |                    |
       +---------+----------+
                 |
                 v
               ACR
                 |
                 v
               Helm
                 |
                 v
                AKS

This gives us traceability between:

Pipeline run
      ↓
Docker image
      ↓
Helm release
      ↓
Kubernetes deployment
      ↓
Running application

76. Final Pipeline Structure

The pipeline has two main stages:

stages:

- stage: Build
  displayName: Build and Push Images

- stage: DeployDEV
  displayName: Deploy to DEV
  dependsOn: Build

The relationship is:

Build
  |
  | success
  v
DeployDEV

The deployment cannot start until the build stage has succeeded.

77. Verify the Deployment From the Pipeline

After Helm deployment, I verify:

kubectl get deployments -n frontend
kubectl get deployments -n backend

kubectl get services -n frontend
kubectl get services -n backend

I also use:

kubectl get pods -n frontend
kubectl get pods -n backend

The objective is:

Frontend Pod     Running
Backend Pod      Running
Frontend Service LoadBalancer
Backend Service  ClusterIP

78. Useful Kubernetes Debugging Commands

When a deployment fails, these are some of the commands I use most often.

List Pods

kubectl get pods -n frontend

kubectl get pods -n backend

Check Pod Details

kubectl describe pod <pod-name> -n backend

This is useful for:

Scheduling problems
Image pull errors
Environment problems
Mount problems
Events

Check Logs

kubectl logs <pod-name> -n backend

For the previous crashed container:

kubectl logs <pod-name> -n backend --previous

Check Deployment

kubectl describe deployment k8app-backend -n backend

Check Services

kubectl get services -n backend

kubectl describe service k8app-frontend -n frontend

Check ConfigMaps

kubectl get configmaps -n backend

kubectl describe configmap k8app-backend-config -n backend

Check Secrets

kubectl get secrets -n backend

Do not casually print secret values into logs or screenshots.

79. Useful Helm Debugging Commands

Render the chart:

helm template test-release helm/k8app

Render with debug:

helm template test-release helm/k8app --debug

List releases:

helm list -n backend

Check release:

helm status k8app -n backend

Check release history:

helm history k8app -n backend

Upgrade:

helm upgrade k8app helm/k8app \
  --namespace backend \
  --set backend.image.tag=21 \
  --set frontend.image.tag=21 \
  --wait \
  --timeout 5m

80. Why `helm template` Is So Useful

One of the biggest Helm lessons for me was that Helm is essentially a templating and release-management layer around Kubernetes manifests.

For example:

image: "{{ .Values.backend.image.repository }}:{{ .Values.backend.image.tag }}"

is not valid final Kubernetes configuration until Helm renders it.

Running:

helm template

lets me see what Kubernetes will actually receive.

The debugging process becomes:

Helm template
     |
     v
Rendered YAML
     |
     v
Check values
     |
     v
Fix template
     |
     v
Deploy

81. Verify the ACR Images

From Azure CLI:

az acr repository list \
  --name dessydevopsacr \
  -o table

You should see repositories such as:

k8app-backend
k8app-frontend

List tags for the backend:

az acr repository show-tags \
  --name dessydevopsacr \
  --repository k8app-backend \
  -o table

Frontend:

az acr repository show-tags \
  --name dessydevopsacr \
  --repository k8app-frontend \
  -o table

This helps verify that the CI stage actually pushed the images.

82. Verify Which Image Kubernetes Is Running

You can inspect the Deployment:

kubectl get deployment k8app-backend \
  -n backend \
  -o jsonpath='{.spec.template.spec.containers[0].image}'

And frontend:

kubectl get deployment k8app-frontend \
  -n frontend \
  -o jsonpath='{.spec.template.spec.containers[0].image}'

This should show the expected Build ID.

For example:

dessydevopsacr.azurecr.io/k8app-backend:21

This is useful when troubleshooting:

Did the pipeline build the correct image, or is AKS still running an older version?

83. Verify the Application

After the deployment succeeds:

kubectl get service k8app-frontend -n frontend

Get the external IP.

Then open the IP in a browser.

For this deployment, the frontend became accessible through:

http://4.253.76.190

The application loaded successfully.

That gave me the final checkpoint:

GitHub
   ↓
Azure DevOps
   ↓
Build
   ↓
Docker
   ↓
ACR
   ↓
Helm
   ↓
AKS
   ↓
Frontend
   ↓
Running Application

84. The Complete DEV Environment

At the end, the important Azure resources were:

Resource Group
devops-aks-lab-rg-sa

ACR
dessydevopsacr

AKS
dessy-aks-cluster

PostgreSQL
dessy-k8app-postgres

PostgreSQL subnet
postgres-subnet
10.0.2.0/24

Private DNS
private.postgres.database.azure.com

The Kubernetes namespaces were:

frontend
backend

The Helm release was:

k8app

The container images were:

dessydevopsacr.azurecr.io/k8app-frontend:<BuildID>

dessydevopsacr.azurecr.io/k8app-backend:<BuildID>

85. A Simple Way to Think About the Whole System

After building the project, this is the mental model I use:

SOURCE CODE
    |
    v
Azure DevOps
    |
    | Build
    v
Docker Image
    |
    v
ACR
    |
    | Pull
    v
AKS
    |
    +--------------------+
    |                    |
    v                    v
Frontend              Backend
    |                    |
LoadBalancer          ClusterIP
                         |
                         v
                PostgreSQL Flexible
                      Server
                         |
                         v
                  Private Network
                         |
                         v
                   Private DNS

Each component has one main responsibility.

Component	Responsibility
GitHub	Source code
Azure DevOps	CI/CD orchestration
Docker	Package applications
ACR	Store container images
Helm	Package/deploy Kubernetes resources
AKS	Run containers
Kubernetes Service	Provide stable networking
ConfigMap	Non-secret configuration
Secret	Sensitive configuration
PostgreSQL Flexible Server	Managed database
Private DNS	Resolve private database hostname
Azure RBAC	Azure authorization
Kubernetes RBAC	Kubernetes authorization

86. Important Lessons I Learned

Lesson 1 — Authentication and authorization are different

I initially thought:

If Azure DevOps can authenticate to Azure,
it should automatically be able to deploy to Kubernetes.

That is not necessarily true.

We had:

OIDC
 ↓
Azure authentication

then:

Azure RBAC
 ↓
Azure permissions

and separately:

Kubernetes RBAC
 ↓
Kubernetes permissions

Lesson 2 — ACR push and AKS pull are different operations

Azure DevOps needs permission to:

PUSH

images into ACR.

AKS needs permission to:

PULL

images from ACR.

Therefore:

Azure DevOps
     |
     | Push
     v
ACR
     |
     | Pull
     v
AKS

Lesson 3 — Private networking requires DNS

It is not enough to have:

Private IP

The application normally uses:

hostname

Therefore:

Hostname
   ↓
Private DNS
   ↓
Private IP
   ↓
Database

The Private DNS zone was:

private.postgres.database.azure.com

Lesson 4 — ConfigMap and Secret solve different problems

ConfigMap:

Normal configuration

Secret:

Sensitive configuration

So:

ConfigMap
 |
 +-- PROJECT_NAME
 +-- POSTGRES_SERVER
 +-- POSTGRES_DB

while:

Secret
 |
 +-- POSTGRES_USER
 +-- POSTGRES_PASSWORD

Lesson 5 — Kubernetes Pods are not permanent endpoints

Pods can be recreated.

Therefore applications should normally communicate through Services rather than hardcoding Pod IPs.

Pod
 ↓
Service
 ↓
Stable endpoint

Lesson 6 — `kubectl describe` is extremely useful

When something is:

Pending

or:

Failed

don't immediately change configuration.

First investigate.

For example:

kubectl describe service k8app-frontend -n frontend

revealed:

PublicIPCountLimitReached

That saved a lot of unnecessary Kubernetes troubleshooting.

87. Troubleshooting Cheat Sheet

Pod is CrashLoopBackOff

Start with:

kubectl logs <pod> -n <namespace>

Then:

kubectl describe pod <pod> -n <namespace>

Look for:

missing environment variables
bad configuration
application startup failure
database connection failure

Pod is Pending

Run:

kubectl describe pod <pod> -n <namespace>

Look at:

Events

Possible causes include:

insufficient resources
scheduling problem
node problem

ImagePullBackOff

Check:

kubectl describe pod <pod> -n <namespace>

Verify:

Image name
Image tag
ACR repository
ACR permissions
AKS AcrPull role

Service EXTERNAL-IP is Pending

Run:

kubectl describe service <service> -n <namespace>

Check the events.

Don't assume the Kubernetes Service itself is broken.

The problem could be Azure infrastructure or quota.

Helm deployment fails

First run:

helm template test-release helm/k8app

Then:

helm upgrade --install ...

If necessary:

helm template test-release helm/k8app --debug

Backend cannot connect to PostgreSQL

Check:

1. DNS
2. Private IP
3. Network connectivity
4. PostgreSQL server
5. Database name
6. Username
7. Password
8. TLS configuration

For DNS:

kubectl run psql-test \
  --rm -it \
  --image=postgres:16 \
  --restart=Never \
  -- \
  getent hosts dessy-k8app-postgres.postgres.database.azure.com

88. Security Notes

There are a few things I would improve before calling this a production-ready implementation.

First, never commit:

POSTGRES_PASSWORD

to Git.

Never put it in:

values.yaml

or:

Deployment YAML

The real password should also never appear in:

screenshots
Dev.to article
GitHub repository
pipeline logs

For a production implementation, I would also consider:

Azure Key Vault
External Secrets
Workload Identity
More restrictive RBAC
Private ACR
Private AKS networking
Network policies
TLS
Ingress
Health probes
Database migrations

The current project is a learning environment designed to understand the complete flow.

89. What I Would Improve Next

The current DEV environment works, but there are several areas I want to improve.

1. Database migrations

The PostgreSQL database currently has no application tables.

The next step is to introduce migrations so the deployment process can create/update the schema safely.

2. Better secret management

The Azure DevOps variable group works for the learning environment.

A more production-oriented design would use:

Azure Key Vault
       |
       v
Workload Identity / Secret integration
       |
       v
AKS

3. Ingress

The frontend currently uses:

LoadBalancer

A production environment could instead use an ingress architecture:

Internet
   |
   v
WAF / Application Gateway
   |
   v
Ingress
   |
   +---- frontend
   |
   +---- backend

This also provides a cleaner way to route traffic between frontend and backend.

4. Health probes

The Deployments should eventually have:

livenessProbe
readinessProbe

This allows Kubernetes to understand whether an application is:

alive

and whether it is:

ready to receive traffic

5. Pipeline error handling

The deployment script should explicitly fail when an important command fails.

For example, I would eventually add:

set -euo pipefail

This prevents a later command from accidentally hiding an earlier deployment failure.

6. Environment-specific Helm values

Eventually I want:

values-dev.yaml
values-uat.yaml
values-prod.yaml

Then:

DEV
 ↓
UAT
 ↓
Approval
 ↓
PROD

90. What Comes Next: UAT and PROD

For now, I deliberately stopped at DEV.

The next architecture will be:

             Build
               |
               v
              DEV
               |
               v
              UAT
               |
               v
        Approval Gate
               |
               v
             PROD

The idea is that the same application artifact should move through environments rather than rebuilding different artifacts for each environment.

That gives us:

Build once
    |
    v
Promote the same version
    |
    +---- DEV
    |
    +---- UAT
    |
    +---- PROD

This is the direction I want to take the Azure DevOps pipeline next.

91. Final DEV Checklist

Before considering the DEV environment complete, I verify:

[✓] Resource Group created

[✓] Azure Container Registry created

[✓] AKS created

[✓] PostgreSQL Flexible Server created

[✓] PostgreSQL private subnet created

[✓] Private DNS configured

[✓] AKS can resolve PostgreSQL hostname

[✓] AKS can connect to PostgreSQL

[✓] ACR contains frontend image

[✓] ACR contains backend image

[✓] AKS has AcrPull permission

[✓] frontend namespace created

[✓] backend namespace created

[✓] ConfigMap created

[✓] Kubernetes Secret created

[✓] Backend Deployment running

[✓] Frontend Deployment running

[✓] Backend Service is ClusterIP

[✓] Frontend Service is LoadBalancer

[✓] Helm chart renders successfully

[✓] Helm release deployed

[✓] Azure DevOps build succeeds

[✓] Azure DevOps deploys to AKS

[✓] Frontend receives public IP

[✓] Application accessible from browser

Conclusion

This project started as a way for me to practice Azure DevOps and Kubernetes.

It became much more useful once I stopped looking at the individual commands and started understanding how the components fit together.

The final DEV deployment looks like:

                    GitHub
                       |
                       v
                Azure DevOps
                       |
                       v
              Build + Test
                       |
                       v
                Docker Build
                       |
                       v
                     ACR
                       |
                       v
                    Helm
                       |
                       v
                     AKS
                  /       \
                 /         \
                v           v
          Frontend       Backend
                |           |
          LoadBalancer   ClusterIP
                            |
                            v
                     Private Network
                            |
                            v
                  PostgreSQL Flexible
                         Server

The most important thing I learned was that DevOps is not simply about writing YAML.

When something fails, I need to understand:

What failed?
     |
     v
Which component owns the failure?
     |
     v
How does that component communicate
with the other components?
     |
     v
What identity is being used?
     |
     v
What permission is required?
     |
     v
What evidence can I collect?
     |
     v
What should I change?

For example:

Backend failed
      |
      v
Check logs
      |
      v
Missing environment variables
      |
      v
ConfigMap + Secret
      |
      v
Deployment references them
      |
      v
Backend starts

And:

Frontend LoadBalancer pending
      |
      v
kubectl describe service
      |
      v
PublicIPCountLimitReached
      |
      v
Check Azure resources
      |
      v
Find unused AKS cluster
      |
      v
Remove unused resource
      |
      v
LoadBalancer receives public IP

Those troubleshooting experiences were more valuable than simply getting a green pipeline on the first attempt.

The final goal is not just:

"I know the commands."

The goal is:

I understand what each component does, why it exists, how the components communicate, how identities and permissions work, and how to troubleshoot the environment when something breaks.

That is the foundation I now have for moving from DEV to UAT, approval gates and PROD.

Migrating CI/CD from Azure DevOps to GitHub Actions with Azure OIDC, ACR, Helm and AKS

Desmond Goldsmith — Tue, 08 Sep 2026 21:25:06 +0000

Introduction

As part of learning and preparing for a migration from Azure DevOps pipelines to GitHub Actions, I decided to build a small practice project before working with a real application.

The project started very simply. I wanted to get the application building, run tests, create a Docker image, authenticate GitHub Actions to Azure without storing a client secret, and push the image into Azure Container Registry.

After I got that working, I continued with the Kubernetes side and deployed the application to Azure Kubernetes Service using Helm.

Project repository: https://github.com/Desmondgoldsmith/REST-go-k8s-example
github url

What made the exercise useful for me was that things did not always work on the first attempt.

I had to figure out problems such as:

How GitHub Actions can authenticate to Azure using OIDC
Which Azure RBAC permissions the GitHub identity actually needs
How AKS authenticates GitHub Actions
Why kubelogin was required
Why a Kubernetes LoadBalancer service was stuck on <pending>
How an Azure public IP quota problem was affecting the Kubernetes service
How to make the Docker image name come from the GitHub repository instead of hard-coding it
How to use the GitHub Actions run number as the Docker image tag

So this article is not meant to be a perfect "copy these commands and everything will work" tutorial.

It is more of a record of how I built the pipeline, what I configured, what failed, how I investigated the failures, and what I learned from the process.

What I wanted to build

The end goal was roughly:

GitHub Pull Request
        ↓
GitHub Actions
        ↓
Go Build + Test
        ↓
Azure OIDC Authentication
        ↓
Azure Container Registry
        ↓
Docker Image
        ↓
Helm
        ↓
Azure Kubernetes Service
        ↓
Running Application

The project started with CI and pushing the Docker image to ACR.

I then extended it to CD by deploying that image to AKS using Helm.

1. The Practice Application

I used a small Go REST API for the exercise.

The application itself was not the main focus. I just needed something simple enough that I could concentrate on the CI/CD process.

The application needed to:

Build successfully with Go
Pass tests
Run inside a Docker container
Expose an HTTP endpoint
Provide a health-check endpoint

The health-check endpoint is:

/health-check

I first ran the application locally:

go run .

Then I tested it:

curl http://localhost:10000/health-check

The application returned:

{"healthy":true}

That gave me confidence that the application itself was working before I started adding GitHub Actions and Azure.

Testing the Docker container locally

Before involving Azure, I also wanted to make sure the Dockerfile worked.

I built the image:

docker build -t go-rest-api:1.0 .

Then I ran it:

docker run --rm -p 10000:10000 go-rest-api:1.0

And tested it again:

curl http://localhost:10000/health-check

The response was:

{"healthy":true}

At this point I had two things working locally:

Go application
      ↓
Docker image
      ↓
Running container
      ↓
HTTP health check

That was important because if something later failed in GitHub Actions, I knew the problem was probably in the pipeline or cloud configuration rather than the application itself.

2. Connecting the Project to GitHub

The source code was stored in my GitHub repository:

Desmondgoldsmith/REST-go-k8s-example

I created a separate branch for testing:

git checkout -b test-branch

Then pushed it:

git push -u origin test-branch

I opened a Pull Request from:

test-branch → main

One of the requirements I was working toward was having the pipeline run from Pull Requests rather than simply running after every normal push.

So the workflow eventually used:

on:
  pull_request:

This means GitHub Actions can run checks when a Pull Request is opened or updated.

That gives me a useful CI/CD flow:

Developer changes code
        ↓
Push branch
        ↓
Create/update Pull Request
        ↓
GitHub Actions runs
        ↓
Build / Test / Deploy

3. Preparing Azure

For the Azure side of the project, I used an Azure for Students subscription.

I created the resources needed for the lab in:

South Africa North

The main resources were:

Resource Group:
devops-aks-lab-rg-sa

Azure Container Registry:
dessydevopsacr

AKS Cluster:
rest-go-aks

The Azure Container Registry login server was:

dessydevopsacr.azurecr.io

The purpose of ACR was to act as the place where GitHub Actions would push the Docker image.

AKS would later pull that image and run it.

So the basic relationship became:

GitHub Actions
      ↓
Docker Image
      ↓
Azure Container Registry
      ↓
AKS
      ↓
Kubernetes Pod

4. Why I Used Azure OIDC

One of the things I wanted to understand properly was authentication.

A simple approach would have been to create an Azure service principal with a client secret and store that secret inside GitHub.

That would look something like:

GitHub Actions
      ↓
Client ID + Client Secret
      ↓
Azure

But that means having a long-lived secret that has to be stored and managed.

Instead, I used OpenID Connect (OIDC).

The authentication flow became:

GitHub Actions
      ↓
OIDC Token
      ↓
Microsoft Entra ID
      ↓
Azure

The important thing for me was that there was no Azure client secret being stored in GitHub.

The GitHub Actions workflow receives an OIDC identity token, and Azure checks whether that token matches the federated credential that I configured.

5. Creating the Azure App Registration

In the Azure Portal, I went to:

Microsoft Entra ID
    ↓
App registrations
    ↓
New registration

I created an application called:

github-actions-rest-go-acr

I used:

Accounts in this organizational directory only

After creating the application, Azure provided several identifiers.

The important ones for the workflow were:

Application (client) ID
Directory (tenant) ID
Azure subscription ID

There is also an important distinction between the Application/Client ID, the Application Object ID, and the Service Principal Object ID.

I initially had to understand this distinction because Azure role assignments are associated with the service principal identity.

_Azure Portal → Microsoft Entra ID → App registrations → github-actions-rest-go-acr → Overview

_

6. Creating the Federated Credential

Creating the App Registration was not enough.

I also needed to tell Azure:

"I trust this specific GitHub Actions identity."

Inside the App Registration, I went to:

Certificates & secrets
    ↓
Federated credentials
    ↓
Add credential

I selected:

GitHub Actions deploying Azure resources

Then I configured the GitHub repository information.

For this project:

GitHub owner:
Desmondgoldsmith

Repository:
REST-go-k8s-example

I selected:

Entity type:
Pull request

That choice was important because the workflow was configured to run on:

on:
  pull_request:

I named the federated credential:

github-actions-rest-go-acr-pr

The audience was:

api://AzureADTokenExchange

Conceptually, this created a trust relationship between:

GitHub repository
       ↓
GitHub Actions
       ↓
OIDC identity
       ↓
Microsoft Entra ID
       ↓
Azure App Registration

The Federated credential configuration in Azure.

7. Understanding What the Federated Credential Actually Does

This was one of the most important concepts I learned during the project.

The federated credential does not simply mean:

"GitHub is allowed to do anything in Azure."

It establishes a trust relationship.

When the workflow runs, GitHub can provide an OIDC token.

Azure receives that token and checks whether it matches the federated credential.

Conceptually:

GitHub Actions
       ↓
OIDC token
       ↓
Microsoft Entra ID
       ↓
"Does this identity match my trust configuration?"
       ↓
Yes
       ↓
Application identity authenticated

But authentication is only one part of the process.

Even after Azure knows who the identity is, Azure still needs to know what that identity is allowed to do.

That is where Azure RBAC comes in.

So I think of it as:

Federated Credential
        ↓
WHO are you?

Azure RBAC
        ↓
WHAT are you allowed to do?

8. Giving GitHub Actions Permission to Push to ACR

Next, I needed to give the GitHub Actions identity permission to push images into ACR.

I opened:

Azure Portal
    ↓
Container Registries
    ↓
dessydevopsacr
    ↓
Access control (IAM)

Then:

Add
    ↓
Add role assignment

For the role, I selected:

AcrPush

The important part here was that I did not give the GitHub identity broad permissions such as Contributor.

The pipeline only needed permission to work with container images in the registry.

So the assignment was essentially:

github-actions-rest-go-acr
          ↓
       AcrPush
          ↓
  dessydevopsacr

This follows the principle of least privilege.

Azure Portal → ACR → Access control (IAM) → Role assignments showing the AcrPush assignment.

9. Adding the Azure Values to GitHub

I then went to the GitHub repository:

Settings
    ↓
Secrets and variables
    ↓
Actions
    ↓
Secrets

I created:

AZURE_CLIENT_ID
AZURE_TENANT_ID
AZURE_SUBSCRIPTION_ID

The values were:

AZURE_CLIENT_ID       = <AZURE_CLIENT_ID>
AZURE_TENANT_ID      = <AZURE_TENANT_ID>
AZURE_SUBSCRIPTION_ID = <AZURE_SUBSCRIPTION_ID>

Also notice what is missing:

AZURE_CLIENT_SECRET

There was no client secret.

That is because the authentication was based on OIDC and the federated credential.

GitHub → Settings → Secrets and variables → Actions

10. Allowing GitHub Actions to Request an OIDC Token

The workflow also needed permission to request an OIDC token.

I added:

permissions:
  id-token: write
  contents: read

The important permission here is:

id-token: write

This allows the workflow to request an OIDC identity token.

The:

contents: read

permission allows the workflow to read the repository contents.

So the beginning of the workflow became:

name: Go CI/CD

on:
  pull_request:

permissions:
  id-token: write
  contents: read

This was another piece of the authentication chain.

11. Logging Into Azure from GitHub Actions

Once the Azure App Registration, federated credential, GitHub secrets, and permissions were configured, I added the Azure login action:

- name: Azure login
  uses: azure/login@v2
  with:
    client-id: ${{ secrets.AZURE_CLIENT_ID }}
    tenant-id: ${{ secrets.AZURE_TENANT_ID }}
    subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}

This is where the configuration started coming together.

The workflow:

Requests OIDC token
       ↓
Azure validates token
       ↓
Federated credential matches
       ↓
GitHub Actions is authenticated
       ↓
Azure CLI can be used

12. Testing Azure Authentication

I pushed the workflow changes to my test branch and opened a Pull Request.

GitHub Actions ran the workflow.

This time the Azure login step completed successfully.

That was an important checkpoint.

It proved that this part of the setup was working:

GitHub Actions
       ↓
OIDC
       ↓
Microsoft Entra ID
       ↓
Federated Credential
       ↓
Azure Application Identity

GitHub Actions run showing the successful Azure login

13. Logging Docker Into ACR

Being authenticated to Azure does not automatically mean Docker is logged into the container registry.

So I added:

- name: Log in to Azure Container Registry
  run: az acr login --name dessydevopsacr

This uses the Azure identity that was established by azure/login.

The registry is:

dessydevopsacr.azurecr.io

After this step, Docker can authenticate against the ACR registry.

The flow is now:

GitHub Actions
      ↓
Azure OIDC Login
      ↓
Azure CLI
      ↓
ACR Login
      ↓
Docker

14. Moving From CI to CD

At this point I had the CI side working.

The next step was to actually deploy the Docker image to Kubernetes.

The target platform was:

Azure Kubernetes Service (AKS)

My AKS cluster was:

rest-go-aks

I also created a Kubernetes namespace for the application:

go-rest-api

The deployment flow would now become:

Pull Request
      ↓
GitHub Actions
      ↓
Go Build
      ↓
Go Test
      ↓
Docker Build
      ↓
Push Image to ACR
      ↓
Helm
      ↓
AKS
      ↓
Pod

15. Why I Used Helm

Instead of putting the Kubernetes Deployment and Service YAML directly into the GitHub Actions workflow, I used Helm.

My Helm chart looked roughly like this:

helm/
├── Chart.yaml
├── values.yaml
└── templates/
    ├── _helpers.tpl
    ├── deployment.yaml
    └── service.yaml

The chart contains the Kubernetes resources needed by the application.

The important idea is that the chart provides the structure, while values such as the image repository and image tag can be supplied when the deployment runs.

For example:

image:
  repository: dessydevopsacr.azurecr.io/go-rest-api
  pullPolicy: IfNotPresent
  tag: "latest"

The latest value here is simply the default in the values file.

During the actual deployment, GitHub Actions overrides it with the image we just built.

16. The Kubernetes Deployment

The Helm Deployment template contains the container definition.

The important part is:

containers:
  - name: {{ .Chart.Name }}
    image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"

This means Helm takes:

image.repository

and:

image.tag

and combines them into:

registry/repository:tag

For example:

dessydevopsacr.azurecr.io/rest-go-k8s-example:12

The Deployment also has readiness and liveness probes:

readinessProbe:
  httpGet:
    path: /health-check
    port: http
  initialDelaySeconds: 5
  periodSeconds: 10

livenessProbe:
  httpGet:
    path: /health-check
    port: http
  initialDelaySeconds: 10
  periodSeconds: 20

This allows Kubernetes to use the application's health endpoint when determining whether the container is ready and whether it is still healthy.

17. The Kubernetes Service

I also created a Kubernetes Service.

For the lab, I used:

service:
  type: LoadBalancer
  port: 10000
  targetPort: 10000

The Service exposes the application outside the Kubernetes cluster.

The basic flow is:

Internet
   ↓
Azure Load Balancer
   ↓
Kubernetes Service
   ↓
Pod
   ↓
Go Application

18. Connecting GitHub Actions to AKS

The GitHub Actions identity also needed permission to interact with the AKS cluster.

The AKS cluster was configured to use Microsoft Entra ID and Azure RBAC for Kubernetes authorization.

I assigned the GitHub identity the permissions required to access the cluster and perform the deployment.

The important distinction here is that ACR access and AKS access are separate permissions.

For example:

GitHub Actions Identity
       │
       ├──────────────→ ACR
       │                  │
       │                AcrPush
       │
       └──────────────→ AKS
                          │
                   Cluster User Role
                          +
                    RBAC Writer

This was important because having AcrPush does not automatically give the identity permission to deploy to Kubernetes.

19. Getting the AKS Context

I used Microsoft's AKS context action:

- name: Set AKS context
  uses: azure/aks-set-context@v5
  with:
    resource-group: devops-aks-lab-rg-sa
    cluster-name: rest-go-aks
    admin: 'false'
    use-kubelogin: 'true'

I deliberately used:

admin: 'false'

because I wanted the workflow to authenticate using the configured Azure identity and Kubernetes RBAC rather than using the AKS administrator credentials.

20. The `kubelogin` Problem

This was one of the first real problems I encountered when connecting GitHub Actions to AKS.

The workflow initially failed with an error saying that kubelogin could not be found.

The important part of the error was:

Unable to locate executable file: kubelogin

The AKS credentials were being retrieved, but the GitHub runner did not have the required kubelogin executable available.

Instead of assuming the AKS configuration was broken, I looked at the error carefully.

It was telling me that the authentication mechanism needed kubelogin.

So I added:

- name: Install kubelogin
  uses: azure/use-kubelogin@v1.2
  with:
    kubelogin-version: 'v0.0.24'

I pinned the version rather than relying on an automatically resolved latest version.

The AKS context step then used:

use-kubelogin: 'true'

After this change, the workflow was able to configure the AKS context successfully.

The failed GitHub Actions run containing the kubelogin error

The later successful GitHub Actions run with the `Install kubelogin` and `Set AKS context` steps passing

21. Checking the AKS Context Locally

While troubleshooting the Kubernetes side, I also used kubectl locally to understand what was happening.

To see available contexts:

kubectl config get-contexts

My AKS context was:

rest-go-aks

I could switch to it with:

kubectl config use-context rest-go-aks

Then verify the cluster:

kubectl get nodes

This was useful because it gave me a way to distinguish between:

Kubernetes problem

and:

GitHub Actions authentication problem

22. Deploying With Helm

Once the GitHub runner could authenticate to AKS, the deployment step was:

- name: Deploy to AKS with Helm
  run: |
    helm upgrade --install go-rest-api ./helm \
      --namespace go-rest-api \
      --set image.repository=dessydevopsacr.azurecr.io/${{ env.IMAGE_NAME }} \
      --set image.tag=${{ github.run_number }}

There are a few things happening here.

First:

helm upgrade --install

means:

If the release exists, upgrade it. If it does not exist, install it.

The release name is:

go-rest-api

The namespace is:

go-rest-api

The image repository and tag are supplied dynamically.

23. Why I Changed From Commit SHA to Run Number

Originally, I used the GitHub commit SHA as the Docker image tag.

That looked like:

go-rest-api:<commit-sha>

Using the SHA has an advantage because the image can be traced directly back to a commit.

However, my supervisor asked me to change this.

The requirement was to use the pipeline build/run number instead because it is easier to follow as a simple linear sequence.

So instead of:

${{ github.sha }}

I changed the workflow to:

${{ github.run_number }}

Now the image tags look like:

:10
:11
:12

For example:

dessydevopsacr.azurecr.io/rest-go-k8s-example:12

This makes it much easier to say:

"Deployment 12 is running image 12."

The important part is that the same run number is used when building, pushing, and deploying the image.

24. Making the Docker Image Name Dynamic

Another requirement from my supervisor was that the Docker image name should come from the GitHub repository.

I did not want to hard-code:

go-rest-api

as the image name.

The GitHub context provides:

github.event.repository.name

which returns the repository name.

For this repository:

REST-go-k8s-example

Docker image repository names need to be lowercase, so I added:

- name: Set image name
  run: echo "IMAGE_NAME=$(echo '${{ github.event.repository.name }}' | tr '[:upper:]' '[:lower:]')" >> "$GITHUB_ENV"

This creates:

IMAGE_NAME=rest-go-k8s-example

The image can then be built using:

- name: Build Docker image
  run: docker build -t dessydevopsacr.azurecr.io/${{ env.IMAGE_NAME }}:${{ github.run_number }} .

And pushed using:

- name: Push Docker image
  run: docker push dessydevopsacr.io/${{ env.IMAGE_NAME }}:${{ github.run_number }}

The actual registry should of course be:

dessydevopsacr.azurecr.io

So the final image becomes:

dessydevopsacr.azurecr.io/rest-go-k8s-example:12

This is better than hard-coding the application name because the workflow can be reused for another repository without changing the Docker image name logic.

25. The Final CI/CD Workflow

After putting the pieces together, my workflow became:

name: Go CI/CD

on:
  pull_request:

permissions:
  id-token: write
  contents: read

jobs:
  build:
    runs-on: ubuntu-latest

    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up Go
        uses: actions/setup-go@v5
        with:
          go-version: '1.27'

      - name: Azure login
        uses: azure/login@v2
        with:
          client-id: ${{ secrets.AZURE_CLIENT_ID }}
          tenant-id: ${{ secrets.AZURE_TENANT_ID }}
          subscription-id: ${{ secrets.AZURE_SUBSCRIPTION_ID }}

      - name: Install kubelogin
        uses: azure/use-kubelogin@v1.2
        with:
          kubelogin-version: 'v0.0.24'

      - name: Set AKS context
        uses: azure/aks-set-context@v5
        with:
          resource-group: devops-aks-lab-rg-sa
          cluster-name: rest-go-aks
          admin: 'false'
          use-kubelogin: 'true'

      - name: Log in to Azure Container Registry
        run: az acr login --name dessydevopsacr

      - name: Download dependencies
        run: go mod download

      - name: Build Go application
        run: go build -v ./...

      - name: Run Go tests
        run: go test ./...

      - name: Set image name
        run: echo "IMAGE_NAME=$(echo '${{ github.event.repository.name }}' | tr '[:upper:]' '[:lower:]')" >> "$GITHUB_ENV"

      - name: Build Docker image
        run: docker build -t dessydevopsacr.azurecr.io/${{ env.IMAGE_NAME }}:${{ github.run_number }} .

      - name: Push Docker image
        run: docker push dessydevopsacr.azurecr.io/${{ env.IMAGE_NAME }}:${{ github.run_number }}

      - name: Deploy to AKS with Helm
        run: |
          helm upgrade --install go-rest-api ./helm \
            --namespace go-rest-api \
            --set image.repository=dessydevopsacr.azurecr.io/${{ env.IMAGE_NAME }} \
            --set image.tag=${{ github.run_number }}

The important thing about this workflow is not memorizing every line.

The workflow is basically doing this:

1. Checkout source code
        ↓
2. Install Go
        ↓
3. Authenticate to Azure
        ↓
4. Install kubelogin
        ↓
5. Connect to AKS
        ↓
6. Login to ACR
        ↓
7. Download Go dependencies
        ↓
8. Build Go application
        ↓
9. Run tests
        ↓
10. Determine image name from GitHub repository
        ↓
11. Build Docker image
        ↓
12. Push image to ACR
        ↓
13. Deploy image to AKS with Helm

26. Verifying the Image in ACR

After a successful workflow run, I checked Azure Container Registry.

The repository name was now based on the GitHub repository:

rest-go-k8s-example

The image tag was based on the GitHub Actions run number.

For example:

rest-go-k8s-example:12

The complete image reference was:

dessydevopsacr.azurecr.io/rest-go-k8s-example:12

This confirmed that the Docker build and push stages were working.

Azure Portal → Container Registry → Repositories → `rest-go-k8s-example`

27. Verifying the Helm Release

After deployment, I checked the Helm release:

helm list -n go-rest-api

The release showed as deployed:

NAME          NAMESPACE     REVISION    STATUS
go-rest-api   go-rest-api   3           deployed

The important thing here is that Helm was managing the Kubernetes deployment rather than GitHub Actions manually applying individual Kubernetes YAML files.

To see releases across all namespaces, I could also use:

helm list -A

And if I wanted to see the Kubernetes resources:

kubectl get pods -n go-rest-api

kubectl get svc -n go-rest-api

kubectl get deployments -n go-rest-api

28. Checking What Image AKS Is Running

One of the checks I found useful was looking directly at the Deployment.

For example:

kubectl get deployment go-rest-api \
  -n go-rest-api \
  -o jsonpath='{.spec.template.spec.containers[0].image}'

This lets me verify the exact image Kubernetes is configured to run.

The result was:

dessydevopsacr.azurecr.io/rest-go-k8s-example:12

This was a useful confirmation because it connected the different parts of the pipeline:

GitHub Actions Run 12
        ↓
Docker Image Tag 12
        ↓
ACR Image :12
        ↓
Helm
        ↓
AKS Deployment
        ↓
Image :12

29. The LoadBalancer Problem

One of the more interesting problems happened when I created the Kubernetes Service.

The Service was:

type: LoadBalancer

but when I checked it:

kubectl get svc -n go-rest-api

the external IP was:

<pending>

At first, this could have been a Kubernetes configuration problem.

So rather than guessing, I described the Service:

kubectl describe svc go-rest-api -n go-rest-api

The Events section showed:

Warning CreateOrUpdatePublicIPAddress
ERROR CODE: PublicIPCountLimitReached
Cannot create more than 3 public IP addresses for this subscription in this region.

That immediately changed the direction of the investigation.

The Kubernetes Service itself was not the main problem.

Azure was refusing to create another public IP because I had reached the public IP quota for the subscription and region.

30. Finding the Resource Consuming the Public IP

I checked the public IP resources in Azure.

There were multiple public IP resources associated with the Kubernetes environment.

I also discovered that I had an older Helm release in the default namespace:

go-rest-api

while I was now using:

go-rest-api

in the intended namespace:

go-rest-api

I removed the old Helm release:

helm uninstall go-rest-api -n default

I then investigated the remaining Azure public IP resources.

This was a good reminder that deleting a Kubernetes object and checking Azure resources are sometimes two separate parts of troubleshooting.

Kubernetes may request Azure resources such as:

Public IP
Load Balancer
Network Interface

and when troubleshooting, it is useful to look at both sides.

31. The Service Finally Received an External IP

After cleaning up the unused resources and allowing Azure to create the required public IP, the Service received an external IP.

I checked it again:

kubectl get svc -n go-rest-api

The Service was now showing an external IP instead of:

<pending>

I then tested the application from outside the cluster:

curl http://<EXTERNAL-IP>:10000/health-check

The application returned:

{"healthy":true}

That was the final confirmation that the complete deployment path was working.

The successful external curl request returning {"healthy":true}

32. What the Complete Architecture Looks Like

At this point the project had moved beyond simply pushing an image to ACR.

The complete flow was:

Developer
    ↓
GitHub Pull Request
    ↓
GitHub Actions
    ↓
Go Build
    ↓
Go Tests
    ↓
GitHub OIDC
    ↓
Microsoft Entra ID
    ↓
Azure RBAC
    ↓
Azure Container Registry
    ↓
Docker Image
    ↓
Helm
    ↓
AKS
    ↓
Kubernetes Deployment
    ↓
Kubernetes Service
    ↓
Azure Load Balancer
    ↓
Go REST API

There are actually two important Azure authentication/authorization paths involved.

For pushing the image:

GitHub Actions
      ↓
OIDC
      ↓
Microsoft Entra ID
      ↓
GitHub Actions App
      ↓
AcrPush
      ↓
ACR

For running the application:

AKS
      ↓
Kubelet Identity
      ↓
AcrPull
      ↓
ACR
      ↓
Docker Image
      ↓
Pod

And for deploying to Kubernetes:

GitHub Actions
      ↓
OIDC
      ↓
Microsoft Entra ID
      ↓
GitHub Actions Identity
      ↓
AKS Cluster User Role
      +
Azure Kubernetes Service RBAC Writer
      ↓
Kubernetes API
      ↓
Helm Deployment

Understanding these as separate permission paths helped me understand why one permission does not automatically give another.

33. ACR Push vs AKS Pull

This was another concept that became much clearer after actually doing the project.

There are two different actions:

GitHub Actions pushes the image

GitHub Actions
      ↓
AcrPush
      ↓
ACR

GitHub Actions needs permission to push the image.

AKS pulls the image

AKS Node/Kubelet
      ↓
AcrPull
      ↓
ACR

The AKS kubelet identity needs permission to pull the image.

So I did not need to create a Kubernetes imagePullSecret for this setup.

The Azure identities handled the registry authentication.

34. Checking Kubernetes Resources

During the troubleshooting process, I also learned that I don't need to memorize every resource name.

If I forget which namespaces exist:

kubectl get ns

If I want to see Pods everywhere:

kubectl get pods -A

Deployments:

kubectl get deployments -A

Services:

kubectl get svc -A

All common resources:

kubectl get all -A

For Helm releases:

helm list -A

The -n option means namespace.

For example:

helm list -n go-rest-api

means:

Show Helm releases in the go-rest-api namespace.

And:

helm list -A

means:

Show Helm releases across all namespaces.

35. Validating Helm Before Deploying

Another useful command during the Helm work was:

helm template test-release ./helm

This renders the Helm templates locally without actually deploying them.

That helped catch template problems before sending the chart to AKS.

At one point, I encountered an error related to:

.Values.serviceAccount.create

The chart still contained generated templates that expected values I was no longer using.

I simplified the chart and removed the unused generated ServiceAccount and test templates.

After that:

helm template test-release ./helm

rendered successfully.

This reinforced an important lesson for me:

Helm templates are just templates until Helm renders them.

So if something is wrong with the chart, rendering it locally is often a much faster way to find the problem.

36. The Final Image Naming

The final image naming approach was one of the changes I made based on feedback from my supervisor.

Instead of:

go-rest-api:<commit-sha>

the pipeline now creates:

<acr>/<github-repository-name>:<github-run-number>

For this project:

dessydevopsacr.azurecr.io/rest-go-k8s-example:12

The important parts are:

dessydevopsacr.azurecr.io
        ↓
ACR registry

rest-go-k8s-example
        ↓
GitHub repository name

12
        ↓
GitHub Actions run number

This means the workflow logic is reusable without hard-coding the Docker repository name.

37. What I Learned

The biggest thing I learned from this exercise was that CI/CD is not just about writing a YAML file.

The YAML is actually the easy part once you understand what each component needs to do.

For example, when the pipeline failed because of kubelogin, the important thing was not memorizing:

uses: azure/use-kubelogin@v1.2

The important thing was understanding what the error meant.

The same thing happened with the LoadBalancer.

Instead of immediately changing Kubernetes configuration, I checked:

kubectl describe svc go-rest-api -n go-rest-api

and found:

PublicIPCountLimitReached

That told me the problem was actually an Azure quota issue.

So one of the biggest lessons for me was:

Don't guess when troubleshooting. Start with the error, identify which component is reporting it, and investigate that component.

38. Authentication vs Authorization

Another concept that became much clearer through the project was the difference between authentication and authorization.

Authentication

Authentication answers:

Who are you?

In this project:

GitHub Actions
      ↓
OIDC
      ↓
Microsoft Entra ID
      ↓
Federated Credential

Authorization

Authorization answers:

What are you allowed to do?

For example:

GitHub Actions Identity
      ↓
AcrPush
      ↓
ACR

and:

GitHub Actions Identity
      ↓
AKS RBAC permissions
      ↓
Kubernetes API

This distinction is easy to read about, but it became much more obvious after configuring both sides myself.

39. Final Result

I started with a small Go REST API running locally.

Then I gradually built the following:

Go Application
      ↓
Docker
      ↓
GitHub Repository
      ↓
GitHub Pull Request
      ↓
GitHub Actions
      ↓
Go Build + Test
      ↓
OIDC Authentication
      ↓
Microsoft Entra ID
      ↓
Azure RBAC
      ↓
Azure Container Registry
      ↓
Docker Image
      ↓
Helm
      ↓
AKS
      ↓
Kubernetes Deployment
      ↓
Kubernetes Service
      ↓
External Load Balancer
      ↓
Go REST API

The final Docker image looked like:

dessydevopsacr.azurecr.io/rest-go-k8s-example:12

And Kubernetes was running that image through the Helm-managed deployment.

The application's health endpoint was reachable externally:

curl http://<EXTERNAL-IP>:10000/health-check

and returned:

{"healthy":true}

That was a good point to stop and look back at the whole process because I had gone from a local Go application to a working CI/CD pipeline and Kubernetes deployment.

40. What I Want to Explore Next

There is still another part of the project that I want to investigate.

In Jenkins, one common approach is using Shared Libraries.

The idea is that instead of every repository maintaining its own large pipeline, a central repository contains the reusable pipeline logic.

Then individual repositories can call that shared workflow with very little configuration.

The GitHub Actions equivalent I want to investigate is reusable workflows.

The goal would be something like:

Central GitHub Actions workflow
              ↓
       Reusable workflow
        ↙     ↓      ↘
     Repo A  Repo B  Repo C

Instead of copying the complete CI/CD workflow into every repository, the workflow logic could live centrally.

Then if the central workflow is updated, the repositories using it can benefit from the change without duplicating the whole pipeline.

That is the next part I want to understand and implement properly.

I don't want to simply copy an example and call it done. I want to understand how the reusable workflow receives things such as:

Repository name
Azure credentials
AKS cluster
Resource group
ACR
Environment
Image tag

and how the calling repository can keep its configuration minimal.

Conclusion

This project started as a small exercise to understand GitHub Actions, but it ended up teaching me much more about how the different pieces of a real CI/CD system fit together.

The main things I worked with were:

GitHub Pull Requests
GitHub Actions
Go builds and tests
Docker
Azure OIDC
Microsoft Entra ID
Azure RBAC
Azure Container Registry
AKS
Kubernetes RBAC
Helm
Kubernetes Services
Azure Load Balancers
Troubleshooting Azure and Kubernetes

More importantly, I learned that when something fails, I should not immediately start changing random configuration.

I can first ask:

What component failed?
        ↓
What exactly is the error saying?
        ↓
What resource is involved?
        ↓
What command can I use to inspect it?
        ↓
What does the result tell me?
        ↓
What is the smallest change needed?

That is probably the most useful part of the exercise for me.

The project is now at the point where the application can go from a Pull Request through GitHub Actions, into ACR, and then into AKS using Helm.

The next step is to make the workflow more reusable so that the same CI/CD logic can be shared across multiple repositories without copying the entire workflow each time.

Building a DevOps Pipeline from Scratch: Local Deployment with Vagrant, Docker & Nginx - Part 1

Desmond Goldsmith — Mon, 23 Mar 2026 18:06:24 +0000

This is Part 1 of a series where I build a complete DevOps pipeline from the ground up; local deployment, CI/CD, security scanning, monitoring, and alerting.

Series Overview

This series documents the end-to-end process of building a production-grade DevOps pipeline. Here's where we're headed:

Part 1: Local Deployment with Vagrant, Docker, and Nginx
Part 2: CI/CD Pipeline with GitHub Actions (automated testing and deployment on every push)
Part 3: Security Scanning with Trivy (container image and code vulnerability scanning)
Part 4: Monitoring and Logging
Part 5: Slack Alerting for Failed Builds and Deployments (real-time notifications when things go wrong)

Resources

The Vagrantfile, Nginx config, and Docker Compose files used in this guide are available here: devops-pipeline-series
The application code is sourced from Docker's official Repo

What We're Building in Part 1

In this first part, we deploy a full-stack web application locally using Vagrant as the infrastructure layer, Docker Compose to orchestrate the containers, and Nginx as the reverse proxy sitting in front of everything.

Here's the architecture:

All four services run as Docker containers on a shared private network. Nothing is exposed to the outside except Nginx on port 80.

Why This Stack

Vagrant lets us provision a real Ubuntu server locally. Everything practiced here transfers directly to cloud deployments. You can destroy and rebuild the environment as many times as needed , no cloud costs, no configuration drift etc.
Docker Compose manages the multi-container setup through a single declarative file. Rather than running and networking containers manually, Docker compose handles the entire lifecycle with one command.
Nginx as a reverse proxy is standard practice in production. It gives us with a single entry point for all traffic, cleanly separating routing concerns from application logic, and makes it straightforward to add SSL termination, rate limiting, or load balancing later.

Prerequisites

Vagrant 2.x
VirtualBox 7.x
Git
Docker Engine (installed automatically)

To verify installation, run this:

vagrant --version
git --version

Step 1 : Create the Vagrantfile

Before we touch anything, make sure you're working on your local machine; not inside a VM or remote server.Open your terminal and follow along.

Create a new directory for the Vagrant project and add a Vagrantfile

mkdir ~/vagrant-devops-project
cd ~/vagrant-devops-project

In the vagrant-devops-project directory, create a vagrant file called Vagrantfile using this command touch Vagrantfile

Open the Vagrantfile in a vim editor using this command vim Vagrantfile and paste the command below:

Vagrant.configure("2") do |config|

  # Ubuntu 22.04 
  config.vm.box = "ubuntu/jammy64"
  config.vm.hostname = "devops-server"

  # Forward VM port 80 to localhost:8080
  config.vm.network "forwarded_port", guest: 80, host: 8080

  config.vm.provider "virtualbox" do |vb|
    vb.memory = "2048"
    vb.cpus = 2
  end

  # Provision Docker using the official install script
  config.vm.provision "shell", inline: <<-SHELL
    curl -fsSL https://get.docker.com | sh
    usermod -aG docker vagrant
  SHELL

end

When you're done pasting, press Esc to exit insert mode, then type :wq and hit Enter to save and close the file.

Note:
The provision script uses Docker's official installer to install Docker and Docker Compose on the Ubuntu vm.

The usermod line grants the vagrant user permission to run Docker without sudo.

Port forwarding maps localhost:8080 on your machine to port 80 inside the VM, which is where Nginx listens.

Step 2 : Boot the VM

Run the command below to boot the VM.

vagrant up

Step 3 : SSH into the VM

Run the command below to SSH into the VM.

vagrant ssh

Step 4 : Get the Application Code

We're using Docker's official awesome-compose repository as the application base. The focus of this series is on infrastructure and pipeline , not the application code itself.

Inside your VM, clone the project and cd into the awesome-compose/react-express-mysql using the code below:

git clone https://github.com/docker/awesome-compose.git
cd awesome-compose/react-express-mysql

This is the project structure:

react-express-mysql/
├── frontend/        ← React application
├── backend/         ← Node.js / Express API
├── db/              ← Database init scripts
├── compose.yaml     ← Docker Compose config
└── README.md

Step 5 : Add Nginx

The repository doesn't include Nginx. We add it ourselves.
In the react-express-mysql directory, create an nginx directory using the command below:

mkdir nginx
cd nginx

In the nginx directory, run this command vim nginx.conf and paste the code below:

server {
    listen 80;

    # Route frontend traffic
    location / {
        proxy_pass http://frontend:3000;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
    }

    # Route API traffic to the backend
    location /api/ {
        proxy_pass http://backend:5000/;
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
    }
}

After pasting the code above, press Esc to exit insert mode, then type :wq and hit Enter to save and close the file.

Note:
frontend and backend _in the _proxy_pass directives are Docker container names, not folder references. When containers share a Docker network, they can resolve each other by service name. Docker handles the DNS internally , no IP addresses required.

proxy_set_header forwards the original client information to the upstream service. This matters when your backend needs the real client IP for logging or rate limiting.

Step 6: Update Docker Compose

Replace the contents of compose.yaml with this code below:

services:

  frontend:
    build:
      context: frontend
      target: development
    networks:
      - app-network
    depends_on:
      - backend
    restart: always

  backend:
    build: ./backend
    environment:
      - DB_HOST=db
      - DB_USER=root
      - DB_PASSWORD=password
      - DB_NAME=example
    networks:
      - app-network
    depends_on:
      - db
    restart: always

  db:
    image: mariadb:10.6.4-focal
    command: '--default-authentication-plugin=mysql_native_password'
    environment:
      - MYSQL_DATABASE=example
      - MYSQL_ROOT_PASSWORD=password
    volumes:
      - db-data:/var/lib/mysql
    networks:
      - app-network
    restart: always

  nginx:
    image: nginx:alpine
    ports:
      - "80:80"
    volumes:
      - ./nginx/nginx.conf:/etc/nginx/conf.d/default.conf
    depends_on:
      - frontend
      - backend
    networks:
      - app-network
    restart: always

volumes:
  db-data:

networks:
  app-network:
    driver: bridge

Key design decisions in this config:

Only _Nginx _has a ports mapping. Every other service communicates internally over app-network. This is intentional , the database and backend are never directly reachable from outside the VM.

depends_on enforces startup ordering. Nginx won't start until the frontend and backend containers are up. The backend won't start until the database is ready.

restart: always ensures containers recover automatically from crashes.

The db-data named volume persists database state independently of the container lifecycle. Removing and recreating the database container does not wipe the data.

Your updated project structure:

react-express-mysql/
├── frontend/
├── backend/
├── db/
├── nginx/
│   └── nginx.conf     
├── compose.yaml        
└── README.md

Step 7: Deploy the Application

Run these commands below to start docker compose

docker compose up --build -d

Verify all containers are running:

docker compose ps

Expected output:

NAME                              STATUS
react-express-mysql-frontend-1    running
react-express-mysql-backend-1     running
react-express-mysql-db-1          running
react-express-mysql-nginx-1       running

Step 8: Verify the Deployment

Open your browser on your local machine:

http://localhost:8080

The React application loads. The request path was: browser → Vagrant port forwarding → VM port 80 → Nginx → React frontend container.

Useful Commands

# View logs for a specific service
docker compose logs nginx
docker compose logs backend

# Stream logs in real time
docker compose logs -f nginx

# Open a shell inside a running container
docker exec -it react-express-mysql-nginx-1 sh

# Monitor container resource usage
docker stats

# Stop the stack
docker compose down

# Stop and remove all volumes (full reset)
docker compose down -v

# Restart a single service
docker compose restart nginx

What's Coming in Part 2

With the application running locally, the next step is automation.
In Part 2 we'll build a CI/CD pipeline using GitHub Actions that triggers on every push to the main branch. It will run automated tests, and if they pass, SSH into the server and deploy the latest version with zero manual intervention.

After that, the series continues with:

Trivy for scanning container images and source code for known vulnerabilities before they ever reach the server
Centralized logging so you have visibility into what's happening across all containers
Slack notifications for failed builds and deployments because the worst way to find out something broke is from a user

Final Thoughts

I've been learning DevOps for a while now ; going through Docker, Linux, AWS, Vagrant, and currently working towards my Kubernetes certification. But there's a point where consuming content stops being enough and you just have to build something real.

This project was that moment for me. And honestly, the thing I learned most wasn't about Docker or Nginx. It was that the tools make a lot more sense when they're solving a real problem in front of you. Why does Nginx sit in front of everything? Because you need one controlled entry point. Why do containers talk by name instead of IP? Because IPs change, names don't. Those things click differently when you're the one wiring them together.

This setup is intentionally close to how production pipelines are structured .. a single entry point through Nginx, services isolated on a private network, infrastructure defined as code. Each part of this series builds directly on the previous one. By the end you'll have a complete pipeline, from a local vagrant up all the way through automated deployment, security scanning, and real-time alerting.

Follow along and drop questions in the comments. Thanks.