How to automate deployment of applications to Kubernetes cluster using GitHub Actions

Você pode ler este artigo em Português clicando aqui

While in last article of this series, I've teach you how to set up a free Kubernetes cluster running on Oracle Cloud's always free tier today you will learn how to automate the deployment of your applications from a GitHub repository to Kubernetes cluster using the GitHub Actions, building and publishing a Docker image to OCI Registry and empower you to move once and for all from any Vercel alike service.

We'll walk you through setting up policies in OCI Console, configuring your GitHub repository for automated builds and deployments, and ultimately deploying your Dockerized application to your OKE cluster.

Before You Begin

To follow this guide I presume that you already have:

• A free Kubernetes cluster running on OCI (follow our previous article to set it up), also all the credentials necessary to interact with the cluster.
• A basic understanding of Docker and Kubernetes concepts.
• A GitHub account.

Also I would like to introduce you to the infrastructure architecture that will be used in this guide.

Let's Get Started!

1. Securing Your OKE Cluster with Network Security Groups

OCI recommends managing security rules for your OKE cluster using Network Security Groups (NSGs). These groups allow you to define granular access controls for your applications running within the cluster.

1.1 Configuring policies to allow cluster manage NSGs.

In your OCI console, navigate to Identity & Security > Policies.

• Click Create Policy.
• Name your policy (e.g., "OKE-Security") and provide a description.
• Select the Show manual editor option.
• Add the following policy statements to the editor:

ALLOW any-user to manage network-security-groups in compartment <compartment-name> where request.principal.type = 'cluster'
ALLOW any-user to manage vcns in compartment <compartment-name> where request.principal.type = 'cluster'

Replace with the name of your OKE cluster's compartment (if you have created one).

If your cluster was created at the root compartment, use compartment id and the tenant OCID instead. ... in compartment id <tenant-OCID>

1.2 Creating a default NSG for NLB backends

The next step is create a NSG for managing the nlbs' backends. In your OCI console navigate to Virtual Cloud Networks > select the vcn where OKE is running.

• In resources, select Network Security Groups.
• Click Create Network Security Group.
• Name your new NSG (e.g., "nlb-nsg-backend") and click Next.
• In Security Rules don't add any rules and click Create.

1.3 Attaching NSG created to node workers.

To have effect and oci-cloud-controller-manager make his magic orchestrating security rules in cluster's VCN and exposing correctly to the internet the Kubernetes LoadBalancer services type we need to attach the created NSG to all the node workers.

In OCI Console go to Developer Services > Containers & Artifacts > Kubernetes Clusters (OKE).

• Select the cluster previously created.
• In resources, select node pools and click in the pool name.
• Click node name, a page with instance information should be shown.
• In VNICS, seek for Network security groups and click edit.
• Select the previous created NSG, repeat the process to all node instances.

2. Setting Up Your GitHub Repository

To Dockerize our application and create a valid workflow for GitHub Actions we need to set up some things in our repository.

• Create a new directory in your repository for your workflow files:

.github/workflows

• Create a Dockerfile within this directory to define your application's build instructions.

• Example Dockerfile for a Go Lang application:
  

FROM golang:1.22.5-alpine3.19 AS build
WORKDIR /app
COPY go.mod ./
RUN go mod download
COPY . ./
RUN CGO_ENABLED=0 GOOS=linux go build -a -installsuffix cgo -o hello main.go

FROM gcr.io/distroless/static-debian11:nonroot
COPY --from=build /app/hello /hello
USER nonroot:nonroot
EXPOSE 8080
ENTRYPOINT ["/hello"]

• Create a directory for deployments manifests (optional):

/kubernetes

2.1 Preparing an YAML manifest to deployment:

Imagine that your repo hosts the code of a Go Lang application, let's prepare a YAML file to describe the deployment and service that will be deployed to Kubernetes cluster.

• Example Deployment YAML manifest file:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: go-hello-world
  labels:
    app: go-hello-world
spec:
  replicas: 2
  selector:
    matchLabels:
      app: go-hello-world
  template:
    metadata:
      labels:
        app: go-hello-world
    spec:
      containers:
      - name: go-hello-world
        image: $IMAGE_NAME
        imagePullPolicy: Always
        ports:
          - containerPort: 8080
            protocol: TCP
      imagePullSecrets:
        - name: $DOCKER_REGISTRY_SECRET_NAME

The deployment kind defines the application go-hello-world details, including pod configs, see we're referring to an environment variable to set image and a docker registry secret $IMAGE_NAME and $DOCKER_REGISTRY_SECRET_NAME that will be created and set dynamically when running the workflow action that will be created soon.

• Example Service YAML manifest

---
apiVersion: v1
kind: Service
metadata:
  name: go-hello-world-lb
  labels:
    app: go-hello-world
  annotations:
    oci.oraclecloud.com/load-balancer-type: "nlb"
    oci-network-load-balancer.oraclecloud.com/backend-policy: "THREE_TUPLE"
    oci.oraclecloud.com/security-rule-management-mode: "NSG"
    oci-network-load-balancer.oraclecloud.com/security-list-management-mode:  "None"
    oci.oraclecloud.com/oci-backend-network-security-group: $OCI_NETWORK_SG_BACKEND
spec:
  type: LoadBalancer
  externalTrafficPolicy: Local
  ports:
    - name: http
      port: 80
      targetPort: 8080
  selector:
    app: go-hello-world

The service kind exposes the app go-hello-world to the internet through a LoadBalancer service type, listening on port 80 and redirecting the traffic to 8080 port of the pods. As seen, there's specific OCI annotations used to define the type of load balancer (NLB) and handle the network security policies, in details, the oci.oraclecloud.com/oci-backend-network-security-group annotation refers to an another environment variable to set the OCID of NSG for Network Load Balancer created and attached to the nodes instances earlier in this guide, again, when running the workflow action we will set dynamically this data.

Example repository structure after all these changes:

.dockerignore
.github
   |-- workflows
   |   |-- main.yml
.gitignore
Dockerfile
go.mod
kubernetes
   |-- main.yml
main.go

3. Configuring GitHub Actions Secrets

To automate deployments, we'll use GitHub Actions secrets to store sensitive information like your OCI credentials and cluster details.

• Navigate to your repository's Settings > Secrets & variables > Actions.
• Create the following repository's secrets, replacing the placeholders with your actual values:
  • OKE_CLUSTER_OCID: The cluster id of OKE cluster, available at cluster details page.
  • OCI_CLI_REGION: Your OCI region (e.g., sa-saopaulo-1).
  • OCI_CLI_TENANCY: Your OCI tenancy OCID.
  • OCI_CLI_USER: Your OCI user OCID.
  • OCI_CLI_FINGERPRINT: Your OCI user's public key fingerprint.
  • OCI_CLI_KEY_CONTENT: Your OCI user's private key content.
  • OCIR_AUTH_TOKEN: Your OCI Registry authentication token (generated in your OCI Console).
  • OCIR_REGISTRY: Your OCI Registry domain (e.g., ocir.oraclecloud.com).
  • OCIR_USERNAME: Your OCI Registry username (namespace + email address, e.g., sbataw3dd/[e-mail]).
  • OCIR_USERNAME_EMAIL: The email from username
  • OCIR_REPO_NAME: The name of your OCI Registry repository.
  • OCI_NETWORK_SG_BACKEND: The OCID of the Network Security Group attached to your OKE cluster's worker nodes.
  • OCI_TENANCY_NAMESPACE: The namespace generated automatically for your OCI tenancy's Object Storage.

4. Creating Your GitHub Actions Workflow

Create a file named main.yml within your workflows directory. This file will define the steps for your automated deployment process. The file we will create is available in example GitHub repository for this article, link at the end of the article.

Before we proceed, we have to back to OCI Console and certify that is possible to create a new container registry repository at first push to root compartment.

Navigate to Developer Services > Containers & Artifacts > Container Registry

• Click Settings, check Create repository on first push in root compartment option and Save changes.

Fine, let's finally to GitHub Actions Workflow

name: Build, push and deploy example

on:
  push:
    branches:
      - main
  pull_request:
    branches:
      - main
jobs:
  build-and-publish-docker-image:
    name: Build and publish Docker image
    runs-on: ubuntu-latest
    steps:
      - name: Checkout
        uses: actions/checkout@v4
      - name: Set up QEMU
        uses: docker/setup-qemu-action@v3
      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3
      - name: Login to Oracle Cloud Registry
        uses: docker/login-action@v3
        with:
          registry: ${{ secrets.OCIR_REGISTRY }}
          username: ${{ secrets.OCIR_USERNAME }}
          password: ${{ secrets.OCIR_AUTH_TOKEN }}
      - name: Build Docker image
        uses: docker/build-push-action@v6
        with:
          platforms: linux/arm64
          context: .
          file: Dockerfile
          tags: ${{ secrets.OCIR_REGISTRY }}/${{ secrets.OCI_TENANCY_NAMESPACE }}/${{ secrets.OCIR_REPO_NAME }}:latest
          push: false
          load: true
          cache-from: type=gha
          cache-to: type=gha,mode=max
      - name: Push Docker Image to Oracle Cloud Registry
        run: |
          docker push ${{ secrets.OCIR_REGISTRY }}/${{ secrets.OCI_TENANCY_NAMESPACE }}/${{ secrets.OCIR_REPO_NAME }}:latest

  deploy-to-oke:
    needs: build-and-publish-docker-image
    name: Deploy container image to an OKE Cluster
    runs-on: ubuntu-latest 
    env:
      OCI_CLI_USER: ${{ secrets.OCI_CLI_USER }}
      OCI_CLI_TENANCY: ${{ secrets.OCI_CLI_TENANCY }}
      OCI_CLI_FINGERPRINT: ${{ secrets.OCI_CLI_FINGERPRINT }}
      OCI_CLI_KEY_CONTENT: ${{ secrets.OCI_CLI_KEY_CONTENT }}
      OCI_CLI_REGION: ${{ secrets.OCI_CLI_REGION }}
      OCI_NETWORK_SG_BACKEND: ${{ secrets.OCI_NETWORK_SG_BACKEND }}
      IMAGE_NAME: ${{ secrets.OCIR_REGISTRY }}/${{ secrets.OCI_TENANCY_NAMESPACE }}/${{ secrets.OCIR_REPO_NAME }}:latest
      DOCKER_REGISTRY_SECRET_NAME: 'name-of-your-secret'
      NAMESPACE: 'name-of-desired-kubernetes-namespace'
    steps:
      - name: Configure Kubectl
        uses: oracle-actions/configure-kubectl-oke@v1.5.0
        with:
          cluster: ${{ secrets.OKE_CLUSTER_OCID }}
      - name: Checkout
        uses: actions/checkout@v4
      - name: Create namespace if not exists
        run: |
          kubectl create namespace ${{ env.NAMESPACE }} || echo "namespace already exists"
      - name: Create Docker registry secret
        run: | 
          kubectl -n preview create secret docker-registry ${{ env.DOCKER_REGISTRY_SECRET_NAME }} \
          --docker-server=${{ secrets.OCIR_REGISTRY}} \
          --docker-username=${{ secrets.OCIR_USERNAME}} \
          --docker-password='${{ secrets.OCIR_AUTH_TOKEN }}' \
          --docker-email=${{ secrets.OCIR_USERNAME_EMAIL}} || echo "secret already exists"
      - name: Deploy to Kubernetes Cluster
        run: |
          envsubst < kubernetes/main.yml | kubectl apply -f - -n ${{ env.NAMESPACE }}

This workflow is triggered on push and pull request events to the main branch. Let's dive deeper in the workflow and understand better the jobs and steps of workflow:

build-and-publish-docker-image Job

This job is responsible for building and publishing a Docker image.

1. Checkout: This step checks out the code from the GitHub repository inside the runner environment. It uses the actions/checkout@v4 official GitHub action.

2. Set up QEMU: This step sets up QEMU, a virtual machine emulator, for building and testing Docker images. It uses the docker/setup-qemu-action@v3 official Docker action.

3. Set up Docker Buildx: This step sets up Docker Buildx, a tool for building and pushing Docker images. It uses the docker/setup-buildx-action@v3 official Docker action.

4. Login to Oracle Cloud Registry: This step logs in to the Oracle Cloud Registry using the provided credentials (username, password, and registry URL). It uses the docker/login-action@v3 official Docker action.

5. Build Docker image: This step builds a Docker image using the docker/build-push-action@v6 official Docker action. The build context is the current directory (.), the Dockerfile is used as the build script, and the image is tagged with the Oracle Cloud Registry URL and the latest tag. The push option is set to false to prevent the image from being pushed to the registry immediately, for some weird reason, when pushing direct to OCIR the image created it's split into layers.

6. Push Docker Image to Oracle Cloud Registry: This step pushes the built Docker image to the Oracle Cloud Registry using the docker push command.

deploy-to-oke Job

This job is responsible for deploying the built Docker image to an Oracle Kubernetes Engine (OKE) cluster.

1. Needs: This directive specifies that the build-and-publish-docker-image job must complete successfully before this job can start.

2. Configure Kubectl: This step configures the kubectl command-line tool to connect to the specified OKE cluster. It uses the oracle-actions/configure-kubectl-oke@v1.5.0 action.

3. Checkout: This step checks out the code from the GitHub repository inside the runner environment. It uses the actions/checkout@v4 official Docker action.

4. Create namespace if not exists: This step creates a Kubernetes namespace if it does not already exist. It uses the kubectl create namespace command.

5. Create Docker registry secret: This step creates a Kubernetes secret to store the Oracle Cloud Registry credentials. It uses the kubectl create secret command.

6. Deploy to Kubernetes Cluster: This step deploys the Docker image to the OKE cluster using the kubectl apply command. The deployment configuration is read from a file named kubernetes/main.yml and the envsubst command is used to replace environment variables in the file.

If all occurs well in the next push or pull request to the main branch of our repository will trigger the action and it will build and publish the image to OCI private registry and deploy the Dockerized application to Kubernetes cluster.

The OKE itself will provision an OCI NLB and will set up all network related config and security rules. At the end, you will have an NLB with a public IPV4 that you can aim for a domain, also it's possible to set up a reserved IP in Kubernetes service type manifest, please for more details of how to, read OCI documentation.

GitHub Action executed successfully:

Pods running:

NAME                              READY   STATUS    RESTARTS   AGE
go-hello-world-6cbdd98797-hs7lv   1/1     Running   0          7m10s
go-hello-world-6cbdd98797-t4d5f   1/1     Running   0          7m10s

NLB provisioned by OKE Cluster:

Conclusion

This is a simple guide to automate the deployment of your applications to Kubernetes cluster and empower you to move from any PaaS with Docker approach to a public cloud. You can include job workflows to test your code before deploying it to a Kubernetes cluster, also you can use GitHub Actions workflows to improve your entire CI/CD cycle managing multiple environments (e.g., production, preview, staging).

I hope this article has been useful for you, let me know if you have any insights or comments about it. See you!

Links & references

• GitHub example repository
• GitHub Actions documentation
• OCI Docs:
  • Pushing Images Using the Docker CLI
  • Setting Up Cluster Access
  • Provisioning OCI Network Load Balancers for Kubernetes Services of Type LoadBalancer
  • Specifying Network Security Groups

Comments

[Mike Stemle]

This article describes a number of approaches which break security best practices. In addition to missing on the principle of least privilege, commingling your build and deployment stages with the same set of permissions can be disastrous in the event of a supply-chain attack.

I love GitHub actions, but these approaches are dangerous.