Deploying a Kubernetes application with ArgoCD and Gitlab CI

Hello there !

In the part 4, we deployed the Scaleway Infrastructure with GitLab and Terraform.

In this last part, we will deploy a simple application using ArgoCD and Gitlab.

Plan

• Building and publishing our Docker image using Gitlab and Google Cloud Build.
• Configuring and deploying our Docker image to Kubernetes using Gitlab and ArgoCD.

Docker image

As mentioned in the part 2, we need to build the docker image after each git tag. Once the new docker image is published we edit the Kubernetes manifests using Kustomize and an env repo pipeline is triggered from the app-repo pipeline.

That mechanism is described in the demo-app repo pipeline:

.gitlab-ci.yaml

stages:
  - publish

publish docker image:
  stage: publish
  image: 
    name: eu.gcr.io/${GCP_PROJECT_ID}/tools
  script: 
    - eval $(ssh-agent -s)
    - echo "$GITLAB_SSH_KEY" | tr -d '\r' | ssh-add -
    - mkdir -p ~/.ssh
    - chmod 700 ~/.ssh
    - '[[ -f /.dockerenv ]] && echo -e "Host *\n\tStrictHostKeyChecking no\n\n" > ~/.ssh/config'
    - ssh -T [e-mail]

    - git config --global user.name "${GITLAB_USER_NAME}"
    - git config --global user.email "${GITLAB_USER_EMAIL}"
    - git config --global push.followTags true

    - IMAGE_TAG=$CI_COMMIT_TAG-$CI_COMMIT_SHORT_SHA
    - cd src
    - gcloud config set project ${GCP_PROJECT_ID}
    - gcloud builds submit . --tag=eu.gcr.io/${GCP_PROJECT_ID}/demo:$IMAGE_TAG --project ${GCP_PROJECT_ID} --gcs-log-dir=gs://${GCP_PROJECT_ID}_cloudbuild/logs
    - git clone "[e-mail]:stack-labs/internal/sandbox/chabanerefes/meetup/demo-env.git"
    - cd demo-env/envs/dev
    - kustomize edit set image eu.gcr.io/${GCP_PROJECT_ID}/demo:$IMAGE_TAG
    - cd ../..
    - git add .
    - 'git commit -m "ci: update image to $IMAGE_TAG"'
    - git tag -a -m "New release available - $CI_COMMIT_TAG" $CI_COMMIT_TAG
    - git push -o ci.skip
  tags:
    - k8s-dev-runner
  only:
    refs:
      - tags
    changes:
      - src/**/*

Share the specific runner k8s-dev-runner created previously with this project. You will need Maintainer permission in Gitlab.

Now you can run the Gitlab pipeline with the following Gitlab CI/CD Variables:

GCP_PROJECT_ID=$GCP_PROJECT_ID

Deployment

In the demo-env repo we have our Kubernetes manifests. Each time the docker image version is edited and a git tag is created, the Gitlab pipeline is executed.

The following files describe the Kubernetes manifests to deploy:

base/demo-deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: demo
  labels:
    app: demo
spec:
  selector:
    matchLabels:
      app: demo
  template:
    metadata:
      labels:
        app: demo
    spec:
      containers:
      - image: eu.gcr.io/<GCP_PROJECT_ID>/demo:v0.1.0
        name: demo
        ports:
        - containerPort: 8080

base/demo-svc.yaml

apiVersion: v1
kind: Service
metadata:
  name: demo
spec:
  ports:
  - port: 80
    targetPort: 8080
  selector:
    app: demo

base/kustomization.yaml

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

# Adds namespace to all resources.
namespace: app-dev
resources:
- demo-deployment.yaml
- demo-svc.yaml

envs/dev/kustomization.yaml

namespace: app-dev
apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization
resources:
- ../../base

The ArgoCD application is also defined in a yaml file:

envs/dev/application.yaml

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: app-demo-dev
  namespace: argocd
spec:
  project: demo-dev
  source:
    repoURL: <GIT_REPOSITORY_URL>
    targetRevision: HEAD
    path: envs/dev
  destination:
    server: <SW_KAPSULE_CLUSTER_URL>
    namespace: app-dev
  syncPolicy:
    automated:
      prune: true

In the gitlab pipeline we have two stages init and deploy.

• The first stage adds Kapsule cluster in ArgoCD and initializes the dev environment in Kubernetes.
• The second stage creates and configures the application in ArgoCD.
• I added a third stage for manual sync on ArgoCD.

envs/dev/application.yaml

stages:
  - init
  - deploy
  - sync

# Get Scaleway credentials from Vault
before_script:
    - export VAULT_TOKEN="$(gcloud secrets versions access latest --secret=vault-token --project ${GCP_PROJECT_ID})"
    - export SCW_SECRET_KEY="$(vault kv get -field=key scaleway/project/${SW_PROJECT_NAME}/credentials/secret)"
    - export AROGOCD_TOKEN="$(gcloud secrets versions access latest --secret=argocd-token --project ${GCP_PROJECT_ID})"

init sw k8s project 🔬:
  stage: init
  when: manual
  image:
    name: eu.gcr.io/${GCP_PROJECT_ID}/tools
  script:
    # Connect to GCP GKE DevOps Cluster
    - gcloud container clusters get-credentials gke-cluster-devops --zone europe-west1-b --project ${GCP_PROJECT_ID}
    # Connect to scaleway
    - scw init secret-key=$SCW_SECRET_KEY
    # Get kubeconfig from sw kapsule cluster
    - scw k8s kubeconfig get $(scw k8s cluster list | grep kapsule-cluster-dev-demo | awk '{ print $1 }') region=fr-par > kapsule_config
    # Register kapsule cluster on Argocd
    - export KUBECONFIG=~/.kube/config:$(pwd)/kapsule_config
    - argocd cluster add admin@kapsuleclusterdevdemo --name kapsule-cluster-dev-demo --kubeconfig kapsule_config --auth-token=${AROGOCD_TOKEN} --server ${ARGOCD_ADDR} --grpc-web 
    # Create namespace on kapsule cluster
    - kubectl config use-context admin@kapsuleclusterdevdemo
    - kubectl create namespace app-dev || echo 'namespace app-dev already exists'
    # To access GCR service, create the json key file and associate it with the service account
    - gcloud iam service-accounts keys create sw-gcr-auth-ro.json --iam-account=sw-gcr-auth-ro@${GCP_PROJECT_ID}.iam.gserviceaccount.com
    - export GCR_SECRET_NAME=gcp-gcr-auth-ro
    - |
      kubectl create secret docker-registry $GCR_SECRET_NAME -n app-dev \
      --docker-server=https://eu.gcr.io \
      --docker-username=_json_key \
      --docker-email=ci@<MY_COMPANY>.com \
      --docker-password="$(cat sw-gcr-auth-ro.json)" || echo 'secret $GCR_SECRET_NAME already exists'
    - |
      kubectl patch serviceaccount default -n app-dev \
      -p "{\"imagePullSecrets\": [{\"name\": \"$GCR_SECRET_NAME\"}]}"
  tags:
    - k8s-dev-runner
  only:
    - master 

deploy sw k8s project 🚀:
  stage: deploy
  when: manual
  image:
    name: eu.gcr.io/${GCP_PROJECT_ID}/tools
  script:
    # Connect to scaleway
    - scw init secret-key=$SCW_SECRET_KEY
    # Get sw kapsule cluster url
    - export SW_KAPSULE_CLUSTER_URL=$(scw k8s cluster get $(scw k8s cluster list | grep kapsule-cluster-dev-demo | awk '{ print $1 }') | grep ClusterURL | awk '{ print $2 }' | tr -d '\r')
    - cd envs/dev
    - sed -i "s,<SW_KAPSULE_CLUSTER_URL>,$SW_KAPSULE_CLUSTER_URL,g;s,<GIT_REPOSITORY_URL>,$CI_PROJECT_URL.git,g" application.yaml
    # Connecto to gcp gke devops cluster
    - gcloud container clusters get-credentials gke-cluster-devops --zone europe-west1-b --project ${GCP_PROJECT_ID}
    # Create ArgoCD project
    - argocd proj create demo-dev -d $SW_KAPSULE_CLUSTER_URL,app-dev -s $CI_PROJECT_URL.git --auth-token=${AROGOCD_TOKEN} --server ${ARGOCD_ADDR} --grpc-web 
    # Create ArgoCD application
    - kubectl apply -n argocd -f application.yaml
  tags:
    - k8s-dev-runner
  only:
    - master

sync dev 🔨:
  stage: sync
  when: manual
  image:
    name: eu.gcr.io/${GCP_PROJECT_ID}/tools
  script:
    - argocd app sync app-demo-dev
  tags:
    - k8s-dev-runner
  only:
    - master

Share the specific runner k8s-dev-runner created previously with this project. You will need Maintainer permission in Gitlab.

Now you can run the Gitlab pipeline with the following Gitlab CI/CD Variables:

GCP_PROJECT_ID=$GCP_PROJECT_ID
SW_PROJECT_NAME=$SW_PROJECT_NAME
ARGOCD_ADDR=$ARGOCD_ADDR
VAULT_ADDR=$VAULT_ADDR
ENV=dev

That's it!

Final Words

The source code is available on Gitlab.

The demonstration is also available (but in 🇫🇷) on Youtube at 32:12.

If you have any questions or feedback, please feel free to leave a comment.

Otherwise, I hope I have helped you answer some of the hard questions about building a multi-cloud between an European cloud and an American cloud leader.

By the way, do not hesitate to share with peers 😊

Thanks for reading!