Deploying and Securing an App on AWS EKS with Gitlab CI/CD and Checkov

Introduction

Deploying an application on AWS EKS (Elastic Kubernetes Service) can be a powerful way to ensure scalability and reliability for your application. However, the process can be complex and time-consuming, especially when it comes to ensuring the security and compliance of your deployment. In this article, we'll show you how to simplify the process and ensure your deployment is secure with GitLab CI/CD and Checkov. GitLab CI/CD provides a powerful toolset for automating the deployment process and improving collaboration among team members, while Checkov is a Security as Code tool that can help you automatically scan your configuration files for potential security and compliance issues. By integrating these tools into your deployment pipeline, you can ensure your deployment is secure and compliant with industry best practices, all while saving time and effort.

Prerequisites

Before proceeding you will need the following :

• Set up a GitLab project with runners to execute CI/CD jobs

• A container registry (a docker hub repo is more than enough)

• A running AWS EKS cluster

• Some knowledge of Docker and Kubernetes

The different steps are as follows

1. Set up the application code and the Dockerfile

2. Define CI/CD GitLab variables

3. Set Kubernetes manifest files

4. Set up the CI/CD pipeline

5. Trigger the pipeline with a git push.

Directory structure

├── Dockerfile
├── .gitlab-ci.yml
├── .k8s
│   ├── deployment.yaml
│   └── services.yaml
└── src
    ├── app.py
    └── requirements.txt

The application files and the Kubernetes configurations are respectively in the src and .k8s directories. and the Dockerfile and the GitLab-ci script are at the root of the directory.

Set up the application code and the Dockerfile

Use whatever language or framework you want to create the application you want, the main thing is to have an application that you can containerize with a Dockerfile. Personally, I used a simple Python code that uses the Flask framework to create a web application that displays a "Hello, World!" message.

For the Dockefile, here is an example of what it could look like :

FROM python:3.9-slim-buster

WORKDIR /app

COPY src/requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt

COPY src/app.py .

CMD ["python", "./app.py"]

I recommend you test your docker image locally before continuing.

Define CI/CD GitLab variables

To connect to AWS, Kubernetes, and Docker Hub from GitLab CI, you need to define variables in the GitLab CI/CD pipeline. You can define these variables in the GitLab project settings under CI/CD > Variables.

To connect to AWS

• ${AWS_ACCESS_KEY_ID}: This variable contains the access key ID for the AWS account used to deploy the application.

• ${AWS_SECRET_ACCESS_KEY}: This variable contains the secret access key for the AWS account used to deploy the application.

• ${AWS_DEFAULT_REGION}: This variable contains the AWS region where the application will be deployed.

The variables related to the Docker hub or container registry

• ${CI_REGISTRY_USER}: This variable contains the username used to authenticate with Container Registry, it can be docker hub, GitLab registry or whatever you want

• ${CI_REGISTRY_PASSWORD}: This variable contains the password used to authenticate with the Container Registry.

• ${CI_REGISTRY_IMAGE}: This variable contains the name of the Docker image in the Container Registry.

• ${CI_REGISTRY_IMAGE_VERSION}: This variable contains the version or tag of the Docker image in the Container Registry.

The configuration file to access Kubernetes

• ${KUBECONFIG}: This variable of type file contains the Kubernetes configuration file used to authenticate with the Kubernetes cluster. this file is available at this path ~/.kube/config so when adding it to Gitlab make sure you choose File as the type

Set Kubernetes manifest files

Now it's time to define manifest files for Kubernetes deployments. As you would have seen above these files are located in the .k8s folder.

apiVersion: apps/v1
kind: Deployment
metadata:
  creationTimestamp: null
  labels:
    app: my-app
  name: my-app
spec:
  replicas: 1
  selector:
    matchLabels:
      app: my-app
  strategy: {}
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: my-app
    spec:
      containers:
      - name: my-app
        image: ${CI_REGISTRY_USER}/${CI_REGISTRY_IMAGE}:${CI_REGISTRY_IMAGE_VERSION}
        resources: {}
status: {}

deployment.yaml defines a Kubernetes Deployment for the application named "my-app". The Deployment creates a single replica of the application and specifies the container image to be used for the application using the variable ${CI_REGISTRY_USER}/${CI_REGISTRY_IMAGE}:${CI_REGISTRY_IMAGE_VERSION}. This variable references the Docker image built and pushed to a Docker registry.

apiVersion: v1
kind: Service
metadata:
  creationTimestamp: null
  labels:
    app: my-app
  name: my-app
spec:
  ports:
  - port: 80
    protocol: TCP
    targetPort: 5000
  selector:
    app: my-app
status:
  loadBalancer: {}

services.yaml defines Kubernetes Service for the "my-app" application. The Service exposes the application on port 80 and routes traffic to the listening port of our hello world app which is port 5000 on the application container. The file also specifies the labels to be used to identify the application in Kubernetes.

Hint: To generate a quick template on which to base and write these configuration files and thus reduce the risk of error and save time, there is an option of the kubectl command that can be very useful.

The --dry-run=client -o yaml option in the kubectl command generates a YAML representation of the Kubernetes resource that would be created or modified without actually creating or modifying the resource. Here is an example of how to generate our yaml file

#deployment
kubectl create deployment <app_name> \
     --image=<the_docker_image> \
     --dry-run=client -o yaml > deployment.yaml

#service
kubectl expose deployment <app_name> \
     --port=80 --target-port=5000 \
     --dry-run=client -o yaml > service.yaml

This should generate the two yaml files that you will adjust to fit your use. You can also use the --dry-run option with the kubectl command to validate a Kubernetes YAML file without actually applying it to a cluster.

To use the --dry-run option to validate a Kubernetes YAML file, you can run the following command:

kubectl apply --dry-run=client -f <yaml_file_path>

Set up the CI/CD pipeline

Now we can start setting up the GitLab script for our pipeline. The script in our case here will have 5 steps.

The script consists of the following stages:

• docker build: Builds the Docker image and tags it with the registry information.

• docker push: Pushes the Docker image to the registry.

• test: Runs the Checkov tool to validate the Kubernetes deployment and service files.

• deploy services: Deploy the Kubernetes services to EKS using the kubectl command.

• deploy the app: Deploys the Kubernetes application to EKS using the kubectl command.

Let's go a little further into the test phase, our test here is much more security oriented. we have integrated Checkov into the pipeline. Checkov is an open-source tool used for static code analysis of infrastructure-as-code (IAC) files. In this case, it will be used to perform security and compliance checks on the Kubernetes YAML files in the .k8s directory.

By running Checkov on the .k8s/deployments.yaml and .k8s/services.yaml files, the GitLab CI/CD pipeline can ensure that the Kubernetes resources being deployed meet the security and compliance requirements defined in the policies and rules being enforced by Checkov.

stages:
  - docker build
  - docker push
  - test
  - deploy services
  - deploy app


Build docker image:
  image: docker:stable
  stage: docker build
  before_script:
    - docker login -u ${CI_REGISTRY_USER} -p ${CI_REGISTRY_PASSWORD}
  script:
    - docker build -t the-app .
    - docker tag the-app:latest ${CI_REGISTRY_USER}/${CI_REGISTRY_IMAGE}:${CI_REGISTRY_IMAGE_VERSION} 

Push to registry:
  image: docker:stable
  stage: docker push
  before_script:
    - docker login -u ${CI_REGISTRY_USER} -p ${CI_REGISTRY_PASSWORD}
  script:
    - docker push ${CI_REGISTRY_USER}/${CI_REGISTRY_IMAGE}:${CI_REGISTRY_IMAGE_VERSION}

Test:
  image: bridgecrew/checkov:latest
  stage: test
  script:
    - checkov -d .k8s/deployments.yaml
    - checkov -d .k8s/services.yaml
  allow_failure: true



Deploy services on EKS:
  image: ${CI_REGISTRY_USER}/${CI_REGISTRY_IMAGE}:${CI_REGISTRY_IMAGE_VERSION}
  stage: deploy services
  before_script:
    - export AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID}
    - export AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY}
    - export AWS_DEFAULT_REGION=${AWS_DEFAULT_REGION}
  script:
    - cd .k8s
    - kubectl --kubeconfig ${KUBECONFIG} apply -f services.yaml
  rules:
    - changes:
      - .k8s/services.yaml


Deploy app on EKS:
  image: ${CI_REGISTRY_USER}/${CI_REGISTRY_IMAGE}:${CI_REGISTRY_IMAGE_VERSION}
  stage: deploy app
  before_script:
    - export AWS_ACCESS_KEY_ID=${AWS_ACCESS_KEY_ID}
    - export AWS_SECRET_ACCESS_KEY=${AWS_SECRET_ACCESS_KEY}
    - export AWS_DEFAULT_REGION=${AWS_DEFAULT_REGION}
  script:
    - cd .k8s
    - kubectl --kubeconfig ${KUBECONFIG} apply -f deployment.yaml
    - kubectl --kubeconfig ${KUBECONFIG} rollout status deployments

• Docker build stage: Builds a Docker image for the application specified by the Dockerfile using the official docker:stable image. Before building the image, it logs in to the Docker registry using the username and password provided as GitLab CI environment variables ${CI_REGISTRY_USER} and ${CI_REGISTRY_PASSWORD}. After building the image, it tags it with ${CI_REGISTRY_USER}/${CI_REGISTRY_IMAGE}:${CI_REGISTRY_IMAGE_VERSION}.

• Docker push stage: Pushes the Docker image created in the previous stage to the GitLab CI registry using the docker push command.

• Test stage: Runs checkov tool for Kubernetes manifest files deployment.yaml and services.yaml. in this case, this stage is allowed to fail and does not prevent the pipeline from continuing.

• Deploy services on EKS stage: Deploys the Kubernetes services specified in the services.yaml file to the Amazon Elastic Kubernetes Service (EKS) cluster. Before deploying, it sets the AWS credentials and region environment variables. It only runs the deployment if the services.yaml file has been modified since the last pipeline run.

• Deploy app on EKS stage: Deploys the Kubernetes deployment specified in the deployment.yaml file to the EKS cluster. Before deploying, it sets the AWS credentials and region environment variables. After deploying, it checks the rollout status of the deployment.

To make sure that this script is valid, you can use a lint tool to validate the script, it will help to quickly fix the errors and save time. For my part, I used glab CLI tool which with the command glab ci lint allows validating the script to make sure that everything is correct.

Trigger the pipeline with a git push.

To trigger this GitLab CI pipeline, you need to commit and push the code changes to the GitLab repository that contains this GitLab CI script.

Once you have pushed the changes to the repository, GitLab CI automatically detects the changes and starts running the pipeline. The pipeline can also be triggered manually by clicking the "CI/CD" tab in the GitLab repository and clicking the "Run Pipeline" button.

Note that to run the pipeline successfully, you need to ensure that you have configured the necessary environment variables on GitLab CI, such as CI_REGISTRY_USER, CI_REGISTRY_PASSWORD, AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY, AWS_DEFAULT_REGION, and KUBECONFIG. These variables are used to log in to the GitLab registry, authenticate with AWS, and connect to the Kubernetes cluster.

You can find the files I used here on my GitHub or GitLab

I remain open to any contribution and suggestion to improve my work. Do not hesitate to let me know your contribution or suggestion by opening an issue.

Thanks for reading :)