DEV Community: Alvaro Valarezo de la Fuente

Deploy a ML model using Google Cloud Run, Github Actions and Terraform

Alvaro Valarezo de la Fuente — Tue, 25 Oct 2022 14:17:12 +0000

In this post. I will explain how to expose an API in Python using FastAPI from a trained model, use best CI/CD practices (Github Actions) and IaC (Terraform) to automate infrastructure creation.

Prerrequisites

Docker Desktop
Git
Github Account
Google Cloud Platform with owner permissions
Clone this repo

Google Cloud Run

Cloud Run is a serverless platform from Google Cloud to deploy and run containers. Cloud Run can be used to serve Restful web APIs, WebSocket applications, or microservices connected by gRPC.

In this project we will need:

An IAM account with permissions to create a service account
Cloud Storage Admin permissions
Cloud Registry Admin permissions
Google Cloud Run Admin permissions

In case you don't want to expose the API for public access:

In the terraform/main.tf:

Remove the resource "google_cloud_run_service_iam_member" "run_all_users".

Ideally, you can set the IAM accounts that can access this api using Google Cloud Run UI or using Terraform. This approach doesn't add any latency to the customer because it uses built-in IAM roles and permissions from Google Cloud.

Terraform

Terraform is a popular open-source tool for running infrastructure as code. It uses HCL which is a declarative language to declare infrastructure.
The basic flow is:

Terraform init: Initializes the plugins, backend and many config files Terraform uses to keep tracking of the infrastructure.
Terraform plan: Generates an execution plan for all the infrastructure which is in terraform/main.tf
Terraform apply: Apply all the changes that were on the plan.

All steps are declared in the .github/workflows/workflow.yaml

Machine Learning Model

It's a logistic regression model that is serialized as a pickle file. It takes as an input a list with 37 parameters and returns a number between 0-1 which determines the probability of a delayed flight.

Eg:

  {
"test_array": [
0,0,0,0,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,0,1,0,0,1,0,1,0,0,1,1,1,0,1,0,1,0]

}

Returns: 0
In this case it means the flight isn't going to be delayed.

Run it locally

Fork the repo
Clone it in your computer
Run docker build -t ml-api . in the root of the project to build the image of the api.
Run docker run -d --name ml -p 80:8080 ml-api to create the container using ml-api image built.
Open localhost to test the project.
On /predict/ post endpoint, you can use this body as an example:

  {
"test_array": [
0,0,0,0,0,1,0,1,0,0,1,0,1,0,1,0,1,0,1,0,0,1,0,0,1,0,1,0,0,1,1,1,0,1,0,1,0]

}

You should expect a response 200 with a "prediction": 0 which means the flight wasn't delayed.

How to deploy it with your GCP account

Generate a Service Account key and upload it in Github Secrets as GCLOUD_SERVICE_KEY
Push any change in the main branch
That's it! :)

A bit of stress testing for the API

On Mac brew install wrk
Run wrk -t12 -c200 -d45s -s request.lua https://mlops-api-backend-1-5gdi5qltoq-uc.a.run.app/predict/ to open 12 threads with 200 open http connections during 45 seconds.

How can we improve the results

The best approach would be using horizontal scaling. in this case we can create a 2nd Google Cloud Run instance and use load balancing to distribute the traffic between both instances.

Develop an API with FastApi, Kubernetes, Traefik and Ansible

Alvaro Valarezo de la Fuente — Mon, 19 Sep 2022 07:00:12 +0000

kubernetes-fastapi-traefik

Backend API (Python) in FastApi in a Kubernetes Cluster using Traefik as an API Gateway and Ansible as a configuration management tool.

Clone Github repo

Clone my repo and follow the steps below

Description of the directories

ansible : Folder with the resources to deploy the traefik pods using Ansible
kubernetes: Folder with the YAML files to deploy the API and the Postgres Database
sa-service-2: Folder with the source code of the API

Requirements for installing the project

Docker

To install Docker: https://docs.docker.com/desktop/mac/install/

Note: In Docker Desktop Preferences, enable Kubernetes to run the cluster

Ansible

To install ansible use pip: python -m pip install --user ansible

Minikube (kubectl)

Follow these steps: https://minikube.sigs.k8s.io/docs/start/

Python (PIP)
openshift
pyyaml
kubernetes

To install python libraries run Eg: pip install openshift

Helm V3 (Kubernetes Package Manager)

To install run brew install helm

1.- Steps to start the K8s cluster

After installing everything above and running Docker Desktop in your PC

Run in your terminal:

minikube start (Used to initiate a default local cluster with the default namespace)
minikube tunnel (Used to get an external ip of the cluster to access in the host OS) (Run it in a separate terminal, Don’t close it)
kubectl proxy (Used to forwading the port 8001 to access the k8s dashboard)
Go to: http://127.0.0.1:8001/api/v1/namespaces/kubernetes-dashboard/services/http:kubernetes-dashboard:/proxy/#/deployment?namespace=default

In this URL, you can check that there aren’t any deployments yet in the default namespace.

2.- Steps to deploy Traefik using Ansible

The Traefik Service is managed by using Ansible, so you can open the file ansible/templates/traefik_values_default.yml.j2 where you can check that the type of deployment is a DaemonSet

Traefik listens on port 80 on all interfaces of the host for incoming HTTP requests
Traefik listens on port 443 on all interfaces of the host for incoming HTTPS request
Traefik dashboard is enabled and is not exposed to the public internet.

Ports exposed:


# To access the dashboard you can use "kubectl port-forward" e.g.:

# kubectl -n traefik port-forward $(kubectl get pods --selector "app.kubernetes.io/name=traefik" --output=name -A | head -1) 9000:9000

# Opening http://localhost:9000/dashboard/ should show the dashboard.

traefik:

port: 9000

expose: true

protocol: TCP

Resources:


# CPU and RAM resource limits. These settings should also be set to

# reasonable values in case of a memory leak e.g.

resources:

requests:

cpu: "100m"

memory: "50Mi"

limits:

cpu: "300m"

memory: "150Mi"

In the ansible/defaults/main.yml, you can change the namespace for the deployment (Traefik) , name for the ingress controller,etc.

The ansible playbook will detect these settings and apply them to the cluster when it runs the playbook.

In the ansible/k8s.yml you can look that the host is traefik. to recognize this dns we need to edit the /etc/ansible/hosts file.

** If you don’t have the file or folder, please create it using sudo mkdir /etc/ansible and sudo touch hosts then edit it:

and paste this:


[traefik]

localhost

We are telling Ansible that the traefik dns will translate to localhost

After doing this we are ready to deploy:

To deploy the Traefik Service in the traefik namespace:

cd ansible
ansible-playbook --tags=role-traefik-kubernetes --extra-vars action=install k8s.yml

One of the final tasks is called TASK [githubixx.traefik-kubernetes : Output rendered template]. This allows to check the YAML file before Traefik gets deployed

If everything is OK , run kubectl -n traefik get pods -o wide and you will see a pod with the traefik initials.

To forward the ports to check the traefik dashboard, run:

kubectl -n traefik port-forward $(kubectl get pods --selector "app.kubernetes.io/name=traefik" --output=name -A | head -1) 9000:9000

Open:

http://localhost:9000/dashboard/#/http/services to check that everything is working correctly.

Steps to deploy the Postgres Database

To deploy the database in the K8s cluster, we are going to use diferent YAML files.

cd kubernetes

In this folder, there are 4 YAML files for the Postgres Deployment.

postgres-configmap.yaml A K8s ConfigMap allows me to save important variables like the name of the db, POSTGRES_USER, POSTGRES_PASSWORD.
postgres-storage.yaml 2 different kinds of volume: PersistentVolume and PersistentVolumeClaim used to preserve the data if a pod is deleted.
postgres-service.yaml A K8s Service that allows me to connect to the Postgres DB and with other pods in the same cluster.
postgres-deployment.yaml A K8s Deployment with a Postgres 10.1 image.

In the kubernetes folder:

Run:

kubectl apply -f postgres-configmap.yaml
kubectl apply -f postgres-storage.yaml
kubectl apply -f postgres-deployment.yaml
kubectl apply -f postgres-service.yaml
If you run kubectl get all you can check that everything is deployed.
Run kubectl port-forward --namespace default svc/postgres-ip-service 5432:5432 to port forwarding and access the database using localhost:5432
Connect to the database using your favorite DB client like Datagrip, DBeaver,etc.

Inside the users database, run this query:


CREATE TABLE rappiuser (

user_id serial PRIMARY KEY,

name VARCHAR ( 50 ) UNIQUE NOT NULL,

lastname VARCHAR ( 50 ) NOT NULL,

address VARCHAR ( 250 ) NOT NULL,

phone INTEGER NOT NULL,

age INTEGER NOT NULL,

hire_date DATE,

fire_date DATE




);

That’s all for the db configuration.

Steps to deploy the FastApi Backend

Finally, the last step is deploying the API.

cd sa-service-2

Inside the app folder you can check that is structured in models, schemas, services and utils with a main.py file that contains the routes.

Inside the sa-service-2 folder, run:

minikube image build -t sa-service-2 .

This command upload the backend image to the minikube local registry, to use it for the deployment.

cd .. return to the previous dir.
kubectl apply -f resource-manifests/service-two.yml Apply a Deployment, Service and an Ingress Controller.

Deployment:

Define the name for the deployment, and the image that will be used sa-service-2.

Service:

Define the port that will be exposed in this case is the Port 80

Ingress:

The Ingress Controller will be handled by Traefik and will listen in the port 80

Now you can go to http://localhost/docs and start testing the API :)

In the http://localhost:9000/dashboard/#/http/services you can check there is a new ingress controller that handles 2 pods