DEV Community: Israel Aminu

Dockerize and Deploy your Machine Learning Application on AWS

Israel Aminu — Sat, 27 Jun 2020 12:10:11 +0000

In my previous post, I talked about how you can containerize your Machine Learning application using Docker, but unfortunately, I was only able to build and deploy that application locally on my machine. In this article, I will show you how you can deploy that dockerized image to the cloud with AWS using AWS EC2 instance.

Pushing Image to Docker Hub

The first step is to push your custom image to a container registry, in this case, I’ll be using Docker Hub. To use Docker Hub you’ll have to create an account, you can do that here. Then head over to your terminal and run the following commands:

To check all the images you have locally, enter the command below:

israel@israel:~$ sudo docker images

For this article, the images I’ll be pushing to Docker Hub is the israelaminu/diabetes-prediction-model. To check how I built this image, you can check here.

If you’re doing this for the first time, you’ll have to log in to Docker Hub using the command below:

israel@israel:~$ sudo docker login --username=yourhubusername

Next, copy the IMAGE ID for that particular image and tag it using the command below:

israel@israel:~$ sudo docker tag b4bacf7fb8a4 yourhubusername/model-deployment:latest

The alphanumeric digit must match the IMAGE ID for that image, yourhubusername/model-deployment represents the repository you’re creating on Docker Hub and :latest is the tag.

Then push your image to Docker Hub using the repository you created with the command:

israel@israel:~$ sudo docker push yourhubusername/model-deployment

You should see the following output:

The push refers to repository [docker.io/israelaminu/model-deployment]
dc29bdeebb3b: Pushed 
87397f1e0ca2: Pushed 
0b2a46dd78d7: Pushed
abe258d83872: Pushed 
21fe8a57cd01: Pushed 
0f3ecf0004ca: Pushed 
...
55109ad1bdfa: Pushed
46829331b1e4: Pushed 
d35c5bda4793: Pushed 
a3c1026c6bcc: Pushed
f1d420c2af1a: Pushed 
461719022993: Pushed 
latest: digest: sha256:221424ffd34434c7834504980b845889722d3aa1bed0535fe666dbae25072357 size: 3676

If you go to Docker Hub, you’ll see your Docker Image in your repository.

Setting up AWS EC2 Instance

Scroll to the Launch Instance and click the Launch instance button

Next, choose an Amazon Machine Image (AMI), for my case, I’m selecting the Amazon Linux AMI 2018.03.0 (HVM) because it has all the dependencies and it’s free tier which means I can run the instance with little or no cost.

For the Instance Type, I selected the General Purpose t2.micro

Also in the Configure Security Group section, I created a new security group. The SSH allows me to connect to my instance on my machine and the HTTP routes my server IP to the DNS for me to make the DNS accessible anywhere.

Those are the configuration I’ll be doing. You can choose to set other configurations also.

Click on Review and Launch.

You will be prompted to create or use an existing pair key which allows you to SSH to your instance in your local machine. After you select the preferred option, click on Launch Instance
You should see the image below:

Run the Docker Image on the EC2 instance

SSH to your EC2 instance on your machine using the command below:

sudo ssh -i yourpairkey.pem ec2-user@my-instance-public-dns-name

You will see the following output:

Run the following to update your instance packages

[ec2-user@ip-172–31–52–68 ~]$ sudo yum update

Install Docker:

[ec2-user@ip-172–31–52–68 ~]$ sudo yum install docker

After installation, pull the docker image we pushed to the repository.

[ec2-user@ip-172–31–52–68 ~]$ docker pull israelaminu/model-deployment:latest

Testing and Making Predictions using POSTMAN

I tested the endpoint by passing in the features and the values to get the Diabetic Type of the patient and the confidence score, it took 492 milliseconds to return the response, which is fair enough.

So far we’ve been able to deploy a Machine Learning model by running it on a Docker Container and using hosting it on an AWS EC2 instance.

Using JSON Web Token(JWT) with Python

Israel Aminu — Fri, 15 May 2020 09:28:04 +0000

Authorization and security remain a key feature when building Web API for users or for an organization and knowing the amount of information you want to make accessible. Most web apps take security measures to make sure the user(s) information is safe and secure.

What is a JSON Web Token(JWT)?
JSON Web Token (JWT) is an open standard (RFC 7519) that defines a compact and self-contained way for securely transmitting information between a client and a server as a JSON object. This information can be verified and trusted because it is digitally signed. - jwt.io

How JWT Works?
A server generates a token that certifies the user identity and sends it to the client. The client will send the token back to the server for every subsequent request to an endpoint to access a particular service, the client can send the token via a header or a query parameter so the server then knows the request comes from a particular identity or user. Also, we can set the validity of the token by setting an elapsed time for the token to expire. Whereafter the user is authenticated when we perform API requests either to a REST API.

To read more about JWT, you can check here

Using JWT in python

pip install pyjwt

Example Usage



>>> import jwt
>>> secretKey = 'secret'
>>> encoded_jwt = jwt.encode({'some': 'payload'},secretKey, algorithm='HS256')
>>> encoded_jwt
'eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJzb21lIjoicGF5bG9hZCJ9.4twFt5NiznN84AWoo1d7KO1T_yoc0Z6XOpOVswacPZg'

>>> jwt.decode(encoded_jwt,secretKey, algorithms=['HS256'])
{'some': 'payload'}

What just happened is simply encoding the information present in that JSON as a token using a secret key and decrypting it to access the information. I'll show you a little demo of how I implemented a JSON web token using flask, you can also apply the same logic to Django also.

My Working Directory.



.
├── app.py
├── authenticate.py

authenticate.py

This is the python file I use to accept and verify tokens and then use it as a python decorator in my flask app.



import json
import jwt
from flask import Flask, request

flask_app = Flask(__name__)

SECRET_KEY = "hkBxrbZ9Td4QEwgRewV6gZSVH4q78vBia4GBYuqd09SsiMsIjH"

def token_required(something):
    def wrap():
        try:
            token_passed = request.headers['TOKEN']
            if request.headers['TOKEN'] != '' and request.headers['TOKEN'] != None:
                try:
                    data = jwt.decode(token_passed,SECRET_KEY, algorithms=['HS256'])
                    return something()
                except jwt.exceptions.ExpiredSignatureError:
                    return_data = {
                        "error": "1",
                        "message": "Token has expired"
                        }
                    return flask_app.response_class(response=json.dumps(return_data), mimetype='application/json'),401
                except:
                    return_data = {
                        "error": "1",
                        "message": "Invalid Token"
                    }
                    return flask_app.response_class(response=json.dumps(return_data), mimetype='application/json'),401
            else:
                return_data = {
                    "error" : "2",
                    "message" : "Token required",
                }
                return flask_app.response_class(response=json.dumps(return_data), mimetype='application/json'),401
        except Exception as e:
            return_data = {
                "error" : "3",
                "message" : "An error occured"
                }
            return flask_app.response_class(response=json.dumps(return_data), mimetype='application/json'),500

    return wrap


#

I used the request.headers to accept the token from the client using the "TOKEN" header variable, then I try to decrypt the token, if the token is valid, the client will have access to that endpoint. If the token has expired or the client decides to use and old token it will return the "Token has Expired" response and throw a 401, also if the client tries to create any form of token that's not correct he'll get an "Invalid token" response. If no token is passed to access a particular endpoint in my web app it will ask for a token to access the endpoint. And if there's any other issue it will return "An error occurred" response.

app.py



import json
import jwt
import datetime
from authenticate import token_required #The token verification script
from flask import Flask, request

SECRET_KEY = "hkBxrbZ9Td4QEwgRewV6gZSVH4q78vBia4GBYuqd09SsiMsIjH"

flask_app = Flask(__name__)

@flask_app.route('/loginEndpoint', methods=['POST'])
def loginFunction():
    userName = request.form.get('username')
    passWord = request.form.get('password')
    #Generate token
    timeLimit= datetime.datetime.utcnow() + datetime.timedelta(minutes=30) #set limit for user
    payload = {"user_id": userName,"exp":timeLimit}
    token = jwt.encode(payload,SECRET_KEY)
    return_data = {
        "error": "0",
        "message": "Successful",
        "token": token.decode("UTF-8"),
        "Elapse_time": f"{timeLimit}"
        }
    return flask_app.response_class(response=json.dumps(return_data), mimetype='application/json')


@flask_app.route('/anEndpoint',methods=['POST'])
@token_required #Verify token decorator
def aWebService():
    return_data = {
        "error": "0",
        "message": "You Are verified"
        }
    return flask_app.response_class(response=json.dumps(return_data), mimetype='application/json')

if __name__ == "__main__":
    flask_app.run(port=8080, debug=True)

In the python file I imported the necessary libraries and also the python script where I use to authenticate client tokens.

The flask app contains a simple login function which requests a username and password, then a token is generated which stores the username to the token and also the token also expires 30 mins from when it is generated, after that the token will no longer be valid. When the token is generated it will be sent as a response to the client, which the client can then use to access other endpoints that require a token

To access the second endpoint created above, the user needs to pass in the token generated hence the decorator @token_required above the function, if the token is correct the client will be able to access the services in that function and if the token is wrong or expired the client cannot have access to that service.

And there you have it, that's how you can simply add a JSON web token(JWT) to your REST API python project to authenticate your users or client.

Build and Deploy your Machine Learning Application with Docker

Israel Aminu — Wed, 22 Apr 2020 08:04:49 +0000

Ever deployed a Machine Learning model that works perfectly fine on your computer locally, but then the code breaks on another machine or worse when it's been deployed into production? Well, in this article I will walk you through how you can use a popular tool named 'Docker' to run and also deploy your Machine Learning model(s).

So what is Docker?

Docker is a tool that makes it easier to create, deploy and run any application by using what is called a container. It's also a software platform, which is used to create Docker images that will be referred to as a Docker container once it's been deployed.

A Docker Container is an isolated environment which contains all the required dependencies for your application to run, it is often referred to as a running instance of a Docker image.

A Docker image is a file(read-only), comprised of multiple layers, that is used to execute code in a Docker container. Docker images are found in a large hub which is referred to as Docker Hub. So you pull images from the hub or you build a custom image from a base image and when these images are being executed they serve as containers for your application.

So combining the pieces together we can simply define Docker as:

A Software platform which makes it easier to create and deploy any application by creating a Docker image which will then be a Docker container which contains all the dependencies and packages we need for our application to work once it's been deployed.

Benefits of Docker

Docker solves the problem of having an identical environment across various stages of development and having isolated environments in your individual applications.
Docker allows you to run your application from anywhere as long as you have docker installed on that machine.
Docker gives you the liberty to scale up quickly.
Expand your development team painlessly.

Installing Docker

Docker is available across various platforms whether if you're using a Linux, Windows or a Mac computer, you can follow the installation guide here

Now that we've understood the basics of Docker and you've gotten Docker running on your machine, let us go ahead and deploy a Machine Learning Application with it.

My Working directory

For the model I want to deploy, this is how my working directory looks like:

.
├── app.py
├── Dockerfile
├── ML_Model
│   ├── Diabetestype.csv
│   ├── model.pkl
│   └── model.py
└── requirements.txt
1 directory, 6 files

app.py
The app.py is a python script which contains the API I built for my Machine Learning model using flask. I defined the API endpoint and the path, how we receive data from the web, how the data is being processed and how predictions are being returned as a response.

import json
import pickle
import numpy as np
from flask import Flask, request
# 

flask_app = Flask(__name__)

#ML model path
model_path = "ML_Model/model.pkl"


@flask_app.route('/', methods=['GET'])
def index_page():
    return_data = {
        "error" : "0",
        "message" : "Successful"
    }
    return flask_app.response_class(response=json.dumps(return_data), mimetype='application/json')

@flask_app.route('/predict',methods=['GET'])
def model_deploy():
    try:
        age = request.form.get('age')
        bs_fast = request.form.get('BS_Fast')
        bs_pp = request.form.get('BS_pp')
        plasma_r = request.form.get('Plasma_R')
        plasma_f = request.form.get('Plasma_F')
        HbA1c = request.form.get('HbA1c')
        fields = [age,bs_fast,bs_pp,plasma_r,plasma_f,HbA1c]
        if not None in fields:
            #Datapreprocessing Convert the values to float
            age = float(age)
            bs_fast = float(bs_fast)
            bs_pp = float(bs_pp)
            plasma_r = float(plasma_r)
            plasma_f = float(plasma_f)
            hbA1c = float(HbA1c)
            result = [age,bs_fast,bs_pp,plasma_r,plasma_f,HbA1c]
            #Passing data to model & loading the model from disk
            classifier = pickle.load(open(model_path, 'rb'))
            prediction = classifier.predict([result])[0]
            conf_score =  np.max(classifier.predict_proba([result]))*100
            return_data = {
                "error" : '0',
                "message" : 'Successfull',
                "prediction": prediction,
                "confidence_score" : conf_score
            }
        else:
            return_data = {
                "error" : '1',
                "message": "Invalid Parameters"             
            }
    except Exception as e:
        return_data = {
            'error' : '2',
            "message": str(e)
            }
    return flask_app.response_class(response=json.dumps(return_data), mimetype='application/json')


if __name__ == "__main__":
    flask_app.run(host ='0.0.0.0',port=8080, debug=False)

ML_Model
The ML_Model directory contains the ML model, the data I used to train the model and the pickle file generated after model is being trained which the API will make use of.

requirements.txt
The requirements.txt file is a text file which contains all the required python packages we need for our application to run. Some of the packages I made of use were:

Flask==1.1.2
pandas==1.0.3
numpy==1.18.2
sklearn==0.0

Dockerfile
A Dockerfile is a text file that defines a Docker image. You'll use a Dockerfile to create your own custom Docker image when the base image you want to use for your project doesn't meet your required needs. For the model I'll be deploying, this is how my Dockefile looks like:

#I specify the parent base image which is the python version 3.7
FROM python:3.7

MAINTAINER aminu israel <aminuisrael2@gmail.com>

# This prevents Python from writing out pyc files
ENV PYTHONDONTWRITEBYTECODE 1
# This keeps Python from buffering stdin/stdout
ENV PYTHONUNBUFFERED 1

# install system dependencies
RUN apt-get update \
    && apt-get -y install gcc make \
    && rm -rf /var/lib/apt/lists/*

# install dependencies
RUN pip install --no-cache-dir --upgrade pip

# set work directory
WORKDIR /src/app

# copy requirements.txt
COPY ./requirements.txt /src/app/requirements.txt

# install project requirements
RUN pip install --no-cache-dir -r requirements.txt

# copy project
COPY . .

# Generate pikle file
WORKDIR /src/app/ML_Model
RUN python model.py

# set work directory
WORKDIR /src/app

# set app port
EXPOSE 8080

ENTRYPOINT [ "python" ] 

# Run app.py when the container launches
CMD [ "app.py","run","--host","0.0.0.0"]

In my Dockerfile, I pulled the Docker base image which is python:3.7, updated the system dependencies, installed the packages in the requirements.txt file, ran the ML code to train the model and generate the pickle file which the API will use and lastly run the server locally.

Now let's build our Docker image from the Dockerfile we've created using this command:

israel@israel:~/Documents/Projects/Docker_ML$ docker build aminu_israel/ml_model:1.0 .

I named my custom image "aminu_israel/ml_model" and set the tag to 1.0. Notice the "." at the end of the command, it means I'm telling Docker to locate the Dockerfile in my current directory, which is my project folder. If it's successful you should have a result like this:

Sending build context to Docker daemon  249.3kB
Step 1/16 : FROM python:3.7
 ---> cda8c7e31f89
Step 2/16 : MAINTAINER aminu israel <aminuisrael2@gmail.com>
 ---> Running in cea1c80b990f
Removing intermediate container cea1c80b990f
 ---> 2c82fc9c1b5a
Step 3/16 : ENV PYTHONDONTWRITEBYTECODE 1
 ---> Running in 6ee3497a7ff4
Removing intermediate container 6ee3497a7ff4
 ---> 56f5f9838610
Step 4/16 : ENV PYTHONUNBUFFERED 1
 ---> Running in 1f53b581eed7
...

Step 16/16 : CMD [ "app.py","run","--host","0.0.0.0"]
 ---> Running in 1f7fc05b4e12
Removing intermediate container 1f7fc05b4e12
 ---> 8636b5bc482e
Successfully built 8636b5bc482e
Successfully tagged aminu_israel/ml_model:1.0

You can check the new image you've created using this command:

israel@israel:~$ docker images

Now that we've successfully built image, lets run the docker images using the command:

israel@israel:~$ docker run --name deployML -p 8080:8080 aminu_israel/ml_model:1.0

If successful you should see a result like this:

* Serving Flask app "app" (lazy loading)
 * Environment: production
   WARNING: This is a development server. Do not use it in a production deployment.
   Use a production WSGI server instead.
 * Debug mode: off
 * Running on http://0.0.0.0:8080/ (Press CTRL+C to quit)

To check if your docker container is running, use this command:

israel@israel:~$ docker ps

And you'll see a result like this:

CONTAINER ID        IMAGE                       COMMAND                  CREATED             STATUS              PORTS                    NAMES
dc5c417d893f        aminu_israel/ml_model:1.0   "python app.py run -…"   24 seconds ago      Up 20 seconds       0.0.0.0:8080->8080/tcp   deployML

Which shows that the new container is currently running. To get full docker documentation, you can check here

And there you have it, you've successfully deployed your ML model using docker.

You can get the code for this article here

Thanks for reading 😀