Deploy Existing Web Applications as Serverless using AWS Lambda, Zappa and DuploCloud

Serverless is the latest paradigm in cloud computing where software engineers can deploy their application without having to create or manage server infrastructure. In this tutorial, we describe how in about 30 mins, you can learn Serverless and deploy an existing python web application without making any code change to your existing application.

Further, using DuploLive, you won’t even need your own AWS account, let alone having to deal with any aws policies and configuration. Get an account for free @ https://www.duplocloud.com that will create a secure and isolated workspace for you in a common AWS account. This comes with $100 in free AWS usage. No credit card is needed.

Enjoy Serverless!!

Readers who are well versed with the basic concepts of Serverless can directly skip to the section “AWS Lambda, Zappa and DuploCloud”. Following is a video version of this tutorial

What is Server less

Traditionally, developers write their code and create a deployment package that would be deployed in servers in the cloud. The code would be packaged for deployment using packaging technologies like Docker, debian packages, Windows MSI etc and deployed to servers using deployment tools like Kubernetes, Chef etc.

In the Serverless model, instead of deploying on servers, developers deploy their application by providing the Cloud Platform (e.g. AWS) with:

1. Code Package and Infrastructure Configuration:

Code package needs to be in one of the supported languages (like python, java, NodeJs) from the cloud provider. The infrastructure configuration comprises of two categories

a) Application needs like CPU, Memory, Environmental variable and execution timeout. The provider will terminate the process after the timeout.

b) Infrastructure Policies: Permissions, network Access so that the execution environment has access to other components of the system like databases, other services (Serverless or otherwise), internet etc.

2. Triggers:

The code would be executed based on these trigger definitions. Some common triggers are REST endpoints in case of web applications, file change notifications from data stores like S3, event streams like kinesis.

Why and Why Not Server less?

AWS Lambda, Zappa and DuploCloud

Lambda is the Serverless platform by AWS. It supports Python, NodeJS, Java, and .NET Core. In this tutorial we will be deploying a web application so we will use AWS API gateway to host and expose the REST endpoints to the internet

Zappa

“ Zappa _ makes it super easy to build and deploy all Python WSGI applications on AWS Lambda + API Gateway. Think of it as “serverless” web hosting for your Python apps. That means infinite scaling, zero downtime, zero maintenance — and at a fraction of the cost of your current deployments!” — [_https://github.com/Miserlou/Zappa.](https://github.com/Miserlou/Zappa.)

Zappa has two core functions:

1. Code Packaging: It takes a legacy python web application and creates a Lambda compatible code package. Internally, it creates a lambda function that wraps the original application. Inside the Lambda function it parses the trigger and routes it to the right web API in the application. For example it hides all the REST endpoints defined in a flask App and instead exposes only the lambda function it has created. However, when the request is received, the function parses the request to route it to the appropriate flask API as an inline function call.
2. AWS Infrastructure Management: It automatically uploads packages to S3, creates IAM roles, policies, API Gateway configuration. However, it needs practically administrator level access to the user-provided AWS account. Further, it is a CLI tool so it has no notion of multi-developer isolation workspaces. DevOps tasks like integration of the Lambda deployment with rest of the application topology (like databases, ELB, data strores like DynamoDB) or other parts of the organization’s AWS deployment are out-of-scope.

Zappa’s infrastructure management scheme works for a Hobby developer or a very small team but are not really meant for an Enterprise-class environment where giving every developer these permissions is out of the question. In addition, there needs to be a substantial DevOps effort to put things together. This is where DuploCloud comes into the picture.

DuploCloud

While in a broader sense DuploCloud is a DevOps-as-a-service platform, here its purpose is to enable multiple developers to deploy lambda applications on AWS w/o needing ANY permissions at all on the AWS account itself. DuploCloud operates by creating isolated and secure workspace slices on a single AWS account.

In corporate environments, the infrastructure admin would install DuploCloud in the company’s AWS account and instantiate workspaces and give access to the workspace(s) to developers.

Another way to access this is by using DuploLive where a user signs up for a workspace simply with their gmail\google address. They will be provided an isolated and secure workspace slice on a common AWS account owned by DuploCloud. You will need an account in DuploCloud for this tutorial that you can get for free @ https://www.duplocloud.com.

Tutorial: Hello World Django application on Lambda

We will convert a HelloWorld Django application that runs on a server to deploy and function on AWS Lambda. This tutorial was built on a macOS Sierra. A video of the entire tutorial is available @

1. Signup for a DuploLive account

Visit https://portal.duplocloud.net and click on signup. Provide your email (a google account). You will receive an email with an activation link.

2. Install Zappa and DuploCli

We need to work in a virtual env so that we can package all the dependencies in the lambda package

> virtualenv duplo 

> source duplo/bin/activate

> pip install zappa

> pip install git+git://github.com/duplodemo/duplocli.git

3. Clone a sample Django Hello World Repo

> git clone https://github.com/duplodemo/demoservice.git

4. Create zappa settings file

This file is needed to tell zappa about your application so that we can convert the Django application into a lambda package

> cd demoservice

> cd mysite

> vi zappa_settings.json

Paste the following contents and save the file

{
    "dev": {
        "django_settings": "mysite.settings",
        "binary_support": false
    }
}

You can reference https://github.com/Miserlou/Zappa for details and other python libraries like flask.

5. Create duplo_compose.json

In the mysite directory where we created the zappa_settings.json create a file called duplo_compose.json with the following contents. The handler created by zappa is always “handler.lambda_handler” so that should not be changed, for the rest you can create custom values

{
 "LambdaFunctions": [{
      "Description":"Hello World",
      "Environment":{
              "Variables":{
                    "foo":"bar"
                }
      },
      "FunctionName":"helloworld",
      "Handler":"handler.lambda_handler",
      "MemorySize":128,
      "Runtime":"python2.7",
      "Timeout":7
 }]
}

This file provides the infrastructure needs of the application.

6. Build the lambda package

In the mysite directory install the requirements in the virtual environment

> pip install –r requirements.txt

Invoke zappa to create the package.

> zappa package dev

This will create a zip file for example mysite-dev-1506182688.zip

7. Deploy the lambda package in AWS using DuploCloud

If you have signed up for a DuploCloud account account in step 1, login to the portal. In the Dashboard click on the API icon and it will display the cli command like the following example screen shot

Copy the CLI connection command and run it in the virtual environment we have been working in i.e. the mysite directory.

> duplocli connection connect ….

Now run the following command to deploy the lambda function. Note the command has two aa in the word lambdaa This is because we wrote the cli using python and lambda was a language keyword. Let us know @ [e-mail] if you a workaround.

> duplocli lambdaa create-function -n helloworld -c duplo_compose.json -p <zipfilepath>

Here helloworld is the name of the function in the duplo_compose.json. DuploCli will look for the specifications of the function helloworld and use it to create a deployment in your workspace in AWS. is the package zappa created in step 6. The output of the above command will be like

8. Test the lambda application in AWS

From a web browser, login to duplocloud again and go to AWS tab. You will see your S3 bucket and Lambda function. Following is a screen shot of my workspace

Let’s get into AWS Lambda console from here by clicking on the second AWS Console button below the table of Lambda function. It might seem magical to have access to a slice of AWS console w/o having to owning or managing one.

Following is a screen shot of my lambda function.

Click on Test and run the test. It should succeed. A screen shot of ours is

9. Expose the service via APIGateway

Create an AWS api gateway to expose the lambda function to the internet. In this case we will create a simple proxy API. This will route all traffic received to the lambda function. This is done in two steps: A) Creating an API B) Deploying the API in a stage.

> duplocli apigateway create-api -n helloworld -l duploservices-ca3o68i-helloworld

The arguments above were a desired name (-n) and the name of the lambda function (-l). The output will be like

Config in the Duplo Portal that the API has been created. It takes at-times upto a minute. The AWS tab in Duplo portal will look like the following

10. Test the APIGateway Lambda integration

In the Duplo Portal, select the API Gateway line in the table and click on AWS Console button below and it will take you to API Gateway screen that will look like

Select the second ANY resource and click on test as shown in the screen shot below

This will take you to the test screen below where choose “GET” in the Method Dropdown and type getInfo in the path text box. Click Test at the bottom. The result should be a success with the text “CICD Demo!” as shown in the picture below.

Now we have tested the application E2E. Let us “deploy” the API. In the second pane, Actions dropdown choose “Deploy API”. You will need to create a new stage, call it “dev” and hit Deploy. This will make the API public and give you a URL as shown in the picture below.

Now you can take the URL, append /getInfo and open it in a browser and you should see “CI/CD Demo!” as shown below.

Summary

We convert a HelloWorld Django application without any code change into a lambda function using Zappa. We further deployed it on AWS via DuploCloud without requiring ANY permissions from the developer.

In this tutorial we used DuploLive (duplocloud.com) which is a public hosted service that acts an abstraction on top of an AWS account owned by DuploCloud. Each developer has his own secure and isolated workspace. In a corporate environment DuploCloud enterprise would need to be installed by the infrastructure admin (devops) on company AWS account. Developers (application teams) would have their workspaces in that environment.

If you are interested to know more about DuploCloud, reach us @ [e-mail]

Coming Next

In the subsequent two tutorials we will

1. Integrate Lambda application with other services like Docker web apps, S3, Dynamo and SQL. We will describe how in a Duplo Workspace a user can compose different forms of services w/o having to work about any devops constructs.
2. Lambda deployments and QA in a CI/CD environment integrated with github.com

The post Deploy Existing Web Applications as Serverless using AWS Lambda, Zappa and DuploCloud appeared first on DuploCloud.