Deploy Lambda Applications with the Serverless Framework

This topic is the subject of my presentation at the AWS User Group Mega Manila Meetup last May 21, 2020.

Functions-as-a-service (FaaS) has ushered in an era of serverless applications. Developers, thrilled with the promise of no servers to manage, suddenly find that while FaaS removes many of the pain points of managing servers, it introduces its problems as well.

1 | Problems of FaaS

One of those problems is how to deploy. With a single-function "hello world" example, FaaS looks very simple. But as we build fully-fledged serverless applications, the complexity starts to grow: we add more files, our directories start to grow, and we add more and more dependencies. Manually deploying all of this would be even be slower than deploying traditional monolith applications.

The solutions available for deploying FaaS applications can vary per cloud provider. For this post, we focus on AWS Lambda applications.

CloudFormation

One way to make deployments easier is through CloudFormation (CF). With CF, we write templates that describe each component of our infrastructure. Since we feed the template directly to AWS, we have to very verbose in writing these templates. A simple setup often takes 100+ lines to YAML/JSON code.

Lambda Deployment Frameworks

To ease this burden, deployment frameworks build a layer of abstraction on top of CloudFormation. Instead of writing 100+ lines of code, we use the framework's special syntax to write the same code in a few lines. While this means the framework makes assumptions on specific components of our Lambda function, we can always add more lines to be more explicit.

Serverless Framework

In this post, we look at the Serverless Framework. The framework itself is cloud-agnostic and supports many cloud providers (e.g. AWS, GCP, Azure, etc.). In this post, we focus on its AWS functionality.

2 | Setup

To get started, follow the steps below:

• Let's install the Serverless Framework from here. If you're installing from Mac, don't forget to place this export command export PATH="$HOME/.serverless/bin:$PATH" to your .bashrc / .zshrc.
• Then, we have to make sure that our local machine has the AWS CLI installed and configured to have access to an admin-level user.

3 | Create our Lambda application with Serverless

The hands-on example we are about to do basically provisions a Lambda function and a DynamoDB table. It also creates an IAM role so that the Lambda function can access DynamoDB. We also install Severless Framework plugins to allow us to upload Python dependencies (Quart and Mangum, to be discussed later).

(3.1) To create our first Lambda application using the Serverless Framework, let's run serverless create --template aws-python3 --path helloWorld.

This command should create two files:

• handler.py - a sample python file for our Lambda app
• serverless.yml - in this file, we define our infrastructure. The pre-generated snippet defines a simple Lambda function. It also has many comments showing you how to utilize the framework for more complex use cases.

If you already have a Lambda function, you can just add a serverless.yml file on the root directory. The Serverless framework integrates right in.

(3.2) To show the Serverless Framework's power, we construct on a more complicated example. We create an application that displays blog posts. We start by replacing the contents of the serverless.yml file with the snippet below.

Mainly, the new template is divided into five sections:

• service - the name of the service
• provider - we define the cloud provider and the default runtime for our Lambda functions. We also defined a default IAM role that will be created and used for all the Lambda functions (if none was explicitly defined). The role we defined allows our Lambda functions to have full access to all DynamoDB tables in our account.
• functions - we define the lambda functions for our application. We define one Lambda function named "hello". In this function, the handler defines the entry point for our application. We also define the "/" route to be accessible via API Gateway. This means we get to call our Lambda function from our browser with this route. If this doesn't make sense now, follow along and I'll demonstrate it.
• package - we can exclude/include certain files and directories from the package that will be uploaded to Lambda.
• resources - this is the CloudFormation part of our template. Here, we define our AWS resources that we may need as part of our infrastructure. In this case, we defined a simple DynamoDB table to store our blog post data. The table has a primary key of "post_number" and an attribute of "message".

service: helloworld

provider:
  name: aws
  runtime: python3.7
  region: us-west-2
  iamRoleStatements:
    - Effect: "Allow"
      Action: "dynamodb:*"
      Resource: "*"

functions:
  hello:
    handler: handler.hello
    events:
      - http:
          path: /
          method: get

package:
  exclude:
    - venv/**

resources:
  Resources:
    orderTwo:
      Type: AWS::DynamoDB::Table
      Properties:
        TableName: post
        AttributeDefinitions:
          - AttributeName: "post_number"
            AttributeType: "N"
        KeySchema:
          - AttributeName: post_number
            KeyType: HASH
        ProvisionedThroughput:
          ReadCapacityUnits: 1
          WriteCapacityUnits: 1

(3.3) To test our application, let's deploy this right now with the serverless deploy command. You should see the public URL of your application.

Then, visit your browser using the link. You should see JSON displayed in your screen

4 | Adding Python Dependencies

Lambda applications often come with dependencies. In this section, we will work with Quart, a web framework alternative to Flask. We will also install Mangum so we can port our Quart web application to Lambda.

(4.1)

In order to install Python dependencies with the Serverless Framework, we have to install a Serverless Framework plugin called serverless-python-requirements. We will use this command serverless plugin install -n serverless-python-requirements. This plugin detects the dependencies we write on the requirements.txt file, and packages them up with our Lambda function.

The Serverless Framework ecosystem is rich. It has a lot more plugins that can help solve some problems that you may be having.

(4.2)

In the root directory of your project, execute the following commands. Here, we start a Python virtual env and install the web framework Quartz. Quartz is a modern alternative to Flask, using ASGI instead of WSGI technology for its application server.

# create a python virtual env to keep our Python installation clean
python3 -m venv venv
source venv/bin/activate

# install mangum and quartz
pip install mangum
pip install quart
pip freeze > requirements.txt

(4.3) Now, let's replace the contents of our handler.py with the snippet below.

from mangum import Mangum
from quart import Quart

app = Quart(__name__)

@app.route("/")
async def hello():
    return "hello world!"

hello = Mangum(app)

(4.5) To see the changes live in Lambda, execute serverless deploy.

5 | Using DynamoDB

(5.1) Create a file named database_wrapper.py and add the snippet below. In this file, we create code that reads and writes from our DynamoDB table.

import boto3
from boto3.dynamodb.conditions import Key, Attr
from botocore.exceptions import ClientError
from datetime import datetime

def try_ex(func):
    try:
        return func()
    except KeyError:
        return None

def read_post(post_number):
    dynamodb = boto3.resource("dynamodb", region_name='us-west-2')
    table = dynamodb.Table('post')
    item = None

    try:
        response = table.get_item(Key={'post_number': post_number})
    except ClientError as e:
        print(e.response['Error']['Message'])
    else:
        item = try_ex(lambda: response['Item'])
        if item is None:
            return False


    return item

def write_post(post_number):
    dynamodb = boto3.resource("dynamodb", region_name='us-west-2')
    table = dynamodb.Table('post')
    response = table.put_item(
       Item={
            'post_number': post_number,
            'message': 'this information is read from a DynamoDB table',
        }
    )

(5.2) Modify the handler.py to reference the read_post and write_post functions from the database_wrapper.py file. Essentially, we added code that reads the post with the post_number 1 from the post table. If the post doesn't exist yet, we create a new record for it.

from mangum import Mangum
from quart import Quart
from database_wrapper import read_post, write_post

app = Quart(__name__)

@app.route("/")
async def hello():
    post = read_post(1)
    if not post:
        write_post(1)
        post = read_post(1)

    return post['message']

hello = Mangum(app)

(5.3) Then, we deploy once again using serverless deploy. We should be able to see a new message. This message is retrieved from the DynamoDB table.