Note: This article was authored by Michael Lavers
In Part 1 we covered the serverless basics and got our feet wet with the Serverless Framework. If you haven't read that part yet, it's highly encouraged that you start there. Assuming you've followed along in Part 1, you now know how to create and deploy a simple AWS Lambda function in Python; including a web API using AWS API Gateway. But we've only just scratched the surface of what you can do in serverless with Python. In this post, we'll introduce a library that allows you to plug any web framework that speaks WSGI into serverless (pretty much all of them); and we'll take a look at AWS' own Python web framework for creating web APIs.
Zappa
Zappa, named after the legendary Frank Zappa, is the original Python serverelss library/framework. It was originally conceived as django-zappa, a library to allow you to deploy Django web applications to AWS Lambda. It has since grown in leaps and bounds and now supports any WSGI compatible Python web framework. It has also expanded it's feature set and is very much a "batteries included" tool for creating, deploying and managing serverless web applications written in Python.
What is WSGI?
If you've done Python web development before, you've likely heard of WSGI. But what is it? WSGI stands for "Web Server Gateway Interface" and it's an implementation agnostic interface between web servers and web applications/frameworks. It was created by the Python community back in 2003 because there wasn't a standard way to develop a web application in Python back then. Instead you first had to pick a web server, which could use CGI, FastCGI, mod_python or something else entirely and develop against that. Which meant your web application was very much coupled to the web server in which it was built upon (remember Zope?). Now we have WSGI and all modern Python web frameworks support it, which means you can use whatever web server you like -- or no server at all? Yes, thanks to Zappa, WSGI is supported on serverless, too.
The reason I refer to Zappa as a "library/framework" is that it kind of lives in both worlds. Zappa at it's core is a compatibility/translation layer between serverless and WSGI. For example, with AWS Lambda, Zappa takes the HTTP event we covered in Part 1 and converts it into a WSGI compatible equivalent representation that all WSGI web frameworks can use. This is where Zappa feels like a "library", but then Zappa packs a ton of useful features on top and starts to feel more like a "framework". Although, like WSGI, Zappa is very much agnostic when it comes to the actual web framework you use.
Which web framework should I use?
If you're asking this question, then I would recommend you start with either Django or Flask. Django is the most popular web framework in Python and is very much a "batteries included" option. Flask is arguably the second most popular web framework in Python and calls itself a "micro framework"; it has a simpler API and doesn't come with all the bells and whistles that Django does. Both are excellent options and are very well documented. In this post, we'll focus on these two web frameworks when it comes to Zappa, but Zappa should work with pretty much any Python web framework. Suffice to say, if you're using Bottle, Pyramid, Tornado, Falcon or something else -- it should be easy to get your web application running on serverless using Zappa.
Django Meets Zappa
As we mentioned in Part 1, the rule of thumb for new Python serverless projects is to use Python 3.6, and true to form, Django has dropped Python 2.7 support in Django 2.0. This may seem like a sad day for all the Python 2.7 loyalists out there, but the pros greatly outweigh the cons here. Think about it: no more six. That alone should put a smile on your face -- at least until Python 4 is released.
Now before we dive into Zappa, we first need to create a Django web app. Django makes this really easy:
pip install -U django
django-admin startproject mydjangoapp
cd mydjangoapp
This will install the latest version of django using pip and the django-admin startproject command will create a directory called mydjangoapp with a preconfigured Django web app ready to go. Now if you've done Python web development before, you're likely asking "what about the virtualenv?" If you don't know what a "virtualenv" is, it's a best practice in Python to create a "virtual environment" for each project so that all your dependencies remain in one place and don't get mixed up with other projects. There are a number of tools out there to make managing virtual environments easy. for example, the one gaining a lot of traction recently is Kenneth Reitz' Pipenv. If you don't know who that is, he's the guy behind requests, you likely know his work. But I'm still partial to virtualenvwrapper I'm going to treat virtual environments as a personal preference. It's still a best practice, though, so pick whichever tool fits your workflow the best.
Now that we have a Django project created and we're in our mydjangoapp directory let's fire up Zappa:
pip install -U zappa
zappa init
Zappa is going to ask a series of questions about your web app. For now, you can stick with the defaults that Zappa suggests. As we mentioned in Part 1, like with the Serverless Framework, Zappa is going to check that your AWS credentials are properly configured. If you have multiple AWS profiles configured, Zappa wil ask which one to use, but if you only have one, then the default is what you want. After running zappa init and answering the questions, you'll find that Zappa has created a zappa_settings.json file in your project directory. This file serves the same purpose as the serverless.yml in Part 1, but because it is JSON there are no comments. Thankfully, Zappa documents the available settings.
Now to deploy:
zappa deploy dev
Zappa will perform steps similar to Serverless Framework covered in Part 1, but out-of-the-box it will do some handy Python-specific things like sort out dependencies for us. Once it is done deploying, you should see something like this:
Deployment complete!: https://xxxxxxxxxx.execute-api.us-east-1.amazonaws.com/dev
If you've read Part 1 this is going to look familiar. It's a URL provided by AWS API Gateway. Let's open it up in a browser and see what we get:
DisallowedHost at /
Invalid HTTP_HOST header: 'xxxxxxxxxx.execute-api.us-east-1.amazonaws.com'. You may need to add 'xxxxxxxxxx.execute-api.us-east-1.amazonaws.com' to ALLOWED_HOSTS.
Huh. Ok, let's do that. Open up mydjangoapp/settings.py and look for the ALLOWED_HOSTS setting and change it to the following:
ALLOWED_HOSTS = [".execute-api.us-east-1.amazonaws.com"]
Why aren't we including the xxxxxxxxxx part? Well, this part of the hostname is prone to change. For example, if you decide to deploy a staging or production environment with Zappa, then this part of the hostname is going to be different. Setting your ALLOWED_HOSTS this way allows you to support every environment you deploy with Zappa in AWS us-east-1. If you decide to deploy in another AWS region, then you'll need to update your ALLOWED_HOSTS accordingly. Let's save and re-deploy:
zappa update dev
Notice here the command is zappa update and not zappa deploy. This is because most of your serverless web app is already deployed. All we need to do now is replace our package in the S3 bucket Zappa created for us. You should notice that zappa update runs noticeably faster. Once done, let's reload our https://xxxxxxxxxx.execute-api.us-east-1.amazonaws.com/dev URL in a browser:
Page not found (404)
Ok, we're making progress. The reason we're getting a "Page Not Found" error is that Django doesn't configure a page for / by default. Let's fix that. First, we'll need to create a Django app in our project. A Django web application is comprised of "apps". Say your web application had a blog and a store, each of those would be an "app" in your Django project. To create an app:
python manage.py startapp myapp
This command will create a myapp directory in your project. Inside that directory are a handful of modules to support an app. But before we start working on our app we must first register it within our Django project. To do this, we need to open our mydjangoapp/settings.py again and look for the INSTALLED_APPS setting. There we want to add our app to the end of the list, so it should look something like this:
INSTALLED_APPS = [
...
'myapp',
]
Note that INSTALLED_APPS will have several Django apps already installed. We want to keep those and just add a new app to the list. So the .... represents all the Django apps that are already configured. Save this file and open up mydjangoapp/urls.py. This file is where Django configures all the routes of our web application. In fact, if you notice, there's already a route configured:
urlpatterns = [
path('admin/', admin.site.urls),
]
Interesting. What if we open https://xxxxxxxxxx.execute-api.us-east-1.amazonaws.com/dev/admin/ in a browser? How about that. The Django admin app is already there running. Unfortunately we haven't configured any users in our Django project, so we won't be able to get beyond this login page. Back to what we were originally doing. Let's add a route for / and serve up an HTML template. First we'll need to import Django's TemplateView. After the from django.urls import path line, add the following:
from django.views.generic import TemplateView
And then update our urlpatterns like so:
urlpatterns = [
path('admin/', admin.site.urls),
path('', TemplateView.as_view(template_name='index.html')),
]
Here we're configuring an empty route, which is equivalent to / which is to be served by TemplateView using a template_name of index.html. Now to create the template, we need to go back to our app. From our project directory, we create a templates directory in our app and add a template to it:
mkdir myapp/templates
echo "<h1>Hello Django</h1>" > myapp/templates/index.html
Note that the template's name is index.html like we specified in our urls.py. Django will automatically load any templates it finds in any of the INSTALLED_APPS, assuming a templates directory is present for that app. Now re-deploy:
zappa update dev
And reload https://xxxxxxxxxx.execute-api.us-east-1.amazonaws.com/dev in a web browser:
Hello Django
And just like that you have a serverless Django web app. Zappa truly couldn't make it any easier. Deploying the same app to a regular web server would actually be more work. Aren't you glad you ditched those things?
Now as cool as a stock Django app running on serverless is, if we refer back to Part 1, we're really looking to build a web API and workers, right? With Django, we're in luck, as it has one of the most powerful and well-documented REST frameworks out there called Django REST Framework. And if you follow the installation instructions you'll have a fully-featured REST API running on serverless in no time. Or if you're building a GraphQL API, then the excellent Graphene integrates with Django.
As for workers, Zappa offers a solution not unlike the one we described in Part 1. If you had a function called scheduled_job in myapp/tasks.py that you wanted to execute every ten minutes, you would open your zappa_settings.json and add the following:
{
"dev": {
...
"events": [{
"function": "myapp.tasks.scheduled_job",
"expression": "rate(10 minutes)"
}],
...
}
}```
{% endraw %}
And then to deploy it:
{% raw %}
```bash
zappa schedule dev
And you're good to go. And what about custom domains? Zappa has you covered there, too. Before we continue, assuming you don't need this app, Zappa makes it easy to clean up:
zappa unschedule dev
zappa undeploy dev
Want to learn more about Django? Check out the Django documentation. Or even better, check out these Django tutorials.
Flask Meets Zappa
Now if the Django setup wasn't straightforward enough, Flask might be even easier. Assuming we're starting from a new virtualenv (see Django section for suggested tools for creating a virtualenv). We first install Flask and Zappa:
pip install -U flask zappa
Then create a project directory:
mkdir myflaskapp
cd myflaskapp
And within that directory add a app.py with the following:
from flask import Flask
app = Flask(__name__)
@app.route('/')
def hello():
return {'message': 'hi there'}
if __name__ == '__main__':
app.run()
Then fire up Zappa:
zappa init
Like with our Django app, Zappa will auto-detect that our app is using Flask. And to deploy:
zappa deploy dev
And we should get:
Deployment complete!: https://xxxxxxxxxx.execute-api.us-east-1.amazonaws.com/dev
And to test it:
curl https://xxxxxxxxxx.execute-api.us-east-1.amazonaws.com/dev
Which should return:
{"message": "hi there"}
Talk about an instant web API. So what's happening here? Zappa probably feels like magic, but it's really just clever. In Django, the WSGI is exposed in mydjangoapp/wsgi.py. So Zappa creates a AWS Lambda handler for us that accepts HTTP requests. This handler transforms these requests into equivalent WSGI requests and passes them onto mydjangoapp/wsgi.py. With Flask, it's pretty much the same idea. The only difference is that Flask exposes it's WSGI via the app.run() method in our app.py above.
The Zappa topics we've covered already with Django are applicable to Flask as well. From scheduled jobs to custom domains, they're all supported.
Want to learn more about Flask? Check out the Flask documentation. Or even better, check out this Flask mega tutorial.
Oh, and don't forget to clean up:
zappa undeploy dev
More Zappa?
In this post, we've only scratched the surface of what you can do with Zappa. Check out the Zappa documentation for more, including how to rollback deployments, tail logs, keep your functions warm, fire asynchronous jobs, etc.
Chalice
If Zappa didn't inspire you to start building a serverless Python web app, I don't know what will, but it also appears to have inspired (at least in part) a team at AWS to start their own web framework specifically built for AWS Lambda. They call it Chalice.
This project is interesting for a couple reasons. First is that it is one of the few Python web frameworks out there that doesn't implement WSGI. While I've been waxing poetic on the virtues of WSGI, Chalice is built specifically for AWS LAmbda, and as such doesn't need an abstraction layer like WSGI. The trade off here is simplicity for lock in: the Chalice implementation is simplified by only having to target AWS LAmbda, but that also means you can only use Chalice with AWS Lambda. Second, Chalice's API is clearly influenced by another web framework we've already covered in this post, which helps with the learning curve. Which one you ask? Let's take a look.
Assuming we're starting with a fresh virtualenv, install Chalice with:
pip install -U chalice
And to create a project:
chalice new-project mychaliceapp
This command creates a directory called mychaliceapp with a pre-configured Chalice project. Inside there's a .chalice hidden directory that contains config files, a requirements.txt and an app.py. There's our first clue, if you recall, in our Flask app we created a app.py as well, which is a common Flask idiom. But if we take a look at the app.py it will become clear:
from chalice import Chalice
app = Chalice(app_name='mychaliceapp')
@app.route('/')
def index():
return {'hello': 'world'}
Look familiar? They say imitation is the sincerest form of flattery. And if I'm Armin Ronacher (creator of Flask), I'm feeling pretty flattered. And to deploy:
chalice deploy
When the deploy finishes, Chalice will return a URL that should be familiar by now: https://xxxxxxxxxx.execute-api.us-east-1.amazonaws.com/api
You guessed it, there's our old friend, API Gateway. And to test it:
curl https://xxxxxxxxxx.execute-api.us-east-1.amazonaws.com/api
Which should return:
{"hello": "world"}
We didn't even have to write any code for that one. Does that make it codeless as well?
Chalice offers a Flask-esque API and is very much designed for JSON REST APIs. It also comes bundled with some nice integrations with other AWS services, including authentication with AWS Cognito. It is clear that the project is focused on the problem they're trying to solve, as such, as long as your problem is JSON REST APIs then Chalice can help you get up and running on AWS Lambda quickly. Anything else? You're going to want to look at the other options we've covered so far.
In Summation
In this post, we've taken a look a two Python tools looking to make developing, deploying and managing serverless apps easier. One that is agnostic as to what web framework you use, the other is very much opinionated. One thing is clear, the Python community is getting a lot right when it comes to serverless.
In Part 3 we're going to compare and contrast all the options available in the serverless space for Python web apps. We'll also introduce you to a serverless observability tool that will give you visibility into your serverless web apps like never before. Stay tuned.
Top comments (0)