Originally posted at Serverless
AWS re:Invent is in full swing, with AWS announcing a slew of new features. Most notably, we’re pretty excited about AWS Lambda’s support for Layers.
Layers allows you to include additional files or data for your functions. This could be binaries such as FFmpeg or ImageMagick, or it could be difficult-to-package dependencies, such as NumPy for Python. These layers are added to your function’s zip file when published. In a way, they are comparable to EC2 AMIs, but for functions.
The killer feature of Lambda’s Layers is that they can be shared between Lambda functions, accounts, and even publicly!
There are two aspects to using Lambda Layers:
Publishing a layer that can be used by other functions
Using a layer in your function when you publish a new function version.
We’re excited to say that the Serverless Framework has day 1 support for both publishing and using Lambda Layers with your functions with Version 1.34.0!
See how you can publish and use Lambda Layers with the Serverless Framework below.
Example use case: Creating GIFs with FFmpeg
For a walkthrough, let’s make a service that takes an uploaded video and converts it to a GIF.
We’ll use FFmpeg, a open source tool for manipulating video and audio. FFmpeg is a binary program and a great example use case for a layer as managing the binary falls outside the responsibility of your runtime’s packaging system.
In this example, we’ll build and publish a layer that contains FFmpeg. Then, we’ll create a Lambda function that uses the FFmpeg layer to convert videos to GIFs.
To get started, create a serverless project for your layer & service:
Then at the bottom of your serverless.yml add the following to define your layer that will contain FFmpeg. The path property is a path to a directory that will be zipped up and published as your layer:
Run the following commands to download the contents of your layer:
You’re ready to test deployment of your layer. Deploy and you’ll see the layer’s ARN in the output info:
Next, we’ll add a custom section to serverless.yml to specify the S3 bucket name (choose your own unique bucket name):
Now rename your function from hello to mkgif, specify that your function uses the layer you’re publishing, and add an S3 event configuration:
You’ll also need to give your service permission to read & write your S3 bucket, add the following in the provider section of your serverless.yml file:
Your serverless.yml should now look like this.
We need to make our handler. Replace the contents of handler.js with the following code, which gets the file from S3, downloads it to disk, runs ffmpeg on it, reads the GIF, and finally puts it back to S3:
Now you can deploy both the layer & updated function with sls deploy. Let’s test it out by uploading a video to our S3 bucket:
You now have a GIF copy of the mp4 you uploaded!
For the full source of this example, check it out in our examples repo.
Some tips on working with layers
In the example above, instead of specifying an ARN for the layer that the function is using, we used {Ref: FfmpegLambdaLayer}. This is a CloudFormation Reference.
The name is derived from your layer’s name, e.g., ffmpeg becomes FfmpegLambdaLayer. If you're not sure what your layer's name will be, you can find it by running sls package then searching for LambdaLayer in .serverless/cloudformation-template-update-stack.json.
You may have noticed that every time you deploy your stack, a new layer version is created. This is due to limitations with CloudFormation. The best way to deal with this is by keeping your layer and your function in separate stacks.
Let’s try that with the example we just made.
First, create a new folder and move the layers directory into it:
Remove the top-level layers section in gifmaker/serverless.yml, then create a new serverless.yml in the ffmpeg-layer folder containing:
Now you can run sls deploy to publish your layer!
Go back to the gifmaker service directory and change {Ref: FfmpegLambdaLayer} in the serverless.yml to ${cf:ffmpeg-layer-dev.FfmpegLayerExport}. You can now run sls deploy and it'll use the layer from the other service. Note that the dev in the variable above is the stage of your layer service.
More Examples
You can see the following projects for some examples of using this plugin to build a layer. They all leverage Docker and the docker-lambda images to compile for AWS’s Lambda environment on any operating system:
geoip-lambda-layer — A layer containing MaxMind’s GeoIP libraries
sqlite-lambda-layer — A layer to fix SQLite support in Python 3.6 runtimes
Awesome layers
Also check out this repository of awesome layers: https://github.com/mthenw/awesome-layers
Custom runtime support: even better!
Along with layers support, AWS also just announced support for building your own runtime using the Runtime API.
This allows you to build, use, and share runtime support for Lambda outside of what AWS officially supports.
Custom runtimes with the Serverless Framework
To utilize custom runtimes with Serverless, specify the runtime as provided in your serverless.yml and include a layer that provides a custom runtime. For documentation on building your own runtime, see AWS’s documentation here
More re:Invent news
Top comments (0)