DEV Community

loading...
Cover image for How to Create a AWS Lambda Layer of Your Gemfile / Ruby Gem Dependencies

How to Create a AWS Lambda Layer of Your Gemfile / Ruby Gem Dependencies

peterc profile image Peter Cooper Updated on ・3 min read

If you use rooling like AWS SAM or Serverless Framework to build and deploy your serverless Ruby functions, they'll take care of building your dependencies on a function by function basis and package them up for deployment. Great! Use them!

But if you prefer to have your own hands on the controls at all times, using gems with native dependencies (e.g. Nokogiri) can be a nightmare! If you prefer to travel light (or, maybe, like me you like coding adhoc functions in the AWS console), you can instead build your dependencies into a Lambda Layer and then use that layer across whatever Lambda functions you like without worrying about dependencies again.

But how? After an hour of trial and error plus a few tips from this neat article (though that one still used SAM), I figured out the magic spell required to locally build Ruby gems and their dependencies in a local Amazon Linux container and turn them into a layer.

Note: This post assumes you're familiar with the aws client and have the credentials to use Lambda with it. If not, this is too advanced for now. Google the aws CLI client and how to create an IAM user with the right permissions.

The code

Without further ado, here's the magic:

LAYER_NAME="my-ruby-layer"
mkdir $LAYER_NAME && cd $_

bundle init
bundle add http --skip-install
bundle add nokogiri --skip-install
rm Gemfile.lock

docker run --rm -v $PWD:/var/layer \
           -w /var/layer \
           amazon/aws-sam-cli-build-image-ruby2.7 \
           bundle install --path=ruby

mv ruby/ruby ruby/gems
zip -r layer.zip ruby

aws lambda publish-layer-version \
           --layer-name $LAYER_NAME \
           --region eu-west-1 \
           --compatible-runtimes ruby2.7 \
           --zip-file fileb://layer.zip
Enter fullscreen mode Exit fullscreen mode

This results in a LayerVersionArn you can use with your Lambda functions. Once you've done this, loading the gems you need in the usual way (e.g. require 'nokogiri') will Just Work™.

If you use the AWS console, it'll let you pick this from a drop down menu which is how I like to do it:

Picking a Lambda layer in the AWS console

How the code works

Here's a quick break down of what the code above does. First, we create a folder with the name of our layer:

LAYER_NAME="my-ruby-layer"
mkdir $LAYER_NAME && cd $_
Enter fullscreen mode Exit fullscreen mode

Next, we create a Gemfile and add some gems to it. If you already have a Gemfile, copy it in at this point instead of doing this:

bundle init
bundle add http --skip-install
bundle add nokogiri --skip-install
rm Gemfile.lock
Enter fullscreen mode Exit fullscreen mode

Note: I delete Gemfile.lock because the version of Bundler in the Lambda container clashes with mine.

Let's now instruct Docker to run the image Amazon provides for building Ruby 2.7 dependencies and do the bundle install there (while saving it to our normal filesystem):

docker run --rm -v $PWD:/var/layer \
           -w /var/layer \
           amazon/aws-sam-cli-build-image-ruby2.7 \
           bundle install --path=ruby
Enter fullscreen mode Exit fullscreen mode

A quick directory name tweak is necessary to match the structure that Lambda expects. We can then zip up the ruby folder:

mv ruby/ruby ruby/gems
zip -r layer.zip ruby
Enter fullscreen mode Exit fullscreen mode

Finally, publish the layer to AWS Lambda:

aws lambda publish-layer-version \
           --layer-name $LAYER_NAME \
           --region eu-west-1 \
           --compatible-runtimes ruby2.7 \
           --zip-file fileb://layer.zip
Enter fullscreen mode Exit fullscreen mode

Make sure to set the region to the one you actually want to use the layer from. Every region has its own layers.

Note: If you want to deploy your layer to the public or use it more broadly across all regions, you'll want to search up the AWS Serverless Application Repository.

Discussion (3)

Collapse
pamit profile image
Payam Mousavi

Thanks for sharing.
I think if we use serverless, it will also take care of the gems layer.

Collapse
peterc profile image
Peter Cooper Author

If you mean the Serverless Framework, it looks like that's the case. I dislike using frameworks and prefer to understand all of the moving parts myself (at least until I understand it perfectly), although there are downsides as this article demonstrates ;-)

Collapse
cmar profile image
Chris Mar

Like you, I prefer to build manually first to understand the moving parts. Thanks for sharing.

Forem Open with the Forem app