Running a git binary in AWS Lambda

In some scenarios it can be easier to use a binary in a lambda function. In this blog post I will walk you through how you can run a git binary in an AWS Lambda.

For this blog post I will use the AWS Serverless Application Model(AWS SAM). AWS SAM makes it easy to build, package and deploy lambda functions. The used template can be found on GitHub. AWS SAM will look at the Runtime of the AWS::Serverless::Function resource. In this example I used python so it will look at the requirements.txt. It will look at the packages defined in the file. AWS SAM installs them in .aws-sam/build/<ResourceName>. This folder is compressed and uploaded to Amazon S3 when you package and deploy the template.

You can also invoke the function on your own computer. This makes it easier to develop and troubleshoot. To invoke the function we first need to build it. Afterwards you need to invoke it. I created a Makefile, for more information on this you can read one of my previous blogs.

make invoke

You will see that you will see the following error:

{
    "errorMessage": "[Errno 2] No such file or directory: 'git'",
    "errorType": "FileNotFoundError"
}

And that makes sense right? We do not have the git client installed. You cannot install it using the requirements.txt file. This is because it is not in the PyPi package manager.

Install the git client

If you look at the Makefile you can see that it will:

• Build a docker container.
  • Install git on the container.
  • Move all the dependencies into the /var/task structure.
  • Zip the content of the /var/task folder.
• Extract the zip file from the container.
• Unzip the contents into the git_client folder.
• sam build will copy the content to .aws-sam/build/GitClient.
• sam build installs the dependencies from the requirements.txt.
• sam invoke will use the .aws-sam/build/GitClient content to invoke the function.

The Git binary is located in /var/task/bin. Because when we invoke the git command we also supply the /var/task/bin as PATH. This allows us to call git and the container knows it needs to invoke /var/task/bin/git. The libraries are located in /var/task/lib. Because we have set the LD_LIBRARY_PATH to /var/task/lib the libraries can be found.

I created a Dockerfile that uses public.ecr.aws/lambda/python:3.8 as the base. This is the same runtime of the Lambda function. I prepare 2 folders /var/task/bin and /var/task/lib. Then I install the git and zip packages within the docker context.

I then copy the /usr/bin/git and the /usr/libexec/git-core/git-remote-http executables to the /var/task/bin folder.
When you try to build the Lambda function and run it it fails on a missing library.

{
    "errorMessage": "git: error while loading shared libraries: libpcre2-8.so.0: cannot open shared object file: No such file or directory",
    "errorType": "Exception"
}

By using the ldd command you can find all the shared libraries. We will use this to identify the needed libraries and copy them into the /var/task/lib/ folder.
Once you have all dependencies in place. You can fetch the git --version running inside of the Lambda function.

Git version: git version 2.32.0

If you combine this with the awscli and git-remote-codecommit packages. You can clone a CodeCommit repository and perform your actions and commit it again.

Conclusion

It is possible to embed binaries in AWS Lambda and use them. It might take you some effort. But it opens new possibilities that you might not considered before.
In my next blog you will read how you can use this to synchronize two CodeCommit repositories.

Photo by Emile Perron on Unsplash