Deploying AWS Lambda functions often involves packaging various dependencies. Managing these dependencies directly within Lambda can be cumbersome and error-prone. Using Docker and Amazon Elastic Container Registry (ECR) can streamline this process, ensuring consistency and reliability in your deployments. In this blog, we will walk through how to install pip packages in Lambda packages using Docker and ECR.
AWS Lambda is a powerful service that allows you to run code without provisioning or managing servers. However, managing dependencies for your Lambda functions can be challenging, especially when dealing with complex Python packages. By leveraging Docker, you can package your Lambda function and its dependencies in a container, which ensures a consistent runtime environment. Additionally, using ECR to store your Docker images simplifies the deployment process.
Before reading the remainder, be sure to subscribe and support the channel if you have not!
Subscribe:
Youtube
Support:
https://www.buymeacoffee.com/mmshilleh
Hire me at UpWork to build your IoT projects:
https://www.upwork.com/freelancers/~017060e77e9d8a1157
Prerequisites
Before we get started, make sure you have the following installed and configured:
- AWS CLI
- Docker
Step 1: Install AWS CLI and Configure
First, install the AWS CLI and configure it with your credentials.
Installation
On Windows: Download the AWS CLI MSI installer from the official website. Run the installer and follow the on-screen instructions.
On macOS: Use Homebrew to install the AWS CLI:
brew install awscli
On Linux: Use the following commands:
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install
Configure the AWS CLI by running:
aws configure
Enter your AWS Access Key ID, Secret Access Key, Default region name, and Default output format when prompted.
Step 2: Create an ECR Repository
If you haven't already created an ECR repository, do so by running:
aws ecr create-repository --repository-name my-lambda-repo --region us-east-1
This command creates a new ECR repository where you'll store your Docker images.
Step 3: Authenticate Docker to ECR
Authenticate your Docker CLI to your ECR registry, make sure you are signed into Docker desktop and have it running on your computer:
aws ecr get-login-password --region us-east-1 | docker login --username AWS --password-stdin <aws_account_id>.dkr.ecr.us-east-1.amazonaws.com
Replace with your actual AWS account ID.
Step 4: Create a .dockerignore File
Create a .dockerignore file in your project directory to exclude unnecessary files from the Docker build context:
__pycache__
.git
tests
*.md
*.txt
Step 5: Create a Dockerfile
Create a Dockerfile in your project directory. This file defines the environment in which your Lambda function will run and specifies the necessary dependencies.
# Builder stage - you can use other AWS runtimes and python versions
FROM public.ecr.aws/lambda/python:3.12-x86_64 as builder
# Install Python packages (replace as needed)
RUN pip install --no-cache-dir requests numpy pandas --target "/var/task"
# Final stage
FROM public.ecr.aws/lambda/python:3.12-x86_64
# Copy necessary files from the builder stage - add whatever code you need
COPY --from=builder /var/task /var/task
COPY handler.py ./
# Set the CMD to your handler
CMD ["handler.lambda_handler"]
Step 6: Build the Docker Image
Build the Docker image using the following command:
docker build -t my-lambda-image .
This command creates a Docker image named my-lambda-image.
Step 7: Tag the Docker Image
Tag the Docker image for pushing to your ECR repository:
docker tag my-lambda-image <aws_account_id>.dkr.ecr.us-east-1.amazonaws.com/my-lambda-repo:latest
Step 8: Push the Docker Image to ECR
Push the Docker image to your ECR repository:
docker push <aws_account_id>.dkr.ecr.us-east-1.amazonaws.com/my-lambda-repo:latest
Step 9: Update Lambda Function
Finally, update your Lambda function to use the new Docker image. You can do this through the AWS Management Console or using the AWS CLI:
aws lambda update-function-code --function-name my-lambda-function --image-uri <aws_account_id>.dkr.ecr.us-east-1.amazonaws.com/my-lambda-repo:latest
Replace my-lambda-function with the name of your Lambda function.
Alternatively, you can update your Lambda function through the AWS Management Console:
- 1. Navigate to the AWS Lambda console.
- 2. Select your Lambda function from the list.
- 3. In the "Code" tab, choose "Image" as the code source.
- 4. Select the ECR image you pushed earlier.
- 5. Save the changes.
Creating a Lambda Function from Scratch with a Docker Image
You can also create a new AWS Lambda function from scratch using a Docker image. This is useful when you are starting a new project or want to create a fresh Lambda function with a specific Docker environment. Here's how to do it through the AWS Management Console:
Navigate to the AWS Lambda console.
- Click on "Create function."
- Select "Container image" as the code source.
- Specify your container image details by selecting the ECR repository and image tag you pushed earlier.
- Configure the remaining settings (function name, role, etc.).
- Click on "Create function." This process sets up a new Lambda function that uses your Docker image as its execution environment, ensuring all dependencies and configurations are included from the start.
Conclusion
Using Docker and ECR to manage dependencies for your AWS Lambda functions significantly streamlines the deployment process. This method ensures that your functions have a consistent runtime environment, reducing the potential for errors caused by missing or incompatible dependencies.
By following the steps outlined in this tutorial, you can efficiently package, test, and deploy your Lambda functions with all necessary dependencies included. This approach not only simplifies the deployment process but also enhances the reliability and performance of your serverless applications.
Remember to subscribe to the channel and support our content creation efforts:
Subscribe on YouTube
Support on Buy Me a Coffee
Hire me for your IoT projects on UpWork
Top comments (0)