Using LocalStack to Test AWS Cloud Applications Locally

You can read the original post here.

LocalStack is a free and open-source tool that allows you to run AWS services locally on your machine, enabling you to develop and test your cloud applications without having to connect to the real AWS cloud. This is accomplished by emulating AWS APIs and providing you a local interface to interact with these APIs.

The LocalStack API is a RESTful API that mimics the AWS API. This means that you can use the same tools and libraries that you use to test your applications against the real AWS cloud to test your applications against LocalStack.

LocalStack allows you to emulate different services like Lambda, SQS, SNS or EC2, and also provides different levels of access to these APIs. You can check their feature coverage page here, to understand the level of parity to the real AWS API and the pricing tier under which these APIs are available in LocalStack.

LocalStack is a great way to:

• Speed up your development process
• Reduce costs
• Test you application and Improve reliability

Why and When Should You Use LocalStack?

There are many reasons why you might want to use LocalStack. Here are a few of the most common:

• To speed up your development process. When you're developing a cloud application, you often need to make changes to your code and then deploy those changes to the AWS cloud. This can be a time-consuming process, especially if you're making a lot of changes. LocalStack allows you to test your changes locally, which can save you a lot of time.
• To reduce costs. If you're developing a cloud application, you'll need to pay for AWS resources like EC2 instances and S3 buckets. If you can test your application locally, you can reduce your AWS costs significantly.
• To improve reliability. When you deploy your application in the real AWS cloud, you have the peace of mind of having performed integration tests in addition to unit tests, and other software tests. LocalStack allows you to test your application in a more isolated environment, using the same AWS APIs you would use once deployed to AWS, which can help you to improve the reliability of your application by finding errors while developing it, rather than after deploying it. It's always cheaper to fix bugs or change your architecture in development rather than in production!

How to Install LocalStack

There are three ways to install LocalStack:
Manually using Homebrew. If you're on a Mac, you can install LocalStack using Homebrew. To do this, run the following command in your terminal:

brew install localstack

Using the installer. You can also download the LocalStack installer from the LocalStack website: https://docs.localstack.cloud/getting-started/installation/. Once you've downloaded the installer, run it to install LocalStack.
LocalStack runs inside a Docker container, and there are different ways to start and manage the LocalStack Docker containerUsing Docker images. For example, run the following command in your terminal:

docker run -it -p 4566:4566 localstack/localstack

How to Use LocalStack

Once you've installed LocalStack, you can start using it to test your cloud applications. To do this, first you need start LocalStack with:

localstack start

Examples of Testing an AWS Service Locally

Here are a few examples of how you can use LocalStack to test an AWS service locally:

• Testing an S3 bucket. You can test an S3 bucket by creating a file in your local directory and then uploading it to the S3 bucket in LocalStack. You can then download the file from the S3 bucket and verify that it's the same file that you uploaded.
• Testing an SQS queue. You can test an SQS queue by sending a message to the queue and then receiving the message from the queue. You can then verify that the message that you received is the same message that you sent.
• Testing a Lambda function. You can test a Lambda function by invoking the function and then verifying that the function returns the expected result.

I will provide a small snippet of how to use LocalStack and Python, using the AWS SDK (boto3) to create an Amazon Simple Notification Service (SNS) topic and subscription. The idea can easily be expanded to test more complex applications.

"""
Using AWS Boto3 create the following:
An SNS topic with to notify the user of a change in the AWS EKS Parameter
An SNS subscription for the topic with the email address "XXXXXXXXXXXXXXXXXXXXXXXXXX"
"""


import boto3
from botocore.config import Config
from botocore.exceptions import ClientError
from datetime import datetime
import logging
import sys


logging.basicConfig(
    stream=sys.stdout,
    level=logging.INFO,
    format="[%(levelname)s] %(asctime)s: %(message)s",
)
logger = logging.getLogger()


RUN_LOCAL = True
SNS_EMAIL_ENDPOINT = "[e-mail]"  # Add email address to send notifications to.

################ Setup Localstack Endpoint ################

endpoint_url = "http://localhost.localstack.cloud:4566"

################ Setup Localstack Endpoint ################


def create_boto3_client_session(service_name: str, run_local=RUN_LOCAL):
    """Cheks if we are running locally using localstack or in AWS.
        If we are running locally we set the endpoint to point to localstack endpoint.

    Args:
            service_name (str): Servie name to create the client for (e.g. ec2, s3, sns, etc.)
            run_local (bool, optional): Run locally? Defaults to False.

    Returns:
            boto3.client: A boto3 client for the service requested.

    """

    if run_local:
        logger.info(f"Creating boto3 client for {service_name} in localstack")
        return boto3.client(service_name, endpoint_url=endpoint_url)
    else:
        logger.info(f"Creating boto3 client for {service_name} in AWS")
        return boto3.client(service_name)


# SNS TOPIC


def create_sns_topic(name: str) -> dict:
    """Create an SNS topic for eventBridge to publish to.

    Returns:
        dict: SNS topic ARN.
    """
    sns_client = create_boto3_client_session(service_name="sns")

    try:
        topic = sns_client.create_topic(Name=name)
        logger.info(f"Created SNS topic {topic}")
        topic_arn = topic["TopicArn"]
    except ClientError:
        logger.exception("Failed to create SNS topic {topic}")
        raise
    else:
        return topic


def create_sns_subscription(topic_arn: str, endpoint: str) -> dict:
    """Create an SNS subscription for the topic.

    Args:
        topic_arn (str): SNS topic ARN.
        endpoint (str): SNS subscription endpoint.

    Returns:
        dict: SNS subscription dictionary.
    """
    sns_client = create_boto3_client_session(service_name="sns")

    try:
        subscription = sns_client.subscribe(
            TopicArn=topic_arn, Protocol="email", Endpoint=endpoint
        )
        logger.info(f"Created SNS subscription {subscription}")
    except ClientError:
        logger.exception("Failed to create SNS subscription")
        raise
    else:
        return subscription


if __name__ == "__main__":
    sns_topic = create_sns_topic(
        name=f"MyTestSNSTopic"
    )
    sns_subscription = create_sns_subscription(
        topic_arn=sns_topic["TopicArn"], endpoint=SNS_EMAIL_ENDPOINT
    )

After importing the required modules and setting up logging, I decided to have a flag to signal my script whether I want to run the script locally using LocalStack or against the real AWS APIs.

RUN_LOCAL = True

After I'm done testing, I can flip the flag to False and deploy the the application to AWS.

Next step is to store the LocalStack endpoint in a variable to tell our application how to connect to the emulated AWS APIs:

endpoint_url = "http://localhost.localstack.cloud:4566"

When we create instances of AWS services using the SDK, first thing we need to do is to create a client session. If you are composing an application that creates multiple services, you have to create a client session for each service. This creates repetition of code as we instantiate multiple clients. Following the DRY principle, I've created the function create_boto3_client_session that takes two arguments, the service that we want to create a session for and whether to run locally or not, returning a client session for service passed onto it.

I then create two functions to create the SNS topic create_sns_topic and create subscription to the topic create_sns_subscription based on a passed email address as a subscription endpoint I've stored as global variable SNS_EMAIL_ENDPOINT. Finally, I call these two functions to create the SNS topic and subscription.

You could expand the script I've provided here to include other services that trigger messages to the SNS topic, such as AWS Lambda functions or Amazon EventBridge. The code provided here should provide a good starting point to show you how you can develop applications to be deployed to AWS, testing them locally with the same tools and services you would use against real AWS APIs.

Conclusion

LocalStack is a powerful tool that can help you to speed up your development process, reduce costs, improve reliability, and test your applications in an isolated local environment. If you're developing a cloud application, I highly recommend checking out LocalStack.

For more information, please visit the LocalStack website: https://localstack.cloud/.

Cover photo by Glenn Carstens-Peters on Unsplash