How to design and create an AWS Serverless API Builder with CDK Python

Serverless logic tier

One of the most known serverless pattern is AWS API gateway and AWS Lambda to create a microservice which might represent the logic tier of the three-tier architecture. The union of these two services build a serverless application that is secure, highly available and scalable, using the serverless approach you are not responsible for server management in any capacity.

Scale project-architecture

For larger scale project-architecture, you might think about migrating web applications associating one API Gateway with one Lambda function if they are using frameworks like Flask (for Python). These web frameworks support routing and separate user contexts that are well suited if the application is running on a web server.
You could think same approach with a new cloud native application and add more functionality in one single Lambda.
With this approach, an Amazon API Gateway proxies all requests to the same Lambda function to handle routing. As the application develops more routes, the Lambda function grows in size (physical file size of packages) and deployments of new versions replace the entire function. It becomes harder for multiple developers to work on the same project in this context.
Moreover, AWS states these deployment package sizes as the limit. It makes sense to break down different parts of the program which can be independent by domain as a separate serverless function.

Deployment package (.zip file archive) size
50 MB (zipped, for direct upload)
250 MB (unzipped)
This quota applies to all the files you upload, including layers and custom runtimes.
3 MB (console editor)

Apart from that, if you design that every AWS Lambda deploys with its own Amazon API Gateway endpoint, you will end up with multiple different URL endpoints and to keep track of the different endpoints and updating different clients will become a massive hassle at scale.
A better architecture is to take advantage of the native routing functionality available in API Gateway. You do not need a web framework in many cases, which increases the size of the package. AWS API Gateway validates parameters, reducing the need for checking parameters inside the lambda code. In addition, you can configure mechanisms for authentication and authorization and other features to lighten the code.
The new architecture looks like this:

When the approach is clear it is time to develop. The first thing that we need to create is the infrastructure.

Automate infrastructure creation

We use infrastructure as code (IaC) to create this architecture. A few of the many benefits of this approach of using infrastructure as code as opposed to building applications in the console are flexibility, repeatability, and reusability.
You have the flexibility to manage all the resources in one place and deploy a single template to any environment if needed. You can consistently deploy the same application in multiple Regions with one command. These characteristics avoid the environment drift problem. By using infrastructure as code, you can manage your applications more easily in one place.
We will not only use IaC, but also AWS CDK which lets build applications in the cloud with the considerable expressive of a programming language.

Serverless API builder

So, we take the advantage of using a programming language like Python to develop a builder whose function is to automate the creation of the lambdas and their integration with the gateway api to avoid boilderplate code.
With this builder, we just need to specify the method and the necessary api resources and configure the lambda values in our property file (with the power of property files we can set different values for different environments with development and production) and our code in cdk takes care of creating the lambda and the integration with the gateway api.

Let's see how!

Firstly, we will create a Python AWS CDK project and in the main class app.py where stack are defined, we will create an api gateway stack along with its properties:

api_gateway_props = ApiGatewayProps(
    props=properties,
    template=env_template
)

api_gateway_stack = ApiGatewayDPPStack(
    app,
    api_gateway_props=api_gateway_props,
    env=cdk_env,
    tags=tags)

Secondly, we will create an API Gateway stack and a private method __get_build_integrations is invoked where is the serverless builder:

class ApiGatewayDPPStack(core.Stack):


    def __init__(self, scope: core.Construct, api_gateway_props: api_gateway_props, **kwargs) -> None:
        props = api_gateway_props.props
        template = api_gateway_props.template
        id = template.substitute(name=props["api_gateway"]["stack_name"])
        super().__init__(scope, id, stack_name=id,description=props["api_gateway"]["description"], **kwargs)
        self.__create_api_gateway_role(props, template)
        api = api_gateway.RestApi(self, template.substitute(name=props["api_gateway"]["api_rest_name"]),
                                  description=template.substitute(name=props["api_gateway"]["api_rest_description"]),
                                  deploy_options=api_gateway.StageOptions(
                                      stage_name=template.substitute(name=props["api_gateway"]["stage_name"]),
                                      data_trace_enabled=True,
                                      logging_level=api_gateway.MethodLoggingLevel.INFO,
                                      access_log_destination=log_group_destination)
                                  )

self.__get_build_integrations(
    props,
    api,
    template,
    api_gateway_props
)

Finally, we will develop the __get_build_integrations:

def __get_build_integrations(self, props: dict,
                             api: api_gateway.RestApi, template: Template,
                             api_gateway_props: api_gateway_props):
    for integration in props["api_gateway"]["integrations"]:
        name = integration["name"]
        environment, id_prefix, request_schema, response_schema, role =  
    self.build_api_method(api_gateway_props,integration, name, template)
        lambda_integration = lambda_builder.create_lambda(self,
                                                          template,
                                                          integration["lambda"],
                                                          role,
                                                          environment)
method_builder_props = method_props.MethodBuilderProps(response_schema, lambda_integration,name, api, self, integration["base_resource"],integration["method"], id_prefix, request_schema)
api_method.MethodBuilder(method_builder_props)

To develop this method, we will have a loop which will iterates per integration and need three functions:
1.- Obtain properties and resources of API definition such as request, response schema to be validate.
2.- Create and configurates a lambda with its properties and layers. To make the example more complete, we have created two lambdas that create and delete an item with Amazon DynamoDB.

def create_lambda(self, template, props, execution_role, environment = None ):

    lambda_name = template.substitute(name=props['name'])
    lambda_function = _lambda.Function(
        self,
        id=lambda_name,
        function_name=lambda_name,
        description=props['description'],
        runtime=_lambda.Runtime.PYTHON_3_8,
        code=_lambda.Code.from_asset(props['code_from_asset']),
        handler=props['handler'],
        memory_size=props['cpu'],
        timeout=core.Duration.minutes(props['timeout']),
        role=execution_role,
        environment=environment
    )
    __add_layer(lambda_function, props, self, template)
    return lambda_function


def __add_layer(lambda_function, props, self, template):
    if "layer" in props:
        layer_name = template.substitute(name=props["layer"]["name"])
        layer = _lambda.LayerVersion(self,
                                     layer_name,
                                     layer_version_name=layer_name,
                                     compatible_runtimes=[_lambda.Runtime.PYTHON_3_8],
                                     code=_lambda.Code.from_asset(props["layer"]["code"]),
                                     description=props["layer"]["description"])
        lambda_function.add_layers(layer)

3.- Add a new method in API Gateway with all required information:

def addMethod(self, baseResource, props, responseModel, requestModel, requestValidator):
    baseResource.add_method(
        http_method=props.httpMethod,
        integration=self.lambdaIntegration,
        request_parameters=props.queryParameters,
        method_responses=[
            api_gateway.MethodResponse(status_code=str(props.responseStatusCode),
                                       response_models=responseModel)],
        request_validator=requestValidator,
        request_models=requestModel,
        operation_name=self.methodName)

All the code of the project is in the following github repository:

https://github.com/ysyzygy/aws-serverless-api-builder-cdk

The example in the AWS CDK documentation gives a basic idea of the methods you need, but the more complex the API Rest you are building, the more complex the code you need to automate it.

https://docs.aws.amazon.com/cdk/api/v1/python/aws_cdk.aws_apigateway/README.html#aws-lambda-backed-apis

This is all, as always feedback is welcome :)