Private serverless APIs in AWS

Our application landscaping is growing to more interfacing applications; A website, mobile apps and customer service tooling just to give a few examples. We've been using the AWS Serverless Application Model to quickly create stacks that are consumable by multiple applications. One downside of using AWS SAM is that the API Gateway endpoints are public by default. We protect these endpoints against abuse via an authentication, but for some endpoints we would like to have an additional layer of protected by making these endpoints only available from inside our VPC. In other words make our endpoints private.

NB This article specifically outlines making an AWS Serverless Application Model stack private. A serverless infrastructure has a great amount of similarities with AWS::ApiGateway::RestApi, but is still subtly different. I like to look at it as a simplification of AWS::ApiGateway::RestApi and AWS::Lambda::Function.

A serverless stack

A typical serverless stack consists of two parts: an AWS Lambda function and a trigger. This trigger can be anything from an upload on an S3 bucket, a message being pushed on a SQS or a call to an HTTP endpoint. In this last scenario the endpoint is defined as an API Gateway resource.

We use Cloudformation to define our stacks. Let's consider a minimal function that allows retrieval of entities of the type entity via the call GET /entities:

Parameters:
  Environment:
    Type : "String"
    Default: "development"
    Description: "Environment in which resources are deployed."

Resources:
  # Lambdas
  GetEntitiesFunction:
    Type: AWS::Serverless::Function
    Properties:
      Handler: index.getEntities
      FunctionName: !Sub "${AWS::StackName}-get-entities"
      CodeUri: .
      Role: "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
      Events:
        GetResource:
          Type: Api
          Properties:
            Path: /entities
            Method: get
            RestApiId: !Ref "ServerlessApi"

  # Api gateway
  ServerlessApi:
    Type: "AWS::Serverless::Api"
    Properties:
      StageName: !Ref "Environment"

When deployed this should result in a serverless stack with a single endpoint that is publicly available for everyone that knows the url.

VPC endpoint

Making an API Gateway private requires a VPC. Every stack that requires access to the private API must be part of the VPC. Benefit of having a VPC is that access control is centralized in the VPC configuration.

In order to make the API private the traffic will have to pass through the VPC. This is done via a VPC endpoint:

Resources:
  APIGatewayVpcEndpoint:
    Type: "AWS::EC2::VPCEndpoint"
    Properties:
      SubnetIds:
        - !ImportValue "exported-subnet-id"
      SecurityGroupIds:
        - !Ref "resource of type AWS::EC2::SecurityGroup"
      ServiceName: "com.amazonaws.eu-west-1.execute-api"
      VpcId: !ImportValue "exported-vpc-id"
      VpcEndpointType: "Interface"
      PrivateDnsEnabled: false

Disabling the private DNS is a deliberate choice. The reason for that is explained later in this article.

Marking an API private

By default API Gateway resources are publicly available. In 2018 AWS introduced the possibility to mark API Gateway resources as private. Marking an API Gateway as private is easily done in the Cloudformation definition of our stack:

Resources:
  # ...

  # Api gateway
  ServerlessApi:
    Type: "AWS::Serverless::Api"
    Properties:
      StageName: !Ref "Environment"
      EndpointConfiguration:
        Type: "PRIVATE"
        VPCEndpointIds:
          - !ImportValue "infrastructure-api-gateway-vpc-endpoint"
      Auth:
        ResourcePolicy:
          IntrinsicVpceWhitelist:
            - !ImportValue "infrastructure-api-gateway-vpc-endpoint"

NB This example imports a VPC endpoint. We try to separate stacks as much as possible for ownership and maintainability. In a separate infrastructure stack the VPC endpoint(s) are defined and the required identifiers are exported from those stacks. That way every other stack can use them while the hosting team still has the possibility to make changes to the endpoint.

The EndpointConfiguration tells AWS that the API should no longer be publicly available. A required property of EndPointConfiguration is a list of VPC endpoints.

Next to marking the API private we also need to tell the API it is allowed to be called from the endpoint. This is done via a resource policy. In a AWS::Serverless::Api this is done via the Auth property.

When we now deploy this stack, the API will no longer be publicly available.

Consuming the private API

The downside of making an API Gateway resource is that consuming becomes a little bit more complicated. There are two methods that work out-of-the-box. Those methods use the VPC endpoint as entry for the request:

curl -v https://{public-dns-hostname}.execute-api.{region}.vpce.amazonaws.com/{stage}/{path}

The {public-dns-hostname} of the VPC endpoint is visible in the AWS Console. For the following examples I will call the /entities endpoint defined in the Cloudformation for region eu-west-1 and stage testing.

The first method uses the Host header:

curl -v https://vpce-01234567abcdef012-01234567.execute-api.eu-west-1.vpce.amazonaws.com/testing/entities \
-H 'Host: 01234567ab.execute-api.eu-west-1.amazonaws.com'

The second method uses the x-apigw-api-id header:

curl -v https://vpce-01234567abcdef012-01234567.execute-api.eu-west-1.vpce.amazonaws.com/testing/entities \
-H 'x-apigw-api-id: 01234567ab'

Both the host and the API id can be found in the AWS Console after deploying your stack.

There are other options to call your private endpoints that a bit more user-friendly, but that have their own caveats. You can enable the "Private DNS Name" while creating an interface VPC endpoint for API Gateway, but this also means that the VPC where the VPC endpoint is present will no longer be able to access public APIs:

When you select the Enable Private DNS Name option while creating an interface VPC endpoint for API Gateway, the VPC where the VPC Endpoint is present won't be able to access public (edge-optimized and regional) APIs. For more information, see Why can't I connect to my public API from an API Gateway VPC endpoint?.

Source: Invoking your private API using private DNS names

As we have both public and private APIs and we also consume public APIs from within our VPC we are not able to use this approach.

Another option is to use a Route53 alias. We have not yet tried this, so I cannot make any comments on this option.

Conclusion

The SAM model offered by AWS allows for fast development of applications as you don't have to worry too much about your infrastructure choices. AWS will scale if needed. Building a modest sized API can be achieved using the tandem Lambda and API Gateway, but this will result in a public endpoint by default.

If you want to utilize the benefits of the SAM model, but you don't want to expose your API to the public you can make your API private. This will increase security on your endpoint, but it will also introduce some downsides like the necessity to have a VPC and a VPC endpoint.