CloudWatch Logging for Web Apps (Part 1)

In this post, I'm describing how to replicate the LambdaSharp app capability to log to CloudWatch Logs using an Amazon API Gateway REST API.

Observability is a critical building block for developers. Therefore, it is an integral part of the LambdaSharp developer experience. For reference, this is how a Blazor WebAssembly app is created and automatically wired for CloudWatch logging in LambdaSharp.

Module: Sample.BlazorWebAssembly
Items:
  - App: MyBlazorApp

Yes, that is really it! Nothing additional is needed, but there are plenty of additional capabilities. However, non-LambdaSharp developers may want to achieve the same capability for their apps using their preferred framework. That is the purpose of this post. It shows how to build the CloudWatch logging capability for any frontend app using any framework.

Overview

This implementation does not use any Lambda functions. Instead, we enable logging to CloudWatch by directly integrating the API Gateway REST API with the CloudWatch Logs API using Apache Velocity templates. This design means there is only minimal code involved, no Lambda cold-start latencies, and no Lambda invocation costs.

The implementation is described in terms of CloudFormation resources using the YAML notation, but the same outcome can be achieved by using AWS Console instead.

Logging REST API

First, we need to create a new API Gateway resource. We define the top-level .app resource to anchor our API. In LambdaSharp, this top-level resource name is configurable via CloudFormation parameters, which are omitted here.

RestApi:
  Type: AWS::ApiGateway::RestApi
  Properties:
    Name: !Sub "${AWS::StackName} App API"

RestApiAppResource:
  Type: AWS::ApiGateway::Resource
  Properties:
    RestApiId: !Ref RestApi
    ParentId: !GetAtt RestApi.RootResourceId
    PathPart: .app

CloudWatch Log Group

The CloudWatch Log Group should be created explicitly for each app to make it is easy to distinguish them across apps. In addition, a log retention policy should be set to limit the amount of storage the log group uses to avoid being billed indefinitely for it.

LogGroup:
  Type: AWS::Logs::LogGroup
  Properties:
    RetentionInDays: 90

API Gateway IAM Role

API Gateway needs permission to create log streams in the log group and write to them. This is achieved by the following IAM role definition.

RestApiRole:
  Type: AWS::IAM::Role
  Properties:
    AssumeRolePolicyDocument:
      Version: 2012-10-17
      Statement:
        - Sid: ApiGatewayPrincipal
          Effect: Allow
          Principal:
            Service: apigateway.amazonaws.com
          Action: sts:AssumeRole
    Policies:
      - PolicyName: ApiLogsPolicy
        PolicyDocument:
          Version: 2012-10-17
          Statement:
            - Sid: LogGroupPermission
              Effect: Allow
              Action:
                - logs:CreateLogStream
                - logs:PutLogEvents
              Resource:
                - !Sub "arn:${AWS::Partition}:logs:${AWS::Region}:${AWS::AccountId}:log-group:${LogGroup}"
                - !Sub "arn:${AWS::Partition}:logs:${AWS::Region}:${AWS::AccountId}:log-group:${LogGroup}:log-stream:*"

API Validation

A neat feature of API Gateway REST API is that it can validate requests against a JSON schema model. This capability prevents unnecessary invocations of the API when the incoming payload is not valid. Validation is enabled by associating each API Gateway method with the following validator declaration.

RestApiValidator:
  Type: AWS::ApiGateway::RequestValidator
  Properties:
    RestApiId: !Ref RestApi
    ValidateRequestBody: true
    ValidateRequestParameters: true

REST API

This next section is a bit heavy, because how API Gateway resources, methods, and integrations are built.

First, we create the logs resource associated with the API methods.

RestApiAppLogsResource:
  Type: AWS::ApiGateway::Resource
  Properties:
    RestApiId: !Ref RestApi
    ParentId: !Ref RestApiAppResource
    PathPart: "logs"

Next, we need to create the OPTIONS method to handle CORS requests. Note this implementation uses Allow-Origin: '*', which should be replaced with the actual host scheme and name from which the application is served. LambdaSharp uses CloudFormation parameters to make it configurable, but these were omitted for brevity.

RestApiAppLogsResourceOPTIONS:
  Type: AWS::ApiGateway::Method
  Properties:
    AuthorizationType: NONE
    RestApiId: !Ref RestApi
    ResourceId: !Ref RestApiAppLogsResource
    HttpMethod: OPTIONS
    Integration:
      IntegrationResponses:
        - StatusCode: 204
          ResponseParameters:
            method.response.header.Access-Control-Allow-Headers: "'Content-Type,X-Amz-Date,Authorization,X-Api-Key,X-Amz-Security-Token'"
            method.response.header.Access-Control-Allow-Methods: "'OPTIONS,POST,PUT'"
            method.response.header.Access-Control-Allow-Origin: "'*'"
            method.response.header.Access-Control-Max-Age: "'600'"
          ResponseTemplates:
            application/json: ''
      PassthroughBehavior: WHEN_NO_MATCH
      RequestTemplates:
        application/json: '{"statusCode": 200}'
      Type: MOCK
    MethodResponses:
      - StatusCode: 204
        ResponseModels:
          application/json: 'Empty'
        ResponseParameters:
            method.response.header.Access-Control-Allow-Headers: false
            method.response.header.Access-Control-Allow-Methods: false
            method.response.header.Access-Control-Allow-Origin: false
            method.response.header.Access-Control-Max-Age: false

Once the browser is authorized via the OPTIONS request, we need to provide two additional endpoints: one for creating a new log stream using POST and another for writing to a log stream using PUT. In addition, we define a JSON schema model for each endpoint to validate requests before they are executed.

Note that the app is responsible for creating a new log stream. For single page apps (SPA), a new log stream should be created each time the app loads. This is also the behavior for Blazor WebAssembly apps built with LambdaSharp.

Create LogStream - POST:/.app/logs

The POST method creates a new log stream in the associated log group. Some of the response handling relates to how errors are returned to the calling application. Emphasis of the handling is on providing useful feedback without revealing too many internal details.

Similar to the OPTIONS method, this configurations uses Allow-Origin: '*', which should be replaced with the actual host scheme and name from which the application is served. LambdaSharp uses CloudFormation parameters to make it configurable, but these were omitted for brevity.

RestApiAppLogsResourcePOST:
  Type: AWS::ApiGateway::Method
  Properties:
    OperationName: CreateLogStream
    ApiKeyRequired: true
    RestApiId: !Ref RestApi
    ResourceId: !Ref RestApiAppLogsResource
    AuthorizationType: NONE
    HttpMethod: POST
    RequestModels:
      application/json: !Ref RestApiAppLogsResourcePOSTRequestModel
    RequestValidatorId: !Ref RestApiValidator
    Integration:
      Type: AWS
      IntegrationHttpMethod: POST
      Uri: !Sub "arn:${AWS::Partition}:apigateway:${AWS::Region}:logs:action/CreateLogStream"
      Credentials: !GetAtt RestApiRole.Arn
      PassthroughBehavior: WHEN_NO_TEMPLATES
      RequestParameters:
        integration.request.header.Content-Type: "'application/x-amz-json-1.1'"
        integration.request.header.X-Amz-Target: "'Logs_20140328.CreateLogStream'"
      RequestTemplates:
        application/json: !Sub |-
          #set($body = $input.path('$'))
          {
            "logGroupName": "${LogGroup}",
            "logStreamName": "$body.logStreamName"
          }
      IntegrationResponses:
        - SelectionPattern: "200"
          StatusCode: 200
          ResponseParameters:
            method.response.header.Access-Control-Allow-Origin: "'*'"
          ResponseTemplates:
            application/x-amz-json-1.1: |-
              { }

        - SelectionPattern: "400"
          StatusCode: 400
          ResponseParameters:
            method.response.header.Access-Control-Allow-Origin: "'*'"
          ResponseTemplates:
            application/x-amz-json-1.1: |-
              #set($body = $input.path('$'))
              {
              #if($body.message.isEmpty())
                "error": "Unknown error"
              #else
                "error": "$util.escapeJavaScript($body.message).replaceAll("\\'","'")"
              #end
              }

        - StatusCode: 500
          ResponseParameters:
            method.response.header.Access-Control-Allow-Origin: "'*'"
          ResponseTemplates:
            application/x-amz-json-1.1: |-
              {
                "error": "Unexpected response from service."
              }

    MethodResponses:
      - StatusCode: 200
        ResponseModels:
          application/json: Empty
        ResponseParameters:
            method.response.header.Access-Control-Allow-Origin: false

      - StatusCode: 400
        ResponseModels:
          application/json: Empty
        ResponseParameters:
            method.response.header.Access-Control-Allow-Origin: false

      - StatusCode: 500
        ResponseModels:
          application/json: Empty
        ResponseParameters:
            method.response.header.Access-Control-Allow-Origin: false

RestApiAppLogsResourcePOSTRequestModel:
  Type: AWS::ApiGateway::Model
  Properties:
    Description: CreateLogStream
    ContentType: application/json
    RestApiId: !Ref RestApi
    Schema:
      $schema: http://json-schema.org/draft-04/schema#
      type: object
      properties:
        logStreamName:
          type: string
      required:
        - logStreamName

Append to Log Stream - POST .app/logs

Similar to the POST method, the PUT method validates incoming requests and limits what internal details are exposed when errors occur.

Similar to the OPTIONS method, this configurations uses Allow-Origin: '*', which should be replaced with the actual host scheme and name from which the application is served. LambdaSharp uses CloudFormation parameters to make it configurable, but these were omitted for brevity.

RestApiAppLogsResourcePUT:
  Type: AWS::ApiGateway::Method
  Properties:
    OperationName: PutLogEvents
    ApiKeyRequired: true
    RestApiId: !Ref RestApi
    ResourceId: !Ref RestApiAppLogsResource
    AuthorizationType: NONE
    HttpMethod: PUT
    RequestModels:
      application/json: !Ref RestApiAppLogsResourcePUTRequestModel
    RequestValidatorId: !Ref RestApiValidator
    Integration:
      Type: AWS
      IntegrationHttpMethod: POST
      Uri: !Sub "arn:${AWS::Partition}:apigateway:${AWS::Region}:logs:action/PutLogEvents"
      Credentials: !GetAtt  RestApiRole.Arn
      PassthroughBehavior: WHEN_NO_TEMPLATES
      RequestParameters:
        integration.request.header.Content-Type: "'application/x-amz-json-1.1'"
        integration.request.header.X-Amz-Target: "'Logs_20140328.PutLogEvents'"
        integration.request.header.X-Amzn-Logs-Format: "'json/emf'"
      RequestTemplates:
        application/json: !Sub |-
          #set($body = $input.path('$'))
          {
            "logEvents": [
          #foreach($logEvent in $body.logEvents)
                {
                  "message": "$util.escapeJavaScript($logEvent.message).replaceAll("\\'","'")",
                  "timestamp": $logEvent.timestamp
                }#if($foreach.hasNext),#end
          #end
            ],
            "logGroupName": "${LogGroup}",
            "logStreamName": "$body.logStreamName",
            "sequenceToken": #if($body.sequenceToken.isEmpty()) null#else "$body.sequenceToken"#end
          }
      IntegrationResponses:
        - SelectionPattern: "200"
          StatusCode: 200
          ResponseParameters:
            method.response.header.Access-Control-Allow-Origin: "'*'"
          ResponseTemplates:
            application/x-amz-json-1.1: |-
              {
                "nextSequenceToken": "$input.path('$.nextSequenceToken')"
              }

        - SelectionPattern: "400"
          StatusCode: 400
          ResponseParameters:
            method.response.header.Access-Control-Allow-Origin: "'*'"
          ResponseTemplates:
            application/x-amz-json-1.1: |-
              #set($body = $input.path('$'))
              #if($body.expectedSequenceToken.isEmpty())
              {
              #if($body.message.isEmpty())
                "error": "Unknown error"
              #else
                "error": "$util.escapeJavaScript($body.message).replaceAll("\\'","'")"
              #end
              }
              #else
              {
              #if($body.message.isEmpty())
                "error": "unknown error",
              #else
                "error": "$util.escapeJavaScript($body.message).replaceAll("\\'","'")",
              #end
                "nextSequenceToken": "$body.expectedSequenceToken"
              }
              #end

        - StatusCode: 500
          ResponseParameters:
            method.response.header.Access-Control-Allow-Origin: "'*'"
          ResponseTemplates:
            application/x-amz-json-1.1: |-
              {
                "error": "Unexpected response from service."
              }

    MethodResponses:
      - StatusCode: 200
        ResponseModels:
          application/json: Empty
        ResponseParameters:
            method.response.header.Access-Control-Allow-Origin: false

      - StatusCode: 400
        ResponseModels:
          application/json: Empty
        ResponseParameters:
            method.response.header.Access-Control-Allow-Origin: false

      - StatusCode: 500
        ResponseModels:
          application/json: Empty
        ResponseParameters:
            method.response.header.Access-Control-Allow-Origin: false

RestApiAppLogsResourcePUTRequestModel:
  Type: AWS::ApiGateway::Model
  Properties:
    Description: PutLogEvents
    ContentType: application/json
    RestApiId: !Ref RestApi
    Schema:
      $schema: http://json-schema.org/draft-04/schema#
      type: object
      properties:
        logEvents:
          type: array
          items:
            - type: object
              properties:
                message:
                  type: string
                timestamp:
                  type: integer
              required:
                - message
                - timestamp
        logStreamName:
          type: string
        sequenceToken:
          type:
            - string
            - "null"
      required:
        - logEvents
        - logStreamName

API Key & Usage Plan

The following resources declare an API key and usage plan. The API key is set by default to the Base64 value of the CloudFormation stack GUID. It is recommended to explicitly set the API key since the frontend app will need access to it to use the logging REST API. The API key can be further obfuscated by combining it with an internal value of the app. In LambdaSharp, the API key is generated by combining the CloudFormation stack GUID and the compiled .NET Core assembly identifier GUID.

RestApiKey:
  Type: AWS::ApiGateway::ApiKey
  Properties:
    Description: !Sub "${AWS::StackName} App API Key"
    Enabled: true
    StageKeys:
      - RestApiId: !Ref RestApi
        StageName: !Ref RestApiStage
    Value:
      Fn::Base64: !Select [ 2, !Split [ "/", !Ref AWS::StackId ]]

RestApiUsagePlan:
  Type: AWS::ApiGateway::UsagePlan
  Properties:
    ApiStages:
      - ApiId: !Ref RestApi
        Stage: !Ref RestApiStage
    Description: !Sub "${AWS::StackName} App API Usage Plan"
    Throttle:
      BurstLimit: 200
      RateLimit: 100

RestApiUsagePlanKey:
  Type: AWS::ApiGateway::UsagePlanKey
  Properties:
    KeyId: !Ref RestApiKey
    KeyType: API_KEY
    UsagePlanId: !Ref RestApiUsagePlan

API Deployment

Finally, we define a stage called LATEST, which is used by the deployment resource. Note that CloudFormation only runs the deployment once. Subsequent CloudFormation stack updates need to be manually deployed when the REST API changes. LambdaSharp uses the Finalizer construct to allow for configuration changes to be always applied automatically.

RestApiStage:
  Type: AWS::ApiGateway::Stage
  Properties:
    DeploymentId: !Ref RestApiDeployment
    Description: App API LATEST Stage
    RestApiId: !Ref RestApi
    StageName: LATEST

RestApiDeployment:
  Type: AWS::ApiGateway::Deployment
  Properties:
    Description: !Sub "${AWS::StackName} App API"
    RestApiId: !Ref RestApi
  DependsOn:
    - RestApiAppLogsResource
    - RestApiAppLogsResourcePOST
    - RestApiAppLogsResourcePOSTRequestModel
    - RestApiAppLogsResourcePUT
    - RestApiAppLogsResourcePUTRequestModel

Conclusion - To be continued...

In this post, we created the resources required to enable a frontend apps to log to CloudWatch directly. In the next post, we will cover the protocol for logging via this REST API.

Happy Hacking!