Serverless API Development on AWS with TypeScript - Part 2

Part 1 of this series was an introduction to the Tenant Service project, the AWS services used and configurations for the Serverless Framework.

In this article, part 2, we will take a deeper look at the composition of the entities because they play a vital role in how we define our table schema when adopting the Single Table design strategy. Also contained in this part 2 is Lambda function configuration with the Serverless Framework, configurations for creating DynamoDB table, keys and Global Secondary Index.

charlallison / tenant-service

Simple example of building Serverless API on AWS using TypeScript

Tenant Service

The Tenant Service is a serverless application built on AWS Lambda and other services using the Serverless Framework. It provides management of tenants and payments made for a property, as well as basic CRUD operations for properties.

Features

• Tenant Management: Create, update, and delete tenant information, including name, phone, and property allocation.
• Payment Management: Record tenant payments, including payment amount and date.
• Property CRUD: Perform basic CRUD operations (Create, Read, Update) on property information, such as property details, address, and rental rates.

Architecture

The Tenant Service utilizes the following AWS services:

• AWS Lambda: Handles the serverless functions for processing tenant, payment and property operations.
• API Gateway: Provides a RESTful API to interact with the service.
• DynamoDB: Stores tenant and property data, as well as payment records.
• EventBridge (CloudWatch Events): Triggers lambda function on schedule.
• AWS CloudFormation: Automates the provisioning and management of AWS resources.
• Amazon SNS Sends SMS…

View on GitHub

Entities

The Tenant Service contains these entities tenant, payment and property. Let's take a look at the Tenant entity.

import {v4} from "uuid";

export enum TenantStatus {
  Active = "Active",
  InActive = "Inactive"
}

export class Tenant {
  readonly id: string;
  readonly PK: string;
  readonly SK: string;
  readonly name: string;
  readonly phone: string;
  readonly status = TenantStatus.InActive;
  readonly Type: string = Tenant.name;
  readonly GSI1PK: string;
  readonly GSI2PK: string;

  constructor(data: Partial<Tenant>){
    this.id = data.id ?? v4();
    this.name = data.name;
    this.phone = data.phone;
    const { PK, SK } = Tenant.BuildPK(this.id);
    this.PK = PK;
    this.SK = SK;

    const { GSI1PK, GSI2PK } = Tenant.BuildGSIKeys({ id: this.id });
    this.GSI1PK = GSI1PK;
    this.GSI2PK = GSI2PK;
  }

  static BuildPK(id: string) {
    return {
      PK: `tenant#id=${id}`,
      SK: `profile#id=${id}`
    }
  }

  static BuildGSIKeys(prop?: {id: string}) {
    return {
      GSI1PK: `tenant#id=${prop?.id}`,
      GSI2PK: `type#${Tenant.name}`
    };
  }
}

In the src/tenant.ts file, we have pretty self-explanatory fields and two static functions. We also have an enumeration that is used to categorise tenants into;

• those whose payments are still valid and have not expired as Active and

• those whose payments have expired and need to be renewed or no longer reside in the property as Inactive.

The static functions, BuildPK and BuildGSIKeys are for formatting primary keys for both our default table and for Global Secondary Index (GSIs). These keys help with data manipulation in DynamoDB. These static functions are used as a convention in the Tenant Service such that the BuildPK is used to format the Primary key and BuildGSIKeys is used to format GSI keys where applicable. More about keys in the DynamoDB section.

Lambda Configuration

Lambda function configuration refers to parameters needed for the execution of a lambda function. This includes the function trigger and memory requirements, environment variables, AWS role and other services that the function will need to interact with. The Serverless Framework makes developing and deploying a Lambda function easy. However, it is up to you to organise your code in your preferred way that will be easy to navigate. I have organised the Tenant service in such a way that each lambda function resides in a separate directory to isolate them from the other functions. This provides a clear visual indication that all the required files for a function are located within a single directory.

The ./lambda directory contains all the functions that make our API. I prefer to name a function directory using _verb_s as it is easy to spot where a function resides based on its name and the action it performs. Let's take a look at the CreateTenant function.

CreateTenant Function

The CreateTenant directory contains three (3) files; config.yml, schema.ts and index.ts.

Config.yml

The config.yml file contains necessary function configurations, description, deployment and invocation mechanism using the Serverless Framework. Every necessary configuration as per application requirement for this function has to be declared here. The important keys in this file are described below:

• handler - points to the TypeScript function to be invoked when the corresponding event is triggered - in this case, it is the main function.

• memorySize and timeOut - has to do with the amount of memory needed and execution duration of the function

• environment: key for specifying environment variables if needed

• iamRoleStatements: lists permissions and the object of action in the AWS ecosystem. In the configuration below we have the dynamodb:PutItem action with permission set to allow on a database table resource via its resource name.

• events: lists the triggers for the function - in this case, it is http which refers to an invocation from API Gateway. It also specifies the request method as POST and its path as /tenants.

schema.ts

This file contains the JSON schema for validating the request payload for the CreateProperty function. This JSON schema is used by middy middleware to validate the request payload before the lambda function is executed. An error is thrown if the shape of the payload does not conform to the schema.

export default {
  type: 'object',
  required: ['body'],
  properties: {
    body: {
      type: 'object',
      required: ['name', 'phone'],
      properties: {
        name: {type: 'string'},
        phone: {type: 'string'}
      }
    }
  }
} as const

index.ts

contains the code that will be executed when the function is triggered

• handler is a function of type ValidatedEventAPIGatewayProxyEvent<T>. It receives an argument name event that has been formatted to match a specified schema via the generic <typeof S> construct. The event argument contains event-related fields which include body, queryStringParameters and pathParameters fields from where the user data is gotten.

• The exported main function is lambda's entry point. It takes in the handler function and schema field via the middyfy function and chains jsonBodyParser, validator and httpErrorHandler middlewares together before finally calling the handler function.

Type definition function

The ValidatedAPIGatewayProxyEvent is the type definition for the handler function used in index.ts. This function sets the shape of the body and queryStringParameters fields in the request payload proxied by API Gateway.

This src/libs/api-gateway.ts file also contains a function formatJSONResponse that formats response for API Gateway.

    import type { APIGatewayProxyEvent, APIGatewayProxyResult, Handler } from "aws-lambda"
    import type { FromSchema } from "json-schema-to-ts";

    type ValidatedAPIGatewayProxyEvent<S> = Omit<APIGatewayProxyEvent, 'body' | 'queryStringParameters'>
      & { body: FromSchema<S> }
      & { queryStringParameters: FromSchema<S> }

    export type ValidatedEventAPIGatewayProxyEvent<S> = Handler<ValidatedAPIGatewayProxyEvent<S>, APIGatewayProxyResult>

    export const formatJSONResponse = (response: Record<string, unknown>, statusCode = 200) => {
      return {
        statusCode,
        body: JSON.stringify(response)
      }
    }

DynamoDB Configuration

Single-Table approach...certainly not for the faint-hearted.

• Paul Swali

Designing for a relational database software system sometimes requires that one achieves normalization - a 3NF, by splitting data into separate tables where each table represents an entity. You could have a join table as needed. The Single Table design requires a different approach which could be challenging at least for the first time and in some complex scenarios.

It is important to note that DynamoDB does not constrain you to the use of only the Single table design. You are welcome to design your DynamoDB tables as though you were working on a relational database system with several tables in place. With that said, let's dive into the Tenant service table schema.

The configuration for the DynamoDB resource is found in ./resource/database-table.yml which contains some sort of DDL statement. Given the fact that all entities will be stored in a single table, we have to define generic keys that will be meaningful across all entities. In other words, the attributes selected to serve as keys and ensure data integrity must apply to Tenant, Payment and Property entities. To make that generic and easy to manipulate, it is best to name them by their function. For the Tenant service, PK refers to the partition key and SK, sort key.

The DynamoDB resource file database-table.yml reveals self-descriptive fields except for a few. The table is designed to have a composite primary key and two Global Secondary Indexes that have their respective primary keys; GSI1PK and GSI2PK. The table also enables DynamoDB Streaming via the StreamSpecification key.

TenantServiceTable:
  Type: AWS::DynamoDB::Table
  Properties:
    TableName: ${self:service}-${self:provider.stage}-Table
    BillingMode: PAY_PER_REQUEST
    AttributeDefinitions:
      - AttributeName: PK
        AttributeType: S
      - AttributeName: SK
        AttributeType: S
      - AttributeName: GSI1PK
        AttributeType: S
      - AttributeName: GSI2PK
        AttributeType: S
    KeySchema:
      - AttributeName: PK
        KeyType: HASH
      - AttributeName: SK
        KeyType: RANGE
    StreamSpecification:
      StreamViewType: NEW_IMAGE
    GlobalSecondaryIndexes:
      - IndexName: GSI1
        KeySchema:
          - AttributeName: GSI1PK
            KeyType: HASH
        Projection:
          ProjectionType: ALL
      - IndexName: GSI2
        KeySchema:
          - AttributeName: GSI2PK
            KeyType: HASH
        Projection:
          ProjectionType: ALL

DynamoDB Keys - PK & SK

A composite primary key is a primary key that consists of more than one attribute - which is used to uniquely identify a record in a table. Leveraging the composite key concept, we have the PK and SK defined in the KeySchema field. The primary key determines the Data Access Patterns in DynamoDB which is how data is accessed within a table. Care has to be taken when deciding on how a primary key is constructed. In the snippet below, we see how the Tenant's entity primary key is constructed; an id is passed in as an argument to the BuildPK function and we have an object with two attributes as the primary key - this primary key has to match the requirement of the primary key definition in the ./resource/database-table.yml file.

static BuildPK(id: string) {
  return {
    PK: `tenant#id=${id}`
    SK: `profile#id=${id}`
  }
}

To retrieve a tenant's record from the table, this primary key is used. By using the primary key, a full table scan is avoided because of its performance and cost implications. We may need to retrieve a tenant's record by a different attribute other than the initial primary key. When this happens we then need to create an alternative primary key typically seen as another Data Access Pattern. In doing so, a tenant record can be retrieved based on that attribute that now forms the primary key. How can this alternative primary key be created?

Given the needs of an application, multiple Data Access Patterns would need to be utilized for effective data retrieval

Global Secondary Index

To keep it simple, a Global Secondary Index - (GSI) enables you to create an alternative primary key using a different attribute. This ultimately provides more options with which we can query a DynamoDB table.

The Tenant entity has two fields; GSI1PK and GSI2PK that serve as primary keys to the GSIs defined in the GlobalSecondaryIndexes key. There are two indexes; GSI1 and GSI2, the key type and projection which specifies the attributes you want to return when that index is used.

The Payment and Property entities have just one GSI key which is the GSI1PK . In Part 3 of this series, we will look at the usefulness of having a generic key such that entities share similar keys but with different functionality. It is important to note that GSIs and their corresponding keys are added on a per-need basis - the needed Data Access Pattern should determine what keys are created and their composition.

Keys In Action

// ...imports

const handler: ValidatedEventAPIGatewayProxyEvent<typeof schema> = async (event) => {
  const { status }: {[key: string]: any} = event.queryStringParameters;
  const { GSI2PK } = Tenant.BuildGSIKeys();

  const result = await ddbDocClient.send(new QueryCommand({
    TableName: process.env.TENANT_TABLE_NAME,
    IndexName: GSIs.GSI2,
    KeyConditionExpression: 'GSI2PK = :gsi2pk',
    FilterExpression: '#status = :status',
    ExpressionAttributeValues: {
      ':gsi2pk': GSI2PK,
      ':status': status,
    },
    ExpressionAttributeNames: {
      '#name': 'name',
      '#status': 'status'
    },
    ProjectionExpression: 'id, #name'
  }));

  const tenants = result.Items.map(item => item as Pick<Tenant, 'id' | 'name'>);

  return formatJSONResponse({
    count: tenants.length,
    tenants
  });
}

The snippet above contains a DynamoDB Query command operation where an Index and its corresponding primary key are used to fetch one or more records from a DynamoDB table. This is achieved using the IndexName and KeyConditionExpression keys. You can also specify filter conditions using the FilterExpression key as necessary.

Conclusion

In this article we took a step further into entities and their composition, using the tenant entity as a case study. We also looked at keys, how they were constructed and their usefulness in index operations.

Lambda configuration came next as we looked at how the Serverless Framework applies our configuration for AWS Lambda. The configuration also affects how functions are written as we have to specify the services, interactions and the actual operations (permissions) that can be carried out by the function.

A major section of this article looked at the DynamoDB table service. While this isn't an in-depth review of the DynamoDB service and features, we saw the usefulness of primary key - composite and non-composite types, Global Secondary Index and how you can select what Index you intend to use for a query operation.

In Part 3 of this article, we will go deeper into how primary keys determine what data is returned. We will also see DynamoDB streams in action - with this, you can track changes to your table immediately after a DML operation without issuing another query.