DEV Community: Luis Valdés

How to generate a presigned url to upload images to S3

Luis Valdés — Tue, 19 Mar 2024 15:58:47 +0000

In this post I want to share an implementation on how to create presigned url to upload files to an S3 bucket, the piece of technology we are going to use is an aws library that provides the mechanism needed to upload a file to an S3 bucket by performing a POST request with a specific body and url

Requirements

git
NodeJS 14 or later, my version is v18.18.0
An AWS account and configured credentials
Install cdk command
docker
Basic knowledge of TypeScript
AWS Lambda user

TL;DR;

Clone the repo and follow the instructions to deploy the project, you can use the gitpod configuration which comes with nodejs, aws cli v2, docker and cdk installed

New structure of our graphql schema

We are ignoring the resources not related to image upload, so in this case we have the query getImageUploadUrl which returns a PresignedImageUrl.

input ImageInput {
  filename: String!
  contentType: String!
}

type Query {
  getImageUploadUrl(input: ImageInput!): PresignedImageUrl!
}

type PresignedField {
  name: String!
  value: String!
}

type PresignedImageUrl {
  id: ID!
  fields: [PresignedField!]!
  url: String!
}

Create an S3 bucket for the images

In our current scenario we need to create a bucket with specific configurations, we need to allow objects to be public and we also need to declare the cors configuration, we set the owner of the file as the object writer, this bucket is setup to allow files to be public by disabling blockPublicAcls

const assetsBucket = new s3.Bucket(this, 'AssetsBucket',
 {
  objectOwnership: s3.ObjectOwnership.OBJECT_WRITER,
  blockPublicAccess: new s3.BlockPublicAccess({ blockPublicAcls: false }),
  cors: [
    {
      id: "corsRule",
      allowedMethods: [s3.HttpMethods.GET, s3.HttpMethods.POST, s3.HttpMethods.PUT],
      allowedHeaders: ['*'],
      allowedOrigins: ['*'],
      exposedHeaders: [
        "Access-Control-Allow-Origin"
      ]
    } as s3.CorsRule
  ]
 }
);

Images Table

As we did in our previous article, we are going to create a table with a global secondary index that will allow us to filter records by owner

    const cfnImagesTable = new dynamodb.CfnTable(this, 'CfnImagesTable', {
      keySchema: [{
        attributeName: 'id',
        keyType: 'HASH',
        {
          attributeName: 'createdAt',
          keyType: 'RANGE',
        }
      }],
      attributeDefinitions: [
        {
          attributeName: 'id',
          attributeType: 'S',
        },
        {
          attributeName: 'owner',
          attributeType: 'S',
        },
        {
          attributeName: 'createdAt',
          attributeType: 'S',
        }
      ],
      billingMode: 'PAY_PER_REQUEST',
      globalSecondaryIndexes: [
        {
          indexName: 'byOwner',
          keySchema: [
            {
              attributeName: 'owner',
              keyType: 'HASH',
            },
            {
              attributeName: 'id',
              keyType: 'RANGE',
            }
          ],
          projection: {
            projectionType: 'ALL',
          },
        }
      ],
    });

Create the get image upload url function

In order to configure our function, we will use the NodejsFunction construct. In our scenario we are adding the library ulid to generate the id of our records. Another thing to point out is that we are providing environment variables for out images table and our assets bucket name

const getImageUploadUrl = new lambda_nodejs.NodejsFunction(this, "GetPresignedImageUrlLambdaFunction", {
  entry: path.join(__dirname, '../functions/createPresignedPost/index.ts'),
  bundling: {
    nodeModules: ['ulid'],
  },
  projectRoot: path.join(__dirname, '../functions/createPresignedPost'),
  depsLockFilePath: path.join(__dirname, '../functions/createPresignedPost/package-lock.json'),
  handler: 'getImageUploadUrl',
  runtime: lambda_.Runtime.NODEJS_20_X,
  environment: {
    IMAGES_TABLE: imagesTable.tableName,
    ASSETS_BUCKET: assetsBucket.bucketName,
  },
  role: lambdaRole,
  timeout: cdk.Duration.seconds(30)
})

What the source code of the function looks like?

This article assumes you have basic level of TypeScript and AWS Lambda, so we can import all the needed modules and consider them to be self explanatory.

We then declare the clients for S3 and DynamoDB, also we create a type for our input arguments that are coming from the graphql query.

The text content-length-range is a condition for the input file to be from 1024 bytes to 10485760, which is 10 megabytes maximum size for the file.

We also insert a new record in the images table with the details of the key of the image and the url

import { AppSyncResolverEvent , AppSyncIdentityCognito} from "aws-lambda";
import { DynamoDBClient } from "@aws-sdk/client-dynamodb";
import { PutCommand, DynamoDBDocumentClient } from "@aws-sdk/lib-dynamodb";
import { PresignedPostOptions, createPresignedPost } from "@aws-sdk/s3-presigned-post";
import { S3Client } from "@aws-sdk/client-s3";
import { ulid } from 'ulid'
import * as path from 'path';
import { Conditions } from "@aws-sdk/s3-presigned-post/dist-types/types";

const s3Client = new S3Client({ region: "us-east-1" });
const client = new DynamoDBClient({});
const documentClient = DynamoDBDocumentClient.from(client);

type InputArguments = {
  input: {
    filename: string
    contentType: string
  }
}

export const getImageUploadUrl = async (event: AppSyncResolverEvent<InputArguments> ) => {

  const id = ulid()
  const createdAt = new Date().toJSON()
  const identity = event.identity as AppSyncIdentityCognito

  const extension = path.extname(event.arguments.input.filename)
  const Bucket = process.env.ASSETS_BUCKET || ""
  const Key = `uploaded-images/${id}${extension}`

  const conditions: Conditions[] = [
    ["starts-with", "$Content-Type", "image/"],
    ["content-length-range", 1024, 10485760],
  ]
  const Fields = {
    "Content-Type": event.arguments.input.contentType
  };
  const presignedPostOptions: PresignedPostOptions = {
    Bucket,
    Key,
    Conditions: conditions,
    Fields,
    Expires: 600
  }
  const { url, fields } = await createPresignedPost(s3Client, presignedPostOptions);

  const newFields = Object.keys(fields).map(fieldName => ({name: fieldName, value: fields[fieldName]}))

  const data = {
    id,
    owner: identity.username,
    url,
    fields: newFields,
    key: Key,
    status: "waiting_upload",
    createdAt
  }

  const command = new PutCommand({
    TableName: process.env.IMAGES_TABLE,
    Item: data
  });

  await documentClient.send(command)

  const result = {
    id,
    url,
    fields: newFields
  }

  return result
}

We are using the library @aws-sdk/s3-presigned-post, from this library we import a function that returns structure of the data. The data is a struct with url and fields, these values are going to be needed in a frontend application, let say some like inside an input tag

<label htmlFor='image' >Upload image</label>
<input id="image"
      type="file"
      name="image"
      onChange={handleFileChange} />
</div>

A quick and dirty implementation of the handler, in a future post I’ll share the details of the front end application

const handleFileChange = async (event: React.ChangeEvent<HTMLInputElement> ) => {
     const selectedFile = event.target.files![0]
     const filename = selectedFile.name
     const contentType = selectedFile.type
     let response = await amplifyClient.graphql<GraphQLQuery<GetImageUploadUrlQuery>>({
       query: queries.getImageUploadUrl,
       variables: { input: {
         filename,
         contentType
       } },
     });
     const presignedImage = response.data?.getImageUploadUrl!
     const url = presignedImage.url;
     const imageFile = selectedFile;
     const formData = new FormData();
     presignedImage.fields.forEach((item) => {
       formData.append(item.name, item.value);
     });
     formData.append('file', imageFile);
     try {
      response = await axios.post(url!, formData, {
        headers: {'Content-Type': 'multipart/form-data'},
      });
    } catch (e) {
      console.log(e);
      alert("There was an error while uploading the image");
      return
    }
}

Create datasource and resolver

We first need to create a lambda data source using our graphql api and our get image upload url resources as parameters, once we create our data source we call the method createResolver, in this case that is a standard implementation for a lambda data source resolver for a query

const getImageUploadUrlDataSource = new appsync.LambdaDataSource(this, "GetImageUploadUrlDataSource", {
  api: graphqlApi,
  lambdaFunction: getImageUploadUrl
})

getImageUploadUrlDataSource.createResolver("GetImageUploadUrlResolver", {
  typeName: 'Query',
  fieldName: 'getImageUploadUrl',
  requestMappingTemplate: appsync.MappingTemplate.lambdaRequest(),
  responseMappingTemplate: appsync.MappingTemplate.lambdaResult()
})

Next Steps

Image moderation and image recognition

Create an AppSync API using Terraform

Luis Valdés — Sun, 10 Mar 2024 15:11:21 +0000

In this article we are going to dive into the details of creating a graphql api using AWS AppSync, the infrastructure is written using terraform.

The graphql api needs to provide two functionalities, a query to get tasks and a mutation to add tasks, the api needs to be authenticated and the data stored.

In the AppSync context we need to create two resolvers, one for getTasks and one for addTasks

Requirements

git
NodeJS 14 or later, my version is v18.18.0
An AWS account and configured credentials
An S3 bucket for the terraform backend
Install terraform cli

TL;DR;

Clone the repo and follow the instructions to deploy the project, you can use the gitpod configuration which comes with nodejs, aws cli v2, docker and terraform installed

Architecture

Here is an overview of the architecture.

Now let’s jump into the details of the different services we defined in our architecture.

AWS Identity and Access Management (IAM) is a web service that helps you securely control access to AWS resources. With IAM, you can centrally manage permissions that control which AWS resources users can access. You use IAM to control who is authenticated (signed in) and authorized (has permissions) to use resources.

Amazon Cognito delivers frictionless customer identity and access management (CIAM) with a cost-effective and customizable platform that allows you to implement secure, frictionless customer identity and access management that scales.

Amazon DynamoDB is a serverless, NoSQL, fully managed database service with single-digit millisecond response times at any scale, enabling you to develop and run modern applications while only paying for what you use.

AWS Lambda runs code without provisioning or managing servers, creating workload-aware cluster scaling logic, maintaining event integrations, or managing runtimes.

AWS AppSync simplifies application development with GraphQL APIs by providing a single endpoint to securely query or update data from multiple databases, microservices, and APIs.

Creating the tasks table

To store the data generated by the users of the api, we are going to use DynamoDB, here we are creating the tasks table with a key named id and an attribute named owner, we are also creating a global secondary index to be able to retrieve records by owner

resource "aws_dynamodb_table" "tasks_table" {
  name = "TasksTable"
  billing_mode = "PAY_PER_REQUEST"
  hash_key = "id"

  attribute {
    name = "id"
    type = "S"
  }

  attribute {
    name = "owner"
    type = "S"
  }

  global_secondary_index {
    name = "byOwner"
    hash_key = "owner"
    range_key = "id"
    projection_type = "ALL"
  }

}

Create a Cognito User Pool and User Pool Client

Since we are focusing on learning concepts, our cognito user pool configuration is not for production, for a production deployment you will need to modify the password policy

resource "aws_cognito_user_pool" "user_pool" {
  name = "UserPool"
  username_attributes = ["email"]
  auto_verified_attributes = ["email"]
  password_policy {
    minimum_length = 8
    require_lowercase = false
    require_uppercase = false
    require_numbers = false
    require_symbols = false
  }
  admin_create_user_config {
    allow_admin_create_user_only = false # enable self sign in
  }
}

resource "aws_cognito_user_pool_client" "user_pool_client" {
  name = "UserPoolClient"
  user_pool_id = aws_cognito_user_pool.user_pool.id
}

Creating the AppSync API

To create the graphql api resource we need a schema file, define a visibility, in our scenario is GLOBAL, then we need to set our authentication configuration, we are using cognito user pools and we provide the configuration for the user pool

resource "aws_appsync_graphql_api" "graphql_api" {
  name = "terraform-todos-api"

  schema = file("schema.graphql")

  visibility = "GLOBAL"

  authentication_type = "AMAZON_COGNITO_USER_POOLS"

  user_pool_config {
    default_action = "ALLOW"
    user_pool_id = aws_cognito_user_pool.user_pool.id
    app_id_client_regex = aws_cognito_user_pool_client.user_pool_client.id
  }

}

I the file schema.graphql, we define the type Task, the query getTasks and the mutation addTask, TaskConnection is a type we are using to iterate the result from getTasks

type Task {
  id: ID!
  text: String!
  owner: ID!
  createdAt: String!
}

type TaskConnection {
 items: [Task!]
 nextToken: String
}

input TaskInput {
  text: String!
}

type Query {
  getTasks(limit: Int, nextToken: String): TaskConnection!
}

type Mutation {
  addTask(input: TaskInput!): Task!
}

Create a policy that adds write access to Tasks table

The lambda function running the code for addTask needs the dynamodb::PutItem permission on the Tasks table, to be able to do that we are creating a policy with the permissions we need

data "aws_iam_policy_document" "lambda_put_item_policy" {
  statement {
    actions = ["dynamodb:PutItem"]
    resources = [
      aws_dynamodb_table.tasks_table.arn
    ]
  }
}

Create an IAM role for the lambda function

The IAM role for the lambda function needs some permissions for execution, the assume role policy allows lambda to assume this role, the IAM role also have a managed policy AWSLambdaBasicExecutionRole that provides the lambda function access to create logs. To finish the configuration of the role we add an inline policy, we load the policy document and provide a name for it

resource "aws_iam_role" "lambda_execution_role" {
  name = "lambdaExecutionRole"
  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Action = "sts:AssumeRole"
        Effect = "Allow"
        Principal = {
          Service = "lambda.amazonaws.com"
        }
      },
    ]
  })
  managed_policy_arns = ["arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"]

  inline_policy {
    name = "lambda_put_item_inline"
    policy = data.aws_iam_policy_document.lambda_put_item_policy.json
  }
}

Create the addTask lambda function

We are providing a name and a filename, the filename is used to load a packaged version of our function as a zip file.

We are using the IAM role described previously, our function will be able create a record on the Tasks table, we define the handler and the runtime of our function.

To finish our lambda function we add an environment variable with the name of the Tasks table

resource "aws_lambda_function" "add_task_lambda_function" {
  function_name = "addTaskLambdaFunction"
  filename = "add_task_lambda_function.zip"
  role = aws_iam_role.lambda_execution_role.arn
  handler = "index.addTask"
  runtime = "nodejs20.x"
  timeout = 30
  environment {
    variables = {
      TASKS_TABLE = aws_dynamodb_table.tasks_table.name
    }
  }
}

The source code of the lambda function for addTask uses DynamoDB to store a new item, the id of the item is generated using the ulid library, ULID stands for U niversally unique L exicographically sortable ID entifiers, the values are sortable even when generated within a millisecond, this is an example

ulid() // 01HR8FDWSNDC63YF1HBHCB1QBF

The field id is an ulid generated value, and it is sortable.

The value of field owner will have the username field of the cognito identity parameter, and our last parameter is the text that comes from the arguments.

The typescript implementation needs the type InputArguments with the structure needed to be used by AppSyncResolverEvent , by configuring this the application is able to call event.arguments.input.text to obtain the text input of an invocation

import { AppSyncResolverEvent , AppSyncIdentityCognito} from "aws-lambda";
import { DynamoDBClient } from "@aws-sdk/client-dynamodb";
import { PutCommand, DynamoDBDocumentClient } from "@aws-sdk/lib-dynamodb";
import { ulid } from 'ulid'

const client = new DynamoDBClient({});
const documentClient = DynamoDBDocumentClient.from(client);

type InputArguments = {
  input: {
    text: String
  }
}

export const addTask = async (event: AppSyncResolverEvent<InputArguments> ) => {

  const id = ulid()
  const createdAt = new Date().toJSON()
  const identity = event.identity as AppSyncIdentityCognito

  const data = {
    id,
    owner: identity.username,
    text: event.arguments.input.text,
    createdAt
  }
  const command = new PutCommand({
    TableName: process.env.TASKS_TABLE,
    Item: data
  });

  await documentClient.send(command)

  return data
}

Configuring the IAM role for the AppSync data sources

The data source needs to have Query permissions on the global secondary index called byOwner

data "aws_iam_policy_document" "appsync_query_index_inline_policy" {
  statement {
    actions = ["dynamodb:Query"]
    resources = [
      "${aws_dynamodb_table.tasks_table.arn}/index/byOwner"
    ]
  }
}

For the sake of keeping this deployment simple, we are going to use the same IAM role to call our add task lambda function, in order to do it we need to add InvokeFunction permission

data "aws_iam_policy_document" "appsync_invoke_lambda_inline_policy" {
  statement {
    actions = ["lambda:InvokeFunction"]
    resources = [
      aws_lambda_function.add_task_lambda_function.arn
    ]
  }
}

Now we have two policies, one to query the index byOwner and another to invoke the lambda function for addTask, we are going to use these policies in our role for appsync data sources

Creating the IAM role for the appsync data sources

The role needs to be assumed by the appsync service and as we explained previously, we need our role to be able to query an index, and invoke a lambda function, we add two inline policies, use the query index and invoke lambda policies we created before

resource "aws_iam_role" "appsync_datasource_role" {
  name = "appsync_datasource_role"
  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Action = "sts:AssumeRole"
        Effect = "Allow"
        Principal = {
          Service = "appsync.amazonaws.com"
        }
      },
    ]
  })
  inline_policy {
    name = "appsync_query_index_inline"
    policy = data.aws_iam_policy_document.appsync_query_index_inline_policy.json
  }
  inline_policy {
    name = "appsync_invoke_lambda_inline"
    policy = data.aws_iam_policy_document.appsync_invoke_lambda_inline_policy.json
  }
}

Creating the tasks table data source

We create the tasks table datasource using our graphql api, in this scenario it is of type AMAZON_DYNAMODB, so we are going to use a table as data source, we provide the dynamodb_config configuration with the name of the table from our previously created Tasks table and the region

resource "aws_appsync_datasource" "tasks_table_datasource" {
  api_id = aws_appsync_graphql_api.graphql_api.id
  name = "TasksTableDataSource"
  type = "AMAZON_DYNAMODB"
  service_role_arn = aws_iam_role.appsync_datasource_role.arn
  dynamodb_config {
    table_name = aws_dynamodb_table.tasks_table.name
    region = var.region
  }
}

Creating the getTasks resolver using JavaScript

After creating our tasks data source, we are now ready to create our JavaScript resolver, we do this by creating an appsync resolver, using our graphql api, with type of Query and field equals to getTasks.

The runtime of the resolver is APPSYNC_JS, we provide a path to the code and a data source

resource "aws_appsync_resolver" "get_tasks_resolver" {
  api_id = aws_appsync_graphql_api.graphql_api.id
  type = "Query"
  field = "getTasks"
  runtime {
    name = "APPSYNC_JS"
    runtime_version = "1.0.0"
  }
  code = file("resolvers/getTasks.js")
  data_source = aws_appsync_datasource.tasks_table_datasource.name
}

The resolver getTask.js exports two functions, the request function returns an operation Query on the index byOwner, ordered in ascending order by the value of the id field, the value for the owner variable is obtained from the current authenticated user making the request, the value is located in the context variable of the function

import { util } from '@aws-appsync/utils';

export function request(context) {
  return {
      operation: "Query",
      query: {
          expression: "#owner = :userId",
          expressionNames: {
            "#owner": "owner"
          },
          expressionValues: {
              ":userId": util.dynamodb.toDynamoDB(context.identity.username)
          }
      },
      index: "byOwner",
      nextToken: context.arguments.nextToken,
      limit: context.arguments.limit,
      scanIndexForward: true, // true order ASC, false order DESC
      consistentRead: false,
      select: "ALL_ATTRIBUTES"
  }
}

export function response(context) {
  return {
    items: context.result.items,
    nextToken: context.result.nextToken
  }
}

The response function returns a structure of items and pagination.

Creating the data source for the addTask lambda function

We are close to the end of our application, we need to add the add task data source, which in this scenario is of type AWS_LAMBDA.

The service role for the data source is the appsync_datasource_role, which allows AppSync to invoke the lambda function add task

resource "aws_appsync_datasource" "add_task_datasource" {
  api_id = aws_appsync_graphql_api.graphql_api.id
  name = "AddTaskDataSource"
  type = "AWS_LAMBDA"
  service_role_arn = aws_iam_role.appsync_datasource_role.arn
  lambda_config {
    function_arn = aws_lambda_function.add_task_lambda_function.arn
  }
}

The last part of the puzzle is to add the resolver for our mutation addTask

resource "aws_appsync_resolver" "add_task_resolver" {
  api_id = aws_appsync_graphql_api.graphql_api.id
  type = "Mutation"
  field = "addTask"
  data_source = aws_appsync_datasource.add_task_datasource.name
}

Deploy the project

We need the name of the previously created bucket, to use it as parameter for the terraform init command

terraform init -backend-config="bucket=<YOUR_BACKEND_S3_BUCKET_NAME>"

Then we need to run to see the details of the resources to te created

terraform plan

To make changes and create the resources we run

terraform apply

How to test the project

Open your AWS console, the first thing we need to is to create an user in our user pool, to do that we are going to the search bar and search for cognito

Click on the User Pool we created using CDK

Once in the details page of the user pool, click on create user, fill the user email and mark as verified, last fill a password for the user, you will need this credentials later on.

Now open the search bar and search for appsync

On the list, select the todos-api and once on the details page for the api, on the left bar, go to the menu Queries and click in Login with User Pools

Select the client ID that is available and login with the user created in the previous step.

We want to create a task, in the explorer select query and change for mutation then click on the + plus button, choose the addTask and click on the input text, in the example text is Hello World then click on Run MyMutation

When you run the mutation you will have a result like this one

Now go to the explorer and change the mutation to query and click on + plus, click on getTasks and click on all fields, then run MyQuery

Creating a Cognito Trigger using CDK and TypeScript

Luis Valdés — Mon, 04 Mar 2024 21:49:44 +0000

In the current scenario we want our users to be able to sign up for an account, and when the user confirms its account, in the post confirmation event from Cognito, we want to run a lambda function to add the record from the user to a DynamoDB table

Requirements

git
NodeJS 14 or later, my version is v18.18.0
An AWS account and configured credentials
Install cdk command
docker

TL;DR;

Clone the repo and follow the instructions to deploy the project, you can use the gitpod configuration which comes with nodejs, aws cli v2, docker and cdk installed

Create the Users Table

In this snippet we are creating a table with a partition key with the name of id and type string, then we are granting read and write data permission to a role called lambdaRole

const usersTable = new dynamodb.Table(this, 'UsersTable', {
  partitionKey: {
    name: 'id',
    type: dynamodb.AttributeType.STRING,
  },
});
usersTable.grantReadWriteData(lambdaRole)

Create post confirmation lambda function

We fill the parameters with their respective values, notice that we are passing the DynamoDB table name of usersTable as an environment variable, we are going to use this environment variable in the lambda code, in the last part of the snippet we se that we call the addTrigger method of the userPool, we pass as parameter the type of operation and a lambda function construct

const postConfirmation = new lambda_nodejs.NodejsFunction(this, "PostConfirmationLambdaFunction", { 
  entry: path.join(__dirname, '../functions/postConfirmation/index.ts'),
  handler: 'postConfirmation',
  runtime: lambda.Runtime.NODEJS_20_X,
  environment: {
    'USERS_TABLE': usersTable.tableName
  },
  role: lambdaRole,
  timeout: cdk.Duration.seconds(30)
})

userPool.addTrigger(cognito.UserPoolOperation.POST_CONFIRMATION, 
  postConfirmation
);

Lets take a look at the code that runs inside the lambda function, we import the type of event from the aws-lambda module, and then we create a PutCommand using the enviroment variable USERS_TABLE

import { PostConfirmationTriggerEvent } from "aws-lambda";
import { DynamoDBClient } from "@aws-sdk/client-dynamodb";
import { PutCommand, DynamoDBDocumentClient } from "@aws-sdk/lib-dynamodb";

const client = new DynamoDBClient({});
const documentClient = DynamoDBDocumentClient.from(client);

export const postConfirmation = async (event: PostConfirmationTriggerEvent ) => {

  const createdAt = new Date().toJSON()
  const { userAttributes } = event.request

  const data = {
    id: userAttributes['sub'],
    email: userAttributes['email'],
    createdAt
  }

  const command = new PutCommand({
    TableName: process.env.USERS_TABLE,
    Item: data
  });

  await documentClient.send(command);

  return event
}

Conclusion

We created a DynamoDB table, granted permission to an iam role, then created a lambda function and added a trigger to the user pool, in future posts we are going to use our users table

Using CDK and Typescript to create an AppSync GraphQL API

Luis Valdés — Mon, 04 Mar 2024 14:41:11 +0000

In our current scenario we need to provide a GraphQL API that allows users to create new tasks and get the list of tasks they created.

We have two functionalities in our api, addTask and getTasks, both of them need an authenticated user, also it is important to mention that we need to store the data in a database.

In order to solve our current scenario we are going to use AWS IAM, Amazon Cognito, Amazon DynamoDB, AWS Lambda and AWS AppSync, to create our infrastructure we are using CDK (Cloud Development Kit) and TypeScript

Requirements

git
NodeJS 14 or later, my version is v18.18.0
An AWS account and configured credentials
Install cdk command
docker

TL;DR;

Clone the repo and follow the instructions to deploy the project, you can use the gitpod configuration which comes with nodejs, aws cli v2, docker and cdk installed

Architecture

Here is an overview of the architecture.

Now let’s jump into the details of the different services we defined in our architecture.

AWS Lambda runs code without provisioning or managing servers, creating workload-aware cluster scaling logic, maintaining event integrations, or managing runtimes.

AWS AppSync simplifies application development with GraphQL APIs by providing a single endpoint to securely query or update data from multiple databases, microservices, and APIs.

Now that we have the concepts for the services built up we can start to take a look at the details of the implementation of the infrastructure using CDK (Cloud Development Kit)

Creating a DynamoDB table with a global secondary index

We want to create a table to store the data from the Tasks, so we will create such table using the CfnTable construct from module dynamodb.

On the keySchema will will only have the attribute id as type of hash, also there is attribute definitions where we have id as string and owner as string. The billing mode is Pay per Request.

The global secondary index name is byOwner, where the key schema has owner as hash and id as range as key type, this means that we will be able to make a query on the index byOwner using the field owner to retrieve the results sorted by the field id, another thing to point out is that the projectionType is ALL, which will retrieve all fields from each record

const cfnTaskTable = new dynamodb.CfnTable(this, 'CfnTasksTable', {
    keySchema: [{
      attributeName: 'id',
      keyType: 'HASH',
    }],
    attributeDefinitions: [
      {
        attributeName: 'id',
        attributeType: 'S',
      },
      {
        attributeName: 'owner',
        attributeType: 'S',
      }
    ],
    billingMode: 'PAY_PER_REQUEST',
    globalSecondaryIndexes: [
      {
        indexName: 'byOwner',
        keySchema: [
          {
            attributeName: 'owner',
            keyType: 'HASH',
          },
          {
            attributeName: 'id',
            keyType: 'RANGE',
          }
        ],
        projection: {
          projectionType: 'ALL',
        },
      }
    ],
  });

const taskTable = dynamodb.Table.fromTableArn(this, 'TasksTable', cfnTaskTable.attrArn)

We need the table name which will be generated at deployment time, while I was building this project, the table name returned by CfnTable was null, but it returned a valid when calling attrArn, so by having the arn of the table is possible to retrieve its the construct of the table by calling dynamodb.Table.fromTableArn and store it in variable taskTable.

Creating a user pool and user pool client

Inside the Amazon Cognito service we are going to create a resource called user pool, which is a user directory for web and mobile app authentication and authorization.

Another resource that is created is a user pool client which is a configuration within a user pool that interacts with mobile or web application that authenticates with Amazon Cognito.

A web application will use the user pool client id to interact with the cognito user pool.

Another detail to point out is that our UserPool configuration for password policy is weak, only for development purposes.

const userPool = new cognito.UserPool(this, 'UserPool',
  {
    signInAliases: {
      email: true ,
      username: false
    },
    autoVerify: {
        email: true
    },
    passwordPolicy: {
      minLength: 8,
      requireLowercase: false,
      requireUppercase: false,
      requireDigits: false,
      requireSymbols: false,
    },
  }
);

const userPoolClient = userPool.addClient('UserPoolClient', {
  userPoolClientName: 'web'
})

Creating the GraphQL API

In order to create our graphql api we will use the GraphqlApi construct from the appsync module, there are many configurations for it but in our current scenario we will only provide a name, a schema and an authorizationConfig where we are going to use the User Pool authorization tipe and provide the user pool we created before as a param for the userPoolConfig

const graphqlApi = new appsync.GraphqlApi(this, 'AppSyncApi', {
  name: 'todos-api',
  definition: appsync.Definition.fromFile(path.join(__dirname, '../schema.graphql')),
  authorizationConfig: {
    defaultAuthorization: {
      authorizationType: appsync.AuthorizationType.USER_POOL,
      userPoolConfig: {
        userPool: userPool
      }
    }
  }
});

The schema of our graphql api is loaded in the definition parameter, this is what it looks like, we have one type, one input that is used in a mutation, we have the Query getTasks and the Mutation addTask, we will see their resolvers later on

type Task {
  id: ID!
  text: String!
  owner: ID!
  createdAt: String!
}

type TaskConnection {
 items: [Task!]
 nextToken: String
}

input TaskInput {
  text: String!
}

type Query {
  getTasks(limit: Int, nextToken: String): TaskConnection!
}

type Mutation {
  addTask(input: TaskInput!): Task!
}

Lambda execution role

We are going to create a lambda function that will write data to a dynamodb table, in order for the function to be able to put an item in the table, it needs permissions, we provide this permission to the function using an IAM Role, our lambda role has the permission to be assumed by the lambda service, also has a managed policy AWSLambdaBasicExecutionRole.

Last thing we have is the taskTable grating access to read and write to the lambda role

const lambdaRole = new iam.Role(this, 'LambdaExecutionRole', {
  assumedBy: new iam.ServicePrincipal('lambda.amazonaws.com'),
  managedPolicies: [
    iam.ManagedPolicy.fromAwsManagedPolicyName('service-role/AWSLambdaBasicExecutionRole')
  ]
})
taskTable.grantReadWriteData(lambdaRole)

Task data source role

The task data source role will be assumed by appync service to run the JavaScript resolver code, the code uses an index called byOwner, to give access to this index we create a policy.

We use taskTable to grant read and write permissions to the task datasource role and attach an inline policy for the index policy.

const tasksDataSourceRole = new iam.Role(this, 'TasksDataSourceRole', {
  assumedBy: new iam.ServicePrincipal('appsync.amazonaws.com')
})
taskTable.grantReadWriteData(tasksDataSourceRole)

const queryIndexPolicy = new iam.Policy(this, "QueryIndexByOwnerPolicy", {
  policyName: "QueryIndexByOwnerPolicy",
  statements: [
    iam.PolicyStatement.fromJson({
        Effect: 'Allow',
        Action: 'dynamodb:Query',
        Resource: `${taskTable.tableArn}/index/byOwner`
    })
  ]
})
tasksDataSourceRole.attachInlinePolicy(queryIndexPolicy)

Create a dynamodb datasource

To create our dynamodb data source we are going to use the taskTable, the taskDataSourceRole we created earlier and our graphqlApi

const taskTableDataSource = new appsync.DynamoDbDataSource(this, 'TaskTableDataSource', {
  table: taskTable,
  serviceRole: tasksDataSourceRole,
  api: graphqlApi
})

Create a JavaScript resolver

We are going to create a resolver using our taskTableDataSource, this resolver has for fieldName 'getTasks', typename 'Query', the code is loaded from a local filesystem and the runtime is JS_1_0_0

taskTableDataSource.createResolver('getTasksResolver', {
  fieldName: 'getTasks',
  typeName: 'Query',
  code: appsync.Code.fromAsset(path.join(__dirname, '../resolvers/getTasks.js')),
  runtime: appsync.FunctionRuntime.JS_1_0_0
})

Before 17 November 2022 the only way to write resolvers was using Apache Velocity Template Language (VTL), while writing mapping templates for resolvers I learned it requires more effort when debugging, having JavaScript opens more possibilities.

Now lets take a look at our JavaScript resolver, the value that is returned follows a syntax from DynamoDB resolver, the Query request mapping document lets you tell the AWS AppSync DynamoDB resolver to make a Query request to DynamoDB, one thing to point out is that we are using the byOwner index that we created with our dynamodb table tasksTable, this query will return the tasks created by the current authenticated user, the field :userId will have the value of context.identity.username, which is the id of the user in the Cognito User Pool

import { util } from '@aws-appsync/utils';

export function request(context) {
  return {
      operation: "Query",
      query: {
          expression: "#owner = :userId",
          expressionNames: {
            "#owner": "owner"
          },
          expressionValues: {
              ":userId": util.dynamodb.toDynamoDB(context.identity.username)
          }
      },
      index: "byOwner",
      nextToken: context.arguments.nextToken,
      limit: context.arguments.limit,
      scanIndexForward: true, // true order ASC, false order DESC
      consistentRead: false,
      select: "ALL_ATTRIBUTES"
  }
}

export function response(context) {
  return {
    items: context.result.items,
    nextToken: context.result.nextToken
  }
}

When the request resolver is called, in the cloudwatch logs of the resolver it is possible to see the field :userId with a value:

Create a NodeJS Function

To create our addTask function we are going to use the NodejsFunction construct where we are providing a path to the source code to the parameter entry, inside bundling we are adding ulid module as dependency, among other parameters we have the handler which is the name of the exported function that will be called, in the environment we are passing TASK_TABLE from the taskTable construct using its tableName attribute, this environment variable is used inside the code of the function, other important parameter is the role, we are using the lambdaRole we creater previously, and finally we add a timeout of 30 seconds which is the maximum amount of time a function can run in AppSync.

const addTask = new lambda_nodejs.NodejsFunction(this, "AddTaskLambdaFunction", { 
  entry: path.join(__dirname, '../functions/addTask/index.ts'),
  bundling: {
    nodeModules: ['ulid'],
  },
  projectRoot: path.join(__dirname, '../functions/addTask'),
  depsLockFilePath: path.join(__dirname, '../functions/addTask/package-lock.json'),
  handler: 'addTask',
  runtime: lambda.Runtime.NODEJS_20_X,
  environment: {
    'TASKS_TABLE': taskTable.tableName
  },
  role: lambdaRole,
  timeout: cdk.Duration.seconds(30)
})

The source code for the function is, we use aws sdk javascript v3 libraries that are part of the lambda runtime, the library we added is ulid, ULID is a universally unique lexicographically sortable identifier, it generates a string that is random and can be sorted, different from uuids

import { AppSyncResolverEvent , AppSyncIdentityCognito} from "aws-lambda";
import { DynamoDBClient } from "@aws-sdk/client-dynamodb";
import { PutCommand, DynamoDBDocumentClient } from "@aws-sdk/lib-dynamodb";
import { ulid }  from 'ulid'

const client = new DynamoDBClient({});
const documentClient = DynamoDBDocumentClient.from(client);

type InputArguments = {
  input: {
    text: String
  }
}

export const addTask = async (event: AppSyncResolverEvent<InputArguments> ) => {

  const id  = ulid()
  const createdAt = new Date().toJSON()
  const identity = event.identity as AppSyncIdentityCognito

  const data = {
    id,
    owner: identity.username,
    text: event.arguments.input.text,
    createdAt
  }
  const command = new PutCommand({
    TableName: process.env.TASKS_TABLE,
    Item: data
  });

  const response = await documentClient.send(command);
  console.log(response)

  return data
}

Lambda Data Source

To be able to user our lambda function addTask, we need to create a data source using the construct LambdaDataSource and provide graphqlApi as api, and addTask as lambdaFunction parameters.

const lambdaDataSource = new appsync.LambdaDataSource(this, "AddTaskDataSource", {
  api: graphqlApi,
  lambdaFunction: addTask
})

Create resolver for lambda data source

Then we use our lambda data source to create a resolver, which in this case is of typeName Mutation with the fieldName "addTask", the mapping templates are the default ones for lambda data sources.

lambdaDataSource.createResolver("AddTaskResolver", {
  typeName: 'Mutation',
  fieldName: 'addTask',
  requestMappingTemplate: appsync.MappingTemplate.lambdaRequest(),
  responseMappingTemplate: appsync.MappingTemplate.lambdaResult()
})

Deploying the project

Clone the repo and follow the instructions to the deploy the project.

How to test the project

Open your AWS console, the first thing we need to is to create an user in our user pool, to do that we are going to the search bar and search for cognito

Click on the User Pool we created using CDK

Once in the details page of the user pool, click on create user, fill the user email and mark as verified, last fill a password for the user, you will need this credentials later on.

Now open the search bar and search for appsync

On the list, select the todos-api and once on the details page for the api, on the left bar, go to the menu Queries and click in Login with User Pools

Select the client ID that is available and login with the user created in the previous step.

When you run the mutation you will have a result like this one

Now go to the explorer and change the mutation to query and click on + plus, click on getTasks and click on all fields, then run MyQuery

Conclusion and next steps

This introduction to AWS AppSync and how to create resolvers for queries and mutations will enable you to extend the project with your own ideas in case you need a solution as the one provided in this article.

The next steps will be to add a frontend application, which I will do it in a future publication.