DEV Community: Binh Bui

Validating Request Parameters and Body in Amazon API Gateway with AWS CDK

Binh Bui — Tue, 25 Apr 2023 04:14:57 +0000

Note: The code for this post is available on GitHub

Amazon API Gateway provides a powerful way to create REST APIs that can be integrated with other AWS services. However, it's important to validate incoming requests to ensure that they are in the correct format and contain the necessary parameters. This can help to prevent errors and security vulnerabilities.

What is Request Validation?

Request validation is the process of checking that a request contains all the necessary parameters and that they are in the correct format. This can help to prevent errors and security vulnerabilities. For example, if you have an API endpoint that accepts a JSON body with a username and password, you might want to validate that the username is a valid email address and that the password is at least 8 characters long.

How to Validate Request Parameters

Amazon API Gateway allows you to validate request path and query parameters using request validation rules. Request validation helps ensure that incoming requests to your API conform to the expected format, reducing the risk of errors and vulnerabilities.

To validate request path and query parameters in Amazon API Gateway using AWS CDK:

Create a new REST API using the RestApi construct.

const restApi = new apigateway.RestApi(this, 'MyApi', {
  restApiName: 'My API',
  description: 'My sample API',
})

Define a resource and method for your API using the addResource and addMethod methods. Specify a Lambda integration for the method using the LambdaIntegration class.

const getResource = restApi.root.addResource('resource')
const getMethod = getResource.addMethod('GET', new apigateway.LambdaIntegration(myLambdaFunction))

Define the request parameters you want to validate using the requestParameters property. For example, to require a path parameter called id and a query parameter called name, set the requestParameters property like this:

getMethod.addMethod('GET', new apigateway.LambdaIntegration(myLambdaFunction), {
  requestParameters: {
    'method.request.path.id': true,
    'method.request.querystring.name': true,
  },
})

Enable request validation for the method by setting the requestValidatorOptions.validateRequestParametersproperty to true.

getMethod.addMethod('GET', new apigateway.LambdaIntegration(myLambdaFunction), {
  requestParameters: {
    'method.request.path.id': true,
    'method.request.querystring.name': true,
  },
  requestValidatorOptions: {
    validateRequestParameters: true,
  },
})

If you call the API with a request that contains an invalid path parameter, API Gateway returns a 400 Bad Request response with an error message that describes the validation errors.

{
  "message": "Missing required request parameters: [name, id]"
}

That's it! With these steps, you can validate request path and query parameters in Amazon API Gateway using AWS CDK. By validating requests, you can ensure that your API only receives requests that conform to the expected format, reducing the risk of errors and vulnerabilities.

How to Validate Request Body

To validate the request body in Amazon API Gateway, you can use a JSON schema. A JSON schema is a JSON document that defines the structure of a JSON object. You can use JSON schemas to validate the structure of a JSON object, and to validate the values of the object's properties.

To validate the request body in Amazon API Gateway using AWS CDK:

Create a new REST API using the RestApi construct.

const restApi = new apigateway.RestApi(this, 'MyApi', {
  restApiName: 'My API',
  description: 'My sample API',
})

Define a resource and method for your API using the addResource and addMethod methods. Specify a Lambda integration for the method using the LambdaIntegration class.

const getResource = restApi.root.addResource('resource')
const postMethod = getResource.addMethod('POST', new apigateway.LambdaIntegration(myLambdaFunction))

Define the request body schema using the requestModels property. For example, to require a JSON request body with a name field, set the requestModels property like this:

const requestBodySchema = new apigateway.Model(this, 'RequestBodySchema', {
  restApi: restApi,
  contentType: 'application/json',
  schema: {
    type: apigateway.JsonSchemaType.OBJECT,
    properties: {
      name: { type: apigateway.JsonSchemaType.STRING },
    },
    required: ['name'],
  },
})

postMethod.addMethod('POST', new apigateway.LambdaIntegration(myLambdaFunction), {
  requestModels: {
    'application/json': requestBodySchema,
  },
})

Enable request validation for the method by setting the requestValidatorOptions.validateRequestBody and requestValidatorOptions.validateRequestParameters properties to true.

postMethod.addMethod('POST', new apigateway.LambdaIntegration(myLambdaFunction), {
  requestParameters: {
    'method.request.path.id': true,
    'method.request.querystring.name': true,
  },
  requestModels: {
    'application/json': requestBodySchema,
  },
  requestValidatorOptions: {
    validateRequestBody: true,
    validateRequestParameters: true,
  },
})

If you call the API with a request that contains an invalid request body, API Gateway returns a 400 Bad Request response with an error message that describes the validation errors.

{
  "message": "Invalid request body"
}

That's it! With these steps, you can validate request path and query parameters, as well as the request body, in Amazon API Gateway using AWS CDK. By validating requests, you can ensure that your API only receives requests that conform to the expected format, reducing the risk of errors and vulnerabilities.

Troubleshooting

How to Fix "There is already a Construct with name 'validator' in RestApi"

If you get the following error when trying to add a request validator to an API Gateway resource:

There is already a Construct with name 'validator' in RestApi

If you get this error, it means that you already have a request validator defined for the resource. You can only have one request validator per resource, so you need create specify a different name for the request validator.

To fix this error, specify a different name for the request validator. For example:

const bodyValidator = new apigateway.RequestValidator(this, 'BodyValidator', {
  restApi: restApi,
  requestValidatorName: 'BodyValidator',
  validateRequestBody: true,
  validateRequestParameters: false,
})
resource.addMethod('POST', new apigateway.LambdaIntegration(myLambdaFunction), {
  requestValidator: bodyValidator,
  requestModels: {
    'application/json': requestBodySchema,
  },
})

Conclusion

In conclusion, validating incoming requests is an important part of creating a secure and reliable API. Amazon API Gateway provides several options for validating request parameters and request body. By using AWS CDK to define your API infrastructure, you can automate the process of creating and managing your API, including the request validation process.

In this blog post, we explored how to define request parameters and request body schema in Amazon API Gateway using AWS CDK. We also demonstrated how to enable request validation for a method using the requestValidatorOptions property. By following these steps, you can create a more secure and reliable API that is less prone to errors and vulnerabilities.

Overall, using AWS CDK to define your API infrastructure provides a powerful and flexible way to manage your API. By incorporating request validation into your API design, you can ensure that your API only receives valid requests, which can improve the reliability and security of your application.

References

Thanks for reading! If you have any questions or comments, please send me a note on Twitter. I'd love to hear from you!

Connecting AWS Lambda with Amazon RDS using AWS CDK and Node.js

Binh Bui — Fri, 10 Mar 2023 05:54:25 +0000

Photo by [mollyblackbird](https://unsplash.com/@mollyblackbird)

AWS Lambda is a popular serverless compute service that lets you run code in response to events without having to manage the underlying infrastructure. Amazon RDS is a managed relational database service that makes it easy to set up, operate, and scale a relational database in the cloud. In this tutorial, we will explore how to connect an AWS Lambda function to an RDS instance to perform database operations.

Key Points

Here are the key points to keep in mind when connecting an AWS Lambda function to an RDS instance:

Accessing RDS over the Internet or VPC

An RDS instance can be accessed over the Internet using a public IP address or DNS name, or over a VPC using a private IP address.
If your RDS instance is publicly accessible, you can use the public IP address or DNS name to connect to it. If your Lambda function is running inside a VPC, you can use a private IP address to connect to the RDS instance. Here's an example of how to create an RDS instance that is publicly accessible:

const rds = new aws.RDS({
  engine: 'mysql',
  instanceClass: 'db.t3.micro',
  allocatedStorage: 10,
  dbName: 'my_database',
  masterUsername: 'admin',
  masterUserPassword: 'my_password',
  publiclyAccessible: true,
  vpcSecurityGroupIds: [mySecurityGroup.securityGroupId],
})

Here, we're creating an RDS instance using the aws-sdk in Node.js, and setting the publiclyAccessible property to true to allow connections from the Internet. We also specify a security group ID to allow inbound traffic from our Lambda function's security group.

Connecting Lambda to RDS inside VPC

If your Lambda function is running inside a VPC, you can use a private IP address to connect to the RDS instance. Here's an example of how to create a Lambda function inside a VPC:

const myVpc = new aws.ec2.Vpc('my-vpc', {
  cidrBlock: '10.0.0.0/16',
  tags: {
    Name: 'My VPC',
  },
})

const mySubnet = new aws.ec2.Subnet('my-subnet', {
  cidrBlock: '10.0.0.0/24',
  vpcId: myVpc.id,
  availabilityZone: 'us-east-1a',
  tags: {
    Name: 'My Subnet',
  },
})

const mySecurityGroup = new aws.ec2.SecurityGroup('my-security-group', {
  vpcId: myVpc.id,
  ingress: [
    {
      protocol: 'tcp',
      fromPort: 3306,
      toPort: 3306,
      cidrBlocks: [myVpc.vpcCidrBlock],
    },
  ],
  tags: {
    Name: 'My Security Group',
  },
})

const myLambdaFunction = new aws.lambda.Function('my-lambda-function', {
  runtime: 'nodejs14.x',
  code: new aws.lambda.AssetCode('path/to/lambda/code'),
  handler: 'index.handler',
  vpcConfig: {
    subnetIds: [mySubnet.id],
    securityGroupIds: [mySecurityGroup.id],
  },
})

Here, we're creating a VPC with a subnet and a security group that allows inbound traffic on port 3306 (for MySQL connections). We're then creating a Lambda function that is associated with the VPC and the security group.

Making RDS Publicly Accessible

If your Lambda function is not running inside a VPC, the RDS instance must be publicly accessible or located in a public subnet that has a route to the Internet. Here's an example of how to create an RDS instance in a public subnet:

const myVpc = new aws.ec2.Vpc('my-vpc', {
  cidrBlock: '10.0.0.0/16',
  tags: {
    Name: 'My VPC',
  },
})

const myPublicSubnet = new aws.ec2.Subnet('my-public-subnet', {
  cidrBlock: '10.0.1.0/24',
  vpcId: myVpc.id,
  mapPublicIpOnLaunch: true,
  availabilityZone: 'us-east-1a',
  tags: {
    Name: 'My Public Subnet',
  },
})

const mySecurityGroup = new aws.ec2.SecurityGroup('my-security-group', {
  vpcId: myVpc.id,
  ingress: [
    {
      protocol: 'tcp',
      fromPort: 3306,
      toPort: 3306,
      cidrBlocks: [myVpc.vpcCidrBlock],
    },
  ],
  tags: {
    Name: 'My Security Group',
  },
})

const rdsInstance = new aws.rds.Instance('my-rds-instance', {
  engine: 'mysql',
  instanceClass: 'db.t3.micro',
  allocatedStorage: 10,
  dbName: 'my_database',
  masterUsername: 'admin',
  masterUserPassword: 'my_password',
  vpcSecurityGroupIds: [mySecurityGroup.securityGroupId],
  subnetIds: [myPublicSubnet.id],
})

Here, we're creating a VPC with a public subnet that has a route to the Internet, and a security group that allows inbound traffic on port 3306. We're then creating an RDS instance that is associated with the VPC, the security group, and the public subnet.

Using IAM Role to Connect to RDS

Instead of hard-coding the database credentials in your Lambda function, you can use an IAM role to provide temporary credentials to access the RDS instance. Here's an example of how to create an IAM role for your Lambda function to access the RDS instance:

const rdsRole = new aws.iam.Role('rds-lambda-role', {
  assumeRolePolicy: JSON.stringify({
    Version: '2012-10-17',
    Statement: [
      {
        Effect: 'Allow',
        Principal: {
          Service: 'lambda.amazonaws.com',
        },
        Action: 'sts:AssumeRole',
      },
    ],
  }),
})

const rdsPolicy = new aws.iam.Policy('rds-lambda-policy', {
  policy: JSON.stringify({
    Version: '2012-10-17',
    Statement: [
      {
        Effect: 'Allow',
        Action: ['rds-db:connect'],
        Resource: [rdsInstance.arn],
      },
    ],
  }),
})

new aws.iam.RolePolicyAttachment('rds-lambda-role-policy', {
  policyArn: rdsPolicy.arn,
  role: rdsRole.name,
})

const myLambdaFunction = new aws.lambda.Function('my-lambda-function', {
  runtime: 'nodejs14.x',
  code: new aws.lambda.AssetCode('path/to/lambda/code'),
  handler: 'index.handler',
  role: rdsRole.arn,
})

Here, we're creating an IAM role that allows the Lambda function to assume the role and connect to the RDS instance. We're also attaching a policy to the role that allows the Lambda function to connect to the RDS instance using the rds-db:connect action. Finally, we're creating a Lambda function that is associated with the IAM role.

Sample Lambda Function Code

Here's an example of how to write a Lambda function in Node.js that connects to an RDS instance:

const AWS = require('aws-sdk')
const mysql = require('mysql')

exports.handler = async (event) => {
  // Read environment variables
  const dbHost = process.env.DB_HOST
  const dbUser = process.env.DB_USER
  const dbPassword = process.env.DB_PASSWORD
  const dbName = process.env.DB_NAME

  // Create a MySQL connection pool
  const pool = mysql.createPool({
    host: dbHost,
    user: dbUser,
    password: dbPassword,
    database: dbName,
  })

  // Perform a SQL query using the connection pool
  const query = 'SELECT * FROM my_table'
  const results = await new Promise((resolve, reject) => {
    pool.query(query, (error, results) => {
      if (error) {
        reject(error)
      } else {
        resolve(results)
      }
    })
  })

  // Format the results as a JSON object
  const response = {
    statusCode: 200,
    headers: {
      'Content-Type': 'application/json',
    },
    body: JSON.stringify(results),
  }

  return response
}

This Lambda function reads the RDS connection details from environment variables and uses them to create a MySQL connection pool. It then performs a SQL query and formats the results as a JSON object that is returned as the response.

Conclusion

We've also provided code samples for creating a VPC and security group using the AWS CDK and for writing a Lambda function that connects to an RDS instance using the mysql2 library.

Remember that while VPCs provide an added layer of security, they also add complexity to your architecture. Make sure to carefully design and test your VPC and security group configurations to ensure that they meet your specific needs.

Finally, it's important to regularly monitor and review your VPC and security group settings to ensure that they remain up-to-date and secure. By following these best practices, you can ensure that your Lambda function and RDS instance are always properly secured and functioning as intended.

Amazon API Gateway integration with AWS WAF

Binh Bui — Sat, 21 Jan 2023 17:09:32 +0000

Amazon API Gateway integration with AWS WAF

Introduction

Amazon API Gateway is a fully managed service that makes it easy for developers to create, publish, maintain, monitor, and secure APIs at any scale. With a few clicks in the AWS Management Console, you can create an API that acts as a “front door” for applications to access data, business logic, or functionality from your back-end services, such as workloads running on Amazon Elastic Compute Cloud (Amazon EC2), code running on AWS Lambda, or any web application.

Amazon API Gateway supports several types of integrations with AWS services and third-party services. In this post, we will focus on the integration with AWS WAF. AWS WAF is a web application firewall that helps protect your web applications from common web exploits that could affect application availability, compromise security, or consume excessive resources. AWS WAF gives you control over which traffic to allow or block to your web applications by defining customizable web security rules. You can use AWS WAF to create security rules that block common attack patterns, such as SQL injection or cross-site scripting, and rules that filter specific traffic patterns, such as requests from specific IP addresses or referring websites.

Why use Amazon API Gateway integration with AWS WAF?

Amazon API Gateway integration with AWS WAF is a great way to protect your APIs from common web exploits that could affect application availability, compromise security, or consume excessive resources. AWS WAF gives you control over which traffic to allow or block to your APIs by defining customizable web security rules. You can use AWS WAF to create security rules that block common attack patterns, such as SQL injection or cross-site scripting, and rules that filter specific traffic patterns, such as requests from specific IP addresses or referring websites.

Demo - Amazon API Gateway integration with AWS WAF by using AWS CDK

In this demo, we will create an API Gateway API with a single GET method that will return a JSON response. We will then create a WAF rule that will block requests from a specific IP address. We will then test the API and see that the request is blocked. We will then remove the WAF rule and test the API again to see that the request is allowed.

Create an API Gateway API

First, we will create an API Gateway API with a single GET method that will return a JSON response. We will use the AWS CDK to create the API Gateway API.

// lambda function
const testFn = new NodejsFunction(this, 'MyFunction', {
  entry: './function/index.ts',
  runtime: lambda.Runtime.NODEJS_18_X,
  handler: 'main',
  bundling: {
    externalModules: ['aws-sdk'],
    minify: true,
  },
})

// api gateway, this is just for testing
const api = new apigateway.LambdaRestApi(this, 'test-api', {
  handler: testFn,
  deployOptions: {
    stageName: 'test',
  },
})

Create a WAF ACL and WAF Rule

Next, we will create a WAF ACL and a WAF Rule allowing only requests from a specific location. We will use the AWS CDK to create the WAF ACL and WAF Rule.

const webACL = new wafv2.CfnWebACL(this, 'webACL', {
  name: 'webACL',
  description: 'This is WebACL for Auth APi Gateway',
  scope: 'REGIONAL', // or CLOUDFRONT for CloudFront
  defaultAction: { block: {} }, // default action is block all request
  visibilityConfig: {
    cloudWatchMetricsEnabled: true,
    metricName: 'webACL', // metric name for CloudWatch
    sampledRequestsEnabled: true,
  },

  rules: [
    {
      name: `demo-api-auth-gateway-geolocation-rule`,
      priority: 30,
      action: { allow: {} },
      visibilityConfig: {
        metricName: `demo-AuthAPIGeoLocation`,
        cloudWatchMetricsEnabled: true,
        sampledRequestsEnabled: false,
      },
      statement: {
        geoMatchStatement: {
          countryCodes: ['US', 'VN'], // allow US and VN IP. You can add more country codes
        },
      },
    },
  ],
})

This will create a WAF ACL with a single WAF Rule allowing only requests from a specific location(US and VN). We will then associate the WAF ACL with the API Gateway API.

// Web ACL Association
const webACLAssociation = new wafv2.CfnWebACLAssociation(this, 'webACLAssociation', {
  webAclArn: webACL.attrArn, // Web ACL ARN from above
  // For an Amazon API Gateway REST API: arn:aws:apigateway:region::/restapis/api-id/stages/stage-name
  resourceArn: Fn.join('', [
    'arn:aws:apigateway:', // service
    Stack.of(this).region, // region
    '::/restapis/', // resource type
    api.restApiId, // api id
    '/stages/', // resource type
    api.deploymentStage.stageName, // stage name
  ]),
})

// make sure api gateway is deployed before web ACL association
webACLAssociation.node.addDependency(api)

This will associate the WAF ACL with the API Gateway API. We will then test the API and see that the request is blocked. We will then remove the WAF rule and test the API again to see that the request is allowed.

The code for this demo is available on GitHub. You can deploy the demo by running the following commands.

git clone
cd apig-waf-cdk
yarn install
yarn deploy

Testing the API

After deploying the demo, we will test the API. We will use the following command to test the API.

curl -X GET https://<api-id>.execute-api.<region>.amazonaws.com/test

The response should be similar to the following.

{
  "message": "Hello from Lambda!"
}

Now, we will remove 'VN'(or your country code) from the WAF Rule and test the API again. We will use the following command to test the API.

curl -X GET https://<api-id>.execute-api.<region>.amazonaws.com/test

The response should be similar to the following.

{
  "message": "Forbidden"
}

The request is blocked because the request is from a location that is not allowed by the WAF Rule.

Conclusion

In this post, we have seen how to use Amazon API Gateway integration with AWS WAF by using AWS CDK. We have also seen how to test the API to see that the request is blocked.

The source code for this demo is available on GitHub.

That's all for now. Thanks for reading. If you have any questions or comments, please leave them below. I will try to answer them as soon as possible.

References

How to link a Cognito account with a Google account(source code full stack)

Binh Bui — Thu, 29 Dec 2022 04:15:00 +0000

Photo by [Flo Karr](https://unsplash.com/@flo_karr)

Introduction

Recently, I worked on a project that required users to be able to sign in with their Google account and users to be able to link their Google account with their Cognito account. I found that there is not much information on the internet about how to do this. So I decided to write this article to share my experience. I will show you how to link a Cognito account with a Google account in the front end and back end.

Links an existing user account in a user pool (DestinationUser) to an identity from an external IdP (SourceUser) based on a specified attribute name and value from the external IdP. This allows you to create a link from the existing user account to an external federated user identity that has not yet been used to sign in. You can then use the federated user identity to sign in as the existing user account.

For example, if there is an existing user with a username and password, this API links that user to a federated user identity. When the user signs in with a federated user identity, they sign in as the existing user account.

By default, AWS Cognito does not support linking social account (google/facebook) to AWS Cognito user. This project is to provide a solution to link social account to AWS Cognito user. The solution is to use pre-signup trigger to check if the user already exists in the user pool. If the user already exists, then link the social account to the user. If the user does not exist, then create a new user in the user pool.

All the code is available on GitHub. You can clone the repo and follow the steps below to run the project.

Run yarn install in the cdk and webapp folders.
Rename env.sh.template to env.sh. Update the env.sh file with your own values. 2. GOOGLE_CLIENT_ID - The google client ID. 3. GOOGLE_CLIENT_SECRET - The google client secret.
Run deploy.sh to deploy the backend.
Update the webapp/src/aws-exports.js file with the Cognito user pool information.
Run yarn dev in the webapp folder to start the frontend application.

How to link a Cognito account with a IDP account

The key to link a Cognito account with a IDP account is to use the pre-signup trigger. In the pre-signup trigger, we can check if the user already exists in the user pool. If the user already exists, then link the IDP account to the user. If the user does not exist, then create a new user in the user pool.

To order to link a Cognito account with a IDP account, we need to use the adminLinkProviderForUser API. The adminLinkProviderForUser API is used to link an existing user account in a user pool (DestinationUser) to an identity from an external IdP (SourceUser) based on a specified attribute name and value from the external IdP. This allows you to create a link from the existing user account to an external federated user identity that has not yet been used to sign in. You can then use the federated user identity to sign in as the existing user account.

Below is the code for the pre-signup trigger. The pre-signup trigger is used to check if the user already exists in the user pool. If the user already exists, then link the IDP account to the user. If the user does not exist, then create a new user in the user pool.

export const preSignup: PreSignUpTriggerHandler = async (event: PreSignUpTriggerEvent) => {
  console.log('preSignup event', event)

  const { triggerSource, userPoolId, userName, request } = event

  // Note: triggerSource can be either PreSignUp_SignUp or PreSignUp_ExternalProvider depending on how the user signed up
  // incase signup is done with email and password then triggerSource is PreSignUp_SignUp
  // incase signup is done with Google then triggerSource is PreSignUp_ExternalProvider

  if (triggerSource === 'PreSignUp_ExternalProvider') {
    // if user signed up with Google then we need to link the Google account to the user pool
    const {
      userAttributes: { email, given_name, family_name },
    } = request

    // if the user is found then we link the Google account to the user pool
    const user = await findUserByEmail(email, userPoolId)

    // userName example: "Facebook_12324325436" or "Google_1237823478"
    // we need to extract the provider name and provider value from the userName
    let [providerName, providerUserId] = userName.split('_')

    // Uppercase the first letter because the event sometimes
    // has it as google_1234 or facebook_1234. In the call to `adminLinkProviderForUser`
    // the provider name has to be Google or Facebook (first letter capitalized)
    providerName = providerName.charAt(0).toUpperCase() + providerName.slice(1)

    // if the user is found then we link the Google account to the user pool
    if (user) {
      await linkSocialAccount({
        userPoolId: userPoolId,
        cognitoUsername: user.Username,
        providerName: providerName,
        providerUserId: providerUserId,
      })

      // return the event to continue the signup process
      return event
    } else {
      // if the user is not found then we need to create the user in the user pool

      // 1. create a native cognito account
      const newUser = await createUser({
        userPoolId: userPoolId,
        email,
        givenName: given_name,
        familyName: family_name,
      })

      if (!newUser) {
        throw new Error('Failed to create user')
      }

      // 2. change the password, to change status from FORCE_CHANGE_PASSWORD to CONFIRMED
      await setUserPassword({
        userPoolId: userPoolId,
        email,
      })

      // 3. merge the social and the native accounts
      await linkSocialAccount({
        userPoolId: userPoolId,
        cognitoUsername: newUser.Username,
        providerName: providerName,
        providerUserId: providerUserId,
      })

      // set the email_verified to true so that the user doesn't have to verify the email
      // set the autoConfirmUser to true so that the user doesn't have to confirm the signup
      event.response.autoVerifyEmail = true
      event.response.autoConfirmUser = true
    }
  }

  // if the user signed up with email and password then we don't need to do anything
  return event
}

const linkProviderForUserCommand = new AdminLinkProviderForUserCommand({
  UserPoolId: userPoolId,
  DestinationUser: {
    ProviderName: 'Cognito', // Cognito is the default provider
    ProviderAttributeValue: cognitoUsername, // this is the username of the user
  },
  SourceUser: {
    ProviderName: providerName, // Google or Facebook (first letter capitalized)
    ProviderAttributeName: 'Cognito_Subject', // Cognito_Subject is the default attribute name
    ProviderAttributeValue: providerUserId, // this is the value of the provider
  },
})

await cognitoIdentityProviderClient.send(linkProviderForUserCommand)

By default, every user sign up with social identity providers (Google/Facebook) will have a Cognito_Subject attribute. The Cognito_Subject attribute is the unique identifier of the user in the social identity provider. We can use the Cognito_Subject attribute to link the social account to the Cognito user. The Cognito_Subject attribute is the same as the sub claim in the ID token.

Troubleshooting

1. "Already found an entry for username" exception when linking a user

This is known issue and discussing in this. The solution is catch this error in your front end, perhaps show a message like "Accounts have been successfully linked" and then prompt users to re-login with Hosted UI.
This is hacky solution but no other solution at the moment. I hope AWS will fix this issue in the future.

2. Every user sign-in with social identity providers (Google/Facebook) then the email becomes unverified

This is very annoying. After a user signs in with social identity providers (Google/Facebook), the email address becomes unverified. It makes the user have to verify the email address again every time they sign in username and password.

This is default behavior of AWS Cognito. When a user signs up with social identity providers (Google/Facebook), the email address is not verified. And it will update every time the user signs in with social identity providers (Google/Facebook). The solution is to use the post-authentication trigger to verify the email address.

export const postAuthentication = async (event: PostAuthenticationTriggerEvent) => {
  console.log('postAuthentication event', event)

  // set email_verified to true so that the user doesn't have to verify the email
  if (event.request.userAttributes.email_verified !== 'true') {
    await updateUserAttributes(event.userPoolId, event.userName, {
      email_verified: 'true',
    })
  }

  return event
}

Conclusion

Merge accounts with Cognito User Pools and Identity Providers still too complicated. Some developers are not happy with this feature. I hope AWS will improve this feature in the future. And I hope this article will help you to link a Cognito account with a IDP account.

You can find the source code of this article in this. If you have any questions, please leave a comment below. I will try to answer your questions. Thank you for reading this article.

AWS Lambda support Node.js 18 now. Should we update the version of Node.js in the Lambda runtime?

Binh Bui — Wed, 30 Nov 2022 03:18:04 +0000

Recently, AWS Lambda announced that it supports Node.js 18 and it is available in all AWS regions. This is a big news for Node.js developers. It means that we can use the latest version of Node.js in AWS Lambda. However, should we update the version of Node.js in the Lambda runtime? Let's find out.

Why we should update the version of Node.js in the Lambda runtime?

The reason is simple. The new version of Node.js has new features and bug fixes. We should always use the latest version of Node.js in the Lambda runtime. Besides, the new version of Node.js is faster and more secure. It is a good idea to update the version of Node.js in the Lambda runtime.

Some of benefits of using the latest version of Node.js in the Lambda runtime:

Support ES2021 features

Node.js 18 supports ES2021 features. We can use them in the Node.js runtime environment. For example, we can use the optional chaining operator (?.) and the nullish coalescing operator (??).

// index.mjs
export async function handler(event, context) {
  const data = {
    foo: {
      bar: 'baz',
    },
  }
  const value = data?.foo?.bar ?? 'default'
  return value
}

Support top-level await

Node.js 18 supports top-level await. We can use it in the Node.js runtime environment. For example, we can use it to make HTTP requests. I really like this feature because it really useful when we want to re-use same initialization code in multiple Lambda invocations. We can use top-level await to initialize the code once and then re-use it in multiple Lambda invocations.

// index.mjs
const response = await fetch('https://example.com')
const data = await response.json()

export async function handler(event, context) {
  return data
}

AWS SDK v3 support in the Node.js runtime environment

Lambda supports AWS SDK v3 in the Node.js runtime environment. This enables us to use the new AWS SDK v3 in the Node.js runtime environment. We can use the new AWS SDK v3 to make AWS API calls in the Node.js runtime environment. For example, we can use the new AWS SDK v3 to make DynamoDB API calls in the Node.js runtime environment without include the AWS SDK v3 in the Lambda deployment package.

// index.mjs
import { DynamoDBClient, PutItemCommand } from '@aws-sdk/client-dynamodb'

This is a big news for Node.js developers. We can use the new AWS SDK v3 to make AWS API calls in the Node.js runtime environment. We don’t need to use the old AWS SDK v2 anymore. This especially useful when creating functions in the AWS console or inline code in the CloudFormation template.

Node.js 18 language updates

The Lambda Node.js 18 runtime also enables you to take advantage of new Node.js 18 language features. This includes improved performance for class fields and private class methods, JSON import assertions, and experimental features such as the Fetch API, Test Runner module, and Web Streams API.

JSON import assertion

The import assertions feature allows module import statements to include additional information alongside the module specifier. Now the following code is valid:

// index.mjs

// static import with JSON import assertion
import fooData from './foo.json' assert { type: 'json' }

// dynamic import with JSON import assertion
const { default: barData } = await import('./bar.json', { assert: { type: 'json' } })

export const handler = async (event) => {
  console.log(fooData)
  // logs data in foo.json file
  console.log(barData)
  // logs data in bar.json file

  const response = {
    statusCode: 200,
    body: JSON.stringify('Hello from Lambda!'),
  }
  return response
}

Support fetch API in the Node.js runtime environment

The fetch API is a standard API for making HTTP requests in the browser. It is also available in Node.js 16 and above. We can use it to make HTTP requests in the Node.js runtime environment. It is a standard API and we don’t need to install any third-party libraries to use it.
In a previous article, I wrote about how to use the fetch API in the Node.js runtime environment. You can read it here.

// index.mjs
export async function handler(event, context) {
  const response = await fetch('https://example.com')
  const data = await response.json()
  return data
}

How to update the version of Node.js in the Lambda runtime?

We can update the version of Node.js in the Lambda runtime by using the AWS CDK. The following code shows how to update the version of Node.js in the Lambda runtime.

import * as lambda from '@aws-cdk/aws-lambda'

const lambdaFunction = new lambda.Function(this, 'LambdaFunction', {
  runtime: lambda.Runtime.NODEJS_18_X,
  handler: 'index.handler',
  code: lambda.Code.fromAsset('lambda'),
})

Or we can update the version of Node.js in the Lambda runtime by using the AWS CloudFormation. The following code shows how to update the version of Node.js in the Lambda runtime.

Resources:
  LambdaFunction:
    Type: AWS::Lambda::Function
    Properties:
      Runtime: nodejs18.x
      Handler: index.handler
      Code:
        ZipFile: |
          exports.handler = async (event) => {
            return {
              statusCode: 200,
              body: JSON.stringify('Hello from Lambda!'),
            };
          };

Conclusion

Node.js 18 is now supported by Lambda. We should consider updating the version of Node.js to Node.js 18 in the Lambda runtime as soon as possible. For existing Node.js functions, review your code for compatibility with Node.js 18, including deprecations, then migrate to the new runtime by changing the function’s runtime configuration to nodejs18.x.

Thanks for reading. If you have any questions, please leave a comment below. If you like this article, please share it with your friends. You can also follow me on Twitter and LinkedIn.

Quasar 2 with Nuxt3 (Starter Template)

Binh Bui — Wed, 16 Nov 2022 10:47:10 +0000

Quasar 2 with Nuxt3 (Starter Template)

This is a starter template for Quasar 2 with Nuxt3. Source code is available on GitHub.

What is Quasar?

Quasar is a full stack framework for building modern web applications. It is based on Vue.js and uses the same tooling and workflow. It is a collection of plugins, components, and libraries that allow you to build your app in a matter of minutes.

Quasar can be used to build web apps, PWAs, and mobile apps (Android, iOS, and Electron). It is also possible to build desktop apps with Quasar.

What is Nuxt3?

Nuxt 3 is a framework for creating Vue.js applications. It is based on Vue 3, Vue Router 4 and Vite. It is designed from the ground up to be incrementally adoptable. Nuxt 3 is the next evolution of the framework and it is not a rewrite. It is built with the future in mind and leverages the best features of Vue 3 and Vite.

Why Quasar 2 with Nuxt3?

I really like Quasar. It has a lot of UI components and it's easy to use. But I also like Nuxt3. It's fast and it's easy to use. So I decided to combine them together.

How to use Quasar 2 with Nuxt3?

Quasar 2 is a great framework for building web apps. It is easy to use and has a lot of features. However, it is not easy to use with Nuxt 3. There are some issues with Nuxt 3 and Quasar 2. For example, Nuxt 3 does not support the quasar.conf.js file. It also does not support the quasar command. This is why I created this starter template.

By using JSDOM to mock the DOM, we can use Quasar 2 with Nuxt 3. This is the main idea of this starter template. You can use this starter template to build your own web app.

Download the source code from GitHub

Aurora Serverless V2 with AWS CDK

Binh Bui — Wed, 09 Nov 2022 07:16:44 +0000

Introduction

Aurora Serverless is a fully managed, auto-scaling configuration for Amazon Aurora. It automatically starts up, shuts down, and scales capacity up or down based on your application’s needs. You can use Aurora Serverless to run your most demanding production workloads with less administrative effort than with other database options. Aurora Serverless is available in two versions: Aurora Serverless v1 and Aurora Serverless v2.

Aurora Serverless v2 is a new version of Aurora Serverless that provides a new storage engine, Aurora Storage Engine (Aurora SE), and a new query engine, Aurora Query Engine (Aurora QE). Aurora Serverless v2 is a drop-in replacement for Aurora Serverless v1. You can use the same API calls and tools to manage your Aurora Serverless v2 clusters as you use for Aurora Serverless v1.

In this article, we will use the AWS CDK to create an Aurora Serverless v2 cluster with a database and a database user. We will also create a Lambda function that will connect to the database and query the data.

Now AWS CDK has support for Aurora Serverless v2. You can use the AWS CDK to create an Aurora Serverless v2 cluster with a database and a database user.

Let’s get started.

// create a vpc
const vpc = new Vpc(this, 'VPC', {
  cidr: '10.0.0.0/16',
  subnetConfiguration: [{ name: 'egress', subnetType: SubnetType.PUBLIC }], // only one subnet is needed
  natGateways: 0, // disable NAT gateways
})

// create a security group for aurora db
const dbSecurityGroup = new SecurityGroup(this, 'DbSecurityGroup', {
  vpc: vpc, // use the vpc created above
  allowAllOutbound: true, // allow outbound traffic to anywhere
})

// allow inbound traffic from anywhere to the db
dbSecurityGroup.addIngressRule(
  Peer.anyIpv4(),
  Port.tcp(5432), // allow inbound traffic on port 5432 (postgres)
  'allow inbound traffic from anywhere to the db on port 5432'
)

// create a db cluster
// https://github.com/aws/aws-cdk/issues/20197#issuecomment-1117555047
const dbCluster = new rds.DatabaseCluster(this, 'DbCluster', {
  engine: rds.DatabaseClusterEngine.auroraPostgres({
    version: rds.AuroraPostgresEngineVersion.VER_13_6,
  }),
  instances: 1,
  instanceProps: {
    vpc: vpc,
    instanceType: new InstanceType('serverless'),
    autoMinorVersionUpgrade: true,
    publiclyAccessible: true,
    securityGroups: [dbSecurityGroup],
    vpcSubnets: vpc.selectSubnets({
      subnetType: SubnetType.PUBLIC, // use the public subnet created above for the db
    }),
  },
  port: 5432, // use port 5432 instead of 3306
})

// add capacity to the db cluster to enable scaling
Aspects.of(dbCluster).add({
  visit(node) {
    if (node instanceof CfnDBCluster) {
      node.serverlessV2ScalingConfiguration = {
        minCapacity: 0.5, // min capacity is 0.5 vCPU
        maxCapacity: 1, // max capacity is 1 vCPU (default)
      }
    }
  },
})

Let see what we have done here.

We created a VPC with a public subnet. We will use this VPC to create the Aurora Serverless v2 cluster.
We created a security group for the Aurora Serverless v2 cluster. We will use this security group to allow inbound traffic from anywhere to the Aurora Serverless v2 cluster.
We created an Aurora Serverless v2 cluster with a single instance. We used the serverless instance type to create an Aurora Serverless v2 cluster. We also set public accessibility to true to allow the Lambda function to connect to the Aurora Serverless v2 cluster. We also set the port to 5432 to use the Postgres port.
We added capacity to the Aurora Serverless v2 cluster to enable scaling. We set the minCapacity to 0.5 and maxCapacity to 1. This will allow the Aurora Serverless v2 cluster to scale up to 1 vCPU and scale down to 0.5 vCPU.

Create a lambda function to connect to the database and query the data

In this section, we will create a lambda function that will connect to the database and query the data. We will use the AWS CDK to create the lambda function.

const fn = new NodejsFunction(this, 'Lambda', {
  entry: './lambda/index.ts',
  runtime: Runtime.NODEJS_16_X,
  handler: 'main',
  bundling: {
    externalModules: ['aws-sdk', 'pg-native'],
    minify: false,
  },
  environment: {
    databaseSecretArn: dbCluster.secret?.secretArn ?? '', // pass the secret arn to the lambda function
  },
})

// allow the lambda function to access credentials stored in AWS Secrets Manager
// the lambda function will be able to access the credentials for the default database in the db cluster
dbCluster.secret?.grantRead(fn)

const api = new LambdaRestApi(this, 'Api', {
  handler: fn,
})

Let see what we have done here.

We created a lambda function with the AWS CDK. We used the NodejsFunction construct to create the lambda function. We used the externalModules option to exclude the aws-sdk and pg-native modules from the lambda function. We also used the minify option to disable minification.
We passed the secret arn to the lambda function using the environment option. The lambda function will use the secret arn to access the credentials for the default database in the Aurora Serverless v2 cluster.
We allowed the lambda function to access credentials stored in AWS Secrets Manager. The lambda function will be able to access the credentials for the default database in the Aurora Serverless v2 cluster.
And finally, we created a REST API using the lambda function. We will use this REST API to test the lambda function.

You can read more about how to Building Lambda functions with TypeScript in AWS CDK here

Below is the code for the lambda function.

import { GetSecretValueCommand, SecretsManagerClient } from '@aws-sdk/client-secrets-manager'
import { APIGatewayEvent, APIGatewayProxyResult } from 'aws-lambda'
import { Client } from 'pg'

export async function main(event: APIGatewayEvent): Promise<APIGatewayProxyResult> {
  // get the secret from secrets manager.
  const client = new SecretsManagerClient({})
  const secret = await client.send(
    new GetSecretValueCommand({
      SecretId: process.env.databaseSecretArn,
    })
  )
  const secretValues = JSON.parse(secret.SecretString ?? '{}')

  console.log('secretValues', secretValues)

  // connect to the database
  const db = new Client({
    host: secretValues.host, // host is the endpoint of the db cluster
    port: secretValues.port, // port is 5432
    user: secretValues.username, // username is the same as the secret name
    password: secretValues.password, // this is the password for the default database in the db cluster
    database: secretValues.dbname ?? 'postgres', // use the default database if no database is specified
  })

  await db.connect()

  // execute a query
  const res = await db.query('SELECT NOW()')

  // disconnect from the database
  await db.end()

  return {
    body: JSON.stringify({
      message: `DB Response: ${res.rows[0].now}`,
    }),
    statusCode: 200,
    isBase64Encoded: false,
    headers: {
      'Content-Type': 'application/json',
    },
  }
}

Let see what we have done here.

We created a SecretsManagerClient to get the secret from AWS Secrets Manager.
We used the SecretsManagerClient to get the secret from AWS Secrets Manager. We used the SecretId option to specify the secret arn. We used the GetSecretValueCommand to get the secret from AWS Secrets Manager.
We parsed the secret string to get the secret values. The secret string contains the credentials for the default database in the Aurora Serverless v2 cluster.
We created a Client to connect to the database. We used the host, port, user, password, and database options to connect to the database.
We executed a query to get the current time from the database.

That's it. We have created an Aurora Serverless v2 cluster with a single instance. We have also created a lambda function that will connect to the database and query the data. Now, we can deploy the stack to AWS.

Deploy the stack to AWS

To deploy the stack to AWS, run the following command.

cdk deploy

After the stack is deployed, you can verify that the Aurora Serverless v2 cluster is created in the AWS console.

Besides, you can test the lambda function by making a request to the REST API.

curl -X GET https://<api-id>.execute-api.<region>.amazonaws.com/prod

You should get a response similar to the one below.

{
  "message": "DB Response: 2022-11-08T15:52:00.000Z"
}

Clean up

Don't forget to delete the stack after you are done. This will prevent you from incurring any charges.

cdk destroy

Conclusion

In this article, we created an Aurora Serverless v2 cluster with a single instance. We also created a lambda function that will connect to the database and query the data. We used the AWS CDK to create the Aurora Serverless v2 cluster and the lambda function. You can find the code for this article here

That's it for this article. I hope you found it useful. If you have any questions or suggestions, please leave a comment below. You can also reach out to me on Twitter or LinkedIn.

AWS API Gateway Access Logs. How to get them and what to do with them.

Binh Bui — Mon, 26 Sep 2022 07:15:50 +0000

Photo by Marek Piwnicki Boxed Water Is Better

What are API Gateway Access Logs?

API Gateway Access logs are a feature of API Gateway that allows you to log all requests made to your API. This is a great feature to have enabled for debugging purposes. You can also use this data to create metrics and dashboards to monitor your API. You can read more about API Gateway Access Logs here.

Access logs are useful for two main reasons:

Debugging: If you get a spike in 500 Internal Server Error responses, you can locate an access log to point you in the right direction to start your investigation.
Performance analysis: You can analyze your access logs to look for performance degradations over time or to identify slow endpoints.

While API Gateway Access Logs are a great feature, they are not enabled by default. You will need to enable them in order to start logging requests.

Access logs vs. execution logs

There are two types of API logging in CloudWatch: execution logging and access logging.

Execution logging: This is the default logging that is enabled for all API Gateway APIs. It logs the execution of your API. This includes the request, response, and any errors that occur during the execution of your API. You can read more about execution logging here.
Access logging: This is a feature that you can enable to log all requests made to your API. You can read more about access logging here.

In the API Gateway console, you can configure them in the following screen:

Figure 1: Execution logging vs. access logging

As noted above, access logs are a single log line that is logged out on each request that comes to API Gateway, and they’re often used for detecting errors or performing data analysis. This is in contrast to execution logs, which are a series of log lines that are logged out on each request that comes to API Gateway, and they’re often used for debugging. Here’s the execution log output for a single request I made to API Gateway

Note: When you deploy an API, API Gateway creates a log group and log streams under the log group. The log group is named following the API-Gateway-Execution-Logs\_{'{'}rest-api-id{'}'}/{'{'}stage_name{'}'} format.

Figure 2: Execution log output. Many log lines per request

As you can see, there are a lot of log lines here. This is because execution logs are a series of log lines that are logged out on each request that comes to API Gateway.

In general, I disable API Gateway execution logs in the normal course of business. I only enable them when I need to debug an issue. I find that execution logs are too verbose for my needs. I prefer to use access logs for debugging and performance analysis.

How to enable access logs

To enable access logs, you will need to do the following:

Create a CloudWatch log group
Enable access logging in API Gateway and point it to the log group you created
Define the format of the access logs(You can use the default format or define your own). This is optional.

In this section, I will walk you through each of these steps in AWS CDK snippets.

Create a CloudWatch log group

First, you will need to create a CloudWatch log group. You can do this using the LogGroup construct.

const logGroup = new LogGroup(this, 'LogGroup', {
  logGroupName: '/aws/apigateway/my-api',
  removalPolicy: RemovalPolicy.DESTROY,
})

Enable access logging in API Gateway and point it to the log group you created

Next, you will need to enable access logging in API Gateway and point it to the log group you created. You can do this using the AccessLogDestination and AccessLogFormat constructs.

const accessLogDestination = new apigateway.LogGroupLogDestination(logGroup)

Define the format of the access logs

You can define the format of the access logs using the AccessLogFormat construct. You can use the default format or define your own. Here’s an example of the default format:

const accessLogFormat = apigateway.AccessLogFormat.jsonWithStandardFields()

// Create an API Gateway REST API with access logging enabled
const api = new apigateway.RestApi(this, 'MyApi', {
  deployOptions: {
    accessLogDestination,
    accessLogFormat,
  },
})

That is it! You have now enabled access logging in API Gateway and pointed it to the log group you created. You can now start logging requests made to your API.

Source code for this example can be found here

Access log format

When configuring your access logs, you get to choose an output format for your access logs. In doing so, you’ll be constructing a string to be formatted by API Gateway. These strings can use values from the $context object that will be formatted based on the actual values of your specific request.

The API Gateway docs show four general formats that you can use for your access logs:

Common Log Format (CLF). This is the default format.

$context.identity.sourceIp $context.identity.caller  \
$context.identity.user [$context.requestTime] \
"$context.httpMethod $context.resourcePath $context.protocol" \
$context.status $context.responseLength $context.requestId $context.extendedRequestId

JSON format. You can use the JSON format to have a detailed string describing your requests.

{ "requestId":"$context.requestId", \
  "extendedRequestId":"$context.extendedRequestId", \
  "ip": "$context.identity.sourceIp", \
  "caller":"$context.identity.caller", \
  "user":"$context.identity.user", \
  "requestTime":"$context.requestTime", \
  "httpMethod":"$context.httpMethod", \
  "resourcePath":"$context.resourcePath", \
  "status":"$context.status", \
  "protocol":"$context.protocol", \
  "responseLength":"$context.responseLength" \
}

XML format. You can use the XML format to have a detailed string describing your requests.

<request id="$context.requestId"> \
    <extendedRequestId>$context.extendedRequestId</extendedRequestId>
    <ip>$context.identity.sourceIp</ip> \
    <caller>$context.identity.caller</caller> \
    <user>$context.identity.user</user> \
    <requestTime>$context.requestTime</requestTime> \
    <httpMethod>$context.httpMethod</httpMethod> \
    <resourcePath>$context.resourcePath</resourcePath> \
    <status>$context.status</status> \
    <protocol>$context.protocol</protocol> \
    <responseLength>$context.responseLength</responseLength> \
</request>

CSV (comma-separated values) format. You can use the CSV format to have a brief string describing your requests.

$context.identity.sourceIp,$context.identity.caller,\
$context.identity.user,$context.requestTime,$context.httpMethod,\
$context.resourcePath,$context.protocol,$context.status,\
$context.responseLength,$context.requestId,$context.extendedRequestId

I prefer to use the JSON format for my access logs. It gives me a detailed string describing my requests. I can then use this string to parse out the information I need. I can also use this string to send to other services like Elasticsearch or Splunk.

Access logging fields

There are a lot of fields that you can use in your access log format. Here are some of the most common fields that you can use:

$context.identity.sourceIp - The IP address of the client that sent the request.
$context.identity.caller - The identity of the caller making the request. This is the identity that was used to call the API. If the caller is not authenticated, this value is null.
$context.identity.user - The user name of the authenticated caller making the request. If the caller is not authenticated, this value is null.
$context.requestTime - The time when API Gateway received the request from the client, in ISO 8601 format.
$context.httpMethod - The HTTP method used. Valid values include: DELETE, GET, HEAD, OPTIONS, PATCH, POST, and PUT.
$context.resourcePath - The resource path invoked by the request.
authorizer.claims.sub - The subject of the JWT. This is the unique identifier of the user.
integration.latency - The time between when API Gateway receives a request from a client and when it returns a response to the client. The latency includes the integration latency and other API Gateway overhead.
integration.status - The HTTP status code that is returned by the integration back to API Gateway.

I recommend using the following fields for your access logs:

{
  "error.message": "$context.error.message", // The error message returned by API Gateway.
  "httpMethod": "$context.httpMethod", // The HTTP method used. Valid values include: DELETE, GET, HEAD, OPTIONS, PATCH, POST, and PUT.
  "identity.sourceIp": "$context.identity.sourceIp", // The IP address of the client that sent the request.
  "integration.error": "$context.integration.error", // The error message returned by the integration.
  "integration.integrationStatus": "$context.integration.integrationStatus", // The status code returned by the integration.
  "integration.latency": "$context.integration.latency", // The time between when API Gateway receives a request from a client and when it returns a response to the client. The latency includes the integration latency and other API Gateway overhead.
  "integration.requestId": "$context.integration.requestId", // The ID of the request sent to the integration.
  "integration.status": "$context.integration.status", // The HTTP status code that is returned by the integration back to API Gateway.
  "path": "$context.path", // The resource path invoked by the request.
  "requestId": "$context.requestId", // The ID of the API Gateway request.
  "responseLatency": "$context.responseLatency", // The time between when API Gateway receives a request from a client and when it returns a response to the client. The latency does not include the integration latency.
  "responseLength": "$context.responseLength", // The length of the API Gateway response in bytes.
  "stage": "$context.stage", // The name of the API Gateway stage that processes the request.
  "status": "$context.status" // The HTTP status code that is returned by API Gateway.
}

Access logging to CloudWatch

You can use the CloudWatch Logs service to store your access logs. You can then use the CloudWatch Logs service to search, monitor, and analyze your logs. You can also use the CloudWatch Logs service to send your logs to other services like Elasticsearch or Splunk.

In AWS console, navigate to the AWS cloudwatch service. Then click on the Logs tab. Click on the log groups tab. Open the log group that you want to use for your access logs.

Figure 3: API access log

Querying access logs with Log Insights

After you have enabled access logging for your API Gateway, you can query your access logs using Log Insights. Log Insights is a query language that you can use to search and analyze log data in CloudWatch Logs. You can use Log Insights to search for specific log events, filter log events, and aggregate log data. You can also use Log Insights to create dashboards and alerts.

Some of the most common Log Insights queries that you can use to query your access logs are:

Finding 5XX responses in CloudWatch Logs

fields @timestamp, @message
| filter @message like /"status":5/
| sort @timestamp desc
| limit 10

Finding aggregates with CloudWatch Logs Insights

Imagine you want to find the number of 5XX responses for each day. You can use the following query to find the number of 5XX responses for each day.

fields @timestamp, @message
| filter @message like /"status":5/
| stats count(*) by bin(1d)

Finding the most common 5XX responses

Imagine you want to find the most common 5XX responses. You can use the following query to find the most common 5XX responses.

fields @message
| filter @message like /"status":5/
| stats count(*) by @message
| sort count_* desc
| limit 10

Conclusion

In this article, you learned how to enable access logging for your API Gateway. You also learned how to query your access logs using Log Insights. I hope you found this article useful. If you have any questions or comments, please leave them below. Thanks for reading!

Source code for this example can be found here

CloudScape Design with NextJS

Binh Bui — Tue, 23 Aug 2022 09:57:18 +0000

As of July 19th, 2022 Amazon Web Services released their Design System as open-source along with a library of React components that implement the design system. This blog post is a quick overview of the design system and how it can be used with NextJS.

What is Cloudscape?

Cloudscape Design System is an open source solution for building intuitive, engaging, and inclusive user experiences at scale. Cloudscape consists of an extensive set of guidelines to create web applications, along with the design resources and front-end components to streamline implementation.
Cloudscape was built for and is used by Amazon Web Services (AWS) products and services. They created it in 2016 to improve the user experience across web applications owned by AWS services, and also to help teams implement those applications faster.

You can read more about Cloudscape here.

It's a beautiful design system and offers a lot in terms of component functionality out of the box. I would highly encourage everyone to try it out.

Installing Cloudscape

For NextJS users, you can install Cloudscape by running the following command:

npm install @cloudscape-design/global-styles @cloudscape-design/components

CloudScape Integration with NextJS

Next, we will need to integrate them into our application. First we will import the CloudScape global styles package into our app. Let's do this in our pages/_app.{jsx,tsx} file

import '@cloudscape-design/global-styles'

After we have included our global styles, let's try to use some of the components in our application.

import { useState } from 'react'
import Header from '@cloudscape-design/components/header'
import Container from '@cloudscape-design/components/container'
import SpaceBetween from '@cloudscape-design/components/space-between'
import Input from '@cloudscape-design/components/input'
import Button from '@cloudscape-design/components/button'

export default function Home() {
  const [value, setValue] = useState('')

  return (
    <SpaceBetween size="m">
      <Header variant="h1">Hello World!</Header>

      <Container>
        <SpaceBetween size="s">
          <span>Start editing to see some magic happen</span>
          <Input value={value} onChange={(event) => setValue(event.detail.value)} />
          <Button variant="primary">Click me</Button>
        </SpaceBetween>
      </Container>
    </SpaceBetween>
  )
}

Run it and see what happens.

This error occurs due to NextJS implementing native CSS modules. We can fix this by use a NextJS plugin to transpile our styles into native CSS modules that NextJS can understand

npm install -D next-transpile-modules next-compose-plugins

After install the plugin, we need edit next.config.js as follows:

const withTranspileModules = require('next-transpile-modules')
const withPlugins = require('next-compose-plugins')

const nextConfig = {
  // other config options
}

// Here we will add a plugin to our configuration to transpile the CloudScape components into
module.exports = withPlugins([withTranspileModules(['@cloudscape-design/components'])], nextConfig)

Bonus, in case you are using typescript then the file next.config.js should be like this

const withTM = require('next-transpile-modules')(['@cloudscape-design/components']);

/** @type {import('next').NextConfig} */
const nextConfig = {
  reactStrictMode: true,
  swcMinify: true,
}

const buildConfig = _phase => {
  const plugins = [withTM]
  const config = plugins.reduce((acc, next) => next(acc), {
    ...nextConfig
  })
  return config
}

module.exports = buildConfig();

Restart NextJS dev server we should no longer see any error messages. You also see the following output:

Wow, it seem to work! But wait a second, some error messages are still showing up in console.

Warning: useLayoutEffect does nothing on the server, because its effect cannot be encoded into the server renderer's output format. This will lead to a mismatch between the initial, non-hydrated UI and the intended UI. To avoid this, useLayoutEffect should only be used in components that render exclusively on the client. See https://reactjs.org/link/uselayouteffect-ssr for common fixes.
    at InternalContainer (webpack-internal:///./node_modules/@cloudscape-design/components/container/internal.js:32:21)
    at Container (webpack-internal:///./node_modules/@cloudscape-design/components/container/index.js:24:17)

This is because Cloudscape uses the useLayoutEffect hook to render the components.
And the useLayoutEffect hook is not supported in the server renderer.
The Cloudscape team does not intend to Replace useLayoutEffect at the moment.
You can read more about it here.

To fix this issue, you can add this into _app.js.

import React from 'react'
if (typeof window === 'undefined') React.useLayoutEffect = () => {}

That all, we are good to go! The code is now working as expected.

Thanks for reading! Hope you find this useful! :)

Code available on GitHub

Use Route53 + S3 to forward your domain to absolute URLs.

Binh Bui — Fri, 05 Aug 2022 03:59:34 +0000

Suppose that you have a domain name that you want to forward to an absolute URL. For example, you have a domain name dev.codewithyou.com and you want to forward it to https://www.codewithyou.com/blog/cloudfront-restrict-user-access-by-signed-urls. You can do this with Route53 and S3.

How to do it

Step 1: Create a bucket and redirect it to an absolute URL

Open the Amazon S3 console at https://console.aws.amazon.com/s3/.
Under Buckets, create a new bucket with the name dev.codewithyou.com. Name of the bucket MUST is the name of the domain. If the name is different, you will not see the bucket in the route53 alias record.
Choose the bucket that you created in step 2 and click on the Properties tab.
Under Static website hosting, choose Edit.
Choose Redirect requests for an object.
In the Host name box, enter www.codewithyou.com/blog/cloudfront-restrict-user-access-by-signed-urls. This is the absolute URL that you want to forward to your domain. You can use any domain name you want. The URL must not include the protocol (http:// or https://).
For Protocol, choose none

Choose Save changes.

You can test the redirect by visiting the bucket url in your browser. For example, http://dev.codewithyou.com.s3-website-ap-southeast-1.amazonaws.com/. You should see it redirect to the https://www.codewithyou.com/blog/cloudfront-restrict-user-access-by-signed-urls URL.

Step 2: Create a Route53 alias record

Now you need to create a Route53 alias record. You can do this by going to the Route53 console at https://us-east-1.console.aws.amazon.com/route53/v2/home#Dashboard

Click on the Create record button.
Specify the following values:
1. Record name dev
2. Record type A Choose A – IPv4 address.
3. Alias. Select Yes. Choose the bucket that you created in step 1.

Click on the Create records button and wait for the record to be created.

After the record is created, you can test the redirect by visiting the alias record url in your browser. For example, http://dev.codewithyou.com/. You should see it redirect to the https://www.codewithyou.com/blog/cloudfront-restrict-user-access-by-signed-urls URL.

Conclusion

This is a simple example of how to use Route53 and S3 to forward your domain to absolute URLs.
You can also do many other things with Route53 and S3.

Redirect requests for your bucket's website endpoint to another bucket or domain
Configure redirection rules to use advanced conditional redirects
Redirect requests for an object

I will continue to add more examples in the future. Thanks for reading!

How to use parameters in AWS CDK?

Binh Bui — Wed, 20 Jul 2022 06:52:38 +0000

Defining CDK Parameters

Use the optional Parameters section to customize your templates. Parameters enable you to input custom values to your template each time you create or update a stack.

To define a parameter, you use the CfnParameter construct.

// parameter of type String
const applicationPrefix = new CfnParameter(this, 'prefix', {
  description: 'parameter of type String',
  type: 'String',
  allowedPattern: '^[a-z0-9]*$', // allowed pattern for the parameter
  minLength: 3, // minimum length of the parameter
  maxLength: 20, // maximum length of the parameter
}).valueAsString // get the value of the parameter as string

console.log('application Prefix 👉', applicationPrefix)

// parameter of type Number
const applicationPort = new CfnParameter(this, 'port', {
  description: 'parameter of type Number',
  type: 'Number',
  minValue: 1, // minimum value of the parameter
  maxValue: 65535, // maximum value of the parameter
  allowedValues: ['8080', '8081'], // allowed values of the parameter
}).valueAsNumber // get the value of the parameter as number

console.log('application Port 👉', applicationPort)

// parameter of type CommaDelimitedList
const applicationDomains = new CfnParameter(this, 'domains', {
  description: 'parameter of type CommaDelimitedList',
  type: 'CommaDelimitedList',
}).valueAsList // get the value of the parameter as list of strings

console.log('application Domains 👉', applicationDomains)

Note that the name (logical ID) of the parameter will derive from its name and location within the stack. Therefore, it is recommended that parameters are defined at the stack level.

Let's over go what we did in the code snippet above.

We defined 3 parameters:

prefix: a parameter of type String, with a minimum length of 3 and a maximum length of 20.
port: a parameter of type Number, with a minimum value of 1 and a maximum value of 65535.
domains: a parameter of type CommaDelimitedList.

CloudFormation currently supports the following parameter types:

String – A literal string
Number – An integer or float
List – An array of integers or floats
CommaDelimitedList – An array of literal strings that are separated by * commas
AWS-specific parameter types
SSM parameter types

If you try to deploy the stack without defining the parameters, the stack will fail.

CdkStarterStackStack failed: Error: The following CloudFormation Parameters are missing a value: port, domains

The parameter prefix is not required because it has a default value.

Deploy a stack with parameters

npx aws-cdk deploy \
    --parameters prefix=demo \
    --parameters port=8081 \
    --parameters domains=www.codewithyou.com,www.freedevtool.com \
    --outputs-file ./cdk-outputs.json

Note that we have to use the --parameters flag for every parameter we pass into the template.

Now that we've successfully deployed our CDK application, we can inspect the parameters section in the CloudFormation console:

Or we can use the cdk-outputs.json file to get the values of the parameters:

CdkStarterStackStack.applicationDomains = www.codewithyou.com,www.freedevtool.com
CdkStarterStackStack.applicationPort = 8081
CdkStarterStackStack.applicationPrefix = demo

If you are look into the cdk.out/CdkStarterStackStack.template.json file, you will see that parameters are defined in the Parameters section.

{
  "Parameters": {
    "prefix": {
      "Type": "String",
      "Default": "sample",
      "AllowedPattern": "^[a-z0-9]*$",
      "Description": "parameter of type String",
      "MaxLength": 20,
      "MinLength": 3
    },
    "port": {
      "Type": "Number",
      "AllowedValues": ["8080", "8081"],
      "Description": "parameter of type Number",
      "MaxValue": 65535,
      "MinValue": 1
    },
    "domains": {
      "Type": "CommaDelimitedList",
      "Description": "parameter of type CommaDelimitedList"
    },
    "BootstrapVersion": {
      "Type": "AWS::SSM::Parameter::Value<String>",
      "Default": "/cdk-bootstrap/hnb659fds/version",
      "Description": "Version of the CDK Bootstrap resources in this environment, automatically retrieved from SSM Parameter Store. [cdk:skip]"
    }
  }
}

Cleanup

Don't forget to delete the stack before leaving this article. Thanks for reading!

npx aws-cdk destroy

The code for this article is available on GitHub

Improve the Security of API Keys [Checklist included]

Binh Bui — Fri, 08 Jul 2022 04:40:06 +0000

Photo by Kristina Flour

Storing and managing secrets like API keys and other credentials can be challenging. Your API key is valuable information and an accidental leak could result in unwanted calls, app blockage, or worse. Publicly exposing your API keys can lead to unexpected charges on your account.

As a developer, API Keys are typically issued to you to identify the project you are working on and to enforce rate and access limits on proper API usage. These API keys are typically just static secrets baked into your app or web page, and they are pretty easy to steal but painful to replace. You can do better.

Managing API Keys

[x] Using separate API keys for each app, environment, and service.

This limits the scope of each key. If an API key is compromised, you can delete or regenerate the impacted key without needing to update your other API keys.

[x] Monitoring your API usage

Monitoring your API usage to detect unauthorized usage is a good idea.

[x] Applying API key restrictions

By adding restrictions, you can reduce the impact of a compromised API key on your application. Based on the API provider, you can restrict the API key to a specific app, environment, or service.

For example,

You can restrict the API key to a specific app, environment, or service.
You can also restrict the API key to a specific region.
You can also restrict the API key to a specific IP address.
You can also restrict the API key to a specific user.
...

[x] Restrict API access permissions

Add default to minimal permission scope for APIs. Choose right scope for each API.

[x] Delete unneeded API keys to minimize exposure to attacks.

[x] Generate a new key if you suspect a compromise

Store secrets safely

[x] Don’t store your API key directly in your code.

Don’t expose your API key or secret in your code. Instead, use a library to manage your API keys. For example, you can use Node.js’s dotenv to store your API key and secret in a file called .env in the root of your project. This file is ignored by Git, so it is safe to store your API key and secret in this file.

[x] Use local environment variables, when feasible

When you are working on a local environment, you can use local environment variables to store your API key and secret. This is a good idea because it reduces the risk of your API key and secret being exposed to your code. Example of a local environment variable:

export API_KEY=<your-api-key>
export API_SECRET=<your-api-secret>

[x] Consider using an API secret management service

Secret management services are a great way to manage your API keys and secrets. They are a great way to manage your API keys and secrets. One solution for convenience and peace of mind is to use a secret management service like AWS Secret Manager.