DEV Community: se-piyush

How to Resolve 403 Access Issues When Deploying a SPA on AWS with S3 and CloudFront

se-piyush — Tue, 10 Sep 2024 13:34:03 +0000

When deploying a Vue.js Single Page Application (SPA) on AWS, developers often encounter a common issue: the 403 access error. This typically happens when refreshing a page or accessing a non-root URL after the application has already loaded. Here’s my experience troubleshooting and resolving this issue using S3 for static website hosting and CloudFront for distribution.

The Problem: 403 Access Error After Refreshing Routes

While working on deploying my Vue.js application on AWS, I stored the bundled JavaScript files in a S3 bucket and used it for static website hosting. I then created a CloudFront distribution and linked it to my S3 bucket.

The default route pointed to the index.html page, where my app was bootstrapped. Everything worked well when loading the app using the CloudFront domain. However, once the app was loaded, a path would be appended to the domain, like cloud-domain.com/dashboard.

The problem occurred when I refreshed the /dashboard route or navigated to another path directly—it returned a 403 Access Denied error.

Why It Happened: CloudFront's Path Request Mechanism

Here’s how CloudFront works: when you request a URL with a path, CloudFront treats it as a request for a specific file or object. It checks whether the file exists in its cache or in the linked S3 bucket. In this case, since my S3 bucket didn’t contain an actual dashboard file (or any other path-specific file), CloudFront returned the 403 error.

The Solution: Redirect All Requests to `index.html`

The solution to this issue is straightforward: I needed to redirect all requests to index.html, as my Vue.js app handles routing internally. After some research, I discovered that I needed to update configurations in both my S3 bucket and CloudFront distribution.

Here’s what I did:

Step 1: Update S3 Bucket Settings

In the S3 bucket, under the Static Website Hosting section, I had already set index.html as the default document. However, I also needed to set index.html as the Error Document. This way, when any request came in that didn’t match an existing file (like /dashboard), it would return index.html instead of throwing a 403 error.

Here’s how the S3 settings looked after I made the change:

Step 2: Configure CloudFront Custom Error Responses

Next, I needed to configure my CloudFront distribution to handle these requests properly. In the Error Pages section of CloudFront, I created a custom error response for 403 errors.

I set the response code to 200 (OK).
I specified /index.html as the page to serve in the case of a 403 error.
This ensured that when CloudFront couldn’t find the requested path (like /dashboard), it would serve the index.html file instead.

Here’s how I configured CloudFront:

Final Configuration in CloudFront:

After configuring these settings, the issue was resolved. Now, any request to a specific path is properly redirected to the index.html page, allowing Vue.js to handle the routing internally.

Voila! Issue Resolved

By making these two key changes—updating the S3 error document settings and configuring CloudFront's custom error responses—the 403 error was eliminated. Now, my application routes work perfectly, even when refreshing or navigating directly to different URLs.

Conclusion

Deploying a Vue.js Single Page Application on AWS using S3 and CloudFront can result in 403 errors when refreshing or navigating to non-root URLs. However, by redirecting all paths to the index.html page, you allow Vue.js to manage routing internally, solving the issue.

How to Host a Vue.js Application Using S3 and CloudFront

se-piyush — Tue, 10 Sep 2024 13:33:07 +0000

In this post, I’ll guide you through the steps to host a Vue.js Single Page Application (SPA) on AWS using S3 and CloudFront. This setup is perfect for ensuring that your Vue.js app functions efficiently, with all internal routing handled properly.

Step 1: Create a Private S3 Bucket

Start by creating a private S3 bucket to store your Vue.js app files. It's essential to keep the bucket private, as we’ll be using CloudFront to serve content securely.

Here’s how you create the S3 bucket:

After creating the bucket, make sure its permissions are set to private.

Now that your S3 bucket is created, it’s time to upload your Vue.js app bundle.

Step 2: Upload Your Vue.js Application Bundle

Once you’ve built your Vue.js app, upload the bundled files (such as JavaScript, CSS, and HTML files) into the S3 bucket. After uploading, your bucket contents might look like this:

Step 3: Enable Static Website Hosting

Next, enable Static Website Hosting for your S3 bucket. Set both the Index Document and Error Document to index.html. This ensures that all routing will be handled by the index.html page, and your Vue.js application can manage the internal routing.

Here’s what the static website hosting setup should look like:

Step 4: Create a CloudFront Distribution

Now it’s time to set up CloudFront to serve your app. Go to CloudFront and create a new distribution. When setting the origin, choose the S3 bucket you just created. For enhanced security, select Origin Access Control (OAC), which limits direct access to the bucket and serves files only through CloudFront.

Here’s what the CloudFront origin settings look like:

During the setup, you’ll need to create an OAC policy that allows CloudFront to access your S3 bucket securely:

Once the CloudFront distribution is created, update your bucket’s permissions by editing the Bucket Policy. This ensures that CloudFront can access your S3 bucket contents. Here’s a sample policy that allows CloudFront to fetch your files:

{
    "Version": "2008-10-17",
    "Id": "PolicyForCloudFrontPrivateContent",
    "Statement": [
        {
            "Sid": "AllowCloudFrontServicePrincipal",
            "Effect": "Allow",
            "Principal": {
                "Service": "cloudfront.amazonaws.com"
            },
            "Action": "s3:GetObject",
            "Resource": "arn:aws:s3:::<your-bucket-name>/*",
            "Condition": {
                "StringEquals": {
                    "AWS:SourceArn": "arn:aws:cloudfront::<your-aws-account-id>:distribution/<your-cloudfront-distribution-id>"
                }
            }
        }
    ]
}

You can get the policy directly from AWS after creating the distribution or use the above template, replacing the placeholders with your details. To apply it, go to the Bucket Policy section under the Permissions tab in S3.

Step 5: Configure CloudFront Distribution Settings

After your CloudFront distribution is created, configure it to point to index.html as the default root object. This ensures that your root URL always loads the Vue.js entry point.

Here’s how the settings should look:

Step 6: Create a Custom Error Response for 403 Errors

Vue.js SPAs rely on internal routing for navigation. When you refresh a page or try to access a direct route, CloudFront may not find the requested file and will return a 403 error. To avoid this, create a custom error response in CloudFront.

Under the Error Pages tab, add a new custom error response for 403 errors, pointing the Response Page Path to index.html and setting the HTTP response code to 200. This will ensure that CloudFront serves the index.html file for any undefined route, letting Vue.js handle the internal routing. Described more about the error here.

Here’s how to configure the custom error response:

Once set up, your CloudFront custom error response should look like this:

Conclusion

By following these steps, you can successfully deploy your Vue.js Single Page Application using AWS S3 and CloudFront. With this setup, all your routes will work correctly, even on refresh, thanks to the proper configuration of the static website hosting, CloudFront distribution, and custom error responses.

Common Issues and Resolutions When Setting Up a VPC in AWS

se-piyush — Fri, 09 Aug 2024 16:39:11 +0000

Setting up a Virtual Private Cloud (VPC) in AWS can be a challenging task, especially if you encounter unexpected issues. In this post, I’ll share the problems I faced while configuring a VPC, along with the solutions that worked for me. As I'm still working on my VPC setup, I’ll continue to update this post with any new issues and resolutions I come across.

The approach in this blog is straightforward: I’ll first describe the problem and then provide the solution I used. Please note that while these solutions worked for me, there might be more efficient ways to resolve these issues.

Issue 1: Timeout When Communicating with DynamoDB or Other AWS Services from Lambda/EC2 after Setting Up a VPC Endpoint (Interface)

Problem:

After setting up a VPC Endpoint of type Interface, I encountered timeouts when my Lambda function or EC2 instance tried to communicate with DynamoDB or other AWS services.

Solution:

When you create an Interface-type VPC Endpoint, AWS provides a DNS name in its configuration. To ensure successful communication with DynamoDB or other AWS services, you need to use this DNS name in your code.

Additionally, double-check the security group (SG) rules:

The outbound rule in your Lambda-linked security group should be configured to allow connections to AWS services via the endpoint.
The security group linked with your VPC Endpoint must have an inbound rule that permits connections from the Lambda’s security group.

For more detailed steps, you can refer to my other blog post on creating and linking VPC Endpoints of type Interface.

Issue 2: Timeout When Connecting to Third-Party APIs Outside AWS (e.g., Stripe, Firebase)

Problem:

I experienced timeouts when trying to connect to third-party APIs, such as Stripe or Firebase, that are external to the AWS environment.

Solution:

This issue is often caused by misconfigured Network Address Translation (NAT) or Internet Gateway (IG) settings. Here’s what worked for me:

Ensure the Internet Gateway is associated with the route table of the subnet designated as a public subnet.
If your Lambda function or EC2 instance is in a private subnet and needs to connect to the public internet, configure a NAT. When setting up the NAT, make sure it’s associated with the public subnet where the Internet Gateway is linked. Then, associate the NAT with the route table of the private subnet.

For more details on setting up VPCs, check out these posts: Setting Up a VPC for Your App Using AWS Management Console and Setting Up a VPC and Lambda Function with Terraform.

Issue 3: Unable to Delete Subnets Linked to Lambda, Even After Detaching the VPC

Problem:

I wasn’t able to delete subnets linked to my Lambda function, even after detaching the VPC from Lambda.

Solution:

Before you can delete subnets, you need to remove any Network Interfaces (ENIs) associated with the Lambda function. However, in my case, the Network Interfaces were still marked as In-use despite detaching the VPC.

This was due to Lambda versioning. Even though I removed the VPC from the latest Lambda version, previous versions were still attached to the VPC. The solution was to delete the older Lambda versions, which then allowed me to remove the Network Interfaces and delete the subnets.

If you're facing similar issues, this GitHub repo was helpful for me. It contains a script that identifies what might be holding up your Network Interface. If the script doesn’t find any issues, consider contacting AWS support for further assistance.

Conclusion

Setting up a VPC in AWS can be tricky, but by understanding the common issues and how to resolve them, you can avoid a lot of frustration. Whether you're dealing with timeouts when connecting to services or issues with deleting resources, the solutions provided here should help you navigate these challenges.

I’ll continue updating this blog as I encounter new issues and find solutions, so be sure to check back for more insights!

Creating and Linking VPC Endpoint of Type Interface using Terraform and AWS management console

se-piyush — Thu, 08 Aug 2024 15:03:50 +0000

In my previous posts, Setting Up VPC and Lambda Function with Terraform and Setting Up a VPC for Your App Using AWS Management Console, I talked about setting up VPC with Lambda and how my Lambda function interacts with DynamoDB and the external internet.

To connect to DynamoDB, we used a VPC Endpoint of type Gateway, which is easier to manage and doesn’t come with additional costs if the data transfer is within the VPC.

However, Gateway endpoints only support S3 and DynamoDB resources. All other resources (including DynamoDB and S3) require an Interface endpoint.

The Interface type VPC Endpoint requires a Security Group and some changes in your Lambda function code, specifically the DynamoDB configuration, unlike the Gateway type Endpoint, which doesn't require them.

This means that Interface Endpoints can provide more granular restrictions.

In this post, I’ll explain how to create and link a VPC endpoint of type Interface using both the AWS Management Console and Terraform.

Steps to Create a VPC Endpoint of Type Interface

Create Security Groups for Lambda and DynamoDB:
- For the DynamoDB Security Group, specify an inbound rule that accepts incoming requests from the Lambda Security Group.
- For the Lambda Security Group, specify an outbound rule that permits sending requests to DynamoDB.
Create the VPC Endpoint:
- Choose the Interface type service, select the private subnet ID, and link your DynamoDB Security Group to it.
- After creating the VPC Endpoint, retrieve the DNS name from the description, and use it in your Lambda code to connect to DynamoDB.

Creating a VPC Endpoint of type Interface Using Terraform

resource "aws_vpc" "main" {
  cidr_block = "<YOUR_CIDR_BLOCK>"
}

# Create private subnets
resource "aws_subnet" "private" {
  count                   = <NUMBER OF PRIVATE SUBNETS>
  vpc_id                  = aws_vpc.main.id
  cidr_block              = cidrsubnet(aws_vpc.main.cidr_block, 8, count.index * 2 + 1)
  availability_zone       = element(data.aws_availability_zones.available.names, count.index)
  map_public_ip_on_launch = false
}

resource "aws_security_group" "dynamodb_vpc_endpoint_sg" {
  name        = "dynamo-sg"
  description = "Security group for Lambda"
  vpc_id      = aws_vpc.main.id

  ingress {
    from_port       = 0
    to_port         = 0
    protocol        = "-1"
    security_groups = [aws_security_group.lambda.id]
  }
}

resource "aws_security_group" "lambda" {
  name        = "lambda-sg"
  description = "Security group for Lambda"
  vpc_id      = aws_vpc.main.id

  egress {
    from_port   = 443
    to_port     = 443
    protocol    = "TCP"
    cidr_blocks = [aws_vpc.main.cidr_block]
  }
}

resource "aws_vpc_endpoint" "dynamodb" {
  vpc_id             = aws_vpc.main.id
  service_name       = "com.amazonaws.${var.region}.dynamodb"
  vpc_endpoint_type  = "Interface"
  subnet_ids         = aws_subnet.private[*].id
  security_group_ids = [aws_security_group.dynamodb_vpc_endpoint_sg.id]
}

# Use the endpoint as an environment variable in your Lambda function
output "dynamodb_vpc_endpoint_dns" {
  value = aws_vpc_endpoint.dynamodb.dns_entry[0].dns_name
}

const { DynamoDBClient } = require("@aws-sdk/client-dynamodb");
const client = new DynamoDBClient({
    endpoint: `https://<THE VPC ENDPOINT DNS>`,
});
const dynamoDbClient = DynamoDBDocumentClient.from(client);

Creating a VPC Endpoint of Type Interface Using AWS Management Console

Create the DynamoDB Security Group with the appropriate inbound rule (In this example, I’ve created an outbound rule to the Lambda Security Group).
Create the Lambda Security Group with the appropriate outbound rule (Here, I’ve added my VPC CIDR as an outbound rule).
Create a VPC Endpoint of type Interface with the intended VPC and private subnet, then link your DynamoDB Security Group to it. After creation, get the DNS name from the description and use it in your Lambda function to connect to DynamoDB.

You can use this endpoint like this in your Node.js Lambda function:

const { DynamoDBClient } = require("@aws-sdk/client-dynamodb");
const client = new DynamoDBClient({
    endpoint: `https://<THE VPC ENDPOINT DNS>`,
});
const dynamoDbClient = DynamoDBDocumentClient.from(client);

Conclusion

In this post, we explored how to create and link a VPC Endpoint of type Interface using both the AWS Management Console and Terraform. By leveraging Interface endpoints, you can secure your Lambda function's communication with DynamoDB and other AWS services, while maintaining fine-grained control over traffic and security.

We contrasted the Interface endpoint with the Gateway endpoint, noting the additional configuration required, such as setting up security groups and modifying Lambda function code. However, this additional complexity provides enhanced security and control, making Interface endpoints a powerful tool for managing internal communication within your AWS environment.

With the provided Terraform code and AWS Management Console steps, you should now be able to integrate Interface endpoints into your architecture, ensuring secure and efficient access to DynamoDB from your Lambda functions.

Setting Up VPC and Lambda Function with Terraform

se-piyush — Wed, 07 Aug 2024 13:05:35 +0000

In my previous post, Setting Up a VPC for Your App Using AWS Management Console, I talked about how to set up a VPC using the AWS Management Console.

In this blog, I will show you how to achieve a similar setup using Terraform. I have a Lambda function that is behind a custom domain API Gateway. This Lambda function interacts with DynamoDB (without going through the internet) and some third-party APIs.

Below is the Terraform code to set up the required infrastructure.

Terraform Code

`main.tf` File

resource "aws_vpc" "main" {
  cidr_block = "<YOUR_PREFERRED_IP_CIDR_BLOCK>"
}

# Create private subnets
resource "aws_subnet" "private" {
  count                   = <NUMBER_OF_SUBNETS_YOU_WANT>
  vpc_id                  = aws_vpc.main.id
  cidr_block              = cidrsubnet(aws_vpc.main.cidr_block, 8, count.index * 2 + 1)
  availability_zone       = element(data.aws_availability_zones.available.names, count.index)
  map_public_ip_on_launch = false
}

# Create public subnets
resource "aws_subnet" "public" {
  count                   = <NUMBER_OF_SUBNETS_YOU_WANT>
  vpc_id                  = aws_vpc.main.id
  cidr_block              = cidrsubnet(aws_vpc.main.cidr_block, 8, count.index * 2)
  availability_zone       = element(data.aws_availability_zones.available.names, count.index)
  map_public_ip_on_launch = true
}

# Create Internet Gateway
resource "aws_internet_gateway" "igw" {
  vpc_id = aws_vpc.main.id
}

# Create Elastic IP for NAT Gateway
resource "aws_eip" "nat" {
  domain = "vpc"
}

# Create NAT Gateway
resource "aws_nat_gateway" "nat" {
  allocation_id = aws_eip.nat.id
  subnet_id     = element(aws_subnet.public.*.id, 0)
}

# Create public route table
resource "aws_route_table" "public" {
  vpc_id = aws_vpc.main.id

  route {
    cidr_block = "0.0.0.0/0"
    gateway_id = aws_internet_gateway.igw.id
  }
}

# Associate public subnets with public route table
resource "aws_route_table_association" "public" {
  count          = length(aws_subnet.public.*.id)
  subnet_id      = element(aws_subnet.public.*.id, count.index)
  route_table_id = aws_route_table.public.id
}

# Create private route table
resource "aws_route_table" "private" {
  vpc_id = aws_vpc.main.id

  route {
    cidr_block     = "0.0.0.0/0"
    nat_gateway_id = aws_nat_gateway.nat.id
  }
}

# Associate private subnets with private route table
resource "aws_route_table_association" "private" {
  count          = length(aws_subnet.private.*.id)
  subnet_id      = element(aws_subnet.private.*.id, count.index)
  route_table_id = aws_route_table.private.id
}

# Create security group for Lambda
resource "aws_security_group" "lambda" {
  name        = "lambda-sg"
  description = "Security group for Lambda"
  vpc_id      = aws_vpc.main.id

  egress {
    from_port   = 443
    to_port     = 443
    protocol    = "TCP"
    cidr_blocks = [aws_vpc.main.cidr_block]
  }
}

# Lambda VPC Endpoint for DynamoDB
resource "aws_vpc_endpoint" "dynamodb" {
  vpc_id             = aws_vpc.main.id
  service_name       = "com.amazonaws.${var.region}.dynamodb"
  vpc_endpoint_type  = "Gateway"
  route_table_ids    = aws_route_table.private[*].id
}
# The above terraform command automatically creates prefix list in the private route table.

# Lambda Function
resource "aws_lambda_function" "lambda_function" {
  filename      = <PATH_TO_YOUR_ZIP_FILE.zip>
  function_name = "<LAMBDA_FUNCTION_NAME>"
  role          = aws_iam_role.lambda_role.arn
  handler       = "index.handler"
  runtime       = "nodejs20.x"
  memory_size   = 512
  timeout       = 10

  logging_config {
    log_format = "Text"
    log_group  = "/aws/lambda/<LAMBDA_FUNCTION_NAME>"
  }
  tracing_config {
    mode = "PassThrough"
  }

  ephemeral_storage {
    size = "512"
  }

  vpc_config {
    subnet_ids         = aws_subnet.private[*].id
    security_group_ids = [aws_security_group.lambda.id]
  }
}

resource "aws_lambda_alias" "lambda_alias" {
  name             = var.stage
  function_name    = aws_lambda_function.lambda_function.function_name
  function_version = aws_lambda_function.lambda_function.version
}

resource "aws_lambda_permission" "apigw" {
  statement_id  = "AllowAPIGatewayInvoke"
  action        = "lambda:InvokeFunction"
  function_name = aws_lambda_function.lambda_function.function_name
  principal     = "apigateway.amazonaws.com"
  source_arn    = "${aws_api_gateway_rest_api.api.execution_arn}/${var.stage}/*"
}

resource "aws_iam_role" "lambda_role" {
  name = "lambdaRole"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Action = "sts:AssumeRole"
        Effect = "Allow"
        Principal = {
          Service = "lambda.amazonaws.com"
        }
      }
    ]
  })
}

# DynamoDB Tables
resource "aws_dynamodb_table" "dynamo_table" {
  // YOUR TABLE CONFIG
}

resource "aws_iam_role_policy" "lambda_policy" {
  name = "lambda-policy"
  role = aws_iam_role.lambda_role.id

  policy = jsonencode({
    Version = "2012-10-17",
    Statement = [
      {
        Effect = "Allow",
        Action = [
          "logs:CreateLogGroup",
          "logs:CreateLogStream",
          "logs:TagResource",
          "logs:PutLogEvents"
        ],
        Resource = [
          "arn:aws:logs:${var.region}:${data.aws_caller_identity.current.account_id}:log-group:/aws/lambda/<LAMBDA_FUNCTION_NAME>-${var.stage}*:*"
        ]
      },
      {
        Effect = "Allow",
        Action = [
          "logs:PutLogEvents"
        ],
        Resource = [
          "arn:aws:logs:${var.region}:${data.aws_caller_identity.current.account_id}:log-group:/aws/lambda/<LAMBDA_FUNCTION_NAME>-${var.stage}*:*:*"
        ]
      },
      {
        Effect = "Allow",
        Action = [
          "dynamodb:Query",
          "dynamodb:Scan",
          "dynamodb:GetItem",
          "dynamodb:PutItem",
          "dynamodb:UpdateItem",
          "dynamodb:DeleteItem"
        ],
        Resource = [
          aws_dynamodb_table.dynamo_table.arn
        ]
      },
      {
        Effect = "Allow",
        Action = [
          "ec2:CreateNetworkInterface",
          "ec2:DescribeNetworkInterfaces",
          "ec2:DeleteNetworkInterface",
          "ec2:DescribeInstances",
          "ec2:AttachNetworkInterface"
        ],
        Resource = "*"
      }
    ]
  })
}

resource "aws_iam_role_policy_attachment" "policy_attach" {
  role       = aws_iam_role.lambda_role.name
  policy_arn = "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
}

# API Gateway
resource "aws_api_gateway_rest_api" "api" {
  name        = aws_lambda_function.lambda_function.function_name
  description = "API for your Service in ${var.stage} environment"
}

resource "aws_api_gateway_resource" "proxy" {
  rest_api_id = aws_api_gateway_rest_api.api.id
  parent_id   = aws_api_gateway_rest_api.api.root_resource_id
  path_part   = "{proxy+}"
}

resource "aws_api_gateway_method" "proxy_method" {
  rest_api_id   = aws_api_gateway_rest_api.api.id
  resource_id   = aws_api_gateway_resource.proxy.id
  http_method   = "ANY"
  authorization = "NONE"
}

resource "aws_api_gateway_integration" "proxy_integration" {
  rest_api_id             = aws_api_gateway_rest_api.api.id
  resource_id             = aws_api_gateway_resource.proxy.id
  http_method             = aws_api_gateway_method.proxy_method.http_method
  type                    = "AWS_PROXY"
  integration_http_method = "POST"
  uri                     = aws_lambda_function.lambda_function.invoke_arn
}

resource "aws_api_gateway_deployment" "api_deployment" {
  depends_on = [
    aws_api_gateway_method.proxy_method,
    aws_api_gateway_integration.proxy_integration,
  ]
  rest_api_id = aws_api_gateway_rest_api.api.id
  lifecycle {
    create_before_destroy = true
  }
}

resource "aws_api_gateway_stage" "api_stage" {
  stage_name    = var.stage
  rest_api_id   = aws

_api_gateway_rest_api.api.id
  deployment_id = aws_api_gateway_deployment.api_deployment.id
}

resource "aws_api_gateway_base_path_mapping" "base_path_mapping" {
  depends_on  = [aws_api_gateway_stage.api_stage]
  domain_name = var.api_gateway_domain_name
  api_id      = aws_api_gateway_rest_api.api.id
  stage_name  = aws_api_gateway_stage.api_stage.stage_name
  base_path   = var.base_path
}

data "aws_caller_identity" "current" {}

Notice the use of subnet and security group while creating the Lambda function.

Note: If you are using VPC Endpoint of type Gateway than you dont need to make any changes in your code specially in dynamo db configurations.

Conclusion

By following these steps and utilizing the Terraform code provided, you can set up a secure and efficient environment for your application on AWS. This setup involves creating and configuring a VPC, subnets, route tables, internet gateway, NAT gateway, VPC endpoints, and associating them appropriately. Following these steps will help you establish a secure and efficient environment for your application on AWS.

Setting Up a VPC for Your App Using AWS Management Console

se-piyush — Tue, 06 Aug 2024 20:57:59 +0000

In this blog, I will demonstrate how to set up a Virtual Private Cloud (VPC) using the AWS Management Console for your application. I have a Lambda function that is behind a custom domain API Gateway. This Lambda function interacts with DynamoDB (without going through the internet) and some third-party APIs. The Lambda function is behind a custom domain API Gateway.

Things to Keep in Mind

Resources inside a VPC must use security groups (SG) which act as firewalls, defining inbound and outbound rules for each resource.
Once inside the VPC, the Lambda function cannot talk with DynamoDB or any other third-party API due to the security group.
To allow the Lambda function to talk with DynamoDB while keeping the connection within AWS VPC, we use a VPC Endpoint of type Gateway (which is cost-effective compared to Interface type).
Currently, Gateway type VPC Endpoints are only supported by S3 and DynamoDB. To communicate with other services, Interface type VPC Endpoints are required.
Gateway type VPC Endpoints require a record in the route table and do not depend on security group outbound rules, whereas Interface type VPC Endpoints require security group inbound and outbound rules.
To allow our Lambda function to talk to public APIs over the internet, we need to create a NAT Gateway in a public subnet and associate that NAT Gateway with the private subnet's route table. Public subnets are those with a route table linked to an Internet Gateway.
Although resources inside a private subnet should not talk to the public internet, for convenience, we will set this up.

Creating a VPC

In your AWS account, there is always a default VPC. You have the option to create only a VPC or a VPC with additional resources (subnets, NAT Gateway, VPC Endpoint). In this guide, we will create everything one by one.

Integrating VPC with Your Lambda Function

Create VPC

This will result in your VPC entry:
Create a Public Subnet
Note: Ideally, create multiple subnets in multiple availability
zones, but for this demonstration, I will create only one per subnet.
Create a Private Subnet

Now you can see your entries:
Create Route Tables for Each Subnet Type (Public and Private)
Although there is a default route created at the time of VPC creation, we will create separate routes for better management.
Associate the Subnets with the Created Routes

Your route configuration for private and public subnets should look something like this:

Make sure to associate the intended subnets.
Provide Internet Access to Lambda Function Inside VPC
- Create Internet Gateway and Attach it to VPC
- Associate Internet Gateway with Public Subnet Route
- Create NAT Gateway on a Public Subnet and Associate it with Private Subnet Route
Create VPC Endpoint for DynamoDB to Avoid Internet Connections

As we have selected the private route table for the above VPCE, a new entry will be created inside the private route table of type prefix list as shown below:
Create Security Group for More Granular Restrictions

Generally, it is a bad practice for a security group linked to a private subnet resource to have a default outbound rule allowing outgoing internet connections for every protocol, port, and IP.
Link the VPC to Lambda Function
Linking the VPC to a Lambda function is straightforward:

Conclusion

By following these steps, you can set up a secure and efficient environment for your application on AWS, ensuring that your Lambda function can interact with DynamoDB without going through the public internet. This setup involves creating and configuring a VPC, subnets, route tables, internet gateway, NAT gateway, and VPC endpoints, along with associating them appropriately. Following these steps will help you establish a secure and efficient environment for your application on AWS.

Creating Cross-Account DynamoDB Backups with Terraform

se-piyush — Wed, 10 Jul 2024 15:13:21 +0000

In my previous post, Creating Secure Backups for DynamoDB Tables with Terraform, I discussed how to create DynamoDB table backups within the same AWS account. In this post, I will explain how to create cross-account DynamoDB backups, where the tables are in one AWS account and the backups are created in another AWS account for enhanced security and limited access.

Steps to Configure Cross-Account Backups

To achieve cross-account DynamoDB backups, follow these steps:

Step 1: Create Destination Vault in Backup Account

First, set up the backup vault in the AWS account where you want to store the backups. This includes configuring a KMS key for encryption and setting up the vault policy to allow cross-account access.

resource "aws_backup_vault" "destination_backup_vault" {
  name        = "destination-backup-vault"
  kms_key_arn = aws_kms_key.backup_vault_key.arn
}

resource "aws_kms_key" "backup_vault_key" {
  description         = "KMS key for backup vault encryption"
  enable_key_rotation = true
}

resource "aws_backup_vault_policy" "destination_backup_vault_policy" {
  backup_vault_name = aws_backup_vault.destination_backup_vault.name
  policy = jsonencode({
    Version = "2012-10-17",
    Statement = [
      {
        Sid : "AllowCrossAccountAccess",
        Effect = "Allow",
        Principal = {
          AWS = "arn:aws:iam::[YOUR SOURCE ACCOUNT ID]:root"
        },
        Action = [
          "backup:CopyIntoBackupVault"
        ],
        Resource = "*"
      }
    ]
  })
}

Step 2: Create Source Vault in Source Account

Next, create the backup vault in the source account where the DynamoDB tables reside.

resource "aws_backup_vault" "source_backup_vault" {
  name        = "source-backup-vault"
  kms_key_arn = aws_kms_key.backup_vault_key.arn
}

resource "aws_kms_key" "backup_vault_key" {
  description         = "KMS Key for Backup"
  enable_key_rotation = true
}

Step 3: Create IAM Role with Relevant Permissions

Create an IAM role in the source account with the necessary permissions to perform backups, including the AWSBackupServiceRolePolicyForBackup policy.

data "aws_iam_policy_document" "assume_role" {
  statement {
    effect = "Allow"

    principals {
      type        = "Service"
      identifiers = ["backup.amazonaws.com"]
    }

    actions = ["sts:AssumeRole"]
  }
}

resource "aws_iam_role" "source_backup_role" {
  name               = "source-backup-role"
  assume_role_policy = data.aws_iam_policy_document.assume_role.json

  inline_policy {
    name = "backup-policy"

    policy = jsonencode({
      Version = "2012-10-17",
      Statement = [
        {
          Effect = "Allow",
          Action = [
            "dynamodb:CreateBackup",
            "dynamodb:DeleteBackup",
            "dynamodb:DescribeBackup",
            "dynamodb:ListBackups",
            "dynamodb:ListTables",
            "dynamodb:RestoreTableFromBackup",
            "dynamodb:ListTagsOfResource",
            "dynamodb:StartAwsBackupJob",
            "dynamodb:RestoreTableFromAwsBackup"
          ],
          Resource = "*"
        },
        {
          Effect = "Allow",
          Action = [
            "backup:StartBackupJob",
            "backup:StopBackupJob",
            "backup:TagResource",
            "backup:UntagResource"
          ],
          Resource = "*"
        }
      ]
    })
  }
}

resource "aws_iam_role_policy_attachment" "service_backup_policy" {
  policy_arn = "arn:aws:iam::aws:policy/service-role/AWSBackupServiceRolePolicyForBackup"
  role       = aws_iam_role.source_backup_role.name
}

Step 4: Create a Backup Plan

Configure a backup plan to periodically back up the DynamoDB tables and copy them to the destination vault.

resource "aws_backup_plan" "cross_account_cross_region_copy" {
  name = "cross-account-cross-region-copy"

  rule {
    rule_name         = "copy-to-destination"
    target_vault_name = aws_backup_vault.source_backup_vault.name
    schedule          = "cron(38 13 * * ? *)"

    copy_action {
      destination_vault_arn = var.destination_backup_vault_arn
      lifecycle {
        delete_after = 30 # Retain for 30 days
      }
    }
  }
}

resource "aws_backup_selection" "dynamodb_cross_account_copy_selection" {
  plan_id      = aws_backup_plan.cross_account_cross_region_copy.id
  name         = "copy-backup-selection"
  iam_role_arn = aws_iam_role.source_backup_role.arn

  resources = [
    //table arns
  ]
}

resource "aws_backup_vault_policy" "source_backup_vault_policy" {
  backup_vault_name = aws_backup_vault.source_backup_vault.name
  policy = jsonencode({
    Version = "2012-10-17",
    Statement = [
      {
        Sid : "AllowCrossAccountAccess",
        Effect = "Allow",
        Principal = {
          AWS = "arn:aws:iam::[YOUR BACKUP ACCOUNT ID]:root"
        },
        Action = [
          "backup:CopyFromBackupVault",
          "backup:CopyIntoBackupVault"
        ],
        Resource = "*"
      }
    ]
  })
}

So your complete destination terraform file would look something like this

resource "aws_backup_vault" "source_backup_vault" {
  name        = "source-backup-vault"
  kms_key_arn = aws_kms_key.backup_vault_key.arn
}

resource "aws_kms_key" "backup_vault_key" {
  description         = "KMS Key for Backup"
  enable_key_rotation = true
}

resource "aws_backup_vault_policy" "source_backup_vault_policy" {
  backup_vault_name = aws_backup_vault.source_backup_vault.name
  policy = jsonencode({
    Version = "2012-10-17",
    Statement = [
      {
        Sid : "AllowCrossAccountAccess",
        Effect = "Allow",
        Principal = {
          AWS = "arn:aws:iam::[YOUR BACKUP ACCOUNT ID]:root"
        },
        Action = [
          "backup:CopyFromBackupVault",
          "backup:CopyIntoBackupVault"
        ],
        Resource = "*"
      }
    ]
  })
}

data "aws_iam_policy_document" "assume_role" {
  statement {
    effect = "Allow"

    principals {
      type        = "Service"
      identifiers = ["backup.amazonaws.com"]
    }

    actions = ["sts:AssumeRole"]
  }
}

resource "aws_iam_role" "source_backup_role" {
  name               = "source-backup-role"
  assume_role_policy = data.aws_iam_policy_document.assume_role.json

  inline_policy {
    name = "backup-policy"

    policy = jsonencode({
      Version = "2012-10-17",
      Statement = [
        {
          Effect = "Allow",
          Action = [
            "dynamodb:CreateBackup",
            "dynamodb:DeleteBackup",
            "dynamodb:DescribeBackup",
            "dynamodb:ListBackups",
            "dynamodb:ListTables",
            "dynamodb:RestoreTableFromBackup",
            "dynamodb:ListTagsOfResource",
            "dynamodb:StartAwsBackupJob",
            "dynamodb:RestoreTableFromAwsBackup"
          ],
          Resource = "*"
        },
        {
          Effect = "Allow",
          Action = [
            "backup:StartBackupJob",
            "backup:StopBackupJob",
            "backup:TagResource",
            "backup:UntagResource"
          ],
          Resource = "*"
        }
      ]
    })
  }
}

resource "aws_iam_role_policy_attachment" "service_backup_policy" {
  policy_arn = "arn:aws:iam::aws:policy/service-role/AWSBackupServiceRolePolicyForBackup"
  role       = aws_iam_role.source_backup_role.name
}

resource "aws_backup_plan" "cross_account_cross_region_copy" {
  name = "cross-account-cross-region-copy"

  rule {
    rule_name         = "copy-to-destination"
    target_vault_name = aws_backup_vault.source_backup_vault.name
    schedule          = "cron(38 13 * * ? *)"

    copy_action {
      destination_vault_arn = var.destination_backup_vault_arn
      lifecycle {
        delete_after = 30 # Retain for 30 days
      }
    }
  }
}

resource "aws_backup_selection" "dynamodb_cross_account_copy_selection" {
  plan_id      = aws_backup_plan.cross_account_cross_region_copy.id
  name         = "copy-backup-selection"
  iam_role_arn = aws_iam_role.source_backup_role.arn

  resources = [
    //table arns
  ]
}

Understanding the Cron Schedule

The schedule parameter in the backup plan uses a cron expression to determine when the backup should run. Here is an explanation of the cron expression used in this configuration:

schedule = "cron(38 13 * * ? *)"

38: Minute (38th minute of the hour).
13: Hour (1 PM UTC).
*: Day of the month (any day).
*: Month (any month).
?: Day of the week (any day of the week).
*: Year (optional, any year).

This configuration schedules the backup job to run daily at 1:38 PM UTC.

Conclusion

Creating cross-account backups for DynamoDB tables enhances the security and availability of your data. By following the steps outlined in this guide, you can use Terraform to automate the process of setting up secure backups across different AWS accounts. This approach ensures that your backups are stored securely in a separate account, providing an additional layer of protection against data loss or unauthorized access.

If you have any suggestions or improvements, please feel free to comment. Happy Terraforming!

Creating Secure Backups for DynamoDB Tables with Terraform

se-piyush — Wed, 03 Jul 2024 10:43:50 +0000

Creating DynamoDB tables using Terraform is straightforward, but ensuring these tables are securely backed up is crucial for data protection and recovery. In this blog post, I will guide you through configuring secure backups for your DynamoDB tables, storing them in a secure AWS vault using Terraform. Additionally, I will explain the configuration of the cron expression used to schedule these backups.

Step-by-Step Guide to Secure DynamoDB Backups with Terraform

Step 1: Define Your Resources

We need to define several resources to achieve secure backups:

AWS Backup Vault: A secure vault to store backups.
KMS Key: For encrypting the backups.
AWS Backup Plan: Defines when and how often to create backups.
AWS Backup Selection: Specifies which resources to back up.
IAM Role: Grants necessary permissions to AWS Backup service.

Step 2: Create Terraform Configuration

Here is the Terraform configuration template to set up secure backups for DynamoDB tables.

# Define KMS Key for Backup Vault Encryption
resource "aws_kms_key" "backup_vault_key" {
  description = "KMS key for backup vault encryption"
}

# Define AWS Backup Vault
resource "aws_backup_vault" "source_backup_vault" {
  name        = "source-backup-vault"
  kms_key_arn = aws_kms_key.backup_vault_key.arn
}

# Define AWS Backup Plan
resource "aws_backup_plan" "dynamodb_backup_plan" {
  name = "dynamodb-backup"

  rule {
    rule_name         = "daily-backup"
    target_vault_name = aws_backup_vault.source_backup_vault.name
    schedule          = "cron(0 12 * * ? *)" # Daily at 12 PM UTC

    lifecycle {
      delete_after = 30 # Retain for 30 days
    }
  }
}

# Define AWS Backup Selection
resource "aws_backup_selection" "dynamodb_backup_selection" {
  plan_id      = aws_backup_plan.dynamodb_backup_plan.id
  name         = "dynamodb-backup-selection"
  iam_role_arn = aws_iam_role.backup_role.arn

  resources = [
    "<your-table-arn>"
  ]
}

# Define IAM Role for Backup
resource "aws_iam_role" "backup_role" {
  name = "backup-role"

  assume_role_policy = jsonencode({
    Version = "2012-10-17",
    Statement = [
      {
        Effect = "Allow",
        Principal = {
          Service = "backup.amazonaws.com"
        },
        Action : "sts:AssumeRole"
      }
    ]
  })

  inline_policy {
    name = "backup-policy"

    policy = jsonencode({
      Version = "2012-10-17",
      Statement = [
        {
          Effect = "Allow",
          Action = [
            "dynamodb:CreateBackup",
            "dynamodb:DeleteBackup",
            "dynamodb:DescribeBackup",
            "dynamodb:ListBackups",
            "dynamodb:ListTables",
            "dynamodb:RestoreTableFromBackup",
            "dynamodb:ListTagsOfResource",
            "dynamodb:StartAwsBackupJob",
            "dynamodb:RestoreTableFromAwsBackup"
          ],
          Resource = "*"
        },
        {
          Effect = "Allow",
          Action = [
            "backup:StartBackupJob",
            "backup:StopBackupJob",
            "backup:TagResource",
            "backup:UntagResource"
          ],
          Resource = "*"
        }
      ]
    })
  }
}

Explanation of the Configuration

KMS Key: This resource aws_kms_key defines a KMS key used to encrypt the backups stored in the backup vault.
AWS Backup Vault: This vault aws_backup_vault will store the backups securely, using the KMS key for encryption.
AWS Backup Plan: This aws_backup_plan plan schedules the backup jobs. In this example, backups are scheduled to run daily at 12 PM UTC and are retained for 30 days.
AWS Backup Selection: This selection aws_backup_selection specifies which resources (DynamoDB tables) to back up.
IAM Role: This role aws_iam_role grants AWS Backup the necessary permissions to create, delete, describe, and list backups, as well as manage tags and restore tables.

Working of Cron and Its Configuration

The cron expression in the aws_backup_plan resource specifies the schedule for the backup jobs. Here’s a breakdown of how the cron expression works:

schedule = "cron(0 12 * * ? *)"

0: The first field specifies the minute (0).
12: The second field specifies the hour (12 PM UTC).
*: The third field specifies the day of the month (any day).
*: The fourth field specifies the month (any month).
?: The fifth field specifies the day of the week (any day of the week).
*: The sixth field specifies the year (optional field, any year).

In this example, the backup job is scheduled to run every day at 12 PM UTC.

Conclusion

By using Terraform, you can automate the setup of secure backups for your DynamoDB tables, ensuring that your data is safely stored and easily recoverable. This approach leverages AWS Backup, KMS encryption, and IAM roles to provide a robust backup solution. Additionally, the use of cron expressions allows you to customize the backup schedule to meet your requirements. Following the steps outlined above, you can set up a secure backup system that meets your organization's data protection needs.

Using Terraform to Manage Resources in Multiple AWS Accounts

se-piyush — Wed, 03 Jul 2024 10:05:18 +0000

While working on deploying resources on AWS using Terraform, I encountered a scenario where I needed to work with more than one AWS account within the same Terraform configuration. The use case was to deploy two resources in different AWS accounts but manage their states in the same Terraform state file. Here’s how I achieved this using Terraform’s module feature.

Steps to Use Multiple AWS Accounts in Terraform

Step 1: Create AWS CLI Profiles

First, I created two profiles using the AWS CLI for the different accounts:

aws configure --profile aws-profile-a
aws configure --profile aws-profile-b

These profiles store the credentials and configuration for each AWS account.

Step 2: Update the Main Terraform Configuration

Next, I made changes to my main.tf file to configure the AWS providers and use aliases for each account. Here’s what the configuration looked like:

provider "aws" {
  alias   = "accountA"
  region  = "us-east-1"  # Replace with your region
  profile = "aws-profile-a"
}

provider "aws" {
  alias   = "accountB"
  region  = "us-west-2"  # Replace with your region
  profile = "aws-profile-b"
}

module "some_resource_in_aws_account_A" {
  providers = {
    aws = aws.accountA
  }
  source = "./path-to-module-A"
}

module "some_resource_in_aws_account_B" {
  providers = {
    aws = aws.accountB
  }
  source = "./path-to-module-B"
}

In this configuration:

I defined two AWS providers with aliases accountA and accountB.
Each module specifies the provider alias it should use.

Step 3: Update Module Configuration

I also made sure my modules could accept the provider configurations. Here’s an example of how to define the required providers within a module:

terraform {
  required_providers {
    aws = {
      source                = "hashicorp/aws"
      version               = ">= 4.0"
      configuration_aliases = [aws]
    }
  }
}

resource "aws_s3_bucket" "example" {
  bucket = "example-bucket"
  acl    = "private"
}

Full Example of Main Configuration and Module

main.tf:

provider "aws" {
  alias   = "accountA"
  region  = "us-east-1"  # Replace with your region
  profile = "aws-profile-a"
}

provider "aws" {
  alias   = "accountB"
  region  = "us-west-2"  # Replace with your region
  profile = "aws-profile-b"
}

module "resource_in_accountA" {
  providers = {
    aws = aws.accountA
  }
  source = "./moduleA"
}

module "resource_in_accountB" {
  providers = {
    aws = aws.accountB
  }
  source = "./moduleB"
}

moduleA/main.tf:

terraform {
  required_providers {
    aws = {
      source                = "hashicorp/aws"
      version               = ">= 4.0"
      configuration_aliases = [aws]
    }
  }
}

resource "aws_s3_bucket" "example" {
  bucket = "example-bucket-accountA"
  acl    = "private"
}

moduleB/main.tf:

terraform {
  required_providers {
    aws = {
      source                = "hashicorp/aws"
      version               = ">= 4.0"
      configuration_aliases = [aws]
    }
  }
}

resource "aws_s3_bucket" "example" {
  bucket = "example-bucket-accountB"
  acl    = "private"
}

Explanation

Provider Configuration: The provider "aws" blocks in main.tf use aliases accountA and accountB to specify different AWS profiles.
Module Configuration: Each module (moduleA and moduleB) specifies which provider alias to use through the providers attribute.
Required Providers in Modules: The terraform block within each module defines the required provider and ensures it can accept the alias.

Benefits

Separation of Concerns: Different resources can be managed in different AWS accounts, improving security and management.
Single State File: By using a single state file, you maintain a consistent view of your infrastructure.
Modularity: Modules help organize and reuse code, making the Terraform configuration more manageable.

Conclusion

Using multiple AWS accounts in a single Terraform configuration can be achieved by leveraging Terraform's provider aliasing and module features. This approach allows for better separation of concerns, centralized state management, and modularity. By following the steps outlined above, you can deploy resources across different AWS accounts efficiently while maintaining a unified state file.

Automate REST API Management with Terraform: AWS API Gateway Proxy Integration and Lambda

se-piyush — Mon, 01 Jul 2024 18:06:24 +0000

In my previous post, I talked about how to simplify REST API management with AWS API Gateway Proxy Integration and Lambda using the AWS Console. In this blog, I will discuss how to achieve the same setup using Terraform.

Overview

Using Terraform to manage your infrastructure as code (IaC) provides several advantages, such as version control, automation, and consistency across environments. The steps to integrate an API Gateway with Lambda using Terraform involve:

Creation of API Gateway.
Granting it AllowAPIGatewayInvoke permission to invoke the desired Lambda function.
Creating a proxy resource with the path {proxy+}.
Creating an ANY proxy method for the proxy resource.
Creating an API Stage.
Creating a deployment.

Terraform Code Example

Here is the example Terraform code that includes the creation of a Lambda function and the necessary API Gateway configuration:

# Lambda Function
resource "aws_lambda_function" "lambda" {
  function_name = var.lambda_function_name
  role          = aws_iam_role.lambda_role.arn
  filename      = "location/of/package.zip"
  handler       = "index.handler"
  runtime       = "nodejs20.x"
  memory_size   = 512
  timeout       = 10
}

#Grants API Gateway permission to invoke the Lambda function.
resource "aws_lambda_permission" "apigw" {
  statement_id  = "AllowAPIGatewayInvoke"
  action        = "lambda:InvokeFunction"
  function_name = aws_lambda_function.lambda.function_name
  principal     = "apigateway.amazonaws.com"
  source_arn    = "${aws_api_gateway_rest_api.api.execution_arn}/${var.stage}/*"
}

#Creates an IAM role with the necessary permissions 
resource "aws_iam_role" "lambda_role" {
  name = "lambda_role"

  assume_role_policy = jsonencode({
    Version = "2012-10-17",
    Statement = [
      {
        Action = "sts:AssumeRole",
        Effect = "Allow",
        Principal = {
          Service = "lambda.amazonaws.com"
        }
      }
    ]
  })
}

resource "aws_iam_role_policy_attachment" "lambda_policy" {
  role       = aws_iam_role.lambda_role.name
  policy_arn = "arn:aws:iam::aws:policy/service-role/AWSLambdaBasicExecutionRole"
}

# API Gateway
resource "aws_api_gateway_rest_api" "api" {
  name = var.api_name
}

# Creates a proxy resource with the path `{proxy+}`.
resource "aws_api_gateway_resource" "proxy" {
  rest_api_id = aws_api_gateway_rest_api.api.id
  parent_id   = aws_api_gateway_rest_api.api.root_resource_id
  path_part   = "{proxy+}"
}

# Defines the `ANY` method for the proxy resource.
resource "aws_api_gateway_method" "proxy_method" {
  rest_api_id   = aws_api_gateway_rest_api.api.id
  resource_id   = aws_api_gateway_resource.proxy.id
  http_method   = "ANY"
  authorization = "NONE"
}

# Integrates proxy_method and proxy resource with each other
resource "aws_api_gateway_integration" "proxy_integration" {
  rest_api_id             = aws_api_gateway_rest_api.api.id
  resource_id             = aws_api_gateway_resource.proxy.id
  http_method             = aws_api_gateway_method.proxy_method.http_method
  type                    = "AWS_PROXY"
  integration_http_method = "POST"
  uri                     = aws_lambda_function.lambda.invoke_arn
}

resource "aws_api_gateway_deployment" "api_deployment" {
  depends_on = [
    aws_api_gateway_method.proxy_method,
    aws_api_gateway_integration.proxy_integration,
  ]
  rest_api_id = aws_api_gateway_rest_api.api.id
  lifecycle {
    create_before_destroy = true
  }
}

# Defines the deployment stage for the API Gateway.
resource "aws_api_gateway_stage" "api_stage" {
  stage_name    = var.stage
  rest_api_id   = aws_api_gateway_rest_api.api.id
  deployment_id = aws_api_gateway_deployment.api_deployment.id
}

data "aws_caller_identity" "current" {}

variable "api_name" {
  description = "The name of the API Gateway"
}

variable "stage" {
  description = "The deployment stage"
}

variable "lambda_function_name" {
  description = "The name of the Lambda function"
}

Conclusion

Using Terraform to manage your API Gateway and Lambda integration provides a consistent and automated way to handle your infrastructure as code. By following the steps outlined in this blog, you can easily set up a REST API with AWS API Gateway Proxy Integration and Lambda, making your serverless application more manageable and scalable.

This approach not only simplifies the management of your routes but also ensures a more efficient and streamlined serverless architecture. By centralizing your API management with Terraform, you gain greater control and flexibility over your deployments.

Simplify REST API Management with AWS API Gateway Proxy Integration and Lambda

se-piyush — Mon, 01 Jul 2024 14:04:55 +0000

Deploying a REST API using AWS Lambda and API Gateway with proxy integration can greatly streamline your serverless architecture. This method eliminates the need to maintain individual routes in the API Gateway, allowing you to manage all routing within your Express.js application. In this blog post, I’ll guide you through setting up an Express.js application deployed on a Lambda function and integrated with an API Gateway configured as a proxy resource.

Benefits of Using API Gateway as a Proxy

Simplified Route Management: No need to define each route in the API Gateway. All routing is handled within your Express app.
Centralized CORS Management: Manage CORS policies directly in your application code.
Flexibility: Easily add or modify routes without changing the API Gateway configuration.

Step-by-Step Guide to Deploying REST API with Lambda and API Gateway as a Proxy

Step 1: Set Up Your Express.js Application

First, set up your Express.js application. Here’s a basic example:

const express = require('express');
const app = express();
const port = process.env.PORT || 3000;

app.get('/', (req, res) => {
  res.send('Hello World!');
});

app.get('/api/example', (req, res) => {
  res.json({ message: 'This is an example route' });
});

module.exports = app;

Step 2: Create a Lambda Function

Create a Lambda function that will serve as the backend for your API. The function will use the Express.js application.

Handler Function: Create a handler file (index.js).

const awsServerlessExpress = require('aws-serverless-express');
const app = require('./app'); // Path to your Express app
const server = awsServerlessExpress.createServer(app);

exports.handler = (event, context) => {
  awsServerlessExpress.proxy(server, event, context);
};

Deploy Lambda: Package and deploy your Lambda function using the AWS CLI or any CI/CD pipeline.

Step 3: Set Up API Gateway

Create API Gateway: In the AWS Management Console, create a new API Gateway of type REST API.

Create Proxy Resource:
- Go to Resources.
- Create a new resource and enable the proxy resource toggle.
- Name the resource {proxy+}.

Integration with Lambda:
- After creating the proxy resource, select it and click on Actions -> Create Method.
- Choose ANY method and click the checkmark.
- Select Lambda Function as the integration type.
- Enable Use Lambda Proxy Integration.
- Enter the name of your Lambda function.
- Click Save.

Deploy API:
- Go to Stages.
- Create a new stage (e.g., dev).
- Deploy the API to the newly created stage.

Step 4: Testing and Configuring CORS

With the proxy integration, you don't need to configure each route in the API Gateway. Your Lambda function handles all the routing. This setup also simplifies CORS configuration.

Enable CORS in Code:
- Use the cors middleware in your Express app.

const cors = require('cors');
app.use(cors());

Testing:
- Test your API endpoints to ensure they work as expected.
- You can use tools like Postman or curl to send requests to your API.

Conclusion

By using API Gateway as a proxy for your Lambda function, you can deploy a scalable and flexible REST API with minimal configuration overhead. This setup allows you to focus on your application logic and routing within the Express.js framework, while AWS handles the infrastructure scaling and availability.

This method not only simplifies the management of your routes but also ensures a more efficient and streamlined serverless architecture. By centralizing your CORS management and routing logic within your codebase, you gain greater control and flexibility over your API deployments.

Managing Multiple Environments with Terraform Workspaces

se-piyush — Mon, 01 Jul 2024 12:13:50 +0000

Today, I’ll be discussing Terraform workspaces, a powerful feature that helps manage multiple environments using Terraform. Imagine you’ve deployed resources in your cloud using Terraform, and now there’s a requirement to have two environments: a production environment that you just deployed and a development environment. Terraform workspaces can help you manage these environments efficiently.

The Challenge

When you deploy resources using Terraform, a state file (terraform.tfstate) is generated. This state file maintains the configuration of the deployed resources. To manage multiple environments, you need to maintain more than one state file. Terraform workspaces provide a way to manage multiple state files within a single Terraform configuration.

Introducing Terraform Workspaces

Terraform workspaces allow you to manage different sets of infrastructure within the same configuration by maintaining separate state files for each workspace. Here’s how you can use Terraform workspaces to manage your production and development environments.

Steps to Use Terraform Workspaces

Create a Production Workspace: First, create a new workspace for your production environment using the following command:

   terraform workspace new prod

After executing this command, a folder named terraform.tfstate.d is created, containing a subfolder named prod. This folder will hold the state file for the production environment.

Migrate Existing State File to Production Workspace: Move your existing state file to the newly created prod folder:

   mv terraform.tfstate terraform.tfstate.d/prod/terraform.tfstate

This command migrates your current state file to the production workspace.

Create a Development Workspace: Next, create a new workspace for your development environment:

   terraform workspace new dev

This command creates a dev folder inside the terraform.tfstate.d directory. The development environment’s state file will be maintained here.

Select the Development Workspace: To start working in the development environment, select the dev workspace:

   terraform workspace select dev

Deploy Resources in the Development Workspace: Now, you can deploy resources in the development environment. When you run the terraform apply command, Terraform will use the state file in the dev folder to manage the development resources:

   terraform apply

Switching Between Workspaces

To switch between different environments, you simply need to select the appropriate workspace. For example, to switch back to the production environment:

terraform workspace select prod

And to switch to the development environment:

terraform workspace select dev

Benefits of Using Terraform Workspaces

Isolation: Each workspace maintains its own state file, ensuring that the environments are isolated from each other.
Convenience: Workspaces make it easy to switch between different environments without needing separate configuration files or directories.
Scalability: Workspaces allow you to manage multiple environments (e.g., staging, testing) in addition to production and development.

Conclusion

Terraform workspaces provide an elegant solution for managing multiple environments with separate state files. By following the steps outlined above, you can efficiently manage your production and development environments using Terraform workspaces. This approach not only keeps your environments isolated but also makes it easier to switch between them and manage infrastructure as code.

DEV Community: se-piyush

How to Resolve 403 Access Issues When Deploying a SPA on AWS with S3 and CloudFront

The Problem: 403 Access Error After Refreshing Routes

Why It Happened: CloudFront's Path Request Mechanism

The Solution: Redirect All Requests to index.html

Step 1: Update S3 Bucket Settings

Step 2: Configure CloudFront Custom Error Responses

Final Configuration in CloudFront:

Voila! Issue Resolved

Conclusion

How to Host a Vue.js Application Using S3 and CloudFront

Step 1: Create a Private S3 Bucket

Step 2: Upload Your Vue.js Application Bundle

Step 3: Enable Static Website Hosting

Step 4: Create a CloudFront Distribution

Step 5: Configure CloudFront Distribution Settings

Step 6: Create a Custom Error Response for 403 Errors

Conclusion

Common Issues and Resolutions When Setting Up a VPC in AWS

Issue 1: Timeout When Communicating with DynamoDB or Other AWS Services from Lambda/EC2 after Setting Up a VPC Endpoint (Interface)

Problem:

Solution:

Issue 2: Timeout When Connecting to Third-Party APIs Outside AWS (e.g., Stripe, Firebase)

Problem:

Solution:

Issue 3: Unable to Delete Subnets Linked to Lambda, Even After Detaching the VPC

Problem:

Solution:

Conclusion

Creating and Linking VPC Endpoint of Type Interface using Terraform and AWS management console

Steps to Create a VPC Endpoint of Type Interface

Creating a VPC Endpoint of type Interface Using Terraform

Creating a VPC Endpoint of Type Interface Using AWS Management Console

Conclusion

Setting Up VPC and Lambda Function with Terraform

Terraform Code

main.tf File

Conclusion

Setting Up a VPC for Your App Using AWS Management Console

Things to Keep in Mind

Creating a VPC

Integrating VPC with Your Lambda Function

Conclusion

Creating Cross-Account DynamoDB Backups with Terraform

Steps to Configure Cross-Account Backups

Step 1: Create Destination Vault in Backup Account

Step 2: Create Source Vault in Source Account

Step 3: Create IAM Role with Relevant Permissions

Step 4: Create a Backup Plan

Understanding the Cron Schedule

Conclusion

Creating Secure Backups for DynamoDB Tables with Terraform

Step-by-Step Guide to Secure DynamoDB Backups with Terraform

Step 1: Define Your Resources

Step 2: Create Terraform Configuration

Explanation of the Configuration

Working of Cron and Its Configuration

Conclusion

Using Terraform to Manage Resources in Multiple AWS Accounts

Steps to Use Multiple AWS Accounts in Terraform

Step 1: Create AWS CLI Profiles

Step 2: Update the Main Terraform Configuration

Step 3: Update Module Configuration

Full Example of Main Configuration and Module

Explanation

Benefits

Conclusion

Automate REST API Management with Terraform: AWS API Gateway Proxy Integration and Lambda

Overview

Terraform Code Example

Conclusion

Simplify REST API Management with AWS API Gateway Proxy Integration and Lambda

Benefits of Using API Gateway as a Proxy

Step-by-Step Guide to Deploying REST API with Lambda and API Gateway as a Proxy

Step 1: Set Up Your Express.js Application

Step 2: Create a Lambda Function

Step 3: Set Up API Gateway

Step 4: Testing and Configuring CORS

Conclusion

Managing Multiple Environments with Terraform Workspaces

The Solution: Redirect All Requests to `index.html`

`main.tf` File