Modern Directory Architecture: AWS Serverless with Containerized Frontend

The purpose of this project was to create a solution for a hypothetical company looking to host their directory application in the cloud. If needed to be private Cognito could be implemented to provide authentication flow (Covered in another post). The solution consists of the following:

• Frontend hosted in ECS containers behind an Application Load Balancer.
• Backend hosted in RDS (MySQL).
• API Gateway + Lambda to facilitate communication between the frontend and backend
• AWS Secrets Manager to store DB credentials
• Optionally connect to Route53 & Amazon Certificate Manager (DNS & HTTPS flow covered in a another post)

---

HLD of proposed architecture.

Application Flow

1. User makes request the Application Load Balancer and subsequently routes to one of the containers.
2. Users sends query requests through the application which gets passed to API gateway and subsequently Lambda.
3. Lambda queries Secrets Manager to retrieve database secrets and shortly after queries the database to complete the user request.
4. Data is sent back to the container/user.

---

Components

VPC

1. Create a VPC with 2 private subnets.
2. Create an Interface Endpoint for Secrets Manager

Security Groups

There's likely an optimal order of creating these but make all of them and add the rules after. Also, Try give them identifiable names - It can become quite tiresome figuring what sg1, sg2, sg3 etc.. are for.

Lambda

• Inbound: 443 from secrets endpoint
• Outbound: 443 to Secrets Interface Endpoint SG, 3306 to RDS SG

Containers

• Inbound: 443 and/or 80 from ALB SG
• Outbound: All (or restrict to your needs)

ALB

• Inbound: 443 & 80 from 0.0.0.0 or defined range
• Outbound: 443 & 80 to containers SG

Interface Endpoint

• Inbound: 443 from Lambda SG
• Outbound: None required

RDS

• Inbound: 3306 from Lambda SG
• Outbound: None

Lambda

Due to the number of requests my application required, I created a single lambda_function (single lambda_handler and several imported sub-functions). As such, my lambda_function folder looked something like this:

company_requests/
├── pymysql/
├── PyMySQL-1.1.1.dist-info/
├── lambda_function.py
├── deleteEmployee.py
├── getAll.py
├── getAllDepartments.py
├── getAllLocations.py
...

Within your development environment (or within Lambda), Create a folder with all of your required Python functions for their associated requests and be sure to include any dependencies. Zip the folder (I used 7zip) and then upload it to Lambda.
Heres a brief snippet of my lambda_function.py:

def lambda_handler(event, context):

  execution_start_time = time.time()

  # Retrieving secrets from the function defined above.
  secret = get_secret()
  username = secret["username"]
  password = secret["password"]
  db_host = "<your-rds-endpoint"
  db_name = "<your-table-name"

  creds = {"user":username, "pass":password, "db_host":db_host, "db_name":db_name}

  # Determine which function to call based on the HTTP method and path
  http_method = event.get('httpMethod')
  resource = event.get('resource')

  # Personnel operations
  if resource == "/all" and http_method =="GET":
      return get_all(event, creds, execution_start_time)
  elif resource == "/personnel" and http_method == "GET":
      return get_all_personnel(event, creds, execution_start_time)
  ...

Function to retrieve secrets from Secrets Manager (also in lambda_function.py.

import boto
import json

def get_secret():
     secret_name = <your_secret_name>
     region_name = <your_region>

     # Create a Secrets Manager client
     session = boto3.session.Session()
     client = session.client(
         service_name='secretsmanager',
         region_name=region_name
     )

     try:
         get_secret_value_response = client.get_secret_value(
             SecretId=secret_name
         )
     except ClientError as e:
         # Handle exceptions
         raise e

     # Decrypts secret using the associated KMS key.
     secret = get_secret_value_response['SecretString']
     return json.loads(secret)

1. Once you've created your Lambda function, Go to IAM and assign it the following policies:
   1. RDS read access - Creating an inline policy can allow for stronger enforcement of least privilege.
   2. EC2 Create Network Interface - Needed for placing Lambda inside your VPC.
   3. Secrets Manager read secret - Again create an inline policy only allowing access to the necessary secret.
2. Go into your Lambda function’s network configuration and place it inside the same subnet/s of your VPC that hold your RDS instances. Attach the security group you created earlier.
3. You may need to increase the functions timeout to ensure it has time to complete requests. This can be done in general configuration.

API Gateway

1. Create a new API gateway with all of the required resource paths and methods. My resource tree looked something like: / /all /departments /locations /personnel -GET -POST ...
2. When creating each resource ensure Lambda integration is ticked on all of them and link your Lambda function.
3. Make sure to enable CORS on each resource path too
4. Once done, Deploy your api.

ECR

This step assumes you have already containerised your application (Steps on how to do so are covered in a different post).

1. Create a new repository and follow the provided push steps
2. Once uploaded take note of the image URI.

ECS

1. Create a task definition using the URI of the image you uploaded
   • Ensure you select the same ports used in your container.
2. Create a service
   1. Select the task definition you just made, Give the service a name and specify 2 tasks.
   2. Open the networking tab and select your VPC. Select your private subnets and assign the appropriate security group.
   3. Open the load balancing tab and create a new ALB with the containers as targets.
   4. After creating go to the load balancer settings in EC2 and assign the appropriate security group.
   5. If implementing ACM for HTTPS specify the certificate here and add HTTPS traffic to the container and ALB security groups.

RDS

1. Create a new DB with whichever engine you require and place it in your private subnets.
2. Assign it the appropriate security group.

Secrets Manager

1. Create a new secret store and save your RDS DB credentials there.

---

After having to create around 12 API resource paths I realised how useful infrastructure as code (IaS) can be in situation like this so in later posts I will cover defining this project (and likely others) in Terraform.

Although tutorial/project write-up achieves the intended outcome, Im sure there's several tweaks that can be made to make it more streamline and improve security so if this post happens to catch any attention and you have a suggestion please do share.

Yuh.