Automating Cost Optimization Insights with AWS SAM: A Well-Architected Framework Solution

Introduction

Cost management is a core pillar in the AWS Well-Architected Framework. Integrating real-time insights into cloud cost efficiency and over-provisioning can significantly enhance resource utilization. This guide explores deploying a Serverless Application Model (SAM) solution to automate cost optimization insights using AWS EventBridge, API Gateway, Lambda, and DynamoDB. This setup provides cost-focused metrics from AWS Trusted Advisor to help maintain efficient workload configurations in line with AWS Well-Architected Framework best practices.

Architecture Overview

The solution uses AWS services configured as follows:

• AWS SAM: Orchestrates the serverless deployment, managing resource configurations and dependencies.
• AWS EventBridge: Listens for events related to the creation of new Well-Architected Framework workloads and triggers AWS Lambda.
• AWS Lambda: Fetches relevant cost-optimization metrics from AWS Trusted Advisor (such as over-provisioned EC2 instances) and writes them to DynamoDB.
• DynamoDB: Stores cost pillar data for each workload, facilitating quick access and historical analysis.
• API Gateway: Allows for RESTful access to retrieve workload metrics and display cost optimization insights in applications or dashboards.

Step-by-Step Deployment

Step 1: Configure the SAM Template

Define the resources in your SAM template (template.yaml) for API Gateway, Lambda functions, DynamoDB, and EventBridge rules. This file acts as the foundation for automating deployments. An example snippet to trigger Lambda via EventBridge could look like this:

Resources:
  WorkloadEventRule:
    Type: AWS::Events::Rule
    Properties:
      EventPattern:
        source: ["aws.wellarchitected"]
        detail:
          eventName: ["CreateWorkload"]
      Targets:
        - Arn: !GetAtt CostOptimizationLambda.Arn
          Id: "CostOptimizationTarget"

Step 2: Set Up Lambda Function for Cost Optimization Insights

The Lambda function is the heart of the solution. It is triggered by EventBridge when a new Well-Architected Framework workload is created. The Lambda then queries AWS Trusted Advisor for cost optimization insights, such as identifying over-provisioned EC2 instances.

1. Fetch Trusted Advisor Insights: Configure Lambda to call AWS Trusted Advisor API endpoints, particularly for EC2 instances, to pull data on over-provisioned or underutilized resources.
2. Write to DynamoDB: Once the data is fetched, Lambda writes the insights to DynamoDB. The data can be structured with workload IDs as primary keys and specific metrics as attributes, allowing for easy querying and retrieval.

Example Lambda function snippet:

import boto3

def lambda_handler(event, context):
    # Set up Trusted Advisor and DynamoDB clients
    trusted_advisor = boto3.client('support')
    dynamodb = boto3.resource('dynamodb')

    # Fetch cost-related insights from Trusted Advisor
    response = trusted_advisor.describe_trusted_advisor_checks(language='en')

    # Write relevant data to DynamoDB
    table = dynamodb.Table('CostOptimizationTable')
    table.put_item(
        Item={
            'WorkloadID': event['detail']['workloadId'],
            'OptimizationMetrics': response['checks']
        }
    )
    return {"status": "Data saved"}

Step 3: Define DynamoDB Table for Storing Metrics

In your SAM template, define a DynamoDB table to store cost metrics. This table serves as a persistent repository for historical cost pillar data, making it accessible for analytics and reporting.

CostOptimizationTable:
  Type: AWS::DynamoDB::Table
  Properties:
    TableName: "CostOptimizationMetrics"
    AttributeDefinitions:
      - AttributeName: "WorkloadID"
        AttributeType: "S"
    KeySchema:
      - AttributeName: "WorkloadID"
        KeyType: "HASH"
    BillingMode: PAY_PER_REQUEST

Step 4: Expose Data Through API Gateway

Set up an API Gateway to provide access to the cost pillar data. This API can be used by external applications or dashboards to display the cost optimization insights generated by Trusted Advisor and stored in DynamoDB.

Example SAM template configuration for API Gateway:

CostOptimizationApi:
  Type: AWS::Serverless::Api
  Properties:
    StageName: "prod"
    DefinitionBody:
      paths:
        /workload/{id}:
          get:
            x-amazon-apigateway-integration:
              uri: !Sub arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${CostOptimizationLambda.Arn}/invocations
              httpMethod: POST

Testing and Monitoring

• Test Lambda and EventBridge Integration: Create a new Well-Architected Framework workload to see if the Lambda is triggered and if data is written to DynamoDB.
• Access Data via API Gateway: Use the API endpoint to retrieve cost metrics for verification.
• Monitor with CloudWatch Logs: Check logs for Lambda execution to ensure there are no errors in fetching Trusted Advisor data or writing to DynamoDB.

Benefits of the Solution

This automated solution aligns with AWS Well-Architected Framework best practices by providing continuous visibility into cost optimization metrics. Key benefits include:

• Real-Time Cost Insights: Automatically pulls cost-related insights for each workload, helping identify potential savings on over-provisioned resources.
• Improved Resource Utilization: Regular updates ensure that workloads stay optimized based on Trusted Advisor’s recommendations.
• Scalable Solution: Serverless architecture scales to handle multiple workloads and large amounts of data without manual intervention. Refer below screenshot to know how above solution will provide insights.

Conclusion

Automating cost insights with AWS SAM, Lambda, EventBridge, Trusted Advisor, and DynamoDB provides a proactive approach to managing cloud spending. This solution demonstrates how AWS serverless services can integrate seamlessly to drive compliance and cost optimization in line with Well-Architected best practices. With these capabilities, your team can more easily maintain efficient, cost-effective workloads in AWS.

This solution enables continuous cost optimization insights, making it a practical addition to any cloud financial management strategy.