DEV Community

Cover image for Automate Deployment Testing | 🏗️ Build A Version Management System
AWS Community Builders profile image Ntombizakhona Mabaso
Ntombizakhona Mabaso Subscriber for AWS Community Builders

Posted on

Automate Deployment Testing | 🏗️ Build A Version Management System

#aws #certification #cloud #developer

Exam Guide: Developer - Associate
🏗️ Domain 3: Deployment
📘 Task 3: Automate Deployment Testing

This task is about automating tests as part of your deployment workflow: managing Lambda versions and aliases, using weighted aliases for canary deployments, deploying and reviewing IaC templates with change sets, and creating test events for automated validation. Understand version management, safe deployment strategies, and how to preview infrastructure changes before applying them.

📘 Concepts

Lambda Versions vs Aliases

Concept What It Is Mutable? Use Case
$LATEST The default, mutable version of your function Yes Development and testing only
Published version (v1, v2, v3...) An immutable snapshot of code + configuration No Production references, rollback targets
Alias A named pointer to a specific version Yes (can be repointed) Environment mapping (dev, staging, prod)

💡 $LATEST is mutable: every code update changes it. Published versions are immutable: once published, the code and configuration can't change. Aliases are mutable pointers to immutable versions. Event source mappings in production should reference aliases, never $LATEST.

$LATEST vs Published Versions

Aspect $LATEST Published Version
Code Changes Updated on every deploy Frozen at publish time
Configuration Changes Updated immediately Frozen at publish time
Concurrency Settings Shared across all unpublished invocations Can have provisioned concurrency per version
ARN Format arn:aws:lambda:region:account:function:name arn:aws:lambda:region:account:function:name:1
Use In Production No (too risky) Yes (immutable and predictable)

Weighted Aliases for Canary Deployments

An alias can split traffic between two versions using routing configuration:

Deployment Type Traffic Split SAM Config Use Case
AllAtOnce 100% to new version immediately Type: AllAtOnce Non-critical functions, dev environments
Canary X% to new, then 100% after interval Type: Canary10Percent5Minutes Test with small traffic before full rollout
Linear Increase by X% every N minutes Type: Linear10PercentEvery1Minute Gradual rollout with monitoring

💡 Weighted aliases route a percentage of invocations to a secondary version. If the canary version triggers a CloudWatch alarm, CodeDeploy automatically rolls back to the previous version. SAM's AutoPublishAlias + DeploymentPreference handles this automatically.

IaC Templates

SAM vs CloudFormation

Feature SAM CloudFormation
What It Is _Extension of CloudFormation with serverless shortcuts _ AWS's core IaC service
Template Format YAML/JSON with Transform: AWS::Serverless-2016-10-31 YAML/JSON
Serverless Resources AWS::Serverless::Function, AWS::Serverless::Api, etc. Must define each resource individually (Lambda, API GW, IAM roles)
Deployment sam deploy (wraps CloudFormation) aws cloudformation deploy
Local Testing sam local invoke, sam local start-api Not available
Policy Templates DynamoDBCrudPolicy, S3ReadPolicy, etc. Write full IAM policies manually

CloudFormation Change Sets

Concept What It Does
Change Set A preview of what CloudFormation will add, modify, or delete before you apply it
Create Generates the change set without applying anything
Review Shows each resource and whether it will be Added, Modified, or Removed
Execute Applies the changes to the stack
Delete Discards the change set without applying
Replacement Indicates a resource will be deleted and recreated (data loss risk!)

💡 Change sets are critical for production deployments. If a change set shows "Replacement" for a DynamoDB table or RDS instance, that means the resource will be deleted and recreated — you'll lose data. Always review change sets before executing in production. SAM's confirm_changeset = true in samconfig.toml forces this review.

Amazon Q Developer for Test Generation

Capability What It Does
Unit Test Generation Generates pytest/unittest tests from your function code
Mock Setup Creates moto and unittest.mock boilerplate
Edge Case Detection Identifies missing input validation, error handling gaps
Test Coverage Suggests tests for happy path, error cases, and boundary conditions

💡Know that Amazon Q Developer can generate tests and suggest improvements. But, just use Kiro. 👇

⚠️ Amazon Q Developer end-of-support announcement ⚠️

🏗️ Build A Version Management System

Build a Version Management System that demonstrates deployment testing automation:

  • Published Lambda versions with immutable snapshots
  • Aliases pointing to specific versions for environment management
  • Weighted aliases routing traffic for canary testing
  • CloudFormation change sets reviewed and executed through the console
  • Lambda test events created and used for automated validation

Prerequisites

Part I

Publish Lambda Versions

Step 01: Create the Lambda Function

Step 01.1: Open the Lambda console → Create function

  • Function name: VersionedOrderAPI
  • Runtime: Python 3.13

Click Create function

Step 01.2: Paste this code

import json

VERSION = "1.0.0"

def lambda_handler(event, context):
    """
    Version 1 of the Order API.
    Each published version is an immutable snapshot of this code.
    """
    return {
        'statusCode': 200,
        'headers': {'Content-Type': 'application/json'},
        'body': json.dumps({
            'message': 'Order API',
            'version': VERSION,
            'functionVersion': context.function_version,
            'action': event.get('action', 'none')
        })
    }
Enter fullscreen mode Exit fullscreen mode

Step 01.3: Click Deploy

Step 02: Publish Version 1

Step 02.1: Click Actions ▼ → Publish new version

Step 02.2: Version description - optional: v1 - Initial release

Click Publish

💡 You're now viewing Version 1: notice the Function ARN includes :1 at the end

Step 02.3: Click Version 1 ▼ dropdown → select Qualifier: VersionedOrderAPI to go back to the editable version

Step 03: Update and Publish Version 2

Step 03.1: Back on VersionedOrderAPI, update the code:

import json

VERSION = "2.0.0"

def lambda_handler(event, context):
    """
    Version 2 of the Order API — added order validation.
    This version is a new immutable snapshot.
    """
    action = event.get('action', 'none')

    # New in v2: input validation
    if action == 'create' and not event.get('customerId'):
        return {
            'statusCode': 400,
            'body': json.dumps({'error': 'customerId is required'})
        }

    return {
        'statusCode': 200,
        'headers': {'Content-Type': 'application/json'},
        'body': json.dumps({
            'message': 'Order API',
            'version': VERSION,
            'functionVersion': context.function_version,
            'action': action,
            'improvement': 'Added input validation'
        })
    }
Enter fullscreen mode Exit fullscreen mode

Step 03.2: Click Deploy

Step 03.3: Click Actions ▼ → Publish new version

Step 03.4 Version description - optional: v2 - Added input validation

Click Publish

Step 04: Verify Versions

Step 04.1 Click the Version 2 ▼ dropdown, you should see:

  • Qualifier: VersionOrderAPI — the mutable working copy
  • Version: 2 — v1 initial release (immutable)
  • Version: 1 — v2 with validation (immutable)

💡 Each published version gets its own ARN (function-name:1, function-name:2). The code and configuration are frozen. You can't edit a published version. You must publish a new one. This is what makes rollbacks possible: just point the alias back to the previous version.

Part II

Create Aliases Pointing to Versions

Step 01: Create the Production Alias

Step 01.1: On the VersionedOrderAPI function page, click the Aliases tab

Step 01.2: Click Create alias

Step 01.3: Create alias

  • Name: prod
  • Description - optional: Production traffic
  • Version: 1

Click Save

Step 02: Create the Staging Alias

Step 02.1: Click Create alias again

Step 02.2: Create alias

  • Name: staging
  • Description - optional: Staging and QA testing
  • Version: 2

Click Save

Step 03: Create the Dev Alias

Step 03.1: Click Create alias again

Step 02.2: Create alias

  • Name: dev
  • Description - optional: Development and testing
  • Version: $LATEST

Click Save

Step 04: Test Each Alias

Step 04.1: Click on the prod alias

Step 04.2: Go to the Test tab → create a new event:

{"action": "create", "customerId": "CUST-001"}
Enter fullscreen mode Exit fullscreen mode

Step 04.3: Click Test

💡The response should show "version": "1.0.0" and "functionVersion": "1"

Step 04.4: Go back and click on the staging alias

Step 04.5: Run the same test

💡The response should show "version": "2.0.0" and "functionVersion": "2"

Step 05: Update an Alias to Point to a New Version

Step 05.1: Click on the prod alias → Configuration → Edit

Step 05.2: Change Version to 2

Click Save

⚠️ Now production traffic uses version 2. If something goes wrong, you can repoint prod back to version 1 instantly.

💡Aliases decouple consumers from specific versions. Event source mappings, API Gateway integrations, and other services should reference the alias ARN (function-name:prod), not a version number. This lets you promote versions by updating the alias, not reconfiguring every consumer.

Part III

Set Up Weighted Aliases for Canary Testing

Step 01: Configure Traffic Splitting

Step 01.1: Click on the prod alias → Configuration → Edit

  • Version: 1 (current stable)

Step 01.2: Under Weighted alias ▼

  • Additional version - optional: 2
  • Weight 10 (meaning 10% of traffic goes to v2)

Click Save

💡 Now 90% of invocations through the prod alias go to version 1, and 10% go to version 2.

Step 02: Test the Traffic Split

Step 02.1: Go to the Test tab on the prod alias

Step 02.2: Run the test event multiple times (at least 10 times)

Step 02.3: Observe the functionVersion in the responses. Most will show 1, some will show 2

Step 03: Monitor the Canary

Step 03.1: Open the CloudWatch console → Metrics → Lambda

Step 03.2: Look at By Function Name metrics

💡 You can see invocation counts and error rates per version
If version 2 has higher error rates, revert by removing the weighted configuration

Step 04: Remove the Canary (Promote or Rollback)

Step 04.1: To promote v2 to 100%:
Edit the prod alias → set Version to 2 → remove the weighted configuration → Save

Step 04.2: To rollback to v1:
Edit the prod alias → set Version to 1 → remove the weighted configuration → Save

💡 In SAM, weighted aliases are managed automatically with DeploymentPreference. SAM creates a CodeDeploy deployment that shifts traffic gradually and monitors CloudWatch alarms. If an alarm fires, CodeDeploy rolls back automatically. For example, Canary10Percent5Minutes and Linear10PercentEvery1Minute.

Part IV

Create and Review CloudFormation Change Sets

Step 01: Create a CloudFormation Stack

Step 01.1: Open the CloudFormation console

Step 01.2: Click Create stack ▼ → With new resources (standard)

Step 01.3: Create stack

  • Prepare template: Choose an existing template
  • Template source: Upload a template file

Step 01.4: Create and upload this template

AWSTemplateFormatVersion: '2010-09-09'
Description: Version Management Demo Stack

Parameters:
  Stage:
    Type: String
    Default: dev
    AllowedValues: [dev, staging, prod]

Resources:
  OrdersTable:
    Type: AWS::DynamoDB::Table
    Properties:
      TableName: !Sub "${Stage}-version-demo-orders"
      BillingMode: PAY_PER_REQUEST
      KeySchema:
        - AttributeName: PK
          KeyType: HASH
        - AttributeName: SK
          KeyType: RANGE
      AttributeDefinitions:
        - AttributeName: PK
          AttributeType: S
        - AttributeName: SK
          AttributeType: S
      Tags:
        - Key: Environment
          Value: !Ref Stage

  ConfigTable:
    Type: AWS::DynamoDB::Table
    Properties:
      TableName: !Sub "${Stage}-version-demo-config"
      BillingMode: PAY_PER_REQUEST
      KeySchema:
        - AttributeName: configKey
          KeyType: HASH
      AttributeDefinitions:
        - AttributeName: configKey
          AttributeType: S
Enter fullscreen mode Exit fullscreen mode

Step 01.5: Click Next

Step 01.6: Specify stack details:

  • Stack name: version-demo-dev
  • Stage: dev

Step 01.7: Click Next → Next → Submit

⚠️ Wait for the stack to reach CREATE_COMPLETE

Step 02: Create a Change Set to Add a Resource

Step 02.1: Click on the version-demo-dev stack

Step 02.2: Click Stack actions ▼ → Create change set

Step 02.3: Create change set for version-demo-dev

  • Change set type: Standard change set
  • Prepare template: Replace existing template
  • Template source: Upload a template file

Step 02.4: Upload an updated template that adds a new resource (add this to the Resources section):

  AuditTable:
    Type: AWS::DynamoDB::Table
    Properties:
      TableName: !Sub "${Stage}-version-demo-audit"
      BillingMode: PAY_PER_REQUEST
      KeySchema:
        - AttributeName: PK
          KeyType: HASH
        - AttributeName: SK
          KeyType: RANGE
      AttributeDefinitions:
        - AttributeName: PK
          AttributeType: S
        - AttributeName: SK
          AttributeType: S
Enter fullscreen mode Exit fullscreen mode

Step 02.5 Click Next

Step 02.6: Specify change set details

Change set name: add-audit-table

Step 02.7: Click Next → Next → Create change set

Step 03: Review the Change Set

Step 03.1: Click on the add-audit-table change set

Step 03.2: Review the Changes section

  • AuditTable → Action: Add → Resource type: AWS::DynamoDB::Table
  • OrdersTable | no change
  • ConfigTable | no change

💡 This is safe. Only adding a new resource

Step 04: Execute the Change Set

Click Execute change set → Click Execute change set to confirm

⚠️ Wait for the stack to reach UPDATE_COMPLETE

Go to the Resources tab. You should now see three DynamoDB tables

💡 Always check for "Replacement" in change sets. If a change set shows Replacement for a DynamoDB table, that table will be deleted and recreated. All data is lost. Common causes: changing the table name, changing the key schema, or changing certain properties that require replacement. Use DeletionPolicy: Retain on critical resources to prevent accidental deletion.

Part V

Create Lambda Test Events and Automate Testing

Step 01: Create Shareable Test Events

Step 01.1: Open the Lambda console → click on VersionedOrderAPI

Step 01.2: Go to the Test tab

Step 01.3: Click Create new event

  • Event name: CreateOrderValid
  • Event sharing settings: Shareable (visible to all IAM users)
  • Event JSON: 
{
  "action": "create",
  "customerId": "CUST-001",
  "items": [{"productId": "PROD-A", "quantity": 2}]
}
Enter fullscreen mode Exit fullscreen mode

Step 01.4: Click Save

Step 01.5: Create another test event:

  • Event name: CreateOrderMissinCustomer
  • Event sharing settings: Shareable
  • Event JSON: 
{
  "action": "create",
  "items": [{"productId": "PROD-A", "quantity": 1}]
}
Enter fullscreen mode Exit fullscreen mode

Step 01.6: Click Save

Step 01.7: Create a third test event

  • Event name: GetOrderStatus
  • Event sharing settings: Shareable
  • Event JSON: 
{
  "action": "status",
  "orderId": "ORD-2024-001"
}
Enter fullscreen mode Exit fullscreen mode

Step 01.8: Click Save

Step 02: Run All Test Events

Step 02.1: Select CreateOrderValid from the dropdown → click Test

  • Expected: 200 with version info

Step 02.2: Select CreateOrderMissingCustomer → click Test

  • Expected: 400 with validation error (on version 2)

Step 02.3 Select GetOrderStatus → click Test

  • Expected: 200 with status info

Step 03: Automate Testing in a CI/CD Pipeline

In a real pipeline, you'd automate these tests in a CodeBuild buildspec:

# buildspec.yml — automated testing in CI/CD
version: 0.2
phases:
  install:
    commands:
      - pip install pytest moto boto3
  build:
    commands:
      - sam build
  post_build:
    commands:
      # Run unit tests
      - python -m pytest tests/unit/ -v --junitxml=test-results/unit.xml

      # Deploy to test environment
      - sam deploy --config-env test --no-confirm-changeset

      # Get the deployed API URL from stack outputs
      - |
        API_URL=$(aws cloudformation describe-stacks \
          --stack-name my-app-test \
          --query 'Stacks[0].Outputs[?OutputKey==`ApiUrl`].OutputValue' \
          --output text)

      # Run integration tests against the deployed stack
      - API_URL=$API_URL python -m pytest tests/integration/ -v --junitxml=test-results/integration.xml

      # Invoke Lambda directly with test events
      - |
        aws lambda invoke \
          --function-name VersionedOrderAPI:staging \
          --payload '{"action":"create","customerId":"CUST-001"}' \
          response.json
        cat response.json

reports:
  test-reports:
    files:
      - "**/*.xml"
    base-directory: test-results
Enter fullscreen mode Exit fullscreen mode

💡 Shareable test events are visible to all IAM users with access to the function which is useful for team collaboration. Private test events are only visible to the creator. In CI/CD, use aws lambda invoke with the --payload flag to run automated tests against specific aliases. Use --qualifier to target a specific version or alias.

🏗️ What You Built | 📘 Exam Concepts Recap

What You Built Exam Concept
Published immutable Lambda versions (v1, v2) Versions freeze code + config. $LATEST is mutable
Created prod/staging/dev aliases Aliases as mutable pointers for environment mapping
Repointed the prod alias to a new version Instant promotion and rollback via aliases
Configured a weighted alias (90/10 split) Canary deployments with traffic shifting
Created and reviewed a CloudFormation change set Previewing infra changes before applying
Watched for "Replacement" in the change set Avoiding accidental resource deletion / data loss
Created shareable Lambda test events Team-wide test standardization
Automated tests in a CodeBuild buildspec Test automation in CI/CD with aws lambda invoke

⚠️ Clean Up Protocol

  1. Lambda → Delete VersionedOrderAPI (this also deletes all versions and aliases)
  2. CloudFormation → Delete the version-demo-dev stack (this deletes all three DynamoDB tables)
  3. IAM → Delete Lambda execution roles
  4. CloudWatch → Delete log groups

Key Takeaways

  1. Versions are immutable snapshots of code + configuration.
  2. Aliases are mutable pointers to versions.
  3. $LATEST is mutable. Don't use it in production.
  4. Weighted aliases split traffic between two versions for canary deployments. SAM's DeploymentPreference automates this with CodeDeploy.
  5. Canary10Percent5Minutes sends 10% to the new version, waits 5 minutes, then shifts 100%. Linear10PercentEvery1Minute increases by 10% each minute.
  6. Change sets preview CloudFormation changes before applying. Watch for "Replacement". Tt means resource deletion and recreation.
  7. Event source mappings should reference aliases, not $LATEST or version numbers, so you can promote versions by updating the alias.
  8. Shareable test events in the Lambda console are visible to all IAM users. Use them for team-wide test standardization.
  9. Amazon Q Developer can generate unit tests, mock setup, and edge case tests from your function code. > ⚠️ Amazon Q Developer end-of-support announcement ⚠️
  10. In CI/CD, use aws lambda invoke --qualifier alias-name to test specific versions and aws cloudformation describe-stacks to get deployed resource URLs for integration tests.

Additional Resources

⚠️ Amazon Q Developer end-of-support announcement ⚠️

🏗️

Top comments (0)