DEV Community: Ntombizakhona Mabaso

Instrument Code For Observability | 🏗️ Build An Observable Order Service

Ntombizakhona Mabaso — Tue, 14 Jul 2026 20:20:30 +0000

Exam Guide: Developer - Associate
🏗️ Domain 4: Troubleshooting And Optimization
📘 Task 2: Instrument Code For Observability

Observability goes beyond monitoring: it's about understanding the internal state of your system from its external outputs. This task tests your ability to instrument applications with AWS X-Ray for distributed tracing, implement structured logging with correlation IDs, set up CloudWatch alarms and notifications, and build health check endpoints. X-Ray concepts (traces, segments, subsegments, annotations vs metadata), structured logging best practices, and alarm configuration.

📘Concepts

Logging vs Monitoring vs Observability

These three concepts build on each other.

Aspect	Logging	Monitoring	Observability
What It Is	Recording discrete events	Tracking predefined metrics over time	Understanding system behaviour from outputs
Question It Answers	What happened?	Is the system healthy?	Why is the system behaving this way?
Data Type	Text entries (structured or unstructured)	Numeric time-series data	Logs + metrics + traces combined
Proactive vs Reactive	Reactive. Search after an issue	Proactive. Alerts before users notice	Both. Explore unknown unknowns
AWS Service(s)	CloudWatch Logs	CloudWatch Metrics, Alarms	X-Ray, CloudWatch Logs Insights, ServiceLens
`Example`	`ERROR: DynamoDB timeout on table Orders`	`Lambda Duration > 5s for 3 consecutive periods`	Trace shows DynamoDB call took 4.8s due to throttling

The Three Pillars Of Observability

Pillar	What It Provides	AWS Service
Logs	Detailed event records for debugging	CloudWatch Logs
Metrics	Numeric measurements for alerting and trending	CloudWatch Metrics
Traces	Request flow across distributed services	AWS X-Ray

Monitoring tells you something is wrong. Observability tells you why. Metrics show a problem (high latency) but you need traces to find the root cause (a specific downstream service is slow)

AWS X-Ray Concepts

💡 X-Ray is the answer to "How do you find which service is causing latency?"

X-Ray provides distributed tracing. It tracks requests as they flow through your application across multiple services.

Concept	What It Is	Example
Trace	The complete journey of a single request through all services	API Gateway → Lambda → DynamoDB → SNS
Segment	A unit of work done by a single service	The Lambda function's execution
Subsegment	A subdivision of a segment for more detail	A specific DynamoDB call within the Lambda function
Trace ID	Unique identifier for the entire trace	`1-5f84c7a1-abcdef1234567890`
Sampling	Controls what percentage of requests are traced	Default: first request each second + 5% of additional
Service Map	Visual representation of your architecture and request flow	Shows services as nodes with latency and error rates
Trace Map	Visual representation of a single trace	Shows the path of one request through services

X-Ray Annotations vs Metadata

Feature	Annotations	Metadata
What It Stores	Key-value pairs (string, number, boolean)	Any JSON-serializable object
Indexed	Yes. Searchable in the X-Ray console	No. Not searchable
Use For	Filtering and grouping traces	Storing detailed debug data
Max Size	50 annotations per trace	No practical limit
Example	`userId: "CUST-001"`, `orderType: "premium"`	Full request/response bodies, stack traces
Search Syntax	`annotation.userId = "CUST-001"`	Cannot search by metadata

Use annotations for values you want to search and filter by user IDs, order types, environment names.
Use metadata for detailed debugging data you only need when examining a specific trace.
Annotations are indexed and searchable, metadata is not.

💡"How do you find all traces for a specific user?" The answer is annotations.

X-Ray Sampling Rules

Setting	`Default`	Purpose
Reservoir	`1` per second	Guaranteed minimum traces per second
Rate	`5%` (0.05)	Percentage of additional requests to trace
Service Name	`*` (all)	Which services the rule applies to
HTTP Method	`*` (all)	Filter by GET, POST, etc.
URL Path	`*` (all)	Filter by specific paths
Priority	`10000` (default)	Lower number = higher priority

Custom Sampling Rule Example:

Trace 100% of requests to /api/orders (high-value endpoint)
Trace 1% of requests to /api/health (noisy, low-value)

💡 The default sampling rule traces the first request each second plus 5% of additional requests. This keeps costs manageable for high-traffic applications. You can create custom rules to trace more (or fewer) requests for specific paths or services. Sampling rules are evaluated in priority order. Lower priority number wins.

X-Ray Integration with AWS Services

Service	How to Enable	What Gets Traced
Lambda	Toggle "Active tracing" in function configuration	Function execution, SDK calls
API Gateway	Enable X-Ray tracing on the stage	API request processing
ECS/Fargate	Run X-Ray daemon as sidecar container	Application SDK calls
EC2	Install and run X-Ray daemon	Application SDK calls
Elastic Beanstalk	Enable in `.ebextensions` or console	Application SDK calls
SNS/SQS	Automatic when upstream service is traced	Message propagation

💡 For Lambda, enabling "Active tracing" is all you need for basic traces. To add custom subsegments and annotations, you need the X-Ray SDK in your code. The X-Ray daemon runs automatically in Lambda yiu don't need to install it. For EC2 and ECS, you must install and run the daemon yourself.

Structured Logging Best Practices

Practice	Why It Matters	Example
Use JSON Format	Machine-parseable, works with Logs Insights	`{"level":"ERROR","message":"timeout"}`
Include Correlation ID	Track requests across services	`{"correlationId":"abc-123","service":"orders"}`
Include Request Context	Debug without searching multiple logs	`{"requestId":"req-1","userId":"user-5"}`
Use Log Levels	Filter by severity	DEBUG, INFO, WARN, ERROR, FATAL
Avoid Sensitive Data	Security and compliance	Never log passwords, tokens, PII
Include timestamps	Accurate event ordering	ISO 8601 format: `2024-01-15T10:30:00Z`

Correlation IDs for Distributed Tracing

A Correlation ID is a unique identifier that follows a request across all services:

Client → API Gateway → Lambda A → SQS → Lambda B → DynamoDB
         correlationId: "abc-123" flows through every service

Where to Set It	How
API Gateway	Pass `X-Correlation-Id` header or generate in mapping template
Lambda	Read from event headers or generate if missing
SQS	Include in message attributes
SNS	Include in message attributes
DynamoDB	Not applicable (use in logs only)

💡 Correlation IDs complement X-Ray traces. X-Ray gives you the visual trace map. Correlation IDs let you search logs across services for the same request. Use both together: add the correlation ID as an X-Ray annotation so you can find the trace from the log and vice versa.

CloudWatch Alarms

Alarm Component	What It Does	Options
Metric	What you're monitoring	Any CloudWatch metric (built-in or custom)
Statistic	How to aggregate data points	Average, Sum, Min, Max, p99, SampleCount
Period	Time window for each data point	10s, 30s, 60s, 300s, etc.
Evaluation periods	How many periods to evaluate	1-100
Datapoints to alarm	How many periods must breach threshold	1 to evaluation periods
Threshold	The value that triggers the alarm	Any number
Comparison	How to compare metric to threshold	>, >=, <, <=
Actions	What happens when alarm fires	SNS notification, Auto Scaling, EC2 action, Lambda
Missing data	How to treat gaps in data	missing, notBreaching, breaching, ignore

Alarm States:

State	Meaning
OK	Metric is within the threshold
ALARM	Metric has breached the threshold
INSUFFICIENT_DATA	Not enough data to determine state

Composite Alarms

Feature	Standard Alarm	Composite Alarm
Based On	A single metric	Multiple alarms combined with AND/OR/NOT
Use Case	Monitor one metric	Reduce alarm noise, complex conditions
Example	Error rate > 5%	Error rate > 5% AND latency > 2s
Actions	SNS, Auto Scaling, EC2, Lambda	SNS, Lambda

💡 Composite alarms reduce alert fatigue. Instead of getting separate alerts for high errors and high latency, create a composite alarm that only fires when both conditions are true. Use AND for "all conditions must be true" and OR for "any condition triggers." Composite alarms can suppress actions on child alarms to prevent duplicate notifications.

Health Check Patterns

Pattern	What It Checks	Response
Shallow Health Check	Is the service running?	200 OK (no dependency checks)
Deep Health Check	Are all dependencies healthy?	200 OK with dependency status
Liveness Probe	I_s the process alive?_	200 OK (minimal check)
Readiness Probe	Can the service handle requests?	200 OK only when fully initialized

Deep Health Check Response Example:

{
  "status": "healthy",
  "timestamp": "2026-07-20T10:30:00Z",
  "dependencies": {
    "dynamodb": {"status": "healthy", "latency": "12ms"},
    "sqs": {"status": "healthy", "latency": "8ms"},
    "external-api": {"status": "degraded", "latency": "2500ms"}
  }
}

💡Health checks should have their own endpoint (e.g., /health) that doesn't require authentication. Shallow checks are fast and cheap: use them for load balancer health checks. Deep checks verify dependencies: use them for monitoring dashboards. Don't make deep health checks too expensive. Cache dependency status for a few seconds.

🏗️ Build An Observable Order Service

Build an Observable Order Service from scratch using the AWS Console:

A Lambda function with X-Ray active tracing enabled
Custom X-Ray subsegments and annotations in the code
Structured JSON logging with correlation IDs
CloudWatch alarms with SNS email notifications
A health check endpoint that verifies dependencies
Traces visible in the X-Ray console service map

Prerequisites

Part I

Create a Lambda Function with X-Ray Tracing

Step 01: Create the Function

Open the Lambda console → Create function

Function name: ObservableOrderService
Runtime: Python 3.13

→ Click Create function

Step 02: Enable X-Ray Active Tracing

Go to Configuration → Monitoring and operations tools → Edit

→ Under CloudWatch Application Signals and AWS X-Ray → ✔ Lambda services traces

→ Click Save

Step 03: Add the X-Ray SDK Layer

⚠️ The instrumented code below uses aws_xray_sdk for custom subsegments and annotations. That package is **not in the Lambda runtime (and the AWSSDKPandas layer doesn't include it), so you'll get No module named 'aws_xray_sdk' unless you add it. Build a layer in CloudShell (no local setup needed):**

mkdir -p xray-layer/python
pip install aws-xray-sdk -t xray-layer/python/
cd xray-layer && zip -r ../xray-sdk-layer.zip python/ && cd ..

⚠️ Download it (Actions → Download file → xray-sdk-layer.zip), then in the Lambda console:

→ Layers → Edit → Add layer → Create a new layer

→ Name: aws-xray-sdk

→ Source code: Create from a .zip file → Choose file → Upload the zip → Compatible runtimes optional: Python 3.13 → Create → Copy arn

→ Open ObservableOrderService → Code → Layers → Edit → Add a layer → Specify an ARN → Paste arn → Verify → Add

💡 Prefer no layer? You can skip the SDK entirely and rely on active tracing alone It auto-traces the invocation and any boto3/HTTP calls without importing anything. You'd just lose in-code custom subsegments/annotations. If you go that route, remove the aws_xray_sdk imports and xray_recorder calls from the code below.

⚠️ X-Ray permissions: if traces don't appear, attach AWSXRayDaemonWriteAccess to the function's execution role (Configuration → Permissions → role → Add permissions). Active tracing needs xray:PutTraceSegments and xray:PutTelemetryRecords.

⚠️ Alternatively, you can use the aws_xray_sdk by packaging it with your function code. For this tutorial, we'll use the built-in boto3 tracing capabilities.

Step 04: Add Instrumented Code

Replace the function code with:

import json
import time
import boto3
from aws_xray_sdk.core import xray_recorder
from aws_xray_sdk.core import patch_all

patch_all()

def generate_correlation_id(event):
    headers = event.get('headers', {}) or {}
    return headers.get('X-Correlation-Id',
           headers.get('x-correlation-id',
           f"gen-{int(time.time() * 1000)}"))

def structured_log(level, message, correlation_id, **kwargs):
    print(json.dumps({
        "timestamp": time.strftime('%Y-%m-%dT%H:%M:%SZ', time.gmtime()),
        "level": level, "message": message,
        "correlationId": correlation_id,
        "service": "ObservableOrderService", **kwargs
    }))

def lambda_handler(event, context):
    correlation_id = generate_correlation_id(event)
    action = event.get('action', event.get('path', '/process'))

    # Facade-segment fix: annotate our OWN subsegment, not the Lambda root.
    with xray_recorder.in_subsegment('request-handler') as subsegment:
        subsegment.put_annotation('correlationId', correlation_id)
        subsegment.put_annotation('action', str(action))
        subsegment.put_metadata('event', event, 'request')

        structured_log("INFO", "Request received", correlation_id,
                       action=action, requestId=context.aws_request_id)

        if action in ('/health', 'health'):
            return health_check(correlation_id)

        return process_order(event, correlation_id, context)

def health_check(correlation_id):
    with xray_recorder.in_subsegment('HealthCheck') as subsegment:
        subsegment.put_annotation('checkType', 'deep')
        dependencies = {}
        try:
            start = time.time()
            boto3.client('sts').get_caller_identity()
            latency = int((time.time() - start) * 1000)
            dependencies['sts'] = {'status': 'healthy', 'latency': f'{latency}ms'}
        except Exception as e:
            dependencies['sts'] = {'status': 'unhealthy', 'error': str(e)}
        overall = 'healthy' if all(d['status'] == 'healthy' for d in dependencies.values()) else 'degraded'
        structured_log("INFO", f"Health check: {overall}", correlation_id, dependencies=dependencies)
        return {
            'statusCode': 200 if overall == 'healthy' else 503,
            'headers': {'Content-Type': 'application/json'},
            'body': json.dumps({'status': overall, 'correlationId': correlation_id, 'dependencies': dependencies})
        }

def process_order(event, correlation_id, context):
    with xray_recorder.in_subsegment('ValidateInput') as subsegment:
        order_id = event.get('orderId', f"ORD-{int(time.time())}")
        customer_id = event.get('customerId')
        subsegment.put_annotation('orderId', order_id)

        if not customer_id:
            subsegment.put_annotation('validationResult', 'failed')
            structured_log("WARN", "Missing customerId", correlation_id, orderId=order_id)
            return {
                'statusCode': 400,
                'headers': {'Content-Type': 'application/json'},
                'body': json.dumps({'error': 'customerId is required', 'correlationId': correlation_id})
            }
        subsegment.put_annotation('validationResult', 'passed')

    with xray_recorder.in_subsegment('ProcessBusinessLogic') as subsegment:
        subsegment.put_annotation('orderId', order_id)
        subsegment.put_annotation('customerId', customer_id)
        processing_time = 0.1 + (hash(order_id) % 5) / 10
        time.sleep(processing_time)
        subsegment.put_metadata('processingDetails',
            {'orderId': order_id, 'processingTime': f'{processing_time:.1f}s', 'items': event.get('items', [])}, 'order')

    structured_log("INFO", "Order processed successfully", correlation_id,
                   orderId=order_id, customerId=customer_id, processingTime=f"{processing_time:.1f}s")
    return {
        'statusCode': 200,
        'headers': {'Content-Type': 'application/json'},
        'body': json.dumps({'message': 'Order processed', 'orderId': order_id,
                            'customerId': customer_id, 'correlationId': correlation_id})
    }

→ Click Deploy

⚠️ If the aws_xray_sdk import fails, you'll need to add it as a Lambda layer. Create a layer with the X-Ray SDK package, or use the simplified version without the SDK (X-Ray will still trace boto3 calls automatically with active tracing enabled).

Part II

Add Custom Subsegments and Annotations

Step 01: Test the Function

→ Go to the Test tab

→ Create a test event:

{
  "action": "process",
  "orderId": "ORD-2026-001",
  "customerId": "CUST-100",
  "items": [{"productId": "PROD-A", "quantity": 2}],
  "headers": {"X-Correlation-Id": "test-corr-001"}
}

→ Click Test → ⚠️ **Verify the response includes the correlation ID

→ Run it several more times with different order IDs

Step 02: Test the Health Check

→ Create another test event

{
  "action": "health"
}

→ Click Test → ⚠️ Verify the health check response with dependency status

Step 03: Test Validation Failure

Create a test event without customerId

{
  "action": "process",
  "orderId": "ORD-2026-002"
}

→ Click Test → ⚠️ verify the 400 response with validation error

Part III

View Traces in the X-Ray Console

Step 01: Open the X-Ray Trace Map

→ Open the X-Ray console (or CloudWatch → Trace Map → ObservableOrderService)

→ Set the time range to the last 30 minutes

→ You should see a service map showing:

Client → ObservableOrderService (Lambda Context) → ObservableOrderService (Lambda Function)
If the function called other AWS services, those appear as nodes too

Step 02: Examine Individual Traces

→ Click on the ObservableOrderService node in the service map

→ Click View traces

→ Click on a specific trace to see the timeline:

The Lambda segment (total execution time)
Initialization subsegment (cold start, if applicable)
ValidateInput subsegment (your custom subsegment)
ProcessBusinessLogic subsegment (your custom subsegment)
HealthCheck subsegment (for health check requests)

Step 03: Search Traces by Annotation

→ Go to Traces

→ In the filter expression box, type:

   annotation.correlationId = "test-corr-001"

→ Click Run query → You should see only the traces with that correlation ID

💡This is exactly why annotations are indexed and metadata is not. You can search for annotation.userId = "CUST-100" to find all traces for a specific user. You cannot search by metadata values. Use annotations for the fields you'll want to filter by. Use metadata for the detailed data you'll examine once you've found the right trace.

Part IV

Set Up CloudWatch Alarms with SNS Notifications

Step 01: Create an SNS Topic

→ Open the SNS console → Topics → Create topic

→ Type: Standard

→ Name: OrderServiceAlerts

→ Click Create topic

Step 02: Create an Email Subscription

→ Click Create subscription

→ Protocol: Email

→ Endpoint: your email address

→ Click Create subscription

→ Check your email and confirm the subscription

Step 03: Create an Error Rate Alarm

→ Open the CloudWatch console → Alarms → Create alarm

→ Click Select metric

→ Navigate: Lambda → By Function Name → ObservableOrderService → Errors

→ Click Select metric

Statistic: Sum
Period: 60 seconds
Threshold type: Static
Whenever Errors is: Greater than 0
Datapoints to alarm: 1 out of 1

→ Notification:

Alarm state trigger: In alarm
SNS topic: OrderServiceAlerts

→ Alarm name: OrderService-Errors

→ Click Create alarm

Step 04: Create a Duration Alarm

→ Create another alarm:

Metric: Lambda → ObservableOrderService → Duration
Statistic: p95
Period: 300 seconds (5 minutes)
Threshold: Greater than 3000 (3 seconds)
Datapoints to alarm: 2 out of 3
SNS topic: OrderServiceAlerts
Alarm name: OrderService-HighLatency

Step 05: Create a Composite Alarm

→ Go to Alarms → Create alarm → Create composite alarm

→ Alarm rule:

   ALARM("OrderService-Errors") AND ALARM("OrderService-HighLatency")

→ Notification: OrderServiceAlerts

→ Alarm name: OrderService-Critical

→ Click Create alarm

⚠️ This composite alarm only fires when both errors AND high latency occur simultaneously indicating a real problem, not just a transient spike.

Part V

Verify the Complete Observability Stack

Generate Traffic and Verify

→ Go back to the Lambda function and run various test events:

Normal orders (several times)
Health checks
Validation failures (missing customerId)

→ Check CloudWatch Logs:

Open the log group /aws/lambda/ObservableOrderService
Verify structured JSON logs with correlation IDs
Use Logs Insights to query:

   parse @message '"correlationId":"*","service":"*"' as corrId, svc
   | filter corrId like /test-corr/
   | fields @timestamp, @message
   | sort @timestamp desc

→ Check X-Ray:

View the trace map. Verify your function appears
Search traces by annotation
Examine subsegment timing

→ Check CloudWatch Alarms:

Verify alarms are in OK state
Trigger errors to test the alarm → check your email for the SNS notification

→ Check the Dashboard (optional):

Create a dashboard combining Lambda metrics, custom metrics, and Logs Insights queries

💡 A fully observable application has all three pillars working together: structured logs with correlation IDs (searchable in Logs Insights), metrics with alarms (proactive alerting), and traces with annotations (distributed request tracking). The correlation ID ties them all together. From a log entry, you can find the X-Ray trace, and from a trace, you can search the logs.

🏗️ What You Built | 📘 Exam Concepts Recap

What You Built	Exam Concept
Added structured logging with correlation IDs	Tracking a request across services
Created custom X-Ray subsegments	Measuring specific code sections for bottlenecks
Added X-Ray annotations and metadata	Annotations are indexed/searchable. Metadata is not
Searched traces by `annotation.correlationId`	Filtering traces by indexed fields
Enabled active tracing on Lambda	X-Ray daemon runs automatically, no install
Created error and latency CloudWatch alarms	Proactive alerting with SNS notifications
Built a composite alarm with AND logic	Reducing alert noise by combining conditions
Tied logs + metrics + traces via correlation ID	The three pillars of observability working together

⚠️ Clean Up Protocol

Lambda → Delete ObservableOrderService
SNS → Delete the OrderServiceAlerts topic (this also deletes subscriptions)
CloudWatch → Delete all three alarms (Errors, HighLatency, Critical composite)
CloudWatch → Delete the log group /aws/lambda/ObservableOrderService
IAM → Delete the Lambda execution role

Key Takeaways

Observability = logs + metrics + traces. Monitoring tells you something is wrong. Observability tells you why. X-Ray is the answer when the question asks about finding the root cause of latency in distributed systems.
X-Ray annotations are indexed and searchable: use them for filtering traces by user ID, order ID, etc. Metadata is not searchable: use it for detailed debugging data.
X-Ray subsegments let you measure specific parts of your code. Create subsegments for database calls, external API calls, and business logic to identify bottlenecks.
Correlation IDs track a request across services. Generate one at the entry point and pass it through headers, message attributes, and log entries. Add it as an X-Ray annotation.
Structured JSON logging is essential for Logs Insights queries. Include level, message, correlation ID, timestamp, and relevant context in every log entry.
CloudWatch alarms have three states: OK, ALARM, INSUFFICIENT_DATA. Configure TreatMissingData carefully. notBreaching is usually the right choice for Lambda.
Composite alarms combine multiple alarms with AND/OR logic to reduce alert noise. They only fire when the combined condition is true.
X-Ray sampling defaults to 1 request/second + 5% of additional requests. Create custom rules to trace more of high-value endpoints and less of health checks.
Health checks should have shallow (is it running?) and deep (are dependencies healthy?) variants. Use shallow for load balancers, deep for dashboards.
For Lambda, enable Lambda service traces in the function configuration. The X-Ray daemon runs automatically no installation needed. Use the X-Ray SDK to add custom subsegments and annotations.

Additional Resources

🏗️

Assist In Root Cause Analysis | 🏗️ Build A Debugging Dashboard

Ntombizakhona Mabaso — Mon, 13 Jul 2026 20:16:57 +0000

Exam Guide: Developer - Associate
🏗️ Domain 4: Troubleshooting And Optimization
📘 Task 1: Assist In A Root Cause Analysis

When something breaks in production, you need to find out why. Fast. This task tests your ability to use CloudWatch Logs, Logs Insights, CloudWatch metrics, dashboards, and CloudTrail to debug application issues. Writing Logs Insights queries. Understanding Lambda-specific log fields. Using Embedded Metric Format for custom metrics. And recognizing common failure patterns across AWS service integrations.

📘 Concepts

CloudWatch Logs Architecture

Component	What It Is	`Example`
Log Group	A collection of log streams that share retention and access settings	`/aws/lambda/my-function`
Log Stream	A sequence of log events from a single source	`2024/01/15/[$LATEST]abc123`
Log Event	A single log entry with a timestamp and message	`{"level":"ERROR","message":"DynamoDB timeout"}`
Retention	How long logs are kept (1 day to 10 years, or never expire)	`30 days` for production
Subscription Filter	Real-time stream of log events to another destination	Stream to Kinesis, Lambda, or OpenSearch
Metric filter	Extract metric values from log text patterns	Count `ERROR` occurrences

💡 Lambda automatically creates a log group named /aws/lambda/{function-name}. Each function instance creates a new log stream. Log retention defaults to "Never expire". Always set a retention policy to control costs. You need logs:CreateLogGroup and logs:PutLogEvents permissions in the Lambda execution role.

CloudWatch Logs Insights Query Syntax

Logs Insights is a query language for searching and analysing log data. The exam tests specific query commands:

Command	Purpose	`Example`
fields	Select which fields to display	`fields @timestamp, @message`
filter	Filter log events by condition	`filter @message like /ERROR/`
stats	Aggregate data	`stats count(*) by bin(5m)`
sort	Order results	`sort @timestamp desc`
limit	Cap the number of results	`limit 20`
parse	Extract fields from unstructured log text	`parse @message "user=* action=*" as user, action`
display	Choose which fields appear in results	`display @timestamp, @message`

Common Query Patterns

Find the most recent errors

fields @timestamp, @message
| filter @message like /ERROR/
| sort @timestamp desc
| limit 25

Count errors per 5-minute window

filter @message like /ERROR/
| stats count(*) as errorCount by bin(5m)
| sort errorCount desc

Find slow Lambda invocations

filter @duration > 5000
| fields @requestId, @duration, @billedDuration, @memorySize, @maxMemoryUsed
| sort @duration desc
| limit 10

Parse custom log fields

parse @message '"level":"*","requestId":"*","error":"*"' as level, reqId, error
| filter level = "ERROR"
| stats count(*) by error

Find cold starts

filter @type = "REPORT" and @initDuration > 0
| fields @requestId, @initDuration, @duration, @memorySize
| sort @initDuration desc
| limit 20

💡The parse command uses glob patterns (*) to extract fields from unstructured text. The stats command supports count(), sum(), avg(), min(), max(), pct() (percentiles). Use bin() to group by time intervals. The @ prefix denotes system-generated fields like @timestamp, @message, @logStream, etc.

Lambda-Specific Log Fields

Lambda automatically adds these fields to REPORT lines:

`Field`	What It Contains	Why It Matters
`@type`	Log line type: `START`, `END`, `REPORT`	Filter for REPORT to get invocation summaries
`@requestId`	Unique invocation ID	Correlate logs for a single invocation
`@duration`	Actual execution time (ms)	Performance monitoring
`@billedDuration`	Rounded up to nearest ms (or 1ms minimum)	Cost tracking
`@memorySize`	Configured memory (MB)	Right-sizing analysis
`@maxMemoryUsed`	Peak memory used (MB)	Detect memory pressure
`@initDuration`	Cold start initialization time (ms)	Only present on cold starts
`@xrayTraceId`	X-Ray trace ID	Link logs to traces

Memory right-sizing query

filter @type = "REPORT"
| stats avg(@maxMemoryUsed) as avgMem,
        max(@maxMemoryUsed) as peakMem,
        avg(@memorySize) as configuredMem,
        avg(@maxMemoryUsed) / avg(@memorySize) * 100 as utilizationPct

💡 If @maxMemoryUsed is consistently close to @memorySize, the function may be running out of memory. If it's consistently low (under 50%), you're over-provisioned and wasting money. The @initDuration field only appears on cold starts. If you see it frequently, consider provisioned concurrency.

CloudWatch Embedded Metric Format (EMF) vs PutMetricData

There are two ways to publish custom metrics.

Aspect	Embedded Metric Format (EMF)	PutMetricData API
How It Works	Write structured JSON to stdout: CloudWatch extracts metrics automatically	Call the CloudWatch API directly
Latency	Async (written with logs)	Sync (API call blocks execution)
Cost	Log ingestion cost only (no per-metric API charge)	$0.01 per 1,000 PutMetricData API calls
Dimensions	Up to 30 dimensions	Up to 30 dimensions
Resolution	Standard (60s) or high (1s)	Standard (60s) or high (1s)
Batching	Automatic (one log line = one metric data point)	Manual (up to 1,000 values per API call)
Lambda Overhead	None. Just a print statement	API call adds latency to each invocation
Best For	Lambda functions, high-throughput applications	EC2 applications, infrequent metric publishing

EMF format

import json

def lambda_handler(event, context):
    # Your business logic here
    order_total = process_order(event)

    # Emit EMF metric. CloudWatch extracts this automatically
    print(json.dumps({
        "_aws": {
            "Timestamp": 1234567890000,
            "CloudWatchMetrics": [{
                "Namespace": "OrderService",
                "Dimensions": [["Environment", "OrderType"]],
                "Metrics": [
                    {"Name": "OrderTotal", "Unit": "None"},
                    {"Name": "ProcessingTime", "Unit": "Milliseconds"}
                ]
            }]
        },
        "Environment": "production",
        "OrderType": "standard",
        "OrderTotal": order_total,
        "ProcessingTime": 145
    }))

PutMetricData approach

import boto3

cloudwatch = boto3.client('cloudwatch')

def lambda_handler(event, context):
    order_total = process_order(event)

    # Sync API call — adds latency
    cloudwatch.put_metric_data(
        Namespace='OrderService',
        MetricData=[{
            'MetricName': 'OrderTotal',
            'Value': order_total,
            'Unit': 'None',
            'Dimensions': [
                {'Name': 'Environment', 'Value': 'production'},
                {'Name': 'OrderType', 'Value': 'standard'}
            ]
        }]
    )

💡> EMF is the preferred approach for Lambda because it adds zero latency. You're just writing to stdout, which Lambda already sends to CloudWatch Logs. CloudWatch automatically extracts the metrics from the structured JSON. Use PutMetricData when you need to publish metrics from EC2 or when you need more control over metric timing.

Common Integration Failure Patterns

Scenarios to identify the root cause. Know these patterns:

`Symptom`	Likely Cause	How to Debug	Fix
`502 Bad Gateway` (API GW)	Lambda returned invalid response format	Check Lambda logs for response structure	Return `statusCode`, `headers`, `body`
`504 Gateway Timeout` (API GW)	Lambda exceeded API Gateway's 29-second timeout	Check `@duration` in logs	Reduce Lambda timeout, optimize code, or use async
`429 Too Many Requests`	Lambda concurrency throttled or API GW rate limit	Check `Throttles` metric	Increase reserved concurrency or request limit increase
`Access Denied`	Missing IAM permissions	Check CloudTrail for `AccessDenied` events	Update IAM policy
`Task timed out` (Lambda)	Function exceeded its configured timeout	Check `@duration` vs timeout setting	Increase timeout or optimize code
`Out of memory`	Function exceeded configured memory	Check `@maxMemoryUsed` vs `@memorySize`	Increase memory allocation
Connection timeout	Lambda in VPC can't reach internet or service	Check VPC config, NAT Gateway, security groups	Add NAT Gateway or VPC endpoint
`ResourceNotFoundException`	DynamoDB table or resource doesn't exist in the region	Check region configuration	Verify resource exists in the correct region

502 Bad Gateway Deep Dive

The most common Lambda + API Gateway error. API Gateway expects this exact response format:

# CORRECT: API Gateway accepts this
return {
    'statusCode': 200,
    'headers': {'Content-Type': 'application/json'},
    'body': json.dumps({'message': 'success'})  # body must be a STRING
}

# WRONG: causes 502
return {
    'statusCode': 200,
    'body': {'message': 'success'}  # body is a dict, not a string!
}

# WRONG: causes 502
return "success"  # not the expected format at all

# WRONG: causes 502 (unhandled exception)
def handler(event, context):
    raise Exception("oops")  # no try/except = 502

💡 A 502 from API Gateway almost always means the Lambda function returned a response that doesn't match the expected format, or the function threw an unhandled exception. Check the Lambda logs first: the error will be there. The API Gateway execution logs (if enabled) will show "Malformed Lambda proxy response."

CloudTrail for Debugging Permission Issues

CloudTrail Feature	What It Does	Use Case
Event History	Last 90 days of management events (free)	Quick lookup of recent API calls
Trail	Continuous logging to S3 (all events)	Long-term audit, compliance
Data Events	S3 object-level and Lambda invocation events	Debug specific resource access
Insights	Detect unusual API activity	Identify anomalous patterns

Finding AccessDenied events

1. Open CloudTrail console → Event history
2. Filter by Event name or Error code
3. Look for events with errorCode: AccessDenied or errorCode: UnauthorizedAccess
4. The event details show:

userIdentity: who made the call (which role/user)
eventName: what API was called (e.g., PutItem)
requestParameters: what resource was targeted
errorMessage: the specific permission that's missing

💡 CloudTrail logs every AWS API call. When you get an AccessDenied error, CloudTrail tells you exactly which principal tried to call which API on which resource and which permission was missing. This is the fastest way to debug IAM issues. Event history is free and covers the last 90 days.

Deployment Failure Troubleshooting

Failure Type	Where to Look	Common Causes
CloudFormation stack failure	CloudFormation Events tab	IAM permissions, resource limits, invalid template
CodeBuild failure	CodeBuild build logs	Missing dependencies, test failures, buildspec syntax
CodeDeploy failure	CodeDeploy deployment logs	Hook script errors, health check failures, timeout
Lambda deployment failure	CloudWatch Logs + CodeDeploy	New version crashes, alarm triggers rollback
SAM deploy failure	CloudFormation Events + SAM CLI output	Transform errors, packaging issues, capability missing

💡 CloudFormation failures show the specific resource and error in the Events tab. Look for CREATE_FAILED or UPDATE_FAILED status. The most common cause is insufficient IAM permissions because the CloudFormation role doesn't have permission to create the resource. For SAM, make sure you include CAPABILITY_IAM and CAPABILITY_AUTO_EXPAND when deploying.

🏗️ Build A Debugging Dashboard

Build a Debugging Dashboard from scratch using the AWS Console:

A Lambda function that emits structured logs and EMF custom metrics
CloudWatch Logs Insights queries that find errors, slow invocations, and cold starts
A CloudWatch dashboard displaying key application metrics
Simulated failures (timeout, permission denied, 502) with debugging walkthroughs
CloudTrail queries to find access denied events

Prerequisites

An AWS Account
Python

Part I

Create a Lambda Function with Structured Logging and EMF

Step 01: Create the Function

Open the Lambda console → Create function

Function name: DebugDemoFunction
Runtime: Python 3.13

Click Create function

Step 02: Add Structured Logging with EMF

import json
import time
import os
import random

def emit_emf_metric(metric_name, value, unit, dimensions):
    """Emit a CloudWatch metric using Embedded Metric Format."""
    print(json.dumps({
        "_aws": {
            "Timestamp": int(time.time() * 1000),
            "CloudWatchMetrics": [{
                "Namespace": "DebugDemo",
                "Dimensions": [list(dimensions.keys())],
                "Metrics": [{"Name": metric_name, "Unit": unit}]
            }]
        },
        **dimensions,
        metric_name: value
    }))

def lambda_handler(event, context):
    start_time = time.time()
    action = event.get('action', 'process')
    request_id = context.aws_request_id

    # Structured log entry
    log_entry = {
        "level": "INFO",
        "requestId": request_id,
        "action": action,
        "timestamp": time.strftime('%Y-%m-%dT%H:%M:%SZ', time.gmtime()),
        "functionName": context.function_name,
        "memoryLimit": context.memory_limit_in_mb
    }

    try:
        if action == 'slow':
            # Simulate a slow operation
            delay = random.uniform(2, 5)
            log_entry["level"] = "WARN"
            log_entry["message"] = f"Slow operation detected: {delay:.1f}s"
            time.sleep(delay)

        elif action == 'error':
            # Simulate an error
            raise ValueError("Invalid order data: missing required field 'customerId'")

        elif action == 'oom':
            # Simulate high memory usage
            log_entry["message"] = "Processing large dataset"
            data = "x" * (50 * 1024 * 1024)  # 50MB string

        else:
            # Normal processing
            log_entry["message"] = "Order processed successfully"
            time.sleep(random.uniform(0.1, 0.5))

        print(json.dumps(log_entry))

        # Emit EMF metrics
        processing_time = (time.time() - start_time) * 1000
        emit_emf_metric("ProcessingTime", processing_time, "Milliseconds",
                        {"Environment": "demo", "Action": action})
        emit_emf_metric("SuccessCount", 1, "Count",
                        {"Environment": "demo", "Action": action})

        return {
            'statusCode': 200,
            'body': json.dumps({
                'message': 'Processed successfully',
                'action': action,
                'requestId': request_id,
                'processingTime': f"{processing_time:.0f}ms"
            })
        }

    except Exception as e:
        processing_time = (time.time() - start_time) * 1000
        log_entry["level"] = "ERROR"
        log_entry["error"] = str(e)
        log_entry["errorType"] = type(e).__name__
        print(json.dumps(log_entry))

        emit_emf_metric("ErrorCount", 1, "Count",
                        {"Environment": "demo", "Action": action})

        return {
            'statusCode': 500,
            'body': json.dumps({
                'error': str(e),
                'requestId': request_id
            })
        }

Click Deploy

Step 03: Generate Test Data

Go to the Test tab

Create and run these test events multiple times:

Normal processing

{"action": "process"}

Slow operation

{"action": "slow"}

Error case

{"action": "error"}

⚠️ Run each test event 5-10 times to generate enough log data

Part II

Write CloudWatch Logs Insights Queries

Step 01: Open Logs Insights

Open the CloudWatch console → ** ▼ Logs** → Log Management

Select the log group: /aws/lambda/DebugDemoFunction

⚠️ Set the time range to the last 1 hour

→ View in Logs Insights

Query 01: Find All Errors

fields @timestamp, @message
| filter @message like /ERROR/
| sort @timestamp desc
| limit 25

Click Run query. You should see the structured error logs from the error test events.

Query 02: Analyze Invocation Performance

filter @type = "REPORT"
| stats avg(@duration) as avgDuration,
        max(@duration) as maxDuration,
        min(@duration) as minDuration,
        count(*) as invocations

Query 03: Find Cold Starts

filter @type = "REPORT" and @initDuration > 0
| fields @requestId, @initDuration, @duration, @maxMemoryUsed
| sort @initDuration desc

Query 04: Parse Structured Logs for Error Breakdown

parse @message '"level":"*","requestId":"*"' as level, reqId
| filter level = "ERROR"
| stats count(*) as errorCount by level

Query 05: Memory Utilization Analysis

filter @type = "REPORT"
| stats avg(@maxMemoryUsed) as avgMemUsed,
        max(@maxMemoryUsed) as peakMemUsed,
        avg(@memorySize) as configuredMem,
        avg(@maxMemoryUsed) / avg(@memorySize) * 100 as utilizationPct

Query 06: Slow Invocations Over Time

filter @type = "REPORT"
| stats avg(@duration) as avgDuration,
        pct(@duration, 95) as p95Duration,
        pct(@duration, 99) as p99Duration
        by bin(5m)
| sort bin(5m) asc

💡 Save useful queries by clicking **Save in Logs Insights. You can also add query results directly to a CloudWatch dashboard. The pct() function calculates percentiles: pct(@duration, 95) gives you the 95th percentile duration, which is more useful than averages for identifying tail latency.**

Part III

Create a CloudWatch Dashboard

Step 01: Build the Dashboard

Open the CloudWatch console → Dashboards → Create dashboard

Dashboard name: DebugDemo-Dashboard

Click Create dashboard

Step 02: Add an Invocation Count Widget

Click Add widget → Number

Select Lambda → By Function Name → DebugDemoFunction → Invocations

Period: 5 minutes

Click Create widget

Step 03: Add an Error Rate Widget

Click Add widget → Line

Add two metrics:

Lambda → By Function Name → DebugDemoFunction → Errors
Lambda → By Function Name → DebugDemoFunction → Invocations

Click Create widget

Step 04: Add a Duration Widget

Click Add widget → Line

Select Lambda → By Function Name → DebugDemoFunction → Duration

Add statistics: Average, p99, Maximum

Click Create widget

Step 05: Add a Custom EMF Metric Widget

Click Add widget → Number

Select DebugDemo (custom namespace) → Environment, Action → ProcessingTime

Click Create widget

Step 06: Add a Logs Insights Query Widget

Click Add widget → Logs table

Select log group: /aws/lambda/DebugDemoFunction

Enter query:

filter @message like /ERROR/
| fields @timestamp, @message
| sort @timestamp desc
| limit 10

Click Create

**Step 07:* Click Save dashboard

Part IV

Simulate and Debug Common Failures

Step 01: Simulate a 502 Bad Gateway

Create a new Lambda function: BadResponseFunction and add this intentionally broken code

import json

def lambda_handler(event, context):
    # BUG: body should be a string, not a dict
    return {
        'statusCode': 200,
        'body': {'message': 'this will cause a 502'}  # Wrong!
    }

Create an API Gateway REST API pointing to this function

💡 Test the API endpoint. You'll get a 502

⚠️ Debug steps:

Check Lambda logs → the function executed successfully (no error)

Check API Gateway execution logs → "Malformed Lambda proxy response"

Fix: Change 'body': {'message': '...'} to 'body': json.dumps({'message': '...'})

Step 02: Simulate a Timeout

On DebugDemoFunction, go to Configuration → General configuration → Edit

Set Timeout to 3 seconds

Run the slow test event (which sleeps 2-5 seconds)

⚠️ Debug steps:

Check logs → Task timed out after 3.00 seconds

Check the REPORT line → @duration close to 3000ms

Query in Logs Insights:

   filter @message like /Task timed out/
   | fields @requestId, @duration, @memorySize

Fix: Increase timeout or optimize the slow operation

Step 03: Simulate Permission Denied

On DebugDemoFunction, update the code to call DynamoDB:

import boto3
import json

dynamodb = boto3.resource('dynamodb')

def lambda_handler(event, context):
    table = dynamodb.Table('NonExistentTable')
    try:
        table.put_item(Item={'PK': 'test', 'SK': 'test'})
    except Exception as e:
        print(json.dumps({
            "level": "ERROR",
            "error": str(e),
            "errorType": type(e).__name__
        }))
        return {
            'statusCode': 500,
            'body': json.dumps({'error': str(e)})
        }

💡Run the test. You'll get an AccessDeniedException

Debug steps:

Check Lambda logs → AccessDeniedException

Open CloudTrail → Event history

Filter by Event name: PutItem

Find the event with errorCode: AccessDenied

The event shows which role tried to access which resource

Part V

Use CloudTrail to Find Access Denied Events

Step 01: Search Event History

Open the CloudTrail console → Event history

Change the filter to Error code

Type AccessDenied and press Enter

You should see the PutItem event from the previous step

Step 02: Examine the Event Details

Click on the event to expand it

Key fields to examine

userIdentity.arn: the Lambda execution role that made the call
eventName: PutItem
requestParameters: the table name and item
errorCode: AccessDeniedException
errorMessage: the specific permission needed

💡To fix: add dynamodb:PutItem permission to the Lambda execution role for the specific table ARN

Step 03: Create a Logs Insights Query for CloudTrail

If you have a CloudTrail trail sending logs to CloudWatch:

fields eventTime, eventName, errorCode, errorMessage,
       userIdentity.arn as principal,
       requestParameters.tableName as table
| filter errorCode = "AccessDenied"
| sort eventTime desc
| limit 20

💡 CloudTrail Event history is free and covers the last 90 days of management events. For longer retention or data events (S3 object access, Lambda invocations), you need a trail. When debugging permission issues, CloudTrail is your first stop, it tells you exactly what permission is missing, who tried to use it, and on which resource.

🏗️ What You Built | 📘 Exam Concepts Recap

What You Built	Exam Concept
Emitted EMF metrics from structured logs	Zero-latency custom metrics via Embedded Metric Format
Wrote CloudWatch Logs Insights queries	`fields`, `filter`, `stats`, `parse`, `sort`, `pct()` syntax
Queried `@initDuration` for cold starts	Reading Lambda REPORT fields for performance
Built a CloudWatch dashboard with mixed widgets	Single pane of glass for application health
Reproduced and fixed a 502 Bad Gateway	Malformed Lambda proxy response (body must be string)
Reproduced a timeout and read the REPORT line	Diagnosing `Task timed out` from logs
Found AccessDenied events in CloudTrail	Debugging permission issues: principal, API, resource

⚠️ Clean Up Protocol

Lambda → Delete DebugDemoFunction and BadResponseFunction
API Gateway → Delete any test APIs created
CloudWatch → Delete the DebugDemo-Dashboard
CloudWatch → Delete log groups for the Lambda functions
IAM → Delete Lambda execution roles

Key Takeaways for the Exam

CloudWatch Logs Insights uses fields, filter, stats, parse, sort, and limit commands. Know the syntax for each.
Lambda REPORT lines contain @duration, @billedDuration, @memorySize, @maxMemoryUsed, and @initDuration (cold starts only). Use these for performance analysis.
EMF is preferred over PutMetricData for Lambda because it adds zero latency. Metrics are extracted from structured JSON written to stdout. Use PutMetricData for EC2 or infrequent publishing.
502 Bad Gateway from API Gateway almost always means the Lambda response format is wrong: body must be a string, not a dict.
504 Gateway Timeout means Lambda exceeded API Gateway's 29-second limit. The Lambda timeout must be less than 29 seconds for synchronous API Gateway integrations.
CloudTrail logs every AWS API call. Use Event history to find AccessDenied errors. It shows the principal, API, resource, and missing permission.
Metric filters extract CloudWatch metrics from log text patterns. Subscription filters stream log events to Kinesis, Lambda, or OpenSearch in real time.
CloudWatch dashboards can include metrics widgets, Logs Insights query results, and alarms giving you a single pane of glass for application health.
The parse command in Logs Insights extracts fields from unstructured text using glob patterns (*). The stats command supports count(), avg(), sum(), min(), max(), and pct() for percentiles.
When debugging deployment failures, check CloudFormation Events (stack failures), CodeBuild logs (build failures), and CodeDeploy logs (deployment failures). Each service has its own log location.

Additional Resources

🏗️

Deploy Code By Using AWS Continuous Integration And Continuous Delivery (CI/CD) Services | 🏗️ Build A Complete CI/CD Pipeline

Ntombizakhona Mabaso — Wed, 08 Jul 2026 00:12:51 +0000

Exam Guide: Developer - Associate
🏗️ Domain 3: Deployment
📘 Task 4: Deploy Code By Using AWS Continuous Integration And Continuous Delivery (CI/CD) Services

This task tests your ability to build and manage CI/CD pipelines using AWS developer tools. You need to understand how CodeCommit, CodeBuild, CodeDeploy, and CodePipeline work together, how to write buildspec and appspec files, how deployment strategies differ, and how to configure automatic rollbacks. Deployment strategies for Lambda and EC2, SAM deployment preferences, and pipeline orchestration.

📘 Concepts

AWS CI/CD Pipeline Overview

The four AWS developer tools form a complete CI/CD pipeline:

Service	Role	Input	Output
CodeCommit	Source control	Git push	Source artifact
CodeBuild	Build and test	Source artifact	Build artifact
CodeDeploy	Deploy	Build artifact	Running application
CodePipeline	Orchestration	Trigger (push, schedule)	Coordinated pipeline execution

How they connect:

CodeCommit (source) → CodeBuild (build/test) → CodeDeploy (deploy)
         ↑                                              |
         └──────── CodePipeline (orchestrates all) ─────┘

💡CodePipeline is the orchestrator. It doesn't build or deploy anything itself. It connects stages (source, build, test, deploy) and manages transitions between them. Each stage can use different providers (GitHub instead of CodeCommit, Jenkins instead of CodeBuild, etc.).

CodeCommit Fundamentals

Feature	Details
What It Is	Managed Git repository hosted in AWS
Authentication	HTTPS (Git credentials or credential helper) or SSH (SSH keys)
Encryption	Encrypted at rest (AWS managed keys) and in transit (HTTPS/SSH)
Triggers	SNS notifications or Lambda functions on repository events
Cross-account	Use IAM roles with `AssumeRole` for cross-account access
Branching	Standard Git branching: main, develop, feature branches

💡CodeCommit supports triggers for push events that can invoke Lambda functions or send SNS notifications. This is different from CodePipeline's source stage: triggers are repository-level events, while CodePipeline polls or uses CloudWatch Events to detect changes.

CodeBuild `buildspec.yml` Structure

The buildspec file tells CodeBuild what to do. It has four phases:

version: 0.2

env:
  variables:
    ENV_NAME: "production"
  parameter-store:
    DB_PASSWORD: "/myapp/db-password"
  secrets-manager:
    API_KEY: "myapp/api-key:API_KEY"

phases:
  install:
    runtime-versions:
      python: 3.13
    commands:
      - pip install -r requirements.txt

  pre_build:
    commands:
      - echo "Running tests..."
      - python -m pytest tests/ -v

  build:
    commands:
      - echo "Building..."
      - sam build

  post_build:
    commands:
      - echo "Packaging..."
      - sam package --s3-bucket my-bucket --output-template-file packaged.yaml

artifacts:
  files:
    - packaged.yaml
    - appspec.yml
  discard-paths: yes

cache:
  paths:
    - '/root/.cache/pip/**/*'

reports:
  test-reports:
    files:
      - "**/*.xml"
    base-directory: test-results

`Phase`	When It Runs	Typical Use
`install`	First	Install dependencies, runtime versions
`pre_build`	Before build	Run tests, log in to ECR, lint code
`build`	Main phase	Compile code, run SAM build, create artifacts
`post_build`	After build	Package artifacts, push images, notifications

`Section`	Purpose
`env.variables`	Plain text environment variables
`env.parameter-store`	Values pulled from SSM Parameter Store at build time
`env.secrets-manager`	Values pulled from Secrets Manager at build time
`artifacts`	Files to pass to the next pipeline stage
`cache`	Paths to cache between builds (speeds up installs)
`reports`	Test report files for CodeBuild report groups

💡The buildspec file must be named buildspec.yml and placed in the root of your source directory (unless you override the name in the CodeBuild project settings).

The env section can pull secrets from Parameter Store and Secrets Manager. This is the secure way to use credentials in builds. Never hardcode secrets in the buildspec.

CodeDeploy `appspec.yml` Structure

The appspec file tells CodeDeploy how to deploy your application. The structure differs by compute platform:

For Lambda:

version: 0.0
Resources:
  - MyFunction:
      Type: AWS::Lambda::Function
      Properties:
        Name: my-function
        Alias: prod
        CurrentVersion: 1
        TargetVersion: 2
Hooks:
  - BeforeAllowTraffic: PreTrafficCheckFunction
  - AfterAllowTraffic: PostTrafficCheckFunction

For EC2/On-Premises:

version: 0.0
os: linux
files:
  - source: /
    destination: /var/www/html
permissions:
  - object: /var/www/html
    owner: www-data
    group: www-data
    mode: "755"
hooks:
  BeforeInstall:
    - location: scripts/before_install.sh
      timeout: 300
  AfterInstall:
    - location: scripts/after_install.sh
      timeout: 300
  ApplicationStart:
    - location: scripts/start_server.sh
      timeout: 300
  ValidateService:
    - location: scripts/validate.sh
      timeout: 300

CodeDeploy Lifecycle Hooks

Lambda Deployment Hooks:

`Hook`	When It Runs	Purpose
`BeforeAllowTraffic`	Before traffic shifts to new version	Run validation tests against the new version
`AfterAllowTraffic`	After traffic shifts to new version	Run integration tests, verify health

**EC2 Deployment Hooks (in order):**

`Hook`	When It Runs	Purpose
`BeforeInstall`	Before files are copied	Clean up old files, stop services
`AfterInstall`	After files are copied	Configure app, set permissions
`ApplicationStart`	After AfterInstall	Start services, warm up
`ValidateService`	After ApplicationStart	Health checks, smoke tests
`BeforeBlockTraffic`	Before deregistering from ELB	Graceful connection draining
`AfterBlockTraffic`	After deregistering from ELB	Run tasks while instance is out of service
`BeforeAllowTraffic`	Before registering with ELB	Final checks before receiving traffic
`AfterAllowTraffic`	After registering with ELB	Verify instance is healthy in ELB

💡 For Lambda deployments, the hooks are Lambda functions themselves: they run your validation code and must return Succeeded or Failed to CodeDeploy.

For EC2, hooks are shell scripts that run on the instance.

If a hook fails (the deployment rolls back).

Deployment Strategies Comparison

Know each strategy of by heart:

`Strategy`	How It Works	Downtime	Rollback Speed	Risk	Best For
`AllAtOnce`	Deploy to all instances simultaneously	Brief	Redeploy previous version	High: all instances affected	Dev/test environments
`Rolling`	Deploy in batches (one batch at a time)	None (if batch size < total)	Redeploy previous version	Medium: one batch at a time	EC2 fleets with ELB
`Rolling with additional batch`	Adds new instances before removing old ones	None	Redeploy previous version	Low: maintains full capacity	Production EC2 fleets
`Blue/Green`	Create entirely new environment, switch traffic	None	Switch back to blue	Low	Production with zero downtime
`Canary`	Shift X% traffic, wait, then shift 100%	None	Shift back to 0%	Low: limited blast radius	Lambda, API Gateway
`Linear`	Shift X% every N minutes	None	Shift back to 0%	Low: gradual exposure	Lambda, API Gateway

SAM `DeploymentPreference` Types

SAM automates Lambda deployment strategies through CodeDeploy:

`Type`	Traffic Shift Pattern	Example
`AllAtOnce`	100% immediately	Instant cutover
`Canary10Percent5Minutes`	10% for 5 min, then 100%	Quick canary validation
`Canary10Percent10Minutes`	10% for 10 min, then 100%	Longer canary validation
`Canary10Percent15Minutes`	10% for 15 min, then 100%	Extended canary validation
`Canary10Percent30Minutes`	10% for 30 min, then 100%	Conservative canary
`Linear10PercentEvery1Minute`	+10% every minute (10 min total)	Fast linear rollout
`Linear10PercentEvery2Minutes`	+10% every 2 min (20 min total)	Moderate linear rollout
`Linear10PercentEvery3Minutes`	+10% every 3 min (30 min total)	Slow linear rollout
`Linear10PercentEvery10Minutes`	+10% every 10 min (100 min total)	Very conservative rollout

SAM Template Example:

MyFunction:
  Type: AWS::Serverless::Function
  Properties:
    Handler: index.handler
    Runtime: python3.13
    AutoPublishAlias: live
    DeploymentPreference:
      Type: Canary10Percent5Minutes
      Alarms:
        - !Ref MyFunctionErrorAlarm
      Hooks:
        PreTraffic: !Ref PreTrafficHookFunction
        PostTraffic: !Ref PostTrafficHookFunction

💡 AutoPublishAlias is required for DeploymentPreference to work. SAM automatically publishes a new version on each deploy and updates the alias. The Alarms section triggers automatic rollback if any alarm enters ALARM state during deployment. Canary shifts traffic in two steps (X% then 100%), while Linear shifts in equal increments.

API Gateway Stages and Custom Domains

Concept	What It Is	Use Case
Stage	A named reference to a deployment (dev, staging, prod)	Environment separation
Stage Variables	Key-value pairs available in the stage	Point to different Lambda aliases per stage
Canary Release	Split traffic between current and canary deployment	Test new API version with real traffic
Custom Domain	Map your domain to API Gateway	`api.example.com` instead of `xyz.execute-api.region.amazonaws.com`
Base Path Mapping	Map URL paths to different APIs	`/v1` → API v1, `/v2` → API v2

Stage Variables For Lambda Integration:

Stage: prod → stageVariables.lambdaAlias = "prod"
Stage: dev  → stageVariables.lambdaAlias = "dev"

Lambda ARN: arn:aws:lambda:region:account:function:name:${stageVariables.lambdaAlias}

API Gateway Canary Settings:

Setting	Description
Canary Percentage	% of traffic routed to canary deployment (0-100)
Stage Variable Overrides	Different stage variables for canary traffic
Promote Canary	Move canary deployment to production
Delete Canary	Remove canary and keep current production

💡API Gateway stages are not the same as Lambda aliases, but they work together. Use stage variables to point each stage to the corresponding Lambda alias.

A common pattern: API Gateway prod stage → stage variable lambdaAlias=prod → Lambda function prod alias → version 5.

Rollback Strategies

Rollback Type	How It Works	When It Triggers
Automatic (alarm-based)	CodeDeploy monitors CloudWatch alarms during deployment	Alarm enters ALARM state
Automatic (hook failure)	CodeDeploy rolls back if a lifecycle hook fails	Hook returns `Failed`
Manual	You stop the deployment and roll back	You detect an issue
SAM Automatic	SAM configures CodeDeploy with alarms from `DeploymentPreference`	Configured alarm triggers

CloudWatch Alarm For Automatic Rollback:

`Metric`	Threshold	Purpose
`Errors`	> 0 for 1 minute	Catch any Lambda errors
`Duration`	*> timeout 0.8 for 5 minutes**	Catch performance degradation
`Throttles`	> 0 for 1 minute	Catch concurrency issues
`5XXError` (API GW)	> 1% for 5 minutes	Catch server errors

💡Automatic rollback with CloudWatch alarms is the recommended approach for production deployments.

SAM makes this easy: just add the Alarms list to DeploymentPreference.

CodeDeploy monitors the alarms during the deployment window. If any alarm fires, it automatically shifts all traffic back to the previous version. The rollback is fast because it just updates the alias pointer.

Pre/Post Traffic Hooks

Traffic hooks are Lambda functions that validate your deployment:

PreTraffic Hook Pattern:

import boto3
import json

codedeploy = boto3.client('codedeploy')

def handler(event, context):
    deployment_id = event['DeploymentId']
    lifecycle_event_hook_execution_id = event['LifecycleEventHookExecutionId']

    # Run your validation tests here
    try:
        # Example: invoke the new version and check the response
        lambda_client = boto3.client('lambda')
        response = lambda_client.invoke(
            FunctionName='my-function:live',  # the alias
            Payload=json.dumps({'test': True})
        )
        result = json.loads(response['Payload'].read())

        if result.get('statusCode') == 200:
            status = 'Succeeded'
        else:
            status = 'Failed'
    except Exception:
        status = 'Failed'

    # Report back to CodeDeploy
    codedeploy.put_lifecycle_event_hook_execution_status(
        deploymentId=deployment_id,
        lifecycleEventHookExecutionId=lifecycle_event_hook_execution_id,
        status=status
    )

💡 The PreTraffic hook runs before any traffic shifts to the new version: use it for smoke tests and validation.

The PostTraffic hook runs after all traffic has shifted: use it for integration tests.

Both hooks must call put_lifecycle_event_hook_execution_status to tell CodeDeploy whether to proceed or roll back.

If a hook times out (default 1 hour), the deployment fails.

CodePipeline Concepts

Concept	Description
Pipeline	The overall workflow definition
Stage	A logical group of actions (Source, Build, Deploy)
Action	A task within a stage (CodeBuild action, CodeDeploy action)
Artifact	Files passed between stages (stored in S3)
Transition	The link between stages (can be disabled to pause the pipeline)
Approval action	Manual approval gate before proceeding

Pipeline Feature	Details
Trigger	CloudWatch Events (push to CodeCommit), webhook (GitHub), S3 upload
Cross-region	Actions can deploy to different regions
Cross-account	Use IAM roles for cross-account deployments
Parallel Actions	Multiple actions in the same stage run in parallel
Sequential stages	Stages run in order. A stage must complete before the next starts

💡 CodePipeline stores artifacts in an S3 bucket (created automatically or specified by you).

Each action produces output artifacts and consumes input artifacts.

If you need a manual approval before deploying to production, add an Approval action between the staging deploy and production deploy stages.

🏗️ Build A Complete CI/CD Pipeline

Build a Complete CI/CD Pipeline from scratch using the AWS Console.

A CodeCommit repository with application code
A CodeBuild project that runs tests and packages the application
A CodePipeline that orchestrates source → build → deploy
A Lambda function with canary deployment configured through SAM
CloudWatch alarms for automatic rollback
A working pipeline that triggers on code push

Prerequisites

Part I

Create A Github Repository And Add Code

💡 Why GitHub instead of CodeCommit? AWS closed CodeCommit to new customers in July 2024. Even accounts that can create a CodeCommit repo often find it's not offered as a CodePipeline source provider. GitHub via CodeConnections is the reliable, current, real-world source integration. And it's fully supported by CodePipeline. CodeCommit, CodeBuild, CodeDeploy, and CodePipeline (If your account does offer CodeCommit and you'd rather use it, use it)

Step 01: Create the GitHub Repository

Go to GitHub → New → Repository name: cicd-demo-app → Choose Private (or Public)

⚠️ Don't initialize with a README (we'll push our own files)

Click Create repository

💡 Prefer CodeCommit? If your account offers CodeCommit and it appears as a CodePipeline source provider, you can use it instead: create the repo in the CodeCommit console, generate Git credentials in IAM (Security credentials → API keys → Generate API key → AWS CodeCommit), attach the AWSCodeCommitPowerUser policy to your user, then clone the HTTPS URL and push. HTTPS uses Git credentials. SSH uses an uploaded SSH key. Everything downstream (CodeBuild, CodePipeline, CodeDeploy) is identical. only the source provider changes.

Step By Step Guide For Setting Up CodeCommit
Step 1: Open the CodeCommit console → Create repository

Repository name: cicd-demo-app

Description - optional: Demo application for CI/CD pipeline

Click Create

Step 2: Configure Git Credentials
💡 CodeCommit HTTPS access uses Git credentials tied to your IAM user (a generated username/password, separate from your AWS access keys).
Step 3: Open the IAM console → IAM users → select your IAM user

Step 4: Click the Security credentials tab

Step 5: In the API keys section, click Generate API key

Step 6: Service ▼: AWS CodeCommit

Click Generate API key

Step 7: Retrieve API key
⚠️ Copy or download the username and password now: This is the only time the password is shown. If you lose it, you must reset it.

Click Close

💡 Git Credentials vs Access Keys. CodeCommit HTTPS uses a dedicated username/password (Git credentials), not your AWS_ACCESS_KEY_ID. These are IAM-managed and require a CodeCommit policy like AWSCodeCommitPowerUser on your user. HTTPS = Git credentials or the credential helper. SSH = an uploaded SSH public key.

Step 8: Now attach a CodeCommit permissions policy to the same IAM user so it can actually use the repository.

Step 9: Still on the user's page, go to the Permissions tab → Add permissions ▼ → Attach policies directly → search for and select AWSCodeCommitPowerUser → Next → Add permissions.

Step 10: Clone And Add Application Code
git clone https://git-codecommit.us-east-1.amazonaws.com/v1/repos/cicd-demo-app
cd cicd-demo-app
python

Step 02: Add the Application Code Locally

mkdir cicd-demo-app
cd cicd-demo-app
git init
git branch -M main

Step 03: Create the Lambda function code `app.py`

import json

VERSION = "1.0.0"

def lambda_handler(event, context):
    """Order API handler with version tracking."""
    return {
        'statusCode': 200,
        'headers': {'Content-Type': 'application/json'},
        'body': json.dumps({
            'message': 'Order API is running',
            'version': VERSION,
            'functionVersion': context.function_version
        })
    }

Step 04: Create the SAM template `template.yaml`

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: CI/CD Demo Application

Globals:
  Function:
    Timeout: 10
    Runtime: python3.13

Resources:
  OrderFunction:
    Type: AWS::Serverless::Function
    Properties:
      Handler: app.lambda_handler
      CodeUri: .
      AutoPublishAlias: live
      DeploymentPreference:
        Type: Canary10Percent5Minutes
        Alarms:
          - !Ref OrderFunctionErrorAlarm

  OrderFunctionErrorAlarm:
    Type: AWS::CloudWatch::Alarm
    Properties:
      AlarmDescription: Errors on OrderFunction
      Namespace: AWS/Lambda
      MetricName: Errors
      Dimensions:
        - Name: FunctionName
          Value: !Ref OrderFunction
      Statistic: Sum
      Period: 60
      EvaluationPeriods: 1
      Threshold: 1
      ComparisonOperator: GreaterThanOrEqualToThreshold
      TreatMissingData: notBreaching

Step 05: Create the buildspec `buildspec.yml`

version: 0.2

phases:
  install:
    runtime-versions:
      python: 3.13
    commands:
      - pip install aws-sam-cli

  build:
    commands:
      - sam build

  post_build:
    commands:
      - sam package --s3-bucket ${ARTIFACT_BUCKET} --output-template-file packaged.yaml

artifacts:
  files:
    - packaged.yaml

Step 06: Push the code

git add .
git commit -m "Initial application code"

Step 07: Push to Github

git remote add origin `https://github.com/YOUR_USERNAME/cicd-demo-app.git`
git push -u origin main

Part II

Set Up a CodeBuild Project

Step 01: Create an S3 Bucket for Artifacts

Open the S3 console → Create bucket → Account Regional namespace (recommended) → Bucket name: cicd-demo-artifacts

Keep defaults → Create bucket

Step 02: Create the CodeBuild Project

Open the CodeBuild console → Create project

Project configuration:

Project name: cicd-demo-build
Project type: Default project

▼ Source:

Source provider ▼: GitHub
Repository: Repository in my GitHub account
Repository: cicd-demo-app

⚠️ If you used CodeCommit instead of GitHub: Set Source provider to CodeCommit

▼ Environment:

Provisioning model: On-demand
Environment image: Managed image
Compute: EC2
Running mode: Container
Operating system: Amazon Linux
Runtime(s): Standard
Image: aws/codebuild/amazonlinux-x86_64-standard:5.0
Image version: Always use the latest image for this runtime version
Service role: New service role

▶ Additional Configuration: → Environment variables:

Name: ARTIFACT_BUCKET → Value: your S3 bucket name

▼ Buildspec:

Build specifications: Use a buildspec file
Buildspec name - optional: buildspec.yml

▼ Artifacts:

Type: Amazon S3
Bucket name: cicd-demo-artifacts-...
Name: build-output

Click Create build project

⚠️ Grant the CodeBuild role access to your artifact bucket.

The auto-created CodeBuild service role can write to CodeBuild's default locations, but sam package uploads to your custom ARTIFACT_BUCKET, and the role won't have permission by default.

If the build fails at post_build with an AccessDenied on S3:

Step 1: Go to IAM → Roles → the codebuild-cicd-demo-build-service-role
Step 2: Add permissions → Attach policies → attach AmazonS3FullAccess (or a scoped inline policy allowing s3:PutObject, s3:GetObject, s3:GetBucketLocation on your bucket)

💡 Also confirm your S3 bucket and CodeBuild project are in the same region

Step 03: Run a Test Build

Click Start build

⚠️ Watch the build logs in real time
Verify each phase completes: INSTALL → PRE_BUILD → BUILD → POST_BUILD
Check the S3 bucket for the packaged.yaml artifact

💡 CodeBuild charges by the minute. Build environments are ephemeral. They're created fresh for each build. Use the cache section in buildspec.yml to cache dependencies between builds and speed up subsequent builds. CodeBuild can also pull secrets from Parameter Store and Secrets Manager using the env section.

Part III

Create a CodePipeline

Step 01: Build the Pipeline

Step 01.1: Open the CodePipeline console → Create pipeline

Step 01.2: Choose creation option

Category: Build custom pipeline

Click Next

Step 01.3: Choose Pipeline settings

Pipeline name: cicd-demo-pipeline
Execution mode: queued
Service role: New service role

Click Next

Step 01.4: Add a source stage

Source provider ▼: GitHub (via GitHub App)
Connection: click Connect to GitHub → authorize the CodeConnections connection (reuse the one from CodeBuild if you already made it)
Repository name: YOUR_USERNAME/cicd-demo-app
Default branch: main
Output artifact format: CodePipeline default

Click Next

💡 CodeConnections. A connection is a reusable, managed OAuth link between AWS and GitHub (or Bitbucket/GitLab). CodePipeline uses it to detect pushes and pull source. No Git credentials or webhooks to manage yourself. The same connection works across CodePipeline and CodeBuild.

Step 01.5: Add build stage - optional

Build provider: Other build providers
▼ AWS CodeBuild
Project name: cicd-demo-build

Click Next

Step 01.6: Add test stage - optional

Click Next

Step 01.7: Add deploy stage

Deploy provider ▼: AWS CloudFormation
Region: United States (North Virginia)
Input artifacts: BuildArtifact
Action mode: Create or update a stack
Stack name: cicd-demo-app
Artifact name: BuildArtifact
File name: packaged.yaml
Capabilities: CAPABILITY_IAM CAPABILITY_AUTO_EXPAND
Role name: create/select a CloudFormation deployment role (see the warning below)

Click Next → Create pipeline

Step 02: Watch the Pipeline Execute

The pipeline starts automatically after creation

⚠️ Watch each stage transition: Source → Build → Deploy

Click on each stage to see details and logs

The deploy stage creates a CloudFormation stack with your Lambda function

Step 03: Add a Manual Approval Stage

Step 03.1: Click Edit on the pipeline

Step 03.2: Click Add stage between Build and Deploy

Step 03.3 Stage name: Approval

Click Add stage

Step 03.4: Click Add action group:

Action name: ManualApproval
Action provider: Manual approval
Click: Done

Click Save

💡Now the pipeline pauses at the Approval stage and waits for someone to approve before deploying.

Part IV

Configure Canary Deployment with Alarms

Step 01: Verify the Deployment Configuration

Open the CodeDeploy console → Applications

💡 You should see a deployment group created by SAM (named something like cicd-demo-app-ServerlessDeploymentApplication-*)

Click on it to see the deployment configuration: CodeDeployDefault.LambdaCanary10Percent5Minutes

Step 02: View the CloudWatch Alarm

Open the CloudWatch console → Alarms

💡 Find the OrderFunctionErrorAlarm created by the SAM template

It should be in OK state (no errors yet)

Step 03: Trigger a Deployment

Step 03.1: Update app.py locally

import json

VERSION = "2.0.0"

def lambda_handler(event, context):
    """Order API handler v2 — added health check."""
    action = event.get('action', 'default')

    if action == 'health':
        return {
            'statusCode': 200,
            'body': json.dumps({'status': 'healthy', 'version': VERSION})
        }

    return {
        'statusCode': 200,
        'headers': {'Content-Type': 'application/json'},
        'body': json.dumps({
            'message': 'Order API is running',
            'version': VERSION,
            'functionVersion': context.function_version,
            'improvement': 'Added health check endpoint'
        })
    }

Step 03.2: Push the change

git add .
git commit -m "v2.0.0 - Add health check"
git push origin main

💡 The pipeline triggers automatically

Step 03.3: After the deploy stage, go to CodeDeploy → watch the canary deployment:

10% of traffic shifts to version 2
CodeDeploy waits 5 minutes while monitoring the alarm
If no alarm fires, 100% shifts to version 2

Step 04: Test Automatic Rollback

Push a broken version that throws errors

import json

VERSION = "3.0.0-broken"

def lambda_handler(event, context):
    """Intentionally broken to test rollback."""
    raise Exception("Something went wrong!")

Push and let the pipeline deploy

💡 During the canary phase, the 10% of traffic hitting the new version generates errors

The CloudWatch alarm triggers → CodeDeploy automatically rolls back

All traffic returns to the previous working version

Part V

Set Up Automatic Rollback with CloudWatch Alarms

Step 01: Create Additional Alarms

Step 01.1: Open the CloudWatch console → Alarms → Create alarm

Step 01.2: Select metric: Lambda → By Function Name → OrderFunction → Duration

Step 01.3: Configure

Statistic: Average
Period: 60 seconds
Threshold: Greater than 5000 (5 seconds)
Evaluation periods: 2 of 3
Alarm name: OrderFunction-HighDuration

Click Create alarm

Step 01.4: Create another alarm for throttles

Metric: Throttles for OrderFunction
Threshold: Greater than 0
Period: 60 seconds
Alarm name: OrderFunction-Throttles

Step 02: Update the SAM Template `template.yaml` with Multiple

      DeploymentPreference:
        Type: Canary10Percent5Minutes
        Alarms:
          - !Ref OrderFunctionErrorAlarm
          - !Ref OrderFunctionDurationAlarm
          - !Ref OrderFunctionThrottleAlarm

Push the change to trigger a pipeline run with the updated alarm configuration.

💡 You can attach multiple CloudWatch alarms to a deployment preference. If any single alarm fires during the deployment window, CodeDeploy rolls back.

This gives you defense in depth: catch errors, latency spikes, and throttling issues. The alarms are only monitored during the active deployment window, not permanently.

🏗️ What You Built | 📘 Exam Concepts Recap

What You Built	Exam Concept
Connected a GitHub repo via CodeConnections	Source stage, managed source integrations
Built a CodeBuild project with `buildspec.yml`	Build phases, environment variables, artifacts
Created a CodePipeline (source → build → deploy)	Pipeline orchestration and artifact flow
Added a manual approval stage	Human gates before production deployment
Configured `Canary10Percent5Minutes` in SAM	Gradual traffic shifting with `DeploymentPreference`
Attached CloudWatch alarms to the deployment	Automatic rollback on alarm breach
Pushed a broken version and watched it roll back	Defense-in-depth deployment safety
Used `AutoPublishAlias: live`	Required for SAM safe deployments

⚠️ Clean Up Protocol

CodePipeline → Delete cicd-demo-pipeline
CodeBuild → Delete cicd-demo-build project
CloudFormation → Delete the cicd-demo-app stack (removes Lambda, alarms, CodeDeploy resources)
Developer Tools → Settings → Connections → delete the GitHub CodeConnections connection (and optionally delete/keep the GitHub repo yourself)
S3 → Empty and delete the artifacts bucket
IAM → Delete the service roles created for CodeBuild, CodePipeline, and the CloudFormation deploy role
CloudWatch → Delete any remaining log groups and alarms

Key Takeaways

CodePipeline orchestrates the pipeline but doesn't build or deploy: it connects CodeCommit (source), CodeBuild (build), and CodeDeploy (deploy) stages.
buildspec.yml has four phases: install, pre_build, build, post_build.
Use the env section to pull secrets from Parameter Store and Secrets Manager securely.
appspec.yml structure differs between Lambda (Resources + Hooks) and EC2 (files + permissions + hooks).
Canary shifts traffic in two steps (X% then 100%).
Linear shifts in equal increments.
Blue/Green creates a new environment and switches all traffic at once.
SAM DeploymentPreference requires AutoPublishAlias. It automatically creates CodeDeploy deployments with traffic shifting and alarm monitoring.
PreTraffic hooks run before traffic shifts: use for smoke tests. 10. PostTraffic hooks run after: use for integration tests. Both must call put_lifecycle_event_hook_execution_status.
Automatic rollback triggers when a CloudWatch alarm fires during deployment or when a lifecycle hook fails. This is the recommended approach for production.
API Gateway stage variables let you point different stages to different Lambda aliases: prod stage → prod alias, dev stage → dev alias.
CodePipeline artifacts are stored in S3. Each action consumes input artifacts and produces output artifacts that flow to the next stage.
Manual approval actions in CodePipeline create gates between stages: useful for requiring human sign-off before production deployments.

Additional Resources

⚠️ The Future of AWS CodeCommit ⚠️

🏗️

Automate Deployment Testing | 🏗️ Build A Version Management System

Ntombizakhona Mabaso — Sat, 04 Jul 2026 10:19:21 +0000

Exam Guide: Developer - Associate
🏗️ Domain 3: Deployment
📘 Task 3: Automate Deployment Testing

This task is about automating tests as part of your deployment workflow: managing Lambda versions and aliases, using weighted aliases for canary deployments, deploying and reviewing IaC templates with change sets, and creating test events for automated validation. Understand version management, safe deployment strategies, and how to preview infrastructure changes before applying them.

📘 Concepts

Lambda Versions vs Aliases

Concept	What It Is	Mutable?	Use Case
$LATEST	The default, mutable version of your function	Yes	Development and testing only
Published version (v1, v2, v3...)	An immutable snapshot of code + configuration	No	Production references, rollback targets
Alias	A named pointer to a specific version	Yes (can be repointed)	Environment mapping (dev, staging, prod)

💡 $LATEST is mutable: every code update changes it. Published versions are immutable: once published, the code and configuration can't change. Aliases are mutable pointers to immutable versions. Event source mappings in production should reference aliases, never $LATEST.

$LATEST vs Published Versions

Aspect	$LATEST	Published Version
Code Changes	Updated on every deploy	Frozen at publish time
Configuration Changes	Updated immediately	Frozen at publish time
Concurrency Settings	Shared across all unpublished invocations	Can have provisioned concurrency per version
ARN Format	`arn:aws:lambda:region:account:function:name`	`arn:aws:lambda:region:account:function:name:1`
Use In Production	No (too risky)	Yes (immutable and predictable)

Weighted Aliases for Canary Deployments

An alias can split traffic between two versions using routing configuration:

Deployment Type	Traffic Split	`SAM Config`	Use Case
AllAtOnce	100% to new version immediately	`Type: AllAtOnce`	Non-critical functions, dev environments
Canary	X% to new, then 100% after interval	`Type: Canary10Percent5Minutes`	Test with small traffic before full rollout
Linear	Increase by X% every N minutes	`Type: Linear10PercentEvery1Minute`	Gradual rollout with monitoring

💡 Weighted aliases route a percentage of invocations to a secondary version. If the canary version triggers a CloudWatch alarm, CodeDeploy automatically rolls back to the previous version. SAM's AutoPublishAlias + DeploymentPreference handles this automatically.

IaC Templates

SAM vs CloudFormation

Feature	SAM	CloudFormation
What It Is	Extension of CloudFormation with serverless shortcuts	AWS's core IaC service
Template Format	YAML/JSON with `Transform: AWS::Serverless-2016-10-31`	YAML/JSON
Serverless Resources	`AWS::Serverless::Function`, `AWS::Serverless::Api`, etc.	Must define each resource individually (Lambda, API GW, IAM roles)
Deployment	`sam deploy` (wraps CloudFormation)	`aws cloudformation deploy`
Local Testing	`sam local invoke`, `sam local start-api`	Not available
Policy Templates	`DynamoDBCrudPolicy`, `S3ReadPolicy`, etc.	Write full IAM policies manually

CloudFormation Change Sets

Concept	What It Does
Change Set	A preview of what CloudFormation will add, modify, or delete before you apply it
Create	Generates the change set without applying anything
Review	Shows each resource and whether it will be Added, Modified, or Removed
Execute	Applies the changes to the stack
Delete	Discards the change set without applying
Replacement	Indicates a resource will be deleted and recreated (data loss risk!)

💡 Change sets are critical for production deployments. If a change set shows "Replacement" for a DynamoDB table or RDS instance, that means the resource will be deleted and recreated. You'll lose data. Always review change sets before executing in production. SAM's confirm_changeset = true in samconfig.toml forces this review.

Amazon Q Developer for Test Generation

Capability	What It Does
Unit Test Generation	Generates pytest/unittest tests from your function code
Mock Setup	Creates moto and unittest.mock boilerplate
Edge Case Detection	Identifies missing input validation, error handling gaps
Test Coverage	Suggests tests for happy path, error cases, and boundary conditions

💡Know that Amazon Q Developer can generate tests and suggest improvements. But, just use Kiro. 👇

⚠️ Amazon Q Developer end-of-support announcement ⚠️

🏗️ Build A Version Management System

Build a Version Management System that demonstrates deployment testing automation:

Published Lambda versions with immutable snapshots
Aliases pointing to specific versions for environment management
Weighted aliases routing traffic for canary testing
CloudFormation change sets reviewed and executed through the console
Lambda test events created and used for automated validation

Prerequisites

An AWS Account

Part I

Publish Lambda Versions

Step 01: Create the Lambda Function

Step 01.1: Open the Lambda console → Create function

Function name: VersionedOrderAPI
Runtime: Python 3.13

Click Create function

Step 01.2: Paste this code

import json

VERSION = "1.0.0"

def lambda_handler(event, context):
    """
    Version 1 of the Order API.
    Each published version is an immutable snapshot of this code.
    """
    return {
        'statusCode': 200,
        'headers': {'Content-Type': 'application/json'},
        'body': json.dumps({
            'message': 'Order API',
            'version': VERSION,
            'functionVersion': context.function_version,
            'action': event.get('action', 'none')
        })
    }

Step 01.3: Click Deploy

Step 02: Publish Version 1

Step 02.1: Click Actions ▼ → Publish new version

Step 02.2: Version description - optional: v1 - Initial release

Click Publish

💡 You're now viewing Version 1: notice the Function ARN includes :1 at the end

Step 02.3: Click Version 1 ▼ dropdown → select Qualifier: VersionedOrderAPI to go back to the editable version

Step 03: Update and Publish Version 2

Step 03.1: Back on VersionedOrderAPI, update the code:

import json

VERSION = "2.0.0"

def lambda_handler(event, context):
    """
    Version 2 of the Order API — added order validation.
    This version is a new immutable snapshot.
    """
    action = event.get('action', 'none')

    # New in v2: input validation
    if action == 'create' and not event.get('customerId'):
        return {
            'statusCode': 400,
            'body': json.dumps({'error': 'customerId is required'})
        }

    return {
        'statusCode': 200,
        'headers': {'Content-Type': 'application/json'},
        'body': json.dumps({
            'message': 'Order API',
            'version': VERSION,
            'functionVersion': context.function_version,
            'action': action,
            'improvement': 'Added input validation'
        })
    }

Step 03.2: Click Deploy

Step 03.3: Click Actions ▼ → Publish new version

Step 03.4 Version description - optional: v2 - Added input validation

Click Publish

Step 04: Verify Versions

Step 04.1 Click the Version 2 ▼ dropdown, you should see:

Qualifier: VersionOrderAPI — the mutable working copy
Version: 2 — v1 initial release (immutable)
Version: 1 — v2 with validation (immutable)

💡 Each published version gets its own ARN (function-name:1, function-name:2). The code and configuration are frozen. You can't edit a published version. You must publish a new one. This is what makes rollbacks possible: just point the alias back to the previous version.

Part II

Create Aliases Pointing to Versions

Step 01: Create the Production Alias

Step 01.1: On the VersionedOrderAPI function page, click the Aliases tab

Step 01.2: Click Create alias

Step 01.3: Create alias

Name: prod
Description - optional: Production traffic
Version: 1

Click Save

Step 02: Create the Staging Alias

Step 02.1: Click Create alias again

Step 02.2: Create alias

Name: staging
Description - optional: Staging and QA testing
Version: 2

Click Save

Step 03: Create the Dev Alias

Step 03.1: Click Create alias again

Step 02.2: Create alias

Name: dev
Description - optional: Development and testing
Version: $LATEST

Click Save

Step 04: Test Each Alias

Step 04.1: Click on the prod alias

Step 04.2: Go to the Test tab → create a new event:

{"action": "create", "customerId": "CUST-001"}

Step 04.3: Click Test

💡The response should show "version": "1.0.0" and "functionVersion": "1"

Step 04.4: Go back and click on the staging alias

Step 04.5: Run the same test

💡The response should show "version": "2.0.0" and "functionVersion": "2"

Step 05: Update an Alias to Point to a New Version

Step 05.1: Click on the prod alias → Configuration → Edit

Step 05.2: Change Version to 2

Click Save

⚠️ Now production traffic uses version 2. If something goes wrong, you can repoint prod back to version 1 instantly.

💡Aliases decouple consumers from specific versions. Event source mappings, API Gateway integrations, and other services should reference the alias ARN (function-name:prod), not a version number. This lets you promote versions by updating the alias, not reconfiguring every consumer.

Part III

Set Up Weighted Aliases for Canary Testing

Step 01: Configure Traffic Splitting

Step 01.1: Click on the prod alias → Configuration → Edit

Version: 1 (current stable)

Step 01.2: Under Weighted alias ▼

Additional version - optional: 2
Weight 10 (meaning 10% of traffic goes to v2)

Click Save

💡 Now 90% of invocations through the prod alias go to version 1, and 10% go to version 2.

Step 02: Test the Traffic Split

Step 02.1: Go to the Test tab on the prod alias

Step 02.2: Run the test event multiple times (at least 10 times)

Step 02.3: Observe the functionVersion in the responses. Most will show 1, some will show 2

Step 03: Monitor the Canary

Step 03.1: Open the CloudWatch console → Metrics → Lambda

Step 03.2: Look at By Function Name metrics

💡 You can see invocation counts and error rates per version
If version 2 has higher error rates, revert by removing the weighted configuration

Step 04: Remove the Canary (Promote or Rollback)

Step 04.1: To promote v2 to 100%:
Edit the prod alias → set Version to 2 → remove the weighted configuration → Save

Step 04.2: To rollback to v1:
Edit the prod alias → set Version to 1 → remove the weighted configuration → Save

💡 In SAM, weighted aliases are managed automatically with DeploymentPreference. SAM creates a CodeDeploy deployment that shifts traffic gradually and monitors CloudWatch alarms. If an alarm fires, CodeDeploy rolls back automatically. For example, Canary10Percent5Minutes and Linear10PercentEvery1Minute.

Part IV

Create and Review CloudFormation Change Sets

Step 01: Create a CloudFormation Stack

Step 01.1: Open the CloudFormation console

Step 01.2: Click Create stack ▼ → With new resources (standard)

Step 01.3: Create stack

Prepare template: Choose an existing template
Template source: Upload a template file

Step 01.4: Create and upload this template

AWSTemplateFormatVersion: '2010-09-09'
Description: Version Management Demo Stack

Parameters:
  Stage:
    Type: String
    Default: dev
    AllowedValues: [dev, staging, prod]

Resources:
  OrdersTable:
    Type: AWS::DynamoDB::Table
    Properties:
      TableName: !Sub "${Stage}-version-demo-orders"
      BillingMode: PAY_PER_REQUEST
      KeySchema:
        - AttributeName: PK
          KeyType: HASH
        - AttributeName: SK
          KeyType: RANGE
      AttributeDefinitions:
        - AttributeName: PK
          AttributeType: S
        - AttributeName: SK
          AttributeType: S
      Tags:
        - Key: Environment
          Value: !Ref Stage

  ConfigTable:
    Type: AWS::DynamoDB::Table
    Properties:
      TableName: !Sub "${Stage}-version-demo-config"
      BillingMode: PAY_PER_REQUEST
      KeySchema:
        - AttributeName: configKey
          KeyType: HASH
      AttributeDefinitions:
        - AttributeName: configKey
          AttributeType: S

Step 01.5: Click Next

Step 01.6: Specify stack details:

Stack name: version-demo-dev
Stage: dev

Step 01.7: Click Next → Next → Submit

⚠️ Wait for the stack to reach CREATE_COMPLETE

Step 02: Create a Change Set to Add a Resource

Step 02.1: Click on the version-demo-dev stack

Step 02.2: Click Stack actions ▼ → Create change set

Step 02.3: Create change set for version-demo-dev

Change set type: Standard change set
Prepare template: Replace existing template
Template source: Upload a template file

Step 02.4: Upload an updated template that adds a new resource (add this to the Resources section):

  AuditTable:
    Type: AWS::DynamoDB::Table
    Properties:
      TableName: !Sub "${Stage}-version-demo-audit"
      BillingMode: PAY_PER_REQUEST
      KeySchema:
        - AttributeName: PK
          KeyType: HASH
        - AttributeName: SK
          KeyType: RANGE
      AttributeDefinitions:
        - AttributeName: PK
          AttributeType: S
        - AttributeName: SK
          AttributeType: S

Step 02.5 Click Next

Step 02.6: Specify change set details

Change set name: add-audit-table

Step 02.7: Click Next → Next → Create change set

Step 03: Review the Change Set

Step 03.1: Click on the add-audit-table change set

Step 03.2: Review the Changes section

AuditTable → Action: Add → Resource type: AWS::DynamoDB::Table
OrdersTable | no change
ConfigTable | no change

💡 This is safe. Only adding a new resource

Step 04: Execute the Change Set

Click Execute change set → Click Execute change set to confirm

⚠️ Wait for the stack to reach UPDATE_COMPLETE

Go to the Resources tab. You should now see three DynamoDB tables

💡 Always check for "Replacement" in change sets. If a change set shows Replacement for a DynamoDB table, that table will be deleted and recreated. All data is lost. Common causes: changing the table name, changing the key schema, or changing certain properties that require replacement. Use DeletionPolicy: Retain on critical resources to prevent accidental deletion.

Part V

Create Lambda Test Events and Automate Testing

Step 01: Create Shareable Test Events

Step 01.1: Open the Lambda console → click on VersionedOrderAPI

Step 01.2: Go to the Test tab

Step 01.3: Click Create new event

Event name: CreateOrderValid
Event sharing settings: Shareable (visible to all IAM users)
Event JSON:

{
  "action": "create",
  "customerId": "CUST-001",
  "items": [{"productId": "PROD-A", "quantity": 2}]
}

Step 01.4: Click Save

Step 01.5: Create another test event:

Event name: CreateOrderMissinCustomer
Event sharing settings: Shareable
Event JSON:

{
  "action": "create",
  "items": [{"productId": "PROD-A", "quantity": 1}]
}

Step 01.6: Click Save

Step 01.7: Create a third test event

Event name: GetOrderStatus
Event sharing settings: Shareable
Event JSON:

{
  "action": "status",
  "orderId": "ORD-2024-001"
}

Step 01.8: Click Save

Step 02: Run All Test Events

Step 02.1: Select CreateOrderValid from the dropdown → click Test

Expected: 200 with version info

Step 02.2: Select CreateOrderMissingCustomer → click Test

Expected: 400 with validation error (on version 2)

Step 02.3 Select GetOrderStatus → click Test

Expected: 200 with status info

Step 03: Automate Testing in a CI/CD Pipeline

In a real pipeline, you'd automate these tests in a CodeBuild buildspec:

# buildspec.yml — automated testing in CI/CD
version: 0.2
phases:
  install:
    commands:
      - pip install pytest moto boto3
  build:
    commands:
      - sam build
  post_build:
    commands:
      # Run unit tests
      - python -m pytest tests/unit/ -v --junitxml=test-results/unit.xml

      # Deploy to test environment
      - sam deploy --config-env test --no-confirm-changeset

      # Get the deployed API URL from stack outputs
      - |
        API_URL=$(aws cloudformation describe-stacks \
          --stack-name my-app-test \
          --query 'Stacks[0].Outputs[?OutputKey==`ApiUrl`].OutputValue' \
          --output text)

      # Run integration tests against the deployed stack
      - API_URL=$API_URL python -m pytest tests/integration/ -v --junitxml=test-results/integration.xml

      # Invoke Lambda directly with test events
      - |
        aws lambda invoke \
          --function-name VersionedOrderAPI:staging \
          --payload '{"action":"create","customerId":"CUST-001"}' \
          response.json
        cat response.json

reports:
  test-reports:
    files:
      - "**/*.xml"
    base-directory: test-results

💡 Shareable test events are visible to all IAM users with access to the function which is useful for team collaboration. Private test events are only visible to the creator. In CI/CD, use aws lambda invoke with the --payload flag to run automated tests against specific aliases. Use --qualifier to target a specific version or alias.

🏗️ What You Built | 📘 Exam Concepts Recap

What You Built	Exam Concept
Published immutable Lambda versions (v1, v2)	Versions freeze code + config. `$LATEST` is mutable
Created `prod`/`staging`/`dev` aliases	Aliases as mutable pointers for environment mapping
Repointed the `prod` alias to a new version	Instant promotion and rollback via aliases
Configured a weighted alias (90/10 split)	Canary deployments with traffic shifting
Created and reviewed a CloudFormation change set	Previewing infra changes before applying
Watched for "Replacement" in the change set	Avoiding accidental resource deletion / data loss
Created shareable Lambda test events	Team-wide test standardization
Automated tests in a CodeBuild buildspec	Test automation in CI/CD with `aws lambda invoke`

⚠️ Clean Up Protocol

Lambda → Delete VersionedOrderAPI (this also deletes all versions and aliases)
CloudFormation → Delete the version-demo-dev stack (this deletes all three DynamoDB tables)
IAM → Delete Lambda execution roles
CloudWatch → Delete log groups

Key Takeaways

Versions are immutable snapshots of code + configuration.
Aliases are mutable pointers to versions.
$LATEST is mutable. Don't use it in production.
Weighted aliases split traffic between two versions for canary deployments. SAM's DeploymentPreference automates this with CodeDeploy.
Canary10Percent5Minutes sends 10% to the new version, waits 5 minutes, then shifts 100%. Linear10PercentEvery1Minute increases by 10% each minute.
Change sets preview CloudFormation changes before applying. Watch for "Replacement". Tt means resource deletion and recreation.
Event source mappings should reference aliases, not $LATEST or version numbers, so you can promote versions by updating the alias.
Shareable test events in the Lambda console are visible to all IAM users. Use them for team-wide test standardization.
Amazon Q Developer can generate unit tests, mock setup, and edge case tests from your function code. > ⚠️ Amazon Q Developer end-of-support announcement ⚠️
In CI/CD, use aws lambda invoke --qualifier alias-name to test specific versions and aws cloudformation describe-stacks to get deployed resource URLs for integration tests.

Additional Resources

⚠️ Amazon Q Developer end-of-support announcement ⚠️

🏗️

Test Applications In Development Environments | 🏗️ Build A Testing Toolkit

Ntombizakhona Mabaso — Thu, 02 Jul 2026 14:14:23 +0000

Exam Guide: Developer - Associate
🏗️ Domain 3: Deployment
📘 Task 2: Test Applications In Development Environments

Testing on AWS isn't just about unit tests. You need to test locally with SAM CLI, write integration tests against deployed stages, mock AWS services and external APIs, validate event-driven flows, and use API Gateway stages for environment isolation.

📘Concepts

The Testing Pyramid

Level	Speed	Scope	AWS Services	Tools
Unit Tests	Fast (ms)	Single function, no external calls	Mocked	pytest, moto, unittest.mock
Integration Tests	Medium (seconds)	Multiple services working together	Real (deployed)	pytest + requests, deployed API Gateway stage
End-to-end Tests	Slow (seconds–minutes)	Full user workflow	Real (deployed)	Selenium, Postman, curl against live API

💡 Unit tests use mocks (moto for AWS, unittest.mock for external APIs). Integration tests run against real deployed resources.

SAM Local Testing Capabilities

`Command`	What It Does	When to Use
`sam local invoke`	Runs a single Lambda function locally in a Docker container	Testing individual function logic with a specific event
`sam local start-api`	Starts a local API Gateway emulator	Testing full API request/response flow locally
`sam local start-lambda`	Starts a local Lambda endpoint for SDK calls	Testing code that invokes Lambda via the SDK
`sam local generate-event`	Generates sample event payloads for AWS services	Creating test events for S3, SQS, API Gateway, etc.

⚠️ Key Details:

SAM local uses Docker to simulate the Lambda runtime environment
It reads your template.yaml to understand function configuration
Environment variables can be overridden with --env-vars env.json
It does not mock AWS services, therefore if your function calls DynamoDB, it calls real DynamoDB (or you need moto)

💡 sam local invoke runs a function once and exits. sam local start-api keeps running and accepts HTTP requests. Both use Docker containers that match the Lambda runtime. Know that SAM local does NOT mock AWS services. Your function still needs real credentials or mocked services.

Mocking Strategies

What to Mock	Tool	How It Works
AWS Services (DynamoDB, S3, SQS, etc.)	moto (`@mock_aws`)	Intercepts boto3 calls and returns realistic responses
External HTTP APIs	unittest.mock (`@patch`)	Replaces the HTTP client with a mock that returns controlled responses
Environment Variables	`os.environ` or `monkeypatch`	Set test values for TABLE_NAME, STAGE, etc.
Lambda Context	Custom object	Create a simple object with `function_name`, `memory_limit_in_mb`, etc.

API Gateway Stages for Environment Isolation

Stage	`URL Pattern`	Use Case
dev	`https://{api-id}.execute-api.{region}.amazonaws.com/dev`	Development and local integration testing
staging	`https://{api-id}.execute-api.{region}.amazonaws.com/staging`	QA and pre-production testing
prod	`https://{api-id}.execute-api.{region}.amazonaws.com/prod`	Production traffic

Each stage can have its own:

Stage Variables: different table names, feature flags, log levels per stage
Throttling Settings: different rate limits per stage
Logging Configuration: verbose logging in dev, errors only in prod

💡 Stage variables are accessible in Lambda via event['stageVariables']. They let you point different stages at different backends (different DynamoDB tables, different Lambda aliases) without changing code.

Event-Driven Testing Approaches

Approach	What It Tests	How
SAM Local With Generated Events	Function logic with realistic payloads	`sam local generate-event` → `sam local invoke`
Console Test Events	Function logic in the deployed environment	Create test events in the Lambda console
EventBridge Test Events	Rule matching and target invocation	Send test events via the EventBridge console
SQS Test Messages	Queue-triggered function processing	Send messages via the SQS console
CloudWatch Logs	Verify function executed correctly	Check logs after sending test events

🏗️ Build A Testing Toolkit

Build a Testing Toolkit that demonstrates testing approaches:

Local Lambda testing with SAM CLI (invoke and start-api)
Unit tests using moto to mock DynamoDB and S3
Integration tests running against a deployed API Gateway stage
EventBridge rule testing with test events in the console
Sample event generation with sam local generate-event

Prerequisites

⚠️ Before You Start Read This

💡 SAM local does NOT mock AWS. sam local invoke and sam local start-api run your code in a container but call real AWS services with your real credentials. So we create the DynamoDB table first.

💡 API Gateway routes come from the template, not your code. A path only exists if it's declared in the template's Events. GET / will always say "Missing Authentication Token" because there's no root route.

💡 Region must be consistent across your credentials, the table, and env.json. We pin us-east-1 everywhere so change it only if you use a different region, but change it everywhere.

💡 Windows files need UTF-8 without BOM. A BOM causes Unable to unmarshal input: Expecting value: line 1 column 1. Create JSON files in VS Code and check the encoding in the bottom-right status bar.

💡 After editing template.yaml, always sam build then restart start-api. The local server serves the built template in .aws-sam/build/, not your edited source.

Step 00: Set your region once for this session so every command agrees:

$env:AWS_REGION = "us-east-1"
aws configure set region us-east-1

⚠️ If you already use a different region, substitute it consistently in every command and file below.

Step 01: Create the DynamoDB Table First

⚠️ Because SAM local calls real DynamoDB, the table must exist before you invoke anything.

aws dynamodb create-table `
  --table-name dev-orders `
  --attribute-definitions AttributeName=PK,AttributeType=S AttributeName=SK,AttributeType=S `
  --key-schema AttributeName=PK,KeyType=HASH AttributeName=SK,KeyType=RANGE `
  --billing-mode PAY_PER_REQUEST `
  --region us-east-1

💡 The backtick ` is PowerShell's line-continuation character. On macOS/Linux/CloudShell, use \ instead.

⚠️ Wait until the table is ACTIVE:

aws dynamodb describe-table --table-name dev-orders --region us-east-1 --query "Table.TableStatus"

⚠️ Repeat until it prints "ACTIVE" (usually a few seconds).

💡 The table's key schema mirrors single-table design. PK (partition key) + SK (sort key) let one table hold many item types.

Step 02: Create the SAM Project

sam init --runtime python3.13 --name testing-toolkit --app-template hello-world
cd testing-toolkit

💡This scaffolds a project with hello_world/app.py, template.yaml, and tests/.

Step 03: Replace the Function Code

Open hello_world/app.py and replace its entire contents with this:

import json
import os
from datetime import datetime

import boto3

# 💡 Initialize the client OUTSIDE the handler so it's reused across warm
# invocations. Creating it inside the handler would rebuild it every call.
dynamodb = boto3.resource("dynamodb")
TABLE_NAME = os.environ.get("TABLE_NAME", "dev-orders")


def lambda_handler(event, context):
    """Order API: create an order (POST) or fetch one (GET)."""
    table = dynamodb.Table(TABLE_NAME)
    method = event.get("httpMethod", "GET")
    path = event.get("path", "/")

    try:
        # --- Create an order ---
        if method == "POST" and path == "/orders":
            body = json.loads(event.get("body") or "{}")
            order_id = f"ORD-{datetime.utcnow().strftime('%Y%m%d%H%M%S')}"

            table.put_item(Item={
                "PK": f"ORDER#{order_id}",
                "SK": "METADATA",
                "orderId": order_id,
                "customerId": body.get("customerId", "unknown"),
                "items": body.get("items", []),
                "status": "created",
                "createdAt": datetime.utcnow().isoformat(),
            })

            return _response(201, {"orderId": order_id, "status": "created"})

        # --- Get a single order ---
        if method == "GET" and path.startswith("/orders/"):
            order_id = event["pathParameters"]["orderId"]
            result = table.get_item(Key={"PK": f"ORDER#{order_id}", "SK": "METADATA"})
            item = result.get("Item")
            if not item:
                return _response(404, {"error": f"Order {order_id} not found"})
            return _response(200, item)

        return _response(200, {"message": "Testing Toolkit API",
                               "stage": os.environ.get("STAGE", "local")})

    except Exception as e:
        # 💡 Always handle exceptions. An uncaught error becomes a generic
        # 500 "Internal server error" at API Gateway, hiding the real cause.
        print(f"ERROR: {type(e).__name__}: {e}")
        return _response(500, {"error": type(e).__name__, "detail": str(e)})


def _response(status_code, body):
    return {
        "statusCode": status_code,
        "headers": {"Content-Type": "application/json"},
        "body": json.dumps(body, default=str),
    }

💡 The try/except matters for debugging. Without it, a failed DynamoDB call throws an uncaught exception and API Gateway returns a bare Internal server error. By catching it and returning the error name, you can see why it failed in the HTTP response itself.

Step 04: Fix the Template Routes

⚠️ The default template only maps GET /hello. Open template.yaml and replace the entire Resources: section's HelloWorldFunction so it matches your code's routes. Your HelloWorldFunction block should look exactly like this:

  HelloWorldFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: hello_world/
      Handler: app.lambda_handler
      Runtime: python3.13
      Timeout: 10
      Environment:
        Variables:
          TABLE_NAME: dev-orders
          STAGE: local
      Policies:
        - DynamoDBCrudPolicy:
            TableName: dev-orders
      Events:
        CreateOrder:
          Type: Api
          Properties:
            Path: /orders
            Method: post
        GetOrder:
          Type: Api
          Properties:
            Path: /orders/{orderId}
            Method: get

💡 Routes are configuration, not code. API Gateway only serves paths declared under Events. Your function's internal if path == "/orders" logic never runs unless the template routes that path to it. This is why unknown paths return "Missing Authentication Token."

💡 DynamoDBCrudPolicy is a SAM policy template. It grants least-privilege CRUD access to just the named table, instead of a broad AmazonDynamoDBFullAccess.

Step 05: Create the Environment Variable File

⚠️ SAM local reads env vars from a JSON file. Create a file named env.json in the project root (same folder as template.yaml) with this content:

{
  "HelloWorldFunction": {
    "TABLE_NAME": "dev-orders",
    "STAGE": "local",
    "AWS_REGION": "us-east-1"
  }
}

⚠️ Windows Issue: Create this file in VS Code. Look at the bottom-right status bar. If it says "UTF-8 with BOM", click it → Save with Encoding → UTF-8. A BOM breaks JSON parsing with Expecting value: line 1 column 1 (char 0).

💡 The top-level key is the function's logical ID. HelloWorldFunction matches the resource name in template.yaml. Pinning AWS_REGION here guarantees the function looks for the table in the same region you created it.

Step 06: Create a Test Event File

Create a file create_order.json inside the events/ folder :

{
  "httpMethod": "POST",
  "path": "/orders",
  "headers": { "Content-Type": "application/json" },
  "pathParameters": null,
  "body": "{\"customerId\": \"CUST-001\", \"items\": [{\"productId\": \"PROD-A\", \"quantity\": 2}]}"
}

💡 This mimics the API Gateway proxy event. When API Gateway invokes Lambda, it wraps the HTTP request in this structure. body is a JSON string (escaped), not an object. That's why the code does json.loads(event["body"]).

⚠️ Save as UTF-8 without BOM.

Step 07: Build

sam build

💡 Always build after editing template.yaml or your code. sam build copies everything into .aws-sam/build/, which is what sam local actually runs. Skipping this means you test stale code/routes.

⚠️ If sam build complains it can't find Python 3.13, run sam build --use-container (needs Docker running). It builds inside a Python 3.13 image so your local Python version doesn't matter.

Step 08: Test with `sam local invoke` (Direct Function Test)

This invokes the function once with your test event. No API Gateway involved.

sam local invoke HelloWorldFunction -e events/create_order.json --env-vars env.json

Expected Result: a 201 response with an orderId, and no errors.

💡 sam local invoke bypasses API Gateway. It calls the handler directly with the event you pass. Routing doesn't matter here, only the function logic and its (real) AWS calls. Because the dev-orders table exists in us-east-1, the put_item succeeds.

⚠️ If you see ResourceNotFoundException, the table isn't in the region the function is using.

Step 09: Verify the item actually landed in DynamoDB:

aws dynamodb scan --table-name dev-orders --region us-east-1 --query "Items"

You should see the order you just created.

Step 10: Test with `sam local start-api` (Full HTTP Test)

Now test the real HTTP routing. Start the local API in one terminal:

sam local start-api --port 3000 --env-vars env.json

Leave this running.

💡 The function logs (including real errors) print in THIS terminal. If an HTTP call returns 500, look here for the actual stack trace.

Open a second PowerShell terminal and send requests.

Step 11: Create an order (POST)

⚠️ On Windows, use Invoke-RestMethod. PowerShell's curl is an alias for Invoke-WebRequest with different syntax, and inline curl.exe JSON escaping is painful.

Invoke-RestMethod -Uri http://localhost:3000/orders -Method Post -ContentType "application/json" -Body '{"customerId": "CUST-001", "items": [{"productId": "PROD-A", "quantity": 2}]}'

macOS / Linux / CloudShell equivalents

curl -X POST http://localhost:3000/orders \
  -H "Content-Type: application/json" \
  -d '{"customerId": "CUST-001", "items": [{"productId": "PROD-A", "quantity": 2}]}'

curl http://localhost:3000/orders/ORD-20260702093000

Expected Result: an object with orderId and status: created.

Step 12: Get that order (GET)

Copy the orderId from the previous response and:

Invoke-RestMethod -Uri http://localhost:3000/orders/ORD-20260702093000

(Replace with your actual order ID.)

Expected Result: the full order item.

orderId    : ORD-20260702112404
status     : created
createdAt  : 2026-07-02T11:24:04.213247
PK         : ORDER#ORD-20260702112404
items      : {@{productId=PROD-A; quantity=2}}
customerId : CUST-001
SK         : METADATA

💡 Expected "errors" that are actually correct:

GET http://localhost:3000/ → "Missing Authentication Token:" there's no root route. Fine.

GET http://localhost:3000/orders (no ID) → "Missing Authentication Token:" — POST /orders and GET /orders/{orderId} exist. Fine.

Stop the server with Ctrl+C when done. If port 3000 stays stuck:

Get-Process -Id (Get-NetTCPConnection -LocalPort 3000).OwningProcess | Stop-Process -Force

Step 13: Unit Tests with moto (No AWS Needed)

💡 It needs no real AWS, no Docker, no network. moto mocks AWS services in memory, so tests are fast and isolated.

pip install pytest moto boto3

⚠️ Windows Path-Length If you installed Python from the Microsoft Store, its packages live under a very long base path (AppData\Local\Packages\PythonSoftwareFoundation...\LocalCache\...), and moto's deeply nested files push past Windows' 260-character limit, failing with OSError: [Errno 2] No such file or directory. Note that moto[dynamodb] does not help The extra only changes dependencies, not the files installed. Two fixes:

Fix A (permanent, recommended): Enable long paths. In an Administrator PowerShell, run New-ItemProperty -Path "HKLM:\SYSTEM\CurrentControlSet\Control\FileSystem" -Name "LongPathsEnabled" -Value 1 -PropertyType DWORD -Force, then reboot and re-run the install.

Fix B (no admin): Use a virtual environment at a short path so the total path stays under the limit: python -m venv C:\venv then C:\venv\Scripts\Activate.ps1 then pip install pytest moto boto3. Don't put the venv inside the deep project folder. Use C:\venv.

Step 14: Create `tests/unit/test_app.py`:

import json
import os

import boto3
import pytest
from moto import mock_aws


@pytest.fixture
def dynamodb_table():
    """💡 moto intercepts boto3 calls and simulates DynamoDB in memory.
    We create the table inside the mock BEFORE importing the handler."""
    with mock_aws():
        os.environ["AWS_DEFAULT_REGION"] = "us-east-1"
        os.environ["TABLE_NAME"] = "dev-orders"

        client = boto3.resource("dynamodb", region_name="us-east-1")
        client.create_table(
            TableName="dev-orders",
            KeySchema=[
                {"AttributeName": "PK", "KeyType": "HASH"},
                {"AttributeName": "SK", "KeyType": "RANGE"},
            ],
            AttributeDefinitions=[
                {"AttributeName": "PK", "AttributeType": "S"},
                {"AttributeName": "SK", "AttributeType": "S"},
            ],
            BillingMode="PAY_PER_REQUEST",
        )
        yield


def test_create_order_returns_201(dynamodb_table):
    # 💡 Import the handler AFTER the mock is active so its boto3
    # client is intercepted by moto.
    from hello_world.app import lambda_handler

    event = {
        "httpMethod": "POST",
        "path": "/orders",
        "body": json.dumps({
            "customerId": "CUST-001",
            "items": [{"productId": "PROD-A", "quantity": 2}],
        }),
    }

    result = lambda_handler(event, None)

    assert result["statusCode"] == 201
    body = json.loads(result["body"])
    assert body["status"] == "created"
    assert body["orderId"].startswith("ORD-")


def test_get_missing_order_returns_404(dynamodb_table):
    from hello_world.app import lambda_handler

    event = {
        "httpMethod": "GET",
        "path": "/orders/DOESNOTEXIST",
        "pathParameters": {"orderId": "DOESNOTEXIST"},
    }

    result = lambda_handler(event, None)
    assert result["statusCode"] == 404

Step 15: Run the tests

python -m pytest tests/unit/ -v

Expected Result: both tests pass with no real AWS calls.

💡 Unit vs Integration: these moto tests are unit tests (fast, isolated, mocked). The Invoke-RestMethod calls against start-api in Step 9 hit real DynamoDB, closer to integration tests. The difference: mock for unit tests, real deployed resources for integration tests.

Step 16: Deploy and Run Integration Tests Against a Real Stage

💡 Unit Tests (moto) prove your logic. Integration Tests prove the whole stack works (API Gateway, Lambda, and DynamoDB together) against a real deployed environment.

sam deploy --guided

💡 sam deploy provisions real infrastructure via CloudFormation: the Lambda, the API Gateway stage, IAM roles, and the event mappings. Unlike sam local, this creates resources that cost money and must be cleaned up.

Step 17: When it finishes, copy the API Gateway endpoint URL from the `Outputs`. It looks like:

https://abc123.execute-api.us-east-1.amazonaws.com/Prod/

⚠️ Note the stage is Prod (SAM's default), so your full path is .../Prod/orders.

Step 18: Run integration tests against the deployed stage

Create tests/integration/test_api.py:

import os
import requests

# 💡 Pass the deployed URL via an environment variable so the same tests
# can run against dev, staging, or prod without code changes.
BASE_URL = os.environ["API_URL"].rstrip("/")


def test_create_and_get_order():
    # Create
    create = requests.post(f"{BASE_URL}/orders", json={
        "customerId": "CUST-TEST-001",
        "items": [{"productId": "PROD-A", "quantity": 1}],
    })
    assert create.status_code == 201
    order_id = create.json()["orderId"]

    # Retrieve the same order
    get = requests.get(f"{BASE_URL}/orders/{order_id}")
    assert get.status_code == 200
    assert get.json()["orderId"] == order_id


def test_missing_order_returns_404():
    resp = requests.get(f"{BASE_URL}/orders/DOESNOTEXIST")
    assert resp.status_code == 404

Step 19: Install `requests` and run the tests, passing your real URL:

pip install requests
$env:API_URL = "https://abc123.execute-api.us-east-1.amazonaws.com/Prod"
python -m pytest tests/integration/ -v

Expected Result: both tests pass against live AWS.

💡 Testing pyramid in action. Many fast unit tests (moto), fewer integration tests (deployed stage), fewest end-to-end tests. Integration tests catch problems mocks can't: IAM permissions, API Gateway mapping, real DynamoDB behaviour.

Step 20: Test an Event-Driven Flow with EventBridge

Not everything is an API. Let's test an event-driven path using the EventBridge console.

Create a rule and target (console)

Step 20.1 Open the EventBridge console → Rules → Create rule

Step 20.2: Define rule detail

Name: order-placed-rule
Event bus: default

Click Next

Step 20.3: Build event pattern

Event source: Other
Creation method: Custom pattern (JSON editor).
Paste:

{
  "source": ["orders.api"],
  "detail-type": ["OrderPlaced"]
}

Click Next

Step 20.4: Select target(s)

Target: AWS service → Lambda function
Target location: Target in this current account
Function: testing-toolkit-HelloWorldFunction...

Click Next → Next

Step 20.5: Click Create rule

Step 21: Send a test event

Step 21.1 EventBridge console → Event buses → default → Send events

Step 21.2 Send events

Event source: orders.api
Detail type: OrderPlaced
**Event detail:

{ "orderId": "ORD-EVT-001", "customerId": "CUST-EVT", "items": [] }

Click Send

Step 22: Verify it was delivered

Open the Lambda function → Monitor → View CloudWatch logs → newest log stream. You'll see the event was received.

💡 Event pattern matching is exact by default. source and detail-type must match precisely. Content-based filtering (prefix, numeric, exists) can match on detail fields. Sending test events from the console is the fastest way to validate rules.

⚠️ Our handler expects an API Gateway event shape, so it'll log the EventBridge event and likely return the default message. That's fine. The point here is proving the rule → target delivery, not the handler's response.

Step 23: Generate Realistic Test Events with SAM

Hand-writing event JSON is error-prone. sam local generate-event produces the exact structure AWS services send.

# API Gateway POST event
sam local generate-event apigateway aws-proxy --method POST --path /orders --body '{\"customerId\":\"CUST-001\"}' > events/api_post.json

# S3 object-created event
sam local generate-event s3 put --bucket my-bucket --key orders/data.json > events/s3_put.json

# SQS message event
sam local generate-event sqs receive-message > events/sqs_message.json

# EventBridge (CloudWatch) scheduled event
sam local generate-event cloudwatch scheduled-event > events/scheduled.json

List what's available:

sam local generate-event --help
sam local generate-event s3 --help

💡 Generated events include all the metadata (request IDs, ARNs, timestamps) real triggers send, so your local tests behave like production. This is far more reliable than guessing the event shape by hand.

Troubleshooting

`Symptom`	Cause	Fix
`Missing Authentication Token`	Path/method not in template `Events`	Use a declared route; `sam build` + restart after edits
`ResourceNotFoundException`	Table missing or wrong region	Create table in the region from `env.json` & keep regions consistent
`Internal server error` (500)	Uncaught exception in the function	Read the `start-api` terminal for the real stack trace
`Expecting value: line 1 column 1`	JSON file saved with a BOM	Save `env.json`/event files as UTF-8 without BOM
`Binary validation failed for python`	Python 3.13 not on PATH	`sam build --use-container` or install Python 3.13
`curl` fails in PowerShell	`curl` is an alias for `Invoke-WebRequest`	Use `Invoke-RestMethod`, or `curl.exe` with a body file
Port 3000 in use	Old `start-api` still running	`Ctrl+C`, or kill the PID on port 3000

⚠️ Clean Up Protocol

1 Delete the deployed stack

sam delete --stack-name testing-toolkit --region us-east-1

2 Delete the DynamoDB table

aws dynamodb delete-table --table-name dev-orders --region us-east-1

3 EventBridge → delete order-placed-rule
4 CloudWatch → delete the function's log groups
5 Remove the local testing-toolkit folder if you're done

💡 sam delete tears down the whole CloudFormation stack, so you don't have to hunt for individual resources. The manually-created DynamoDB table isn't part of the stack, so delete it separately.**

🏗️ What You Built | 📘 Exam Concepts Recap

What You Built	Exam Concept
Created the table before invoking	SAM local calls real* AWS, it does not mock services*
Pinned region in `env.json` and every command	Region consistency across credentials, resources, and config
Declared routes in the template `Events`	API Gateway routing is configuration, not code
Used `DynamoDBCrudPolicy`	SAM policy templates for least privilege
`sam local invoke` with an event file	Direct function testing, bypassing API Gateway
`sam local start-api` + HTTP calls	Local API Gateway emulation (integration-style)
Wrapped the handler in `try/except`	Uncaught errors surface as generic 500s
moto `@mock_aws` unit tests	Mocking AWS in memory for fast, isolated unit tests
Imported the handler after the mock	moto must be active before boto3 clients are created
`sam deploy --guided` + integration tests	Testing against real deployed resources
Passed the API URL via `$env:API_URL`	Same tests run against any environment
EventBridge rule + test event	Validating event-driven delivery and pattern matching
`sam local generate-event`	Realistic event payloads for S3, SQS, API GW, etc.
`sam delete` for teardown	CloudFormation stack removal
Recognised "Missing Authentication Token"	Generic API Gateway "no matching route" message

Additional Resources

🏗️

Prepare Application Artifacts To Be Deployed To AWS | 🏗️ Build A Multi-Environment Serverless App

Ntombizakhona Mabaso — Wed, 01 Jul 2026 15:49:12 +0000

Exam Guide: Developer - Associate
🏗️ Domain 3: Deployment
📘 Task 1: Prepare Application Artifacts To Be Deployed To AWS

Before you can deploy anything to AWS, you need to package it properly. This task covers Lambda deployment packaging (zip vs container), managing dependencies, structuring projects for multi-environment deployment, and using AWS AppConfig for runtime configuration.

📘Concepts

Lambda Deployment Packaging Options

Option	Max Size	Build Complexity	Cold Start	Best For
Zip Package (inline editor)	3 MB (editor limit)	None	Fastest	Simple functions, no dependencies
Zip Package (upload)	50 MB compressed / 250 MB uncompressed	Low	Fast	Most Lambda functions
Zip + Lambda Layers	250 MB total (function + all layers)	Medium	Fast	Shared dependencies across functions
Container Image	10 GB	Higher	Slower (first invoke)	ML libraries, large dependencies, custom runtimes

💡If a scenario is about a deployment package exceeding 250 MB, the answer is container images. If it mentions sharing dependencies across multiple functions, the answer is Lambda Layers. Zip is the default for most workloads.

Lambda Layers

Aspect	Detail
What They Are	Zip archives containing libraries, custom runtimes, or other dependencies
Max Layers Per Function	5
Size Limit	250 MB total (function code + all layers uncompressed)
Versioning	Each publish creates an immutable version
Sharing	Can be shared across functions, accounts, or made public
Path	Contents extracted to `/opt` in the execution environment

Dependency Management Strategies

Strategy	How It Works	Pros	Cons
Bundle In Zip	Install deps into package directory, zip together	Simple, self-contained	Larger package, duplicated across functions
Lambda Layers	Package deps as a layer, attach to functions	Shared across functions, smaller deploys	Layer version management, 5-layer limit
Container Image	Install deps in Dockerfile	Full control, large deps supported	Slower cold starts, ECR management
sam build	SAM resolves deps from requirements.txt automatically	Easiest, handles everything	Requires SAM CLI

Environment-Specific Configuration Approaches

Approach	When Config Is Resolved	Best For
SAM Parameters + samconfig.toml	Deploy time	Resource names, table names, stage-specific infra
Lambda Environment Variables	Deploy time (baked into function config)	Simple key-value config per environment
SSM Parameter Store	Runtime (fetched by function)	Config that changes without redeployment
AWS AppConfig	Runtime (with gradual rollout)	Feature flags, config that needs validation and rollback
Secrets Manager	Runtime (fetched by function)	Credentials that rotate

💡 If the question is change configuration without redeploying, the answer is Parameter Store or AppConfig. If it says gradual rollout or validate configuration before applying, the answer is AppConfig.

AWS AppConfig Overview

Feature	Detail
What It Does	Deploy configuration changes independently from code, with validation and rollout strategies
Deployment Strategies	AllAtOnce, Linear, Exponential
Validators	JSON Schema or Lambda function to validate config before deployment
Rollback	Automatic rollback if CloudWatch alarm triggers during deployment
Integration	Lambda extension for caching, or direct API calls
Cost	Free for configuration retrieval. Charges for feature flag evaluations

Project Structure Best Practices

A well-structured SAM project separates handlers, shared code, tests, and configuration:

my-app/
├── template.yaml              # SAM/CloudFormation template
├── samconfig.toml             # Per-environment deployment config
├── src/
│   ├── handlers/              # One file per Lambda function
│   │   ├── create_order.py
│   │   ├── get_order.py
│   │   └── process_payment.py
│   └── shared/                # Shared utilities across handlers
│       ├── models.py
│       └── utils.py
├── tests/
│   ├── unit/                  # Fast, no AWS calls
│   │   ├── test_create_order.py
│   │   └── test_get_order.py
│   └── integration/           # Against deployed resources
│       └── test_api.py
├── events/                    # Sample event payloads for testing
│   ├── create_order.json
│   └── get_order.json
└── requirements.txt           # Python dependencies

🏗️ Build A Multi-Environment Serverless App

Build a Multi-Environment Serverless App that demonstrates real-world packaging and deployment patterns:

A Lambda function packaged with dependencies via zip upload in the console
A container image built and pushed to ECR
AWS AppConfig set up with feature flags and a deployment strategy
A SAM project structured for multi-environment deployment
A samconfig.toml configured for dev, staging, and prod

Prerequisites

Part I

Package a Lambda Function with Dependencies (Zip Upload)

Create the Deployment Package Locally

Create a zip file on your local machine and upload it through the console.

⚠️ Common Windows mistake: Don't create the zip with Explorer (right-click → Send to → Compressed folder). That wraps everything inside a subfolder, so your handler ends up at package/lambda_function.py and Lambda throws No module named 'lambda_function'. Rather build the zip in CloudShell.

Step 01: Create a project directory and install dependencies

mkdir lambda-package && cd lambda-package

# Create requirements.txt
cat > requirements.txt << 'EOF'
requests==2.31.0
boto3>=1.28.0
EOF

# Install dependencies into a package directory
pip install -r requirements.txt -t package/

# Create your function code
cat > package/lambda_function.py << 'EOF'
import json
import requests
import boto3

def lambda_handler(event, context):
    """
    Lambda function packaged with external dependencies.
    The 'requests' library is bundled in the zip package.
    boto3 is included in the Lambda runtime, but pinning
    a version in requirements.txt ensures consistency.
    """
    # Use the bundled 'requests' library
    response = requests.get('https://httpbin.org/json')

    return {
        'statusCode': 200,
        'body': json.dumps({
            'message': 'Function with bundled dependencies',
            'externalApiStatus': response.status_code,
            'requestsVersion': requests.__version__
        })
    }
EOF

# Create the zip package
cd package && zip -r ../deployment.zip . && cd ..

Upload via the Console

Step 02: Open the Lambda console → Create function

Function name: PackagedFunction
Runtime: Python 3.12

Click Create function

Step 03: In the Code source section, click Update ▼ → Update from a .zip file

Step 04: Upload, select your deployment.zip file

Step 05: Click Update

Test the Function

Step 06: Go to the Test tab → create a test event with {}

Click Test

⚠️ Verify the response shows the requests library version and a successful API call

💡The zip upload limit is 50 MB compressed. If your package exceeds this, upload to S3 first and reference the S3 location. The uncompressed limit is 250 MB. sam build handles all of this automatically. It installs dependencies, creates the zip, and uploads to S3.

Part II

Build And Push A Container Image To ECR

Create an ECR Repository

Step 01: Open the ECR console

Step 02: Click Create repository

Repository name: lambda-container-demo
Image Tag immutability: Mutable
▼ Image scanning settings - deprecated: Scan on push → Enabled

Click Create

⚠️ Click into the repository Summary and note the URI (e.g., 123456789012.dkr.ecr.us-east-1.amazonaws.com/lambda-container-demo)

Build and Push the Image Locally

Step 03: Create a Dockerfile and function code

# Use the official AWS Lambda Python base image
FROM public.ecr.aws/lambda/python:3.12

# Install dependencies
COPY requirements.txt .
RUN pip install -r requirements.txt

# Copy function code
COPY app.py ${LAMBDA_TASK_ROOT}/

# Set the handler
CMD ["app.lambda_handler"]

Step 04: Create app.py

# app.py
import json
import requests

def lambda_handler(event, context):
    """
    Lambda function running as a container image.
    Container images support up to 10 GB — useful for
    ML libraries, large datasets, or custom runtimes.
    """
    return {
        'statusCode': 200,
        'body': json.dumps({
            'message': 'Hello from a container-based Lambda!',
            'runtime': 'Container image (Python 3.12)',
            'maxPackageSize': '10 GB (vs 250 MB for zip)'
        })
    }

Step 05: Build and push

# Authenticate Docker with ECR
aws ecr get-login-password --region us-east-1 | \
  docker login --username AWS --password-stdin 123456789012.dkr.ecr.us-east-1.amazonaws.com

# Build the image
docker build -t lambda-container-demo .

# Tag for ECR
docker tag lambda-container-demo:latest \
  123456789012.dkr.ecr.us-east-1.amazonaws.com/lambda-container-demo:latest

# Push to ECR
docker push 123456789012.dkr.ecr.us-east-1.amazonaws.com/lambda-container-demo:latest

Alternatively

# Authenticate Docker with ECR
aws ecr get-login-password --region us-east-1 | \
  docker login --username AWS --password-stdin 123456789012.dkr.ecr.us-east-1.amazonaws.com

# Build the image — disable attestations so Lambda accepts the manifest
docker buildx build --provenance=false --sbom=false \
  -t 123456789012.dkr.ecr.us-east-1.amazonaws.com/lambda-container-demo:latest \
  --push .

⚠️ Manifest Format Gotcha: If you build with a plain docker build on a recent Docker Desktop, BuildKit adds provenance and SBOM attestations that produce an OCI image index. Lambda rejects this with: "The image manifest, config or layer media type for the source image ... is not supported." Lambda only accepts Docker Image Manifest V2 Schema 2. The --provenance=false --sbom=false flags above force the compatible format. Alternatively, set DOCKER_BUILDKIT=0 before building to use the legacy builder, which always produces the compatible manifest.

If you prefer the separate tag-and-push steps (with BuildKit disabled):

# PowerShell: $env:DOCKER_BUILDKIT=0   |   CMD: set DOCKER_BUILDKIT=0
docker build -t lambda-container-demo .
docker tag lambda-container-demo:latest \
  123456789012.dkr.ecr.us-east-1.amazonaws.com/lambda-container-demo:latest
docker push 123456789012.dkr.ecr.us-east-1.amazonaws.com/lambda-container-demo:latest

Verify in the Console

Step 06: Go back to the ECR console → click into lambda-container-demo

You should see your image with the latest tag

Create a Lambda Function from the Container Image

Step 07: Open the Lambda console → Create function

Step 08: Select Container image

Function name: ContainerLambdaDemo
Container image URI: Click Browse images → select lambda-container-demo → select the latest tag

Click Create function

Step 09: Test with an empty event {}

💡 Container-based Lambda functions use ECR for image storage. The image must implement the Lambda Runtime API. AWS provides base images for all supported runtimes (public.ecr.aws/lambda/python:3.12). You can also use arbitrary base images with the Runtime Interface Client.

Part III

Set Up AWS AppConfig with Feature Flags

Create an AppConfig Application

Step 01: Open the AppConfig console

Step 02: Click Get started

Step 03: Select configuration type

Configuration options: Feature flag
Configuration profile name: feauture-flags

Click Next

Step 04: Specify configuration data

Flag name: New Checkout Flow
Flag key: new-checkout-flow
Flag description - Optional: Enable the redesigned checkout experience
Variants: Basic flag
Enabled value: Toggle **ON**

Click Next

Step 05: Review and save
Save to application

Application name: orders-service
Description: Feature flags and configuration for the orders service

Click Save and continue to deploy

Step 06: Start deployment

Environment: Click Create environment

Step 07: Create environment

Name: production
Description: Production environment

Click Create environment

Add A Feature Flag

Step 08: Click Add flag

Flag name: Dark Mode
Flag key: dark-mode
Description: Enable dark mode UI
Variants: Basic flag
Enabled value: Toggle **OFF**

Click Save new version

Create a Deployment Strategy

Step 09: Deployment strategies → Create deployment strategy

Name: GradualRollout
Description: Deploy over 10 minutes with bake time
Deployment type: Linear ▼
Step percentage: 20
Deployment time: 10 minutes
Bake time: 5 minutes

Click Create deployment strategy

Deploy the Configuration

Step 10: Go to your orders-service application → production environment

Click Start deployment

Step 11: Start deployment

Configuration profile: feature-flags
Hosted configuration version: (latest)
Deployment strategy: GradualRollout

Click Start deployment

⚠️ Watch the deployment progress. It rolls out 20% at a time over 10 minutes

💡AppConfig deployment strategies control how fast configuration changes roll out. If a CloudWatch alarm fires during deployment, AppConfig automatically rolls back. This is safer than changing a Parameter Store value directly, which takes effect immediately for all callers.

Part IV

Structure a SAM Project for Multi-Environment Deployment

Step 01: Initialize the Project

sam init --runtime python3.12 --name multi-env-app --app-template hello-world
cd multi-env-app

Step 02: Create the Handler Files

⚠️ sam init creates a hello_world/ folder, but our template uses a src/handlers/ structure with two functions. Create these files (delete the generated hello_world/ folder afterward. It's no longer referenced):

Step 03: Create `src/handlers/requirements.txt`:

boto3

Step 04: Create `src/handlers/create_order.py`:

import json
import os
import boto3
import uuid

dynamodb = boto3.resource('dynamodb')
table = dynamodb.Table(os.environ['TABLE_NAME'])

def lambda_handler(event, context):
    body = json.loads(event.get('body', '{}'))
    order_id = str(uuid.uuid4())[:8].upper()

    table.put_item(Item={
        'PK': f'ORDER#{order_id}',
        'SK': 'METADATA',
        'customerId': body.get('customerId', 'unknown'),
        'status': 'created'
    })

    return {
        'statusCode': 201,
        'body': json.dumps({'orderId': f'ORD-{order_id}', 'status': 'created'})
    }

Step 05: Create `src/handlers/get_order.py`:

import json
import os
import boto3

dynamodb = boto3.resource('dynamodb')
table = dynamodb.Table(os.environ['TABLE_NAME'])

def lambda_handler(event, context):
    order_id = event['pathParameters']['orderId']

    response = table.get_item(Key={'PK': f'ORDER#{order_id}', 'SK': 'METADATA'})
    item = response.get('Item')

    if not item:
        return {'statusCode': 404, 'body': json.dumps({'error': 'Order not found'})}

    return {'statusCode': 200, 'body': json.dumps(item, default=str)}

⚠️ The template's CodeUri: src/handlers/ and Handler: create_order.lambda_handler must point to real files. If src/handlers/ doesn't exist or is missing requirements.txt, sam build fails with "source ... does not exist" or "requirements.txt file not found." The CodeUri is the folder SAM packages; the Handler is filename.function_name relative to that folder.

Step 06: Configure the SAM Template with Parameters

Replace the contents of template.yaml

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
Description: Multi-environment serverless application

Parameters:
  Stage:
    Type: String
    Default: dev
    AllowedValues: [dev, staging, prod]
  TableName:
    Type: String
    Default: orders

Globals:
  Function:
    Runtime: python3.12
    Timeout: 30
    Environment:
      Variables:
        STAGE: !Ref Stage
        TABLE_NAME: !Sub "${Stage}-${TableName}"

Resources:
  OrdersTable:
    Type: AWS::DynamoDB::Table
    Properties:
      TableName: !Sub "${Stage}-${TableName}"
      BillingMode: PAY_PER_REQUEST
      KeySchema:
        - AttributeName: PK
          KeyType: HASH
        - AttributeName: SK
          KeyType: RANGE
      AttributeDefinitions:
        - AttributeName: PK
          AttributeType: S
        - AttributeName: SK
          AttributeType: S

  CreateOrderFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: src/handlers/
      Handler: create_order.lambda_handler
      Policies:
        - DynamoDBCrudPolicy:
            TableName: !Ref OrdersTable
      Events:
        CreateOrder:
          Type: Api
          Properties:
            Path: /orders
            Method: post

  GetOrderFunction:
    Type: AWS::Serverless::Function
    Properties:
      CodeUri: src/handlers/
      Handler: get_order.lambda_handler
      Policies:
        - DynamoDBReadPolicy:
            TableName: !Ref OrdersTable
      Events:
        GetOrder:
          Type: Api
          Properties:
            Path: /orders/{orderId}
            Method: get

Outputs:
  ApiUrl:
    Description: API Gateway endpoint URL
    Value: !Sub "https://${ServerlessRestApi}.execute-api.${AWS::Region}.amazonaws.com/${Stage}/"

Step 07: Configure samconfig.toml for Multiple Environments

# samconfig.toml — one section per environment
# The version key is REQUIRED — SAM CLI won't parse the file without it
version = 0.1

[default.deploy.parameters]
stack_name = "multi-env-app-dev"
resolve_s3 = true
capabilities = "CAPABILITY_IAM"
parameter_overrides = "Stage=dev"
confirm_changeset = false

[staging.deploy.parameters]
stack_name = "multi-env-app-staging"
resolve_s3 = true
capabilities = "CAPABILITY_IAM"
parameter_overrides = "Stage=staging"
confirm_changeset = false

[prod.deploy.parameters]
stack_name = "multi-env-app-prod"
resolve_s3 = true
capabilities = "CAPABILITY_IAM"
parameter_overrides = "Stage=prod"
confirm_changeset = true

Step 08: Deploy to Different Environments

# Build once — use --use-container to build inside a Lambda-compatible
# Docker image (needs Docker running, but no local Python required)
sam build --use-container

# Or, if you have Python 3.12 installed and on your PATH:
# sam build

# Deploy to dev (default config)
sam deploy

# Deploy to staging
sam deploy --config-env staging

# Deploy to prod (will prompt for changeset confirmation)
sam deploy --config-env prod

⚠️ sam build can't find Python? If you see "Binary validation failed for python ... did you have python for runtime: python3.12 on your PATH?", it means Python 3.12 isn't installed locally (the WindowsApps\python.EXE entries are Microsoft Store stubs, not a real install). Use sam build --use-container to build inside a Docker container that already has the correct runtime, or install Python 3.12 from python.org and check "Add python.exe to PATH" during setup. CloudShell also has Python 3.12 and SAM pre-installed.

Each deployment creates isolated resources: dev-orders table, staging-orders table, prod-orders table.

💡 samconfig.toml uses config environments (sections like [staging.deploy.parameters]) to manage per-environment settings. The --config-env flag selects which section to use. Setting confirm_changeset = true for production forces you to review changes before applying them.

🏗️ What You Built | 📘 Exam Concepts Recap

What You Built	Exam Concept
Bundled `requests` into a zip and uploaded via console	Zip packaging with dependencies, 50/250 MB limits
Built a Dockerfile and pushed to ECR	Container image packaging (up to 10 GB)
Created a Lambda from a container image	When to use containers vs zip
Created an AppConfig application and environment	Runtime configuration management
Added feature flags with enabled/disabled toggles	Feature flag pattern for decoupling release from deploy
Created a Linear deployment strategy with bake time	Gradual config rollout with automatic rollback
Used SAM parameters and `!Sub` for resource names	Environment-specific infrastructure
Configured `samconfig.toml` for dev/staging/prod	Multi-environment deployment with `--config-env`
Set `confirm_changeset = true` for prod	Forcing review before production changes

⚠️ Clean Up Protocol

Lambda → Delete PackagedFunction and ContainerLambdaDemo
ECR → Delete the lambda-container-demo repository (and all images)
AppConfig → Delete the deployment, then the configuration profile, environment, and application (in that order)
CloudFormation → Delete any SAM-deployed stacks (multi-env-app-dev, etc.)
IAM → Delete Lambda execution roles
CloudWatch → Delete log groups
S3 → Delete any SAM deployment buckets (prefixed with aws-sam-cli-managed-default)

Key Takeaways

Zip packages: 50 MB compressed / 250 MB uncompressed. Container images: up to 10 GB. Use containers for large dependencies.
Lambda Layers share dependencies across functions. Up to 5 layers, 250 MB total uncompressed.
sam build resolves dependencies automatically. sam deploy packages and deploys. sam deploy --guided for first-time setup.
AppConfig deploys configuration independently from code, with validation, gradual rollout, and automatic rollback.
samconfig.toml manages multi-environment deployment configs. Use --config-env to select the environment.
ECR stores container images. Use public.ecr.aws/lambda/ base images for Lambda containers.
Use CloudFormation parameters and !Sub for environment-specific resource names.
SAM policy templates (DynamoDBCrudPolicy, S3ReadPolicy) enforce least privilege without writing full IAM policies.

Additional Resources

🏗️

Manage Sensitive Data In Application Code | 🏗️ Build A Secure Configuration Service

Ntombizakhona Mabaso — Fri, 26 Jun 2026 21:20:02 +0000

Exam Guide: Developer - Associate
🏗️ Domain 2: Security
📘 Task 3: Manage Sensitive Data In Application Code

Managing Sensitive Data In Application Code is about keeping secrets out of your code, classifying data properly, and building applications that handle sensitive data safely.

You need to know when to use Secrets Manager vs Parameter Store, how to mask PII in API responses and logs, and how to isolate data in multi-tenant applications.

The ability to choose the right secret management service, implement data sanitization, and enforce tenant-level data boundaries, is vital.

📘Concepts

Data Classification

Understand data sensitivity levels and how each should be handled:

Classification	Examples	Handling Requirements
PII (Personally Identifiable Information)	Name, email, SSN, phone number, address	Encrypt at rest and in transit, mask in logs and API responses, restrict access
PHI (Protected Health Information)	Medical records, insurance IDs, lab results	HIPAA compliance, encryption required, audit trail mandatory
Financial	Credit card numbers, bank accounts, transaction data	PCI DSS compliance, tokenization, never store full card numbers
Public	Marketing content, public API docs	No special handling needed

💡If a scenario mentions compliance or audit trail, think encryption with KMS (for CloudTrail logging) and Secrets Manager (for automatic rotation). If it mentions PII in logs, think data masking and sanitization.

Secrets Manager vs SSM Parameter Store

Both store configuration and secrets.

Feature	Secrets Manager	SSM Parameter Store
Automatic rotation	Yes (built-in for RDS, Redshift, DocumentDB)	No (you build it yourself with Lambda)
Cost	$0.40/secret/month + $0.05 per 10,000 API calls	Free (Standard tier), $0.05/advanced parameter/month
Cross-account access	Yes (via resource policy)	Yes (advanced parameters only)
Max size	64 KB	4 KB (Standard) / 8 KB (Advanced)
Versioning	Automatic (AWSCURRENT, AWSPREVIOUS labels)	Yes (version history)
Encryption	Always encrypted (KMS required)	Optional (SecureString type uses KMS)
CloudFormation resolve	`{{resolve:secretsmanager:...}}`	`{{resolve:ssm:...}}`
Best for	Database credentials, API keys that need rotation	App config, feature flags, non-rotating secrets

💡If it says automatic rotation or database credentials, the answer is Secrets Manager.
If it says application configuration, feature flags, or free, the answer is Parameter Store.
If it says encrypted configuration, Parameter Store with SecureString works and costs nothing.

Data Masking and Sanitization Patterns

Pattern	What It Does	When to Use
Field Masking	Replace characters with asterisks (`j***e@example.com`)	API responses containing PII
Log Sanitization	Strip or redact sensitive fields before logging	Any log output that might contain secrets
Tokenization	Replace sensitive data with a non-reversible token	Credit card numbers, SSNs in databases
Field-Level Encryption	Encrypt individual fields within a record	When only some fields in a record are sensitive

Multi-Tenant Data Isolation Patterns

Pattern	How It Works	Isolation Level
Partition Key Prefix	Prefix DynamoDB keys with tenant ID (`TENANT#123#ORDER#456`)	Logical (application-enforced)
IAM Condition Keys	Use `dynamodb:LeadingKeys` condition to restrict access	Policy-enforced
Cognito Claims	Extract tenant ID from JWT and scope all queries	Token-Enforced
Separate Tables/Accounts	Each tenant gets their own table or AWS account	Physical (strongest isolation)

💡 Partition key isolation with IAM condition keys is the most common pattern. Separate accounts is the strongest but most expensive.
dynamodb:LeadingKeys restricts which partition keys a principal can access.

When to Use Each Secret Management Approach

Scenario	Service	Why
RDS database password that rotates every 30 days	Secrets Manager	Built-in rotation for RDS
API endpoint URL that differs per environment	Parameter Store (String)	Simple config, no encryption needed, free
Third-party API key that doesn't rotate	Parameter Store (SecureString)	Encrypted, free, no rotation needed
OAuth client secret shared across accounts	Secrets Manager	Cross-account resource policies
Feature flag (enable/disable a feature)	Parameter Store (String) or AppConfig	Simple config value
Lambda environment variable with a password	KMS encryption + runtime decryption	Encrypted at rest, decrypted at cold start

🏗️ Build A Secure Configuration Service

Build a Secure Configuration Service that demonstrates secret and configuration management:

Database credentials stored in Secrets Manager with console-based setup
Application configuration stored in SSM Parameter Store
A Lambda function that retrieves secrets and config at runtime
Data masking applied to API responses containing PII
A multi-tenant data isolation pattern using DynamoDB with partition key prefixes and IAM conditions

Prerequisites

An AWS account

Part I

Store Database Credentials in Secrets Manager

Step 01: Open the Secrets Manager console

Step 02: Click Store a new secret

Step 03: Choose secret type

Secret type: Other type of secret
Key: username, Value: admin
Key: password, Value: SuperSecretPass123!
Key: host, Value: mydb.cluster-abc123.us-east-1.rds.amazonaws.com
Key: port, Value: 3306
Key: dbname, Value: orders

Step 04: Encryption key: aws/secretsmanager

Step 05: Click Next

Step 06: Configure secret

Secret name: prod/database/credentials
Description: Production database credentials for the orders service

Step 07: Click Next

Step 08: Configure rotation - optional

Leave disabled for this tutorial (in production, you'd enable automatic rotation for RDS)

Step 09: Click Next

Step 10: Review

Step 11: Click Store

You now have a secret stored in Secrets Manager.
Click into it and note the Secret ARN.

Explore the Secret

Step 12: Click on prod/database/credentials

In the Secret value section, click Retrieve secret value
You'll see your key/value pairs displayed.

Step 13: Click the Versions tab

Notice the AWSCURRENT staging label

💡Secrets Manager automatically versions secrets. When rotation happens, the new value gets AWSCURRENT and the old value gets AWSPREVIOUS. Your application always retrieves AWSCURRENT by default, so rotation is seamless.

Part II

Store Application Config in SSM Parameter Store

Create String Parameters

Step 01: Open the Systems Manager console

Step 02: In the left sidebar ▼ Application Tools, click Parameter Store

Step 03: Click Create parameter

Name: /app/config/api-url
Description: External API endpoint URL
Tier: Standard
Type: String
Data type: text
Value: https://api.example.com/v2

Step 04: Click Create parameter

Create a SecureString Parameter

Step 05: Click Create parameter again

Name: /app/secrets/api-key
Description: Third-party API key (encrypted)
Tier: Standard
Type: SecureString
KMS key source: My current account
KMS Key ID: alias/aws/ssm
Value: sk-abc123-this-is-a-secret-key

Step 06: Click Create parameter

Create a Hierarchical Config Set

Step 07: Create three more parameters following the same steps:

Name: /app/config/max-items | Type: String | Value: 50
Name: /app/config/feature-flags/new-checkout | Type: String | Value: true
Name: /app/config/feature-flags/dark-mode | Type: String | Value: false

You now have a parameter hierarchy. The /app/config/ prefix groups related configuration together.

💡 Parameter Store supports hierarchies with path-based names. You can retrieve all parameters under a path with GetParametersByPath. This is useful for loading all config for an environment at once. SecureString parameters are encrypted with KMS. Use WithDecryption=True when retrieving them.

Part III

Create a Lambda Function That Retrieves Both

Create the Lambda Function

Step 01: Open the Lambda console → Create function

Function name: SecureConfigService
Runtime: Python 3.12

Step 02: Click Create function

Step 03: Paste this code

import json
import boto3
from functools import lru_cache

secrets_client = boto3.client('secretsmanager')
ssm_client = boto3.client('ssm')

@lru_cache(maxsize=None)
def get_secret(secret_name):
    """
    Retrieve a secret from Secrets Manager.
    Cached at cold start to avoid repeated API calls.
    """
    response = secrets_client.get_secret_value(SecretId=secret_name)
    return json.loads(response['SecretString'])

@lru_cache(maxsize=None)
def get_parameter(name, decrypt=True):
    """
    Retrieve a parameter from SSM Parameter Store.
    Use WithDecryption=True for SecureString parameters.
    """
    response = ssm_client.get_parameter(Name=name, WithDecryption=decrypt)
    return response['Parameter']['Value']

def get_all_config(path):
    """Retrieve all parameters under a path hierarchy."""
    response = ssm_client.get_parameters_by_path(
        Path=path,
        Recursive=True,
        WithDecryption=True
    )
    return {p['Name']: p['Value'] for p in response['Parameters']}

def lambda_handler(event, context):
    """
    Demonstrates retrieving secrets and config from both services.

    Key patterns:
    - Cache secrets at cold start (lru_cache) to reduce API calls
    - Use Secrets Manager for credentials that rotate
    - Use Parameter Store for app config and non-rotating secrets
    - Use GetParametersByPath to load config hierarchies
    """
    action = event.get('action', 'get_all')

    if action == 'get_db_credentials':
        # From Secrets Manager
        creds = get_secret('prod/database/credentials')
        return {
            'statusCode': 200,
            'body': json.dumps({
                'message': 'Database credentials retrieved from Secrets Manager',
                'host': creds['host'],
                'port': creds['port'],
                'dbname': creds['dbname'],
                'username': creds['username'],
                'password': '***REDACTED***'  # Never return passwords in responses!
            })
        }

    elif action == 'get_app_config':
        # From Parameter Store
        api_url = get_parameter('/app/config/api-url')
        api_key = get_parameter('/app/secrets/api-key')
        max_items = get_parameter('/app/config/max-items')

        return {
            'statusCode': 200,
            'body': json.dumps({
                'message': 'App config retrieved from Parameter Store',
                'apiUrl': api_url,
                'apiKey': api_key[:8] + '***REDACTED***',  # Mask the key
                'maxItems': int(max_items)
            })
        }

    elif action == 'get_feature_flags':
        # Load all feature flags under a path
        flags = get_all_config('/app/config/feature-flags')
        return {
            'statusCode': 200,
            'body': json.dumps({
                'message': 'Feature flags loaded from Parameter Store',
                'flags': {k.split('/')[-1]: v == 'true' for k, v in flags.items()}
            })
        }

    else:
        return {
            'statusCode': 200,
            'body': json.dumps({
                'message': 'Secure Config Service',
                'availableActions': ['get_db_credentials', 'get_app_config', 'get_feature_flags']
            })
        }

Step 04: Paste Click Deploy

Add Permissions to the Lambda Role

Step 05: Go to Configuration → Permissions → click the role name

Step 06: Click Add permissions → Attach policies

Step 07: Search for and attach:

SecretsManagerReadWrite
AmazonSSMReadOnlyAccess

Step 08: Click Add permissions

Test the Function

Step 09: Go to the Test tab

Step 10: Create test events

Test event 1: Get database credentials:

{"action": "get_db_credentials"}

Test event 2: Get app config:

{"action": "get_app_config"}

Test event 3: Get feature flags:

{"action": "get_feature_flags"}

Step 11: Run each test and verify the responses

💡Notice the caching pattern with @lru_cache. Secrets Manager and Parameter Store charge per API call.

Caching at cold start means you only pay for one call per Lambda instance, not one per invocation.

For secrets that rotate, use a TTL-based cache instead of lru_cache so the Lambda picks up new values.

Part IV

Implement Data Masking in API Responses

Create the Data Masking Lambda

Step 01: Lambda → Create function

Function name: DataMaskingDemo
Runtime: Python 3.12

Step 02: Click Create function

Step 03: Paste this code

import json
import re
import logging

logger = logging.getLogger()
logger.setLevel(logging.INFO)

# --- Data Masking Functions ---

def mask_email(email):
    """jane.doe@example.com → j***e@example.com"""
    local, domain = email.split('@')
    if len(local) <= 2:
        masked = local[0] + '***'
    else:
        masked = local[0] + '***' + local[-1]
    return f"{masked}@{domain}"

def mask_phone(phone):
    """+1-555-123-4567 → ***-***-**67"""
    digits = re.sub(r'\D', '', phone)
    return f"***-***-**{digits[-2:]}"

def mask_ssn(ssn):
    """123-45-6789 → ***-**-6789"""
    return f"***-**-{ssn[-4:]}"

def mask_credit_card(card):
    """4111-1111-1111-1234 → ****-****-****-1234"""
    digits = re.sub(r'\D', '', card)
    return f"****-****-****-{digits[-4:]}"

def sanitize_user(user):
    """Mask all PII fields before returning to the client."""
    return {
        'id': user['id'],
        'name': user['name'],
        'email': mask_email(user['email']),
        'phone': mask_phone(user['phone']),
        'ssn': mask_ssn(user['ssn']),
        'memberSince': user['memberSince']
    }

# --- Log Sanitization ---

SENSITIVE_PATTERNS = {
    'password': r'"password"\s*:\s*"[^"]*"',
    'ssn': r'\d{3}-\d{2}-\d{4}',
    'credit_card': r'\d{4}[\s-]?\d{4}[\s-]?\d{4}[\s-]?\d{4}'
}

def sanitize_for_logging(data):
    """Remove sensitive data before logging."""
    text = json.dumps(data) if isinstance(data, dict) else str(data)
    for field, pattern in SENSITIVE_PATTERNS.items():
        text = re.sub(pattern, f'"***REDACTED-{field}***"', text)
    return text

def lambda_handler(event, context):
    """
    Demonstrates data masking in API responses and log sanitization.

    Key patterns:
    - Mask PII fields (email, phone, SSN) before returning to clients
    - Sanitize log output to prevent sensitive data from reaching CloudWatch
    - Never log passwords, full credit card numbers, or SSNs
    """
    # Simulated user data (in production, this comes from DynamoDB)
    users = [
        {
            'id': 'USR-001',
            'name': 'Jane Doe',
            'email': 'jane.doe@example.com',
            'phone': '+1-555-123-4567',
            'ssn': '123-45-6789',
            'memberSince': '2023-01-15'
        },
        {
            'id': 'USR-002',
            'name': 'John Smith',
            'email': 'john.smith@example.com',
            'phone': '+1-555-987-6543',
            'ssn': '987-65-4321',
            'memberSince': '2023-06-20'
        }
    ]

    # Sanitize before logging — never log raw PII
    logger.info(f"Processing request: {sanitize_for_logging(event)}")

    # Mask PII before returning in the response
    masked_users = [sanitize_user(u) for u in users]

    return {
        'statusCode': 200,
        'headers': {'Content-Type': 'application/json'},
        'body': json.dumps({
            'message': 'User data with PII masked',
            'users': masked_users
        })
    }

Step 04: Click Deploy

Test Data Masking

Step 05: Go to the Test tab → create a test event with {}

Step 06: Run the test

⚠️ Verify the response shows masked values:

Email: j***e@example.com

Phone: ***-***-**67

SSN: ***-**-6789

Step 07: Check CloudWatch Logs to confirm the log output is also sanitized

💡 Data masking happens at the application layer, not the database layer. Always mask PII before returning it in API responses.
Always sanitize data before logging.
CloudWatch Logs can be accessed by anyone with the right IAM permissions, so treat logs as potentially public.

Part V

Build a Multi-Tenant Data Isolation Pattern

Create the DynamoDB Table

Step 01: Open the DynamoDB console → Create table

Table name: MultiTenantApp
Partition key: PK (String)
Sort key: SK (String)
Table settings: Customize settings
Read/write capacity settings: On-demand

Step 02: Click Create table

Add Sample Data

Step 03: Click into the MultiTenantApp table

Step 04: Click Explore table items → Create item

Step 05: Switch to JSON view and add these items one at a time:

Tenant A: Order 1:

{
  "PK": {"S": "TENANT#tenant-a#USER#user-001"},
  "SK": {"S": "ORDER#2024-001"},
  "orderTotal": {"N": "149.99"},
  "status": {"S": "shipped"},
  "tenantId": {"S": "tenant-a"}
}

Tenant A: Order 2:

{
  "PK": {"S": "TENANT#tenant-a#USER#user-001"},
  "SK": {"S": "ORDER#2024-002"},
  "orderTotal": {"N": "89.50"},
  "status": {"S": "processing"},
  "tenantId": {"S": "tenant-a"}
}

Tenant B: Order 1:

{
  "PK": {"S": "TENANT#tenant-b#USER#user-002"},
  "SK": {"S": "ORDER#2024-003"},
  "orderTotal": {"N": "299.00"},
  "status": {"S": "delivered"},
  "tenantId": {"S": "tenant-b"}
}

Create the Multi-Tenant Lambda

Step 06: Lambda → Create function

Function name: MultiTenantQuery
Runtime: Python 3.12

Step 07: Paste this code:

import json
import boto3
from boto3.dynamodb.conditions import Key

dynamodb = boto3.resource('dynamodb')
table = dynamodb.Table('MultiTenantApp')

def lambda_handler(event, context):
    """
    Multi-tenant data isolation using partition key prefixes.

    The tenant ID is extracted from the request context (simulating
    a Cognito JWT claim) and used to scope all database queries.
    A user in Tenant A can never see Tenant B's data.

    In production, the tenant ID comes from:
    - Cognito custom claims (custom:tenantId)
    - Lambda authorizer context
    - API Gateway request context
    """
    # In production: extract from JWT claims
    # tenant_id = event['requestContext']['authorizer']['claims']['custom:tenantId']
    tenant_id = event.get('tenantId', 'tenant-a')
    user_id = event.get('userId', 'user-001')

    # Query is automatically scoped to this tenant
    pk = f"TENANT#{tenant_id}#USER#{user_id}"

    response = table.query(
        KeyConditionExpression=Key('PK').eq(pk) & Key('SK').begins_with('ORDER#')
    )

    orders = []
    for item in response['Items']:
        orders.append({
            'orderId': item['SK'].replace('ORDER#', ''),
            'total': str(item['orderTotal']),
            'status': item['status']
        })

    return {
        'statusCode': 200,
        'body': json.dumps({
            'tenantId': tenant_id,
            'userId': user_id,
            'orderCount': len(orders),
            'orders': orders
        })
    }

Step 08: Click Deploy

Step 09: Add AmazonDynamoDBReadOnlyAccess to the Lambda execution role

Test Tenant Isolation

Step 10: Create test events

Tenant A query:

{"tenantId": "tenant-a", "userId": "user-001"}

Tenant B query:

{"tenantId": "tenant-b", "userId": "user-002"}

⚠️ Run each test and verify that Tenant A only sees their orders and Tenant B only sees theirs

💡 Partition key isolation is the most common multi-tenant pattern on the exam. For stronger isolation, combine it with IAM condition keys using dynamodb:LeadingKeys. This enforces isolation at the IAM policy level so even a bug in your application code can't leak data across tenants.

🏗️ What You Built | 📘 Exam Concepts Recap

What You Built	Exam Concept
Stored database credentials in Secrets Manager	Secret management for rotating credentials
Explored the `AWSCURRENT` staging label	Secrets Manager versioning and seamless rotation
Created String and SecureString parameters	Parameter Store types. Plaintext vs KMS-encrypted
Built a parameter hierarchy under `/app/config/`	Path-based config organization, `GetParametersByPath`
Cached secrets with `@lru_cache` at cold start	Reducing API calls and cost for secret retrieval
Retrieved SecureString with `WithDecryption=True`	KMS decryption of encrypted parameters
Masked email, phone, SSN in API responses	Application-level PII data masking
Sanitized log output before writing to CloudWatch	Preventing sensitive data leakage in logs
Used partition key prefixes (`TENANT#...`)	Multi-tenant data isolation pattern
Scoped queries to a tenant from request context	Token/claim-based tenant boundary enforcement

⚠️ Clean Up Protocol

Secrets Manager → Select prod/database/credentials → Actions → Delete secret (set minimum 7-day waiting period)
Systems Manager → Parameter Store → Select all parameters → Delete
Lambda → Delete SecureConfigService, DataMaskingDemo, and MultiTenantQuery
DynamoDB → Delete the MultiTenantApp table
IAM → Delete Lambda execution roles
CloudWatch → Delete log groups for all three functions

Key Takeaways

Secrets Manager for credentials that need automatic rotation (RDS, Redshift, DocumentDB).
Parameter Store for app config and non-rotating secrets.
SecureString parameters are encrypted with KMS. Always use WithDecryption=True to read them.
Don't resolve secrets at deploy time ({{resolve:...}}) if they rotate fetch at runtime with caching instead.
Lambda environment variables are encrypted at rest by default. Use a CMK + in-code decryption for additional security.
Mask PII in API responses and sanitize logs.
Never log passwords, SSNs, or credit card numbers.
Multi-tenant isolation: use partition key prefixes combined with IAM condition keys (dynamodb:LeadingKeys) for policy-enforced isolation.
Secrets Manager costs $0.40/secret/month. Parameter Store Standard is free. Choose based on rotation needs, not just cost.
Cache secrets at cold start to reduce API calls and cost. Use lru_cache for static secrets, TTL-based cache for rotating secrets.

Additional Resources

🏗️

Implement Encryption By Using AWS Services | 🏗️ Create A KMS Customer Managed Key

Ntombizakhona Mabaso — Wed, 03 Jun 2026 00:45:50 +0000

Exam Guide: Developer - Associate
🏗️ Domain 2: Security
📘 Task 2: Implement Encryption By Using AWS Services.

Encryption shows up everywhere, especially on this exam. S3, DynamoDB, SQS, Lambda environment variables, RDS, and more. You need to know the difference between client-side and server-side encryption, how KMS works, and when to use each approach.

📘Concepts

Encryption at Rest vs Encryption In Transit

Encryption At Rest

Data stored on disk: S3 Objects, DynamoDB tables, EBS volumes, RDS databases.

Encryption In Transit

Data moving between services or between client and server: HTTPS, TLS, VPN.

Where	At Rest	In Transit
S3	SSE-S3, SSE-KMS, SSE-C	HTTPS (enforced via bucket policy)
DynamoDB	Encrypted by default (AWS owned or KMS)	HTTPS (always)
RDS	KMS encryption	SSL/TLS connections
SQS	SSE-KMS	HTTPS
Lambda env vars	KMS (default + optional CMK)	HTTPS

KMS Key Types

Type	Managed By	Cost	Use Case
AWS owned keys	AWS	Free	Default encryption (DynamoDB, S3 SSE-S3)
AWS managed keys	AWS (in your account)	Free (per-use charges)	`aws/s3`, `aws/dynamodb` (you can't manage them)
Customer managed keys (CMK)	You	Monthly + per-use	Full control: rotation, policies, cross-account

Envelope Encryption

KMS can only directly encrypt up to 4 KB. For larger data, it uses envelope encryption:

1. KMS generates a data key (plaintext + encrypted copy)
2. You encrypt your data with the plaintext data key
3. You store the encrypted data key alongside the encrypted data
4. You discard the plaintext data key from memory
5. To decrypt: KMS decrypts the data key → you decrypt the data

The AWS Encryption SDK handles this automatically.

Server-Side Encryption Options for S3

Option	Key Management	Use Case
SSE-S3	AWS manages everything	Simplest, no KMS costs
SSE-KMS	You control the KMS key	Audit trail via CloudTrail, key policies
SSE-C	You provide the key with every request	Full key control, AWS doesn't store the key

Client-Side vs Server-Side Encryption

Aspect	Server-Side	Client-Side
Who encrypts	AWS (after receiving data)	You (before sending to AWS)
Data in transit	Encrypted by HTTPS	Encrypted by you + HTTPS
Key management	AWS or KMS	You (via KMS or your own keys)
Complexity	Simple	More complex
Use case	Most workloads	When you can't trust the storage layer

Key Rotation

AWS managed keys: Rotated every year automatically (can't change this)
Customer managed keys: Enable automatic rotation (every year)
Old key material is kept, existing encrypted data still works
The key ID doesn't change with automatic rotation
Manual rotation: Create a new key, update alias to point to it

Certificate Management

Service	What It Does	Cost
ACM	Free public SSL/TLS certificates for AWS services	Free
AWS Private CA	Issue private certificates for internal services	Paid

💡ACM certificates can't be exported. They're bound to AWS services (CloudFront, ALB, API Gateway). They auto-renew.

🏗️ Create A KMS Customer Managed Key

Now let's put these concepts into practice:

Create a KMS customer managed key
Encrypt and decrypt data using KMS
Upload objects to S3 with different encryption options (SSE-S3, SSE-KMS)
Enable automatic key rotation
Encrypt Lambda environment variables with a custom KMS key

Prerequisites

An AWS account
An S3 bucket (we'll create one)

Part I

Create a KMS Customer Managed Key

Step 01: Open the KMS console
Click Create key

Step 02: Configure key

Key type: Symmetric
Key usage: Encrypt and decrypt

Click Next

Step 03: Add labels

Alias: app-encryption-key
Description: Customer managed key for application data

Click Next

Step 04: Define key administrative permissions - optional

Key administrators: Select your IAM user

Click Next

Step 05: Define key usage permissions - optional

Key administrators: Select your IAM user

Click Next

Step 06: Edit key policy - optional
Click Next

Step 07: Review
Click Finish

✅Green banner: Success
Your AWS KMS key was created with alias app-encryption-key and key ID 17fx7cex-17ae-419x-x816-de16fx50x928.

💡 You now have a customer managed KMS key. Note the Key ID and ARN.

Enable Automatic Key Rotation

Step 08: Click on your key (app-encryption-key)

Step 09: Go to the Key material rotations tab
Click Edit

Step 10: Edit automatic key rotation

Key rotation: Enable
Rotation period in days: 365

Click Save

✅Green banner: Successfully enabled automatic key rotation

💡 With automatic rotation, KMS keeps old key material so existing ciphertext can still be decrypted. The key ID and ARN don't change. With manual rotation (creating a new key), you get a new key ID. Use aliases to abstract this.

Part II

Encrypt and Decrypt Data with KMS

Using the Console

💡 In the KMS console, you can't directly encrypt/decrypt from the UI

Step 01: Create a Lambda function to demonstrate
Lambda → Create function:

Name: KMSEncryptionDemo
Runtime: Python 3.12
Deploy: Code:

import json
import boto3
import base64

kms = boto3.client('kms')
KEY_ID = 'alias/app-encryption-key'

def lambda_handler(event, context):
    """
    Demonstrates KMS encrypt/decrypt operations.

    Key concepts:
    - KMS can directly encrypt up to 4 KB
    - For larger data, use GenerateDataKey (envelope encryption)
    - The AWS Encryption SDK handles envelope encryption automatically
    """
    action = event.get('action', 'encrypt')
    plaintext = event.get('data', 'Hello, this is sensitive data!')

    if action == 'encrypt':
        # Encrypt data (up to 4 KB)
        response = kms.encrypt(
            KeyId=KEY_ID,
            Plaintext=plaintext.encode('utf-8')
        )
        ciphertext_b64 = base64.b64encode(response['CiphertextBlob']).decode('utf-8')

        return {
            'statusCode': 200,
            'body': json.dumps({
                'message': 'Data encrypted successfully',
                'ciphertext': ciphertext_b64,
                'keyId': response['KeyId']
            })
        }

    elif action == 'decrypt':
        # Decrypt data
        ciphertext = base64.b64decode(event['ciphertext'])
        response = kms.decrypt(CiphertextBlob=ciphertext)

        return {
            'statusCode': 200,
            'body': json.dumps({
                'message': 'Data decrypted successfully',
                'plaintext': response['Plaintext'].decode('utf-8')
            })
        }

    elif action == 'generate_data_key':
        # Generate a data key for envelope encryption
        response = kms.generate_data_key(
            KeyId=KEY_ID,
            KeySpec='AES_256'
        )

        return {
            'statusCode': 200,
            'body': json.dumps({
                'message': 'Data key generated (envelope encryption)',
                'plaintextKey': '*** exists in memory only — use it to encrypt, then discard ***',
                'encryptedKey': base64.b64encode(response['CiphertextBlob']).decode('utf-8'),
                'note': 'Store the encrypted key alongside your encrypted data'
            })
        }

Step 02: Add KMS permissions to the Lambda role:
Configuration → Permissions → click role name
Add permissions → Attach policies → AWSKeyManagementServicePowerUser

Step 03: Test events
Encrypt:

{"action": "encrypt", "data": "My secret credit card number"}

Generate data key:

{"action": "generate_data_key"}

Part III

S3 Encryption Options

Create an S3 Bucket

Step 01: Open the S3 console → Create bucket

Step 02: General configuration

Bucket type: General purpose
Bucket namespace: Account Regional namespace (recommended)
Bucket name prefix: dva-encryption-demo
Encryption type: Server-side encryption with Amazon S3 managed keys (SSE-S3)

Step 03: Upload with SSE-S3 (Default)

Click Upload
Click Add files
Add a file (any text file)
Click Upload

💡SSE-S3 is applied automatically

Step 04: Upload with SSE-KMS

Click Upload
Click Add files
Expand ▶ Properties
Server side encryption: Specify an encryption key
Encryption settings: Override bucket settings for default encryption
Encryption type: Server-side encryption with AWS Key Management Service keys (SSE-KMS)
AWS KMS Key: Choose from your AWS KMS keys
Available AWS KMS Key: app-encryption-key
Click Upload

Step 05: Verify Encryption
Click on each uploaded file

Step 06: Go to the Properties tab → Server-side encryption settings
You'll see which encryption method was used

💡SSE-KMS gives you an audit trail in CloudTrail (every encrypt/decrypt call is logged). SSE-S3 does not. If a question mentions "audit" or "compliance," SSE-KMS is usually the answer.

Part IV

Encrypt Lambda Environment Variables

Using a Custom KMS Key

Step 01: Open your KMSEncryptionDemo function

Step 02: Configuration → Environment variables → Edit

Step 03: Edit environment variables
Click Add environment variable

Key: DB_PASSWORD
Value: super-secret-password

Step 04: Expand ▶ Encryption configuration

Encryption in transit: ✔ Check Enable helpers for encryption in transit
AWS KMS key to encrypt at rest: Use a customer managed key
Select app-encryption-key
Click the Encrypt button next to DB_PASSWORD

Click Save

The value is now encrypted.
In your code, you'd decrypt it:

import boto3
import base64
import os

kms = boto3.client('kms')

# Decrypt at cold start, cache the result
ENCRYPTED = os.environ['DB_PASSWORD']
DECRYPTED = kms.decrypt(
    CiphertextBlob=base64.b64decode(ENCRYPTED)
)['Plaintext'].decode('utf-8')

def lambda_handler(event, context):
    # Use DECRYPTED: already decrypted at cold start
    print(f"Password length: {len(DECRYPTED)}")  # Don't log the actual password!

🏗️ What You Built | 📘 Exam Concepts Recap

What You Built	Exam Concept
Created a Customer Managed KMS key	Key types: AWS owned vs AWS managed vs customer managed
Assigned key administrators and key users	KMS key policies: separate admin and usage permissions
Enabled automatic key rotation	Old key material preserved, key ID unchanged, seamless decryption
Encrypted data directly with `kms.encrypt()`	KMS direct encryption: limited to 4 KB
Called `generate_data_key` for AES-256	Envelope encryption: encrypt large data locally, protect the key with KMS
Discarded the plaintext data key after use	Security best practice: plaintext key lives only in memory
Uploaded S3 objects with SSE-S3	Default encryption: AWS manages everything, no audit trail
Uploaded S3 objects with SSE-KMS	Audit trail in CloudTrail, key policy control, cross-account possible
Encrypted Lambda environment variables with a CMK	Protecting secrets at rest in Lambda configuration
Decrypted env vars at cold start and cached the result	Performance pattern: avoid decrypting on every invocation

⚠️ Clean Up Protocol

S3 → Empty and delete the bucket
Lambda → Delete KMSEncryptionDemo
KMS → Schedule key deletion (7-day minimum waiting period)
IAM → Delete Lambda execution roles
CloudWatch → Delete log groups

Key Takeaways

KMS encrypts up to 4 KB directly: use envelope encryption (GenerateDataKey) for larger data
SSE-S3: simplest. SSE-KMS: audit trail + key control. SSE-C: you provide the key.
Client-side encryption: you encrypt before sending to AWS
Automatic key rotation keeps old key material and existing data ia still decryptable
Use aliases for seamless manual key rotation
Cross-account KMS requires both a key policy AND an IAM policy
ACM = free public certs (can't export). Private CA = private certs (costs money).
Envelope encryption: the AWS Encryption SDK handles this for you

Additional Resources

🏗️

Implement Authentication And/Or Authorization For Applications And AWS Services | 🏗️ Build A Secure Notes API

Ntombizakhona Mabaso — Tue, 02 Jun 2026 22:54:48 +0000

Exam Guide: Developer - Associate
🏗️ Domain 2: Security
📘 Task 1: Implement Authentication And/Or Authorisation For Applications And AWS
Services

Authentication (who are you?) and Authorisation (what can you do?) are central to the DVA-C02, and just Cloud Development in general.
You need to know Cognito, IAM Roles and Policies, STS, Lambda Authorizers, and how to secure microservice-to-microservice communication or communication within a Microservices Architectural Environment.

📘 Concepts

Authentication vs Authorisation

Term	What It Answers	AWS Services
Authentication	Who are you?	Cognito User Pools, IAM
Authorisation	What can you do?	IAM Policies, Cognito Identity Pools, Lambda Authorizers

Amazon Cognito

User Pools vs Identity Pools

Feature	User Pool	Identity Pool
Purpose	Authentication (sign up, sign in)	Authorisation (temporary AWS credentials)
Returns	JWT tokens (ID, access, refresh)	AWS credentials (access key, secret, session token)
Use When	You need a user directory	Users need to call AWS services directly
Federation	Google, Facebook, SAML, OIDC	Google, Facebook, SAML, User Pools

User Pools = authentication (get tokens).
Identity Pools = authorisation (get AWS credentials).
They're often used together but serve different purposes.

The Three Cognito Tokens

Token	Contains	Use For
ID Token	User identity claims (name, email, groups)	Your application to identify the user
Access Token	Scopes and permissions	API authorisation
Refresh Token	Long-lived credential	Getting new ID/access tokens without re-authentication

API Gateway Authorisation Options

Option	What It Does	When to Use
None	No auth	Public APIs
IAM	SigV4 signing	Internal/service-to-service calls
Cognito Authorizer	Validates JWTs from User Pool	Simple JWT validation
Lambda Authorizer (Token)	Custom logic on the token	Complex auth, external IDPs
Lambda Authorizer (Request)	Custom logic on full request	Auth based on headers, query strings, source IP

IAM Roles vs Users

Type	Credentials	Use Case
IAM User	Long-lived (access key + secret)	Humans, legacy CLI access
IAM Role	Temporary (via STS AssumeRole)	Applications, EC2, Lambda, cross-account

Roles are always preferred for applications. Never hardcode access keys.
Lambda uses an execution role, EC2 uses an instance profile, ECS uses a task role.
All of these give your code temporary credentials automatically.

STS (Security Token Service) Operations

`Operation`	Use Case
`AssumeRole`	Same or cross-account role assumption
`AssumeRoleWithWebIdentity`	Exchange OAuth/OIDC token for AWS credentials
`AssumeRoleWithSAML`	Exchange SAML assertion for AWS credentials
`GetSessionToken`	Temporary credentials (for MFA-protected API calls)

Credential Resolution Order (SDK Default Chain)

The AWS SDK looks for credentials in this order:

1. Environment variables (AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY)
2. Shared credentials file (~/.aws/credentials)
3. AWS config file (~/.aws/config)
4. Container credentials (ECS task role)
5. Instance profile (EC2 role)
6. SSO credentials (IAM Identity Center)

🏗️ Build A Secure Notes API

Now lets put these concepts into practice, by building a Secure Notes API with Cognito authentication:

A Cognito User Pool with Managed Login for sign-up/sign-in
An App Client configured with the authorization code grant flow
A Lambda function that stores and retrieves notes
An API Gateway REST API protected by a Cognito authorizer
Test users signing in and calling the API with JWT tokens

Prerequisites

An AWS account

Part I

Create the Cognito User Pool

The Cognito console now uses a streamlined, application-focused setup wizard with Managed Login (the successor to the classic Hosted UI). The wizard creates your user pool and app client in one flow.

Step 01: Open the Cognito console

Step 02: Click Create user pool

Step 03: Set up resources for your application
Application type: Traditional web application
Name your application: My Notes App
Options for sign-in identifiers: Email
Self-registration: ✔
Required attributes for sign-up: email name
Click Create user directory

✅Green banner: Your application "My Notes App" and user pool "User pool - gcvhqy" have been created successfully! Follow the instruction to continue the setup.

💡The new console creates the user pool, app client, and managed login domain all in one step. "Traditional web application" configures the authorization code grant flow with a client secret by default. This is the recommended pattern for server-side apps.

Step 04: Get Your Pool Details
After creation, click on the user pool and note:

User pool ID (looks like us-east-1_ABC123XYZ)
App client ID (from the ▼ Applications tab → App clients)
Cognito domain (from the ▼ Branding tab → Domain )

Part II

Create a Test User

Step 01: In the User Pool, click the ▼ Users management tab
Click Users

Step 02: Users
Click Create user
User Information

Invitation message: Send an email invitation
Email address: Use a real email you can access
Temporary password: Generate a password

Click Create user

✅Green banner: User username@example.com has been created successfully.

You'll receive an email with a temporary password.

⚠️ Don't log in yet. We'll do that through the Managed Login in a moment.

Part III

Test with Managed Login (Authorization Code Flow)

Step 01: View the hosted UI URL
From the ▼ Applications tab → App clients
Click View login page

https://YOUR_COGNITO_DOMAIN.auth.us-east-1.amazoncognito.com/login?
  client_id=YOUR_APP_CLIENT_ID&
  response_type=code&
  scope=email+openid+profile&
  redirect_uri=YOUR_CALLBACK_URL

Step 02: Sign in
Sign in with the user credentials from the email
Cognito will force you to set a permanent password

Step 03: Change password

✅: Success page

Successfully signed in
This is the default redirect page for Amazon Cognito user pools.
You're seeing this page because your Amazon Cognito app client doesn't have a return URL set.

Step 04: After login, you'll be redirected to https://d84l1y8p4kdic.cloudfront.net/?code=2f927666-cd13-4e94-9af9-58887a9d9cd3...

💡Copy the code value from the URL fragment.
You'll need it to call the API.

⚠️ This code is NOT a token. It's a one-time authorization code that expires in a few minutes. You need to exchange it for actual JWT tokens in the next step.

Step 05: Exchange the Code for Tokens
Use the Cognito token endpoint to exchange the authorization code for JWT tokens:

curl -X POST "https://YOUR_COGNITO_DOMAIN.auth.us-east-1.amazoncognito.com/oauth2/token" \
  -H "Content-Type: application/x-www-form-urlencoded" \
  -d "grant_type=authorization_code&code=YOUR_AUTH_CODE&client_id=YOUR_APP_CLIENT_ID&client_secret=YOUR_CLIENT_SECRET&redirect_uri=YOUR_CALLBACK_URL"

⚠️ If your app client has a client secret (default for "Traditional web application"), you also need to include a client_secret parameter, or pass the credentials as a Basic auth header: -H "Authorization: Basic BASE64(client_id:client_secret)"

Expected response:

{
  "id_token": "eyJraWQiOiJ...",
  "access_token": "eyJraWQiOiJ...",
  "refresh_token": "eyJjdHkiOiJ...",
  "expires_in": 3600,
  "token_type": "Bearer"
}

💡 Copy the id_token value. You'll need it to call the API.

response_type=token uses the implicit grant flow (returns tokens directly). response_type=code uses the authorization code flow (returns a code you exchange for tokens via the token endpoint).
Auth code is more secure for production apps.

Part IV

Create the DynamoDB Table

Step 01: Open the DynamoDB console → Create table

Step 02: Create table

Table name: Notes
Partition key: userId (String)
Sort key: noteId (String)
Table settings: Default settings

Click Create table

✅Green banner: The Notes table was created successfully.

Part V

Create the Notes Lambda Function

Step 01: Open Lambda → Create function

Step 02: Create function

Function name: NotesAPI
Runtime: Python 3.12

Click Create function

✅Green banner: Successfully created the function "NotesAPI"

Step 03: Configuration → General configuration → click Edit

Memory: 256 MB
Timeout: 0 min 10 sec

Click Save

✅Green banner: Successfully updated the function "NotesAPI".

Step 04: Add DynamoDB Permissions
Configuration → Permissions → click the Role name

Step 05: Add permissions ▼ → Attach policies → search AmazonDynamoDBFullAccess → attach

✅Green banner: Policies have been successfully attached to role.

Step 06: Function Code

import json
import boto3
import uuid
from datetime import datetime
from boto3.dynamodb.conditions import Key

dynamodb = boto3.resource('dynamodb')
table = dynamodb.Table('Notes')

def lambda_handler(event, context):
    """
    Notes API with user isolation.
    Each user can only access their own notes — enforced by extracting
    the userId from the JWT claims and using it as the partition key.
    """
    http_method = event.get('httpMethod', 'GET')
    path = event.get('path', '/')

    # Extract the authenticated user from Cognito claims
    # API Gateway puts the JWT claims here after the authorizer validates the token
    try:
        claims = event['requestContext']['authorizer']['claims']
        user_id = claims['sub']  # Cognito's unique user ID
        user_email = claims.get('email', 'unknown')
    except KeyError:
        return response(401, {'error': 'Unauthorized'})

    print(f"Request from user {user_email} (ID: {user_id})")

    try:
        if path == '/notes' and http_method == 'GET':
            return list_notes(user_id)

        elif path == '/notes' and http_method == 'POST':
            body = json.loads(event.get('body') or '{}')
            return create_note(user_id, body)

        elif path.startswith('/notes/') and http_method == 'GET':
            note_id = path.split('/')[-1]
            return get_note(user_id, note_id)

        elif path.startswith('/notes/') and http_method == 'DELETE':
            note_id = path.split('/')[-1]
            return delete_note(user_id, note_id)

        else:
            return response(404, {'error': 'Not found'})

    except Exception as e:
        print(f"Error: {str(e)}")
        return response(500, {'error': 'Internal server error'})


def list_notes(user_id):
    """List all notes for the authenticated user."""
    result = table.query(
        KeyConditionExpression=Key('userId').eq(user_id)
    )
    return response(200, {
        'notes': result['Items'],
        'count': result['Count']
    })


def create_note(user_id, body):
    """Create a new note owned by the authenticated user."""
    note_id = str(uuid.uuid4())
    note = {
        'userId': user_id,           # ← partition key ensures data isolation
        'noteId': note_id,
        'title': body.get('title', 'Untitled'),
        'content': body.get('content', ''),
        'createdAt': datetime.utcnow().isoformat()
    }
    table.put_item(Item=note)
    return response(201, note)


def get_note(user_id, note_id):
    """Get a specific note. User can only access their own notes."""
    result = table.get_item(Key={'userId': user_id, 'noteId': note_id})
    item = result.get('Item')
    if not item:
        return response(404, {'error': 'Note not found'})
    return response(200, item)


def delete_note(user_id, note_id):
    """Delete a note. User can only delete their own notes."""
    table.delete_item(Key={'userId': user_id, 'noteId': note_id})
    return response(204, {})


def response(status_code, body):
    return {
        'statusCode': status_code,
        'headers': {'Content-Type': 'application/json'},
        'body': json.dumps(body, default=str)
    }

Step 07: Click Deploy.

Notice the user isolation pattern. The userId comes from the JWT claims (set by Cognito, validated by API Gateway), not from the request body. Users can't forge this value because API Gateway won't invoke Lambda without a valid token. This is application-level authorization.

Part VI

Create the API Gateway API

Create the API

Step 01: Open API Gateway → Create API → REST API → Build

Step 02: Create REST API

New API
API name: NotesAPI
API endpoint type: Regional
Security policy - new: TLS_1_0

Click Create API

✅Green banner: Successfully created REST API 'NotesAPI (xoq5e4tu3j)'.

Create the Cognito Authorizer

Step 03: In the left sidebar, click Authorizers
Click Create authorizer

Step 04: Create authorizer

Authorizer name: CognitoAuth
Authorizer type: Cognito
Cognito user pool: Select NotesUserPool
Token source: Authorization (this is the header name)

Click Create authorizer

✅Green banner: Successfully created authorizer 'CognitoAuth'.

Create the Resources and Methods

Step 05: Click Resources → Create resource

Step 06: Create resource

Resource path: /
Resource name: notes
✔ CORS (Cross Origin Resource Sharing)

Click Create resource

✅Green banner: Successfully created resource '/notes'

Step 07: Now create a child resource for /notes/{noteId}:
Select the /notes resource → Create resource

Resource path: /notes/
Resource name: {noteID} (proxy resource)

Click Create resource

✅Green banner: Successfully created resource '/notes/{noteID}'

Add the Methods

Step 08: For each method below, click the resource, then Create method:

Step 09: POST /notes (create a note)

▼ Method type: POST
Integration type: Lambda Function
Lambda proxy integration: ✔
Lambda function: NotesAPI

✅Green banner: Successfully created method 'POST' in 'notes'.

Step 10: GET /notes (list notes)

▼ Method type: GET
Integration type: Lambda Function
Lambda proxy integration: ✔
Lambda function: NotesAPI

✅Green banner: Successfully created method 'GET' in 'notes'.

Step 11: GET /notes/{noteId} (get one note)

▼ Method type: GET
Integration type: Lambda Function
Lambda proxy integration: ✔
Lambda function: NotesAPI

✅Green banner: Successfully created method 'GET' in 'noteID'.

Step 12: DELETE /notes/{noteId} (delete a note)

▼ Method type: DELETE
Integration type: Lambda Function
Lambda proxy integration: ✔
Lambda function: NotesAPI

✅Green banner: Successfully created method 'DELETE' in 'noteID'.

Attach the Authorizer to Each Method

Step 13: For each of the four methods:

13.1: Click the method (e.g., GET under /notes)
13.2: Click Method request → Edit
13.3: Set Authorization to CognitoAuth
13.4: Click Save

✅Green banners: Successfully edited method request for ‘x’

Deploy the API

Step 14: Click Deploy API
Stage: Create a new stage called dev

Click Deploy

✅Green banner: Successfully created deployment for NotesAPI.

Step 15: Copy the Invoke URL

Part VII

Test the Authenticated API

Test One: Unauthenticated Request (Should Fail)

curl -X POST https://YOUR_API_URL/dev/notes \
  -H "Content-Type: application/json" \
  -d '{"title": "My first note", "content": "Hello world"}'

Expected response:

{"message": "Unauthorized"}

⚠️ API Gateway rejected the request because there's no JWT token. Lambda was never invoked.

Test Two: Authenticated Request (Should Succeed)

Use the id_token you got from the token exchange in Part III:

curl -X POST https://YOUR_API_URL/dev/notes \
  -H "Authorization: eyJraWQiOiJ..." \
  -d '{"title": "My first note", "content": "Hello world"}'

Expected response:

{
  "userId": "abc-123-def-456",
  "noteId": "7f8e9d0c-...",
  "title": "My first note",
  "content": "Hello world",
  "createdAt": "2026-04-24T10:00:00"
}

💡 Notice the userId was pulled from the JWT claims. The user didn't have to send it.

Test Three: List Your Notes

curl https://YOUR_API_URL/dev/notes \
  -H "Authorization: YOUR_ID_TOKEN"

You'll only see notes belonging to the authenticated user.

Test Four: Invalid Token (Should Fail)

curl https://YOUR_API_URL/dev/notes \
  -H "Authorization: invalid.token.here"

Expected response:

{"message": "Unauthorized"}

Part VIII

Build a Lambda Authorizer (Advanced)

Cognito authorizers are simple.
They just validate JWT signatures.
Lambda authorizers let you implement custom logic.
Let's see how.

Create the Authorizer Function

Step 01: Open Lambda → Create function

Step 02: Create function

Function name: NotesLambdaAuthorizer
Runtime: Python 3.12

Click Create function

✅Green banner: Successfully created the function "NotesLambdaAuthorizer".

Step 03: Paste this code

import json

def lambda_handler(event, context):
    """
    Custom Lambda authorizer.
    Returns an IAM policy that allows or denies the API call.

    Lambda authorizer results can be cached (default 5 min, max 1 hour)
    to reduce the number of Lambda invocations.
    """
    token = event.get('authorizationToken', '').replace('Bearer ', '')
    method_arn = event['methodArn']

    print(f"Authorizing request with token: {token[:20]}...")

    # In production, validate the JWT signature using the Cognito JWKS
    # For this demo, we just check if the token starts with a specific prefix
    if not token or token == 'invalid':
        # Deny access
        raise Exception('Unauthorized')

    # Simulate extracting user info from the token
    user_id = 'demo-user-001'
    user_role = 'admin' if 'admin' in token else 'user'

    # Build the policy based on the user's role
    if user_role == 'admin':
        effect = 'Allow'
        # Admin can access everything in the API
        resource = method_arn.rsplit('/', 2)[0] + '/*/*'
    else:
        effect = 'Allow'
        # Regular user can only access the specific method
        resource = method_arn

    return {
        'principalId': user_id,
        'policyDocument': {
            'Version': '2012-10-17',
            'Statement': [{
                'Action': 'execute-api:Invoke',
                'Effect': effect,
                'Resource': resource
            }]
        },
        'context': {
            'userId': user_id,
            'userRole': user_role
        }
    }

Step 04: Click Deploy

✅Green banner: Successfully updated the function "NotesLambdaAuthorizer"

Step 05: Attach It to the API (Walkthrough Only)
In API Gateway:
1. Authorizers → Create authorizer
2. Configure:

Name: LambdaAuth
Authorizer type: Lambda
Lambda function: NotesLambdaAuthorizer
Lambda event payload: Token
Token source: Authorization
Authorization caching: 300 seconds 3. Create authorizer

You could switch methods to use this instead of the Cognito authorizer. We won't switch for this tutorial. The Cognito authorizer is already doing its job.

💡Lambda authorizer results are cached (up to 1 hour). This reduces Lambda invocations but means permission changes aren't immediate. Two types:

Token-based: receives the token, caches by token value

Request-based: receives the full request, caches by specified identity sources.

🏗️ What You Built | 📘Exam Concepts Recap

What You Built	Exam Concept
Created a Cognito User Pool	Authentication via managed user directory
Configured the Cognito Hosted UI	OAuth 2.0 authorization code / implicit grant flows
Signed in and received ID, access, and refresh tokens	The three Cognito token types and their purposes
Attached a Cognito Authorizer to API Gateway methods	JWT validation at the API layer
Extracted `sub` and `email` from JWT claims in Lambda	Passing identity from API Gateway to Lambda
Used `userId` as the DynamoDB partition key	Application-level authorization (data isolation)
Built a Lambda authorizer with custom logic	Fine-grained authorization
Returned an IAM policy from the Lambda authorizer	How Lambda authorizers control access
Set the Lambda authorizer cache TTL	Performance vs permission freshness trade-off
Tested with and without a valid token	API Gateway rejects unauthenticated requests before Lambda

⚠️ Clean Up Protocol

API Gateway → Delete NotesAPI
Lambda → Delete NotesAPI, NotesLambdaAuthorizer
DynamoDB → Delete the Notes table
Cognito → Delete the User Pool (also delete the domain first)
IAM → Delete the Lambda execution roles
CloudWatch → Delete the log groups

Key Takeaways

Cognito User Pools = authentication (tokens). Identity Pools = authorization (AWS credentials).
Three tokens: ID (user identity), Access (API auth), Refresh (getting new tokens).
Cognito Authorizer = simple JWT validation. Lambda Authorizer = custom logic.
Lambda authorizer caching: (up to 1 hour) reduces invocations but delays permission changes.
Never hardcode credentials: use IAM roles (execution role for Lambda, task role for ECS, instance profile for EC2).
STS AssumeRole for cross-account access and temporary credentials.
SigV4 signing is used for IAM authorization on API Gateway (service-to-service).
Credential resolution order: env vars → credentials file → config → container → instance profile.
JWT claims are passed to Lambda via event.requestContext.authorizer.claims.
Application-level authorization: use the authenticated user ID as the partition key to enforce data isolation.

Additional Resources

🏗️

Formerly Known As

Ntombizakhona Mabaso — Wed, 20 May 2026 11:25:01 +0000

Keeping Up With The Renamed World

Tech changes fast.
Sometimes too fast.

One day you are confidently talking about a service, framework, tool, or company… and the next day it has a completely different name, a new logo, a rebrand announcement, and an updated pricing page. 😭

That is exactly why I am starting this series:

"Formerly Known As"

A blog series where I keep up with the former names of current resources, services, platforms, tools, and technologies across Cloud, AI, Web Development, and Tech in general.

Because honestly?

A huge part of learning tech is also learning its history.

If you have ever said things like:

“Wait… wasn’t this called something else before?”
“Isn’t this just a rebrand?”
“I still call it Twitter.”
“When did this AWS service change names?”
“Hold on… what happened to that?”

…then this series is for you, and mostly me.😂

I am going to explore:

1. Old names vs new names
2. Why companies rebrand
3. Mergers and acquisitions
4. Product evolution
5. Retired services
6. Tech nostalgia
7. The confusion these changes create for beginners

And maybe most importantly:
How keeping up with naming changes helps us better understand the tech ecosystem.

Some posts may be quick and fun.
Others may dive deeper into the history behind a platform or service.

Either way, this series is for:

Developers 👩🏽‍💻
Cloud learners ☁️
Curious tech people 🔍
Documentation readers 📚
People who still accidentally use old names 😅
Me.

"Formerly Known As."

The internet never forgets old names…
and apparently neither do we.

No particular frequency.
No particular theme.
Just the renamed things that cross my mind...and my notifications.

Use Data Stores In Application Development | 🏗️ Build A Product Catalog API

Ntombizakhona Mabaso — Tue, 12 May 2026 20:45:57 +0000

Exam Guide: Developer - Associate
🏗️ Domain 1: Development with AWS Services
📘 Task 3: Use Data Stores In Application Development

DynamoDB dominates this task. The need to understand table design, key selection, indexing, consistency models, and how to write efficient queries is essential. As well as caching with ElastiCache and DAX. Plus specialized stores like OpenSearch.

📘 Concepts

DynamoDB Key Concepts

Primary Keys

Every table needs one. Two options:

Simple primary key: Partition key only (PK). Each item has a unique PK.
Composite primary key: Partition key (PK) + Sort key (SK). Multiple items can share a PK if they have different SKs.

Partition Key Selection

The partition key determines which physical partition stores your data. A good partition key has high cardinality which refers to many distinct values so that the data spreads evenly.

Good Partition Keys	Bad Partition Keys
userId, orderId, sessionId	status ("active"/"inactive")
deviceId, transactionId	country (few values, uneven distribution)
email, accountId	date (hot partition for today)

Consistency Models

Model	Behaviour	Cost	Available On
Eventually Consistent	May return stale data (usually consistent within 1 second)	1x read capacity	Base table + GSIs
Strongly Consistent	Always returns the most up-to-date data	2x read capacity	Base table only (NOT GSIs)

Query vs Scan

Operation	How It Works	Cost	When to Use
Query	Finds items by partition key + optional sort key condition	Reads only matching items	Always prefer this
Scan	Reads every item in the table	Reads entire table	Analytics, one-time migrations only
GetItem	Fetches one item by its full primary key	Reads exactly one item	When you know the exact key

FilterExpression does NOT reduce the amount of data read. It only filters what's returned to you. You still pay for the full scan/query. To reduce reads, use better key design or GSIs.

Global Secondary Index (GSI) vs Local Secondary Index (LSI)

Feature	GSI	LSI
Partition Key	Different from base table	Same as base table
Sort Key	Different from base table	Different from base table
When To Create	Anytime	At table creation only
Throughput	Has its own (separate from base table)	Shares with base table
Consistency	Eventually consistent only	Supports strongly consistent
Limit	20 per table	5 per table

GSI Projection Types:

ALL: all attributes (most flexible, most storage cost)
KEYS_ONLY: only key attributes (cheapest)
INCLUDE: keys + specified attributes (balanced)

Caching Options

Service	Use Case	Latency	Works With
DAX	DynamoDB read cache	Microseconds	DynamoDB only, eventually consistent only
ElastiCache Redis	General-purpose cache	Sub-millisecond	Any data source, complex data types, persistence
ElastiCache Memcached	Simple caching	Sub-millisecond	Any data source, multi-threaded, no persistence

Specialized Data Stores

Store	Use Case
DynamoDB	Key-value lookups, known access patterns, serverless
RDS/Aurora	Relational data, complex joins, ACID transactions
OpenSearch	Full-text search, log analytics, complex queries
S3	Object storage, data lake, large files
ElastiCache	Session storage, leaderboards, real-time analytics

Data Lifecycle

DynamoDB TTL

Automatically deletes expired items at no cost. Eventually consistent (up to 48 hours delay).

S3 Lifecycle Policies

Transition objects between storage classes (Standard → IA → Glacier) or expire them after a set time.

🏗️ Build A Product Catalog API

Now let's put these concepts into practice by builidng a Product Catalog API backed by DynamoDB:

A DynamoDB table with a composite primary key and a Global Secondary Index (GSI)
A Lambda function that performs queries, scans, and writes
TTL configured for automatic data expiration
DAX caching in front of DynamoDB
A clear understanding of when to use query vs scan, GSI vs LSI, and strong vs eventual consistency

Prerequisites

An AWS account (free tier covers DynamoDB and Lambda)

Part I

Design the DynamoDB Table

Before creating anything, let's think about access patterns. This is the most important step in DynamoDB design.

Our Access Patterns

Access Pattern	How We'll Query
Get a product by ID	Query PK = `PRODUCT#123`
List all products in a category	Query PK = `CATEGORY#electronics`
Get a product's reviews	Query PK = `PRODUCT#123`, SK begins_with `REVIEW#`
Find products by price range in a category	Query GSI with category + price
List recently added products	Query GSI with status + createdAt

Create the Table

Step 01: Open the DynamoDB console

Step 02: Click Create table

Table name: ProductCatalog
Partition key: PK (String)
Sort key — optional: SK (String)

Step 03: Under Table settings, choose Customize settings

Step 04: Read/write capacity settings: On-demand

⚠️ Don't create the just table yet, let's add a GSI first

Add a Global Secondary Index

Step 05: Scroll down to Secondary indexes → click Create global index:

Index name: GSI1
Partition key: GSI1PK (String)
Sort key: GSI1SK (String)
Attribute projections: All

Click Create index, then click Create table.

✅Green banner: The ProductCatalog table was created successfully.

Why This Design?

We're using the single-table design pattern with overloaded keys:

Base table:
  PK = "PRODUCT#laptop-001"    SK = "METADATA"           → product details
  PK = "PRODUCT#laptop-001"    SK = "REVIEW#2026-04-24"  → a review
  PK = "CATEGORY#electronics"  SK = "PRODUCT#laptop-001" → category listing

GSI1:
  GSI1PK = "CATEGORY#electronics"  GSI1SK = "PRICE#00079.99" → find by price
  GSI1PK = "STATUS#active"         GSI1SK = "2026-04-24"     → find by date

Partition Key Selection: A good partition key has high cardinality (many distinct values). Bad examples: status (only a few values → hot partition), country (uneven distribution). Good examples: userId, productId, orderId.

Part II

Add Sample Data

Using the Console

Step 01: In the DynamoDB console, click on ProductCatalog

Step 02: Click Explore table items

Step 03: Click Create item
Switch to JSON view (toggle at the top)
Paste this item:

{
  "PK": {"S": "PRODUCT#laptop-001"},
  "SK": {"S": "METADATA"},
  "GSI1PK": {"S": "CATEGORY#electronics"},
  "GSI1SK": {"S": "PRICE#00999.99"},
  "name": {"S": "Pro Laptop 15"},
  "description": {"S": "15-inch laptop with 16GB RAM"},
  "price": {"N": "999.99"},
  "category": {"S": "electronics"},
  "status": {"S": "active"},
  "createdAt": {"S": "2026-04-20T10:00:00Z"},
  "stock": {"N": "50"}
}

Step 04: Click Create item

Add a few more products:

{
  "PK": {"S": "PRODUCT#mouse-002"},
  "SK": {"S": "METADATA"},
  "GSI1PK": {"S": "CATEGORY#electronics"},
  "GSI1SK": {"S": "PRICE#00029.99"},
  "name": {"S": "Wireless Mouse"},
  "description": {"S": "Ergonomic wireless mouse"},
  "price": {"N": "29.99"},
  "category": {"S": "electronics"},
  "status": {"S": "active"},
  "createdAt": {"S": "2026-04-22T10:00:00Z"},
  "stock": {"N": "200"}
}

{
  "PK": {"S": "PRODUCT#laptop-001"},
  "SK": {"S": "REVIEW#2026-04-24#user-001"},
  "rating": {"N": "5"},
  "comment": {"S": "Great laptop, fast and reliable"},
  "userId": {"S": "user-001"},
  "createdAt": {"S": "2026-04-24T14:30:00Z"}
}

{
  "PK": {"S": "CATEGORY#electronics"},
  "SK": {"S": "PRODUCT#laptop-001"},
  "name": {"S": "Pro Laptop 15"},
  "price": {"N": "999.99"}
}

{
  "PK": {"S": "CATEGORY#electronics"},
  "SK": {"S": "PRODUCT#mouse-002"},
  "name": {"S": "Wireless Mouse"},
  "price": {"N": "29.99"}
}

Part III

Query vs Scan

Query (Efficient)

Step 01: In the ▼ Scan or query items tab, switch from Scan to Query

Step 02: Set:

Partition key Value: PRODUCT#laptop-001
Sort key Value: Begins with ▼ REVIEW Click Run

✅Green banner: Completed · Items returned: 1 · Items scanned: 1 · Efficiency: 100% · RCUs consumed: 0.5

You'll see only the review items for that product. DynamoDB read exactly the items you asked for. Efficient.

Scan (Expensive)

Step 03: Switch back to Scan
Click Run with no filters

✅Green banner: Completed · Items returned: 5 · Items scanned: 5 · Efficiency: 100% · RCUs consumed: 2

You'll see ALL items in the table. DynamoDB read every single item. On a table with millions of items. This is slow and expensive.

Step 04: Click Add filter

Attribute name: category
Type: String
Condition: Equal to
Value: electronics Click Run

✅Green banner: Completed · Items returned: 2 · Items scanned: 5 · Efficiency: 40% · RCUs consumed: 2

You'll see only electronics items, but DynamoDB still read the entire table and filtered afterward. The filter doesn't reduce the read cost.

💡 FilterExpression does NOT reduce the amount of data read. It only filters what's returned to you. You still pay for the full scan. To reduce reads, use better key design or GSIs.

Query the GSI

Step 05: Switch to Query

Step 06: Change the Index dropdown to GSI1

Partition key (GSI1PK) Value: CATEGORY#electronics
Sort key (GSI1SK) Value: Begins with ▼ PRICE# Click Run

✅Green banner:: Completed · Items returned: 2 · Items scanned: 2 · Efficiency: 100% · RCUs consumed: 0.5

This efficiently finds all electronics products, sorted by price. That's the power of a well-designed GSI.

Part IV

Build the API with Lambda

Create the Lambda Function

Step 01: Open the Lambda console → Create function

Function name: ProductCatalogAPI
Runtime: Python 3.12 Click Create function

✅Green banner:: Successfully created the function "ProductCatalogAPI".

Step 02: Configuration → General configuration → click Edit

Memory: 256 MB
Timeout: 10 seconds Click Save

Step 03:Add DynamoDB Permissions
Configuration → Permissions → click the role name
Add permissions → Attach policies
Search for and attach AmazonDynamoDBFullAccess

⚠️ In production, scope this down to only the ProductCatalog table. For this tutorial, full access keeps things simple. Brevity.

Step 04: Write the Function Code

import json
import boto3
from decimal import Decimal
from boto3.dynamodb.conditions import Key, Attr

# Initialize OUTSIDE the handler — reused across warm invocations
dynamodb = boto3.resource('dynamodb')
table = dynamodb.Table('ProductCatalog')

class DecimalEncoder(json.JSONEncoder):
    """DynamoDB returns Decimal types — this converts them to float for JSON."""
    def default(self, obj):
        if isinstance(obj, Decimal):
            return float(obj)
        return super().default(obj)

def lambda_handler(event, context):
    """
    Routes requests based on the HTTP method and path.
    Demonstrates query, scan, get_item, and put_item operations.
    """
    http_method = event.get('httpMethod', 'GET')
    path = event.get('path', '/')
    path_params = event.get('pathParameters') or {}
    query_params = event.get('queryStringParameters') or {}

    try:
        if path == '/products' and http_method == 'GET':
            return list_products(query_params)

        elif path.startswith('/products/') and http_method == 'GET':
            product_id = path_params.get('productId', path.split('/')[-1])
            return get_product(product_id)

        elif path == '/products' and http_method == 'POST':
            body = json.loads(event.get('body', '{}'))
            return create_product(body)

        elif path.startswith('/products/') and path.endswith('/reviews'):
            product_id = path_params.get('productId', path.split('/')[2])
            return get_reviews(product_id)

        else:
            return response(404, {'error': 'Not found'})

    except Exception as e:
        print(f"Error: {str(e)}")
        return response(500, {'error': 'Internal server error'})


def list_products(query_params):
    """
    List products by category using QUERY (efficient).
    Falls back to SCAN if no category is specified (expensive — avoid in production).
    """
    category = query_params.get('category')

    if category:
        # QUERY — efficient, reads only matching items
        print(f"Querying products in category: {category}")
        result = table.query(
            KeyConditionExpression=Key('PK').eq(f'CATEGORY#{category}')
        )
    else:
        # SCAN — reads entire table, expensive!
        # In production, require a category or use pagination
        print("WARNING: Scanning entire table — this is expensive!")
        result = table.scan(
            FilterExpression=Attr('SK').eq('METADATA')
        )

    return response(200, {
        'products': result['Items'],
        'count': result['Count'],
        'scannedCount': result['ScannedCount']  # Shows the difference between query and scan
    })


def get_product(product_id):
    """
    Get a single product by ID using GET_ITEM.
    This is the most efficient read — directly accesses one item by its full key.
    """
    result = table.get_item(
        Key={
            'PK': f'PRODUCT#{product_id}',
            'SK': 'METADATA'
        },
        # ConsistentRead=True  # Uncomment for strongly consistent read (2x cost)
    )

    item = result.get('Item')
    if not item:
        return response(404, {'error': f'Product {product_id} not found'})

    return response(200, item)


def get_reviews(product_id):
    """
    Get all reviews for a product using QUERY with sort key condition.
    begins_with on the sort key efficiently finds all reviews.
    """
    result = table.query(
        KeyConditionExpression=(
            Key('PK').eq(f'PRODUCT#{product_id}') &
            Key('SK').begins_with('REVIEW#')
        ),
        ScanIndexForward=False  # Sort descending (newest first)
    )

    return response(200, {
        'productId': product_id,
        'reviews': result['Items'],
        'count': result['Count']
    })


def create_product(body):
    """
    Create a new product using PUT_ITEM.
    Also creates the category listing item for efficient category queries.
    """
    product_id = body['productId']
    category = body['category']
    price = Decimal(str(body['price']))  # DynamoDB requires Decimal, not float!

    # Write the product metadata
    table.put_item(Item={
        'PK': f'PRODUCT#{product_id}',
        'SK': 'METADATA',
        'GSI1PK': f'CATEGORY#{category}',
        'GSI1SK': f'PRICE#{price:010.2f}',  # Zero-padded for correct sort order
        'name': body['name'],
        'description': body.get('description', ''),
        'price': price,
        'category': category,
        'status': 'active',
        'stock': body.get('stock', 0)
    })

    # Write the category listing (denormalization for efficient queries)
    table.put_item(Item={
        'PK': f'CATEGORY#{category}',
        'SK': f'PRODUCT#{product_id}',
        'name': body['name'],
        'price': price
    })

    return response(201, {'message': 'Product created', 'productId': product_id})


def response(status_code, body):
    return {
        'statusCode': status_code,
        'headers': {'Content-Type': 'application/json'},
        'body': json.dumps(body, cls=DecimalEncoder)
    }

Step 05: Click Deploy

✅Green banner: Successfully updated the function "ProductCatalogAPI".

Notice Decimal(str(body['price'])). DynamoDB does NOT support Python float. You must use Decimal. This is a common gotcha.

Step 06: Test The Function
Click the Test tab
Create test events for each operation:

List products by category:

{
  "httpMethod": "GET",
  "path": "/products",
  "queryStringParameters": {"category": "electronics"},
  "pathParameters": null
}

Get a single product:

{
  "httpMethod": "GET",
  "path": "/products/laptop-001",
  "pathParameters": {"productId": "laptop-001"},
  "queryStringParameters": null
}

Get reviews:

{
  "httpMethod": "GET",
  "path": "/products/laptop-001/reviews",
  "pathParameters": {"productId": "laptop-001"},
  "queryStringParameters": null
}

Create a product:

{
  "httpMethod": "POST",
  "path": "/products",
  "body": "{\"productId\":\"keyboard-003\",\"name\":\"Mechanical Keyboard\",\"category\":\"electronics\",\"price\":89.99,\"stock\":100}",
  "pathParameters": null,
  "queryStringParameters": null
}

💡 Run each test and check the results. Look at the count vs scannedCount in the list response. When using query, they'll be equal. When scanning, scannedCount will be higher.

Part V

Consistency Models

See the Difference

Step 01: In the Lambda function, find the get_product function
Uncomment the ConsistentRead=True line
Deploy

✅Green banner: Successfully updated the function "ProductCatalogAPI".

Now get_product uses strongly consistent reads:

Always returns the latest data

Costs 2x the read capacity

Only works on the base table (not GSIs)

💡 GSIs only support eventually consistent reads.

When to Use Each

Use Case	Consistency	Why
Product catalog browsing	Eventually consistent	Stale data for a second is fine
Shopping cart	Strongly consistent	User expects to see what they just added
Inventory check before purchase	Strongly consistent	Must be accurate to avoid overselling
Analytics dashboard	Eventually consistent	Slight delay is acceptable

Part VI

TTL: Automatic Data Expiration

Enable TTL

Step 01: In the DynamoDB console, click on ProductCatalog

Step 02: Click ▼ Actions
Click Turn on TTL

Step 03: Turn on Time to Live (TTL)
TTL attribute name: ttl
Click Turn on TTL

✅Green banner: Successfully activated Time to Live for the ProductCatalog table with the ttl attribute.

Step 04: Add Items with TTL

Create an item with a TTL value (Unix epoch timestamp):

{
  "PK": {"S": "SESSION#user-001"},
  "SK": {"S": "DATA"},
  "userId": {"S": "user-001"},
  "cartItems": {"L": [{"S": "laptop-001"}, {"S": "mouse-002"}]},
  "ttl": {"N": "1745625600"}
}

The ttl value is a Unix timestamp. Items are deleted after this time. Use an epoch converter to set a time a few minutes in the future for testing.

💡 TTL deletion is eventually consistent: items may persist for up to 48 hours after expiration. Don't rely on TTL for exact timing. Always filter out expired items in your queries as a safety measure.

Part VII

Caching with DAX

DAX (DynamoDB Accelerator) is an in-memory cache that sits in front of DynamoDB. It's a drop-in replacement. Same API, just change the client endpoint.

When to Use DAX

Scenario	Use DAX?
Read-heavy workload, same items queried repeatedly	Yes
Microsecond response times needed	Yes
Write-heavy workload	No (DAX is a read cache)
Need strongly consistent reads	No (DAX returns eventually consistent)
Diverse access patterns, rarely same item twice	No (low cache hit rate)

How DAX Works

Without DAX:
  App → DynamoDB (single-digit millisecond reads)

With DAX:
  App → DAX (microsecond reads if cached) → DynamoDB (on cache miss)

Console Walkthrough (Don't Create.Just Understand)

💸 DAX requires a VPC and costs money even when idle. We'll walk through the setup without creating it.

Step 01: Click ▼ DAX

Step 02: Click Create cluster

Cluster name: product-cache
Node type family: t-type family
Node type: dax.t3.small
Cluster size: 3 nodes (for high availability) Click Next

Step 03: Configure networks

Network Type: IPv4
Subnet group: Create new
Subnet group name: MySubnetGroup
VPC ID: defaultVPC
Subnets: Select all
Security group: default ▼ Click View in EC2 console

Step 04: Allow port 8111 from your Lambda functions

✅Green banner: Inbound security group rules successfully modified on security group

Step 05: Configure Security

IAM Service role for DynamoDB access: Create new
IAM role name: DaxToDynamoDB Click Next → Click Next → Click Create cluster

✅Green banner: Successfully created the cluster product-cache.

Step 06: In your Lambda code, you'd change one line:

# Without DAX
dynamodb = boto3.resource('dynamodb')

# With DAX — same API, just different endpoint
import amazondax
dax_client = amazondax.AmazonDaxClient(
    endpoints=['product-cache.abc123.dax-clusters.us-east-1.amazonaws.com:8111']
)
table = dax_client.Table('ProductCatalog')
# All your existing code works unchanged!

DAX is a drop-in replacement for DynamoDB reads. Same API, same code. Just change the client. 💡 But remember: DAX only supports eventually consistent reads and is for read-heavy workloads.

🏗️ What You Built | 📘Exam Concepts Recap

What You Built	Exam Concept
Designed a table around access patterns first	Access-pattern-driven DynamoDB design
Created a composite primary key (PK + SK)	Single-table design, sort key relationships
Added a Global Secondary Index with different keys	GSI for alternate access patterns
Used overloaded keys (`PRODUCT#`, `CATEGORY#`, `REVIEW#`)	Single-table design pattern
Queried by partition key with `begins_with` on sort key	Efficient query operations
Ran a scan and compared `Count` vs `ScannedCount`	Scan is expensive: it reads the entire table
Added a `FilterExpression` to a scan	Filters run AFTER reading: don't save capacity
Used `Decimal(str(price))` instead of `float`	DynamoDB type system: no float support
Toggled `ConsistentRead=True` on `get_item`	Strongly vs eventually consistent reads
Noted GSIs only support eventual consistency	GSI limitations
Enabled TTL on a `ttl` attribute	Automatic data lifecycle management
Walked through DAX setup	Read caching for DynamoDB, microsecond latency

⚠️ Clean Up Protocol

1. DynamoDB → Delete the ProductCatalog table
2. Lambda → Delete ProductCatalogAPI
3. IAM → Delete the Lambda execution role
4. CloudWatch → Delete the log groups

Key Takeaways

Partition key cardinality: high cardinality = even distribution = good performance
Query > Scan: always prefer query. Scan reads the entire table.
FilterExpression doesn't save reads: it filters after reading. Use key design or GSIs instead.
GSIs can be added anytime. LSIs must be created with the table
GSIs are eventually consistent only: no strongly consistent reads on GSIs
Use Decimal, not float for DynamoDB numbers in Python
TTL is free but eventually consistent (up to 48 hours delay)
DAX = DynamoDB read cache (microsecond reads). Same API as DynamoDB.
DAX doesn't support strongly consistent reads
Single-table design with overloaded keys is the recommended DynamoDB pattern

Additional Resources

🏗️

Develop Code for Lambda | 🏗️ Build A Real-Time Data Processing Pipeline

Ntombizakhona Mabaso — Thu, 07 May 2026 20:34:21 +0000

Exam Guide: Developer - Associate
🏗️ Domain 1: Development with AWS Services
📘 Task 2: Develop Code for Lambda

Lambda is the most heavily tested service on the DVA-C02. It is also perhaps, one of the most used and talked about services in general too, well after EC2 and S3. So, you need to know how to configure it, write code for it, handle errors, tune performance, and integrate it with practically every other AWS service.

📘Concepts

Lambda Execution Model

When you invoke a Lambda function, AWS:
1. Finds or creates an execution environment (container)
2. Loads your code and initializes it (cold start)
3. Runs your handler function
4. Keeps the environment warm for reuse (subsequent invocations skip step 2)

Code outside the handler runs once during cold start and is reused. This is why you initialize SDK clients and database connections outside the handler.

Key Configuration Parameters

Parameter	Range	Default	Notes
Memory	128 MB – 10,240 MB	128 MB	CPU scales proportionally with memory
Timeout	1s – 900s (15 min)	3s	Max 15 minutes: use Step Functions or ECS for longer or Durable functions
Concurrency	Reserved or Provisioned	1000/region	Reserved = guarantee + cap; Provisioned = pre-warmed
Ephemeral storage	512 MB – 10,240 MB	512 MB	/tmp directory for temporary files
Layers	Up to 5	—	250 MB total unzipped (function + layers)

Invocation Types and Error Handling

Invocation Type	Source Examples	Retry Behavior	Error Destination
Synchronous	API Gateway, ALB	No automatic retries caller handles it	Caller gets the error
Asynchronous	S3, SNS, EventBridge	2 automatic retries (3 total)	DLQ or Lambda Destinations
Event source mapping	SQS, Kinesis, DynamoDB Streams	Retries until record expires or succeeds	DLQ (SQS) or on-failure destination

Lambda Destinations vs DLQ

Feature	DLQ	Lambda Destinations
Captures success	No	Yes
Captures failure	Yes	Yes
Context included	Minimal	Full (request, response, error)
Supported targets	SQS, SNS	SQS, SNS, Lambda, EventBridge
Works with	Async invocations	Async invocations

Lambda Destinations are the modern approach and preferred over DLQs. DLQs are still valid for SQS event source mappings.

VPC Access

By default, Lambda runs in an AWS-managed VPC and can access the internet and public AWS services. When you attach Lambda to your own VPC:

Without VPC:  Lambda → Internet → AWS Services ✅  |  Lambda → Private RDS ❌
With VPC:     Lambda → Private RDS ✅  |  Lambda → Internet ❌ (needs NAT Gateway)
              Lambda → VPC Endpoint → AWS Services ✅

Key Points:

1. VPC Lambda needs private subnets with a NAT Gateway for internet access
2. Use VPC endpoints (PrivateLink) to access DynamoDB, S3, SQS without NAT
3. Use RDS Proxy for connection pooling because Lambda can overwhelm databases with concurrent connections

Cold Starts

A cold start happens when Lambda creates a new execution environment. Strategies to reduce them:

1. Provisioned Concurrency: pre-warms environments (costs money when idle)
2. Keep deployment packages small: smaller packages initialize faster
3. Initialize outside the handler: SDK clients, DB connections, config
4. Use ARM (Graviton2): often faster and 20% cheaper

Kinesis And Lambda Key Settings

Setting	What It Does	Relevance
Batch size	Records per invocation (max 10,000)	Larger batches = fewer invocations
Batch window	Wait time to fill batch (max 300s)	Reduces invocations for low-traffic streams
Parallelization factor (1–10)	Concurrent batches per shard	Increases throughput without adding shards
Bisect batch on error	Splits batch in half on failure	Isolates the bad record
Starting position	LATEST or TRIM_HORIZON	LATEST = new only. TRIM_HORIZON = from beginning

🏗️ Build A Real Time Data Processing Pipeline

Now let's put these concepts into practice by building a Real-Time Data Processing Pipeline using Lambda:

A Lambda function with VPC access connecting to an RDS database
Lambda layers for shared dependencies
A DLQ (Dead Letter Queue) and Lambda Destinations for error handling
A Kinesis stream processor that transforms data in near real time
Performance tuning with memory configuration

This covers all the skills for this task: Lambda configuration, VPC access, error handling, event lifecycle, integrations, and performance tuning.

Prerequisites

An AWS account

Part I

Create and Configure a Lambda Function

Create the Function

Step 01: Open the Lambda

Step 02: **Click **Create function

Step 03: Create function
Choose Author from scratch

Function name: DataProcessor
Runtime: Python 3.12

Click Create function

✅Green banner: Successfully created the function "DataProcessor".

Before writing code, let's walk through the configuration tabs.

Step 04: Click the Configuration tab.
General configuration
Click Edit:

Memory: 256 MB (default is 128 MB)
Ephemeral storage: 512 MB (default)
Timeout: 0 min 30 sec (default is 3 seconds)

Click Save.

Lambda allocates CPU proportionally to memory. At 1,769 MB you get one full vCPU. Increasing memory from 128 MB to 256 MB doubles your CPU therefore your function might run in half the time, costing the same or less.

Step 05: Environment variables
Click Environment variables → Edit → Add environment variable:

Key: LOG_LEVEL, Value: INFO
Key: STAGE, Value: dev

Click Save.

✅Green banner: Successfully updated the function "DataProcessor".

import os

# Access environment variables in your code
log_level = os.environ.get('LOG_LEVEL', 'INFO')
stage = os.environ.get('STAGE', 'dev')

Lambda encrypts all environment variables at rest by default using an AWS managed KMS key. For extra security, you can use a customer managed KMS key and decrypt in your code.

Part II

Create a Lambda Layer

Layers let you share code and libraries across multiple functions. Let's create one with a utility module.

Build and Upload a Layer

Layers must be uploaded as a zip file.
We'll use AWS CloudShell: a browser-based terminal built into the AWS Console so that you don't need anything installed locally.

Step 01: Open CloudShell by clicking the terminal icon (>_) in the top navigation bar of the AWS Console next to the search bar

Step 02: Wait for it to initialize as it might take a few seconds the first time

Step 03:
Run these commands:

mkdir -p python
cat > python/utils.py << 'EOF'
"""
Shared utilities for Lambda functions.
This module is packaged as a Lambda Layer so multiple functions can use it.
"""
import json
import time
import random

def retry_with_backoff(func, max_retries=3, base_delay=0.5):
    """Retry with exponential backoff and jitter."""
    for attempt in range(max_retries + 1):
        try:
            return func()
        except Exception as e:
            if attempt == max_retries:
                raise
            delay = base_delay * (2 ** attempt)
            jitter = random.uniform(0, delay)
            time.sleep(delay + jitter)

def format_response(status_code, body):
    """Standard API Gateway response format."""
    return {
        'statusCode': status_code,
        'headers': {'Content-Type': 'application/json'},
        'body': json.dumps(body, default=str)
    }
EOF

zip -r utils-layer.zip python/

Step 04: Download the zip
Click Actions ▼ (top right of CloudShell) → Download file → type utils-layer.zip → click Download

Why the python/ folder? Lambda layers must follow a specific directory structure. For Python, your code must be inside a python/ folder in the zip. Lambda adds this path to sys.path automatically so your functions can import utils directly.

Step 05: Upload the Layer via Console
In the Lambda console, click Layers in the left sidebar

Step 06: Click Create layer

Name: shared-utils
Upload: Choose the utils-layer.zip file
Compatible runtimes - optional: Python 3.12 Click Create

✅Green banner: Successfully created layer shared-utils version 1.

Step 07: Attach the Layer to Your Function
Go back to the DataProcessor function
Scroll down to the Layers section
Click Edit

Step 08: Edit layers
Click Add a layer
Choose Custom layers
Select shared-utils
Version: 1
Click Add

Click Save

✅Green banner: Successfully updated the function "DataProcessor".

Step 09: Now you can use it in your function code:

from utils import format_response, retry_with_backoff

def lambda_handler(event, context):
    return format_response(200, {'message': 'Hello from DataProcessor'})

A function can have up to 5 layers. The total unzipped size of the function + all layers can't exceed 250 MB. Layers are versioned and immutable. Each upload creates a new version.

Part III

Error Handling: DLQ and Lambda Destinations

Create a Dead Letter Queue

Step 01: Open the SQS console

Step 02: Click Create queue

Type: Standard
Name: data-processor-dlq

Click Create queue

✅Green banner: Queue data-processor-dlq created successfully

Step 03: Copy the Queue ARN

Step 04: Create a Success Queue (for Destinations)
Create another queue:

Name: data-processor-success

Copy the Queue ARN

Step 05: Lambda → DataProcessor → Configuration → Permissions → Click the Role name
Add permissions → Attach policies
Search for AmazonSQSFullAccess and attach it

Step 06: Configure the DLQ on the Lambda Function
Go back to the DataProcessor function
Configuration tab → Asynchronous invocation → Edit

Maximum age of event: 1 h 0 min 0 sec
Retry attempts: 2
Dead-letter queue service ▼: Select Amazon SQS
Queue ▼: Select data-processor-dlq

Click Save

✅Green banner: Successfully updated the function "DataProcessor".

Destinations are the modern approach. They capture both success AND failure.

Step 07: Configure Lambda Destinations
Still in Asynchronous invocation, find the Destinations section
Click Add destination

Step 08: Add destination

Source: Asynchronous invocation
Condition: On success
Destination type: SQS queue
Destination: data-processor-success

Click Save

✅Green banner: Your changes have been saved.

Step 09: Click Add destination again

Source: Asynchronous invocation
Condition: On failure
Destination type: SQS queue
Destination: data-processor-dlq

Click Save

✅Green banner: Your changes have been saved.

Lambda Destinations are preferred over DLQs because:

Destinations capture both success and failure (DLQ only captures failure)

Destinations include more context (request payload, response, error details)

Destinations work with async invocations only (same as DLQ)

DLQs are still valid for SQS event source mappings

Test Error Handling

Step 10: Let's create a function that sometimes fails to see the error handling in action:

Deploy this code, then test it

import json
import random

def lambda_handler(event, context):
    """
    This function randomly fails to demonstrate error handling.
    When invoked asynchronously:
    1. First attempt fails → Lambda retries
    2. Second attempt fails → Lambda retries again
    3. Third attempt fails → Event goes to DLQ / failure destination
    """
    action = event.get('action', 'process')

    if action == 'fail':
        # Always fail — will end up in DLQ after 3 attempts
        raise Exception("Simulated failure for testing DLQ")

    if action == 'random':
        # 50% chance of failure
        if random.random() < 0.5:
            raise Exception("Random failure occurred")

    return {
        'statusCode': 200,
        'body': json.dumps({'message': 'Processed successfully', 'action': action})
    }

Step 11: Click Test tab

Step 12: Test event

Test event action: Create new event
Invocation type: Synchronous
Event name: TestFailure
Even JSON:

{
  "action": "fail"
}

Click Test

❌ Executing function: failed

The function will fail. Since this is a synchronous test invocation, you'll see the error immediately. To test the DLQ flow, you need an asynchronous invocation (from S3, SNS, or EventBridge).

Part IV

Lambda with VPC Access

When Lambda needs to access private resources like RDS or ElastiCache, you attach it to a VPC. Let's walk through the configuration.

Understanding VPC Access

Without VPC:
  Lambda → Internet → AWS Services (DynamoDB, S3, SQS)  ✅
  Lambda → Private RDS                                    ❌

With VPC:
  Lambda → VPC → Private RDS                              ✅
  Lambda → VPC → Internet                                 ❌ (no NAT)
  Lambda → VPC → NAT Gateway → Internet                   ✅
  Lambda → VPC → VPC Endpoint → AWS Services              ✅

Configure VPC Access (Console Walkthrough)

Step 01: Open the DataProcessor function
Configuration tab → VPC → Edit

Step 02: Edit VPC
Select the default VPC
Select at least 2 private subnets (for high availability)
Select or create a security group that allows outbound traffic

Click Save

⚠️Important: The Lambda execution role needs the AWSLambdaVPCAccessExecutionRole managed policy.

Step 03: Connection Reuse Pattern

When connecting to databases from Lambda, initialize the connection outside the handler:

import os
import json
import boto3

# This runs ONCE during cold start, then reuses across warm invocations
# This is critical for database connections — you don't want to open
# a new connection on every single invocation
db_host = os.environ.get('DB_HOST')
db_name = os.environ.get('DB_NAME')

# In a real app, you'd initialize your database connection here
# connection = pymysql.connect(host=db_host, ...)

def lambda_handler(event, context):
    """
    The handler runs on every invocation.
    The connection above is reused across warm invocations.
    """
    # Use the connection...
    return {
        'statusCode': 200,
        'body': json.dumps({'message': 'Connected to database'})
    }

Always initialize SDK clients and database connections outside the handler. This code runs once during the cold start and is reused for subsequent warm invocations.
For RDS specifically, use RDS Proxy to manage connection pooling. Lambda can open hundreds of connections simultaneously, which can overwhelm a database.

Part V

Process Streaming Data with Kinesis

Let's build a real-time clickstream processor.

Create a Kinesis Data Stream

Step 01: Open the Kinesis console
Click Create data stream

Data stream name: clickstream
Capacity mode: On-demand

Click Create data stream

✅Green banner: Data stream clickstream successfully created

Step 02: Create the Stream Processor Function
Go to Lambda → Create function
Configure:

Function name: ClickstreamProcessor
Runtime: Python 3.12
Memory: 512 MB
Timeout: 60 seconds

Step 03: Paste this code

import json
import base64
from datetime import datetime

def lambda_handler(event, context):
    """
    Processes clickstream records from Kinesis in near real time.

    Key concepts for the exam:
    - Kinesis data is base64 encoded
    - Records come in batches
    - The partition key determines which shard receives the record
    - Use a high-cardinality partition key (like userId) for even distribution
    """
    processed = 0
    failed = 0

    print(f"Received {len(event['Records'])} records from Kinesis")

    for record in event['Records']:
        try:
            # Kinesis data is base64 encoded — decode it
            raw_data = base64.b64decode(record['kinesis']['data']).decode('utf-8')
            data = json.loads(raw_data)

            # Extract clickstream fields
            user_id = data.get('userId', 'anonymous')
            page = data.get('page', 'unknown')
            action = data.get('action', 'unknown')
            timestamp = data.get('timestamp', datetime.utcnow().isoformat())

            # Process the click event
            print(f"Click: user={user_id}, page={page}, action={action}, time={timestamp}")

            # In a real app, you'd:
            # - Aggregate metrics
            # - Write to DynamoDB or S3
            # - Trigger alerts for specific patterns

            processed += 1

        except Exception as e:
            print(f"Failed to process record: {str(e)}")
            failed += 1

    print(f"Batch complete: {processed} processed, {failed} failed")

    return {
        'statusCode': 200,
        'body': json.dumps({
            'processed': processed,
            'failed': failed
        })
    }

Click Deploy

✅Green banner: Successfully updated the function "ClickstreamProcessor".

Step 04: Add Permissions
The Lambda execution role needs permission to read from Kinesis:
Go to Configuration → Permissions → click the role name
Add permissions → Attach policies
Search for and attach AmazonKinesisReadOnlyAccess

Step 05: Connect Kinesis to Lambda
In the ClickstreamProcessor function, click Add trigger

Step 06: Add trigger
Select a source ▼: Kinesis

Kinesis stream: clickstream
Batch size: 100
Starting position: Latest
Batch window: 5
On-failure destination:
Retry attempts: 3
Split batch on error: ✔ enabled
Concurrent batches per shard: 10

Click Add

✅Green banner: The trigger clickstream was successfully added to function ClickstreamProcessor.

Key Kinesis + Lambda settings:

Batch size: how many records per invocation (max 10,000)

Batch window: how long to wait to fill the batch (max 300s)

Parallelization factor: (1–10) process multiple batches per shard concurrently

Bisect batch on error: splits the batch in half on failure to isolate the bad record

Starting position: LATEST (new records only) or TRIM_HORIZON (from the beginning)

Step 07: Test with Sample Data
Send test records to the stream:
Open the Kinesis console → click clickstream

Step 08: clickstream
Click Data viewer tab
Or use the AWS CLI to send test data:

aws kinesis put-record \
  --stream-name clickstream \
  --partition-key "USER-001" \
  --data '{"userId":"USER-001","page":"/products/laptop","action":"view","timestamp":"2026-04-24T10:00:00Z"}'

Step 09: Check the ClickstreamProcessor CloudWatch logs to see the processed records

Part VI

Performance Tuning

Memory vs Duration Experiment

Let's see how memory affects performance:

Step 01: Open the ClickstreamProcessor function

Step 02: Configuration → General configuration → Edit
Set Memory to 128 MB → Save

Step 03: Run a test → note the Duration and Max Memory Used in the execution results

{
  "Records": [
    {
      "kinesis": {
        "data": "eyJ1c2VySWQiOiJVU0VSLTAwMSIsInBhZ2UiOiIvcHJvZHVjdHMvbGFwdG9wIiwiYWN0aW9uIjoidmlldyIsInRpbWVzdGFtcCI6IjIwMjYtMDQtMjRUMTA6MDA6MDBaIn0="
      },
      "eventSource": "aws:kinesis",
      "eventSourceARN": "arn:aws:kinesis:us-east-1:123456789012:stream/clickstream"
    },
    {
      "kinesis": {
        "data": "eyJ1c2VySWQiOiJVU0VSLTAwMiIsInBhZ2UiOiIvY2hlY2tvdXQiLCJhY3Rpb24iOiJjbGljayIsInRpbWVzdGFtcCI6IjIwMjYtMDQtMjRUMTA6MDE6MDBaIn0="
      },
      "eventSource": "aws:kinesis",
      "eventSourceARN": "arn:aws:kinesis:us-east-1:123456789012:stream/clickstream"
    }
  ]
}

Step 04: Change memory to 256 MB → test again

Step 05: Change to 512 MB → test again

Step 06: Change to 1024 MB → test again

You'll typically see:

Memory	Duration	Billed Duration	Cost per Invocation
128 MB	~200ms	200ms	Low per-ms, but slow
256 MB	~110ms	110ms	Sweet spot for many functions
512 MB	~60ms	60ms	Faster, slightly more per-ms
1024 MB	~55ms	55ms	Diminishing returns

The sweet spot is where increasing memory no longer significantly reduces duration.

Step 07: Concurrency Settings
Configuration → Concurrency and recursion detection
Concurrency→ Edit

Step 08: Edit concurrency

USe unreserved account concurrency: Uses the shared regional pool (default 1000)
Reserve concurrency: Guarantees capacity AND caps the function

Select Reserve concurrency: 50

Click Save

✅Green banner: Your changes have been saved.

This means:

Your function is guaranteed 50 concurrent executions
It can never exceed 50 (acts as a throttle)
The remaining 950 are available for other functions

Reserved concurrency is free and serves two purposes: guaranteeing capacity AND protecting downstream services from being overwhelmed. Provisioned concurrency costs money even when idle but eliminates cold starts.

🏗️ What You Built | 📘Exam Concepts Recap

What You Did	Exam Concept
Created a Lambda function and configured memory/timeout	Lambda configuration parameters
Built and attached a Lambda Layer	Sharing code across functions, layer structure and limits
Set up a DLQ and Lambda Destinations	Async error handling, event lifecycle
Used `--invocation-type Event` to trigger async flow	Synchronous vs asynchronous invocation types
Attached Lambda to a VPC with subnets and security group	VPC access for private resources (RDS, ElastiCache)
Initialized SDK clients outside the handler	Connection reuse, cold start optimization
Created a Kinesis stream and connected it to Lambda	Real-time data processing, event source mappings
Configured batch size, batch window, parallelization factor	Kinesis + Lambda tuning for throughput
Enabled bisect batch on error	Isolating bad records in stream processing
Changed memory settings and compared Duration	Performance tuning, memory = CPU relationship
Set reserved concurrency	Throttling, capacity guarantees, protecting downstream services

⚠️ Clean Up Protocol

1. Lambda → Delete DataProcessor, ClickstreamProcessor
2. Lambda Layers → Delete shared-utils
3. Kinesis → Delete clickstream stream
4. SQS → Delete data-processor-dlq, data-processor-success
5. IAM → Delete the Lambda execution roles
6. CloudWatch → Delete the log groups

Key Takeaways

Memory = CPU: more memory means more CPU. Find the sweet spot where cost and performance balance.
Initialize outside the handler: SDK clients, DB connections, config loading. Reused across warm invocations.
Lambda Destinations > DLQ: Destinations capture success AND failure with more context.
ReportBatchItemFailures for SQS, BisectBatchOnFunctionError for Kinesis to isolate bad records.
VPC Lambda loses internet: needs NAT Gateway or VPC endpoints for AWS services.
RDS Proxy for Lambda-to-RDS: manages connection pooling.
Layers: up to 5 per function, 250 MB total unzipped, versioned and immutable.
15-minute timeout is the max: for longer tasks, use Step Functions or ECS.
Provisioned concurrency: eliminates cold starts but costs money when idle.
Kinesis parallelization factor" (1–10) lets you process multiple batches per shard.

Additional Resources

🏗️