DEV Community: AlpeshKumbhare

CI/CD on AWS: Deployment Strategies, Pipeline Architectures, and the CodePipeline vs GitHub Actions Decision

AlpeshKumbhare — Tue, 18 Aug 2026 06:03:55 +0000

Shipping code to production should be boring. If your deployments are stressful, your pipeline is wrong. A well-designed CI/CD pipeline makes releases routine, reversible, and observable — whether you deploy once a day or fifty times.

AWS provides both native CI/CD services (CodePipeline, CodeBuild, CodeDeploy) and deep integration with third-party tools (GitHub Actions, GitLab CI). The choice depends on your team's workflow preferences and how much you want to stay inside the AWS ecosystem.

This guide covers the deployment strategies that matter, the pipeline architectures that work, and the decision framework for choosing your CI/CD stack.

Deployment Strategies: How Code Reaches Production

Strategy 1: Rolling Deployment

How it works: Replace instances/tasks gradually. Old and new versions run simultaneously during the transition.

Time 0:  [v1] [v1] [v1] [v1]  (4 tasks, all v1)
Time 1:  [v2] [v1] [v1] [v1]  (1 updated)
Time 2:  [v2] [v2] [v1] [v1]  (2 updated)
Time 3:  [v2] [v2] [v2] [v1]  (3 updated)
Time 4:  [v2] [v2] [v2] [v2]  (complete)

Pros: Simple, no extra infrastructure, works everywhere.
Cons: Two versions run simultaneously (must be backward-compatible). Rollback means rolling forward to v1 again.

AWS implementation: ECS rolling update (default), EKS rolling update, EC2 Auto Scaling group instance refresh.

Strategy 2: Blue/Green Deployment

How it works: Deploy new version (green) alongside old version (blue). Switch traffic atomically. Keep blue alive for instant rollback.

┌─────────────────┐         ┌─────────────────┐
│   BLUE (v1)     │         │   GREEN (v2)    │
│   (serving)     │         │   (staged)      │
└────────┬────────┘         └────────┬────────┘
         │                           │
         ▼                           ▼
┌─────────────────────────────────────────────┐
│              ALB / Route53                    │
│         100% → Blue (until cutover)          │
│         Then: 100% → Green                   │
└─────────────────────────────────────────────┘

Pros: Zero downtime. Instant rollback (switch traffic back to blue). Full testing on green before cutover.
Cons: 2x infrastructure during deployment (cost). Database schema changes need careful handling.

AWS implementation:

ECS native blue/green (GA July 2025) — built into ECS service, no CodeDeploy needed
CodeDeploy blue/green for ECS (original approach)
Route53 weighted routing (for broader blue/green)
ALB target group swap

Strategy 3: Canary Deployment

How it works: Route a small percentage of traffic (1-10%) to the new version. Monitor errors. If healthy, gradually shift more traffic. If unhealthy, route all traffic back to old version.

Step 1:  95% → v1,  5% → v2  (canary)
Step 2:  70% → v1, 30% → v2  (expanding)
Step 3:  50% → v1, 50% → v2  (halfway)
Step 4:   0% → v1, 100% → v2 (complete)

Pros: Minimal blast radius. Real production traffic validates new version. Automatic rollback on alarm.
Cons: Complex routing. Must handle session affinity. Slower than blue/green.

AWS implementation:

ECS native canary/linear (GA October 2025) — percentage-based traffic shifting built into ECS
CodeDeploy canary (Lambda, ECS)
App Mesh / VPC Lattice (weighted routing between versions)
ALB weighted target groups

Strategy 4: Feature Flags (Decouple Deploy from Release)

How it works: Deploy code with new features disabled. Enable features independently via configuration (not deployment). Rollback = toggle flag off.

AWS implementation: AppConfig feature flags (native), LaunchDarkly, or custom DynamoDB-backed flags.

Benefit: Deploy anytime. Release (enable feature) separately. Different features for different users (A/B testing).

ECS Native Deployment Capabilities (2025-2026)

ECS received major deployment upgrades — eliminating the need for CodeDeploy in most container scenarios:

Feature	Release	What It Does
Native blue/green	July 2025	Built-in blue/green without CodeDeploy
Linear/canary	October 2025	Percentage-based traffic shifting
NLB support for linear/canary	February 2026	Canary for TCP/gRPC workloads
Pause/continue	May 2026	Pause deployment for manual validation
Configurable circuit breaker	July 2026	Custom failure thresholds for auto-rollback

ECS Deployment Configuration

{
  "deploymentConfiguration": {
    "deploymentType": "BLUE_GREEN",
    "blueGreenDeploymentConfiguration": {
      "trafficRoutingConfig": {
        "type": "CANARY",
        "canaryConfig": {
          "interval": 300,
          "percentage": 10
        }
      },
      "terminationWaitTimeInMinutes": 60,
      "deploymentCircuitBreaker": {
        "enable": true,
        "rollback": true,
        "failureThreshold": 5
      }
    }
  }
}

This deploys with 10% canary, waits 5 minutes, then shifts remaining traffic. If 5+ tasks fail health checks, automatic rollback triggers.

Pipeline Architecture: AWS-Native Stack

AWS CodePipeline + CodeBuild + CodeDeploy

┌──────────┐     ┌───────────┐     ┌───────────┐     ┌───────────┐
│  Source   │────→│   Build   │────→│   Test    │────→│  Deploy   │
│(CodeCommit│     │(CodeBuild)│     │(CodeBuild)│     │(CodeDeploy│
│ or GitHub)│     │           │     │           │     │ or ECS)   │
└──────────┘     └───────────┘     └───────────┘     └───────────┘

CodePipeline: Orchestrates the pipeline stages. Triggers on source change.
CodeBuild: Runs build commands, tests, security scans in managed containers.
CodeDeploy: Handles deployment strategies (rolling, blue/green, canary).

Pipeline Stages Best Practice

Source → Build → Unit Test → SAST Scan → Container Scan → 
  Deploy Dev → Integration Test → Deploy Staging → 
    Load Test → Manual Approval → Deploy Production

Multi-Account Pipeline Pattern

Tooling Account (pipeline lives here)
  │
  ├── Deploy → Dev Account (automatic)
  ├── Deploy → Staging Account (automatic + integration tests)
  └── Deploy → Production Account (manual approval gate)

Cross-account deployment uses IAM roles. Pipeline in tooling account assumes role in target account to deploy.

Pipeline Architecture: GitHub Actions

For teams using GitHub as source control, GitHub Actions provides a complete CI/CD solution with AWS integration:

name: Deploy to ECS
on:
  push:
    branches: [main]

jobs:
  deploy:
    runs-on: ubuntu-latest
    permissions:
      id-token: write  # OIDC for AWS auth
    steps:
      - uses: actions/checkout@v4

      - uses: aws-actions/configure-aws-credentials@v4
        with:
          role-to-assume: arn:aws:iam::123456789:role/github-actions-deploy
          aws-region: eu-west-1

      - uses: aws-actions/amazon-ecr-login@v2

      - name: Build and push image
        run: |
          docker build -t $ECR_REGISTRY/my-app:$GITHUB_SHA .
          docker push $ECR_REGISTRY/my-app:$GITHUB_SHA

      - uses: aws-actions/amazon-ecs-deploy-task-definition@v2
        with:
          task-definition: task-def.json
          service: my-service
          cluster: production
          wait-for-service-stability: true

GitHub Actions + OIDC (No Long-Lived Credentials)

Use OpenID Connect (OIDC) to authenticate GitHub Actions to AWS — no access keys needed:

Create IAM Identity Provider for GitHub in AWS
Create IAM role with trust policy for specific repo/branch
GitHub Actions exchanges OIDC token for temporary AWS credentials

Security: Credentials are short-lived (1 hour), scoped to specific repos, and never stored as secrets.

CodePipeline vs GitHub Actions vs GitLab CI

Criteria	CodePipeline	GitHub Actions	GitLab CI
Source integration	CodeCommit, GitHub, S3, ECR	GitHub (native)	GitLab (native)
Build	CodeBuild (managed)	Hosted runners or self-hosted	Shared or self-hosted runners
AWS integration	Deep (native IAM, VPC, cross-account)	Good (via aws-actions, OIDC)	Good (via CLI, OIDC)
Deployment strategies	CodeDeploy (full support)	Manual (scripts + AWS CLI)	Manual (scripts + AWS CLI)
Pricing	Free pipeline + CodeBuild minutes ($0.005/min)	2000 free min/month, then $0.008/min	400 free min/month, then $0.005/min
Ecosystem	AWS-only	20K+ marketplace actions	500+ templates
Multi-cloud	❌	✅	✅
Pipeline as code	YAML or console	YAML (`.github/workflows/`)	YAML (`.gitlab-ci.yml`)
Approval gates	Manual approval action	Environment protection rules	Manual jobs
Best for	AWS-native teams, complex deployment strategies	Teams on GitHub, multi-cloud	Teams on GitLab, self-hosted preference

When to Choose What

CodePipeline: You want native blue/green/canary via CodeDeploy, cross-account deployment patterns, or deep AWS integration without custom scripting.
GitHub Actions: Your code lives on GitHub, team prefers GitHub's ecosystem, and you want multi-cloud flexibility.
GitLab CI: Your code lives on GitLab, you want self-hosted runners, or you need built-in security scanning (SAST/DAST).

Security in the Pipeline

Shift-Left Security

Stage	Tool	What It Catches
Pre-commit	git-secrets, talisman	Hardcoded credentials before they enter repo
Build	SAST (CodeGuru, Snyk, Semgrep)	Code vulnerabilities (SQL injection, XSS)
Build	SCA (Dependabot, Snyk)	Vulnerable dependencies
Container Build	ECR image scanning, Trivy	Container CVEs
Pre-Deploy	IAM policy validation (IAM Access Analyzer)	Over-privileged roles
Post-Deploy	DAST (OWASP ZAP)	Runtime vulnerabilities

Pipeline Security Best Practices

No long-lived credentials — use OIDC (GitHub) or IAM roles (CodeBuild)
Least privilege — pipeline role can only deploy to specific services/accounts
Artifact signing — sign container images (cosign / Notation) to ensure integrity
Immutable artifacts — tag images with git SHA, never overwrite :latest in production
Approval gates — require human approval before production deployment
Audit trail — CloudTrail logs all deployment actions

Deployment Observability

A deployment isn't done when the pipeline turns green. Monitor after deploy:

Post-Deploy Validation

Deploy v2 → Wait 5 min → Check:
  ├── Error rate increased? → Rollback
  ├── Latency p99 > threshold? → Rollback
  ├── Health check failures? → Rollback
  └── All green → Deployment successful

CloudWatch Alarms as Deployment Gates

Configure CodeDeploy / ECS circuit breaker to monitor CloudWatch alarms:

If alarm triggers during canary/linear deployment → automatic rollback
No human intervention needed for obvious failures

Common Pipeline Anti-Patterns

Anti-Pattern	Problem	Fix
No staging environment	Bugs found in production	Always deploy to staging first
Tests only in CI, not in deployment	Broken integration passes build	Run integration tests post-deploy
Manual deployment to production	Error-prone, unauditable	Automate everything, gate with approval
Same pipeline for all environments	No quality gates between stages	Multi-stage with promotion (dev → staging → prod)
`:latest` tag in production	Can't tell which version is running	Use git SHA or semantic version tags
Secrets in pipeline code	Credential exposure	Use OIDC, Secrets Manager, or Parameter Store
No rollback plan	Stuck with broken deployment	Blue/green or canary with auto-rollback

Summary

CI/CD on AWS comes down to three decisions:

Deployment strategy — Rolling (simple), Blue/Green (zero-downtime), Canary (lowest risk), or Feature Flags (decouple deploy from release)
Pipeline tool — CodePipeline (AWS-native, deep integration), GitHub Actions (flexible, multi-cloud), or GitLab CI (self-hosted, built-in security)
Safety mechanisms — Approval gates, CloudWatch alarm-based rollback, immutable artifacts, shift-left security scanning

The 2026 default for containers: ECS native blue/green with canary traffic shifting + configurable circuit breaker. No CodeDeploy needed for standard ECS workloads.

The 2026 default for pipeline: GitHub Actions with OIDC auth for most teams. CodePipeline when you need native cross-account patterns or CodeDeploy's advanced strategies.

Alpesh Kumbhare is an AWS Architect at Atos, specializing in AWS infrastructure automation and DevOps practices. Connect on LinkedIn.

AWS Database Selection Guide: RDS vs Aurora vs DynamoDB vs ElastiCache — Choosing the Right Database for Every Workload

AlpeshKumbhare — Mon, 17 Aug 2026 05:54:18 +0000

AWS offers 15+ managed database services. Each is purpose-built for specific data models and access patterns. Choosing wrong means either fighting the database's design (performance problems) or over-engineering a simple workload (cost problems).

The key insight: start with your access patterns, not the database. How will data be read and written? What queries are critical? What's the read/write ratio? The answers point directly to the right service.

This guide covers every AWS database service, organized by data model, with a decision framework that gets you to the right choice fast.

The AWS Database Landscape

┌─────────────────────────────────────────────────────────────────────┐
│                       AWS DATABASE SERVICES                           │
├─────────────────────────────────────────────────────────────────────┤
│  RELATIONAL (SQL)                                                    │
│  RDS (MySQL, PostgreSQL, Oracle, SQL Server, MariaDB) | Aurora       │
├─────────────────────────────────────────────────────────────────────┤
│  KEY-VALUE / DOCUMENT (NoSQL)                                        │
│  DynamoDB                                                            │
├─────────────────────────────────────────────────────────────────────┤
│  IN-MEMORY (Caching / Real-Time)                                     │
│  ElastiCache (Redis/Valkey, Memcached) | MemoryDB for Redis          │
├─────────────────────────────────────────────────────────────────────┤
│  GRAPH                                                               │
│  Neptune                                                             │
├─────────────────────────────────────────────────────────────────────┤
│  TIME SERIES                                                         │
│  Timestream                                                          │
├─────────────────────────────────────────────────────────────────────┤
│  WIDE COLUMN                                                         │
│  Keyspaces (Cassandra-compatible)                                    │
├─────────────────────────────────────────────────────────────────────┤
│  SEARCH                                                              │
│  OpenSearch Service                                                  │
├─────────────────────────────────────────────────────────────────────┤
│  LEDGER                                                              │
│  QLDB (Quantum Ledger Database)                                      │
├─────────────────────────────────────────────────────────────────────┤
│  DATA WAREHOUSE                                                      │
│  Redshift                                                            │
├─────────────────────────────────────────────────────────────────────┤
│  VECTOR (AI/ML)                                                      │
│  Aurora (pgvector) | OpenSearch | MemoryDB | Neptune Analytics        │
└─────────────────────────────────────────────────────────────────────┘

The Decision Flowchart

START
│
├── Need SQL, joins, transactions, complex queries?
│     ├── High performance, auto-scaling, global? → Aurora
│     └── Standard workload, specific engine (Oracle/SQL Server)? → RDS
│
├── Need key-value or simple document access at any scale?
│     └── DynamoDB
│
├── Need microsecond latency caching?
│     ├── Cache-aside pattern (volatile)? → ElastiCache
│     └── Need durability (primary datastore)? → MemoryDB
│
├── Need relationship traversal (social graph, fraud, recommendations)?
│     └── Neptune
│
├── Need time-series data (IoT, metrics, logs)?
│     └── Timestream
│
├── Need full-text search, log analytics?
│     └── OpenSearch
│
├── Need analytics on petabytes (BI, reporting)?
│     └── Redshift
│
├── Need vector similarity search (AI/RAG)?
│     └── Aurora pgvector / OpenSearch / MemoryDB
│
└── Need immutable, cryptographically verifiable ledger?
      └── QLDB

Relational Databases: RDS vs Aurora

Amazon RDS

What: Managed relational database for MySQL, PostgreSQL, MariaDB, Oracle, SQL Server.

AWS manages: Patching, backups, Multi-AZ failover, monitoring.
You manage: Instance sizing, parameter tuning, schema design, query optimization.

Key features:

Multi-AZ: synchronous standby for HA (automatic failover ~60 seconds)
Read Replicas: up to 15 (async replication) for read scaling
Automated backups: point-in-time recovery (35-day window)
Storage auto-scaling: up to 64 TiB

Amazon Aurora

What: AWS-rebuilt MySQL/PostgreSQL with cloud-native storage architecture. Compatible at the wire-protocol level (drop-in replacement).

Why Aurora over RDS:

Feature	RDS	Aurora
Performance	1x (standard engine)	3-5x MySQL, 3x PostgreSQL
Storage	EBS (single AZ)	Distributed across 3 AZs (6 copies)
Storage limit	64 TiB	128 TiB (auto-grows)
Failover	~60 seconds	~30 seconds (with Aurora Replicas)
Read replicas	Up to 15 (async)	Up to 15 (same storage, <10ms lag)
Backtrack	❌	✅ (rewind to point in time without restore)
Global Database	❌	✅ (<1 second cross-region replication)
Serverless	❌	✅ (Aurora Serverless v2 — scales to zero)
Multi-master	❌	✅ (Aurora DSQL — distributed SQL, new)

When to Choose RDS vs Aurora

Choose RDS when...	Choose Aurora when...
Need Oracle or SQL Server	MySQL or PostgreSQL workload
Budget-constrained (RDS is cheaper for small instances)	Need high availability (<30s failover)
Simple, low-traffic application	Read-heavy workload (leverage replicas on shared storage)
Specific engine version required	Need global database (cross-region DR)
	Need serverless (variable/unpredictable traffic)
	Need to scale storage beyond 64 TiB

Aurora Serverless v2

Scales compute automatically (0.5 to 256 ACUs) based on demand. You pay per ACU-hour consumed.

Best for: Dev/test environments, variable workloads, new applications with unknown traffic patterns.

Not for: Steady high-traffic production (provisioned instances are cheaper when utilization is consistently high).

DynamoDB: NoSQL at Any Scale

What: Fully managed key-value and document database. Single-digit millisecond performance at any scale.

When to Use DynamoDB

Key-value or simple document access patterns
Extreme scale (millions of requests/second)
Predictable, consistent latency requirements
Serverless architecture (scales to zero with on-demand mode)
Global applications (Global Tables for multi-region active-active)

When NOT to Use DynamoDB

Complex queries with joins, aggregations, ad-hoc SQL
Data model that requires normalization and relational integrity
Analytics/reporting (use Redshift or Athena instead)
Small dataset with complex query needs (Aurora is simpler)

DynamoDB Key Design Decisions

Decision	Options
Capacity mode	On-Demand (unpredictable traffic, pay per request) vs Provisioned (predictable, cheaper at scale)
Primary key	Partition key only (unique lookup) vs Partition + Sort key (range queries within partition)
Global Tables	Multi-region active-active replication (near-zero RPO)
DAX	In-memory cache for DynamoDB (microsecond reads for hot data)
Streams	Capture changes for event-driven processing (CDC)

DynamoDB Pricing Reality

On-Demand: $1.25 per million write requests, $0.25 per million reads
Provisioned: ~$0.00065 per WCU/hour, ~$0.00013 per RCU/hour

Storage: $0.25/GB/month

Example: 10M reads + 1M writes per day
On-Demand: ~$10/day = $300/month
Provisioned: ~$120/month (60% cheaper at consistent load)

ElastiCache vs MemoryDB: Caching and Real-Time

ElastiCache (Redis/Valkey or Memcached)

Use for: Caching layer in front of databases. Session storage. Leaderboards. Real-time analytics.

Feature	Redis/Valkey	Memcached
Data structures	Rich (strings, hashes, lists, sets, sorted sets)	Simple key-value only
Persistence	Optional (snapshot + AOF)	None (volatile)
Replication	Yes (read replicas, Multi-AZ)	No replication
Pub/Sub	Yes	No
Cluster mode	Yes (horizontal sharding)	Yes (simple sharding)
Use case	Primary cache + data structures	Simple caching only

MemoryDB for Redis

What: Redis-compatible, durable, in-memory database. Unlike ElastiCache (which is a cache), MemoryDB is a primary database that happens to be in-memory.

Key difference: MemoryDB writes to a distributed transaction log before acknowledging — data survives node failures. ElastiCache can lose data on failover.

Use MemoryDB when: You need microsecond reads AND durability (session store as source of truth, real-time user profiles, gaming state).

Purpose-Built Databases

Amazon Neptune (Graph)

Use for: Relationship-heavy data where traversals are the primary query pattern.

Social networks (friend-of-friend queries)
Fraud detection (link analysis)
Recommendation engines (collaborative filtering)
Knowledge graphs (entity relationships)
Network topology (impact analysis)

NOT for: Simple lookups, transactional data, analytics. If you're not doing multi-hop traversals, you don't need a graph database.

Amazon Timestream (Time Series)

Use for: Time-stamped data that's primarily queried by time range.

IoT sensor data
Application/infrastructure metrics
Financial market data (ticks)
Fleet/device telemetry

Why over DynamoDB: Built-in time-based retention tiers (hot → cold → archive), time-series functions (interpolation, smoothing, aggregation), and automatic data lifecycle management.

Amazon Keyspaces (Cassandra)

Use for: Teams with existing Cassandra workloads wanting managed service. Wide-column data model for write-heavy workloads with predictable access patterns.

Amazon OpenSearch (Search + Analytics)

Use for: Full-text search, log analytics, application search, observability data.

Powers CloudWatch Log Insights behind the scenes
Kibana/OpenSearch Dashboards for visualization
Vector search for AI/RAG applications

Amazon QLDB (Ledger)

Use for: Immutable, cryptographically verifiable transaction history.

Financial transactions requiring audit trail
Supply chain provenance tracking
Regulatory compliance (tamper-proof records)

Amazon Redshift (Data Warehouse)

Use for: Analytical queries on large datasets (petabytes). BI reporting, data warehousing, complex aggregations across millions of rows.

NOT for: OLTP (transactional workloads) — use Aurora or DynamoDB instead.

Vector Databases for AI/RAG

With generative AI, vector similarity search is a new access pattern. AWS options:

Service	Vector Capability	Best For
Aurora PostgreSQL (pgvector)	Extension on existing Aurora	Teams already on Aurora, moderate scale
OpenSearch	k-NN plugin	Large-scale similarity search + hybrid text/vector
MemoryDB	Vector search support	Ultra-low latency vector retrieval
Neptune Analytics	Graph + vector	Knowledge graph with semantic search
Bedrock Knowledge Base	Managed (any backend)	Easiest path — managed vector storage

Multi-Database Architecture Patterns

Pattern 1: CQRS (Command Query Responsibility Segregation)

Writes → Aurora (source of truth, transactions)
Reads → ElastiCache (cached hot data) + DynamoDB (materialized views)

Pattern 2: Event Sourcing + Materialized Views

Events → DynamoDB (event store) → Streams → Lambda → 
  ├── OpenSearch (search index)
  ├── ElastiCache (real-time aggregations)
  └── Redshift (analytics)

Pattern 3: Polyglot Persistence

User Profiles → DynamoDB (key-value, high scale)
Product Catalog → Aurora (relational, complex queries)
Recommendations → Neptune (graph traversal)
Session State → ElastiCache (microsecond, volatile)
Search → OpenSearch (full-text)
Analytics → Redshift (OLAP, BI)

Common Mistakes

Mistake	Problem	Fix
DynamoDB for ad-hoc SQL queries	Expensive scans, poor performance	Use Aurora or Athena
Aurora for simple key-value access	Over-engineered, slower than DynamoDB	Use DynamoDB
ElastiCache as primary datastore	Data loss on failover	Use MemoryDB if durability needed
Single database for everything	One size fits none	Polyglot persistence (right DB per access pattern)
Redshift for OLTP	Terrible latency for single-row reads	Use Aurora or DynamoDB
Ignoring connection pooling (Aurora)	Connection exhaustion under load	Use RDS Proxy
DynamoDB without understanding key design	Hot partitions, throttling	Design partition key for even distribution

Cost Comparison (Typical Web App)

Service	Starting Cost (dev)	Production Estimate
RDS db.t4g.medium	~$50/month	~$200-500/month (Multi-AZ)
Aurora Serverless v2 (min 0.5 ACU)	~$45/month	Scales with usage
Aurora Provisioned (db.r6g.large)	~$180/month	~$400-800/month (Multi-AZ + replicas)
DynamoDB (On-Demand, light)	~$5-25/month	Scales per request
ElastiCache (cache.t4g.small)	~$25/month	~$100-300/month (Multi-AZ)
OpenSearch (t3.small.search)	~$35/month	~$200-500/month (Multi-AZ)

Summary

AWS database selection is driven by access patterns, not features:

Access Pattern	Database
SQL, joins, transactions, complex queries	Aurora (or RDS for Oracle/SQL Server)
Key-value lookups at massive scale	DynamoDB
Microsecond caching (volatile)	ElastiCache
Microsecond reads (durable)	MemoryDB
Relationship traversal (graph)	Neptune
Time-series data	Timestream
Full-text search	OpenSearch
Analytical queries (BI)	Redshift
Vector similarity (AI/RAG)	Aurora pgvector or OpenSearch
Immutable audit ledger	QLDB

The guiding principle: One database rarely fits all access patterns. Use purpose-built databases for each pattern (polyglot persistence), connected by event-driven synchronization.

Alpesh Kumbhare is an AWS Architect at Atos, specializing in AWS data architecture and cloud infrastructure automation. Connect on LinkedIn.

Event-Driven Architecture on AWS: SQS vs SNS vs EventBridge vs Kinesis — Patterns, Anti-Patterns, and When to Use What

AlpeshKumbhare — Fri, 14 Aug 2026 05:38:42 +0000

Event-driven architecture (EDA) is how modern distributed systems communicate at scale. Instead of services calling each other directly (tight coupling), services emit events ("this happened") and interested consumers react independently.

On AWS, four messaging services form the EDA backbone — but they solve different problems. Choosing wrong means either over-engineering a simple notification into a Kinesis stream, or under-engineering a high-throughput data pipeline onto SQS.

This guide maps each service to its sweet spot, covers the integration patterns that work in production, and highlights the anti-patterns that waste money and create operational pain.

The Four Messaging Services

┌──────────────────────────────────────────────────────────────────────┐
│                    AWS MESSAGING LANDSCAPE                             │
├──────────────┬──────────────┬───────────────────┬────────────────────┤
│     SQS      │     SNS      │   EventBridge     │     Kinesis        │
│              │              │                   │                    │
│  Queue       │  Pub/Sub     │  Event Bus        │  Stream            │
│  (1:1)       │  (1:many)    │  (content-route)  │  (ordered, replay) │
│              │              │                   │                    │
│  Decouple    │  Fan-out     │  Route + Filter   │  Real-time data    │
│  + buffer    │  broadcast   │  + transform      │  high throughput   │
└──────────────┴──────────────┴───────────────────┴────────────────────┘

Quick Decision Matrix

If you need...	Use...
Decouple producer/consumer, buffer load	SQS
Send one event to many subscribers	SNS
Route events based on content/attributes	EventBridge
Receive events from SaaS (Stripe, Auth0, Shopify)	EventBridge
Process ordered, replayable data stream	Kinesis Data Streams
High-throughput ingestion (100K+ events/sec)	Kinesis
Fan-out + per-consumer buffering	SNS → SQS (combined)
Transform/enrich events between services	EventBridge Pipes
Schedule future events (cron)	EventBridge Scheduler

Amazon SQS: The Queue

What it is: Point-to-point message queue. One producer puts messages, one consumer processes them. Messages are buffered until consumed.

Two Flavors

Feature	Standard Queue	FIFO Queue
Ordering	Best-effort (may reorder)	Strict FIFO guaranteed
Deduplication	At-least-once (may duplicate)	Exactly-once processing
Throughput	Unlimited	3,000 msg/sec (with batching: 30,000)
Use case	High throughput, order doesn't matter	Financial transactions, command sequences

When to Use SQS

Load leveling — smooth bursty traffic (e.g., API receives 10K requests/sec, worker processes at 1K/sec)
Decoupling — producer doesn't need to know about consumer (or if it's running)
Retry/DLQ — failed messages automatically route to Dead Letter Queue for investigation
Batch processing — Lambda polls SQS, processes in batches of up to 10

SQS Architecture Pattern: Work Queue

┌──────────┐     ┌──────────┐     ┌──────────────┐
│  API GW  │────→│   SQS    │────→│ Lambda / ECS │
│  (burst) │     │  (buffer)│     │  (steady)    │
└──────────┘     └──────────┘     └──────────────┘
                       │
                       ▼ (after 3 failures)
                 ┌──────────┐
                 │   DLQ    │
                 └──────────┘

Amazon SNS: Pub/Sub Fan-Out

What it is: Publish-subscribe messaging. One publisher sends to a topic, multiple subscribers receive copies.

Subscriber Types

SQS queues (most common — adds buffering per consumer)
Lambda functions (direct invocation)
HTTP/HTTPS endpoints (webhooks)
Email / SMS (notifications)
Kinesis Data Firehose (streaming delivery)
Mobile push (iOS/Android)

When to Use SNS

Fan-out — one event needs to trigger multiple independent actions
Notifications — email alerts, SMS, mobile push
Decoupled fan-out — SNS → multiple SQS queues (each consumer has its own queue)

SNS + SQS: The Fan-Out Pattern

                         ┌──── SQS (Email Service) ──→ Lambda: send email
                         │
Producer ──→ SNS Topic ──┼──── SQS (Analytics) ──→ Lambda: track metrics
                         │
                         └──── SQS (Inventory) ──→ Lambda: update stock

Each consumer gets its own SQS queue with independent retry and DLQ. One consumer's failure doesn't affect others.

Amazon EventBridge: The Event Router

What it is: Serverless event bus with content-based routing. Events flow in, rules match patterns, and events route to targets — all without code.

Why EventBridge Over SNS

Feature	SNS	EventBridge
Filtering	Basic attribute filtering	Rich content-based rules (any JSON field)
Schema	None	Schema registry + discovery
SaaS integration	No	30+ SaaS partners (Stripe, Auth0, Zendesk)
Archive & replay	No	Yes (replay events from any point in time)
Cross-account	Complex	Native (event bus sharing)
Transforms	No	Input transformer (reshape events before delivery)
Pipes	No	EventBridge Pipes (filter → enrich → transform → deliver)
Pricing	Per publish + delivery	Per event ingested ($1/million)

When to Use EventBridge

Content-based routing — route events based on any field in the event body
SaaS events — receive events from third-party services without polling
Cross-account event sharing — centralized event bus for multi-account architectures
Event replay — archive events and replay when debugging or reprocessing
Schema enforcement — discover and validate event schemas automatically

EventBridge Rule Example

{
  "source": ["com.myapp.orders"],
  "detail-type": ["OrderCreated"],
  "detail": {
    "amount": [{"numeric": [">", 1000]}],
    "region": ["eu-west-1", "eu-central-1"]
  }
}

This rule matches only: orders > $1000 from EU regions. Everything else is ignored. No code needed.

EventBridge Architecture Pattern: Event Mesh

┌─────────────┐        ┌─────────────────┐        ┌─────────────┐
│ Order Service│──emit──→│  EventBridge    │──rule──→│ Payment Svc │
└─────────────┘        │  (Central Bus)  │        └─────────────┘
                        │                 │
┌─────────────┐        │  Rules:         │        ┌─────────────┐
│ Auth0 (SaaS)│──emit──→│  • OrderCreated │──rule──→│Warehouse Svc│
└─────────────┘        │  • UserSignedUp │        └─────────────┘
                        │  • PaymentFailed│
┌─────────────┐        │                 │        ┌─────────────┐
│ Stripe(SaaS)│──emit──→│                 │──rule──→│ Notification│
└─────────────┘        └─────────────────┘        └─────────────┘

Amazon Kinesis: The Data Stream

What it is: Real-time data streaming for high-throughput, ordered, replayable event processing.

Kinesis Family

Service	Purpose
Kinesis Data Streams	Custom real-time stream processing (you control consumers)
Kinesis Data Firehose	Managed delivery to S3, Redshift, OpenSearch (zero code)
Kinesis Data Analytics	SQL/Flink on streaming data (real-time analytics)

When to Use Kinesis Over SQS/EventBridge

Requirement	SQS/EventBridge	Kinesis
Ordering guarantee	FIFO SQS (limited)	Per-shard ordering (scalable)
Event replay	EventBridge archive	Native (24h-365d retention)
Multiple consumers on same stream	❌ (SNS fan-out)	✅ (multiple consumers, each at own position)
Throughput	Millions/sec (SQS)	1MB/sec per shard (scale shards)
Real-time analytics	Not designed for	Built for this
Use case	Application events, notifications	IoT telemetry, clickstream, logs, financial ticks

Kinesis Architecture Pattern: Real-Time Analytics

┌────────────┐     ┌─────────────┐     ┌───────────────────┐
│ IoT Devices│────→│  Kinesis     │────→│ Lambda (real-time) │
│ Clickstream│     │  Data Stream │     │ Anomaly detection  │
│ App Logs   │     │  (ordered)   │     └───────────────────┘
└────────────┘     └──────┬──────┘
                          │
                          ├────→ Firehose → S3 (data lake)
                          └────→ Flink (windowed aggregations)

EventBridge Pipes: Connect + Transform

EventBridge Pipes (launched 2023) connects sources to targets with optional filtering, enrichment, and transformation — without writing Lambda glue code.

Source → Filter → Enrich → Transform → Target

Example:
SQS Queue → filter (only "critical") → Lambda (add metadata) → reshape JSON → EventBridge Bus

Supported Sources & Targets

Sources: SQS, Kinesis, DynamoDB Streams, Kafka (MSK), Self-Managed Kafka

Targets: Lambda, Step Functions, ECS Task, EventBridge Bus, API Gateway, SQS, SNS, Kinesis, and more

When to Use Pipes vs Lambda

Scenario	Pipes	Lambda
Filter + route (no business logic)	✅ No code	Overkill
Simple field transformation	✅ Input transformer	Overkill
Complex business logic	❌	✅ Needed
Enrich from external API	✅ (enrichment step)	Also works

EventBridge Scheduler

For time-based events, EventBridge Scheduler replaces CloudWatch Events (cron):

One-time schedules — "send reminder email in 48 hours"
Recurring schedules — "run cleanup every day at 2 AM"
Rate-based — "trigger every 5 minutes"
Timezone-aware — handles DST correctly (CloudWatch Events doesn't)
At-scale — millions of individual schedules (one per user/order/entity)

// Schedule a one-time future event
{
  "ScheduleExpression": "at(2026-08-20T14:00:00)",
  "Target": {
    "Arn": "arn:aws:lambda:...:send-reminder",
    "Input": "{\"orderId\": \"order-123\", \"action\": \"follow-up\"}"
  }
}

Integration Patterns

Pattern 1: Command Queue (SQS)

Use for: async task processing, work distribution

API → SQS → Worker (Lambda/ECS)

Pattern 2: Fan-Out (SNS → SQS)

Use for: one event, multiple independent reactions

Service → SNS → SQS(A) → Consumer A
                → SQS(B) → Consumer B
                → SQS(C) → Consumer C

Pattern 3: Event Router (EventBridge)

Use for: content-based routing, SaaS integration, cross-account

Services → EventBridge Bus → Rules → Targets (Lambda, SQS, Step Functions)

Pattern 4: Streaming Pipeline (Kinesis)

Use for: real-time ordered data, multiple consumers, replay

Producers → Kinesis → Consumer A (real-time alerts)
                    → Consumer B (S3 archive via Firehose)
                    → Consumer C (analytics via Flink)

Pattern 5: Choreography (EventBridge, cross-service)

Use for: loosely coupled microservice workflows

Order Created → (event) → Payment Service reacts
Payment Succeeded → (event) → Shipping Service reacts
Shipping Completed → (event) → Notification Service reacts

No orchestrator. Each service reacts independently to relevant events.

Pattern 6: Orchestration + Events (Step Functions + EventBridge)

Use for: complex workflows with visibility + event-driven triggers

EventBridge detects event → triggers Step Function →
Step Function orchestrates multi-step workflow →
emits completion event back to EventBridge

Anti-Patterns to Avoid

Anti-Pattern	Problem	Fix
Using Kinesis for simple notifications	Expensive, complex for low throughput	SQS or SNS
SQS for fan-out (multiple consumers)	Only one consumer per message	SNS → SQS pattern
EventBridge for 100K+ events/sec	Soft limits, cost adds up	Kinesis for high throughput
Synchronous chains disguised as events	Hidden coupling, hard to debug	True async with DLQs
No Dead Letter Queue	Lost messages on failure	Always configure DLQ
Giant event payloads (>256KB)	Exceeds size limits	Store payload in S3, pass reference
No event schema/contract	Breaking consumers on format change	EventBridge Schema Registry
Ignoring idempotency	Duplicate processing causes data corruption	Design consumers to be idempotent

Cost Comparison

Service	Pricing Model	Cost at 10M events/month
SQS Standard	$0.40/million requests	~$4
SQS FIFO	$0.50/million requests	~$5
SNS	$0.50/million publishes + delivery costs	~$5-15
EventBridge	$1.00/million events	~$10
Kinesis (1 shard)	$0.015/hr + $0.014/million PUT	~$15
Kinesis (10 shards)	$0.15/hr + $0.014/million PUT	~$110

Takeaway: SQS is cheapest for simple queuing. EventBridge costs 2.5x SQS but provides routing, filtering, schema, archive, and replay. Kinesis is the most expensive but provides ordering, replay, and real-time stream processing.

Summary

Event-driven architecture on AWS comes down to four services with distinct strengths:

SQS — queue + buffer. Decouple producer and consumer. Simple, cheap, reliable.
SNS — fan-out. One event to many subscribers. Combine with SQS for durability.
EventBridge — intelligent routing. Content-based filtering, SaaS integration, archive/replay, cross-account. The "default choice" for new EDA designs.
Kinesis — ordered stream. High throughput, multiple consumers, real-time processing. Use for data pipelines, not application events.

The default starting point in 2026: EventBridge for routing + SQS for buffering. Add Kinesis only when you need ordering, replay at volume, or real-time analytics. Use SNS when you need mobile push or email notifications.

Alpesh Kumbhare is an AWS Architect at Atos, specializing in AWS infrastructure automation and event-driven cloud architecture. Connect on LinkedIn.

AWS Observability in 2026: The Complete Stack — CloudWatch, X-Ray, OpenTelemetry, and Application Signals

AlpeshKumbhare — Thu, 13 Aug 2026 09:11:41 +0000

You can't fix what you can't see. As systems grow more distributed — microservices, serverless functions, event-driven flows — the gap between "deployed" and "observable" becomes the difference between resolving incidents in minutes versus hours.

AWS's observability stack has evolved significantly. X-Ray SDK entered maintenance mode in February 2026, replaced by OpenTelemetry. Application Signals introduced SLO-based monitoring. CloudWatch absorbed capabilities that previously required third-party tools.

This post covers the complete AWS observability architecture for modern applications — what to collect, where to send it, and how to build alerting that actually works.

The Three Pillars of Observability

┌─────────────┐    ┌─────────────┐    ┌─────────────┐
│   METRICS   │    │    LOGS     │    │   TRACES    │
│             │    │             │    │             │
│ What's      │    │ What        │    │ Where in    │
│ happening?  │    │ happened?   │    │ the path?   │
│             │    │             │    │             │
│ CPU, memory │    │ Error msgs  │    │ Request     │
│ request rate│    │ debug info  │    │ flow across │
│ error count │    │ audit trail │    │ services    │
│ latency p99 │    │             │    │             │
└─────────────┘    └─────────────┘    └─────────────┘
      │                   │                   │
      └───────────────────┼───────────────────┘
                          │
                    ┌─────────────┐
                    │ CORRELATION │
                    │ Trace ID →  │
                    │ connects    │
                    │ all three   │
                    └─────────────┘

Metrics tell you WHAT is wrong (error rate spiked).
Logs tell you WHY it's wrong (NullPointerException at line 42).
Traces tell you WHERE in the request path it went wrong (payment-service → database timeout).

All three need a common correlation key (trace ID) to be useful together.

The AWS Observability Stack

┌─────────────────────────────────────────────────────────────────┐
│  COLLECTION (Instrumentation)                                    │
│  ADOT (OpenTelemetry) | CloudWatch Agent | SDKs                  │
├─────────────────────────────────────────────────────────────────┤
│  PROCESSING & STORAGE                                            │
│  CloudWatch (Metrics + Logs) | X-Ray (Traces) | S3 (Archive)    │
├─────────────────────────────────────────────────────────────────┤
│  ANALYSIS                                                        │
│  Log Insights | Metrics Insights | Trace Analytics | ServiceLens │
├─────────────────────────────────────────────────────────────────┤
│  SLO & ALERTING                                                  │
│  Application Signals | CloudWatch Alarms | Composite Alarms      │
├─────────────────────────────────────────────────────────────────┤
│  VISUALIZATION                                                   │
│  CloudWatch Dashboards | ServiceLens Map | Managed Grafana        │
└─────────────────────────────────────────────────────────────────┘

Metrics: CloudWatch Metrics + EMF

Default Metrics (Free)

AWS services automatically emit metrics to CloudWatch:

EC2: CPU, network, disk, status checks
ECS/Fargate: CPU, memory utilization per task
Lambda: invocations, duration, errors, throttles, cold starts
ALB: request count, latency, HTTP 4xx/5xx, healthy host count
RDS: connections, IOPS, replication lag, free storage

Custom Metrics

For application-specific metrics, use Embedded Metric Format (EMF):

{
  "_aws": {
    "Timestamp": 1692286800000,
    "CloudWatchMetrics": [{
      "Namespace": "MyApp/Orders",
      "Dimensions": [["Service", "Environment"]],
      "Metrics": [
        {"Name": "OrdersProcessed", "Unit": "Count"},
        {"Name": "ProcessingTime", "Unit": "Milliseconds"}
      ]
    }]
  },
  "Service": "order-service",
  "Environment": "production",
  "OrdersProcessed": 47,
  "ProcessingTime": 230
}

Why EMF over PutMetricData: EMF lets you log structured JSON that CloudWatch automatically extracts as metrics AND preserves as log entries — one write, two outputs. No PutMetricData API calls (cheaper, lower latency).

Container Insights

For ECS and EKS, Container Insights provides:

Per-container CPU and memory
Per-pod/task network I/O
Cluster-level resource utilization
Kubernetes-aware dimensions (namespace, deployment, pod)

Enable with: ADOT collector as DaemonSet (EKS) or sidecar (ECS).

Logs: CloudWatch Logs + Log Insights

Structured Logging (Essential)

Unstructured logs are unsearchable at scale. Always log structured JSON:

{
  "timestamp": "2026-08-13T10:30:00Z",
  "level": "ERROR",
  "service": "payment-service",
  "traceId": "1-abc123-def456",
  "requestId": "req-789",
  "message": "Payment failed",
  "error": "TimeoutException",
  "customerId": "cust-123",
  "amount": 99.99,
  "duration_ms": 5002
}

Log Insights Queries

CloudWatch Log Insights provides SQL-like querying across log groups:

# Find slowest requests in the last hour
fields @timestamp, service, duration_ms, requestId
| filter duration_ms > 1000
| sort duration_ms desc
| limit 20

# Error rate per service
filter level = "ERROR"
| stats count(*) as errors by service
| sort errors desc

# Trace specific request across services
filter traceId = "1-abc123-def456"
| sort @timestamp asc

Log Architecture Patterns

Pattern	When
Direct to CloudWatch	Lambda (automatic), ECS (awslogs driver), EKS (Fluent Bit)
CloudWatch → S3	Long-term retention, compliance archives (subscription filter)
CloudWatch → OpenSearch	Need full-text search, complex aggregations, Kibana dashboards
ADOT → CloudWatch	OpenTelemetry-based collection with CloudWatch backend

Retention and Cost

Set retention policies per log group (don't default to "never expire")
Production: 30-90 days in CloudWatch, archive to S3 Glacier after
Dev/staging: 7-14 days (no archival)
Infrequent Access class: 50% cheaper for logs you rarely query

Traces: X-Ray + OpenTelemetry (ADOT)

The Migration: X-Ray SDK → OpenTelemetry

Important: X-Ray SDK entered maintenance mode in February 2026. AWS now recommends OpenTelemetry for all new instrumentation.

Old Approach	New Approach
X-Ray SDK (language-specific)	AWS Distro for OpenTelemetry (ADOT)
Proprietary trace format	OpenTelemetry (OTel) standard
AWS-only export	Export to X-Ray, Jaeger, Zipkin, Grafana Tempo, etc.
Auto-instrumentation (limited)	Auto-instrumentation (comprehensive)

ADOT Architecture

┌────────────┐     ┌──────────────┐     ┌─────────────┐
│Application │────→│ ADOT Collector│────→│  X-Ray      │
│(OTel SDK)  │     │ (sidecar or  │     │  CloudWatch │
│            │     │  DaemonSet)  │     │  Prometheus │
└────────────┘     └──────────────┘     └─────────────┘

ADOT Collector receives traces/metrics via OTLP protocol and exports to one or more backends. You can send to X-Ray AND Grafana Tempo simultaneously.

Auto-Instrumentation (Zero Code Changes)

For Java, Python, Node.js, and .NET — ADOT auto-instrumentation captures:

HTTP requests (incoming and outgoing)
Database queries (SQL, DynamoDB, Redis)
AWS SDK calls (S3, SQS, SNS, Lambda invocations)
gRPC calls

EKS: Deploy ADOT auto-instrumentation as a Kubernetes operator — injects instrumentation into pods automatically.

Lambda: Enable Lambda X-Ray Active Tracing — one toggle, zero code.

Trace Anatomy

Trace: 1-abc123-def456 (entire request lifecycle)
│
├── Span: API Gateway (12ms)
│     └── Span: Lambda: order-handler (450ms)
│           ├── Span: DynamoDB: GetItem (23ms)
│           ├── Span: HTTP: payment-service (380ms)
│           │     └── Span: RDS: INSERT (45ms)  ← SLOW?
│           └── Span: SQS: SendMessage (15ms)

Each span shows: service name, duration, status, metadata. Find the bottleneck instantly.

Application Signals: SLO-Based Monitoring

Application Signals (GA 2025) is CloudWatch's answer to "monitor what matters to users" — automatically tracks SLIs (Service Level Indicators) and lets you define SLOs (Service Level Objectives).

What It Auto-Discovers

Without any configuration, Application Signals detects:

Services and their dependencies (service map)
Call volume between services
Latency (p50, p90, p99)
Error rate and fault rate
Availability

Defining SLOs

SLO: "Payment Service Availability"
├── SLI: Success rate (HTTP 2xx / total requests)
├── Target: 99.9% over 30-day rolling window
├── Error budget: 0.1% (43 minutes/month of allowed errors)
└── Alert: When burn rate exceeds 10x normal → page on-call

Why SLOs > Threshold Alarms

Traditional Alarm	SLO-Based Alert
"CPU > 80%" → alert	"Error budget burning too fast" → alert
Noisy, often false positive	Only fires when users are impacted
Measures infrastructure health	Measures user experience
Doesn't account for context	Accounts for error budget remaining

Alerting That Works

Alarm Strategy

Layer	What to Alert On	Action
SLO breach	Error budget burn rate > threshold	Page on-call immediately
Service health	Error rate > 5% for 5 minutes	Page on-call
Saturation	CPU > 85%, memory > 90%, disk > 80%	Auto-scale + notify
Dependencies	Upstream latency > SLA	Notify (not page)
Business metrics	Orders/min drops > 50%	Alert business + engineering

Composite Alarms

Reduce noise by combining related alarms:

CompositeAlarm: "Payment Service Degraded"
├── AND: Error rate > 5%
├── AND: Latency p99 > 2000ms
└── AND: NOT in maintenance window

→ Only fires when BOTH error rate AND latency are degraded
→ Eliminates false positives from single-metric spikes

Alert Routing

CloudWatch Alarm → SNS Topic → Multiple targets:
├── PagerDuty/Opsgenie (critical: pages on-call)
├── Slack channel (warning: notification only)
├── Lambda (auto-remediation for known issues)
└── ITSM (ServiceNow incident creation)

Dashboards: What to Show

Per-Service Dashboard (Auto-Generated with Application Signals)

Request rate (rpm)
Error rate (%)
Latency (p50, p90, p99)
Dependency health (downstream services)
Recent deployments (correlate changes with metrics)

Platform Dashboard (SRE/Platform Team)

SLO status across all services (green/yellow/red)
Error budget remaining per service
Top 5 highest-latency services
Recent alerts and resolution time
Deployment frequency and failure rate

Cost Dashboard (FinOps)

Compute spend trend (CloudWatch billing metric)
Data transfer costs by service
Lambda invocations/cost correlation
Over-provisioned resources (Compute Optimizer findings)

Observability for Different Architectures

Serverless (Lambda + API Gateway + DynamoDB)

Pillar	Tool	Setup
Metrics	CloudWatch (auto) + EMF for custom	Lambda Powertools library
Logs	CloudWatch Logs (auto)	Structured JSON, set retention
Traces	X-Ray Active Tracing (one toggle)	Zero code
SLOs	Application Signals	Auto-discovers Lambda services

Containers (ECS/EKS + ALB + RDS)

Pillar	Tool	Setup
Metrics	Container Insights + EMF	ADOT collector as sidecar/DaemonSet
Logs	CloudWatch (awslogs/Fluent Bit)	Structured JSON, per-container log groups
Traces	ADOT (OpenTelemetry)	Auto-instrumentation operator (EKS)
SLOs	Application Signals	Service map auto-discovery

Hybrid (Containers + Lambda + Step Functions)

Pillar	Tool	Setup
Metrics	Container Insights + Lambda metrics	Unified CloudWatch namespace
Logs	CloudWatch Logs (all services)	Common trace ID in all log entries
Traces	ADOT + X-Ray Active Tracing	Trace propagation across Lambda→ECS→SQS
SLOs	Application Signals	End-to-end service map

Cost Optimization for Observability

Observability itself can be expensive. Control costs:

Lever	Savings
Log retention policies (don't keep forever)	50-80% on log storage
Infrequent Access log class	50% for rarely-queried logs
EMF instead of PutMetricData API	Avoid per-metric API charges
Sampling traces (e.g., 10% of requests)	90% trace storage savings
Metric filters instead of full log queries	Reduce Log Insights costs
Archive to S3 after 30 days	CloudWatch → S3 Glacier (95% cheaper)

Summary

AWS observability in 2026 centers on three shifts:

X-Ray SDK → OpenTelemetry (ADOT) — industry-standard instrumentation, multi-backend export, auto-instrumentation for zero code changes
Threshold alarms → SLO-based alerting — Application Signals measures what users experience, not just infrastructure health
Separate tools → unified correlation — trace IDs connect metrics, logs, and traces so you move from "something's wrong" to "here's the failing span" in seconds

The architecture: Instrument with ADOT (OTel), store in CloudWatch + X-Ray, analyze with Log Insights + Trace Analytics, alert on SLOs via Application Signals, visualize in CloudWatch Dashboards or Managed Grafana.

Start with auto-instrumentation and structured logs. Add custom metrics and SLOs as your services mature. Don't over-instrument day one — observability should grow with your system's complexity.

Alpesh Kumbhare is an AWS Architect at Atos, specializing in AWS observability, infrastructure automation, and cloud architecture. Connect on LinkedIn.

Terraform vs AWS CDK vs CloudFormation: The Definitive IaC Decision Guide for 2026

AlpeshKumbhare — Wed, 12 Aug 2026 07:35:58 +0000

Every AWS team eventually asks: "Should we use Terraform, CDK, or CloudFormation?" The answer isn't universal — it depends on your team's skills, organizational constraints, multi-cloud requirements, and how you want to manage infrastructure lifecycle.

This guide compares all three across the dimensions that actually matter in production, with a decision framework to help you choose.

The Three Models at a Glance

┌───────────────────────────────────────────────────────────────────┐
│                    INFRASTRUCTURE AS CODE                           │
├───────────────────┬───────────────────┬───────────────────────────┤
│   CloudFormation  │     AWS CDK       │       Terraform           │
│                   │                   │                           │
│   Declarative     │   Imperative      │   Declarative             │
│   JSON/YAML       │   TypeScript/     │   HCL                    │
│                   │   Python/Java/Go  │                           │
│   AWS-native      │   Synthesizes to  │   Multi-cloud             │
│                   │   CloudFormation   │   (4000+ providers)       │
│   No state file   │   No state file   │   State file required     │
│   (AWS manages)   │   (AWS manages)   │   (you manage)            │
└───────────────────┴───────────────────┴───────────────────────────┘

CloudFormation: The AWS Native

What it is: AWS's own IaC service. Declarative JSON/YAML templates that describe desired state. AWS handles provisioning, ordering, and rollback.

Strengths

Zero setup — no tools to install, no state to manage
Same-day AWS support — new services/features available immediately in CloudFormation
Drift detection — detects when resources deviate from template
Stack operations — create, update, delete as atomic operations with automatic rollback
StackSets — deploy across multiple accounts and regions from one template
Change sets — preview changes before applying
No state file — AWS tracks resource state internally (no S3 backend, no locking concerns)

Weaknesses

Verbose — simple resources require many lines of YAML/JSON
No loops or conditionals (limited) — Conditions and Fn::ForEach are awkward
No abstraction — can't create reusable "classes" of infrastructure
AWS-only — cannot manage non-AWS resources
Slow updates — large stacks take 30+ minutes to update
Error messages — often cryptic, debugging is painful

Best For

Teams deeply committed to AWS with no multi-cloud plans
Organizations using AWS Service Catalog (backed by CloudFormation)
Landing Zone Accelerator (LZA) deployments
Simple, single-account deployments with < 50 resources

AWS CDK: Infrastructure in Real Code

What it is: An open-source framework that lets you define infrastructure using programming languages (TypeScript, Python, Java, Go, C#). CDK synthesizes to CloudFormation — it's an abstraction layer on top.

Strengths

Real programming languages — loops, conditionals, functions, classes, inheritance
Constructs — reusable, composable building blocks (L1, L2, L3)
Type safety — IDE autocomplete, compile-time checks, refactoring support
Abstraction — define a "SecureWebApp" construct once, reuse everywhere
Testing — unit test infrastructure with standard testing frameworks (Jest, pytest)
Same deployment model as CloudFormation — benefits from AWS-managed state, rollback, drift detection
Construct Hub — community library of pre-built patterns (aws-solutions-constructs)

Weaknesses

Still limited by CloudFormation — if CFN can't do it, CDK can't either
Synthesis step — adds complexity to CI/CD pipelines
Learning curve — must know both the programming language AND AWS resource model
Stack size limits — inherits CloudFormation's 500-resource limit per stack
Breaking changes — CDK library updates can break existing constructs (semver issues)
AWS-only — synthesizes to CloudFormation, so no multi-cloud

CDK Construct Levels

Level	Description	Example
L1	Direct CloudFormation mapping (CfnBucket)	1:1 with CFN, verbose
L2	AWS-aware with sensible defaults (Bucket)	Encryption enabled by default, fewer params
L3	Opinionated patterns (ApplicationLoadBalancedFargateService)	Full architecture in one construct

Best For

Developer-heavy teams familiar with TypeScript/Python
Organizations wanting reusable infrastructure libraries (shared constructs)
Complex deployments needing loops, conditionals, and dynamic generation
Teams already using CloudFormation wanting better developer experience
Projects requiring infrastructure unit testing

Terraform: The Multi-Cloud Standard

What it is: HashiCorp's open-source IaC tool using HCL (HashiCorp Configuration Language). Declarative, provider-based architecture supporting 4,000+ providers across all major clouds and SaaS services.

Strengths

Multi-cloud — same workflow for AWS, Azure, GCP, Kubernetes, Datadog, PagerDuty, GitHub, etc.
Mature ecosystem — Terraform Registry with 15,000+ modules
HCL is purpose-built — cleaner than YAML, simpler than full programming languages
Plan before apply — terraform plan shows exactly what will change
Module system — reusable, versioned, composable modules
Import existing resources — bring manually-created resources under management
Fast execution — parallel resource creation, faster than CloudFormation for large deployments
Community — massive community, extensive documentation, Stack Overflow coverage

Weaknesses

State management — YOU manage the state file (S3 + DynamoDB locking is standard)
State drift — if someone changes resources outside Terraform, state diverges
No built-in rollback — failed applies can leave infrastructure in partial state
Provider lag — new AWS features may take days/weeks to appear in the AWS provider
License change — BSL license since 2023 (OpenTofu is the open-source fork)
HCL limitations — no full programming language features (workarounds needed for complex logic)
Sensitive data in state — state file contains secrets (must encrypt S3 backend)

Terraform vs OpenTofu

Since HashiCorp's BSL license change in 2023, OpenTofu exists as a fully open-source fork:

Criteria	Terraform	OpenTofu
License	BSL (Business Source License)	MPL 2.0 (truly open source)
Feature parity	Leading edge	Follows (slight lag)
Provider support	Full	Full (same providers)
Enterprise support	Terraform Cloud/Enterprise	Community + vendors
When to choose	Need Terraform Cloud features	Need fully open-source

Best For

Multi-cloud or hybrid environments
Platform teams managing infrastructure across multiple providers
Organizations with existing Terraform expertise
Projects needing to manage non-AWS resources (GitHub repos, DNS, monitoring, SaaS configs)
Large-scale deployments where CloudFormation is too slow

Head-to-Head Comparison

Criteria	CloudFormation	CDK	Terraform
Language	JSON/YAML	TypeScript/Python/Java/Go	HCL
Paradigm	Declarative	Imperative (synthesizes to declarative)	Declarative
Multi-cloud	❌ AWS only	❌ AWS only	✅ 4000+ providers
State management	AWS-managed	AWS-managed (via CFN)	Self-managed (S3 + DynamoDB)
Rollback	✅ Automatic	✅ Automatic (via CFN)	❌ Manual
New AWS features	Same-day	Same-day (via L1)	Days/weeks lag
Reusability	Nested stacks (limited)	Constructs (excellent)	Modules (excellent)
Testing	❌ No native support	✅ Unit + integration	✅ Terratest, plan assertions
Learning curve	Low (if you know YAML)	Medium (language + AWS)	Low-Medium (HCL)
IDE support	Basic YAML validation	Full (autocomplete, types)	Good (HCL extension)
Drift detection	✅ Built-in	✅ Built-in (via CFN)	`terraform plan` (manual)
Import existing	✅ (limited)	✅ (limited)	✅ (excellent)
Community modules	Limited	Construct Hub (growing)	Registry (massive, 15K+)
CI/CD integration	AWS-native (CodePipeline)	CDK Pipelines	Any CI tool
Cost	Free	Free	Free (OSS) / Paid (Cloud)

State Management: The Critical Difference

CloudFormation / CDK: No State File Worries

AWS tracks state internally. You never see a state file. Benefits:

No S3 backend to configure
No locking mechanism needed
No state corruption risk
Drift detection built-in

Terraform: You Own the State

terraform {
  backend "s3" {
    bucket         = "my-terraform-state"
    key            = "prod/network/terraform.tfstate"
    region         = "eu-west-1"
    dynamodb_table = "terraform-locks"
    encrypt        = true
  }
}

State risks you must manage:

State file contains secrets (encrypt the S3 bucket, restrict access)
Concurrent applies can corrupt state (DynamoDB locking solves this)
Lost state = Terraform doesn't know what it manages (backup state files)
State drift = someone changed resources outside Terraform (regular terraform plan detects this)

Module / Construct Ecosystem

Terraform Registry

module "vpc" {
  source  = "terraform-aws-modules/vpc/aws"
  version = "5.0.0"

  cidr = "10.0.0.0/16"
  azs  = ["eu-west-1a", "eu-west-1b", "eu-west-1c"]
  private_subnets = ["10.0.1.0/24", "10.0.2.0/24", "10.0.3.0/24"]
}

15,000+ community modules. The terraform-aws-modules organization alone covers VPC, EKS, RDS, Lambda, and dozens more.

CDK Constructs

import { ApplicationLoadBalancedFargateService } from 'aws-cdk-lib/aws-ecs-patterns';

new ApplicationLoadBalancedFargateService(this, 'MyApp', {
  taskImageOptions: { image: ecs.ContainerImage.fromRegistry('my-app:latest') },
  desiredCount: 3,
  cpu: 512,
  memoryLimitMiB: 1024,
});
// This single construct creates: ECS cluster, Fargate service, ALB, target group,
// security groups, IAM roles, CloudWatch log group — all with sensible defaults

Team Workflow Patterns

Pattern 1: Terraform for Platform, CDK for Applications

Platform Team → Terraform
  ├── VPCs, Transit Gateway, DNS
  ├── EKS clusters, RDS instances
  └── Shared infrastructure (S3, KMS, IAM boundaries)

Application Teams → CDK
  ├── ECS services, Lambda functions
  ├── Application-specific resources
  └── Deploy via CDK Pipelines

Why: Platform team needs multi-provider support (AWS + GitHub + Datadog). App teams benefit from CDK's developer-friendly constructs.

Pattern 2: Terraform Everywhere

All Teams → Terraform
  ├── Shared modules in internal registry
  ├── Atlantis or Terraform Cloud for PR-based workflow
  └── One language, one workflow, one state backend

Why: Consistency. One tool to learn, one CI/CD pattern, one troubleshooting approach.

Pattern 3: CDK Everywhere

All Teams → CDK (TypeScript)
  ├── Shared construct library (internal npm package)
  ├── CDK Pipelines for deployment
  └── Jest for infrastructure testing

Why: Developer-first organization where infrastructure is code written by application developers.

The Decision Flowchart

START
  │
  ├── Do you need to manage non-AWS resources?
  │     └── YES → Terraform (multi-provider)
  │
  ├── Is your team primarily developers (TypeScript/Python)?
  │     ├── YES → CDK (familiar language, constructs, testing)
  │     └── NO → Continue ↓
  │
  ├── Do you need multi-cloud portability?
  │     └── YES → Terraform
  │
  ├── Is simplicity the priority (small team, few resources)?
  │     └── YES → CloudFormation (no tools to manage)
  │
  ├── Do you need reusable infrastructure libraries?
  │     ├── Developer org → CDK (constructs)
  │     └── Ops/platform org → Terraform (modules)
  │
  └── Already using one tool successfully?
        └── YES → Stay with it (switching cost > marginal benefit)

Migration Considerations

From	To	Effort
CloudFormation → CDK	Low	CDK can import existing CFN stacks. Migrate incrementally.
CloudFormation → Terraform	Medium	Use `terraform import`. Re-create templates in HCL.
Terraform → CDK	High	No migration path. Must re-create and import.
CDK → Terraform	High	No migration path. Must re-create and import.

Key rule: Don't migrate for marginal gains. Only migrate when the current tool is actively blocking you (e.g., multi-cloud requirement, team can't hire CloudFormation skills).

What About Pulumi / CDKTF / Crossplane?

Tool	Niche
Pulumi	Like CDK but multi-cloud. Real languages + any cloud provider. Consider if you want CDK-style + multi-cloud.
CDKTF (CDK for Terraform)	Write CDK-style code that synthesizes to Terraform HCL. Best of both worlds — but adds complexity.
Crossplane	Kubernetes-native IaC. Manages cloud resources via K8s CRDs. For teams running everything on Kubernetes.

Summary

Choose...	When...
CloudFormation	AWS-only, simple deployments, no state management overhead, same-day feature support
CDK	Developer teams, need reusable constructs, want type safety and testing, AWS-only is fine
Terraform	Multi-cloud, platform teams, large module ecosystem, existing HCL expertise, non-AWS resources

The "wrong" choice isn't which tool you pick — it's switching tools every 6 months because someone read a blog post. Pick one, standardize, and build expertise. The best IaC tool is the one your team uses consistently.

Alpesh Kumbhare is an AWS Architect at Atos, specializing in AWS infrastructure automation and IaC best practices. Connect on LinkedIn.

AWS Compute Decision Guide: EC2 vs ECS vs EKS vs Fargate vs Lambda — When to Use What

AlpeshKumbhare — Tue, 11 Aug 2026 06:59:43 +0000

The most expensive mistake in cloud isn't over-provisioning a server — it's choosing the wrong compute model for your workload. Running event-driven functions on EKS? Over-engineered. Running a 24/7 API with 10-second cold starts on Lambda? Under-engineered. Both cost you money, time, and pain.

AWS offers six primary compute options, and they're not interchangeable. This guide provides a decision framework based on workload characteristics — not vendor hype.

The Compute Spectrum

MORE CONTROL                                              LESS CONTROL
MORE RESPONSIBILITY                                       LESS RESPONSIBILITY

┌──────┐   ┌──────┐   ┌──────┐   ┌──────┐   ┌──────┐   ┌──────┐
│  EC2  │   │ECS on│   │EKS on│   │Fargate│   │App   │   │Lambda│
│       │   │ EC2  │   │ EC2  │   │      │   │Runner│   │      │
│You own│   │You   │   │You   │   │No    │   │Just  │   │Just  │
│every- │   │manage│   │manage│   │servers│   │deploy│   │code  │
│thing  │   │nodes │   │nodes │   │      │   │image │   │      │
└──────┘   └──────┘   └──────┘   └──────┘   └──────┘   └──────┘

Quick Decision Matrix

If your workload is...	Use...	Why
Event-driven, < 15 min, bursty	Lambda	Pay per invocation, scales to zero
HTTP API, predictable traffic	ECS + Fargate or App Runner	Simplest container path
Microservices, 5-15 services	ECS + Fargate	AWS-native, simple, low ops
Microservices, 15+ services, CNCF ecosystem	EKS	Kubernetes portability + ecosystem
GPU workloads, ML training	EC2 (or SageMaker)	GPU instance control
Legacy apps, specific OS/kernel needs	EC2	Full machine control
Batch processing, parallel compute	AWS Batch (on Fargate or EC2)	Managed job scheduling
Real-time data streaming	ECS/EKS with persistent connections	Long-lived processes

EC2: Full Control, Full Responsibility

Choose EC2 when:

You need specific instance types (GPU, FPGA, bare-metal)
Workload requires kernel customization or specific OS configurations
Running licensed software tied to machine/CPU
Need persistent local storage (instance store NVMe)
Maximum performance with minimal abstraction

Don't choose EC2 when:

You just need to run containers (use Fargate instead)
Workload is event-driven and bursty (Lambda is cheaper)
You don't want to manage patching, AMIs, and Auto Scaling groups

EC2 Cost Optimization

Strategy	Savings
Graviton (ARM) instances	20-40% better price/performance
Spot Instances	Up to 90% for fault-tolerant workloads
Savings Plans	Up to 72% for steady-state
Right-sizing (Compute Optimizer)	20-30% by eliminating over-provisioning
Scheduled stop/start (dev/staging)	~65% on non-production

ECS (Elastic Container Service): AWS-Native Containers

Choose ECS when:

You want the simplest container orchestration on AWS
Your team doesn't know (or need) Kubernetes
5-15 microservices, all on AWS
You want deep AWS integration (IAM task roles, Service Connect, CloudMap)
No multi-cloud requirement

ECS on Fargate vs ECS on EC2

Criteria	ECS + Fargate	ECS + EC2
Server management	None	You manage ASG + AMIs
Cost at scale	Higher per-vCPU	Lower (especially with Spot/RIs)
Startup time	~30 seconds	Instant (containers already on host)
GPU support	Limited	Full GPU instance access
Bin packing	AWS handles it	You optimize placement
Best for	Variable workloads, small-medium scale	High-scale, cost-sensitive, GPU

ECS Architecture Pattern

┌─────────────┐     ┌──────────────┐     ┌──────────────┐
│    ALB      │────→│  ECS Service │────→│    RDS       │
│             │     │  (Fargate)   │     │  (Aurora)    │
└─────────────┘     │  3 tasks     │     └──────────────┘
                    └──────────────┘
                           │
                    ┌──────────────┐
                    │  Service     │
                    │  Connect     │──→ Other ECS Services
                    └──────────────┘

EKS (Elastic Kubernetes Service): The Kubernetes Path

Choose EKS when:

Team already knows Kubernetes
You need multi-cloud portability (same manifests on GKE/AKS)
15+ microservices benefiting from the CNCF ecosystem
Need advanced traffic management (Istio, Linkerd)
Want Karpenter for intelligent auto-scaling
Need custom operators or CRDs for your domain

Don't choose EKS when:

Your team doesn't know Kubernetes (learning curve is steep)
< 10 services (ECS is simpler and cheaper)
You're 100% AWS with no multi-cloud plans

EKS Cost: The $73/Month Control Plane

EKS charges $0.10/hour ($73/month) for the control plane. Add node compute on top. This makes EKS expensive for small workloads but negligible at scale.

EKS Compute Options

Option	Best For	Cost Model
Managed Node Groups	Standard workloads, simple ops	EC2 instance pricing
Karpenter	Dynamic scaling, mixed instance types	Right-sized EC2 + Spot
Fargate	Isolation-sensitive, no node management	Per-pod vCPU/memory pricing
Spot + Karpenter	Cost-optimized batch/stateless	Up to 90% cheaper than On-Demand

EKS Architecture Pattern

┌──────────────────────────────────────────────────┐
│  EKS Cluster                                      │
│  ┌─────────────┐  ┌─────────────┐  ┌──────────┐ │
│  │ Namespace:  │  │ Namespace:  │  │Namespace:│ │
│  │ team-a      │  │ team-b      │  │ platform │ │
│  │ ┌─────────┐ │  │ ┌─────────┐ │  │ ArgoCD   │ │
│  │ │ pods    │ │  │ │ pods    │ │  │ Prometheus│ │
│  │ └─────────┘ │  │ └─────────┘ │  │ Grafana  │ │
│  └─────────────┘  └─────────────┘  └──────────┘ │
│                                                    │
│  Karpenter → Spot + On-Demand (Graviton)          │
└──────────────────────────────────────────────────┘

Fargate: Serverless Containers

Choose Fargate when:

You want containers without managing servers
Workloads are variable (scale up/down frequently)
Security requires task-level isolation (each task gets its own kernel)
Team doesn't want to manage EC2 instances, ASGs, or AMI patching

Don't choose Fargate when:

Cost is the primary concern at high scale (EC2 is 30-50% cheaper)
Need GPU instances
Need > 4 vCPU / 30 GB memory per task (Fargate has size limits)
Need daemonsets or host-level access (use EC2 nodes)

Fargate Pricing Reality

Fargate is ~3x more expensive per vCPU-hour than equivalent EC2 On-Demand. But you save on:

No idle capacity (pay only for running tasks)
No ops time managing nodes
No AMI patching or security updates on hosts

Break-even: If utilization stays above ~60-70%, EC2 is cheaper. Below that, Fargate wins because you're not paying for idle capacity.

Lambda: Serverless Functions

Choose Lambda when:

Event-driven processing (S3 events, SQS messages, API calls)
Execution time < 15 minutes
Bursty, unpredictable traffic patterns
You want zero infrastructure management
Cost must scale to zero when idle

Don't choose Lambda when:

Functions run > 15 minutes
Need persistent connections (WebSockets, gRPC streams)
Cold starts are unacceptable (use provisioned concurrency or Fargate)
Workload is high-throughput steady-state (EC2/Fargate is cheaper)
Need > 10 GB memory

Lambda Cost Calculation

Cost = (Requests × $0.20/1M) + (GB-seconds × $0.0000166667)

Example: 1M requests/month, 256MB, 200ms average
= $0.20 + (1M × 0.256GB × 0.2s × $0.0000166667)
= $0.20 + $0.85
= $1.05/month

Same workload on Fargate (0.25 vCPU, 512MB, always running):
= $9.47/month (vCPU) + $1.04/month (memory) = $10.51/month

Lambda is 10x cheaper for bursty workloads. But at 10M+ steady requests, Fargate wins.

App Runner: The Simplest Container Path

Choose App Runner when:

You want the fastest path from container image to production URL
Simple web applications or APIs
Don't need VPC integration (or use VPC connector for private resources)
Team has zero AWS infrastructure experience

App Runner vs Fargate:

Feature	App Runner	ECS + Fargate
Setup complexity	Push image → done	Task def + service + ALB + target group
Auto scaling	Built-in, automatic	Configure scaling policies
Custom domains + TLS	One-click	ACM + ALB configuration
VPC access	Via connector	Native
Load balancing	Built-in	You configure ALB
CI/CD	Built-in (from ECR/GitHub)	You configure pipeline
Flexibility	Limited	Full control

The Decision Flowchart

START
  │
  ├── Is it event-driven, < 15 min execution?
  │     └── YES → Lambda
  │
  ├── Is it a simple web app/API needing fastest deployment?
  │     └── YES → App Runner
  │
  ├── Does your team know Kubernetes?
  │     ├── YES + need multi-cloud/CNCF ecosystem → EKS
  │     └── NO → ECS
  │
  ├── Want to manage servers?
  │     ├── NO → Fargate (with ECS or EKS)
  │     └── YES (cost/GPU/custom needs) → EC2 (with ECS or EKS)
  │
  └── Need GPU, bare-metal, or kernel access?
        └── YES → EC2 (raw or with ECS)

Cost Comparison: Same Workload, Different Compute

Workload: Web API, 2 vCPU, 4GB RAM, running 24/7, steady traffic

Compute	Monthly Cost	Notes
EC2 (m7g.large, On-Demand)	~$60	You manage everything
EC2 (m7g.large, 1yr Savings Plan)	~$38	37% savings, committed
ECS + Fargate (2 vCPU, 4GB)	~$120	No server management
EKS + Fargate (2 vCPU, 4GB)	~$193	$73 control plane + Fargate
EKS + EC2 (m7g.large)	~$133	$73 control plane + EC2
App Runner (2 vCPU, 4GB)	~$100	Simplest path
Lambda (if 1M req/month)	~$1	Only if workload fits event model

Takeaway: Lambda is cheapest for bursty workloads. EC2 with Savings Plans is cheapest for steady-state. Fargate/App Runner trade cost for operational simplicity.

Migration Paths

From	To	When
EC2 → Fargate	You're spending too much time on instance management and patching
Lambda → Fargate	Functions hitting 15-min timeout or cold starts are unacceptable
ECS → EKS	Team growing, need CNCF tools, or multi-cloud is on the roadmap
EKS → ECS	Over-engineered; small team drowning in Kubernetes complexity
Fargate → EC2	Scale reached where Fargate premium exceeds ops savings
Monolith EC2 → ECS/EKS	Decomposing into microservices, need orchestration

Summary

There's no "best" AWS compute service — only the best fit for YOUR workload:

Lambda — event-driven, bursty, < 15 min. Cheapest at low scale.
App Runner — simplest container deployment. Zero AWS knowledge needed.
ECS + Fargate — standard container workloads, AWS-native, low ops. Sweet spot for most teams.
EKS — Kubernetes ecosystem, multi-cloud, 15+ services. Higher complexity, more power.
EC2 — full control, GPU, specialized instances. Cheapest at high steady-state utilization.

Choose based on: team skills, workload characteristics, scale, and how much operational overhead you're willing to accept. When in doubt, start with Fargate — you can always move to EC2 underneath later without changing application code.

Alpesh Kumbhare is an AWS Architect at Atos, specializing in AWS infrastructure automation and cloud architecture decisions. Connect on LinkedIn.

Platform Engineering on AWS: Building an Internal Developer Platform That Developers Actually Use

AlpeshKumbhare — Fri, 07 Aug 2026 07:06:23 +0000

Gartner predicted that by 2026, 80% of large software engineering organizations would establish platform engineering teams. That prediction landed. Platform engineering is no longer a trend — it's the default operating model for cloud-native organizations.

But here's what most teams get wrong: they build a platform nobody uses. They create golden paths developers ignore, self-service portals that are harder than Terraform, and governance layers that slow teams down.

This post covers how to build an Internal Developer Platform (IDP) on AWS that developers actually adopt — the architecture, the AWS services that power it, the golden path patterns, and the operating model that sustains it.

What Is Platform Engineering?

Platform engineering is the practice of building and maintaining an Internal Developer Platform — a self-service layer that abstracts infrastructure complexity while enforcing organizational standards.

WITHOUT PLATFORM                    WITH PLATFORM

Developer → learns Terraform        Developer → picks template
         → configures VPC                    → clicks deploy
         → sets up IAM                       → gets environment
         → creates pipeline                  → ships code
         → manages monitoring
         → handles security

Cognitive load: HIGH                 Cognitive load: LOW
Time to first deploy: Weeks          Time to first deploy: Minutes
Consistency: None                    Consistency: Enforced

The goal: Developers get self-service access to production-ready infrastructure without needing to become infrastructure experts.

The IDP Architecture on AWS

┌─────────────────────────────────────────────────────────────────┐
│  DEVELOPER INTERFACE (Portal)                                    │
│  Backstage | Custom Portal | Service Catalog Console             │
├─────────────────────────────────────────────────────────────────┤
│  ORCHESTRATION (Provisioning Engine)                             │
│  Service Catalog | Proton | CDK Pipelines | Crossplane | ArgoCD │
├─────────────────────────────────────────────────────────────────┤
│  TEMPLATES (Golden Paths)                                        │
│  CloudFormation | Terraform | CDK | EKS Blueprints | Cookiecutter│
├─────────────────────────────────────────────────────────────────┤
│  PLATFORM SERVICES (Shared Capabilities)                         │
│  CI/CD | Observability | Security Scanning | Secrets | DNS       │
├─────────────────────────────────────────────────────────────────┤
│  INFRASTRUCTURE (Managed by Platform Team)                       │
│  EKS | ECS | Lambda | RDS | S3 | VPC | IAM | KMS               │
└─────────────────────────────────────────────────────────────────┘

The 5 Capabilities Every IDP Needs

1. Self-Service Infrastructure Provisioning

Developers should deploy environments without filing tickets or waiting for ops teams.

AWS Options:

Service	Best For	Developer Experience
AWS Service Catalog	Governed AWS resource provisioning	Console/API, pre-approved products
AWS Proton	Container and serverless environments	Template-based, managed pipelines
CDK Pipelines	Custom IaC with CI/CD	Code-first, GitOps-style
EKS Blueprints	Kubernetes cluster standardization	Addons + configurations pre-packaged

Service Catalog pattern:

Platform Team creates "Products":
├── "Web Application" → VPC + ALB + ECS + RDS + CloudWatch
├── "Data Pipeline" → S3 + Glue + Step Functions + Athena
├── "API Backend" → API Gateway + Lambda + DynamoDB
└── "ML Workspace" → SageMaker Studio + S3 + IAM role

Developer: Launches product → fills 5 parameters → gets full stack in 10 minutes

2. Golden Paths (Opinionated Templates)

Golden paths are pre-built, production-ready templates that encode best practices. They're not mandatory — but they're so much easier than building from scratch that teams naturally adopt them.

What makes a golden path "golden":

Security pre-configured (IAM roles, encryption, network isolation)
Observability built-in (CloudWatch dashboards, alarms, X-Ray tracing)
CI/CD included (pipeline deploys on merge)
Cost-optimized (right-sized, auto-scaling configured)
Compliance-ready (tagging, logging, Config rules)

Example golden path: "Microservice on ECS"

Developer runs: `platform create service --type=ecs-microservice --name=payment-api`

Platform creates:
├── ECR repository
├── ECS service + task definition (Fargate, Graviton)
├── ALB target group + listener rule
├── CodePipeline (source → build → deploy)
├── CloudWatch dashboard + alarms
├── IAM task role (least privilege)
├── Security group (ingress from ALB only)
├── X-Ray tracing enabled
└── Resource tags (team, cost-center, environment)

Developer gets a working, production-grade service in minutes. No Terraform expertise required.

3. Built-In CI/CD

Every golden path should include a deployment pipeline. Developers shouldn't configure CI/CD — it should come free with the platform.

Pattern: CodePipeline per service (auto-provisioned)

Service creation triggers:
├── CodeCommit/GitHub repo created
├── CodeBuild project (build + test + scan)
├── CodeDeploy / ECS rolling update
├── Pipeline stages: Source → Build → Test → Deploy-Dev → Deploy-Staging → Deploy-Prod
└── Quality gates: unit tests, SAST scan, container scan, integration tests

Alternative: GitOps with ArgoCD (for EKS platforms)

Developer pushes code → CI builds image → updates manifest repo →
ArgoCD detects change → syncs to EKS cluster → progressive rollout

4. Observability as a Platform Service

Don't make developers configure monitoring. Bake it into every golden path:

Layer	Auto-Configured	Service
Metrics	CPU, memory, request count, error rate, latency	CloudWatch Container Insights / EMF
Logs	Structured JSON, centralized, retention policies	CloudWatch Logs + Log Insights
Traces	Distributed tracing across services	X-Ray / OpenTelemetry on ADOT
Dashboards	Per-service dashboard auto-generated	CloudWatch Dashboards
Alerts	SLO-based alerting (error rate > 1%, latency p99 > 500ms)	CloudWatch Alarms → SNS → PagerDuty

Developer experience: Deploy a service → dashboard appears automatically → alerts fire if SLOs breach. Zero configuration.

5. Security and Compliance Guardrails

The platform enforces security without developer intervention:

IAM boundaries — permission boundaries on developer roles prevent privilege escalation
Container scanning — ECR image scan on push, block deployment if critical CVEs
SAST/SCA — code scanning in pipeline (CodeGuru, Snyk, Checkov)
Network isolation — services get private subnets by default, no public access unless explicitly approved
Secrets management — Secrets Manager integrated into task definitions, no hardcoded credentials
Config rules — continuous compliance checking (encryption enabled, public access blocked, tags present)

AWS Proton: The Managed Platform Engine

AWS Proton is purpose-built for platform engineering:

Environment templates — define shared infrastructure (VPC, cluster, logging)
Service templates — define workload patterns (how services deploy INTO environments)
Versioning — templates are versioned; platform team upgrades without breaking existing services
Self-service — developers create services from templates via console, CLI, or API
Sync from Git — templates stored in Git, Proton syncs changes automatically

Proton Architecture

Platform Team                          Developer
     │                                      │
     ▼                                      │
┌──────────────┐                            │
│ Environment  │  (VPC, EKS, RDS, shared)   │
│ Template v2  │                            │
└──────────────┘                            │
                                            ▼
┌──────────────┐                    ┌──────────────┐
│ Service      │  ────────────────→ │  My Service  │
│ Template v3  │  (developer picks) │  (deployed)  │
│ (ECS+ALB+CD)│                    └──────────────┘
└──────────────┘

EKS Blueprints: Platform for Kubernetes

If your IDP is Kubernetes-based, EKS Blueprints (CDK or Terraform) provide standardized cluster provisioning:

What Blueprints package:

EKS cluster with managed node groups (Graviton, Spot)
Core addons: CoreDNS, kube-proxy, VPC CNI, EBS CSI
Platform addons: ArgoCD, Karpenter, Prometheus, Grafana, cert-manager, external-dns
Team namespaces with RBAC and resource quotas
GitOps pipeline for addon management

Developer experience: Request a namespace → get isolated Kubernetes namespace with resource quota, network policy, service account, and ArgoCD project. Deploy via Git push.

Developer Portal: The Front Door

A developer portal gives visibility into all platform services, documentation, and self-service actions.

Backstage (Open Source, Popular Choice)

Service catalog — discover all services, their owners, and documentation
Software templates — create new services from golden paths via UI wizard
TechDocs — documentation lives alongside code
Plugins — integrate AWS services, CI/CD status, cost dashboards
Scorecards — track adoption of best practices per team

Alternative: Custom Portal

Some teams build a simpler portal with:

React frontend
API Gateway + Lambda backend
Service Catalog / Proton API integration
CloudFormation/Terraform execution via CodeBuild

The Platform Team Operating Model

What the Platform Team Owns

Golden path templates (create, update, deprecate)
Shared infrastructure (networking, clusters, CI/CD pipelines)
Security baselines (scanning, guardrails, compliance)
Observability stack (metrics, logs, traces, dashboards)
Developer experience (portal, documentation, onboarding)

What the Platform Team Does NOT Own

Application code
Business logic
Feature decisions
Application-level testing

Team Size Guidance

Organization Size	Platform Team	Ratio
50 developers	3-5 platform engineers	1:10-15
200 developers	8-12 platform engineers	1:15-20
500+ developers	15-25 platform engineers	1:20-30

Measuring Platform Success

Metric	Target	Measures
Time to first deploy	< 1 day	Onboarding friction
Golden path adoption	> 80%	Template quality
Lead time for changes	< 1 hour	CI/CD effectiveness
Developer satisfaction (NPS)	> 40	Overall experience
Mean time to recovery	< 30 min	Platform reliability
Self-service ratio	> 90%	Ticket reduction

Common Anti-Patterns

Anti-Pattern	Problem	Fix
"Build it and they will come"	Developers ignore the platform	Co-design with dev teams, solve THEIR pain points
Too much abstraction	Developers can't debug issues	Provide escape hatches and visibility
Mandatory everything	Creates friction, teams route around	Make golden paths easier than alternatives, not mandatory
Platform as ticket queue	Defeats self-service purpose	If devs still file tickets, the platform isn't done
Over-engineering v1	Launches too late, already outdated	Start with 1-2 golden paths, iterate based on adoption
No documentation	Developers can't onboard	Treat docs as product feature, not afterthought

Getting Started: The 90-Day Plan

Days 1-30:

Interview 5 dev teams about their biggest infrastructure pain points
Identify the #1 most-deployed workload pattern (usually: web API or containerized service)
Build the first golden path template for that pattern

Days 31-60:

Deploy Service Catalog or Proton with the first template
Onboard 2 teams as pilot users
Iterate based on feedback (what's missing? what's confusing?)

Days 61-90:

Add CI/CD and observability to the golden path
Build second template based on demand
Launch developer portal (even basic) with documentation
Measure: adoption rate, deploy frequency, support tickets avoided

Summary

Platform engineering on AWS is about reducing developer cognitive load while maintaining governance:

Self-service provisioning — Service Catalog, Proton, or custom portal. No tickets.
Golden paths — opinionated templates with security, CI/CD, and observability built in.
Shared platform services — CI/CD, observability, security scanning as platform capabilities.
Developer portal — Backstage or custom. Discoverability and documentation.
Operating model — platform team owns templates and shared infra, not application code.

The best platforms aren't built by mandate. They're adopted because they're easier than the alternative. Start with one golden path that solves a real developer pain point, and grow from there.

Alpesh Kumbhare is an AWS Architect at Atos, specializing in AWS infrastructure automation and platform engineering. Connect on LinkedIn.

AWS Disaster Recovery Strategies: From Backup & Restore to Multi-Region Active-Active

AlpeshKumbhare — Thu, 06 Aug 2026 09:34:10 +0000

Every architecture eventually faces "what happens when this fails?" Disaster recovery (DR) isn't about preventing failures — it's about recovering from them within acceptable time and data loss thresholds.

AWS provides four DR strategies with progressively lower RTO/RPO at progressively higher cost. The art is matching the right strategy to each workload's business criticality — not over-engineering (expensive) or under-investing (risky).

This post covers all four strategies, when to use each, the AWS services that implement them, and practical automation patterns for reliable failover.

RTO and RPO: The Two Numbers That Drive Everything

Metric	Definition	Question It Answers
RTO (Recovery Time Objective)	Maximum acceptable downtime	How long can we be down?
RPO (Recovery Point Objective)	Maximum acceptable data loss	How much data can we lose?

        Data Loss                    Downtime
   ◄──────────────────►     ◄──────────────────────►

   Last backup    Disaster    Recovery    Normal ops
       │              │           │            │
       ▼              ▼           ▼            ▼
   ────┼──────────────┼───────────┼────────────┼────
       │              │           │            │
       │◄── RPO ─────►│           │            │
                      │◄── RTO ──►│

The Four DR Strategies

                    Cost / Complexity →

   ┌────────────┐  ┌────────────┐  ┌────────────┐  ┌────────────┐
   │  Backup &  │  │   Pilot    │  │   Warm     │  │ Multi-Site │
   │  Restore   │  │   Light    │  │  Standby   │  │Active-Active│
   └────────────┘  └────────────┘  └────────────┘  └────────────┘

   RTO: Hours       RTO: 10s min    RTO: Minutes    RTO: Near-zero
   RPO: Hours       RPO: Minutes    RPO: Seconds    RPO: Near-zero
   Cost: $          Cost: $$        Cost: $$$       Cost: $$$$

Strategy 1: Backup and Restore

RTO: 1-24 hours | RPO: 1-24 hours | Cost: Lowest

The simplest strategy: back up data and configurations to another region. When disaster strikes, restore from backups and rebuild infrastructure.

Implementation

Component	Backup Method	Target
EC2/EBS	EBS Snapshots (cross-region copy)	DR Region S3
RDS	Automated snapshots + cross-region replication	DR Region
S3	Cross-Region Replication (CRR)	DR Region bucket
DynamoDB	Point-in-time recovery + on-demand backups	DR Region
EFS	AWS Backup with cross-region copy	DR Region
Configuration	CloudFormation/Terraform templates in S3	DR Region

AWS Backup: Centralized Policy

AWS Backup provides a single place to manage backup policies across services:

Backup Plan
├── Rule: Daily backups (retain 30 days)
├── Rule: Weekly backups (retain 90 days)
├── Rule: Monthly backups (retain 1 year)
└── Cross-region copy: All backups → DR region

Recovery Process

Deploy infrastructure from IaC templates (CloudFormation/Terraform)
Restore data from latest backups
Update DNS to point to new resources
Validate application health
Redirect traffic

When to Use

Non-critical workloads where hours of downtime are acceptable
Development/staging environments
Compliance archives (data must exist in DR, but recovery speed isn't critical)
Tightest budget constraints

Strategy 2: Pilot Light

RTO: 10 minutes - 1 hour | RPO: Minutes | Cost: Low-Medium

Keep the core infrastructure "lit" in the DR region — databases replicated, AMIs ready — but no compute running until needed. On failover, spin up compute and scale.

What's Always Running (the "pilot light")

RDS read replica or Aurora Global Database (data continuously replicated)
Core networking (VPC, subnets, security groups, Transit Gateway attachments)
IAM roles and policies
Route53 health checks monitoring primary region

What Starts on Failover

EC2 instances (launch from pre-configured AMIs or Auto Scaling Groups)
ECS/EKS services (scale from 0 to required capacity)
Load balancers (ALB/NLB with target groups)
Caches (ElastiCache — cold start, data rebuilds from DB)

Architecture

PRIMARY REGION                          DR REGION (Pilot Light)
┌─────────────────┐                    ┌─────────────────┐
│  EC2 / ECS      │                    │  (Stopped/Zero) │
│  ALB            │                    │  (No ALB yet)   │
│  RDS Primary    │───replication────→ │  RDS Replica    │
│  ElastiCache    │                    │  (Not running)  │
│  S3 Bucket      │───CRR───────────→ │  S3 Bucket      │
└─────────────────┘                    └─────────────────┘
         │                                      │
    Route53 ─── health check fails ──→ trigger failover

Failover Automation

Route53 health check fails
    │
    ▼
EventBridge rule triggers
    │
    ▼
Step Function: DR Failover
    ├── Promote RDS replica to primary
    ├── Launch EC2 / scale ECS to target capacity
    ├── Create ALB + register targets
    ├── Wait for health checks to pass
    └── Update Route53 to DR region endpoints

Strategy 3: Warm Standby

RTO: Minutes | RPO: Seconds | Cost: Medium-High

A scaled-down but fully functional copy of your production environment runs in the DR region at all times. On failover, scale up to full production capacity.

What's Always Running

Full application stack at reduced scale (e.g., 1 instance instead of 10)
Database with synchronous or near-synchronous replication
Load balancers actively health-checking
Caches warm with subset of data
Background workers at minimum capacity

Architecture

PRIMARY REGION (Full Scale)              DR REGION (Warm Standby)
┌──────────────────────┐                ┌──────────────────────┐
│  ALB (100% traffic)  │                │  ALB (0% traffic)    │
│  EC2: 10 instances   │                │  EC2: 2 instances    │
│  RDS Multi-AZ        │──async repl──→ │  RDS Read Replica    │
│  ElastiCache 3-node  │                │  ElastiCache 1-node  │
│  Workers: 5          │                │  Workers: 1          │
└──────────────────────┘                └──────────────────────┘

Failover Process

Route53 weighted routing shifts traffic (or health check triggers automatic failover)
Auto Scaling scales DR from warm (2 instances) to hot (10 instances)
RDS replica promoted to primary (Aurora Global: ~1 minute)
Caches warm up from database (transient latency spike)
Full production capacity achieved in minutes

Key Services for Warm Standby

Service	DR Capability
Aurora Global Database	<1 second replication lag, ~1 minute failover
DynamoDB Global Tables	Multi-region active-active replication
ElastiCache Global Datastore	Cross-region Redis replication
S3 Cross-Region Replication	Continuous object replication
ECS/EKS	Scale from minimum to target in DR region

Strategy 4: Multi-Site Active-Active

RTO: Near-zero | RPO: Near-zero | Cost: Highest

Both regions serve production traffic simultaneously. No failover required — if one region fails, the other absorbs traffic automatically.

Architecture

                    ┌──────────────┐
                    │   Route53    │
                    │ (Latency or  │
                    │  Weighted)   │
                    └──────┬───────┘
                           │
              ┌────────────┼────────────┐
              │                         │
              ▼                         ▼
    ┌──────────────────┐      ┌──────────────────┐
    │  REGION A (Active)│      │  REGION B (Active)│
    │  ALB + EC2/ECS   │      │  ALB + EC2/ECS   │
    │  Aurora Global    │◄────►│  Aurora Global    │
    │  DynamoDB Global  │◄────►│  DynamoDB Global  │
    │  ElastiCache     │◄────►│  ElastiCache     │
    └──────────────────┘      └──────────────────┘

Design Challenges

Challenge	Solution
Data conflicts	DynamoDB Global Tables (last-writer-wins) or application-level conflict resolution
Session state	Stateless applications + DynamoDB/ElastiCache for session store
Database writes	Aurora Global: single writer region with read replicas, or DynamoDB Global Tables for multi-writer
Data consistency	Eventual consistency between regions (acceptable for most workloads)
Cost	2x infrastructure — only justified for business-critical, zero-downtime requirements

AWS Elastic Disaster Recovery (DRS)

For lift-and-shift DR of EC2 workloads and on-premises servers, AWS Elastic Disaster Recovery provides continuous block-level replication:

Continuous replication — sub-second RPO via block-level data sync
No impact on source — lightweight agent, no snapshots needed
Point-in-time recovery — recover to any point within retention window
Automated failover — launch recovery instances with pre-configured settings
DR drills — test recovery without impacting production
Supports up to 3,000 servers per target account

DRS Architecture

Source Region/On-Prem              DR Region
┌─────────────────┐              ┌─────────────────┐
│  Source Server   │              │  Staging Area   │
│  (DRS Agent)     │──continuous─→│  (Low-cost EBS) │
│                  │  replication │                  │
└─────────────────┘              │  On failover:    │
                                 │  Launch recovery │
                                 │  instances with  │
                                 │  correct config  │
                                 └─────────────────┘

When to Use DRS vs Native DR

Workload	Use DRS	Use Native DR
Legacy EC2 apps (no IaC)	✅	Hard to rebuild
On-premises servers	✅	Not applicable
RDS databases	❌	Cross-region replicas
Containerized apps (ECS/EKS)	❌	Redeploy from images
Serverless (Lambda, Step Functions)	❌	Multi-region deploy from IaC

AWS Resilience Hub: Validate Your DR

Resilience Hub continuously assesses whether your applications meet RTO/RPO targets:

Define application — map resources and their dependencies
Set targets — specify RTO/RPO per application tier
Assess — Resilience Hub evaluates your architecture against targets
Recommendations — suggests improvements (add cross-region replication, implement multi-AZ, etc.)
Drift detection — alerts when changes degrade resilience posture

Service-Specific DR Patterns

Service	DR Pattern	RPO	RTO
Aurora	Global Database (read replicas in DR)	~1 second	~1 minute
RDS	Cross-region read replica + promotion	Minutes	Minutes
DynamoDB	Global Tables (multi-region active)	Near-zero	Near-zero
S3	Cross-Region Replication	Minutes	Seconds
EFS	AWS Backup + cross-region restore	Hours	Hours
ElastiCache Redis	Global Datastore	Seconds	Minutes
Lambda	Multi-region deploy from CI/CD	Near-zero	Near-zero
ECS/EKS	Multi-region service with Route53	Depends on DB	Minutes
SQS/SNS	Recreate from IaC (stateless)	N/A	Minutes

DR Strategy Selection Framework

START
  │
  ├── Can you tolerate hours of downtime?
  │     └── YES → Backup & Restore ($)
  │
  ├── Need recovery in under 1 hour?
  │     └── YES → Pilot Light ($$)
  │
  ├── Need recovery in minutes?
  │     └── YES → Warm Standby ($$$)
  │
  └── Need zero downtime?
        └── YES → Active-Active ($$$$)

Per-workload approach: Most organizations use different strategies for different tiers:

Tier 1 (revenue-generating): Active-Active or Warm Standby
Tier 2 (business operations): Pilot Light
Tier 3 (internal tools): Backup & Restore

DR Testing: The Part Everyone Skips

A DR plan that hasn't been tested is a DR plan that doesn't work. Schedule regular drills:

Monthly: Verify backups are restorable (pick random backup, restore, validate data)
Quarterly: Pilot Light / Warm Standby failover drill (trigger failover, measure RTO)
Annually: Full region failover exercise (for Active-Active workloads)

Use DRS non-disruptive drills — launches recovery instances in isolation without affecting production or replication.

Automate with Resilience Hub — define test schedules, track results, alert on degradation.

Summary

AWS DR isn't one-size-fits-all. Match strategy to business criticality:

Backup & Restore — hours of RTO, lowest cost. Use for non-critical workloads.
Pilot Light — database replicated, compute on-demand. 10 min - 1 hour RTO.
Warm Standby — scaled-down copy always running. Minutes RTO.
Active-Active — both regions serve traffic. Near-zero RTO/RPO. 2x cost.

The critical success factors:

Automate failover — manual runbooks at 3 AM don't work. Step Functions + Route53 health checks.
Test regularly — monthly backup validation, quarterly failover drills.
Use Resilience Hub — continuous validation that architecture meets RTO/RPO targets.
Tier your workloads — not everything needs active-active. Spend where it matters.

Alpesh Kumbhare is an AWS Architect at Atos, specializing in AWS infrastructure automation and resilient cloud architecture. Connect on LinkedIn.

Implementing Zero Trust Architecture on AWS: Verified Access, VPC Lattice, and Identity-Centric Security

AlpeshKumbhare — Wed, 05 Aug 2026 06:01:28 +0000

The traditional security model — "trust everything inside the network perimeter" — doesn't work in cloud. There is no perimeter. Resources span multiple accounts, regions, and connectivity paths. Developers deploy from home networks. APIs are called from everywhere.

Zero Trust flips the model: never trust, always verify. Every request is authenticated, authorized, and encrypted — regardless of where it comes from. On AWS, this isn't a single product. It's an architecture pattern built from multiple services working together.

This post covers how to implement Zero Trust on AWS using the services available today, with practical patterns for user-to-application access, service-to-service communication, and data protection.

Zero Trust Principles on AWS

Principle	AWS Implementation
Verify explicitly	Authenticate and authorize every request based on identity, device, location, and context
Least privilege access	IAM policies, SCPs, resource policies scoped to minimum required permissions
Assume breach	Micro-segmentation, encryption everywhere, continuous monitoring, blast radius isolation

The AWS Zero Trust Stack

┌─────────────────────────────────────────────────────────────────┐
│  USER-TO-APP ACCESS                                              │
│  AWS Verified Access | IAM Identity Center | Cognito             │
├─────────────────────────────────────────────────────────────────┤
│  SERVICE-TO-SERVICE                                              │
│  Amazon VPC Lattice | IAM Auth (SigV4) | PrivateLink            │
├─────────────────────────────────────────────────────────────────┤
│  NETWORK CONTROLS                                                │
│  Security Groups | NACLs | Network Firewall | VPC Endpoints     │
├─────────────────────────────────────────────────────────────────┤
│  DATA PROTECTION                                                 │
│  KMS Encryption | Macie | S3 Access Grants | Lake Formation     │
├─────────────────────────────────────────────────────────────────┤
│  CONTINUOUS VERIFICATION                                         │
│  GuardDuty | Security Hub | CloudTrail | IAM Access Analyzer    │
├─────────────────────────────────────────────────────────────────┤
│  GOVERNANCE                                                      │
│  SCPs | Config Rules | Resource Control Policies                 │
└─────────────────────────────────────────────────────────────────┘

Pillar 1: User-to-Application Access (Replace the VPN)

AWS Verified Access

Traditional pattern: User connects to VPN → gets network-level access to everything inside.

Zero Trust pattern: User requests access to a specific application → identity and device posture are verified → access granted only to that application.

AWS Verified Access implements this:

No VPN required — users access applications directly via browser
Per-request verification — every HTTP request is evaluated against access policies
Identity-aware — integrates with IAM Identity Center, Okta, Azure AD, CrowdStrike, Jamf
Device posture — check device compliance (managed device, antivirus active, OS updated)
Application-level granularity — access to App A doesn't imply access to App B

How Verified Access Works

User (Browser)
    │
    ▼
┌──────────────────────┐
│  Verified Access      │
│  Endpoint             │
│  ┌────────────────┐  │
│  │ Trust Provider  │  │  ← Checks identity (IdP) + device (MDM)
│  │ (IdP + Device) │  │
│  └────────────────┘  │
│  ┌────────────────┐  │
│  │ Access Policy   │  │  ← Evaluates: user group + device trust + context
│  │ (Cedar)         │  │
│  └────────────────┘  │
└──────────┬───────────┘
           │ ✅ Allowed
           ▼
┌──────────────────────┐
│  Internal Application │
│  (ALB / NLB / ENI)   │
└──────────────────────┘

Access Policies with Cedar

Verified Access uses Cedar policy language for fine-grained decisions:

// Allow access only for engineering team on managed devices
permit(principal, action, resource)
when {
    context.identity.groups.contains("engineering") &&
    context.device.status == "compliant" &&
    context.identity.email.endsWith("@company.com")
};

When to Use Verified Access vs VPN

Scenario	Verified Access	VPN
Web applications (HTTP/HTTPS)	✅ Best fit	Overkill
SSH/RDP to EC2	Use SSM Session Manager instead	Legacy option
Non-HTTP protocols (database clients)	Not supported	Still needed
Third-party contractor access	✅ Scoped, auditable	Risky (broad access)
BYOD users	✅ Device posture checks	Hard to enforce

Pillar 2: Service-to-Service Communication

Amazon VPC Lattice

In microservices architectures, services call other services constantly. Without Zero Trust, any service in the VPC can call any other service — one compromised service exposes everything.

VPC Lattice provides identity-based, service-to-service authorization:

Service network — logical boundary grouping related services
Auth policies — IAM-based (SigV4) authentication between services
Per-request authorization — every call verified against policy
Cross-account — services in different accounts can communicate securely without VPC peering
No networking changes — works alongside existing VPCs, no route table modifications

VPC Lattice Architecture

┌─────────────────────────────────────────────────────┐
│               Service Network                         │
│                                                       │
│  ┌─────────┐        ┌─────────┐        ┌─────────┐ │
│  │Service A │──IAM──→│Service B │──IAM──→│Service C │ │
│  │(Account1)│  Auth  │(Account2)│  Auth  │(Account3)│ │
│  └─────────┘        └─────────┘        └─────────┘ │
│                                                       │
│  Auth Policy: Service A can call Service B            │
│  Auth Policy: Service B can call Service C            │
│  Deny all other combinations                          │
└─────────────────────────────────────────────────────┘

VPC Lattice Auth Policy Example

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::111111111111:role/ServiceA-Role"
      },
      "Action": "vpc-lattice-svcs:Invoke",
      "Resource": "arn:aws:vpc-lattice:us-east-1:222222222222:service/svc-abc123/*",
      "Condition": {
        "StringEquals": {
          "vpc-lattice-svcs:RequestMethod": "GET"
        }
      }
    }
  ]
}

This policy: Service A (Account 1) can make GET requests to Service B (Account 2). Nothing else. No network path matters — even if networking allows it, IAM denies it.

VPC Lattice vs PrivateLink vs Service Mesh

Feature	VPC Lattice	PrivateLink	Service Mesh (e.g., App Mesh)
Auth model	IAM (native)	Network-level only	mTLS (self-managed)
Cross-account	Built-in	Complex (endpoint services)	Requires mesh federation
Protocol support	HTTP, HTTPS, gRPC, TCP	Any TCP	HTTP, gRPC
Observability	Built-in access logs	Limited	Sidecar-based (Envoy)
Complexity	Low	Medium	High
Cost model	Per request + data	Per endpoint-hour + data	Proxy instances

Pillar 3: Network Micro-Segmentation

Zero Trust doesn't eliminate networks — it adds identity verification ON TOP of network controls:

Defense in Depth: Layered Network Controls

Layer 1: VPC Isolation (account-per-workload)
  └── Layer 2: Subnet segmentation (public/private/isolated)
       └── Layer 3: Security Groups (instance-level, stateful)
            └── Layer 4: Network Firewall (domain filtering, IPS)
                 └── Layer 5: VPC Lattice / Verified Access (identity-based)

Key Network Zero Trust Patterns

Pattern 1: No public subnets for applications
All applications live in private subnets. Access only through Verified Access (users) or VPC Lattice (services). No direct internet exposure.

Pattern 2: VPC endpoints for AWS services
Don't route AWS API calls through the internet. Use Interface VPC Endpoints — traffic stays on AWS backbone, accessible only from your VPC.

Pattern 3: DNS-based network firewall
AWS Network Firewall with domain-based rules — allow only approved domains for egress. Block everything else by default.

Pattern 4: Eliminate broad security group rules
Replace 0.0.0.0/0 ingress with prefix lists or security group references. Every rule should have a documented justification.

Pillar 4: Data Protection

Zero Trust for data means: even if someone has network access and authenticated identity, they only see data they're explicitly authorized for.

Service	Zero Trust Capability
KMS	Encryption keys with key policies — even admins can't decrypt without explicit grant
S3 Access Grants	Map identities (from Identity Center) to specific S3 prefixes — fine-grained data access
Lake Formation	Column-level and row-level access control for analytics data
Macie	Continuously discover and alert on sensitive data in S3
RDS IAM Auth	Database access via IAM tokens instead of passwords
Secrets Manager	Rotate credentials automatically — no long-lived database passwords

S3 Access Grants: Identity-Based Data Access

Instead of broad S3 bucket policies:

User/Role → Identity Center Group → Access Grant → S3 Prefix

Marketing team → can read s3://data-lake/marketing/*
Engineering team → can read/write s3://data-lake/engineering/*
Finance team → can read s3://data-lake/finance/* (with Macie monitoring)

No bucket policies to manage. Identity drives data access.

Pillar 5: Continuous Verification and Monitoring

Zero Trust isn't set-and-forget. Continuous verification detects drift and threats:

Service	What It Monitors
GuardDuty	Threat detection — compromised credentials, crypto mining, C&C communication
IAM Access Analyzer	External access — finds resources shared outside your organization
Security Hub	Compliance — CIS benchmarks, AWS Foundational Security Best Practices
CloudTrail	Audit — every API call, who did what, when, from where
Config	Drift — detects when resources deviate from compliant state
Detective	Investigation — visualize and investigate security findings
VPC Flow Logs	Network — all traffic flows for forensic analysis

Automated Response Pattern

GuardDuty Finding (e.g., compromised credentials)
    │
    ▼
EventBridge Rule
    │
    ▼
Lambda: Auto-remediate
    ├── Revoke active sessions
    ├── Attach deny-all SCP to affected account
    ├── Isolate EC2 (restrict security group)
    └── Create ITSM incident

Implementation Roadmap

Phase 1: Foundation (Weeks 1-4)

Enable CloudTrail, GuardDuty, Security Hub across all accounts
Implement mandatory tagging and Config rules
Audit existing security group rules — remove 0.0.0.0/0
Enable VPC endpoints for S3, DynamoDB, and frequently-used AWS APIs

Phase 2: Identity-Centric Access (Weeks 5-8)

Deploy Verified Access for top 3 internal web applications
Migrate users off VPN for those applications
Implement IAM Identity Center with MFA and device trust
Enable RDS IAM authentication (eliminate static database passwords)

Phase 3: Service-to-Service (Weeks 9-12)

Deploy VPC Lattice for critical service-to-service paths
Implement IAM auth policies (SigV4 signing)
Remove overly broad security group rules between services
Enable VPC Lattice access logs for audit

Phase 4: Data and Continuous (Ongoing)

Enable S3 Access Grants for data lake
Deploy Macie for sensitive data discovery
Implement automated remediation (GuardDuty → Lambda)
Regular access reviews with IAM Access Analyzer

Common Objections (and Answers)

Objection	Answer
"We already have a VPN"	VPN gives network access. Zero Trust gives application access. Breach one VPN credential = access everything.
"It's too complex to implement"	Phase it. Start with Verified Access for one app. Don't boil the ocean.
"Performance overhead?"	Verified Access adds <10ms latency. VPC Lattice is in-line networking (negligible).
"We trust our internal services"	Assume breach. One compromised container shouldn't access your payment service.
"Compliance requires VPN"	Most compliance frameworks (SOC2, ISO27001) now accept Zero Trust as equivalent or superior to VPN.

Summary

Zero Trust on AWS is built from five pillars:

User-to-app: Verified Access — replace VPN with per-request identity + device verification
Service-to-service: VPC Lattice — IAM-based authentication between services, even cross-account
Network: Micro-segmentation — security groups + Network Firewall + no public exposure
Data: Identity-driven access — KMS, S3 Access Grants, Lake Formation, RDS IAM auth
Continuous: Monitor and respond — GuardDuty, Security Hub, automated remediation

The key shift: network location is no longer a trust signal. Identity is. Every request proves who it is, every service proves it's allowed, and every data access is scoped to exactly what's needed.

Alpesh Kumbhare is an AWS Architect at Atos, specializing in AWS security architecture and cloud infrastructure automation. Connect on LinkedIn.

Building Agentic AI on AWS: From Bedrock Agents to Multi-Agent Orchestration with AgentCore

AlpeshKumbhare — Tue, 04 Aug 2026 07:00:45 +0000

2026 is the year AI moved from "answer questions" to "take actions." Agentic AI — systems that autonomously plan, reason, use tools, and execute multi-step tasks — has become the dominant pattern for building intelligent applications on AWS.

This post covers the full agentic AI stack on AWS: from single-agent basics to multi-agent orchestration, the infrastructure that runs them, and the guardrails that keep them safe in production.

What Makes AI "Agentic"?

Traditional AI: User asks question → Model generates answer → Done.

Agentic AI: User states goal → Agent plans steps → Agent calls tools → Agent evaluates results → Agent iterates → Goal achieved.

The difference is autonomy. An agent decides what to do, executes actions, and self-corrects — without human intervention at each step.

┌─────────────────────────────────────────────────────────────┐
│                    AGENTIC AI LOOP                            │
│                                                              │
│   User Goal → Plan → Act → Observe → Reason → Act → Done   │
│                  ↑                               │           │
│                  └───────── iterate ──────────────┘           │
└─────────────────────────────────────────────────────────────┘

The AWS Agentic AI Stack

┌─────────────────────────────────────────────────────────────────┐
│  APPLICATION LAYER                                               │
│  Amazon Q (Business & Developer) | Custom agents via Bedrock    │
├─────────────────────────────────────────────────────────────────┤
│  AGENT FRAMEWORKS                                                │
│  Bedrock Agents | Strands Agents SDK | LangGraph on AgentCore   │
├─────────────────────────────────────────────────────────────────┤
│  AGENT INFRASTRUCTURE (AgentCore)                                │
│  Runtime | Memory | Identity | Observability | Code Interpreter │
├─────────────────────────────────────────────────────────────────┤
│  TOOLS & KNOWLEDGE                                               │
│  AgentCore Gateway (MCP) | Knowledge Bases (RAG) | Action Groups│
├─────────────────────────────────────────────────────────────────┤
│  SAFETY & GOVERNANCE                                             │
│  Guardrails | IAM | CloudTrail | Model Evaluation               │
├─────────────────────────────────────────────────────────────────┤
│  FOUNDATION MODELS                                               │
│  Claude | Nova | Llama | Mistral | DeepSeek (via Bedrock)       │
└─────────────────────────────────────────────────────────────────┘

Amazon Bedrock Agents: The Managed Path

Bedrock Agents is the fully managed way to build AI agents. You define the agent's instructions, connect tools and knowledge, and Bedrock handles the orchestration loop (ReAct-style reasoning).

Core Concepts

Concept	What It Does
Instructions	System prompt that defines agent's role, behavior, and boundaries
Action Groups	Tools the agent can call (Lambda functions, APIs, or return-of-control)
Knowledge Bases	RAG — grounds agent responses in your data (documents, databases)
Guardrails	Safety controls (content filters, PII masking, denied topics)
Memory	Session persistence — agent remembers context across turns
Code Interpreter	Agent can write and execute code to solve problems

How the Orchestration Loop Works

User sends a message to the agent
Agent's foundation model reasons about what to do (using instructions + context)
Agent decides to call a tool (action group) or query knowledge (RAG)
Tool executes and returns results
Agent evaluates the results — decides if goal is met or needs more steps
Repeat until goal is achieved or max iterations reached
Agent returns final response to user

Building an Agent: Key Design Decisions

Choosing the model: Claude Sonnet or Nova Pro for complex reasoning. Haiku or Nova Micro for simple routing agents.

Instruction design: Be specific about the agent's role, what it should NOT do, and how to handle ambiguity. Vague instructions lead to unpredictable behavior.

Tool design: Each tool should do ONE thing well. Name them clearly (the model uses the name and description to decide when to call them). Include input/output schemas.

# Example: Defining an action group tool
{
  "actionGroupName": "OrderManagement",
  "description": "Manages customer orders - lookup, modify, cancel",
  "apiSchema": {
    "payload": "openapi-schema.json"
  },
  "actionGroupExecutor": {
    "lambda": "arn:aws:lambda:us-east-1:123456789:function:order-api"
  }
}

Multi-Agent Collaboration: Supervisor Pattern

For complex problems, a single agent isn't enough. Multi-agent collaboration lets specialized agents work together:

Architecture: Supervisor + Collaborators

                    ┌──────────────────┐
                    │  Supervisor Agent │
     User ────────→│  (Routes tasks)  │
                    └────────┬─────────┘
                             │
              ┌──────────────┼──────────────┐
              │              │              │
              ▼              ▼              ▼
     ┌──────────────┐ ┌───────────┐ ┌──────────────┐
     │ Research Agent│ │ Code Agent│ │ Review Agent │
     │ (RAG + Web)  │ │ (CodeGen) │ │ (Validation) │
     └──────────────┘ └───────────┘ └──────────────┘

How It Works on Bedrock

Supervisor agent — Receives user request, analyzes complexity, and routes to specialist agents
Collaborator agents — Each has specific tools and knowledge. Execute their specialty and return results to supervisor.
Supervisor synthesizes — Combines collaborator outputs into final response

When to Use Multi-Agent

Scenario	Single Agent	Multi-Agent
FAQ chatbot	✅	Overkill
Code generation only	✅	Unnecessary
Research + summarize + format	⚠️ Gets messy	✅ Clean separation
Customer support (billing + tech + shipping)	⚠️ Tool overload	✅ Specialist routing
Complex analysis with validation	⚠️ Context window limits	✅ Divide and conquer

Rule of thumb: If one agent would need >10 tools or >3 distinct responsibilities, split into multiple agents.

Amazon Bedrock AgentCore: Production Infrastructure

AgentCore is the runtime infrastructure for deploying agents at scale. It provides the "boring but critical" capabilities agents need in production:

AgentCore Components

Component	Purpose
Runtime	Serverless execution environment for agents (auto-scaling, isolation)
Memory	Managed long-term memory across sessions (agent remembers past interactions)
Identity	Authentication for agent-to-service and agent-to-agent communication
Observability	Traces, metrics, and logs for debugging agent behavior
Code Interpreter	Sandboxed code execution (Python/JS) for data analysis tasks
Gateway	Converts APIs and Lambda functions into MCP-compatible tools

AgentCore Gateway: The Tool Layer

The Gateway is particularly powerful — it transforms your existing APIs into tools that any agent can discover and use via the Model Context Protocol (MCP):

Point Gateway at your OpenAPI spec or Lambda function
Gateway auto-generates MCP-compatible tool definitions
Agents discover tools at runtime (no hard-coding)
Security: IAM-based access control per tool

This means agents don't need tools baked into their code. They discover capabilities dynamically — add a new API to Gateway, and all connected agents can immediately use it.

Strands Agents SDK: The Open-Source Path

For teams wanting more control, AWS released Strands Agents SDK — an open-source Python framework for building agents that runs on AgentCore:

Why Strands?

Model-agnostic — works with any Bedrock model (or external models)
Tool-first — tools are defined with @tool decorator, auto-generating schemas
Memory built-in — integrates with AgentCore Memory for persistence
MCP native — discovers tools from MCP servers at runtime
Observable — built-in tracing compatible with AgentCore Observability

Basic Agent Structure

from strands import Agent, tool
from strands.models.bedrock import BedrockModel

@tool
def get_weather(city: str) -> str:
    """Get current weather for a city."""
    # Call weather API
    return f"Weather in {city}: 22°C, sunny"

@tool
def create_ticket(title: str, priority: str) -> str:
    """Create a support ticket in the ticketing system."""
    # Call ticketing API
    return f"Created ticket: {title} (priority: {priority})"

agent = Agent(
    model=BedrockModel(model_id="anthropic.claude-sonnet-4-20250514"),
    tools=[get_weather, create_ticket],
    system_prompt="You are a helpful assistant that can check weather and create tickets."
)

response = agent("Check the weather in London and create a ticket if it's raining")

Strands vs Bedrock Agents: When to Use Which

Criteria	Bedrock Agents (Managed)	Strands SDK (Code-first)
Setup complexity	Low (console/API)	Medium (write code)
Customization	Moderate	Full control
Orchestration logic	AWS-managed ReAct loop	Custom (you define the loop)
Multi-agent	Built-in supervisor pattern	Build your own topology
Deployment	Fully managed	AgentCore Runtime or self-hosted
Best for	Standard use cases, rapid prototyping	Complex custom logic, advanced patterns

Knowledge Bases: Grounding Agents in Facts

Without knowledge, agents hallucinate. Knowledge Bases provide RAG (Retrieval-Augmented Generation):

How It Works

Ingest — Upload documents (PDF, HTML, Markdown, Word, CSV) to S3
Chunk & Embed — Knowledge Base splits documents into chunks, generates embeddings
Store — Embeddings stored in vector database (OpenSearch Serverless, Aurora, Pinecone, or Managed KB)
Retrieve — When agent needs information, relevant chunks are retrieved and injected into prompt
Generate — Model generates response grounded in retrieved facts

Managed Knowledge Base (GA June 2026)

The latest option — fully managed RAG without provisioning anything:

No vector database to manage
Auto-scaling retrieval
Multimodal ingestion (text, images, tables)
Built-in re-ranking for relevance
Agentic retrieval (multi-hop reasoning across documents)

Guardrails: Keeping Agents Safe

Agents that take actions need safety boundaries. Bedrock Guardrails provides:

Policy Type	What It Does
Content filters	Block harmful content (hate, violence, sexual, misconduct) with configurable thresholds
Denied topics	Prevent agent from discussing specific topics (competitor info, legal advice, etc.)
Word filters	Block specific words or phrases
Sensitive information	Detect and mask PII (names, emails, credit cards, SSNs)
Grounding check	Detect hallucinations by comparing response against source documents
Contextual grounding	Verify response relevance to the user's query

Applying Guardrails

Guardrails attach to:

The agent itself (all interactions filtered)
Specific Knowledge Base queries
Individual nodes in a Bedrock Flow

Key insight: Apply guardrails on BOTH input (what users send) AND output (what agents respond). Users can craft prompts to bypass instructions — guardrails are the defense layer.

Orchestration Patterns

Pattern 1: Supervisor-Worker (Hierarchical)

Best for: Customer support, multi-domain queries.
One supervisor routes to specialist workers. Workers don't talk to each other.

Pattern 2: Pipeline (Sequential)

Best for: Document processing, content creation.
Agent A → Agent B → Agent C. Each stage enriches output.

Pattern 3: Parallel Fan-Out

Best for: Research, data gathering from multiple sources.
Multiple agents work simultaneously, results aggregated.

Pattern 4: Debate/Validation

Best for: High-stakes decisions, code review.
Generator agent produces output, critic agent evaluates quality, iterate until criteria met.

Production Checklist for Agentic AI

Before deploying agents to production:

[ ] Guardrails configured — content filters, denied topics, PII masking
[ ] IAM scoped — agent's execution role has minimum required permissions
[ ] Tool permissions bounded — each tool can only access specific resources
[ ] Observability enabled — traces for every agent invocation (debug failed reasoning)
[ ] Cost controls — max iterations per invocation, token budgets
[ ] Fallback behavior defined — what happens when agent can't solve the problem?
[ ] Human-in-the-loop — for high-impact actions (delete, purchase, deploy), require approval
[ ] Testing — evaluate against known good/bad inputs before production
[ ] Rate limiting — prevent runaway agents from flooding APIs
[ ] Audit trail — CloudTrail logging of all agent actions and tool invocations

What's Coming Next

The agentic AI space on AWS is evolving rapidly:

Bedrock Flows — visual builder for chaining agents, prompts, and conditions without code
Agent-to-agent communication — agents that discover and delegate to other agents autonomously
Long-running agents — agents that persist across hours/days (not just request-response)
AgentCore Memory improvements — structured memory with entity relationships, not just conversation history
MCP ecosystem growth — more pre-built tool servers for common services (databases, APIs, SaaS platforms)

Summary

Building agentic AI on AWS in 2026:

Start with Bedrock Agents for managed orchestration — fast to prototype, production-ready
Use Strands SDK when you need custom orchestration logic or advanced patterns
Deploy on AgentCore for production infrastructure (memory, identity, observability)
Connect tools via AgentCore Gateway — MCP-based discovery, zero hard-coding
Ground with Knowledge Bases — RAG prevents hallucination
Protect with Guardrails — content filters, PII masking, grounding checks on every interaction
Scale with multi-agent patterns — supervisor-worker for complex domains, pipeline for sequential processing

The shift from "chatbot that answers" to "agent that acts" is the defining pattern of cloud AI in 2026. The infrastructure is ready — the question is what you build on it.

Alpesh Kumbhare is an AWS Architect at Atos, specializing in AWS infrastructure automation and cloud AI solutions. Connect on LinkedIn.

AWS Cost Optimization Playbook: 20 Strategies That Actually Save Money in Production

AlpeshKumbhare — Mon, 03 Aug 2026 08:10:45 +0000

Industry data consistently shows 28-50% of cloud spend is avoidable. Not because teams are careless, but because cost optimization is a continuous discipline — not a one-time project. Resources get provisioned for peak load and never scaled down, default storage classes persist long after access patterns change, and commitment coverage drifts as workloads evolve.

This playbook covers 20 cost optimization strategies organized by impact area, with specific AWS services and actionable implementation steps for each.

The Cost Optimization Stack

┌─────────────────────────────────────────────────────────────────┐
│  Layer 1: VISIBILITY (you can't optimize what you can't see)     │
│  Cost Explorer | CUR | Budgets | Tags | Cost Anomaly Detection  │
├─────────────────────────────────────────────────────────────────┤
│  Layer 2: PRICING MODELS (biggest single lever)                  │
│  Savings Plans | Reserved Instances | Spot | Graviton            │
├─────────────────────────────────────────────────────────────────┤
│  Layer 3: RIGHTSIZING (match resources to actual demand)          │
│  Compute Optimizer | Trusted Advisor | Auto Scaling              │
├─────────────────────────────────────────────────────────────────┤
│  Layer 4: ARCHITECTURE (design for cost efficiency)              │
│  Serverless | Storage tiering | Data transfer | Caching          │
├─────────────────────────────────────────────────────────────────┤
│  Layer 5: GOVERNANCE (sustain savings over time)                  │
│  FinOps practices | Automation | Accountability                  │
└─────────────────────────────────────────────────────────────────┘

Layer 1: Visibility — Know Where the Money Goes

Strategy 1: Enforce Cost Allocation Tags

Without tags, you can't attribute spend to teams, projects, or environments. This is the foundation.

Mandatory tags:

Environment (prod/staging/dev)
Team or CostCenter
Project or Application
Owner (email of responsible person)

Enforce with: AWS Config rule required-tags + SCP that denies resource creation without mandatory tags.

Strategy 2: Enable Cost and Usage Reports (CUR)

Cost Explorer gives you summaries. CUR gives you line-item detail (every API call, every hour, every resource). Export CUR to S3 and query with Athena for custom analysis.

Key CUR insights:

Which resources have zero utilization?
What percentage of spend is On-Demand vs committed?
Where is data transfer cost concentrated?

Strategy 3: Set Up Cost Anomaly Detection

AWS Cost Anomaly Detection uses ML to identify unexpected spend spikes. Configure alerts per service, account, or cost allocation tag. Catches runaway resources before they become $10K problems.

Strategy 4: Per-Account AWS Budgets

Set monthly budgets per account with alerts at 50%, 80%, and 100% thresholds. Optionally trigger automatic actions (stop EC2 instances, apply restrictive SCP) when budgets are exceeded in non-production.

Layer 2: Pricing Models — The Biggest Lever

Strategy 5: Compute Savings Plans

Savings: Up to 72% vs On-Demand.

Compute Savings Plans are the most flexible commitment — they apply automatically to any EC2, Fargate, or Lambda usage regardless of instance family, size, OS, or region.

Approach:

Analyze 30-day baseline in Cost Explorer → Savings Plans recommendations
Commit to covering your steady-state floor (typically 60-70% of average usage)
Let On-Demand cover the peaks above commitment

Rule: Never commit to more than your minimum usage floor. Over-commitment is waste.

Strategy 6: EC2 Reserved Instances (for specific workloads)

RIs give deeper discounts than Savings Plans (up to 75%) but lock you to specific instance family + region. Use for:

RDS databases (fixed instance type, always running)
ElastiCache clusters
OpenSearch domains
Redshift clusters

Strategy 7: Spot Instances for Fault-Tolerant Workloads

Savings: Up to 90% vs On-Demand.

Use Spot for:

Batch processing, data pipelines, ETL jobs
CI/CD build agents
Stateless web tier (behind ALB with multiple instance types)
EKS/ECS worker nodes (with mixed capacity strategy)
SageMaker training jobs

Key: Use Spot Fleet or EC2 Auto Scaling with multiple instance types and AZs to reduce interruption risk.

Strategy 8: Graviton Migration

Savings: 20-40% better price-performance vs x86.

AWS Graviton (ARM-based) instances offer immediate savings with no code changes for most workloads:

m7g instead of m7i (general purpose)
c7g instead of c7i (compute-optimized)
r7g instead of r7i (memory-optimized)

What works out of the box: Java, Python, Node.js, .NET 6+, Go, containers, databases (RDS, ElastiCache, OpenSearch all support Graviton).

Migration path: Start with non-production → validate performance → switch production during next deployment.

Layer 3: Rightsizing — Match Resources to Demand

Strategy 9: EC2 Rightsizing with Compute Optimizer

AWS Compute Optimizer analyzes 14 days of CloudWatch metrics and recommends:

Downsizing over-provisioned instances
Upsizing under-performing instances
Moving to Graviton

Reality check: Most organizations have 30-40% of EC2 instances over-provisioned by at least one size. A m5.xlarge running at 5% CPU should be an m5.large or smaller.

Strategy 10: Auto Scaling Everything

If a workload doesn't run 24/7, it shouldn't pay 24/7:

Service	Scaling Approach
EC2	Target tracking on CPU/memory + scheduled scaling
ECS/Fargate	Service auto scaling on request count
Lambda	Inherently auto-scales (pay per invocation)
RDS	Aurora Auto Scaling for read replicas
DynamoDB	On-demand mode or auto scaling for provisioned

Strategy 11: Stop/Start Non-Production Resources

Development and staging environments don't need to run nights and weekends. Schedule EC2, RDS, and EKS node groups to stop outside business hours.

Savings: ~65% on dev/staging compute (running 10 hours/day × 5 days vs 24/7).

Tools: AWS Instance Scheduler, EventBridge + Lambda, or Terraform with lifecycle schedules.

Strategy 12: Identify and Terminate Zombie Resources

Common zombies:

Unattached EBS volumes (charged even when not attached)
Idle Elastic IPs ($3.65/month per unused EIP)
Old EBS snapshots beyond retention policy
Idle load balancers (no healthy targets)
Unused NAT Gateways ($32/month + data processing)
Orphaned RDS snapshots

Detection: Trusted Advisor + custom Config rules + Cost Explorer filtering for zero-traffic resources.

Layer 4: Architecture — Design for Cost

Strategy 13: Storage Tiering

S3 storage classes offer massive savings for infrequently accessed data:

Class	Use Case	Savings vs Standard
S3 Standard	Frequently accessed	Baseline
S3 Intelligent-Tiering	Unknown/changing access patterns	Automatic, ~40%
S3 Standard-IA	Accessed < 1x/month	~45%
S3 Glacier Instant Retrieval	Archives needing millisecond access	~68%
S3 Glacier Flexible	Archives (5-12 hour retrieval OK)	~78%
S3 Glacier Deep Archive	Compliance archives (12+ hour retrieval)	~95%

Action: Enable S3 Lifecycle rules to automatically transition objects. Use Intelligent-Tiering as default if access patterns are unpredictable.

EBS: Switch gp2 volumes to gp3 (20% cheaper, better baseline performance). Evaluate whether io2 is truly needed or if gp3 with provisioned IOPS suffices.

Strategy 14: Reduce Data Transfer Costs

Data transfer is the hidden AWS cost killer:

Path	Cost	Mitigation
Cross-region transfer	$0.02/GB	Keep data and compute in same region
Internet egress	$0.09/GB (first 10TB)	CloudFront ($0.085/GB, cheaper at scale)
Cross-AZ transfer	$0.01/GB each direction	Place tightly-coupled services in same AZ
NAT Gateway processing	$0.045/GB	Use VPC endpoints for AWS services ($0)

Quick wins:

VPC endpoints for S3 and DynamoDB (Gateway endpoints are free)
Interface endpoints for frequently-called AWS APIs (cheaper than NAT Gateway data processing)
CloudFront for static content (cheaper than direct S3 egress)
Same-AZ placement for chatty services

Strategy 15: Serverless Where Appropriate

For variable or low-throughput workloads, serverless eliminates idle capacity cost:

Instead of...	Use...	When...
Always-on EC2 for APIs	Lambda + API Gateway	< 1M requests/month
EC2 for scheduled jobs	Lambda + EventBridge	Jobs under 15 min
Self-managed Kafka	EventBridge or SQS	Event routing without ordering guarantees
EC2 for containers	Fargate	Variable workloads, no cluster management
RDS for simple key-value	DynamoDB On-Demand	Unpredictable traffic patterns

Warning: Serverless isn't always cheaper at high scale. Above ~1M invocations/month, compare Lambda cost vs a small EC2 instance.

Strategy 16: Caching to Reduce Downstream Costs

Every cache hit avoids a database query, API call, or compute operation:

CloudFront — Cache static and dynamic content at edge (reduces origin compute + data transfer)
ElastiCache (Redis/Memcached) — Cache database queries (reduces RDS cost/load)
DAX — DynamoDB accelerator (reduces read cost for hot keys)
API Gateway caching — Cache API responses (reduces Lambda invocations)

Layer 5: Governance — Sustain Savings

Strategy 17: FinOps Operating Model

Cost optimization isn't a one-time project. Establish:

Weekly cost review — 15 minutes, engineering leads review team spend vs budget
Monthly optimization sprint — Dedicated time for rightsizing, commitment review, zombie cleanup
Quarterly commitment planning — Review Savings Plan/RI coverage and adjust
Cost ownership — Teams own their account spend. No central team absorbs everyone's waste.

Strategy 18: Automated Cleanup Policies

Automate what humans forget:

Delete EBS snapshots older than retention policy (Lambda + CloudWatch Events)
Terminate dev instances running > 12 hours on weekends
Remove unattached EBS volumes after 7 days
Expire unused Elastic IPs after 48 hours
Tag compliance enforcement (auto-tag with "Owner: unknown" → alert → terminate after 7 days)

Strategy 19: Right-size Commitments Quarterly

Savings Plan and RI coverage drifts as workloads change. Review quarterly:

Are you under-committed? (too much On-Demand spend)
Are you over-committed? (paying for unused commitment)
Has instance family usage shifted? (Graviton migration changes commitment needs)

Use Cost Explorer's Savings Plans utilization report to track coverage.

Strategy 20: Implement Showback/Chargeback

Make cost visible to decision-makers:

Showback — Show teams their spend (awareness, no billing impact)
Chargeback — Charge teams for their consumption (accountability)

Tag-based cost allocation + CUR + Athena dashboards make this possible without third-party tools.

The Quick-Win Priority List

If you can only do 5 things today:

Buy Compute Savings Plans for your usage floor (immediate 30-50% savings on covered compute)
Switch gp2 → gp3 volumes (20% cheaper, zero downtime, no code change)
Enable S3 Lifecycle rules (move old objects to cheaper tiers automatically)
Add VPC endpoints for S3 (eliminates NAT Gateway data processing charges)
Schedule non-prod stop/start (65% savings on dev/staging compute)

These five typically yield 25-35% total savings with minimal effort.

AWS Native Cost Tools Summary

Tool	Purpose
Cost Explorer	Visual spend analysis, forecasting, and Savings Plan recommendations
Cost and Usage Reports	Line-item billing detail for custom analysis
Budgets	Set spending limits with alerts and automated actions
Cost Anomaly Detection	ML-based detection of unexpected spend spikes
Compute Optimizer	Rightsizing recommendations for EC2, EBS, Lambda, ECS
Trusted Advisor	Identifies idle resources and optimization opportunities
Savings Plans	Flexible commitment discounts for compute
S3 Storage Lens	S3-specific cost and usage analytics
VPC IPAM	Tracks IP usage (relevant for NAT Gateway optimization)

Summary

AWS cost optimization is a layered discipline:

Visibility first — Tags, CUR, anomaly detection. You can't optimize what you can't see.
Pricing models — Savings Plans and Spot cover 50-70% of compute savings potential.
Rightsizing — Compute Optimizer + Auto Scaling + scheduled scaling for non-prod.
Architecture — Storage tiering, VPC endpoints, caching, serverless for variable workloads.
Governance — FinOps operating model, automated cleanup, quarterly commitment reviews.

The teams that sustain savings long-term treat cost as an engineering metric — reviewed weekly, optimized continuously, and owned by the teams that provision resources.

Alpesh Kumbhare is an AWS Architect at Atos, specializing in AWS infrastructure automation and cloud cost optimization. Connect on LinkedIn.

Designing a Secure AWS Multi-Account Landing Zone: Architecture Patterns for Enterprise Scale

AlpeshKumbhare — Fri, 31 Jul 2026 07:28:03 +0000

very enterprise AWS deployment eventually hits the same inflection point: a single account becomes ungovernable. Workloads from different teams share blast radius, IAM policies become unwieldy, cost attribution is impossible, and security boundaries blur.

The answer is a multi-account strategy with a well-architected landing zone. This post covers the architecture patterns, OU structures, network topology, and security guardrails needed to build a production-grade AWS landing zone that scales from 10 accounts to 1,000+.

What Is a Landing Zone?

A landing zone is your foundational multi-account AWS environment — the organizational structure, governance controls, networking, and security baseline that exists before any workload arrives. Think of it as the "operating system" for your AWS estate.

Core components:

AWS Organizations — Account hierarchy and policy management
AWS Control Tower — Automated landing zone setup and governance
Landing Zone Accelerator (LZA) — Opinionated, customizable IaC for enterprise landing zones
Service Control Policies (SCPs) — Preventive guardrails at the organization level
Centralized networking — Transit Gateway, shared VPCs, DNS
Security baseline — GuardDuty, Security Hub, Config, CloudTrail across all accounts

Account Structure: The OU Design

The organizational unit (OU) structure is the most critical design decision. It determines how you group accounts, apply policies, and manage access.

Recommended OU Hierarchy

Root
├── Security OU
│   ├── Log Archive Account (centralized logging)
│   ├── Audit Account (security tooling, read-only cross-account)
│   └── Security Tooling Account (GuardDuty, Security Hub delegated admin)
│
├── Infrastructure OU
│   ├── Network Account (Transit Gateway, DNS, shared connectivity)
│   ├── Shared Services Account (AD, CI/CD, artifact repos)
│   └── Backup Account (centralized backup vault)
│
├── Sandbox OU
│   └── Developer sandbox accounts (experimentation, auto-nuke)
│
├── Workloads OU
│   ├── Production OU
│   │   ├── App-A Prod Account
│   │   └── App-B Prod Account
│   ├── Staging OU
│   │   ├── App-A Staging Account
│   │   └── App-B Staging Account
│   └── Development OU
│       ├── App-A Dev Account
│       └── App-B Dev Account
│
├── Policy Staging OU (test SCPs before applying to workloads)
│
└── Suspended OU (quarantined/decommissioned accounts)

Design Principles

One workload, one account — Blast radius isolation. A misconfigured security group affects only one workload.
Environment separation via OUs — Prod/Staging/Dev get different SCPs and guardrails.
Centralize what's shared — Networking, logging, security tooling live in dedicated accounts.
Never put workloads in the management account — It's for Organizations management only.

AWS Control Tower: Automated Governance

Control Tower automates landing zone setup and provides:

Account Factory — Self-service account provisioning with pre-configured baselines
Guardrails (Controls) — Preventive (SCPs) and detective (Config rules) controls
Dashboard — Compliance visibility across all accounts
Region deny — Restrict which AWS regions can be used

Essential Guardrails to Enable

Guardrail	Type	Purpose
Disallow public S3 buckets	Preventive	Prevent data exposure
Require MFA for root user	Detective	Account security
Disallow internet gateways in specific OUs	Preventive	Network isolation
Require encryption on EBS volumes	Detective	Data protection
Deny access to unused regions	Preventive	Reduce attack surface
Require tags on resources	Detective	Cost attribution
Disallow deletion of CloudTrail	Preventive	Audit integrity
Deny root user access	Preventive	Force IAM usage

Landing Zone Accelerator (LZA): Enterprise-Grade IaC

For organizations needing more customization than Control Tower alone provides, the Landing Zone Accelerator (LZA) is an AWS-maintained open-source solution that deploys a comprehensive landing zone via CloudFormation/CDK.

LZA covers:

Multi-account structure with Control Tower integration
Centralized networking (Transit Gateway, route tables, DNS)
Security services deployment across all accounts
Logging and monitoring pipelines
Budget alerts and cost management
Custom CloudFormation stack deployment per account/OU

LZA Configuration Structure

├── accounts-config.yaml      # Account definitions and OU placement
├── global-config.yaml        # Organization-wide settings, regions, tags
├── iam-config.yaml           # IAM roles, policies, identity center
├── network-config.yaml       # VPCs, TGW, route tables, DNS
├── security-config.yaml      # GuardDuty, Security Hub, Config, CloudTrail
├── customizations-config.yaml # Custom stacks deployed per account/OU

This declarative approach means your entire landing zone is version-controlled, auditable, and reproducible.

Network Architecture: Hub-and-Spoke with Transit Gateway

The Pattern

                    ┌──────────────────────┐
                    │   Network Account    │
                    │   (Transit Gateway)  │
                    └──────────┬───────────┘
                               │
        ┌──────────────────────┼──────────────────────┐
        │                      │                      │
        ▼                      ▼                      ▼
┌───────────────┐    ┌──────────────────┐    ┌──────────────────┐
│  Shared VPC   │    │  Workload VPC A  │    │  Workload VPC B  │
│  (Inspection) │    │  (Prod App A)    │    │  (Prod App B)    │
└───────────────┘    └──────────────────┘    └──────────────────┘
        │
        ▼
┌───────────────┐
│ Network       │
│ Firewall /    │
│ Inspection    │
└───────────────┘

Key Networking Decisions

Decision	Recommendation
VPC per account or shared VPC?	VPC per workload account for isolation. Shared VPC only for tightly coupled services.
Transit Gateway vs VPC Peering?	TGW for 5+ accounts. Peering for simple 2-3 account setups.
Centralized egress?	Yes — route internet traffic through a shared inspection VPC with Network Firewall.
DNS resolution	Route53 Resolver in network account with forwarding rules shared via RAM.
On-premises connectivity	Direct Connect to network account TGW, propagate routes to workload VPCs.
IP address management	Use IPAM (VPC IPAM) to prevent CIDR conflicts across accounts.

Network Firewall for Centralized Inspection

All egress traffic from workload accounts routes through a centralized inspection VPC:

Workload VPC → Transit Gateway (default route)
Transit Gateway → Inspection VPC
Network Firewall inspects traffic (domain filtering, IPS/IDS)
Allowed traffic → NAT Gateway → Internet

This gives you a single point of visibility and policy enforcement for all outbound traffic.

Security Baseline: Defense in Depth

Services to Enable Across All Accounts

Service	Purpose	Deployment
CloudTrail	API audit logging	Organization trail → Log Archive account
AWS Config	Resource compliance and change tracking	Aggregator in Audit account
GuardDuty	Threat detection	Delegated admin in Security account
Security Hub	Unified security findings	Aggregator with CIS/AWS Foundational benchmarks
IAM Access Analyzer	Find unintended resource sharing	Per account + organization-level
Macie	S3 sensitive data discovery	Delegated admin
VPC Flow Logs	Network traffic visibility	All VPCs → centralized S3
AWS Backup	Centralized backup policies	Backup account with cross-account vaults

Service Control Policies (SCPs): The Guardrail Layer

SCPs are the most powerful governance tool — they set maximum permissions boundaries that even account administrators cannot exceed.

Essential SCPs:

// Deny leaving the organization
{
  "Effect": "Deny",
  "Action": ["organizations:LeaveOrganization"],
  "Resource": "*"
}

// Deny disabling security services
{
  "Effect": "Deny",
  "Action": [
    "guardduty:DeleteDetector",
    "guardduty:DisableOrganizationAdminAccount",
    "securityhub:DisableSecurityHub",
    "config:StopConfigurationRecorder"
  ],
  "Resource": "*"
}

// Deny root user actions
{
  "Effect": "Deny",
  "Action": "*",
  "Resource": "*",
  "Condition": {
    "StringLike": {"aws:PrincipalArn": "arn:aws:iam::*:root"}
  }
}

Identity and Access: IAM Identity Center

Centralized identity management via IAM Identity Center (formerly SSO):

Single sign-on across all accounts from one portal
Permission sets — predefined access patterns (Admin, ReadOnly, Developer, Auditor)
Integration with corporate IdP (Okta, Azure AD, Active Directory)
Temporary credentials — no long-lived access keys
Attribute-based access control (ABAC) — permissions based on user tags

Permission Set Design

Permission Set	Accounts	Access Level
PlatformAdmin	Infrastructure OU	Full admin
SecurityAuditor	All accounts	Read-only security services
DeveloperAccess	Dev/Staging OUs	Power user (no IAM changes)
ProductionReadOnly	Production OU	Read-only (break-glass escalation for changes)
BillingViewer	Management account	Cost and billing only

Cost Management

AWS Organizations consolidated billing — single payer, volume discounts
Cost allocation tags — mandatory tags enforced via SCP + Config rules
AWS Budgets — per-account and per-OU budgets with alerts
Account-level cost visibility — each team owns their account's spend
Savings Plans and Reserved Instances — purchased at organization level, shared

Common Mistakes to Avoid

Mistake	Impact	Fix
Workloads in management account	Can't apply SCPs to management account	Dedicated workload accounts only
Flat OU structure	Can't differentiate policies per environment	Nest OUs (Workloads → Prod/Staging/Dev)
No network inspection	Blind to outbound data exfiltration	Centralized egress with Network Firewall
Shared VPCs everywhere	Noisy neighbor, no blast radius isolation	VPC per workload account
No SCP testing	Breaking production with bad policies	Policy Staging OU to test SCPs first
Skipping IPAM	CIDR conflicts between accounts	VPC IPAM from day one
Long-lived access keys	Credential exposure risk	IAM Identity Center with temp credentials

Summary

A well-designed AWS landing zone is the foundation everything else builds on:

OU structure — Separate security, infrastructure, and workloads. Nest environments (prod/staging/dev) under workloads.
Control Tower + LZA — Automated governance with guardrails. Version-controlled configuration.
Hub-and-spoke networking — Transit Gateway in a dedicated network account. Centralized inspection for egress.
Security baseline everywhere — CloudTrail, GuardDuty, Security Hub, Config in every account. SCPs as preventive guardrails.
Identity Center — Centralized SSO with permission sets. No long-lived credentials.
Cost governance — Mandatory tags, per-account budgets, organization-level purchasing.

Get the landing zone right, and everything you build on top — workloads, automation, compliance — inherits the security and governance posture from day one.

Alpesh Kumbhare is an AWS Architect at Atos, specializing in AWS infrastructure automation and enterprise cloud architecture. Connect on LinkedIn.