DEV Community: Husain Yusuf

🚀 Streamlining Terraform PR Automation with Atlantis on ECS Fargate

Husain Yusuf — Sun, 06 Apr 2025 19:42:15 +0000

Have you ever experienced these Terraform headaches?

😱 Team members running Terraform commands from their laptops with different versions
😱 Your college is on vacation and you need to take over his/her work. You ask yourself, "what was deployed?"
😱 No visibility into what changes are being applied
😱 Manual processes for reviewing and applying Terraform changes
😱 Forgetting to apply changes after they're approved

Here comes Atlantis. Atlantis solves all these problems by providing a centralized, automated workflow that integrates directly with your Git repositories. It creates a standardized process for Terraform changes that everyone follows.

🤔 What is Atlantis?

Atlantis is an awesome tool that automates your Infrastructure as Code (IaC) processes by integrating directly with your Git workflow. When someone creates a pull request with Terraform changes, Atlantis automatically runs terraform plan and comments the results right on the PR. Once approved, it can run terraform apply too!

In my opinion, one of the greatest benefits of Atlantis is that it doesn't introduce a new user interface - instead, it integrates seamlessly with your existing version control system, allowing teams to perform code reviews and Terraform operations through the same familiar interface.

🚀 Why Atlantis can benefits you

✔️ Enhanced Collaboration with GitOps Approach
Atlantis brings collaboration to the forefront of infrastructure management by integrating with version control systems. Teams can review, comment, and approve Terraform changes directly within pull requests, creating a centralized platform for feedback and reviews

✔️ Increased Efficiency
By automating Terraform commands, Atlantis saves time and reduces the risk of human error. The tool handles the execution of terraform plan and terraform apply, allowing teams to focus on higher-level tasks.

✔️ Infrastructure State Management
Atlantis expertly handles the isolation and management of Terraform state files, preventing conflicts that can occur when multiple developers are changing infrastructure simultaneously using locking.

🔍 Atlantis Workflow for Terraform Automation

This diagram illustrates the Terraform Plan phase of the Atlantis workflow:

A developer creates a pull request with Terraform code changes
GitHub sends a webhook notification to Atlantis
Atlantis automatically runs terraform plan on the proposed changes
The plan output is returned to Atlantis
Atlantis comments the plan results directly in the PR
An approver reviews the PR and the Terraform plan output

This automation eliminates the need for developers to run plans locally and manually share the results, creating a standardized review process.

This diagram illustrates the Terraform Apply phase of the Atlantis workflow that happens after approval:

After approval, a developer comments atlantis apply on the PR
Atlantis executes terraform apply to implement the changes
The apply output is returned to Atlantis
Atlantis comments the results in the PR
The PR can now be merged, completing the workflow

In this blog, I want to demonstrate you how to deploy Atlantis on AWS ECS Fargate using Terraform and setup webhook for Atlantis!

You may ask, why AWS ECS Fargate? In my opinion running Atlantis on AWS ECS Fargate offers several advantages:

Serverless: No need to manage EC2 instances
Scalable: Automatically scales based on demand
Isolated: Runs in its own container for better security
Cost-effective: Pay only for what you use

📋 Prerequisites

To begin this, you will need:

AWS Account with appropriate permissions
Terraform installed
AWS VPC with public and private subnets
GitHub repository with Terraform code
GitHub App ID and GitHub App Installation ID (To create these resources, you can refer to this links GitHub App, Install GitHub App)
GitHub App private key
Webhook secret for GitHub repository
Route53 domain (optional but recommended for HTTPS)

🚶‍♂️ Implementation

For this I created a public repository to give you an example how I deployed Atlantis in AWS ECS Fargate.

atlantis-deployment/
├── main.tf
├── github_webhook.tf
├── secrets.tf
├── variables.tf
├── outputs.tf
├── provider.tf
├── terraform.example-tfvars.
└── tf-modules/
    └── github-repository-webhook/
        ├── main.tf
        ├── variables.tf
        ├── outputs.tf
        ├── version.tf
        └── README.md

The main.tf file contains the core infrastructure for deploying Atlantis on AWS ECS Fargate. It creates CloudWatch log group for the atlantis server. I used this terraform module to deploy Atlantis into ECS Fargate.

data "aws_caller_identity" "current" {}
data "aws_region" "current" {}

################################################################################
# Atlantis Module to run Atlantis on AWS Fargate
################################################################################

### ECS Atlantis CloudWatch Group
resource "aws_cloudwatch_log_group" "atlantis" {
  name              = "/ecs/atlantis"
  retention_in_days = 7
}

### Atlantis Server
module "atlantis" {
  depends_on = [aws_cloudwatch_log_group.atlantis]
  source     = "terraform-aws-modules/atlantis/aws"
  version    = "4.4.0"

  vpc_id          = var.vpc_id
  service_subnets = var.private_subnets_id
  name            = "atlantis"


  ## Atlantis ECS Container Definition
  atlantis = {
    environment = [
      {
        name  = "ATLANTIS_REPO_ALLOWLIST"
        value = join(",", var.atlantis_repo_allowlist)
      },
      {
        name  = "ATLANTIS_GH_APP_ID"
        value = var.github_app_id
      },
      {
        name  = "ATLANTIS_WEB_BASIC_AUTH"
        value = var.web_basic_auth
      },
      {
        name  = "ATLANTIS_WEB_USERNAME"
        value = var.web_username
      },
      {
        name  = "ATLANTIS_WEB_PASSWORD"
        value = var.web_password
      },
      {
        name  = "ATLANTIS_LOG_LEVEL"
        value = "debug"
      },
      {
        name  = "ATLANTIS_WRITE_GIT_CREDS"
        value = true
      },
      {
        name  = "ATLANTIS_ATLANTIS_URL"
        value = join("", ["https://", var.route53_record_name])
      },
      {
        name  = "ATLANTIS_GH_APP_INSTALLATION_ID"
        value = var.github_app_installation_id
      }
    ]
    secrets = [
      {
        name      = "ATLANTIS_GH_WEBHOOK_SECRET"
        valueFrom = aws_secretsmanager_secret_version.github_webhook_secret.arn
      },
      {
        name      = "ATLANTIS_GH_APP_KEY"
        valueFrom = aws_secretsmanager_secret_version.github_app_private_key.arn
      }
    ]
    log_configuration = {
      logDriver = "awslogs"
      options = {
        awslogs-group         = aws_cloudwatch_log_group.atlantis.name
        awslogs-region        = data.aws_region.current.name
        awslogs-stream-prefix = "atlantis"
      }
    }
  }

  ## ECS Service
  service = {
    enable_execute_command  = true
    task_exec_iam_role_name = "atlantis-task-execution-role"
    task_exec_secret_arns   = [aws_secretsmanager_secret_version.github_app_private_key.arn, aws_secretsmanager_secret_version.github_webhook_secret.arn]
    # Provide Atlantis permission necessary to create/destroy resources
    tasks_iam_role_name     = "atlantis-tasks-role"
    tasks_iam_role_policies = var.tasks_iam_role_policies

  }
  ## ALB
  alb_subnets             = var.public_subnets_id
  certificate_domain_name = var.certificate_domain_name
  route53_record_name     = var.route53_record_name
  route53_zone_id         = var.route53_zone_id
  create_alb              = var.create_alb
}

The github_webhook.tf file create the GitHub webhook integration for Atlantis. It uses a local module from ./tf-modules/github-repository-webhook. This configuration ensures that GitHub can send notifications to Atlantis when pull requests are created or updated.

################################################################################
# GitHub repository webhook
################################################################################

module "github_repository_webhooks" {
  depends_on = [module.atlantis]
  source     = "./tf-modules/github-repository-webhook"

  repositories = var.github_repositories

  webhook_url    = "${module.atlantis.url}/events"
  webhook_secret = aws_secretsmanager_secret_version.github_webhook_secret.secret_string
}

Optionally you can update the repository webhook manually from GitHub. Follow this guidance to set this up.

The secrets.tf file create AWS Secrets Manager and SSM Parameter Store resources to store sensitive information.

################################################################################
# Atlantis Server configuration and secrets
################################################################################
resource "aws_ssm_parameter" "atlantis_web_password" {
  name  = "/atlantis/web_password"
  type  = "SecureString"
  value = var.web_password
}
resource "aws_secretsmanager_secret" "github_app_private_key" {
  name        = "github_app_private_key"
  description = "GitHub App private key"
}
resource "aws_secretsmanager_secret_version" "github_app_private_key" {
  secret_id     = aws_secretsmanager_secret.github_app_private_key.id
  secret_string = var.github_app_private_key
}
resource "aws_secretsmanager_secret" "github_webhook_secret" {
  name        = "github_webhook_secret"
  description = "GitHub Webhook Secret"
}
resource "aws_secretsmanager_secret_version" "github_webhook_secret" {
  secret_id     = aws_secretsmanager_secret.github_webhook_secret.id
  secret_string = var.github_webhook_secret
}

Optionally you can configure Atlantis Server setting in repo side. Make sure if your use case required this additional setup.

‼️ Please adjust your own terraform variable. The variable provided in the repo is just a dummy to gives you guidance to use the project.

Run the following commands to initialize and apply your Terraform configuration:

terraform init
terraform plan
terraform apply

🎉 Congratulations that you make this far, then I need to remind you to think about your Atlantis security. In this link Atlantis gives you information what should be done or let's say to improve your security for Atlantis.

Once the deployment is complete, you can verify that Atlantis is running by accessing the URL for Atlantis or (if you not using Route53 domain) DNS name of the Application Load Balancer. Below is an example of Atlantis UI.

🏗️ Test Atlantis using Pull Request in GitHub repository

To test Atlantis in a GitHub pull request, start by creating a new branch in your repository (dev) and adding a simple Terraform configuration. Push this branch and open a pull request. Atlantis will automatically detect the PR and run terraform plan, posting the results as a comment. Review the plan output in the PR comments.

If you're satisfied with the changes, comment atlantis apply on the PR. Atlantis will then execute terraform apply and post the results. Once the changes are applied successfully, you can merge the PR. This process allows you to verify that Atlantis is correctly integrated with your GitHub repository and automates your Terraform workflow through pull requests

🎯 Conclusion – Less Headache managing Terraform!

In my opinion, Atlantis provides a powerful solution for automating Terraform workflows through pull requests. By integrating with version control systems, it enhances collaboration, improves security, and ensures consistency in infrastructure changes 🤖. It remains a valuable tool for teams looking to streamline their infrastructure as code processes. It is for me a tool that reduce the risk of human error while improving the team collaboration.

🔔 Stay tuned for my next blog.

Next topic I will be writing is about to run Atlantis manage Terraform infrastructure in multiple AWS Accounts. See you soon!

🛡️ Centralized Backup Solution in AWS Organization - Because One Backup is never enough!

Husain Yusuf — Sun, 02 Mar 2025 11:26:21 +0000

Data loss, whether due to accidental deletion, cyberattacks, or system failures, can be catastrophic for any organization.

Imagine waking up one day and realizing that your backups have mysteriously vanished. 😱 Maybe someone accidentally deleted them (oops), or worse, a cyberattack wiped them out. Not cool, right?

Enter the AWS Central Backup Account – the superhero 🦸‍♂️ of backups! With this setup, all backups from your AWS Organization are automatically copied to a dedicated AWS account, ensuring an extra layer of protection. No more heart attacks over lost data! 💾✨

🔑 Why You Need This in Your Life

✔️ 🚀 Ultimate Backup Resilience – Even if a backup is deleted in a member account, a copy is safe in the Central Backup Account. Crisis averted!
✔️ 🧐 Compliance Made Easy – Need to meet regulations like GDPR or DORA? Centralized backups make audits a breeze!
✔️ 📝 Automate Everything – AWS Backup Plans take care of everything, so you can relax while your backups work for you.
✔️ 🔒 Backup Security – Protect your backups with Customer Managed KMS Keys and Backup Vault policy!
✔️ 📢 Automated Alerts & Monitoring – Get instant notifications if something goes wrong, so you can fix it before your boss finds out! 😅

🤔 The Problem This Solves

🚨 Backups can be lost! Accidental deletions, cyberattacks, or Murphy’s Law can strike at any time. With this setup, you always have a spare copy.
🚨 Manually copying backups is painful! We automate everything so you never have to worry about forgetting to copy your backups.
🚨 Visibility on backup failures is crucial! AWS EventBridge + Lambda + SNS work together to notify you immediately when something goes wrong.
🚨 AWS Managed Keys don’t work for cross-account backups! (at least now where I wrote this blog in February 2025) That’s why we use Customer Managed KMS Keys to securely share encrypted backups across accounts.

🔍 Centralized Backup Solution Architecture

The diagram illustrates a multi-account AWS backup strategy, ensuring backups are automatically copied from application accounts to a dedicated central backup account for enhanced security and disaster recovery.

🛠 Components in the Architecture

🚀 Application Account (Source Account)

Hosts e.g: Amazon RDS and Amazon EBS volumes that need to be backed up.
Uses AWS Managed Keys or Customer Managed Keys (KMS) to encrypt the snapshots of these resources.
Implements an AWS Backup Plan to schedule automatic snapshots.

🚀 Backup Vaults

Temporary Backup Vault: Stores the initial backup before copying it to the Primary Backup Vault.
Primary Backup Vault: Stores the final backup within the application account, encrypted with a Customer Managed Key (CMK) to enable cross-account copy operations.

🚀 AWS Backup Copy Jobs

Copy Job 1: Copies the backup from the Temporary Backup Vault to the Primary Backup Vault in the same AWS account.
Copy Job 2 (Cross-account copy job) triggered from Lambda: Copies the backup from the Primary Backup Vault to the Central Backup Account.

🚀 AWS Lambda & Amazon EventBridge

EventBridge triggers Lambda functions after each copy job is complete.
Initiating the cross-account copy job once the backup reaches the Primary Backup Vault.
Lambda delete backups from the Temporary Backup Vault after the cross-account copy is successfully complete.
Sending notifications to alert admins of backup failures.

🚀 Parameter Store

Stores backup tag settings used by Lambda functions.

🚀 Central Backup Account

A dedicated AWS account used for long-term storage of backups.
Contains a Backup Vault, where cross-account copies from the application accounts are stored.
Uses a Customer Managed Key (CMK) to encrypt the backups securely.

📝 Prerequisites

✅ An AWS Organization with multiple accounts.
✅ Enable cross-account monitoring in AWS Backup from management account. The steps are described here.
✅ A dedicated AWS Backup Account for centralized backup that already have delegated permission for backup. You can find how to setup here.
✅ Ensure and enable the supported resources for cross-account backup. Check here

🚀 Step-by-Step Deployment

🤖 Deployment in central backup account

Step 1: Create a backup vault in Central Backup Account to store backup copy of member account
📍 Go to AWS Backup → Create a Backup Vault for member account to store the copy of the backup.
📍 Update the Backup Vault Policy → Allow the role in the member account to sent the copy into the backup vault.

📍 Create backup policy that will be implemented across the AWS Organization → The example can be found here.

👨🏼‍🏫 Deployment in member account

Step 1: Set Up a Customer Managed KMS Key 🔑
📍 Go to AWS KMS → Create a Customer Managed Key (CMK).
📍 Update the Key Policy to allow access from the Central Backup Account. → Please refer to this link

Step 2: Configure AWS Backup in Each Member Account 🏗️
📍 Create a Temporary Backup Vault and Primary Backup Vault.
📍 Set up an AWS Backup Plan Rule to back up tagged resources into the Temporary Backup Vault.
📍 Configure the Backup Plan Rule to copy backups to the Primary Backup Vault.

Step 3: Deploy a Lambda Function and EventBridge to Handle Backup Copy Jobs 🤖
📍 Create an EventBridge Rule for successful copy job from Temporary Vault to Primary Vault.
📍 Create an AWS Lambda function triggered by EventBridge. → The function run a copy job from Primary Backup Vault to Central Backup Vault in central backup Account.
📍 If the event is a successful copy from Temporary to Primary Vault, Lambda copies it to the Central Backup Account.

Step 4: Set Up EventBridge to Watch for Backup Jobs Failures 👀
📍 Create an EventBridge Rule for failed backup, copy, or restore jobs (so you know when something’s broken).
📍 Create an SNS Topic and subscribe your email (or Slack, or any your preference endpoint that supported in SNS)
📍 Add SNS as Eventbridge Target to sent the notification.
📍 Get real-time alerts before disaster strikes!

🎯 Conclusion – Your Backups Just Got Smarter!

By implementing this Centralized AWS Backup Solution, you’ve just leveled up your cloud game. No more “oops, my backup is gone” moments, no more compliance headaches, and no more manual backup drudgery.

🚀 Automation? Check.
🔒 Security? Check.
📢 Notifications? Check.

So what are you waiting for? Get started today! 🎉 Your future self will thank you!

‼️ Things to consider

🔔 The time where AWS Backup runs the backup job. In AWS Backup, RDS backups aren't allowed within an hour before the RDS maintenance window or the RDS automated backup window. Therefore, be sure that your backup plans for RDS databases are scheduled more than an hour apart from the RDS maintenance window and the RDS automated backup window.

🥳 How I prepared for AWS DevOps Engineer Professional (DOP-C02) Exam

Husain Yusuf — Sun, 01 Dec 2024 10:26:19 +0000

Passing the AWS DevOps Engineer Professional exam was a significant milestone in my career and marked my 7th AWS certification. It required dedication, strategic planning, and a deep understanding of AWS services and DevOps principles.
This blog post isn’t just a success story—it’s a roadmap for anyone preparing for the exam. Whether you're just getting started or are already knee-deep in AWS, I hope my experiences and tips will inspire you to take on this certification with confidence.

Why Take the AWS DevOps Engineer Professional Exam? 😯

The AWS DevOps Engineer - Professional certification is more than just a badge. It’s a validation of your ability to design, implement, and manage robust CI/CD pipelines, optimize cloud infrastructure, and ensure security and scalability in production environments.

Here’s why you should consider it:

1. Demonstrates Advanced Skills and Knowledge

This certification validates your ability to:

Build robust, scalable, and secure CI/CD pipelines.
Automate complex cloud architectures.
Design fault-tolerant and resilient applications.
Implement best practices for monitoring, governance, and compliance.
It showcases your expertise in bridging the gap between development and operations, a critical skill in modern cloud environments.

2. Career Growth

It demonstrates expertise in DevOps best practices, which are in high demand. In my opinion, roles like Cloud DevOps Engineer, Site Reliability Engineer (SRE), and Cloud Architect often require or prefer professionals with this certification.
Additionally, achieving it demonstrates to employers and peers that you’ve mastered advanced DevOps practices and cloud strategies. It sets you apart as someone capable of handling complex cloud infrastructure challenges.

3. Focus on Real-World Use Cases
Unlike some certifications that are purely theoretical, the DevOps Engineer Professional exam is heavily use-case-driven. By preparing for this exam, you learn how to:

Solve practical challenges in cloud infrastructure.
Optimize deployments for performance, cost, and security.
Apply DevOps methodologies like automation, scaling, and resilience.

4. Encourages a Deeper Understanding of AWS Services
Preparing for this certification forces you to go beyond the basics and develop an advanced understanding of AWS services, including:

How to integrate multiple AWS services into complex architectures.
Best practices for CI/CD, automation, and monitoring.
The trade-offs and limitations of various AWS tools in different scenarios.

This knowledge enhances your problem-solving skills and helps you make better decisions in real-world projects.

5. A Confidence Booster

Tackling one of the most challenging AWS certifications proves your mettle as a cloud and DevOps professional.

My Study Resources 📚

Preparing for this exam was no easy feat—it required a mix of structured study, hands-on projects, and real-world experience. Here’s how I approached it:

1. Stephane Maarek’s Course Udemy
A fantastic starting point! Stephane’s content is well-structured and directly aligned with the exam’s objectives. It helps you build a strong foundation.

2. Adrian Cantrill’s Course
Adrian’s material dives deep into the inner workings of AWS services, giving you a more nuanced understanding. This course is invaluable for grasping the "why" behind AWS architectures.

3. Practice Exams by Jon Bonso
These mock exams are essential. The structure, difficulty, and content are very similar to the real exam, making them great tools for gauging readiness.

4. AWS Skill Builder
I utilized AWS’s official practice exams on Skill Builder to validate my knowledge and identify weak areas. These tests are great for understanding AWS's preferred solutions and best practices.

5. AWS Whitepapers and FAQs
Reading AWS whitepapers, particularly those on CI/CD, monitoring, automation, and security, provided insights into best practices and design principles. The FAQs for services like CodePipeline, CloudFormation, and CloudWatch were also extremely helpful.

6. Do some hands-on project
My work as an AWS Cloud Consultant gave me invaluable hands-on experience with AWS services. Real projects taught me how services are used, their limitations, and the kind of decisions required in practical scenarios.
If you don't have this chance yet, spin up your own projects on AWS Account.

Key takeaway: Don’t just rely on courses. Build pipelines, set up monitoring, deploy scalable architectures, and troubleshoot real-world problems. The exam is heavily focused on use cases, and hands-on experience helps bridge the gap between theory and application.

Key Topics Covered in the Exam ⏳

The exam dives deep into DevOps practices and AWS services. Based on my experience, here are some key areas you’ll need to master:

1. CI/CD Pipelines

Services: AWS CodePipeline, CodeBuild, CodeDeploy, CodeCommit, and GitOps strategies.
Use case: Building scalable and automated CI/CD pipelines for production workloads.

2. Monitoring and Logging

Services: CloudWatch, Eventbridge, X-Ray, and CloudTrail.
Use case: Setting up centralized logging and monitoring for large-scale applications.

3. High Availability and Disaster Recovery

Services: Elastic Load Balancing, Auto Scaling, Route 53, Blue/Green Deployment, AWS Backup, and multi-region architectures.
Use case: Designing fault-tolerant architectures.

4. Infrastructure as Code (IaC)

Services: CloudFormation and AWS CDK.
Use case: Automating infrastructure deployment and management.

5. Security and Governance

Services: IAM, SCPs, AWS Config, Trusted Advisor, GuardDuty and Secrets Manager.
Use case: Implementing least privilege access, threat detection and enforcing compliance.

6. Networking

Services: VPC, Security Group and Transit Gateway.
Use case: Managing secure, scalable, and cost-efficient networks.

Lessons Learned 💪

Hands-On Is a Game-Changer
Understanding AWS concepts is one thing; applying them in real scenarios is another. Building CI/CD pipelines, setting up monitoring systems, and troubleshooting issues in a real-world environment taught me far more than any course.

The Exam Is All About Use Cases

The questions often involve multiple AWS services working together to solve a specific problem. It’s crucial to understand how services interact and which service fits a given scenario best.
Important to mention that all the answer options could be implemented. You need to read carefully what is the requirement mentioned in the question (eg: cost-efficient, reduce management overhead or high avalibility)

Final Thoughts 🏅

This certification is challenging, but with the right preparation, anyone can achieve it. More importantly, certifications are not just about passing an exam—they’re about building a deeper understanding of cloud practices and enhancing your ability to solve real-world problems.

Remember ‼️‼️
Certification is great, but hands-on experience is what truly sets you apart. Dive into projects, experiment with services, and push yourself to apply what you learn.

To everyone considering the AWS DevOps Engineer - Professional exam: go for it! The journey will not only add value to your career but also significantly elevate your cloud expertise.

Good luck, and happy learning! 🌟

🤩How I prepared for AWS Solutions Architect Professional (SAP-C02) Exam

Husain Yusuf — Fri, 18 Oct 2024 16:13:06 +0000

Passing the AWS Solutions Architect Professional exam was one of the most challenging yet rewarding experiences in my cloud journey ✈️ ☁️ . This truly pushed me to deepen my understanding of AWS services and architectures. Throughout this journey, I leveraged a variety of resources and hands-on experiences that helped me succeed. I’ll share here the study materials, strategies, and tips that helped me pass the exam, along with my thoughts on why real-world experience is crucial for success 😉.

Why Take the AWS Solutions Architect Professional Exam❓

The AWS Solutions Architect Professional exam is a highly valued certification in the cloud industry, and here are several reasons why pursuing this certification can be beneficial:

1. Validates Advanced AWS Knowledge 🧑‍🔧

This certification is an excellent way to showcase your deep expertise in designing scalable, fault-tolerant, and cost-efficient systems on AWS. It goes beyond the fundamentals, testing your ability to solve complex architectural challenges using multiple AWS services.

2. Distinguishes You as a Cloud Expert 👩‍🔬

For those seeking a career in cloud architecture, this certification demonstrates to employers that you possess both broad and in-depth knowledge of the AWS ecosystem, making you a valuable asset for architecting solutions for enterprises. You'll feel more confident working on enterprise-level architectures, designing solutions that incorporate multiple services, high availability, disaster recovery, and security best practices.

3. Prepares You for Real-World Scenarios 🏃‍♀️

The exam focuses on solving use-case-based scenarios that mirror the challenges you might face in the real world. You’re not just tested on individual services but rather on how to combine services effectively to meet specific business needs.

4. Opens Up Career Opportunities 🧑‍🚀

As cloud adoption grows, so does the demand for cloud architects with advanced knowledge. Earning this certification can help unlock career growth opportunities and tt is especially relevant for senior-level positions like Cloud Architect, DevOps Engineer, and Cloud Solutions Consultant.

5. Proves Mastery of AWS Best Practices 🏋️

AWS puts a strong emphasis on best practices, scalability, cost optimization, and security in its exam questions. This certification validates that you understand not just how to use services, but how to design architectures that are in line with AWS best practices.

6. Endless Learning Opportunity 🚴

Pursuing this certification also exposes you to new services and features that AWS continuously rolls out. In my preparation, I encountered services and use cases I hadn’t worked with before, which expanded my knowledge base and prepared me to design better solutions for future projects.

So now that you already aware the benefits of taking this exam, I will now give you a deeper insight to my study resources.

My Study Resources 📚

To prepare for the exam, I used a combination of online courses, practice exams, AWS whitepapers, and hands-on experience. I invested my time around 10 hr/week and did 2 month preparation for the exam.
Here’s a breakdown of the resources I used:

1. Stephane Maarek’s Course (Udemy)
Stephane’s course was my main guide for preparing specifically for the exam. His content is structured to mirror the domains of the AWS Solutions Architect Professional certification. His course and practice exams gives you excellent coverage of the required topics.

2. Adrian Cantrill’s Course
While Stephane’s course is great for exam-focused preparation, Adrian Cantrill’s course provided deeper insights into AWS services. Adrian’s content goes beyond exam requirements, explaining the architecture and intricacies of AWS services, giving you an edge in understanding how AWS services work.

3. Jon Bonso’s Practice Exams (Tutorials Dojo)
Jon Bonso’s practice exams are invaluable. They are extremely similar to the actual AWS exam in terms of question style, difficulty, and structure. I used both Jon’s and Stephane’s practice exams, which helped build my confidence for exam day.

4. AWS Skill Builder
AWS offers its own set of practice exams through AWS Skill Builder, which gave me the chance to validate whether I was ready. The questions are similar to what you can expect in the actual exam and give a good feel for how well-prepared you are.

5. AWS Whitepapers
AWS whitepapers were a key resource for best practices. They helped me deepen my understanding of AWS architectural frameworks and gave insight into cost optimization, security, and scalability strategies. The whitepapers that I found particularly useful included the Well-Architected Framework, Security Best Practices, and High Availability Architectures.

6. Do some hands-on project
The more you work on real projects, the easier it will be to understand the scenarios and architectural challenges presented in the exam. Spin up your own projects on AWS Account or participate in client projects if possible.

AWS Services Covered in the Exam ⏳

The AWS Solutions Architect Professional exam covers a vast array of services, with questions that typically combine at least three services in one use case. The exam tests not only your knowledge of services but also your ability to apply them in various real-world scenarios.

Many of the exam questions present real-world scenarios where you need to identify the best architecture or AWS service to meet the specific requirements (e.g., scalability, cost-efficiency, security, or disaster recovery). For instance, a question might ask how to implement a highly available and fault-tolerant architecture across multiple AWS regions using services like S3, Route 53, and CloudFront.

Here’s an overview of some of the key services that appeared in the exam:

1. Compute:
EC2 ,Lambda, Elastic Beanstalk

2. Containers & Orchestration:
ECS, EKS, AWS Fargate, ECR

3. Storage:
S3, EFS, EBS, Amazon FSx, Amazon Glacier

4. Networking:
VPC, AWS Transit Gateway, Direct Connect and VPN, API Gateway (how to secure these resources).

5. Security & Identity:
IAM, AWS Organizations, Service Control Policies (SCPs), AWS SSO, Cross-Account Roles, AWS Managed AD, KMS (key policy), System Manager (Parameter Store, Session Manager, Automation, Run Command), ACM

6. DevOps Tools:
CodeBuild, CodePipeline ,CodeDeploy

7. Data Services:
Amazon RDS, DynamoDB, Redshift, Athena, Amazon ElastiCache (Redis), Amazon Neptune

8. Content Delivery & Analytics:
CloudFront, QuickSight, Amazon WorkSpaces

9.Monitoring and Notification Services:
Cloudwatch, CloudTrail (Organization), SNS, Eventbridge Rules, AWS Config

10. High Availability & Disaster Recovery:
Route 53, Elastic Load Balancers (Application Load Balancer and Network Load Balancer), Auto Scaling, AWS Backup

Hands-On Experience is a must and critical

While courses and practice exams are important, the most valuable part of my preparation came from hands-on experience.

Here’s why hands-on experience matters: 🏅 🏅

Context & Problem-Solving: In the exam, most questions revolve around real-world scenarios. Having worked on AWS implementations, I was able to draw from my experience to solve complex questions on topics like hybrid architectures, scalability, and high availability.
Service Limitations: Some AWS services have limitations that aren’t obvious from reading the documentation. Only by using these services in real-world projects do you truly understand how they behave under different conditions.
Design Trade-offs: Working on projects helped me better understand how to make trade-offs in architectural designs. The exam often presents several valid solutions, and your experience helps you choose the best one based on performance, cost, and scalability considerations.

Final Tips for Preparing for the AWS Solutions Architect Professional Exam

Start with Courses, But Don’t Rely Only on Them: Courses like Stephane Maarek’s will give you a strong foundation, but don’t stop there. Dive deeper into services and AWS best practices.
Do More Hands-On Labs: The more you work on real projects, the easier it will be to understand the scenarios and architectural challenges presented in the exam. Spin up your own projects on AWS or participate in client projects if possible.
Take Practice Exams: Jon Bonso’s and Stephane’s practice exams are fantastic for testing your knowledge. Don’t just take them once—review your mistakes and learn from them. AWS’s official practice exams are also a great way to measure your readiness.
Focus on Domain Mastery: The AWS Solutions Architect Professional exam covers a wide range of topics. The exam domains include:
- Design for Organizational Complexity
- Design for New Solutions
- Continuous Improvement for Existing Solutions
- Accelerate Workload Migration and Modernization
Review Exam Readiness on AWS Skill Builder: AWS’s Skill Builder platform offers exam readiness courses and practice exams that are valuable in fine-tuning your understanding and confidence before the actual test.

My Final Thoughts 🎯

Getting certified is not just about passing an exam—it’s about taking a leap in your cloud journey and gaining a deep understanding of how to design complex, scalable, and secure architectures using AWS.

However, remember that certifications are just one part of the equation. Hands-on experience is what truly pushes your learning curve. Working on real-world projects not only prepares you for the exam but also gives you insights into how AWS services behave, their limitations, and best use cases. Through hands-on experience, you’ll develop a deeper understanding of architectural trade-offs, service integrations, and how to meet the needs of diverse business scenarios.

So, I encourage you to take on this certification challenge, but don’t stop there. Get your hands dirty, dive into projects, and continuously apply what you’ve learned. Have fun and good luck on your cloud journey! 👍 👍 👏

Securely Push Image from GitHub to AWS with GitHub Action

Husain Yusuf — Wed, 09 Oct 2024 10:11:58 +0000

🚀 In modern cloud-native development, automating deployments is key to ensuring fast, secure, and reliable application delivery. GitHub Actions, a powerful CI/CD tool, allows you to automate your workflows directly from your GitHub repository. When combined with AWS and OpenID Connect (OIDC), you can securely deploy your code without managing long-lived AWS credentials and maintaining high security standards.

Here are the key reasons why OIDC is beneficial:

Eliminates Long-Lived Credentials: OIDC allows GitHub Actions to authenticate with AWS without storing permanent credentials like AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY in your GitHub repository. This reduces the risk of credential leakage and simplifies credential management.
Improves Security: By using short-lived tokens generated during each workflow run, OIDC minimizes the attack surface compared to static credentials.
Simplifies Access Management: OIDC enables federated identity management, allowing you to control access permissions centrally in AWS IAM. You can define trust relationships and conditions under which GitHub tokens are considered valid, streamlining access control.
Supports Conditional Access: You can configure policies in AWS IAM to restrict access based on specific conditions, such as the repository or branch from which a workflow is triggered. This granular control helps enforce security best practices
Integrates Easily with GitHub Actions: GitHub provides official actions like aws-actions/configure-aws-credentials that simplify the integration process by automatically exchanging OIDC tokens for AWS temporary credentials during workflow execution

In this blog post, we’ll walk through how to set up a seamless deployment from GitHub to AWS using GitHub Actions and OIDC.
I setup an example of use case using Amazon Elastic Container Registry (ECR) as our image registry and IAM Role that will be assumed through OIDC. After the resources in AWS are created, we'll use GitHub Actions to build docker image of Flask (Python) application using Docker and push the Docker image to Amazon ECR and secure the connection with AWS using OpenID Connect (OIDC).

Prerequisites

A GitHub repository containing a Flask application. There is a simple flask app as an example.
An AWS account with access to ECR.
AWS CLI installed and configured in the GitHub runner. (I am using Github-hosted runner that already have AWS CLI installed).

Step 1: Set Up AWS Resources

1. Create ECR Repository:

Open the AWS Management Console and navigate to ECR.
Click on Create repository.
Name your repository (e.g., flask-app) and create it.

2. Add an Identity Provider in AWS
The first step is to create an Identity Provider in AWS. The following steps are based on the GitHub documentation here.

Go to IAM in the AWS Management Console.
In the left navigation pane, click on Identity providers and choose Add provider.
For Provider type, select OpenID Connect.
For Provider URL, enter https://token.actions.githubusercontent.com. Provider URL is the Github OpenID Connect URL for authentication requests.
For Audience, enter sts.amazonaws.com. Audience is a value that identifies the application that is registered with an OpenID Connect provider.
Click Add provider.

3. Create an IAM Role for OIDC

Create a new role with the Web identity type.
Select GitHub as the provider and specify your repository details.
Attach policies that allow write access to ECR (In this case I am using AWS-managed Policy AmazonEC2ContainerRegistryPowerUser).

Create trust relationships for the role. This allow only specific branch or tags to assume the role.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "Federated": "<Arn of Identity provider>"
            },
            "Action": "sts:AssumeRoleWithWebIdentity",
            "Condition": {
                "StringLike": {
                    "token.actions.githubusercontent.com:sub": [
                        "repo:<GitHub organization name>/<GitHub repo name>:ref:refs/heads/main",
                        "repo:<GitHub organization name>/<GitHub repo name>:ref:refs/heads/dev*",
                        "repo:<GitHub organization name>/<GitHub repo name>:ref:refs/tags/v*"
                    ],
                    "token.actions.githubusercontent.com:aud": "sts.amazonaws.com"
                }
            }
        }
    ]
}

You have to replace the placeholders with correct values.

Arn of Identity Provider - ARN of the Identity Provider that you created in previous step.
GitHub organization name - your GitHub Organization name
GitHub repo name - your GitHub Repository name

You can use the StringLike condition in the IAM policy to allow more flexible pattern matching with wildcards. Instead of using StringEquals, which requires an exact match, StringLike lets you use patterns like * to match multiple variations.
In this case, to allow deployment from branches starting with dev (like dev, develop, etc.) and any version tag that follows the semver rule (e.g., v1.0, v2.1.3), you can set up the condition like this:

Branches: Use StringLike with dev* to match any branch that begins with dev.
Tags: Use StringLike with v* to match any tag that starts with v, allowing flexibility for any versioning system that follows the semantic versioning convention.

Step 2: Update GitHub Repository Secrets

1. Add Repository Secrets:

In your GitHub repository, go to Settings > Secrets and variables > Actions.
Create following secrets:

AWS_REGION = Region of AWS where you deploy your AWS resources.

AWS_ROLE = Arn of the IAM role that the GitHub Action has to assume.

Step 3: Create GitHub Actions Workflow

1. Create Workflow File:

In the GitHub repository, create a .github/workflows/build_image.yml file.

name: Deploy to AWS

on:
  push:
    branches:
      - main

jobs:
  build-and-deploy:
    runs-on: ubuntu-latest

    permissions:
      id-token: write
      contents: read

    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Configure AWS Credentials
        uses: aws-actions/configure-aws-credentials@v4
        with:
          role-to-assume: ${{ secrets.AWS_ROLE }}
          role-session-name: ecr-push-registry-${{ github.run_number }}
          aws-region: ${{ secrets.AWS_REGION }}

      - name: Log in to Amazon ECR
        uses: aws-actions/amazon-ecr-login@v2
        id: login-ecr

      - name: Build, tag, and push Docker image to Amazon ECR
        env:
          ECR_URL: ${{ steps.login-ecr.outputs.registry }}
          REPOSITORY: flask-app
          VERSION_TAG: ${{ github.sha }}
        run: bash ./build_image.sh

      - name: Log out of Amazon ECR
        if: always()
        run: docker logout ${{ steps.login-ecr.outputs.registry }}

This is the overview of the workflow code:

Name: The workflow is named "Deploy to AWS".
Trigger: The workflow is triggered by a push event on the main branch.
Jobs: The workflow consists of a single job named build-and-deploy which runs on an ubuntu-latest virtual machine.
Permissions
- id-token: write: This permission allows the workflow to request an OpenID Connect (OIDC) token, which is used to authenticate with AWS securely.
- contents: read: This permission allows the workflow to read repository contents, which is necessary for checking out the code.
Steps
- Checkout Code: This step checks out the repository code so that subsequent steps have access to it.
- Configure AWS Credentials: This step configures AWS credentials using OIDC. It assumes a role specified by the secret AWS_ROLE and sets the AWS region using the secret AWS_REGION.
- Log in to Amazon ECR: This step logs into Amazon Elastic Container Registry (ECR) so that Docker images can be pushed. The step ID is set to login-ecr, which allows referencing its outputs in later steps.
- Build, Tag, and Push Docker Image to Amazon ECR: This step create environment variables ECR_URL (The URL of the ECR registry obtained from the output of the login step) , REPOSITORY and VERSION_TAG (A unique tag for the Docker image based on the commit SHA (github.sha)). This step executes a shell script (build_image.sh) that builds, tags, and pushes the Docker image to ECR.
- Log out of Amazon ECR: Logs out from Amazon ECR using Docker to ensure that credentials are not left lingering. This step always runs (if: always()) regardless of whether previous steps succeeded or failed.

Step 4: Test the GitHub Action workflow

Once you have configured GitHub Actions, you can test the integration by pushing a change to your GitHub repository. GitHub Actions should automatically trigger a build and deploy process in AWS, using the OpenID Connect authentication data to authenticate the user.

Conclusion

And there you have it, folks! We've successfully sent our trusty Flask app on a whirlwind adventure to the cloud ☁️, all thanks to the magic of GitHub Actions and AWS. With Docker as its suitcase and OIDC as its security blanket, our app is now living its best life.

Stay tuned for my next blog post, where we'll dive even deeper into the world of cloud automation and deployment magic. See you then! 😎