DEV Community: Ismail G.

How I Became an AWS Community Builder (Data Track)

Ismail G. — Sat, 07 Mar 2026 07:17:26 +0000

I got an email a few days ago that made my day. I had been accepted into the Data track of the AWS Community Builders.

This initiative may just look like another badge for a lot of folks. But for me, it means a lot more: months of studying, trying new things, and sharing what I learn with other people.

After I told folks the news, a lot of them asked me the same thing: "What helped you get in?"

There is no one secret, to tell the truth. But one thing was really important: always sharing technical information. One of the best things I did on my trip was to write about my experiences with databases, cloud architectures, and AWS services on Dev.to.

In this article, I want to talk about what I accomplished, what made my application stand out, and what I would tell anyone who wishes to apply to the program in the future.

What is the AWS Community Builders Program?

Before diving into my journey, let's clarify what this program is. The AWS Community Builders program is designed to recognize and support technical community leaders who are passionate about sharing knowledge and connecting with others about AWS technologies.

It provides builders with technical resources, mentorship, $500 in AWS credits, exam vouchers, and a direct line to AWS product teams. It’s not just about what you know; it’s about how you help others learn.

Where I Started: My Background

At first, I focused mainly on getting AWS certifications. But after a while, I realized that passing an exam is really just the starting point. To truly understand the cloud, I needed hands-on experience—working through real database problems, experimenting, sometimes breaking things, and figuring out how they work.

Along the way, I also started documenting what I learned so others could benefit from the same experiences.

The Turning Point: Writing on Dev.to

At some point, I realized that simply learning new things wasn’t enough. I was spending hours troubleshooting systems, experimenting with databases, and figuring out how different AWS services worked together—but most of those lessons stayed in my own notes.

That’s when I started sharing what I learned on Dev.to.

Instead of writing simple notes for myself, I began turning real troubleshooting sessions into structured tutorials. Whenever I solved a problem, I tried to explain the process step by step—what went wrong, what I tried, and what finally worked.

Another place where I learned a lot was AWS re:Post. I started helping people who were facing real problems with AWS services. Sometimes the questions were about databases, sometimes about architecture or infrastructure.

When I encountered an interesting problem there, I didn’t just answer it and move on. I often recreated the scenario in AWS, tested different solutions, and then wrote a detailed article on Dev.to so that the solution could help more people facing the same issue.

Because I applied to the Data track of the AWS Community Builders, many of my articles naturally focused on data-related topics, such as:

AWS DocumentDB — exploring how managed NoSQL databases work in AWS
MongoDB migrations — the challenges of moving on-premise data to the cloud
Database architecture — designing systems for high availability and scalability
Cloud infrastructure — automating data workloads and deployments

Over time, something interesting happened. Writing about these topics didn’t just help others—it also helped me understand them much more deeply. Explaining a solution forces you to truly understand it.

One thing I learned along the way is this:

Don’t just write about what a service is. Write about the problem you solved with it.

That’s the kind of knowledge the cloud community values most.

Community Contribution & Engagement
A significant part of my journey consisted of writing articles; however, I quickly realized that being "constructive" wasn't just about producing content.

Being a member of a community is also an important element. Answering questions posted in online forums, participating in online discussions, and trying to help with challenging database configurations also gave me new experiences.

What Actually Matters When Applying?

If you are planning to apply for the next cohort, here are the four pillars that I believe made my application stand out:

I can't speak for the critics, but in my experience, four elements seem to be the most important:

Technical Information
High-quality blog posts, GitHub repositories, or films that indicate how deep your technical knowledge is and how much practical experience you have.

Consistency:
One article soon before the deadline won't do anything. If you post content regularly over a few months, it demonstrates you're actively participating.

Real-World Experience:
Your information is considerably more useful if you explain how you use AWS services to solve real problems, especially difficulties with infrastructure or data.

Community Impact:
Last but not least, your content should be useful to individuals. People talking about your ideas, leaving comments, and using them prove that your effort is helpful to the community.

My Advice for Aspiring Builders

If you’re thinking about applying to the AWS Community Builders, my biggest advice is simple: start sharing what you learn.

You don’t need to be an expert in everything. In fact, many of the articles I wrote started with something I had just learned while working on a real problem. Instead of keeping that knowledge to myself, I turned those experiences into tutorials and shared them with the community.

One thing that helped me a lot was writing about real challenges. Explaining how you solved a problem—whether it’s a database migration, an architecture decision, or a troubleshooting process—creates content that is actually useful for others.

Another important thing is consistency. You don’t need to publish something every week, but sharing your learning journey over time shows that you’re actively contributing to the ecosystem.

Finally, try to engage with the community whenever you can. Platforms like Dev.to or AWS re:Post are great places to both learn from others and help people solve real problems.

At the end of the day, the goal isn’t just to get accepted into the program. The real value comes from learning in public and helping others along the way.

A New Beginning

Becoming an AWS Community Builder is a milestone, but more importantly, it’s a beginning. It’s an invitation to learn more, share more, and connect with some of the brightest minds in the industry.

Are you planning to apply for the next round? Let me know in the comments if you have any questions about the process!

Secure Terraform CI/CD on AWS with GitHub Actions (OIDC + Remote State)

Ismail G. — Sun, 08 Feb 2026 21:40:52 +0000

For CI/CD processes to work well, they need to be secure and repeatable. Without a strong authentication system and a consistent state management strategy, infrastructure automation quickly becomes vulnerable to security threats.
This blog post explains how to set up a remote Terraform backend with state locking using Amazon S3 and DynamoDB. We will also use OIDC to set up keyless authentication from GitHub Actions to Amazon Web Services (AWS).

Amazon S3 – remote state storage (versioned & encrypted)
DynamoDB – state locking
AWS KMS – encryption
GitHub Actions – CI/CD automation

Why This Setup Matters

AWS access keys are saved as secrets in traditional continuous integration pipelines. This plan is very risky because long-lived credentials could be stolen.

Key rotation is hard, but it's necessary. When CI is compromised, AWS is also compromised.

OpenID Connect (OIDC) solves this problem by letting GitHub Actions get an IAM role dynamically using short-lived credentials from AWS STS.

Terraform also needs to use a remote backend to:

Stop the state from getting corrupted at the same time.
Take care of values that are weak.
Make sure that people can work together.

This architecture solves both of these problems in a way that is easy to use and can grow with your needs.

High-Level Architecture:

GitHub Actions requests an OIDC identity token
AWS validates the token using IAM OIDC Provider
An IAM Role is assumed via sts:AssumeRoleWithWebIdentity
Terraform runs with temporary credentials
State is stored in encrypted S3, locked via DynamoDB

Step 1️: Create AWS OIDC Provider

To allow GitHub Actions to authenticate with AWS, an OIDC provider must be configured in AWS IAM. Before this, if you do not have AWS CLI configured in your local computer, you must setup it.

AWS CLI Setup (macOS)

# Homebrew
brew install awscli

aws --version

aws configure

write those when asked:
AWS Access Key ID [None]: <your-accesskey>
AWS Secret Access Key [None]: <your-secret-accesskey>
Default region name [None]: <region-name>
Default output format [None]: json

# Account test
aws sts get-caller-identity

Create the OIDC Provider

Run the following command using AWS CLI or create the provider via the AWS Console.

aws iam create-open-id-connect-provider \
  --url https://token.actions.githubusercontent.com \
  --client-id-list sts.amazonaws.com \
  --thumbprint-list 6938fd4d98bab03faadb97b34396831e3780aea1

This enables AWS to validate GitHub-issued identity tokens.

Step 2️: Create IAM Role for GitHub Actions

Next, an IAM role must be created so that GitHub Actions workflows can assume it using sts:AssumeRoleWithWebIdentity.

This role explicitly defines:

Who can assume it (GitHub Actions)
From which repository it can be assumed

Create GitHubActionsRole with trust policy sts:AssumeRoleWithWebIdentity

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Principal": {
        "Federated": "arn:aws:iam::YOUR_ACCOUNT_ID:oidc-provider/token.actions.githubusercontent.com"
      },
      "Action": "sts:AssumeRoleWithWebIdentity",
      "Condition": {
        "StringEquals": {
          "token.actions.githubusercontent.com:aud": "sts.amazonaws.com"
        },
        "StringLike": {
          "token.actions.githubusercontent.com:sub": "repo:YOUR_GITHUB_USERNAME/YOUR_REPO_NAME:*"
        }
      }
    }
  ]
}

Attach IAM Policy

For bootstrap simplicity, we attach AdministratorAccess.
Important:
In real production environments, replace this with least-privilege policies.

Step 4️: Configure GitHub Repository Secret

GitHub Actions must now be informed which IAM role to assume.

In the GitHub repository: Settings → Secrets and variables → Actions → New repository secret

Create the following secret:

AWS_ROLE_ARN = arn:aws:iam::YOUR_ACCOUNT_ID:role/GitHubActionsRole

Step 5️: Terraform Remote State

We use a one-time bootstrap workflow to provision:

S3 bucket (versioning + encryption)
DynamoDB table (state locking)
KMS key (state encryption)

Repository Structure

terraform-remote-state/
├── main.tf
├── providers.tf
├── variables.tf
├── terraform.tfvars

From my GitHub repository you can check the terraform files:

GitHub Repo

Step 6️: Bootstrap GitHub Actions Workflow

Below is the final bootstrap workflow.

Uses OIDC for AWS auth

Accepts the S3 bucket name as an input

Pins Terraform version

Verifies AWS identity before provisioning

bootstrap.yml

# This workflow creates the foundational infrastructure for Terraform:
# - S3 bucket for state storage with encryption and versioning
# - DynamoDB table for state locking (prevents concurrent modifications)
# - KMS key for encrypting state files and secrets
#
# Run this ONCE before deploying main infrastructure

name: Bootstrap 

on:  
  workflow_dispatch:

permissions:
  contents: read
  id-token: write

env:
  AWS_REGION: us-east-1
  TF_VERSION: 1.5.0  

jobs: 
  bootstrap:  
    runs-on: ubuntu-latest 

    defaults:
      run:
        working-directory: terraform-remote-state

    steps:
      - name: Checkout
        uses: actions/checkout@v4

      - name: Configure AWS credentials via OIDC
        uses: aws-actions/configure-aws-credentials@v4
        with:
          role-to-assume: ${{ secrets.AWS_ROLE_ARN }}
          aws-region: ${{ env.AWS_REGION }}
          role-session-name: GitHubActions-Bootstrap

      - name: Verify AWS identity
        run: |
          echo "Authenticated as:"
          aws sts get-caller-identity

          ACCOUNT_ID=$(aws sts get-caller-identity --query Account --output text)
          echo "AWS Account ID: $ACCOUNT_ID"
          echo "AWS Region: ${{ env.AWS_REGION }}"

      - name: Setup Terraform
        uses: hashicorp/setup-terraform@v3
        with:
          terraform_version: ${{ env.TF_VERSION }}

      - name: Terraform Init
        run: terraform init

      - name: Terraform Plan
        run: |
          terraform plan -out=plan.tfplan

      - name: Terraform Apply
        run: terraform apply -auto-approve plan.tfplan

      - name: Terraform Output
        run: terraform output

Step 7️: Run the Bootstrap Workflow

Go to GitHub Actions

Select Bootstrap

Click Run workflow

Step 8️: Store Terraform Outputs as GitHub Secrets

After completion, Terraform outputs values required by all future environments.
Store these as GitHub Secrets:
TF_STATE_BUCKET # S3 bucket name
TF_LOCK_TABLE # DynamoDB table name
KMS_KEY_ARN # KMS key ARN

Solving Frontend-Lambda Timeout Issues with AppSync Asynchronous Execution

Ismail G. — Sat, 29 Nov 2025 16:33:36 +0000

A common issue in serverless applications: the frontend receives a timeout error while CloudWatch logs show the Lambda function completed successfully. Users see failed requests, but backend operations succeed.

When a Lambda function is called synchronously, the API waits for it to complete and return a response. For long-running tasks, this might cause considerable delays.

Critical timeout constraints:

Layer	Maximum Timeout	Configurable
Lambda Function	15 minutes	Yes
API Gateway (REST)	29 seconds	No
AppSync (GraphQL)	30 seconds	No

The Solution: AppSync Asynchronous Lambda Execution

AWS AppSync provides asynchronous Lambda resolver support. Asynchronous execution lets a GraphQL mutation trigger a Lambda function without waiting for it to finish. The resolver returns immediately, bypassing the 30-second timeout limit.

With this pattern, the frontend is no longer tied to the duration of the Lambda execution. This enables long-running workflows to complete in the background.

```Before (Synchronous):
Frontend → "Start job" → Wait 30s → Timeout ❌
Lambda still running...

After (Asynchronous):
Frontend → "Start job" → Get job ID immediately ✅
Lambda runs independently → Updates result → Frontend gets notified ✅```

How It Works

When a GraphQL mutation is invoked with an async handler, AppSync invokes the Lambda function using Event invocation type (asynchronous mode). It returns a response—typically containing a job identifier—without waiting for Lambda completion.

The Lambda function then executes independently in the background. The frontend retrieves results through two methods:

Real-time updates: GraphQL subscriptions notify the client when data changes
Polling: Periodic GraphQL queries check job status at defined intervals

This architecture eliminates the 30-second AppSync resolver timeout limitation while maintaining a responsive user experience.

Implementation with AWS Amplify Gen 2

For Amplify applications using AppSync, AWS provides native support for asynchronous Lambda resolvers:

The frontend triggers a GraphQL mutation.
AppSync invokes the Lambda function in asynchronous mode and immediately returns a task reference.
The Lambda executes independently.
Results are written to a datastore.
The frontend retrieves results via follow-up GraphQL queries, or AppSync subscriptions (real-time updates).

AWS Documentation:

https://docs.amplify.aws/react/build-a-backend/data/custom-business-logic/#async-function-handlers

Get Hands-On with Amazon RDS Using AWS’s Getting Started Resource Center

Ismail G. — Sat, 02 Aug 2025 10:15:35 +0000

Understanding Amazon RDS (Relational Database Service) is essential for anyone seeking to gain expertise in cloud technology. You can't beat getting your hands on some real-world experience with managed databases, cloud-native application deployment, or even just learning the ropes of Amazon Web Services (AWS) certification.

Fortunately, the 'Getting Started Resource Center' on AWS provides a curated set of practical lessons tailored to RDS.

What is AWS RDS (Relational Database Service)

Amazon RDS is a managed relational database service provided by AWS (Amazon Web Services). Without worrying about the underlying infrastructure, users may quickly establish, operate, and scale databases in the cloud.

Launching and managing cloud-based relational databases is easy with Amazon RDS. It facilitates numerous engines, including MariaDB, SQL Server, PostgreSQL, and MySQL, and it hides numerous tedious processes, such as backups, scalability, replication, and patching.

Getting RDS knowledge empowers you with the ability to:

Database provisioning that is both secure and scalable

Redundancy and high availability

Tracking and automating performance

Managing a database instance and integrating third-party applications

Unless you have hands-on experience with database launch, connection, and management, these concepts may appear abstract. The value of AWS's practical guides becomes apparent in this context.

Hands-On RDS Tutorials Currently Available

As of now, AWS offers three dedicated hands-on labs for Amazon RDS, each addressing a key learning scenario:

Create and Connect to a MySQL Database

Ideal for beginners
Learn how to launch a MySQL RDS instance, configure access, and connect with a client
Free Tier–eligible

Create and Connect to a Microsoft SQL Server Database

Similar structure, but uses SQL Server as the database engine
Great for Windows-centric or enterprise developers
Learn connectivity, security, and basic DB management

Amazon RDS Backup & Restore Using AWS Backup

Learn how to create an on-demand backup job for an Amazon RDS database
Practice backup planning and restore workflows
Valuable for DevOps and system reliability engineers

Why AWS Hands-on Tutorials Are Valuable

While the number of RDS-related hands-on tutorials is currently limited, they cover core operational skills that are widely applicable:

Almost every cloud project requires database creation and connection. The availability of data in production settings depends on the backup and restore processes.

The ability to use Microsoft SQL Server configurations will give you basic information about managing other databases.

Don’t just read about RDS—build with it. Let's start with this:
https://aws.amazon.com/getting-started/hands-on/amazon-rds-backup-restore-using-aws-backup/?ref=gsrchandson&id=itprohandson

Using SSL with a PostgreSQL DB Instance

Ismail G. — Sun, 15 Jun 2025 13:22:17 +0000

Protecting any app that deals with sensitive information means making sure that it is safe while it is being sent. When you host PostgreSQL on Amazon RDS, it enables Secure Sockets Layer (SSL) connections.

This means that data transfers between your app and the database can be protected. This makes sure that private information is safe from being intercepted or changed while it is being sent.

This post will show you how to use SSL with a PostgreSQL DB instance on Amazon RDS, including what you need to do first, how to set it up, and the best ways to do so.

Enabling SSL on Your RDS PostgreSQL Instance

By default, Amazon RDS for PostgreSQL supports SSL. But to make SSL work and set up your client correctly, you need to do a few more things.

1. Check the SSL Configuration

Go to your RDS instance in the AWS Console and review the associated parameter group. If you are using PostgreSQL version 15 or newer, rds.force_ssl may be enforced by default.

Navigate to RDS > Databases > [your database] > Configuration

Open the linked Parameter group

Find the rds.force_ssl parameter:

If set to 1, SSL is required.
If set to 0, SSL is optional.

2. How to Enforce SSL in RDS (If SSL is not Enforced in RDS)

If rds.force_ssl parameter is 0 you must set it to 1. By default, parameter groups in AWS RDS are read-only and cannot be modified. Therefore, to enable rds.force_ssl = 1, you must create a custom parameter group.

Create a Custom Parameter Group:

By default, parameter groups in AWS RDS are read-only and cannot be modified. Therefore, to enable rds.force_ssl = 1, you must create a custom parameter group.

Go to RDS → Parameter groups → Click “Create parameter group”

Fill in the fields as follows:
- Parameter group family: postgres14
- Group name: custom-postgres14-ssl
- Description: Enable SSL for PostgreSQL 14
Click Create

Set rds.force_ssl = 1 in your new parameter group:

Select your newly created parameter group
Click “Edit parameters”

Search for rds.force_ssl and change its value from 0 ➝ 1
Click Save

Attach the custom parameter group to your RDS instance:

Go to RDS → Databases → Click your instance (database-1)
Click “Modify”

In the DB parameter group dropdown, select the custom group:
custom-postgres14-ssl

Scroll to the bottom and choose 'Apply immediately'.

Click “Continue” and then “Apply changes”

3. Download the AWS RDS Root Certificate

To establish a secure SSL connection, you must download the root certificate authority (CA) file from AWS.

You can find the latest region-specific certificates here:
Using SSL with Amazon RDS PostgreSQL

Connecting with SSL

Once you’ve downloaded the certificate, you can connect to your RDS PostgreSQL instance using several methods.

Using a GUI Tool (e.g., DBeaver)

In your tool of choice, create a new PostgreSQL connection:

Enter the RDS endpoint, port (5432), database name, and credentials.
Under SSL settings:
- Enable SSL (usually a checkbox).
- Set SSL Mode to require or verify-ca.
- Upload the RDS Root CA certificate you previously downloaded.

Using Terminal (psql CLI)

You can also connect securely via the terminal:

psql "host=mydb.xxxxxx.rds.amazonaws.com port=5432 dbname=mydb user=myuser password=mypass sslmode=verify-full sslrootcert=rds-ca-2019-root.pem"

sslmode=verify-full: Ensures both certificate and hostname validation.

sslrootcert: Path to the downloaded certificate file.

This configuration helps ensure both confidentiality and integrity of the data transmitted.

For more information and certificate downloads, refer to the official documentation:
Using SSL with Amazon RDS PostgreSQL

Migrating from MongoDB to Amazon DocumentDB

Ismail G. — Sat, 17 May 2025 12:22:46 +0000

Modern applications today often use document databases. For years, MongoDB has been the preferred choice for developers to build applications using JSON-like document data structures. However, a move to a fully managed service like Amazon DocumentDB is attractive when workloads increase.

Built from the ground up, Amazon DocumentDB (with MongoDB compatibility) is highly available, robust, and scalable. It supports common MongoDB drivers and tools, making it easy to move teams without changing application code. This article will guide you step-by-step through migrating data from MongoDB to Amazon DocumentDB using the AWS Database Migration Service (DMS).

Prerequisites

Before you begin the migration, make sure you have the following:

An Amazon DocumentDB cluster already created and available in your AWS account.

We will use AWS Database Migration Service (DMS) to migrate data from a MongoDB database to Amazon DocumentDB. This will work with minimal downtime and will not require the export and import of collections manually.

Create an AWS DMS Replication Instance for MongoDB Migration

The replication instance is responsible for the actual migration process. It establishes a connection to the source and target endpoints, extracts the data, transforms it (if necessary), and then inserts it into the destination.

Access the AWS DMS Console and navigate to the Replication instances section.

Select "Create replication instance."

Pick an instance identifier and select an instance class, such as dms.t3.medium.
Select the appropriate virtual private cloud (VPC). This must be the same VPC that your DocumentDB cluster refers to as its home.
Multi-AZ should be activated when exceptional availability is required.
Select the "Create" button and await the "Available" status of the instance.

Create Source and Target Endpoints for MongoDB Migration

Once the replication instance is ready, create source and target endpoints to define where the data will be moved from and to.

Source Endpoint (MongoDB):

Endpoint type: Source
Engine: MongoDB
Server name: MongoDB hostname or IP address
Port: 27017
Database name: your MongoDB database (e.g., zips-db)
Authentication mode: default
Username and password: credentials for your MongoDB instance

Target Endpoint (Amazon DocumentDB):

Endpoint type: Target
Engine: MongoDB
Server name: your DocumentDB cluster endpoint (e.g., mydocdbcluster.cluster-xxxxxx.docdb.amazonaws.com)
Port: 27017
Database name: target database name
Authentication: enter your DocumentDB admin username and password
TLS mode: verify-full
TLS CA file: upload global-bundle.pem

To add TLS CA file,first download the global CA certificate. You can find the TLS certificate download command directly in the Amazon DocumentDB console under your cluster's Connectivity & security tab.

After both endpoints are created, test the connections to verify that DMS can reach each database.

Create and Run a MongoDB Migration Task

Now, you can define and launch the migration task.

In the DMS Console, go to Database migration tasks and click Create task.

Choose a task identifier, and select your previously created replication instance.

For migration type, choose one of the following:

Migrate existing data only

Important: Turn off data validation. This feature is not supported for MongoDB endpoints.

Under Table mappings, click Add new selection rule (you must select at least one) :

Schema: your MongoDB database name (e.g., zips-db)

Source table name: %

Action: Include

Choose to start the task automatically on create.

Once created, the task will begin migrating your data to Amazon DocumentDB. You can monitor progress via the DMS console and view logs for detailed insights.

Wrapping Up

AWS DMS makes migrating from MongoDB to Amazon DocumentDB a simple task. Following the procedures outlined above will help you to reduce downtime and transfer your document-based workloads to a completely controlled environment that grows with your requirements. DocumentDB gives you the AWS security, scalability, and dependability advantages without compromising MongoDB compatibility.

This move can be the next step in developing your architecture if you are thinking about integration with other AWS services, operational simplicity, and long-term maintenance.

How to Set Up AWS EFS Static Provisioning Across Multiple Kubernetes Namespaces

Ismail G. — Fri, 11 Apr 2025 15:57:48 +0000

Bitnami PostgreSQL is a widely-used container image with the default being to run safely as a non-root user. But persistent storage—especially shared storage between environments such as dev and test—becomes a problem. Here in this blog post, I'll walk you through how I used AWS EFS static provisioning to share storage between two namespaces with Bitnami PostgreSQL running on Kubernetes.

Why Static Provisioning?

While dynamic provisioning is easy, static provisioning offers full control. It allows you to set a PersistentVolume (PV) by hand which corresponds with an AWS EFS File System or Access Point—ideal for environments where there are multiple environments (e.g., dev and test). As we’re using static provisioning, there’s no need to define a StorageClass for EFS.

Full control over PersistentVolume (PV) setup.
A way to reuse the same EFS volume across different namespaces.
Simpler debugging for permission or access issues.
No need to define a StorageClass

What We’re Building

A PostgreSQL setup running in two separate namespaces: dev and test

Both environments mount the same EFS volume
PostgreSQL data is shared

Prerequisites

Before you begin:

A running Kubernetes cluster (K3s, EKS, etc.)
An AWS EFS file system already created

Project Structure

Your repo should look like this:
deployment-files/
├── deployment-dev/
│ └── pv-dev.yml, pvc-dev.yml, postgres.yml
└── deployment-test/
└── pv-test.yml, pvc-test.yml, postgres*.yml

My GitLab repo: https://gitlab.com/samueldeniz80/aws-efs-static-provisioning-bitnami-postgresqls/-/tree/main/deployment-files?ref_type=heads

Step 1: Create EFS Access Point

To prevent permission issues when mounting EFS across namespaces, create an Access Point from the AWS Console with:

User ID: 1001
Group ID: 1001
Permissions: 0775

Install EFS CSI Driver in your cluster:

kubectl apply -k "github.com/kubernetes-sigs/aws-efs-csi-driver/deploy/kubernetes/overlays/stable/?ref=release-1.7"

You can also use Helm for EFS CSI Driver installation.

Step 2: Define the PV and PVC

Set your pv’s volumeHandle section with both fs file system id and access point id :

volumeHandle: fs-<file-system-id>::fsap-<access-point-id>

Leave storageClassName empty.

pv-dev.yml:

For pvc also leave storageClassName empty.

pvc-dev.yml:

Create PV and PVC as the same for the test namespace. The test namespace PV will also point to the same access point in the EFS.

Step 3: Configure PostgreSQL Deployment

Make sure the deployment uses fsGroup: 1001 in its securityContext to match EFS Access Point permissions:
securityContext:
fsGroup: 1001

Step 4: Deploy Namespaces

Deploy to dev:

kubectl create namespace dev
kubectl apply -f deployment-files/deployment-dev/ -n dev

Check the logs to verify that the PostgreSQL PV and PVC are bound, and the postgres pod is running.

Deploy to test:

kubectl create namespace test
kubectl apply -f deployment-files/deployment-test/ -n test

Check the logs to verify that the PostgreSQL PV and PVC are bound, and the postgres pod is running.

Outcome

You now have a shared EFS volume accessed by PostgreSQL pods running in different namespaces.

Automate Your Python API with AWS Lambda and EventBridge

Ismail G. — Sun, 23 Mar 2025 07:02:54 +0000

In situations when you want to conduct scheduled tasks without having to worry about the infrastructure, serverless architecture is an excellent option to consider. Over the course of this piece, I will demonstrate how I utilized Amazon Web Services Lambda and Amazon EventBridge to automate a Python-based API update script that runs on a regular schedule.

Use Case

Let's assume you have a website or app that requires you to pull information from a third-party API at regular intervals. The information could be prices, exchange rates, weather, or analytics results. You already have an existing Python script that functions to accomplish this upgrade.

Now, the objective is to execute this script on a regular basis without any need to work with any servers.

Step 1: Create Your Lambda Function

Your Python script needs to be prepared and packaged in such a way that it can be executed by AWS Lambda. This is the first stage.

Your working directory ought to have the following:

.
├── lambda_function.py
├── requirements.txt

lambda_function.py:
When you run a Python script on AWS Lambda, the conventional name assists AWS Lambda in identifying and running your function in the right way. Your primary Python script must be lambda_function.py.

Also you must define lambda_handler(event, context). The lambda_handler() function serves as an entry point for AWS Lambda and will directly be called when your function gets invoked.

requirements.txt:
Requirements.txt contains a list of all the dependencies that your code needs.

After you've structured your folder and written your Lambda function code, the next step is to package your Python script along with its dependencies. AWS Lambda does not automatically install Python packages from requirements.txt — you must include all dependencies in your deployment package:

Install dependencies locally into your project folder:

pip install -r requirements.txt -t .

Create a Deployment Package (ZIP):

zip -r lambda_shopify_update.zip .

Double check that you are zipping up the contents of the folder, not the folder itself. That is important — AWS needs to see the handler file (i.e., the lambda_function.py) in the root of the ZIP file.

Step 2. Upload to AWS Lambda

Now go to the AWS Console:

Open AWS Lambda
Click “Create function”
Choose “Upload from → .zip file”
Upload your lambda_shopify_update.zip
Test your lambda function

Step 3: Automating with EventBridge

Once your Lambda function is working correctly, the next step is to automate its execution using Amazon EventBridge. EventBridge is a serverless scheduler that lets you trigger your Lambda function on a regular schedule.

Go to the Amazon EventBridge section in the AWS Console.
Click “Create rule” to set up an automatic trigger for your Lambda function.
Under Rule type, choose “Schedule” to trigger your function based on time (rather than an event).
Select “Rate-based schedule” to run your Lambda at a regular interval.
Alternatively, you can use “Cron-based schedule” for more specific timing (e.g., every day at 08:00).

In the Target section:
- Choose AWS Lambda.
- Select your deployed Lambda function.
- Click Next.
Review your configuration, and click “Create rule”.

Your Lambda function is now scheduled to run automatically at the interval you defined — no manual execution needed.

Creating an AWS DMS Migration Task

Ismail G. — Sun, 09 Mar 2025 16:23:54 +0000

Migrating Data from Local SQL Server to AWS RDS PostgreSQL Using AWS DMS - II

In my last post, I guided you through building AWS Database Migration Service (DMS) to move data from an on-premises SQL Server to an AWS RDS PostgreSQL database instance. We went over establishing the AWS environment, configuring the source database, and building the required AWS DMS resources.

In this article, we will examine the migration process itself—target. With both source and target endpoints configured, the next step is to create a migration task in AWS DMS.

Creating and Running the AWS DMS Migration Task

With both the source (SQL Server) and target (PostgreSQL) endpoints configured (if not check the Migrating Data from Local SQL Server to AWS RDS PostgreSQL Using AWS DMS - I, it's time to create and execute a migration task.

Creating the Migration Task

1.Navigate to Database Migration Tasks:

Go to AWS DMS Console > Database migration tasks > Create task.

2.Configure Task Settings:

Task Identifier: Just give a name, for example, sqlserver-to-postgres-migration.
Replication instance: Select the replication instance configured earlier.
Source and target database: Choose the previously configured SQL Server source and PostgreSQL target endpoints.
Migration type: Select "Migrate" to perform a full load of the existing data.

3.Task Settings:

Editing mode: Wizard
Target table preparation mode: If you wish DMS to drop and replicate tables on the target, choose for "Drop tables on target". If you wish to maintain the current structure and data unaltered, then choose "Do nothing"; DMS just adds fresh entries without altering or deleting any current data. If you have pre-existing data that ought to be kept whole, this option is handy.
Include LOB columns in replication: Enable this option if your tables contain large object (LOB) data types.
Enable validation: When you enable this option, AWS DMS verifies the row counts and checksums of the source and target databases and confirms that they are equal.

4.Table Mapping and Transformations

Editing mode: Wizard
Selection rules: Define at least one selection rule to specify the schemas and tables you want to be included or excluded from the migration. Use % in the Source table name to include all tables.
Transformation rules (optional): Set transformation rules, if you need to rename schemas, tables, or columns.

5.Premigration assessment

A premigration assessment warns you of potential migration issues before starting your migration task. For this tutorial I skipped premigration assessment.

6.Running the Migration Task

Click Create task, then Start the migration.

AWS DMS will begin extracting data from SQL Server, transforming it, and loading it into PostgreSQL.

Migrating Data from Local SQL Server to AWS RDS PostgreSQL Using AWS DMS - I

Ismail G. — Thu, 27 Feb 2025 09:09:24 +0000

Data migration is an essential activity for those organizations that are moving from on-premises database technology to cloud offerings such as AWS RDS. An extremely helpful service that simplifies this task is the AWS Database Migration Service (DMS). With AWS DMS, for example, you can migrate data from an on-premises Microsoft SQL Server database to an AWS RDS PostgreSQL. I am going to explain the process of creating SQL Server parameters and configuring an AWS DMS replication instance in this post.

Preparing the SQL Server for AWS DMS

Before setting up AWS DMS, it is essential to configure your local SQL Server to allow external connections and ensure proper networking settings.

1. Enabling TCP/IP Connections in SQL Server

By default, SQL Server does not allow remote connections unless TCP/IP is explicitly enabled. Follow these steps to enable TCP/IP connections:

Open SQL Server Configuration Manager.

Navigate to SQL Server Network Configuration → Protocols for MSSQLSERVER.

Locate the TCP/IP protocol and right-click to select Enable.

Right-click on TCP/IP, select Properties, and navigate to the IP Addresses tab.

Under the IPAll section, set TCP Port to 1433 (leave TCP Dynamic Ports blank).

2. Configuring Windows Firewall Rules

After enabling TCP/IP, you need to allow inbound connections on port 1433 through Windows Firewall:

Open Windows Defender Firewall with Advanced Security.

Navigate to Inbound Rules → Add New Rule.

Select Port and click Next.

Choose TCP and enter 1433 in the Specific local ports field.

Click Next.

Choose 'Allow the connection'

Apply the rule to Domain, Private, and Public profiles.

Name the rule and complete the setup.

3. Restarting SQL Server

For the changes to take effect, restart the SQL Server service:

Open SQL Server Configuration Manager.

Select SQL Server Services.

Right-click SQL Server (MSSQLSERVER) and select Restart.

Once the SQL Server settings are configured, test the connection from another computer again using:

sqlcmd -S IP-Address -U Username -P Password

If you can connect successfully, your SQL Server is ready for migration.

Setting Up AWS DMS for Migration

Follow these steps to configure AWS DMS:

1. Creating a Replication Instance

Navigate to the AWS DMS Console.

Select Replication Instances and click Create Replication Instance.

Provide a name and choose an appropriate instance class.

Set the replication instance to public since the SQL Server is hosted on a local computer.

Security Group Configuration

DMS relies on security groups to control inbound and outbound traffic between the replication instance and databases. To properly configure the security group:

Navigate to EC2 Security Groups in the AWS Console.
Locate the security group assigned to the replication instance.
Add the following Inbound Rules:
- MSSQL (TCP/1433): Allow traffic from your local SQL Server’s IP or security group.
- PostgreSQL (TCP/5432): Allow traffic to the target AWS RDS PostgreSQL instance.
Add an Outbound Rule that allows all traffic to leave (egress) the VPC. This ensures communication from the replication instance to the source and target database endpoints.

2. Configuring Source Endpoint (SQL Server)

Choose Source Endpoint and enter SQL Server details.

Set the Endpoint Type to Source.

Enter the Server Name (IP Address), Port (1433), Username, and Password.

Click Test Connection and ensure it succeeds.

3. Configuring Target Endpoint (AWS RDS PostgreSQL)

Navigate to Endpoints and select Create Endpoint.

Choose Target Endpoint and enter AWS RDS PostgreSQL details.

Set the Endpoint Type to Target.

Enter the RDS Endpoint, Port (5432), Username, and Password.

Click Test Connection and verify success.

Database migration tasks

After making those settings you are ready to create a database migration task. I will cover this in my next blog post.

AWS Global Infrastructure: Components And Benefits

Ismail G. — Thu, 14 Dec 2023 11:04:41 +0000

The AWS Global Cloud Infrastructure stands as one of the most expansive cloud services, providing a comprehensive array of more than 200 fully-featured services accessible from data centers distributed across the world. Learning about the AWS Global Infrastructure opens the door to a modern world of cloud technology. Understanding this digital world becomes possible as you delve deeper into the many powerful tools and services that Amazon Web Services (AWS) offers worldwide.

The key components of the AWS Global Cloud Infrastructure are Regions, Availability Zones, Edge Locations, Wavelength Zones, and Regional Edge Caches. Exploring these key components provides a comprehensive understanding of how the network is structured and optimized for seamless performance and reliability.

In this blog post, we will explore AWS’s Global Infrastructure. If you’re a cloud enthusiast looking to earn AWS certifications like AWS Certified Cloud Practitioner or AWS Solution Architect, this is where you should begin your adventure.

What is AWS Global Infrastructure?

The AWS Global Infrastructure is a vast network of data centers and resources strategically located throughout the world to provide cloud services. The infrastructure comprises multiple interconnected components, such as Regions, Availability Zones, Local Zones, and Wavelength Zones.

AWS Global Infrastructure’s main component is data centers called Availability Zones (AZ). AZ’s are isolated locations within AWS Regions. The AWS Global Infrastructure Map currently includes 102 Availability Zones spread across 32 geographic regions globally. Moreover, plans are underway to expand with an additional 12 Availability Zones and 4 more AWS Regions, covering countries like Canada, Malaysia, New Zealand, and Thailand.

Additionally, AWS offers 35 Local Zones and 29 Wavelength Zones, catering to the needs of ultralow latency applications. Local Zones extend AWS services to select metropolitan areas, allowing for proximity to end-users and reduced latency. On the other hand, telecommunication providers collaborate in designing Wavelength Zones to bring AWS services even closer to 5G networks, thereby enhancing the performance of latency-sensitive applications.

What are the Key Components of AWS Global Infrastructure?

The AWS Global Infrastructure represents a highly distributed and robust ecosystem of data centers, ensuring global coverage, reliability, and performance for a wide array of cloud-based services and applications. Here is the 6 components of AWS Global Infrastructure:

Regions
Availability Zones (AZs)
Local Zones
Edge Locations
AWS Regional Edge Caches
Wavelength Zones

AWS Regions

An AWS Region is a distinct geographical area worldwide where a collection of data centers is clustered. This model incorporates multiple Availability Zones and physically and logically separate logical data center groups. Each AWS Region comprises at least three such Availability Zones, ensuring geographic redundancy and high availability.

AWS Regions and Availability Zones are not the same. AWS Regions are the larger geographical areas where data centers are located, while Availability Zones are subsets within those regions. We will discuss AZs deeply in the next part.

AWS operates in several geographic regions, including North America, South America, Europe, China, Asia Pacific, South Africa, and the Middle East.

AWS Availability Zones

People often refer to Availability Zones (AZs) in AWS as data center facilities. Each Availability Zone is a physically separated data center with its own power and infrastructure. AWS has strategically designed these Availability Zones to provide redundancy and resiliency for applications and services.

AWS Availability Zones have their own power, cooling, and networking infrastructure. Deploying resources across multiple Availability Zones ensures that if one AZ experiences issues or downtime, it enables the others to continue operating, thereby enhancing the overall resilience of applications hosted on the AWS platform.

Physical distances are important when comparing the distance between an AWS Region and its corresponding AZ. This separation helps minimize the impact of natural disasters or localized incidents. A significant distance physically separates Availability Zones in AWS. It’s important to note that while the distance between AZs is substantial, all AZs within a given AWS Region remain within 100 kilometers (approximately 60 miles) from each other.

Here is the full list of AWS data center locations in all regions:

Europe / Middle East / Africa
11 Geographic Regions
33 Availability Zones
39 Edge Network locations
2 Regional Edge Cache locations
Availability Zones:
Bahrain (3), Cape Town (3), Frankfurt (3), Ireland (3), London (3), Milan (3), Paris (3), Spain (3), Stockholm (3), Zurich (3), and UAE (3)

North America
7 Geographic Regions
25 Availability Zones
44 Edge Network locations
2 Regional Edge Cache locations
Availability Zones:
N. Virginia (6), Ohio (3), N. California (3), Oregon (4), US-East (3), US-West (3), Central (3)

South America
1 Geographic Regions
3 Availability Zones (in São Paulo)
4 Edge Network locations
1 Regional Edge Cache locations
Asia Pacific and China
12 Geographic Regions
38 Availability Zones
34 Edge Network locations
5 Regional Edge Cache locations

AWS Local Zones

AWS Local Zones are a part of the AWS Global infrastructure that brings cloud services locally to certain areas with a high concentration of users and applications. Essentially, these Local Zones form a subset of AWS Availability Zones, but they situate themselves in close proximity to specific metropolitan areas.

Local Zones allow customers to deploy applications that require low latency to end-users or specific resources in those geographic areas. They are particularly useful for applications that require real-time processing, such as gaming, interactive multimedia, and financial services, where reducing latency is critical for a seamless user experience.

AWS Local Zones Locations:

Local Zones are available in 33 metropolitan areas around the world, where 16 of them are located in the US (specific names not provided), and 17 of them are located outside of the US. Here are 17 Local Zones located outside the US:

Lagos
Auckland
Santiago
Bangkok
Kolkata
Copenhagen
Hamburg
Helsinki
Lima
Muscat
Perth
Querétaro
Taipei
Warsaw
Delhi
Buenos Aires
Manila
Additionally, AWS plans to broaden Local Zones to encompass 19 additional locations spanning 16 countries:

Australia
Austria
Belgium
Brazil
Canada
Colombia
Czech Republic
Germany
Greece
India
Kenya
Netherlands
Norway
Portugal
South Africa
Vietnam

AWS Edge Locations

AWS Edge Locations are endpoints for Amazon Web Services’ CloudFront service. They are strategically distributed data centers situated in various areas around the world. These Edge Locations act as caching servers that store copies of frequently accessed content, such as images, videos, web pages, and other static files, closer to the end users.

The Edge Location closest to the client receives requests for content distributed via CloudFront. This reduces latency and improves the overall performance of the application or website if the content already exists at that Edge Location. If the content still needs to be cached, the Edge Location retrieves it from the origin server (such as an Amazon S3 bucket or a custom origin) and caches it for subsequent requests.

AWS Regional Edge Caches

Like edge locations, Regional Edge Caches are CloudFront service sites strategically placed worldwide near your viewers. They deliver content directly to viewers between your origin server and global edge locations.

Regional edge caches assist with many sorts of material, especially with content that becomes less popular over time. Examples are user-generated material, e-commerce assets, news, and other content that may rapidly gain popularity.

Wavelength Zones

AWS Wavelength is an Amazon Web Services (AWS) service that brings cloud services to the edge of 5G networks. It supports real-time gaming, augmented reality (AR), virtual reality (VR), video streaming, and other applications that require ultra-low latency and high bandwidth connectivity.

A Wavelength Zone is essentially a specialized infrastructure deployment within a data center run by a telecommunications provider.

These zones are strategically placed in close proximity to 5G base stations, allowing for a direct connection between the AWS resources and the 5G network. The proximity minimizes latency for applications requiring instant responses.

With Wavelength, developers can run their applications and workloads on AWS infrastructure located at the edge of the 5G network, combining the benefits of both AWS services and high-speed, low-latency 5G connectivity.

What are the 6 benefits of AWS Global Infrastructure?

The AWS Global Infrastructure offers 6 key benefits to businesses and organizations that leverage its services, as mentioned on the AWS website: security, availability, performance, scalability, flexibility, and global footprint.

Security

AWS offers robust security measures, including encryption and constant monitoring. You maintain control over your data with encryption, movement, and retention options.

Availability

AWS ensures the utmost network availability among cloud providers. Each region is isolated and divided into multiple Availability Zones (AZs). If one AZ experiences an issue, other AZs can continue to operate without interruption.

Performance

Low latency, minimal packet loss, and high network quality are all characteristics of AWS Global Infrastructure.

Scalability

AWS enables flexible scaling of resources, eliminating over-provisioning. You can instantly adjust resources based on business needs, rapidly deploying hundreds or thousands of servers.

Flexibility

You have the freedom to choose how and where to run workloads, utilizing the same network, control plane, APIs, and services. Options include global AWS Regions, AWS Local Zones, AWS Wavelength for low latency, and AWS Outposts for on-premises deployment.

Global Footprint

AWS has an extensive global infrastructure presence. You can choose to select technology infrastructure close to your intended users, ensuring ideal assistance for a wide range of applications, from those requiring high throughput to those requiring low-latency performance.

These benefits empower businesses to build, scale, and operate applications efficiently while leveraging a secure and reliable global cloud infrastructure provided by AWS.

What are Essential AWS Certifications and Courses to Learn AWS Global Infrastructure?

To become proficient in understanding and working with AWS Global Infrastructure, you can pursue specific AWS certifications and courses that cover relevant concepts, technologies, and best practices. Here are the top 3 essential AWS certifications and courses that can help you learn about AWS Global Infrastructure: AWS Cloud Practitioner Certification, AWS Certified Solutions Architect – Associate, and AWS Certified Solutions Architect – Professional.

AWS Cloud Practitioner Certification

The AWS Certified Cloud Practitioner certification provides a solid overview of AWS services, pricing models, architecture, security, and the overall benefits of cloud computing. This certification is ideal for beginners who are new to cloud technology and want to grasp the fundamental concepts, services, and benefits of cloud computing.

The AWS Certified Cloud Practitioner certification is a great starting point for anyone looking to build a foundational understanding of Amazon Web Services and cloud computing concepts. It does not dive into the technical details of specific AWS services.

AWS Certified Solutions Architect – Associate

AWS Certified Solutions Architect – Associate certification covers a broad range of AWS services and architectural best practices, including designing and deploying applications on a global scale using multiple regions and Availability Zones.

AWS Certified Solutions Architect – Associate certification is ideal for individuals who want to showcase their expertise in architecting solutions using AWS services, including global infrastructure-related ones.

AWS Certified Solutions Architect – Professional

The AWS Certified Solutions Architect – Professional certification is an advanced-level certification offered by Amazon Web Services (AWS). AWS Cloud professionals with extensive experience and expertise in designing and deploying complex applications are ideal candidates for this program.

Building on the associate-level certification, AWS Certified Solutions Architect – Professional certification goes deeper into advanced architectural concepts, including designing highly available and fault-tolerant systems across a global infrastructure.

How to learn AWS?

AWS provides comprehensive documentation on its global infrastructure, including networking, regions, Availability Zones, and global services. Reading AWS whitepapers related to networking and global infrastructure is also highly recommended.

Remember that hands-on experience and practical projects are crucial to truly understanding and mastering AWS Global Infrastructure. Consider working on real-world projects, setting up multi-region architectures, and experimenting with AWS services to gain practical expertise. Moreover, remain on top of any new AWS courses that may emerge to enhance your knowledge of AWS Global Infrastructure further.

A Comprehensive Guide to AWS Databases

Ismail G. — Sat, 04 Nov 2023 15:50:20 +0000

In the digital age, data is the lifeblood of businesses and organizations. Storing, managing, and accessing this invaluable resource efficiently is essential for making informed decisions, improving operations, and delivering a seamless user experience.

Amazon Web Services (AWS) recognized this need and developed a robust solution – Amazon Relational Database Service (Amazon RDS) – to simplify database management. In this blog post, we'll explore AWS databases, with a focus on Amazon RDS, and discover how it can help you unlock the full potential of your data.

Understanding AWS Databases

AWS offers a variety of database services to cater to the diverse needs of businesses. These services are designed to address different use cases, from simple data storage to complex data analytics. Here are some key AWS database offerings:

Amazon RDS:

Amazon Relational Database Service (RDS) is a managed database service that supports various relational database engines like MySQL, PostgreSQL, Oracle, SQL Server, and MariaDB. It automates routine tasks like patching, backups, and scaling, allowing you to focus on your applications.

Amazon Aurora:

Amazon Aurora is a fully-managed, high-performance relational database service compatible with MySQL and PostgreSQL. It offers enhanced performance and availability, making it an excellent choice for critical applications.

Amazon DynamoDB:

DynamoDB is a fully-managed NoSQL database service designed for high availability and scalability. It's an ideal choice for applications that require seamless scaling with minimal administrative overhead.

Amazon Redshift:

Amazon Redshift is a fully-managed data warehousing service optimized for analytical workloads. It enables you to run complex queries and generate insights from your data.

Amazon Neptune:

Amazon Neptune is a fully-managed graph database service designed for applications that need to work with highly connected data, such as social networks and recommendation engines.

The Power of Amazon RDS

Amazon RDS is a standout among the AWS database services due to its versatility and ease of use. Here are some key benefits of Amazon RDS:

Managed Service:

With Amazon RDS, AWS takes care of the heavy lifting, such as database setup, maintenance, and backups. This frees you from time-consuming administrative tasks and lets you focus on developing your applications.

Multi-Engine Support:

Amazon RDS supports a variety of database engines, providing you with the flexibility to choose the one that best suits your application requirements.

High Availability:

Amazon RDS offers options for high availability, such as Multi-AZ deployments, which automatically replicate data to a standby instance in a different Availability Zone. This ensures minimal downtime and data durability.

Scalability:

Amazon RDS allows you to easily scale your database resources up or down as your application's needs change. You can adjust CPU, memory, and storage without disrupting your operations.

Security:

AWS provides robust security features, such as encryption at rest and in transit, automated software patching, and user authentication and authorization controls, to keep your data safe.

Monitoring and Metrics:

Amazon RDS offers built-in monitoring and alerting tools through Amazon CloudWatch. You can track performance, set alarms, and gain insights into your database's behavior.

Cost Efficiency:

Amazon RDS's pay-as-you-go pricing model allows you to pay only for the resources you use, optimizing costs while ensuring optimal performance.

Conclusion

Amazon RDS is just one piece of the AWS database puzzle, but it plays a crucial role in simplifying database management and maximizing the potential of your data. Whether you are running a small web application or managing a complex enterprise system, AWS databases provide a scalable and reliable foundation for your data needs.

Embracing these services can empower your organization with the ability to make data-driven decisions, enhance customer experiences, and stay competitive in an ever-evolving digital landscape.