DEV Community: abdelino17

How to Stay Motivated When Preparing for a Cloud Certification?

abdelino17 — Tue, 16 Sep 2025 08:00:00 +0000

Preparing for a cloud certification is often a solo journey.
After passing 20+ certs, here’s what I’ve learned to stay motivated along the way:

1. Start with your WHY

Why do you really want this certification?

➡️ A new job opportunity?

➡️ Building new skills?

➡️ Personal challenge?

If your only reason is “because my colleague just passed it,” you’ll probably lose steam quickly.

A strong WHY is what will keep you moving when motivation dips.

2. Set a clear deadline

Saying “I’ll pass this exam this year” is too vague. I’ve seen people repeat it for 3 years in a row.

Instead, be specific: “I’ll pass this exam within 3 months — before November 30.”

Personally, I love buying exam vouchers that expire soon. It forces me to stay disciplined.

3. Choose the right resources

Sometimes it’s not you—it’s the material you’re using. There are tons of courses out there, but the key is to pick the right ones. A well-structured, hands-on course makes it much easier to stay consistent.

🎯 My advice: stick to 2–3 solid resources max. More than that, and you’ll just lose focus.

Now your turn — what are your strategies to stay motivated when preparing for a cloud certification?

Why you should consider the AWS AI Practitioner Certification?

abdelino17 — Sat, 13 Sep 2025 08:00:00 +0000

Last week, someone asked me if they should take the AWS Generative AI Practitioner exam.

Here’s my take — and why you should consider it if you’re working in tech.

1. Generative AI is the biggest revolution since the Internet

It’s disrupting jobs everywhere, and we’re still just at the beginning.

If you’re a builder, you can’t afford to stay a passive chatbot user. You need to understand:

What LLMs are
How tokens, temperature, and context work
How to use them in modern applications

This certification gives you all the fundamental concepts. The goal isn’t to become an expert, but to know enough to build smarter workflows and solutions. LLMs are here to stay — for better or worse — so it’s up to us to adapt and leverage them.

2. “Just start it”

I love the Nike slogan “Just do it”. Let’s tweak it a little: “Just start it.”

Preparing for this cert gives you real appetite for Generative AI. It’s an excellent entry point into advanced topics like MCPs, AI Agents, and Agentic Systems.

If you’re thinking about shifting your career toward AI/ML, this exam is the perfect challenge to get you started.

3. Practitioner level — but not as easy as you think

This exam goes deeper than the AWS Cloud Practitioner. The scope is narrower but more technical. Expect many questions on AWS Data Services (especially SageMaker).

With solid preparation, you can definitely pass on your first try — but don’t underestimate it.

✅ If you’re in tech and curious about the AI wave, this cert is absolutely worth it.

💬 What do you think — will you take the plunge into the GenAI Practitioner?

Why the AWS Solutions Architect Associate Should Be Your First AWS Certification?

abdelino17 — Wed, 10 Sep 2025 16:00:00 +0000

Whenever someone asks me which AWS certification to start with, my answer is always the same: AWS Certified Solutions Architect – Associate.

Here are 3 technical reasons why I recommend it, especially for junior cloud engineers:

1. Solid Understanding of AWS Core Services

AWS offers over 200 services, but you only need about 20% of them to cover 80% of real-world use cases (pareto principle).

This certification focuses on the essentials — EC2, S3, VPC, IAM, RDS, Lambda, and more. Mastering these will give you the foundation to build, troubleshoot, and optimize cloud solutions from day one.

2. Practical Experience With Design Patterns and Best Practices

You’ll learn how to architect scalable, resilient, and secure solutions using AWS-recommended frameworks. Expect to work with networking, storage, compute, and security configurations — skills that apply directly to real projects.

3. The Perfect Entry Point into AWS Certifications

It’s neither too easy nor too difficult — just the right mix to challenge yourself while being achievable with proper preparation.

With the right resources, you can pass it in a few weeks (sometimes even days).

Ready to start? Here are my top course recommendations:

📘 [French] LeCloudFacile.com – Clear explanations for beginners

🎯 [English] Stéphane Maarek’s AWS Certified Solutions Architect – Associate course on Udemy

What was your first AWS certification, and would you choose the same path again?

DEVS HATE NETWORKING (But You Need It!)

abdelino17 — Mon, 21 Jul 2025 18:07:24 +0000

Back in my bachelor’s degree, almost every developer I knew avoided networking. Can you blame us? Terms like Class A/B/C/D/E, Ethernet, BGP, network prefix, subnet mask sound more confusing than helpful.

If you’re a developer, here are five core networking concepts that will take your career to the next level:

1. CIDR Blocks

What it is: A notation that defines IP ranges using a prefix (e.g., /24).

Examples:

1.2.3.4/32 = just 1.2.3.4 (1 IP)
10.0.0.0/24 = 10.0.0.1 → 10.0.0.255 (254 useful IPs)
10.0.0.0/16 = 10.0.0.1 → 10.0.255.255 (~65,534 IPs)
10.0.0.0/8 = 10.0.0.1 → 10.255.255.255 (~16 million IPs)

2. Private IP vs. Public IP

Public IP: Routable on the internet — anyone can reach it (unless blocked by a firewall).

Private IP: Only accessible within a local network.

Reserved Ranges:

10.0.0.0/8
172.16.0.0/12
192.168.0.0/16

Other Special Ranges:

127.0.0.0/8 = Localhost
169.254.0.0/16 = Link-local (APIPA)
100.64.0.0/10 = Carrier-grade NAT (mobile operator, VPN)

Understanding which IP range to use prevents nasty routing and security surprises.

3. DNS (Domain Name System)

Think of DNS as the internet’s phonebook — translating human-friendly domain names into IP addresses. Key record types you should know:

A = maps a domain to an IPv4 address
CNAME = alias to another domain name
MX = mail server record
TXT = arbitrary text (e.g., SPF, DKIM, validation)

If you can’t resolve a hostname, most things in your stack simply won’t work.

4. TCP/IP Model Essentials

TCP (Transmission Control Protocol): A reliable, connection-oriented transport protocol. Used for most web traffic (HTTP, HTTPS, database connections).
ICMP (Internet Control Message Protocol): Used by tools like ping or traceroute to test connectivity and path.
HTTP/HTTPS: Application-layer protocols for web APIs, websites, and microservices.

Knowing when to use TCP vs. UDP vs. ICMP helps you debug connectivity issues quickly.

5. Load Balancers

Purpose: Distribute incoming traffic across multiple servers or containers to ensure reliability and scalability.

Common Example: Nginx (can act as a reverse proxy and load balancer).

Cloud-Native Alternatives: AWS Elastic Load Balancer (ALB/NLB), Google Cloud Load Balancer, etc.

A solid grasp of load balancing means you can design resilient, high-traffic architectures instead of firefighting bottlenecks.

How docker build Really Works?

abdelino17 — Mon, 21 Jul 2025 17:45:02 +0000

If you run docker build, you need to understand what's happening under the hood. It’s not magic — it's a clever use of containers themselves.

Here’s the breakdown 👇

What really happens during `docker build`?

When Docker builds an image, it reads your Dockerfile line by line. But here’s the key: not every instruction creates a new layer.

1. For instructions that change the filesystem (RUN, COPY, ADD):
Docker performs a three-step process for each one:

It spins up a temporary container using the image from the previous step.
It runs the command (e.g., RUN apt-get update) or copies the files inside that container.
It commits the result as a new image layer that contains only the filesystem changes.

2. For instructions that only change metadata (WORKDIR, ENV, CMD, EXPOSE):

No container is created. Docker simply updates the image's metadata (the config.json file) to be used by containers later on. These steps are incredibly fast because they don't involve creating layers.

Your final image is an ordered stack of these layers plus a final configuration.

So, how can a container run without an image?

To run a container, a low-level runtime like runc just needs an OCI bundle:

📁 A root filesystem (a directory with all the necessary files).
📝 A config.json file (specifying what command to run, environment variables, etc.).

That's it. A Docker image is essentially a convenient, portable way to package and distribute this bundle.

The Twist 💡

So, to build an image, Docker runs a series of temporary containers. But to run a container, you don't actually need an image — just the final root filesystem and config file.

When Docker builds an image, it needs containers to run each instruction and commit the result as a new image layer.

But when you have a bundle ready, you can run a container without ever building a Docker image.

Understanding how docker build works under the hood:

Helps you debug weird build issues
Helps you order Dockerfile instructions for better caching
Explains why multi-stage builds are so powerful
Makes you appreciate the layered, file-system-based model of containers

Next time you type docker build, remember: you're running containers to build other containers.

Inspired by the excellent iximiuz article: You need containers to build an image.

How to deploy a SpringBoot API on AWS ECS using CDKTF?

abdelino17 — Fri, 24 Jan 2025 00:18:40 +0000

When a Java developer asked me how to deploy their Spring Boot API on AWS ECS, I saw it as the perfect chance to dive into the latest updates on the CDKTF (Cloud Development Kit for Terraform) project.

In a previous article, I introduced CDKTF, a framework that allows you to write Infrastructure as Code (IaC) using general-purpose programming languages such as Python. Since then, CDKTF has reached its first GA release, making it the perfect time to revisit it. In this article, we’ll walk through deploying a Spring Boot API on AWS ECS using CDKTF.

Find the code of this article on my github repo.

Architecture Overview

Before diving into the implementation, let’s review the architecture we aim to deploy:

From this diagram, we can break down the architecture into 03 layers:

Network:
- VPC
- Public and private subnets
- Internet Gateway
- NAT Gateways
Infrastructure:
- Application Load Balancer (ALB)
- Listeners
- ECS Cluster
Service Stack:
- Target Groups
- ECS Service
- Task Definitions

Step 1: Containerize your Spring Boot Application

The Java API we’re deploying is available on GitHub.

It defines a simple REST API with three endpoints:

/ping: Returns the string "pong". This endpoint is useful for testing the API's responsiveness. It also increments a Prometheus counter metric for monitoring.
/healthcheck: Returns "ok", serving as a health check endpoint to ensure the application is running correctly. Like /ping, it updates a Prometheus counter for observability.
/hello: Accepts a name query parameter (defaults to "World") and returns a personalized greeting, e.g., "Hello, [name]!". This endpoint also integrates with the Prometheus counter.

Let’s add the Dockerfile:

FROM maven:3.9-amazoncorretto-21 AS builder

WORKDIR /app

COPY pom.xml .

COPY src src

RUN mvn clean package

# amazon java distribution
FROM amazoncorretto:21-alpine

COPY --from=builder /app/target/*.jar /app/java-api.jar

EXPOSE 8080

ENTRYPOINT ["java","-jar","/app/java-api.jar"]

Our application is ready to be deployed!

Step 2: Set up AWS CDKTF

AWS CDKTF allows you to define and manage AWS resources using Python.

1. Prerequisites

- [**python (3.13)**](https://www.python.org/)
- [**pipenv**](https://pipenv.pypa.io/en/latest/)
- [**npm**](https://nodejs.org/en/)

2. Install CDKTF and Dependencies

Ensure you have the necessary tools by installing CDKTF and its dependencies:

$ npm install -g cdktf-cli@latest

This installs the cdktf CLI that allows to spin up new projects for various languages.

3. Initialize Your CDKTF Application

We can scaffold a new python project by running:

# init the project using aws provider
$ mkdir samples-fargate

$ cd samples-fargate && cdktf init --template=python --providers=aws

There are many files created by default and all the dependencies are installed.

Below is the initial main.pyfile:

#!/usr/bin/env python
from constructs import Construct
from cdktf import App, TerraformStack

class MyStack(TerraformStack):
    def __init__(self, scope: Construct, id: str):
        super().__init__(scope, id)

        # define resources here

app = App()
MyStack(app, "aws-cdktf-samples-fargate")

app.synth()

Step 3: Building Layers

A stack represents a group of infrastructure resources that CDK for Terraform (CDKTF) compiles into a distinct Terraform configuration. Stacks enable separate state management for different environments within an application. To share resources across layers, we will utilize Cross-Stack references.

1. Network Layer

Add the network_stack.py file to your project

$ mkdir infra

$ cd infra && touch network_stack.py

Add the following code to create all the network resources:

from constructs import Construct
from cdktf import S3Backend, TerraformStack
from cdktf_cdktf_provider_aws.provider import AwsProvider
from cdktf_cdktf_provider_aws.vpc import Vpc
from cdktf_cdktf_provider_aws.subnet import Subnet
from cdktf_cdktf_provider_aws.eip import Eip
from cdktf_cdktf_provider_aws.nat_gateway import NatGateway
from cdktf_cdktf_provider_aws.route import Route
from cdktf_cdktf_provider_aws.route_table import RouteTable
from cdktf_cdktf_provider_aws.route_table_association import RouteTableAssociation
from cdktf_cdktf_provider_aws.internet_gateway import InternetGateway

class NetworkStack(TerraformStack):
    def __init__(self, scope: Construct, ns: str, params: dict):
        super().__init__(scope, ns)

        self.region = params["region"]

        # configure the AWS provider to use the us-east-1 region
        AwsProvider(self, "AWS", region=self.region)

        # use S3 as backend
        S3Backend(
            self,
            bucket=params["backend_bucket"],
            key=params["backend_key_prefix"] + "/network.tfstate",
            region=self.region,
        )

        # create the vpc
        vpc_demo = Vpc(self, "vpc-demo", cidr_block="192.168.0.0/16")

        # create two public subnets
        public_subnet1 = Subnet(
            self,
            "public-subnet-1",
            vpc_id=vpc_demo.id,
            availability_zone=f"{self.region}a",
            cidr_block="192.168.1.0/24",
        )

        public_subnet2 = Subnet(
            self,
            "public-subnet-2",
            vpc_id=vpc_demo.id,
            availability_zone=f"{self.region}b",
            cidr_block="192.168.2.0/24",
        )

        # create. the internet gateway
        igw = InternetGateway(self, "igw", vpc_id=vpc_demo.id)

        # create the public route table
        public_rt = Route(
            self,
            "public-rt",
            route_table_id=vpc_demo.main_route_table_id,
            destination_cidr_block="0.0.0.0/0",
            gateway_id=igw.id,
        )

        # create the private subnets
        private_subnet1 = Subnet(
            self,
            "private-subnet-1",
            vpc_id=vpc_demo.id,
            availability_zone=f"{self.region}a",
            cidr_block="192.168.10.0/24",
        )

        private_subnet2 = Subnet(
            self,
            "private-subnet-2",
            vpc_id=vpc_demo.id,
            availability_zone=f"{self.region}b",
            cidr_block="192.168.20.0/24",
        )

        # create the Elastic IPs
        eip1 = Eip(self, "nat-eip-1", depends_on=[igw])
        eip2 = Eip(self, "nat-eip-2", depends_on=[igw])

        # create the NAT Gateways
        private_nat_gw1 = NatGateway(
            self,
            "private-nat-1",
            subnet_id=public_subnet1.id,
            allocation_id=eip1.id,
        )

        private_nat_gw2 = NatGateway(
            self,
            "private-nat-2",
            subnet_id=public_subnet2.id,
            allocation_id=eip2.id,
        )

        # create Route Tables
        private_rt1 = RouteTable(self, "private-rt1", vpc_id=vpc_demo.id)
        private_rt2 = RouteTable(self, "private-rt2", vpc_id=vpc_demo.id)

        # add default routes to tables
        Route(
            self,
            "private-rt1-default-route",
            route_table_id=private_rt1.id,
            destination_cidr_block="0.0.0.0/0",
            nat_gateway_id=private_nat_gw1.id,
        )

        Route(
            self,
            "private-rt2-default-route",
            route_table_id=private_rt2.id,
            destination_cidr_block="0.0.0.0/0",
            nat_gateway_id=private_nat_gw2.id,
        )

        # associate routes with subnets
        RouteTableAssociation(
            self,
            "public-rt-association",
            subnet_id=private_subnet2.id,
            route_table_id=private_rt2.id,
        )

        RouteTableAssociation(
            self,
            "private-rt1-association",
            subnet_id=private_subnet1.id,
            route_table_id=private_rt1.id,
        )

        RouteTableAssociation(
            self,
            "private-rt2-association",
            subnet_id=private_subnet2.id,
            route_table_id=private_rt2.id,
        )

        # terraform outputs
        self.vpc_id = vpc_demo.id
        self.public_subnets = [public_subnet1.id, public_subnet2.id]
        self.private_subnets = [private_subnet1.id, private_subnet2.id]

Then, edit the main.py file:

#!/usr/bin/env python
from constructs import Construct
from cdktf import App, TerraformStack
from infra.network_stack import NetworkStack

ENV = "dev"
AWS_REGION = "us-east-1"
BACKEND_S3_BUCKET = "blog.abdelfare.me"
BACKEND_S3_KEY = f"{ENV}/cdktf-samples"

class MyStack(TerraformStack):
    def __init__(self, scope: Construct, id: str):
        super().__init__(scope, id)

        # define resources here

app = App()
MyStack(app, "aws-cdktf-samples-fargate")

network = NetworkStack(
    app,
    "network",
    {
        "region": AWS_REGION,
        "backend_bucket": BACKEND_S3_BUCKET,
        "backend_key_prefix": BACKEND_S3_KEY,
    },
)

app.synth()

Generate the terraform configuration files by running the following command:

$ cdktf synth

Deploy the network stack with this:

$ cdktf deploy network

Our VPC is ready as shown in the image below:

2. Infrastructure Layer

Add the infra_stack.py file to your project

$ cd infra && touch infra_stack.py

Add the following code to create all the infrastructure resources:

from constructs import Construct
from cdktf import S3Backend, TerraformStack
from cdktf_cdktf_provider_aws.provider import AwsProvider
from cdktf_cdktf_provider_aws.ecs_cluster import EcsCluster
from cdktf_cdktf_provider_aws.lb import Lb
from cdktf_cdktf_provider_aws.lb_listener import (
    LbListener,
    LbListenerDefaultAction,
    LbListenerDefaultActionFixedResponse,
)
from cdktf_cdktf_provider_aws.security_group import (
    SecurityGroup,
    SecurityGroupIngress,
    SecurityGroupEgress,
)

class InfraStack(TerraformStack):
    def __init__(self, scope: Construct, ns: str, network: dict, params: dict):
        super().__init__(scope, ns)

        self.region = params["region"]

        # Configure the AWS provider to use the us-east-1 region
        AwsProvider(self, "AWS", region=self.region)

        # use S3 as backend
        S3Backend(
            self,
            bucket=params["backend_bucket"],
            key=params["backend_key_prefix"] + "/load_balancer.tfstate",
            region=self.region,
        )

        # create the ALB security group
        alb_sg = SecurityGroup(
            self,
            "alb-sg",
            vpc_id=network["vpc_id"],
            ingress=[
                SecurityGroupIngress(
                    protocol="tcp", from_port=80, to_port=80, cidr_blocks=["0.0.0.0/0"]
                )
            ],
            egress=[
                SecurityGroupEgress(
                    protocol="-1", from_port=0, to_port=0, cidr_blocks=["0.0.0.0/0"]
                )
            ],
        )

        # create the ALB
        alb = Lb(
            self,
            "alb",
            internal=False,
            load_balancer_type="application",
            security_groups=[alb_sg.id],
            subnets=network["public_subnets"],
        )

        # create the LB Listener
        alb_listener = LbListener(
            self,
            "alb-listener",
            load_balancer_arn=alb.arn,
            port=80,
            protocol="HTTP",
            default_action=[
                LbListenerDefaultAction(
                    type="fixed-response",
                    fixed_response=LbListenerDefaultActionFixedResponse(
                        content_type="text/plain",
                        status_code="404",
                        message_body="Could not find the resource you are looking for",
                    ),
                )
            ],
        )

        # create the ECS cluster
        cluster = EcsCluster(self, "cluster", name=params["cluster_name"])

        self.alb_arn = alb.arn
        self.alb_listener = alb_listener.arn
        self.alb_sg = alb_sg.id
        self.cluster_id = cluster.id

Edit the main.py file:

...

CLUSTER_NAME = "cdktf-samples"
...

infra = InfraStack(
    app,
    "infra",
    {
        "vpc_id": network.vpc_id,
        "public_subnets": network.public_subnets,
    },
    {
        "region": AWS_REGION,
        "backend_bucket": BACKEND_S3_BUCKET,
        "backend_key_prefix": BACKEND_S3_KEY,
        "cluster_name": CLUSTER_NAME,
    },
)
...

Deploy the infra stack with this:

$ cdktf deploy network infra

Note the DNS name of the ALB, we will use it later.

3. Service Layer

Add the service_stack.py file to your project

$ mkdir apps

$ cd apps && touch service_stack.py

Add the following code to create all the ECS Service resources:

from constructs import Construct
import json
from cdktf import S3Backend, TerraformStack, Token, TerraformOutput
from cdktf_cdktf_provider_aws.provider import AwsProvider
from cdktf_cdktf_provider_aws.ecs_service import (
    EcsService,
    EcsServiceLoadBalancer,
    EcsServiceNetworkConfiguration,
)
from cdktf_cdktf_provider_aws.ecr_repository import (
    EcrRepository,
    EcrRepositoryImageScanningConfiguration,
)
from cdktf_cdktf_provider_aws.ecr_lifecycle_policy import EcrLifecyclePolicy
from cdktf_cdktf_provider_aws.ecs_task_definition import (
    EcsTaskDefinition,
)
from cdktf_cdktf_provider_aws.lb_listener_rule import (
    LbListenerRule,
    LbListenerRuleAction,
    LbListenerRuleCondition,
    LbListenerRuleConditionPathPattern,
)
from cdktf_cdktf_provider_aws.lb_target_group import (
    LbTargetGroup,
    LbTargetGroupHealthCheck,
)
from cdktf_cdktf_provider_aws.security_group import (
    SecurityGroup,
    SecurityGroupIngress,
    SecurityGroupEgress,
)
from cdktf_cdktf_provider_aws.cloudwatch_log_group import CloudwatchLogGroup
from cdktf_cdktf_provider_aws.data_aws_iam_policy_document import (
    DataAwsIamPolicyDocument,
)
from cdktf_cdktf_provider_aws.iam_role import IamRole
from cdktf_cdktf_provider_aws.iam_role_policy_attachment import IamRolePolicyAttachment

class ServiceStack(TerraformStack):
    def __init__(
        self, scope: Construct, ns: str, network: dict, infra: dict, params: dict
    ):
        super().__init__(scope, ns)

        self.region = params["region"]

        # Configure the AWS provider to use the us-east-1 region
        AwsProvider(self, "AWS", region=self.region)

        # use S3 as backend
        S3Backend(
            self,
            bucket=params["backend_bucket"],
            key=params["backend_key_prefix"] + "/" + params["app_name"] + ".tfstate",
            region=self.region,
        )

        # create the service security group
        svc_sg = SecurityGroup(
            self,
            "svc-sg",
            vpc_id=network["vpc_id"],
            ingress=[
                SecurityGroupIngress(
                    protocol="tcp",
                    from_port=params["app_port"],
                    to_port=params["app_port"],
                    security_groups=[infra["alb_sg"]],
                )
            ],
            egress=[
                SecurityGroupEgress(
                    protocol="-1", from_port=0, to_port=0, cidr_blocks=["0.0.0.0/0"]
                )
            ],
        )

        # create the service target group
        svc_tg = LbTargetGroup(
            self,
            "svc-target-group",
            name="svc-tg",
            port=params["app_port"],
            protocol="HTTP",
            vpc_id=network["vpc_id"],
            target_type="ip",
            health_check=LbTargetGroupHealthCheck(path="/ping", matcher="200"),
        )

        # create the service listener rule
        LbListenerRule(
            self,
            "alb-rule",
            listener_arn=infra["alb_listener"],
            action=[LbListenerRuleAction(type="forward", target_group_arn=svc_tg.arn)],
            condition=[
                LbListenerRuleCondition(
                    path_pattern=LbListenerRuleConditionPathPattern(values=["/*"])
                )
            ],
        )

        # create the ECR repository
        repo = EcrRepository(
            self,
            params["app_name"],
            image_scanning_configuration=EcrRepositoryImageScanningConfiguration(
                scan_on_push=True
            ),
            image_tag_mutability="MUTABLE",
            name=params["app_name"],
        )

        EcrLifecyclePolicy(
            self,
            "this",
            repository=repo.name,
            policy=json.dumps(
                {
                    "rules": [
                        {
                            "rulePriority": 1,
                            "description": "Keep last 10 images",
                            "selection": {
                                "tagStatus": "tagged",
                                "tagPrefixList": ["v"],
                                "countType": "imageCountMoreThan",
                                "countNumber": 10,
                            },
                            "action": {"type": "expire"},
                        },
                        {
                            "rulePriority": 2,
                            "description": "Expire images older than 3 days",
                            "selection": {
                                "tagStatus": "untagged",
                                "countType": "sinceImagePushed",
                                "countUnit": "days",
                                "countNumber": 3,
                            },
                            "action": {"type": "expire"},
                        },
                    ]
                }
            ),
        )

        # create the service log group
        service_log_group = CloudwatchLogGroup(
            self,
            "svc_log_group",
            name=params["app_name"],
            retention_in_days=1,
        )

        ecs_assume_role = DataAwsIamPolicyDocument(
            self,
            "assume_role",
            statement=[
                {
                    "actions": ["sts:AssumeRole"],
                    "principals": [
                        {
                            "identifiers": ["ecs-tasks.amazonaws.com"],
                            "type": "Service",
                        },
                    ],
                },
            ],
        )

        # create the service execution role
        service_execution_role = IamRole(
            self,
            "service_execution_role",
            assume_role_policy=ecs_assume_role.json,
            name=params["app_name"] + "-exec-role",
        )

        IamRolePolicyAttachment(
            self,
            "ecs_role_policy",
            policy_arn="arn:aws:iam::aws:policy/service-role/AmazonECSTaskExecutionRolePolicy",
            role=service_execution_role.name,
        )

        # create the service task role
        service_task_role = IamRole(
            self,
            "service_task_role",
            assume_role_policy=ecs_assume_role.json,
            name=params["app_name"] + "-task-role",
        )

        # create the service task definition
        task = EcsTaskDefinition(
            self,
            "svc-task",
            family="service",
            network_mode="awsvpc",
            requires_compatibilities=["FARGATE"],
            cpu="256",
            memory="512",
            task_role_arn=service_task_role.arn,
            execution_role_arn=service_execution_role.arn,
            container_definitions=json.dumps(
                [
                    {
                        "name": "svc",
                        "image": f"{repo.repository_url}:latest",
                        "networkMode": "awsvpc",
                        "healthCheck": {
                            "Command": ["CMD-SHELL", "echo hello"],
                            "Interval": 5,
                            "Timeout": 2,
                            "Retries": 3,
                        },
                        "portMappings": [
                            {
                                "containerPort": params["app_port"],
                                "hostPort": params["app_port"],
                            }
                        ],
                        "logConfiguration": {
                            "logDriver": "awslogs",
                            "options": {
                                "awslogs-group": service_log_group.name,
                                "awslogs-region": params["region"],
                                "awslogs-stream-prefix": params["app_name"],
                            },
                        },
                    }
                ]
            ),
        )

        # create the ECS service
        EcsService(
            self,
            "ecs_service",
            name=params["app_name"] + "-service",
            cluster=infra["cluster_id"],
            task_definition=task.arn,
            desired_count=params["desired_count"],
            launch_type="FARGATE",
            force_new_deployment=True,
            network_configuration=EcsServiceNetworkConfiguration(
                subnets=network["private_subnets"],
                security_groups=[svc_sg.id],
            ),
            load_balancer=[
                EcsServiceLoadBalancer(
                    target_group_arn=svc_tg.id,
                    container_name="svc",
                    container_port=params["app_port"],
                )
            ],
        )

        TerraformOutput(
            self,
            "ecr_repository_url",
            description="url of the ecr repo",
            value=repo.repository_url,
        )

Update the main.py (for the last time 😁):

...

java_api = ServiceStack(
    app,
    "java_api",
    {
        "vpc_id": network.vpc_id,
        "private_subnets": network.private_subnets,
    },
    {
        "alb_sg": infra.alb_sg,
        "alb_listener": infra.alb_listener,
        "cluster_id": infra.cluster_id,
    },
    {
        "region": AWS_REGION,
        "backend_bucket": BACKEND_S3_BUCKET,
        "backend_key_prefix": BACKEND_S3_KEY,
        "app_name": "java-api",
        "app_port": 8080,
        "desired_count": 1,
    },
)
...

Deploy the service stack with this:

$ cdktf deploy network infra service

Here we go!

We successfully created all the resources to deploy a new service on AWS ECS Fargate.

Run the following to get the list of your stacks

$ cdktf list

Step 4: Github Actions Workflow

To automate deployments, let’s integrate a GitHub Actions workflow to our java-api. After enabling Github Actions, setting the secrets and variables for your repository, create the .github/workflows/deploy.yml file and add the content below:

name: Java API deployment

on:
  workflow_dispatch:

  push:
    branches:
      - main

  pull_request:
    types:
      - opened
      - reopened
    branches:
      - main

concurrency:
  group: ${{ github.workflow }}-${{ github.ref }}
  cancel-in-progress: true

jobs:
  test:
    name: Build job
    runs-on: ubuntu-latest
    # concurrency:
    #   group: ${{ github.job }}
    steps:
      - name: Checkout code
        uses: actions/checkout@v4

      - name: Set up JDK 21
        uses: actions/setup-java@v4
        with:
          distribution: corretto
          java-version: 21

      - name: Build with Maven
        run: mvn clean package

  build:
    runs-on: ubuntu-latest
    needs: test
    steps:
      - name: Checkout
        uses: actions/checkout@v4

      - name: Set up QEMU
        uses: docker/setup-qemu-action@v3

      - name: Set up Docker Buildx
        uses: docker/setup-buildx-action@v3

      - name: Configure AWS credentials
        uses: aws-actions/configure-aws-credentials@v4
        with:
          aws-access-key-id: ${{ secrets.AWS_ACCESS_KEY_ID }}
          aws-secret-access-key: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
          aws-region: ${{ vars.AWS_REGION }}

      - name: Login to Amazon ECR
        id: login-ecr
        uses: aws-actions/amazon-ecr-login@v2

      - name: Build, tag, and push image to Amazon ECR
        id: build-image
        env:
          ECR_REGISTRY: ${{ steps.login-ecr.outputs.registry }}
          IMAGE_TAG: ${{ github.sha }}
          IMAGE_NAME: ${{ vars.IMAGE_NAME }}
        run: |
          # Build a docker container and push it to ECR so that it can be deployed to ECS.
          docker build -t $ECR_REGISTRY/$IMAGE_NAME:$IMAGE_TAG .
          docker push $ECR_REGISTRY/$IMAGE_NAME:$IMAGE_TAG
          echo "image=$ECR_REGISTRY/$IMAGE_NAME:$IMAGE_TAG" >> $GITHUB_OUTPUT
      - name: Fill in the new image ID in the Amazon ECS task definition
        id: task-def
        uses: aws-actions/amazon-ecs-render-task-definition@v1
        with:
          task-definition-arn: ${{ vars.TASK_DEFINITION }}
          container-name: ${{ vars.CONTAINER_NAME }}
          image: ${{ steps.build-image.outputs.image }}

      - name: Deploy Amazon ECS task definition
        uses: aws-actions/amazon-ecs-deploy-task-definition@v2
        with:
          task-definition: ${{ steps.task-def.outputs.task-definition }}
          service: ${{ vars.SERVICE_NAME }}
          cluster: ${{ vars.CLUSTER_NAME }}
          wait-for-service-stability: true

Our workflow is working well:

The service was successfully deployed as shown in the image below:

Step 5: Validate the Deployment

Test your deployment using the following script (replace the ALB URL with yours):

ALB_FQDN="tf-lb-20250118223346384400000002-528284851.us-east-1.elb.amazonaws.com"
until curl -Is --max-time 5 http://$ALB_FQDN/ping | grep "HTTP/1.1 200" >/dev/null 2>&1
do
  echo "Waiting for ALB to serve traffic of java-api (/ping)..."
  sleep 5
done

printf "\nALB is now ready to serve traffic:\n"
printf "http://$ALB_FQDN\n"

The ALB is now ready to serve traffic!

Final Thoughts

By leveraging AWS CDKTF, we can write clean, maintainable IaC code using Python. This approach simplifies deploying containerized applications like a Spring Boot API on AWS ECS Fargate.

CDKTF’s flexibility, combined with Terraform’s robust capabilities, makes it an excellent choice for modern cloud deployments.

While the CDKTF project offers many interesting features for infrastructure management, I have to admit that I find it somewhat too verbose at times.

Do you have any experience with CDKTF? Have you used it in production?

Feel free to share your experience with us.

Is it time to Learn Golang?

abdelino17 — Sun, 24 Sep 2023 19:00:00 +0000

When it comes to programming languages, Go, also affectionately known as Golang, has been making waves in the development community. It's a language that boasts a unique blend of strengths and limitations, and in this article, we're going to dive deep into what makes Go a programming powerhouse while also shining a light on the aspects that developers should keep in mind.

Strengths of Go

Simplicity: The Elegance of Minimalism
Go adheres to the principle that less is more. With its clean syntax and a small set of keywords (just 25 and 1 loop type), it's a programming language that's remarkably easy to learn and even easier to read. The result? Code that remains crystal clear, even when you revisit it months or years down the line. If you appreciate code that stands the test of time, Go's simplicity is your ally.
Efficient Concurrency: Scaling Made Simple
Go was tailor-made for handling concurrency like a champ. It introduces the concept of goroutines — lightweight threads managed by the Go runtime. These little workhorses allow for efficient parallelism. The Go runtime's scheduler juggles goroutines at the OS level, ensuring scalability and top-notch performance. If your project involves handling multiple tasks concurrently, Go is your secret weapon.
Powerful Standard Library: Your Swiss Army Knife
Go comes armed with an extensive standard library that's a treasure trove of functionality. From networking to file handling, encryption, and beyond, this comprehensive library empowers developers to construct complex systems without drowning in external dependencies.
In fact, you can write a full MVC web application (from Web Router to Databases passing by Views) without installing almost any packages.
Think of it as your programming Swiss Army knife, always ready for the task at hand.
Lightning-Fast Compilation Speed: Turbocharge Your Workflow
Time is precious in the world of development, and Go respects that. Its compilation speed is a cut above the rest, providing quick feedback loops that foster rapid iteration. Say goodbye to frustrating waiting times and hello to faster development cycles.
By default, the Go compiler produces static executable binaries that require no external language or runtime. This is one of the reasons, Go is widely used to distribute modern applications: from agents, and plugins to DevOps tools such as Terraform, docker, Kubernetes, etc.
Backward Compatibility: Future-Proof Your Projects
Go doesn't just focus on the present; it has an eye on the future. It maintains a rock-solid commitment to backward compatibility. Your code, written in earlier versions of Go, will seamlessly transition to newer versions. This stability is a lifeline for maintaining and upgrading existing projects.

Limitations of Go

Immature Ecosystem: Growing Pains
Go's ecosystem is dynamic and expanding, but it's relatively young compared to established languages. Consequently, you might find fewer libraries and frameworks for specialized use cases. Nevertheless, the robust Go community and the comprehensive standard library help bridge this gap.
Error Handling: The Double-Edged Sword
Go adopts an explicit error-handling approach through return values, which can sometimes lead to verbose code. While it promotes rigorous error checking, it can also introduce complexity. However, Go provides mechanisms like defer and panic/recover to tame errors more gracefully.
Limited Metaprogramming: The Reflective Edge
When it comes to metaprogramming and reflection, Go offers only limited support compared to some other languages. While Go does provide some reflection capabilities, they are not as potent as those found in alternative languages.

In conclusion, Go is a programming language that packs a punch, emphasizing simplicity and concurrency. Its strengths encompass elegance, efficient concurrency, a robust standard library, lightning-fast compilation, and unwavering backward compatibility. However, it does have its limitations, explicit error handling, an evolving ecosystem, and limited metaprogramming capabilities. Armed with this knowledge, you're well-equipped to make an informed decision when considering Go for your next project.

Ready to harness the power of Go? Dive in and see what this dynamic language can do for you!

P.S. I'm preparing three distinct series on Golang: The first one will focus on the fundamentals of the language. In the second one, we will discuss the strengths of Golang for Web Developers and finally, the third one will concentrate on the unique features of Go for DevOps Engineers.

How to deploy your next FastAPI app on AWS with AppRunner in less than 5 minutes?

abdelino17 — Thu, 20 Apr 2023 18:44:29 +0000

In May 2021, AWS announced the General Availability of its service AWS App Runner, described as "the simplest way to build and run a containerized web application in AWS". At that time, the service was extremely limited (no support for VPC connectivity) and only a few runtimes were supported.

02 years later, the service has evolved and now supports many interesting features.

In this article, you will see what AppRunner is and how it can help you to deploy FastAPI web apps in a couple of minutes.

Why AppRunner?

When it comes to deploying a web application on AWS, you have several ways, depending on the application’s requirements, team capabilities, budget and various constraints.

However, regardless of the deployment method chosen, there are some mandatory configurations that need to be in place to ensure the application is highly available.

This typically falls under the responsibility of the operations team (ops), who must set up:

Load balancing
Target groups
Scaling
Security groups
Domains
Certificates
Observability (monitoring, logging, metrics)
And more.

Despite this complexity, developers simply want to deploy their application and focus on its development. This observation led AWS to offer a fully managed service to simplify the deployment of applications: AppRunner.

Definition

The official page states “AWS App Runner is a fully managed container application service that lets you build, deploy, and run containerized web applications and API services without prior infrastructure or container experience”.

At this moment, you are probably wondering if AppRunner is a:

Platform as a Service (PaaS)?
Container as a Service (CaaS)?
New Serverless Service?
Managed Service
or All at once?

Let’s try to understand how the service operates. This will probably help us to respond to the question.

Modes of Operation

Under the hood, AppRunner creates an ECS cluster and uses Fargate to effortlessly run your containers, saving you time and effort.

While some may argue that sacrificing granularity is a compromise, the benefits of simplicity and speed are undeniable. With built-in continuous integration and integration with CloudWatch for logs and monitoring, App Runner eliminates the need for additional third-party services.

Streamline your container management with App Runner and focus on what matters most – your business!

When it comes to configuring App Runner, you have two options to choose from – manual and automatic.

The manual mode is ideal for those who prefer to have more control over the configuration process. With this mode, you can configure your application's environment variables, networking, and other settings using the AWS Management Console or CLI.

On the other hand, the automatic mode is interesting for those who prefer a more hands-off approach. With this mode, App Runner automatically loads your application configuration from its source code allowing you to focus on other aspects of your project.

In addition to the modes of configuration, App Runner offers two modes of operation – Container Mode and Build Mode.

Container Mode is recommended for running pre-built containers from a container registry, such as Amazon ECR or Docker Hub. With this mode, you can quickly deploy and scale your application without worrying about the underlying infrastructure.

The Build Mode is perfect for building and deploying applications from source code. With this mode, App Runner automatically builds your application from your source code, dependencies, and build instructions, making it easy to deploy and manage your applications.

Whether you prefer to use pre-built containers or build your applications from source code, App Runner offers flexible options to meet your needs.

App Runner in Action

Now that we have a complete overview of the service, let’s practice using build and automatic modes.

In a previous article, I set up a basic python api with FastAPI Framework. I will reuse the code for the rest of this article.

import fastapi, uvicorn
from starlette.requests import Request
import prometheus_client
import os

api = fastapi.FastAPI()

...

if __name__ == "__main__":
    print("Starting webserver...")
    uvicorn.run(
        api,
        host="0.0.0.0",
        port=int(os.getenv("PORT", 8080)),
        debug=os.getenv("DEBUG", False),
        log_level=os.getenv('LOG_LEVEL', "info"),
        proxy_headers=True
    )

The AWS Console provides all you need for the deployment. You should see a page like this:

Click on Create an App Runner Service button, you have now this:

For the while, AppRunner supports only Github as Source code repository. So, click on the Add new button to add a new repository.

A new window is opened. Set a name for your connection and click on the Install button.

Indeed, AWS will install the Github connect and you will be able to select the repository and the branch you want to deploy. The connection could be reused for all your other services.

Don’t forget to check the repositories the AWS Connector has access to.

At this step, you should have all your fields filled in as follows:

On the Next Page, it’s time to configure your build settings.

Add the configuration file apprunner.yaml at the root of your project with the following content:

version: 1.0 # version of the specification
runtime: python3 # runtime to use
build:
  commands:
    build:
      - pip install -r requirements.txt # install dependencies

  env: # define environment variables
    - name: PORT
      value: '8080'

run:
  command: python app.py # command to run fastapi
  network:
    port: 8080 # port AppRunner will expose

On the Next Page, you can enable various options to make stand out your service:

Auto Scaling: Define the scaling behaviour for you service by setting the number of requests, concurrency, etc.
Health Check: Configure the health checks that the load balancer performs on your App Runner service instances (the path to request, the number of consecutive health check failures, etc.). The health check path of our service is /ping.
Security: A custom instance role to call AWS services
Networking: Public, private endpoints, custom VPC, etc.
Observability: Tracing with AWS X-Ray.

The last page is the Review and create Page. You will be able check your settings one last time before the deployment. If everything is good, click on the Create and deploy button, which will trigger the deployment process:

It will take a couple of minutes for AppRunner to deploy your application. After that, the status turns into Running and a message indicates that your application is running.

Congratulations, You nailed it!

Testing the continuous deployment

In automatic mode, every push on your repository will trigger the deployment of your service. Let’s test it!

Open the app.py file and add the following code:

...

@api.get('/hello')
def hello(request: Request):
    REQUESTS.labels(endpoint='/hello').inc()
    return "Hello World!"
...

Commit your changes and push them.

If you go back to your AWS Console, you should see the status of your service status transitioning from running to Operation in progress. This indicates the detection of changes and a redeployment in progress.

A few minutes later, your application should be running again and the new endpoint should now be available:

Awesome! our automatic deployment is fully operational.

If you prefer command line, feel free to check out the AWS CLI Docs page.

Conclusion

To sum up, AWS AppRunner is definitely THE service you should consider to deploy your next FastAPI service quickly and efficiently. It is developer-centric and easy to set up.

A lot of effort of integration with the AWS Ecosystem has been done these last months and the service is now production-ready for most of use cases. Just enjoy!

That’s all for today!

If you have any questions or experience with AppRunner, do not hesitate to share them in the comments section.

I’d love to hear from you 😉.

Unleashing the Power of LocalStack for Faster AWS Development

abdelino17 — Tue, 14 Feb 2023 19:42:21 +0000

AWS is the world’s leading cloud vendor, offering 200+ services operating in 26 regions globally. Beyond the fundamental services proposed by any cloud provider, AWS has a consistent, coherent, and complete environment to innovate, create and develop next-gen applications. This includes APIs, SDKs, CLI tools, etc.

Image having your own local AWS environment in a single Docker Container?

The promise is clear: Run AWS services on your local computer. Interested? Let’s get started!

What’s LocalStack?

To get started with AWS, you have to create a new AWS account and receive a free tier of $300 for 12 months to practice AWS services.

However, even if you are a seasoned AWS user, you may prefer to have some essential AWS services locally for development, testing, and iteration without the need for an AWS account. This is where LocalStack comes in.

"LocalStack is a cloud service emulator that runs in a single docker container on your local machine. It allows you to run, test, and debug your AWS applications without a connection to a remote cloud provider".

As stated on their Getting Started page, LocalStack supports a number of fundamental AWS services including AWS Lambda, S3, DynamoDB, SQS, and many more.

For even more features and advanced APIs, there is also a Pro version of LocalStack. A comprehensive list of supported APIs can be found on their ☑️ Feature Coverage page.

With this understanding of LocalStack, Let’s delve into what makes it so powerful.

Under the hood

In Software Engineering, when designing API, you should model it first. API Modelling helps you to:

Identify the users/actors of your API
Establish better communication practices
Draw out common understanding (ubiquitous language in DDD)
Answer the question of why the API exists.

Let’s take the example of SQS, a message queueing service allowing different software components to exchange messages safely. If we take a look at the service API definition, we can see this:

{
    "CreateQueue":{
      "name": "CreateQueue",
      "HTTP": {
        "method": "POST",
        "requestUri": "/"
      },
      "input":{"shape": "CreateQueueRequest"},
      "output":{
        "shape": "CreateQueueResult",
        "resultWrapper": "CreateQueueResult"
      },
      "errors":[
        {"shape": "QueueDeletedRecently"},
        {"shape": "QueueNameExists"}
      ],
      "documentation": "Creates a new standard or FIFO queue. You can pass one or more attributes in the request ..."
    },
    ...
    ...
    "CreateQueueRequest":{
        "type": "structure",
    "required":["QueueName"],
        "members": {
            "QueueName":{
                "shape": "String",
                "documentation": "The name of the new queue. The following limits apply to this name: ..."
            },
            "Attributes": {
                "shape": "QueueAttributeMap",
                "documentation": "A map of attributes with their corresponding values...",
                "locationName": "Attribute"
            },
            "tags":{
                "shape": "TagMap",
                "documentation": "Add cost allocation tags to the specified Amazon SQS queue. For an overview, ...",
                "locationName": "Tag"
            }
        }
    }
}

This snippet corresponds to the CreateQueue endpoint of the SQS service. The API takes 03 parameters: QueueName, Attributes, and Tags.

We can find the equivalent implementation in the localstack repo:

@handler("CreateQueue")
def create_queue(
    self,
    context: RequestContext,
    queue_name: String,
    attributes: QueueAttributeMap = None,
    tags: TagMap = None,
) -> CreateQueueResult:
    raise NotImplementedError

...
...

class CreateQueueRequest(ServiceRequest):
    QueueName: String
    Attributes: Optional[QueueAttributeMap]
    tags: Optional[TagMap]

As explained in this article, LocalStack’s mission since day one is to keep parity with AWS. This means, whenever you make an AWS API call to LocalStack’s cloud emulator, it behaves the same way AWS would.

Based on the AWS service protocol definition, the LocalStack Team built a framework named AWS Server Framework (ASF). This latter generates server-side stubs for services and all their supported operations. The API definitions come from the python SDK discussed above. All service requests are then routed to their respective server-side implementation through ASF, which implements the AWS protocol in a generalized way.

There are two levels of Emulation:

CRUD: The service accepts requests and returns proper (potentially static) responses. No additional business logic besides storing entities.
Emulated: The service imitates the functionality, including synchronous and asynchronous business logic operating on service entities.

Let’s practice!

There are various ways to start and manage your LocalStack instance. As described in the docs, you can use LocalStack CLI, LocalStack Cockpit, Docker, Docker-Compose, or Helm.

In our example, I will use the CLI. This requires Python 3.7 at least.

Install LocalStack with this command:

$ pip install localstack

Then, run it with the command:

$ localstack start -d

The command above starts LocalStack inside a container.

Alternatively, you can directly run LocalStack from the official image with the following command:

$ docker run --rm -it -p 4566:4566 -p 4510-4559:4510-4559 localstack/localstack

After a few seconds, you should see this:

To have a smooth experience, I recommend you to install the following tools:

# a thin wrapper around the aws cli command that runs the command directly against LocalStack
$ pip install awscli-local

# a thin wrapper around the terraform command line client
$ pip install terraform-local

Additionally, you should install the Docker Desktop LocalStack Extension.

Once the installation is done, you should see this:

You can also query the status of respective services on LocalStack with the CLI:

$ localstack status services

In my previous article, I introduced you to AWS CDK + Terraform (CDKTF) and set up a basic s3 bucket.

Unfortunately, there is no wrapper for CDKTF yet. I will create a basic bucket from python.

import boto3
from pprint import pprint
import glob

client = boto3.resource("s3", endpoint_url="http://s3.localhost.localstack.cloud:4566")

# create a new bucket
blogBucket = client.Bucket('blog.abdelfare.me')

response = blogBucket.create()
pprint(response)

for filename in glob.iglob(f'posts/*'):

    # upload blog article from the "posts" directory
    client.upload_file(filename, filename)

There we go, our bucket was created and you can check it with the following commands:

$ awslocal s3 ls s3://blog.abdelfare.me --recursive

Your development environment is now ready for cloud development with AWS.

LocalStack is ephemeral in nature and will not persist any data across restarts. You can destroy your local infrastructure with the command:

$ localstack stop

Conclusion

To sum up, LocalStack provides a convenient and practical solution for developing with AWS Cloud. With its ability to emulate some services and provide basic CRUD operations for others, it offers a smooth experience that eliminates any surprises when deploying to a real AWS account.

However, it’s worth noting that LocalStack should only be used for development and initial testing. For validating the performance, reliability, and availability of your system, you should use your real AWS account.

That’s all for now.

If you have any questions or experiences with LocalStack, feel free to share them in the comments section. I'd love to hear from you 😉.

Create your next s3 Bucket with AWS CDK and Terraform

abdelino17 — Sat, 30 Jul 2022 07:27:53 +0000

What do you think of using your preferred programming language to provision your infrastructure?

AWS CDK (Cloud Development Kit), based on CloudFormation has been released a couple of years to achieve that.

As a big fan of Terraform, I had a keen interest in the CDKTF project which aims at using AWS CDK with Terraform.

Before jumping into CDKTF, let's revisit some fundamentals concepts.

Infrastructure as Code

Infrastructure as code (IaC) is basically, the process of managing and provisioning IT resources (in the cloud or not), through definition files or code rather than manual or interactive configuration.

The main objective is to provide a centralized way to manage configuration in terms of implementation and version control. Hereby, all changes can be validated, tested, and provisioned as part of a release process using standard deployment pipelines.

There are many benefits to using IaC for your next project:

Speed and safety: You expend a lot less effort on simple and repetitive tasks.
Documentation: You can easily document your code like any software project.
Testing: You can write tests for your project.

Now that, we know what IaC is, let's talk about Terraform which is the most popular Iac tool out there!

Terraform

Terraform is an open-source tool created by HashiCorp in 2014. It allows you to define the infrastructure as code using a simple, declarative language called HCL (stands for Hashicorp Configuration Language) to deploy and manage that infrastructure across a variety of public cloud providers (e.g., Amazon Web Services, Microsoft Azure, Google Cloud Platform, DigitalOcean), private cloud and virtualization platforms.
If you want to deep dive into Terraform, I recommend the excellent book: Terraform: Up & Running by O'Reilly.

AWS CDK

The AWS Cloud Development Kit (AWS CDK) is an open-source framework initiated by AWS to define your cloud infrastructure in code and provision it through AWS CloudFormation.
Using the familiarity and expressive power of current programming languages, It provides high-level components called constructs that preconfigure cloud resources with proven defaults, so you can build cloud applications with ease.

You will find more information in the documentation

Cloud Development Kit for Terraform (CDKTF) is the fruit of a partnership between Hashicorp and AWS teams for leveraging the Terraform ecosystem to define and provision your infrastructure. This allows you to use Terraform without learning HashiCorp Configuration Language (HCL).
CDKTF currently supports TypeScript, Python, Java, C#, and Go (experimental).

Prerequisites

— python (3.10)
— pipenv
— npm

Installation

To use cdk with terraform, you must run the following command:

$ npm install --global cdktf-cli@latest

This installs the cdktf CLI that allows to spin up new projects for various languages.

We can scaffold a new python project by running:

$ mkdir s3-demo && cd s3-demo

$ cdktf init --template="python" --local

We use the option --local to indicate that we will use local state storage for generated Terraform.
In a real project, it is recommended to use a remote backend.

There are many files created by default.

For our demo, we will use AWS as cloud provider so we need to install the required provider.

$ pipenv install cdktf-cdktf-provider-aws

You can edit the main.py at the root of the project and add the code below:

#!/usr/bin/env python
from constructs import Construct
from cdktf import App, TerraformOutput, TerraformStack
from cdktf_cdktf_provider_aws import AwsProvider, s3
# Define your stack

class MyStack(TerraformStack):
    def __init__(self, scope: Construct, ns: str):
        super().__init__(scope, ns)

        # Configure the AWS provider to use the eu-west-3 region
        AwsProvider(self, "AWS", region="eu-west-3")

        # Create your first bucket
        # pass the id of the resource and the name of the bucket
        first_bucket = s3.S3Bucket(
            self, "first_bucket",
            bucket="blog.abdelfare.me",
            acl="public-read"
        )

        TerraformOutput(self, "bucket_name", value=first_bucket.bucket)
        TerraformOutput(self, "bucket_arn", value=first_bucket.arn)

app = App()
MyStack(app, "s3-demo")

app.synth()

I found the SDK really intuitive.

Basically, We created our Main Stack inherited from the TerraformStack class. We set up the aws provider (Don't forget to define the AWS profile) and We create our first S3 bucket in one line of code.

TerraformOutput allows to export some values and reuse them in another stack.

Now that, everything is ready, we can generate the terraform configuration files by running the command below:

$ cdktf synth

We can see the new directory cdktf.out which contains all the terraform configuration.
This step is equivalent to the execution of the command terraform plan.

We can deploy our stack by running the command:

$ cdktf deploy

This command runs terraform apply in the background.

We can see all the new changes and confirm the execution.

There we go!

Our bucket was created successfully.

We can see the output of your stack by using the following command:

cdktf output

We can now destroy our stack by running the command:

$ cdktf destroy

When to use CDK for Terraform

CDKTF offers many benefits, but it is not the right choice for every project. You should consider using CDKTF when:

You have a strong preference or need to use a procedural language to define infrastructure.
You need to create abstractions to help manage complexity.
You are comfortable doing your own troubleshooting and do not require commercial support.

Some pain points to consider:

Initial CDK setup is time consuming
Deployment times can be slow due to the synthesizing steps

Do you have any experience with CDKTF? Have you used it in production?

Feel free to share you experience with us in comment.

You could find the code of this article on my github repo.

In a next article, I will set up a full containerized architecture with *ECS, Fargate, ECR, ALB, CDK + Terraform so stay tuned 😃!*

Package your FastAPI application with “Distroless” Docker Images

abdelino17 — Tue, 19 Jul 2022 08:16:02 +0000

Distroless Docker Images is a project proposed by Google in order to help building slimmer containers. The project description states it’s "Language focused docker images, minus the operating system". Sounds interesting, right?

Compared to common base images like ubuntu, alpine, debian which have lots of OS packages and libraries that might not be required for your application, Distroless Images have only what you need, to run your application.

It’s worth noting that Distroless containers are not always the most secured solution as explained by RedHat in this article.

Although it'd be hard to package a python application since its standard library relies on some higher-level OS capabilities, it’s possible to achieve that.

Python distroless containers require multi-stage building process because the gcr.io/distroless/python3 image has neither pip nor even easy_install.

Let’s try to package a FastAPI api with “Distroless” Docker Images.

Here is a sample api server:

import fastapi, uvicorn
from starlette.requests import Request
import prometheus_client
import os

api = fastapi.FastAPI()

REQUESTS = prometheus_client.Counter(
    'requests', 'Application Request Count',
    ['endpoint']
)

@api.get('/ping')
def index(request: Request):
    REQUESTS.labels(endpoint='/ping').inc()
    return "pong"

@api.get('/metrics')
def metrics():
    return fastapi.responses.PlainTextResponse(
        prometheus_client.generate_latest()
    )

if __name__ == "__main__":
    print("Starting webserver...")
    uvicorn.run(
        api, 
        host="0.0.0.0",
        port=int(os.getenv("PORT", 8080)),
        debug=os.getenv("DEBUG", False),
        log_level=os.getenv('LOG_LEVEL', "info"),
        proxy_headers=True
    )

We will use Pipenv as package manager. Here is our Pipfile:

[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[packages]
fastapi = "==0.77.1"
uvicorn = "==0.17.6"
prometheus-client = "==0.14.1"
Jinja2 = "==3.1.2"

[dev-packages]

[requires]
python_version = "3.10"

So, I ended up with the following Dockerfile (commented) allowing me to package a FastAPI application with the distroless Python image.

First Stage:

FROM python:3.10-slim AS base

# Setup env

## Avoid to write .pyc files on the import of source modules
ENV PYTHONDONTWRITEBYTECODE 1

# Enable fault handler
ENV PYTHONFAULTHANDLER 1

Second Stage:

# Dependencies
FROM base AS python-deps

### Install pipenv and compilation dependencies
RUN pip install pipenv \
    && apt-get update \
    && apt-get install -y --no-install-recommends gcc

### Install python dependencies in /.venv
COPY Pipfile .
COPY Pipfile.lock .

# Allows to install the pipenv packages into the project instead of home user
# --deploy
RUN PIPENV_VENV_IN_PROJECT=1 pipenv install --deploy

Third Stage:

# Runtime
FROM gcr.io/distroless/python3

WORKDIR /app

# Copy the python packages because the distroless base image does 
COPY --from=python-deps /.venv/lib/python3.10/site-packages /app/site-packages

# Set the Python path where the interpreter will look for the packages
ENV PYTHONPATH /app/site-packages
COPY . .

EXPOSE 8080
ENTRYPOINT ["python", "app.py"]

The resulting image was around 25Mb.

You could find all the code on my github repository.

If you have any points to or not to use “Distroless” for python applications, feel free to drop in comment.

Building a python package as per the PyPA Guidelines

abdelino17 — Tue, 14 Jun 2022 22:13:36 +0000

Python is definitely the language you MUST to learn in 2022.
Originally developed by Guido van Rossum in 1991, it is used in a wide variety of industries, ranging from education to medicine.

There are various tools and techniques for creating and distributing packages.

The Python Packaging Authority (PyPA) proposes guidelines for building and sharing python packages.

In this article, we will write a small python package implementing the LRU Cache.

For recall, The Least Recently Used (LRU) is a caching strategy consisting to evict elements from the cache to make room for new elements when the cache is full. In short, the least recently items are discarded first.

Here is the code of the LRU Cache:

from collections import OrderedDict

class LRUCache(OrderedDict):

    def __init__(self, capacity: int):
        self.capacity = capacity

    def put(self, key: str, value: str) -> None:
        if key in self:
            self.move_to_end(key)
        self[key] = value
        if len(self) > self.capacity:
            self.popitem(last=False)

    def get(self, key: str) -> int | str:
        if key not in self:
            return -1
        self.move_to_end(key)
        return self[key]

    def show_entries(self) -> None:
        print(self)

PyPA recommends using a sample project for building reusable packages and is available at: https://github.com/pypa/sampleproject.

The most important files to remember are:

setup.py: script for building and installing the package. It contains a global setup() function.
setup.cfg: this is a configuration file that can be used by setup.py to define default.
setup(): you can pass many arguments to this function such as Name, Version, Description, URL, Author, License.
data: place to add data files if needed.
tests: placeholder to add unit tests for the modules.
src/<package>: top-level package containing all the modules and packages inside it.

Let’s clone the repo and modify it:

$ git clone https://github.com/pypa/sampleproject.git lru-cache

We updated setup.py with arguments as per our package project. Here is a snippet of our file:

setup(
    name="lrucache",
    version="0.1.0",
    description="LRU Cache for illustration purpose",
    long_description=long_description,
    long_description_content_type="text/markdown",  
    url="https://github.com/abdelino17/lru-cache",
    author="Abdel FARE",  
    author_email="fare.abdel@gmail.com", 
    keywords="cache, lru, setuptools, development", 
    package_dir={"": "src"}, 
    packages=find_packages(where="src"),  
    python_requires=">=3.7, <4",
    project_urls={
        "Bug Reports": "https://github.com/abdelino17/lru-cache/issues",
        "Source": "https://github.com/abdelino17/lru-cache"
    }
)

With this setup.py file, we are ready to share our lru-cache package locally as well as to the world.

Before building, here is the structure of our package:

For simplicity purpose, Let's add this to the __init__.py file:

from .lrucache import LRUCache

Here is the content of our test file:

import unittest

from lrucache import LRUCache

CACHE_CAPACITY = 3

class TestLRUCache(unittest.TestCase):

    def setUp(self) -> None:
        super().setUp()
        self.lruCache = LRUCache(CACHE_CAPACITY)

    def test_cache(self):
        self.lruCache.put("1", "1")
        self.lruCache.put("2", "2")
        self.lruCache.put("3", "3")

        self.assertEqual(self.lruCache.get("1"), "1")
        self.assertEqual(self.lruCache.get("3"), "3")

        self.lruCache.put("4", "4")
        self.assertEqual(self.lruCache.get("2"), -1)

        self.lruCache.put("5", "5")
        self.assertEqual(self.lruCache.get("1"), -1)


if __name__ == '__main__':
    unittest.main()

Installing from the local source

Let’s execute the following command to install the package from the local source:

$ pip install <path_to_package>

The following is the console output of the command:

Our package was installed successfully and we can now use it from the command line:

Publishing a package to Test PyPI.

As a next step, we will add our sample package to the PyPI Test repository.

Before executing any command for publishing our package, we will need to create an account and get an API token on Test PyPI.

You could find all the information for creating and adding the token on the Test PyPI website.

To push the package, we will also need the Twine utility. You can install it with pip:

$ pip install twine

Now, we are ready to upload our package, For that, we will execute the following steps:

1.Create a distribution using the following command:

$ python setup.py sdist

2.Upload the distribution file to Test PyPI.

$ twine upload --repository testpypi dist/lrucache-0.1.0.tar.gz

This command will push the package file to our Test PyPI repository.

There we go! Our package is now available for the community.

DEV Community: abdelino17

How to Stay Motivated When Preparing for a Cloud Certification?

1. Start with your WHY

2. Set a clear deadline

3. Choose the right resources

Why you should consider the AWS AI Practitioner Certification?

1. Generative AI is the biggest revolution since the Internet

2. “Just start it”

3. Practitioner level — but not as easy as you think

Why the AWS Solutions Architect Associate Should Be Your First AWS Certification?

1. Solid Understanding of AWS Core Services

2. Practical Experience With Design Patterns and Best Practices

3. The Perfect Entry Point into AWS Certifications

Ready to start? Here are my top course recommendations:

DEVS HATE NETWORKING (But You Need It!)

1. CIDR Blocks

2. Private IP vs. Public IP

3. DNS (Domain Name System)

4. TCP/IP Model Essentials

5. Load Balancers

How docker build Really Works?

What really happens during docker build?

So, how can a container run without an image?

The Twist 💡

How to deploy a SpringBoot API on AWS ECS using CDKTF?

Architecture Overview

Step 1: Containerize your Spring Boot Application

Step 2: Set up AWS CDKTF

1. Prerequisites

2. Install CDKTF and Dependencies

3. Initialize Your CDKTF Application

Step 3: Building Layers

1. Network Layer

2. Infrastructure Layer

3. Service Layer

Step 4: Github Actions Workflow

Step 5: Validate the Deployment

Final Thoughts

Is it time to Learn Golang?

Strengths of Go

Limitations of Go

How to deploy your next FastAPI app on AWS with AppRunner in less than 5 minutes?

Why AppRunner?

Definition

Modes of Operation

App Runner in Action

Testing the continuous deployment

Conclusion

Unleashing the Power of LocalStack for Faster AWS Development

What’s LocalStack?

Under the hood

Let’s practice!

Conclusion

Create your next s3 Bucket with AWS CDK and Terraform

Infrastructure as Code

Terraform

AWS CDK

Prerequisites

Installation

When to use CDK for Terraform

Package your FastAPI application with “Distroless” Docker Images

Building a python package as per the PyPA Guidelines

Installing from the local source

Publishing a package to Test PyPI.

What really happens during `docker build`?