DEV Community: Yusuf Adeyemo

Understanding TOTP: What Really Happens When You Generate That 6-Digit Code

Yusuf Adeyemo — Mon, 08 Dec 2025 16:26:40 +0000

This article started from a tweet.

Someone on Twitter said they "lowkey want to understand the technology behind Google Authenticator" and I dropped a quick reply - explaining that it's basically TOTP: your device and the server share a secret key, both compute a code using HMAC-SHA1 and the current 30-second time window. No network calls. No "previous code." Same secret + same time slice = same 6-digit code.

That reply got some traction, and a few people DM me for a deeper breakdown. So here we are.

If you've ever wondered how your phone generates the exact same 6-digit code the server expects - with no internet request, no sync, nothing - this one's for you.

The Problem With Passwords

Passwords are static. Once someone has it, they have it forever - or until you change it. Even with a strong password, you're one phishing attack or database breach away from compromise.

Two-factor authentication fixes this by adding something that changes. But here's the catch - if your phone needs to call a server every time to get a new code, that's a point of failure. What happens when you're offline? On a plane? In a basement with no signal?

This is where TOTP comes in.

TOTP - Time-based One-Time Password

TOTP is defined in RFC 6238, but don't let the RFC scare you. The core idea is dead simple:

Both your phone and the server share a secret. They both know the current time. They both do the same math. They both get the same answer.

That's it. No network calls. No synchronization requests. Just two parties doing identical calculations independently.

The Setup - That QR Code You Scanned

When you enable 2FA on any service, they show you a QR code. That QR code contains a URL that looks something like this:

otpauth://totp/MyService:yusuf@yusadolat.me?secret=JBSWY3DPEHPK3PXP&issuer=MyService

The important part is the secret. This is a base32-encoded string that both your authenticator app and the server will store. This shared secret is the foundation of everything.

You scan it once. Your app saves it. The server saves it. They never exchange it again.

The Math - How Codes Get Generated

Every 30 seconds, both sides perform this calculation:

Step 1: Get the current time window

Take the current Unix timestamp and divide by 30. Floor it.

time_step = floor(current_unix_time / 30)

Right now, as I write this, the Unix timestamp is around 1733644800. Divided by 30, floored, gives us 57788160. This number changes every 30 seconds.

Step 2: Run HMAC-SHA1

Feed the time step and the shared secret into HMAC-SHA1:

hmac_result = HMAC-SHA1(secret, time_step)

This produces a 20-byte hash. It looks like random garbage, but it's deterministic - same inputs always give same outputs.

Step 3: Dynamic Truncation

20 bytes is too long for humans to type. So we extract 4 bytes from a specific position (determined by the last nibble of the hash), convert to an integer, and take modulo 1,000,000.

offset = hmac_result[19] & 0x0fcode = (hmac_result[offset:offset+4] & 0x7fffffff) % 1000000

Boom. You have your 6-digit code.

Why This Is Actually Clever

Think about what just happened:

No network needed - Your phone doesn't call anyone. The server doesn't push anything. Both just compute.
Codes expire automatically - Because time moves forward, old codes become useless. Even if someone shoulder-surfs your code, they have maybe 30 seconds to use it.
Can't predict future codes - Without the secret, you can't compute tomorrow's codes. The HMAC function is one-way.
Replay attacks fail - Use a code once, the server marks that time window as used. Try it again, rejected.

When Things Go Wrong

The system assumes both parties agree on what time it is. This is usually fine - your phone syncs with NTP servers, and servers have accurate clocks.

But I've seen people with phones that have "manual time" set, drifting by minutes. Their codes stop working and they have no idea why. The server is computing codes for 10:45:00, their phone is computing for 10:43:00. Different time windows, different codes.

Most implementations allow a small tolerance - they'll accept codes from one time window before or after. But drift too far and you're locked out.

The Recovery Code Situation

Those backup codes you're told to save somewhere? They're not TOTP. They're just long random strings stored in a database. Use one, it gets deleted. No time component, no algorithm - just a simple lookup.

Save them. Seriously. Losing access to your authenticator without backup codes is a special kind of pain.

Show Me The Code

Here's a minimal Python implementation to make this concrete:

import hmacimport hashlibimport structimport timeimport base64def generate_totp(secret: str) -> str: # Decode the base32 secret key = base64.b32decode(secret.upper()) # Get current time step (30-second window) time_step = int(time.time()) // 30 # Pack as big-endian 8-byte integer time_bytes = struct.pack('>Q', time_step) # Compute HMAC-SHA1 hmac_hash = hmac.new(key, time_bytes, hashlib.sha1).digest() # Dynamic truncation offset = hmac_hash[-1] & 0x0f code_int = struct.unpack('>I', hmac_hash[offset:offset+4])[0] code_int &= 0x7fffffff code = code_int % 1000000 return f'{code:06d}'# Test itsecret = 'JBSWY3DPEHPK3PXP' # Example secret, This is what you add setup key on Google Authprint(generate_totp(secret))

Want to see it work in real-time? Here's how to test:

Open Google Authenticator (or any TOTP app)
Tap the + button to add a new account
Select "Enter a setup key"
Enter any name (e.g., "TOTP Test")
For the key, enter: JBSWY3DPEHPK3PXP
Make sure it's set to Time-based
Save it

Now run the Python script. The 6-digit code it prints should match what's showing in your authenticator app. If you're a few seconds off, wait for the next 30-second window and try again.

Wrapping Up

There's no cloud magic happening when your authenticator generates codes. It's just math - the same math running independently on your device and the server, anchored to the same clock.

Understanding this changes how you think about 2FA. It's not some opaque security feature. It's a clever application of cryptographic primitives that's been working reliably for over a decade.

Next time you punch in those 6 digits, you'll know exactly what's happening behind the scenes.

If you found this useful, I write about DevOps, security, and cloud infrastructure. Connect with me on Twitter @Yusadolat.

]]>

How to Speed Up AWS CodeBuild Docker Builds by 25% or more Using ECR as a Remote Cache

Yusuf Adeyemo — Mon, 20 Oct 2025 14:25:00 +0000

Have you ever sat there waiting for your CodeBuild project to rebuild your entire Docker image... again? Even though you only changed a single line of code?

Yeah, me too. And it's frustrating.

Today, I'm going to show you how I reduced our Docker build times from ~6 minutes down to ~2 minutes by implementing Amazon ECR as a persistent cache backend. This is based on an official AWS blog post, but I'll walk you through the practical implementation.

The Problem: Why Your Builds Are Slow

Here's the thing about AWS CodeBuild: every build runs in a completely fresh, isolated environment. That means:

No build artifacts carry over between builds
Every build starts from scratch
CodeBuild's "local cache" is temporary and unreliable (works on a "best-effort" basis)
If your builds happen at different times throughout the day, the local cache probably isn't helping you

So even if you only changed one line in your code, CodeBuild rebuilds every single Docker layer. Every. Single. Time.

The Solution: ECR Registry Cache Backend

The solution is surprisingly elegant: store your Docker layer cache persistently in Amazon ECR (Elastic Container Registry). Think of it as a separate "cache image" that lives alongside your actual application image.

Here's how it works:

First Build: Build from scratch, then export the cache to ECR as a separate image
Subsequent Builds: Import the cache from ECR, rebuild only what changed, export the updated cache back

The beauty? Your cache is always available, no matter when you trigger a build.

What You'll Need

Before we start, make sure you have:

An existing AWS CodeBuild project that builds Docker images
An ECR repository where your images are stored
IAM permissions for your CodeBuild role to push/pull from ECR (if you can already push images, you're good!)
About 10 minutes to implement this

Step-by-Step Implementation

Step 1: Understanding Your Current Buildspec

Your current buildspec probably looks something like this:

version: 0.2
phases:
  install:
    commands:
      - aws ecr get-login-password | docker login ...

  build:
    commands:
      - docker build -t myapp:latest .
      - docker tag myapp:latest $ECR_REPO:latest

  post_build:
    commands:
      - docker push $ECR_REPO:latest

This is the "basic" approach. Every build starts from zero.

[📸 IMAGE SUGGESTION: Split screen showing "Basic Build" vs "Cached Build" with layer rebuilding visualization]

Step 2: Add Cache Tag Variable

First, let's define a separate tag for our cache image. In your install phase, add:

install:
  commands:
    - CACHE_TAG=dev-cache  # or prod-cache, staging-cache, etc.
    - IMAGE_TAG=latest     # your actual app image tag

This creates a separate cache image (e.g., myapp:dev-cache) that's distinct from your application image (myapp:latest).

Step 3: Create the Buildx Builder

Here's the key part: Docker's default builder doesn't support registry cache backends. We need to create a new builder using buildx with the containerd driver.

Add this to your install phase:

install:
  commands:
    # ... your existing commands ...
    - docker buildx create --name containerd --driver=docker-container --driver-opt default-load=true --use || docker buildx use containerd

What's happening here?

docker buildx create: Creates a new builder instance
--driver=docker-container: Uses containerd (required for registry cache)
--driver-opt default-load=true: Loads built images into local Docker (important!)
|| docker buildx use containerd: If the builder already exists, just switch to it

Step 4: Replace Your Docker Build Command

Now replace your regular docker build command with the new buildx version:

build:
  commands:
    - |
      docker buildx build \
        --builder=containerd \
        --cache-from type=registry,ref=$ECR_REPO:$CACHE_TAG \
        --cache-to type=registry,ref=$ECR_REPO:$CACHE_TAG,mode=max,image-manifest=true \
        -t $ECR_REPO:$IMAGE_TAG \
        --load \
        .

Let me break down what each flag does:

--builder=containerd: Use the builder we just created
--cache-from type=registry,ref=$ECR_REPO:$CACHE_TAG: Import cache from ECR
--cache-to type=registry,ref=$ECR_REPO:$CACHE_TAG,mode=max,image-manifest=true: Export cache back to ECR
- mode=max: Export all layers (recommended for best caching)
- image-manifest=true: Required for ECR storage
-t $ECR_REPO:$IMAGE_TAG: Tag your final image as usual
--load: Load the built image into local Docker (so you can run it in post_build)
.: Your Dockerfile location

Step 5: Complete Example Buildspec

Here's what a complete, production-ready buildspec looks like:

version: 0.2
phases:
  install:
    commands:
      - echo Logging in to Amazon ECR
      - aws ecr get-login-password --region $AWS_REGION | docker login --username AWS --password-stdin $AWS_ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com
      - CACHE_TAG=dev-cache
      - IMAGE_TAG=latest
      - ECR_REPO=$AWS_ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com/myapp
      - docker buildx create --name containerd --driver=docker-container --driver-opt default-load=true --use || docker buildx use containerd

  build:
    commands:
      - echo Build started on `date`
      - |
        docker buildx build \
          --builder=containerd \
          --cache-from type=registry,ref=$ECR_REPO:$CACHE_TAG \
          --cache-to type=registry,ref=$ECR_REPO:$CACHE_TAG,mode=max,image-manifest=true \
          -t $ECR_REPO:$IMAGE_TAG \
          --load \
          .
      - docker tag $ECR_REPO:$IMAGE_TAG $ECR_REPO:latest

  post_build:
    commands:
      - echo Build completed on `date`
      - docker push $ECR_REPO:$IMAGE_TAG
      - docker push $ECR_REPO:latest

artifacts:
  files: 
    - imageDefinitions.json

Step 6: Update Your CodeBuild Project

You can update your buildspec in two ways:

Option 1: If your buildspec is in your repo
Just commit the changes and push. CodeBuild will pick up the new buildspec automatically.

Option 2: If your buildspec is defined in CodeBuild
Use the AWS CLI:

aws codebuild update-project --name your-project-name --cli-input-json file://buildspec.json

Or update it through the AWS Console: CodeBuild → Your Project → Edit → Buildspec

What to Expect: First Build vs Subsequent Builds

First Build (The Investment)

Your first build after implementing this will actually take slightly longer (maybe 30-60 seconds more). Don't panic! This is normal.

Here's what's happening:

Creating the buildx builder (~5-10 seconds)
Attempting to import cache (fails - no cache exists yet)
Building all layers from scratch
Exporting the cache to ECR (new step, adds ~20-40 seconds)

You'll see messages like:

=> importing cache manifest from $ECR_REPO:dev-cache
=> error: not found

This is expected! The cache doesn't exist yet.

Subsequent Builds (The Payoff)

This is where the magic happens. Your next builds will:

Successfully import the cache from ECR
Identify which layers haven't changed
Reuse cached layers (fast!)
Rebuild only the changed layers
Export the updated cache

Expected time savings:

Before: 6-7 minutes (full rebuild every time)
After: 5-5.5 minutes (25-30% faster!)
Savings: 1-2 minutes per build

If you're doing 10 builds a day, that's 10-20 minutes saved daily. Over a month? That's 5-10 hours of compute time and costs saved.

Verifying It's Working

After your first build completes, check your ECR repository. You should now see two image tags:

[📸 IMAGE SUGGESTION: ECR Console screenshot showing two images: "myapp:latest" and "myapp:dev-cache"]

Your application image (e.g., latest)
Your cache image (e.g., dev-cache)

The cache image will be roughly the same size as your application image - this is normal! It's storing all the layer information.

Troubleshooting Common Issues

Issue 1: "buildx: command not found"

Solution: Update your CodeBuild image to a newer version. Use aws/codebuild/standard:7.0 or later (or the ARM equivalent).

Issue 2: Cache Import Keeps Failing

Solution: Check your IAM permissions. Your CodeBuild role needs:

ecr:BatchGetImage
ecr:GetDownloadUrlForLayer
ecr:BatchCheckLayerAvailability
ecr:PutImage
ecr:InitiateLayerUpload
ecr:UploadLayerPart
ecr:CompleteLayerUpload

Issue 3: Build Hangs at "exporting cache"

Solution: Make sure privilegedMode: true is enabled in your CodeBuild environment settings. This is required for Docker-in-Docker operations.

Advanced: Multi-Environment Setup

If you have multiple environments (dev, staging, prod), use different cache tags for each:

- CACHE_TAG=${ENVIRONMENT}-cache  # Results in: dev-cache, staging-cache, prod-cache

This way:

Dev builds don't invalidate staging cache
Each environment maintains its own optimized cache
You can still share a base cache if needed

Cost Considerations

Storage Cost: You're now storing an additional cache image in ECR. At roughly the same size as your app image, this might add $0.10-0.50/month per repository depending on image size.

Compute Savings: Faster builds = less compute time. If you're saving 1-2 minutes per build and doing 10 builds/day, that's roughly 10-20 fewer compute hours per month. At ~$0.005/minute for BUILD_GENERAL1_SMALL, you could save $3-6/month.

Net Result: Typically a small net savings, plus the huge developer experience win of faster feedback loops.

Conclusion

By implementing ECR as a remote cache backend for your CodeBuild Docker builds, you get:

✅ 25-30% faster build times

✅ Persistent, reliable caching across all builds

✅ Better layer reuse with intelligent cache management

✅ Minimal code changes (just updating your buildspec)

✅ Cost savings from reduced compute time

The implementation is straightforward, and the benefits are immediate (after the first build). Give it a try on your next project!

References

Got questions or run into issues? Drop a comment below - I'd love to hear about your experience implementing this!

How to Speed Up AWS CodeBuild Docker Builds by 25% or more Using ECR as a Remote Cache

Yusuf Adeyemo — Mon, 20 Oct 2025 08:50:14 +0000

Have you ever sat there waiting for your CodeBuild project to rebuild your entire Docker image... again? Even though you only changed a single line of code?

Yeah, me too. And it's frustrating.

Today, I'm going to show you how I reduced our Docker build times from ~7 minutes down to ~2 minutes by implementing Amazon ECR as a persistent cache backend. This is based on an official AWS blog post, but I'll walk you through the practical implementation.

The Problem: Why Your Builds Are Slow

Here's the thing about AWS CodeBuild: every build runs in a completely fresh, isolated environment. That means:

No build artifacts carry over between builds
Every build starts from scratch
CodeBuild's "local cache" is temporary and unreliable (works on a "best-effort" basis)
If your builds happen at different times throughout the day, the local cache probably isn't helping you

So even if you only changed one line in your code, CodeBuild rebuilds every single Docker layer. Every. Single. Time.

The Solution: ECR Registry Cache Backend

Here's how it works:

First Build : Build from scratch, then export the cache to ECR as a separate image
Subsequent Builds : Import the cache from ECR, rebuild only what changed, export the updated cache back

The beauty? Your cache is always available, no matter when you trigger a build.

What You'll Need

Before we start, make sure you have:

An existing AWS CodeBuild project that builds Docker images
An ECR repository where your images are stored
IAM permissions for your CodeBuild role to push/pull from ECR (if you can already push images, you're good!)
About 10 minutes to implement this

Step-by-Step Implementation

Step 1: Understanding Your Current Buildspec

Your current buildspec probably looks something like this:

version: 0.2phases: install: commands: - aws ecr get-login-password | docker login ... build: commands: - docker build -t myapp:latest . - docker tag myapp:latest $ECR_REPO:latest post_build: commands: - docker push $ECR_REPO:latest

This is the "basic" approach. Every build starts from zero.

[📸 IMAGE SUGGESTION: Split screen showing "Basic Build" vs "Cached Build" with layer rebuilding visualization]

Step 2: Add Cache Tag Variable

First, let's define a separate tag for our cache image. In your install phase, add:

install: commands: - CACHE_TAG=dev-cache # or prod-cache, staging-cache, etc. - IMAGE_TAG=latest # your actual app image tag

This creates a separate cache image (e.g., myapp:dev-cache) that's distinct from your application image (myapp:latest).

Step 3: Create the Buildx Builder

Here's the key part: Docker's default builder doesn't support registry cache backends. We need to create a new builder using buildx with the containerd driver.

Add this to your install phase:

install: commands: # ... your existing commands ... - docker buildx create --name containerd --driver=docker-container --driver-opt default-load=true --use || docker buildx use containerd

What's happening here?

docker buildx create: Creates a new builder instance
--driver=docker-container: Uses containerd (required for registry cache)
--driver-opt default-load=true: Loads built images into local Docker (important!)
|| docker buildx use containerd: If the builder already exists, just switch to it

Step 4: Replace Your Docker Build Command

Now replace your regular docker build command with the new buildx version:

build: commands: - | docker buildx build \ --builder=containerd \ --cache-from type=registry,ref=$ECR_REPO:$CACHE_TAG \ --cache-to type=registry,ref=$ECR_REPO:$CACHE_TAG,mode=max,image-manifest=true \ -t $ECR_REPO:$IMAGE_TAG \ --load \ .

Let me break down what each flag does:

--builder=containerd: Use the builder we just created
--cache-from type=registry,ref=$ECR_REPO:$CACHE_TAG: Import cache from ECR
--cache-to type=registry,ref=$ECR_REPO:$CACHE_TAG,mode=max,image-manifest=true: Export cache back to ECR
-t $ECR_REPO:$IMAGE_TAG: Tag your final image as usual
--load: Load the built image into local Docker (so you can run it in post_build)
.: Your Dockerfile location

Step 5: Complete Example Buildspec

Here's what a complete, production-ready buildspec looks like:

version: 0.2phases: install: commands: - echo Logging in to Amazon ECR - aws ecr get-login-password --region $AWS_REGION | docker login --username AWS --password-stdin $AWS_ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com - CACHE_TAG=dev-cache - IMAGE_TAG=latest - ECR_REPO=$AWS_ACCOUNT_ID.dkr.ecr.$AWS_REGION.amazonaws.com/myapp - docker buildx create --name containerd --driver=docker-container --driver-opt default-load=true --use || docker buildx use containerd build: commands: - echo Build started on `date` - | docker buildx build \ --builder=containerd \ --cache-from type=registry,ref=$ECR_REPO:$CACHE_TAG \ --cache-to type=registry,ref=$ECR_REPO:$CACHE_TAG,mode=max,image-manifest=true \ -t $ECR_REPO:$IMAGE_TAG \ --load \ . - docker tag $ECR_REPO:$IMAGE_TAG $ECR_REPO:latest post_build: commands: - echo Build completed on `date` - docker push $ECR_REPO:$IMAGE_TAG - docker push $ECR_REPO:latestartifacts: files: - imageDefinitions.json

Step 6: Update Your CodeBuild Project

You can update your buildspec in two ways:

Option 1: If your buildspec is in your repo Just commit the changes and push. CodeBuild will pick up the new buildspec automatically.

Option 2: If your buildspec is defined in CodeBuild Use the AWS CLI:

aws codebuild update-project --name your-project-name --cli-input-json file://buildspec.json

Or update it through the AWS Console: CodeBuild Your Project Edit Buildspec

What to Expect: First Build vs Subsequent Builds

First Build (The Investment)

Your first build after implementing this will actually take slightly longer (maybe 30-60 seconds more). Don't panic! This is normal.

Here's what's happening:

Creating the buildx builder (~5-10 seconds)
Attempting to import cache (fails - no cache exists yet)
Building all layers from scratch
Exporting the cache to ECR (new step, adds ~20-40 seconds)

You'll see messages like:

=> importing cache manifest from $ECR_REPO:dev-cache=> error: not found

This is expected! The cache doesn't exist yet.

Subsequent Builds (The Payoff)

This is where the magic happens. Your next builds will:

Successfully import the cache from ECR
Identify which layers haven't changed
Reuse cached layers (fast!)
Rebuild only the changed layers
Export the updated cache

Expected time savings:

Before : 6-7 minutes (full rebuild every time)
After : 5-5.5 minutes (25-30% faster!)
Savings : 1-2 minutes per build

If you're doing 10 builds a day, that's 10-20 minutes saved daily. Over a month? That's 5-10 hours of compute time and costs saved.

Verifying It's Working

After your first build completes, check your ECR repository. You should now see two image tags :

Your application image (e.g., latest)
Your cache image (e.g., dev-cache)

The cache image will be roughly the same size as your application image - this is normal! It's storing all the layer information.

Troubleshooting Common Issues

Issue 1: "buildx: command not found"

Solution : Update your CodeBuild image to a newer version. Use aws/codebuild/standard:7.0 or later (or the ARM equivalent).

Issue 2: Cache Import Keeps Failing

Solution : Check your IAM permissions. Your CodeBuild role needs:

ecr:BatchGetImage
ecr:GetDownloadUrlForLayer
ecr:BatchCheckLayerAvailability
ecr:PutImage
ecr:InitiateLayerUpload
ecr:UploadLayerPart
ecr:CompleteLayerUpload

Issue 3: Build Hangs at "exporting cache"

Solution : Make sure privilegedMode: true is enabled in your CodeBuild environment settings. This is required for Docker-in-Docker operations.

Advanced: Multi-Environment Setup

If you have multiple environments (dev, staging, prod), use different cache tags for each:

- CACHE_TAG=${ENVIRONMENT}-cache # Results in: dev-cache, staging-cache, prod-cache

This way:

Dev builds don't invalidate staging cache
Each environment maintains its own optimized cache
You can still share a base cache if needed

Cost Considerations

Storage Cost : You're now storing an additional cache image in ECR. At roughly the same size as your app image, this might add $0.10-0.50/month per repository depending on image size.

Compute Savings : Faster builds = less compute time. If you're saving 1-2 minutes per build and doing 10 builds/day, that's roughly 10-20 fewer compute hours per month. At ~$0.005/minute for BUILD_GENERAL1_SMALL, you could save $3-6/month.

Net Result : Typically a small net savings, plus the huge developer experience win of faster feedback loops.

Conclusion

By implementing ECR as a remote cache backend for your CodeBuild Docker builds, you get:

Faster build times

Persistent, reliable caching across all builds

Better layer reuse with intelligent cache management

Minimal code changes (just updating your buildspec)

Cost savings from reduced compute time

The implementation is straightforward, and the benefits are immediate (after the first build). Give it a try on your next project!

References

Got questions or run into issues? Drop a comment below - I'd love to hear about your experience implementing this!

]]>

Do You Really Know the Difference Between L1, L2, and L3 CDK Constructs?

Yusuf Adeyemo — Sat, 26 Jul 2025 08:23:36 +0000

After you complete this article, you will have a solid understanding of:

What L1, L2, and L3 constructs actually are and when to use each
Why AWS created three different abstraction levels (and the hidden benefits)
How to avoid the most common CDK construct mistakes
When to break the rules and mix construct levels

Have You Ever Been Confused by CDK Construct Levels?

If you've ever started learning AWS CDK, you've probably encountered code like this and wondered why there are so many ways to create the same S3 bucket:

// Wait, what? Three different ways to create a bucket? 
import { CfnBucket } from 'aws-cdk-lib/aws-s3'; 
import { Bucket } from 'aws-cdk-lib/aws-s3'; 
import { StaticWebsite } from '@aws-solutions-constructs/aws-s3-cloudfront';
// Which one should I use? 🤔

And then you see this error that makes you question everything:

Error: Cannot use property type 'BucketProps' with L1 construct 'CfnBucket'

"But they're both S3 buckets! Why can't I use the same properties?"

Let me help you understand these construct levels once and for all.

What Are CDK Constructs Anyway?

Think of CDK constructs as LEGO blocks for your cloud infrastructure. Just like LEGO has basic bricks, specialized pieces, and complete sets, CDK has three levels of constructs.

Level 1 (L1) Constructs: The Raw CloudFormation Experience

L1 constructs are the most basic building blocks. They start with Cfn (short for CloudFormation) and map directly to CloudFormation resources. No magic, no shortcuts.

import { CfnBucket } from 'aws-cdk-lib/aws-s3';

const bucket = new CfnBucket(this, 'MyL1Bucket', {
  bucketName: 'my-raw-bucket-2025',
  versioningConfiguration: {
    status: 'Enabled'
  },
  publicAccessBlockConfiguration: {
    blockPublicAcls: true,
    blockPublicPolicy: true,
    ignorePublicAcls: true,
    restrictPublicBuckets: true
  }
});

Notice how verbose this is? You have to configure EVERYTHING manually. It's like writing CloudFormation in TypeScript.

When Would You Ever Use L1 Constructs?

Brand New AWS Services - When AWS releases a new service, L1 support comes first
Debugging L2/L3 Issues - Sometimes you need to see what's really happening
Migrating from CloudFormation - Direct 1:1 mapping makes migration easier
Edge Cases - When you need a specific CloudFormation property not exposed in L2

Level 2 (L2) Constructs: The Sweet Spot

L2 constructs are what most developers use daily. They provide sensible defaults, helper methods, and hide complexity while still giving you control.

import { Bucket, BucketEncryption } from 'aws-cdk-lib/aws-s3';

const bucket = new Bucket(this, 'MyL2Bucket', {
  bucketName: 'my-friendly-bucket-2025',
  versioned: true,
  encryption: BucketEncryption.S3_MANAGED,
  removalPolicy: RemovalPolicy.DESTROY // Much cleaner!
});

// Look at these helper methods! 
bucket.grantRead(myLambdaFunction);
bucket.addLifecycleRule({
  expiration: Duration.days(90)
});

See the difference? L2 constructs:

Use friendly property names (versioned vs versioningConfiguration)
Provide helper methods (grantRead())
Set security best practices by default
Handle resource dependencies automatically

Level 3 (L3) Constructs: Complete Solutions

L3 constructs (also called patterns) are pre-built architectures for common use cases. They combine multiple resources into a working solution.

import { StaticWebsite } from '@aws-solutions-constructs/aws-s3-cloudfront';

const website = new StaticWebsite(this, 'MyWebsite', {
  websiteIndexDocument: 'index.html',
  websiteErrorDocument: 'error.html'
});

// That's it! You just created:
// - S3 bucket with proper website configuration
// - CloudFront distribution
// - Origin Access Identity
// - Proper IAM policies
// - HTTPS redirect
// - Security headers

With just a few lines, you get a production-ready static website setup that would take hundreds of lines in L1.

Common Mistakes That Will Drive You Crazy

Mistake #1: Mixing Property Types

// 🚫 This won't work!
const bucket = new CfnBucket(this, 'MyBucket', {
  encryption: BucketEncryption.S3_MANAGED // L2 property type
});

// ✅ Use the correct L1 property type
const bucket = new CfnBucket(this, 'MyBucket', {
  bucketEncryption: {
    serverSideEncryptionConfiguration: [{
      serverSideEncryptionByDefault: {
        sseAlgorithm: 'AES256'
      }
    }]
  }
});

Mistake #2: Assuming L3 Constructs Are Always Better

// Using L3 when you need specific customization
const website = new StaticWebsite(this, 'MyWebsite', {
  // Oh no! I can't set specific CloudFront behaviors
  // or custom cache policies here! 😱
});

// Sometimes L2 gives you more control
const bucket = new Bucket(this, 'WebBucket');
const distribution = new CloudFrontWebDistribution(this, 'MyDist', {
  // Full control over every setting
});

Mistake #3: Not Using Escape Hatches

What if you need to modify an L2 construct's underlying L1 resource?

const bucket = new Bucket(this, 'MyBucket');

// Access the L1 construct (escape hatch)
const cfnBucket = bucket.node.defaultChild as CfnBucket;

// Now you can set ANY CloudFormation property
cfnBucket.analyticsConfigurations = [{
  id: 'my-analytics',
  storageClassAnalysis: {
    dataExport: {
      destination: {
        bucketArn: 'arn:aws:s3:::my-analytics-bucket'
      }
    }
  }
}];

The Hidden Benefits of Each Level

L1 Benefits You Didn't Know About

Immediate AWS Feature Support - No waiting for CDK updates
CloudFormation Parity - Easy to convert existing templates
Learning Tool - Understand what L2 constructs do under the hood

L2 Benefits That Save Time

Automatic Security Defaults - Encryption enabled by default
Cross-Service Integration - grant* methods handle IAM for you
Type Safety - Catch errors at compile time, not deployment

L3 Benefits for Real Projects

Proven Architectures - AWS Solutions Constructs follow best practices
Compliance Ready - Many patterns are pre-validated for security
Rapid Prototyping - Get a working system in minutes

Creating Your Own L3 Construct

Here's a practical example of creating your own pattern:

import { Construct } from 'constructs';
import { Bucket, BucketEncryption } from 'aws-cdk-lib/aws-s3';
import { Function, Runtime, Code } from 'aws-cdk-lib/aws-lambda';
import * as path from 'path';

export class SecureDataProcessor extends Construct {
  public readonly bucket: Bucket;
  public readonly processor: Function;

  constructor(scope: Construct, id: string) {
    super(scope, id);

    // Create encrypted bucket
    this.bucket = new Bucket(this, 'DataBucket', {
      encryption: BucketEncryption.KMS_MANAGED,
      versioned: true,
      enforceSSL: true
    });

    // Create processing Lambda
    this.processor = new Function(this, 'Processor', {
      runtime: Runtime.NODEJS_18_X,
      handler: 'index.handler',
      code: Code.fromAsset(path.join(__dirname, 'lambda'))
    });

    // Wire them together
    this.bucket.grantRead(this.processor);
    this.bucket.addEventNotification(
      EventType.OBJECT_CREATED,
      new LambdaDestination(this.processor)
    );
  }
}

// Now anyone can use your pattern!
const dataProcessor = new SecureDataProcessor(this, 'MyProcessor');

When to Use Each Construct Level

Use L1 when:

You need bleeding-edge AWS features
Migrating from CloudFormation
Debugging CDK issues
You need a specific CloudFormation property

Use L2 when:

Building most production applications
You want security best practices by default
You need to integrate multiple services
You value developer productivity

Use L3 when:

Implementing common patterns
Rapid prototyping
Enforcing organizational standards
You don't need heavy customization

The Future of CDK Constructs

AWS is continuously improving CDK constructs. New services get L1 support immediately through CloudFormation, L2 constructs follow within weeks or months, and the community creates L3 patterns for common use cases.

Remember: There's no "wrong" construct level. Each serves a purpose, and experienced CDK developers often mix levels within the same application.

Was this article helpful for you? If so, kindly follow on twitter @yusadolat

]]>

How to Pay AWS Bills in Naira: A Quick Guide

Yusuf Adeyemo — Tue, 14 Jan 2025 20:03:44 +0000

With AWS now supporting Naira, you can skip juggling foreign exchange and just focus on building. Heres how to set it up:

Log In to Your AWS Account

Head to the AWS Console and sign in as usual.

Go to Billing and Cost Management

Youll find this under the main menu.

Preferences and Settings

Once there, look for an option labeled Payment Preferences.

Edit Payment Currency

Click Edit , then pick Nigerian Naira (NGN) from the dropdown.

Save Changes

Congrats, future invoices will be issued in Naira!

Why This Matters

No More FX Fees Youre not burning extra cash on currency conversions.

Local Payment Ease Its simpler to manage local transactions and budgets.

Aligns with the Lagos Local Zone Perfect if youre leveraging AWSs local zone in Nigeria.

Thats it! Its quick, its easy, and it saves you from fiddling with exchange rates. Now you can invest the difference in testing, automations, or that next big idea.

Peace to all, and happy building!

]]>

Nomad 101: The Simpler, Smarter Way to Orchestrate Applications

Yusuf Adeyemo — Tue, 31 Dec 2024 17:34:58 +0000

Nomad is a personal favorite when I need a straightforward, single-binary orchestrator that just works. Its built by HashiCorp, the folks behind Terraform and Vault, and it takes a minimalistic approach to scheduling and managing containerized (and even non-containerized) workloads. Nomad might be the perfect fit if youve ever felt that Kubernetes is overkill for a simpler workload.

In this post, Ill walk you through installing Nomad, spinning it up in a small environment, and running a workload to see it in action. By the end, youll have a solid hands-on feel for how to use Nomad. Lets dive right in.

Why Nomad?

For me, Nomad offers a couple of killer advantages:

Simplicity : Nomad is a single, self-contained binary that can manage containers, VMs, and standalone applications. Configuration is straightforward and uses HCL (HashiCorp Configuration Language), which you might already know from Terraform.
Low Overhead : In contrast to something like Kubernetes, which demands multiple components (etcd, kube-scheduler, kube-apiserver, etc.), Nomad keeps the architecture lean, meaning fewer moving parts and less operational complexity.
Scale Without the Bloat : Just because its simple doesnt mean its small-time. Nomad can run at massive scale. Start small on a single node, then grow into a cluster as your needs evolve.
Broad Workload Support : Containerized apps are the norm these days, but if you have legacy apps or specialized workloads, Nomad accommodates them too. This flexibility makes it easier to transition older systems into orchestrated environments without rewriting everything.

Setting Up Nomad Locally

Lets talk about setting up a Nomad environment on your local machine for a quick test. Ill assume youre running on some flavor of Linux or macOS. If youre on Windows, you can still follow along using WSL2 or a VM.

Download Nomad

Head over to the official Nomad Releases page and download the appropriate binary. Extract it, move it to a directory in your PATH (like /usr/local/bin), and youre good to go. For instance:
Development Agent

Nomad has a dev mode, which is a single-process setup that runs the server and client in one goperfect for local testing. Simply run:

Nomad Architecture at a Glance

In a more robust setup, Nomad is typically deployed as a cluster of server nodes and client nodes:

Server nodes handle scheduling decisions and maintain cluster state.
Client nodes run workloads assigned to them by the servers.

But for now, dev mode is all we need. Later on, you could spin up a 3-node server cluster with as many clients as you want.

Running Your First Nomad Job

A Nomad job describes what you want to run, how many instances, resource constraints, etc. Jobs are written in HCL, so itll feel familiar if youve ever used Terraform. Lets do a quick example by running a Docker-based web server.

Basic HCL Job File

Create a file called nginx.nomad:

job "nginx-web" { datacenters = ["dc1"] type = "service" group "web-group" { count = 1 task "web" { driver = "docker" config { image = "nginx:latest" ports = ["http"] } resources { cpu = 100 memory = 128 } } network { port "http" { static = 8080 } } }}

Lets break it down:

job "nginx-web" : Defines our job name and type. Were calling it a service because its a long-running service.
group "web-group" : A group can contain multiple tasks that share resources and networking. Here, we only have one task.
task "web" : Tells Nomad to run an Nginx container. We specify the docker driver.
network : Maps the containers port to a static host port (8080 in this case) so you can access it on localhost:8080.

Run the Job

Launch it with:

nomad job run nginx.nomad

Nomad will parse the file, create the job, and schedule it on the local dev client. If all goes well, youll see output indicating the job has been placed.

Verify Its Running

Head to your browser at http://localhost:4646 and click on Jobs. You should see nginx-web running. Now try http://localhost:8080 in your browser. Nginxs default Welcome page means its working!

Scaling the Service

Nomad makes scaling super easy. Just update the count parameter in your job file. For instance, change it to:

count = 2

Then run:

nomad job run nginx.nomad

Nomad will place an additional instance of the container, though in dev mode youre still on a single node, so youll have multiple containers on the same host. In a multi-node cluster, Nomad automatically figures out which clients have room.

Stopping or Updating the Job

If you want to stop the job, you can run:

nomad job stop nginx-web

For updates, just modify the HCL file (like changing the Docker image to a different version), then re-run nomad job run nginx.nomad. Nomad will handle rolling updates gracefully, spinning up new tasks before shutting down old ones (as long as you specify appropriate update stanzas).

Integrating with Other HashiCorp Tools

Because Nomad shares the same style of configuration as Terraform and the same developer DNA as Vault and Consul, its easy to create an entire stack:

Consul for service discovery and dynamic DNS.
Vault for secrets management.
Terraform for provisioning the underlying infrastructure.

Nomad can automatically register services with Consul, making them discoverable to other services in your environment. Storing secrets in Vault means you can dynamically inject credentials into your Nomad jobs. It all plays nicely together.

Why I Use Nomad Over Alternatives

Ive used Kubernetes for years, but Nomad is my go-to when:

Speed of Setup : Nomad dev mode is unbelievably quick. One binary, one command, done.
Fewer Dependencies : I dont need etcd or a separate container runtime beyond Docker. Less to break, less to learn.
Flexibility : I can run Docker tasks, raw exec tasks, or even handle batch jobs and system workloads in a single cluster.

Dont get me wrong Kubernetes excels in large, complex ecosystems. But if you want a more lightweight orchestrator or have a hybrid mix of containerized and legacy apps, Nomads a breath of fresh air.

Pro Tips (Anticipating What You Might Need Next)

High Availability : If you plan to run in production, spin up at least three Nomad server nodes. That ensures if one server goes down, the cluster can still schedule workloads.
Autopilot : Nomads built-in autopilot features let you automatically manage upgrades, Raft snapshots, and more to keep the cluster healthy.
Authentication and ACLs : In a multi-user setup, you can integrate Nomads ACL system to restrict who can submit jobs or read cluster data.
Plugins : There are driver plugins for everything from Docker to QEMU to AWS ECS tasks. You can run basically anything that can be launched from a command line or third-party tool.
Monitoring : Nomad exposes metrics that are easy to integrate with Prometheus, Grafana, or whatever your favorite monitoring stack is.

Final Thoughts

Nomad may look unassuming as a single binary, but dont let that fool you. Its a robust orchestrator that simplifies complex workload management. Whether youre prototyping a new service, gradually migrating from manual server management, or just want to avoid the overhead of a full-fledged Kubernetes stack, Nomad can handle it.

Why not give it a shot in your own environment? If youve got that messy monolith or a small container workload, Nomad might be exactly the tool you need to keep everything running smoothly without drowning in complexity.

Sources

Hope this helps you get started with Nomad. Let me know if you run into any snags or come up with a clever integration Im always interested in new ways to push this awesome orchestrator.

]]>

How I Leverage Raspberry Pi as a DevOps Engineer

Yusuf Adeyemo — Mon, 16 Dec 2024 11:00:37 +0000

As a DevOps engineer, I'm always looking for a cost-effective, reliable, and flexible way to prototype new ideas without overcommitting infrastructure resources. Sure, spinning up EC2 instances or provisioning dedicated hardware works, but when you want a low-power, low-cost sandbox, the Raspberry Pi is hard to beat. Its an affordable, credit-card-sized computing powerhouse that helps me test concepts, automate environments, and even experiment with local AI without racking up unnecessary cloud fees or dealing with heavy metal servers.

In this post, I'll share some real-world ways I integrate Raspberry Pi devices into my workflow. If you've never considered them as part of a professional DevOps toolkit, I hope this gives you a few reasons to start.

What is a Raspberry Pi?

If you've never touched one, think of the Raspberry Pi as a tiny Linux-based computer board with just enough CPU, RAM, and storage to run a surprising range of workloads. It's been wildly popular among hobbyists, educators, and professionals alike. Thanks to its Linux roots, you can tap into a massive ecosystem of software, scripting, containers, and automation tools that feel instantly familiar to anyone from a DevOps background.

Why Raspberry Pi Fits My Needs

As a DevOps engineer, I've got plenty of choices for running workloads. But the Pi hits a sweet spot when I need something quick, cheap, and on-prem:

Cost-Effective : For the price of a mid-tier cloud instance running a few weeks, I can own a Pi outright and reuse it a million times over.
Energy-Efficient : A Pi draws minimal power, so I can keep it running 24/7 without worrying about my electric bill.
Exceptionally Versatile : Its a lab in a box. CI/CD runners, IoT hubs, mini Kubernetes clusters, AI inferencing boxes, local proxies, you name it.

How I Use Raspberry Pi in My Workflow

1. Prototyping and Experimental Builds

When experimenting with a new microservice, pipeline, or integration, I often spin it up on a Pi first. This gives me a stable, always-on environment to validate code, run Docker containers, test APIs, and refine configurations. Its a great way to ensure my code and infrastructure definitions hold up before I commit cloud spend.

2. Home Automation and IoT Management

I like to say that my home is my first production environment. Using a Pi as a hub, paired with something like Home Assistant I manage a network of sensors, lights, and other IoT devices. Not only is it fun, but it also lets me practice edge automation. This experience often translates back into my professional work, where edge computing scenarios are becoming more common.

3. Self-Hosted GitHub Actions Runners

If you've worked with GitHub Actions, you know that hosted runners can quickly rack up costs or queue times. By using a Pi as a self-hosted runner, I keep certain build and test pipelines local and cost-controlled. Best of all, I have full control over the environment and dependencies, making it easy to debug issues right in my home office.

4. Local AI Experiments

While you won't train GPT-4 on a Raspberry Pi, its still possible to run smaller models like Googles Gemma2 (2 billion parameters) for inference tasks. This is a great way to experiment with local AI workloads or test model-serving pipelines without relying on GPU-backed cloud instances. It's not going to replace a beefy workstation, but it's enough to poke around with models and APIs before deciding to scale up.

5. Network-Attached Storage (NAS) and Local File Serving

If I need a quick-and-dirty NAS solution, I can set up a Pi with Samba or OpenMediaVault, attach some external storage, and voil: a lightweight NAS on my local network. It's not enterprise-level, but it's perfect for stashing logs, artifacts, or just sharing files among devices at home.

Why It Works for Me

Raspberry Pi devices are more than just cheap boards; they represent a frictionless approach to experimentation. Instead of spending hours setting up cloud VMs or maintaining bulky servers, I have a small fleet of Pis that act as a test bed for ideas. They let me:

Quickly spin up and tear down environments on a budget.
Learn and iterate with minimal risk.
Scale horizontally by adding more boards when I need them.
Develop intuition for edge, IoT, and ARM-based workloads.

Final Thoughts

The Raspberry Pi offers a unique blend of accessibility, affordability, and versatility. Whether I'm refining a new CI pipeline, tinkering with home automation, or trialing lightweight AI inference, the Pi is my go-to platform for hands-on exploration. Its a genuine force multiplier that's expanded the way I think about infrastructure and small-scale deployments.

What about you? How have you put Raspberry Pi to work? If youve got a unique use case or a clever hack, let me know. Im always looking for fresh ways to push these tiny boards to their limits.

TDD vs BDD: Navigating the Testing Landscape in Modern Software Development

Yusuf Adeyemo — Tue, 27 Aug 2024 08:00:12 +0000

Introduction

In the ever-evolving world of software development, testing methodologies play a crucial role in ensuring the quality and reliability of applications. Two prominent approaches that have gained significant traction in recent years are Test-Driven Development (TDD) and Behavior-Driven Development (BDD). While both methodologies share some common ground, they each bring unique perspectives to the testing process. This article delves into the intricacies of TDD and BDD, exploring their benefits, key differences, and how they can be effectively implemented in software projects.

Understanding Test-Driven Development (TDD)
Exploring Behavior-Driven Development (BDD)
Comparing TDD and BDD
Implementing TDD and BDD in Your Projects
Conclusion

Understanding Test-Driven Development (TDD)

Test-Driven Development is a software development process that relies on the repetition of short development cycles. This methodology encourages simple designs and instills confidence in the code by ensuring that every piece of functionality is thoroughly tested.

The TDD Process

The TDD process follows a specific cycle:

Write a test for a new feature before implementing the code.
Run the new test to verify that it fails (as expected).
Write the minimum amount of code necessary to make the test pass.
Run all tests to ensure the new code passes without breaking existing functionality.
Refactor the code to improve its structure and remove any duplication.
Repeat the cycle for each new feature or functionality.

Benefits of TDD

Encourages simple, modular designs
Provides immediate feedback on code correctness
Builds a comprehensive suite of unit tests
Improves code quality and reduces bugs
Facilitates easier refactoring and maintenance

Exploring Behavior-Driven Development (BDD)

Behavior-Driven Development is an agile software development process that extends the principles of TDD. BDD emphasizes collaboration among developers, quality assurance professionals, and business partners to create a shared understanding of how an application should behave.

Key Features of BDD

Utilizes domain-specific scripting languages (DSLs)
Defines user behavior in simple English
Converts English descriptions into automated test scripts
Focuses on the behavior of the application from an end-user perspective

BDD Scenario Examples

BDD often uses scenario-based descriptions to define expected behavior. For example:

Scenario: User adds an item to their shopping cart Given the user is on the product details page When the user selects a size "Medium" And the user clicks the "Add to Cart" button Then the item should be added to the user's shopping cart And the cart total should increase by 1 And the user should see a confirmation message "Item added to cart"

Comparing TDD and BDD

While TDD and BDD share some common ground, they have distinct focuses and approaches:

Shared Benefits

Both TDD and BDD offer several advantages to development teams:

Early detection of errors in requirements
Improved communication between team members
Reduced overall development costs
Higher code quality and fewer bugs

Focus and Approach

TDD focuses on the functionality of individual components
BDD emphasizes the behavior of the application from a user's perspective
TDD tests are typically written in the same programming language as the application
BDD tests are often written in a more accessible, natural language format

Implementing TDD and BDD in Your Projects

To successfully implement TDD or BDD in your software projects:

Choose the appropriate methodology based on your project's needs and team structure
Invest in training and tools to support the chosen approach
Start small and gradually expand the use of TDD or BDD across your projects
Regularly review and refine your testing processes
Foster a culture of collaboration and continuous improvement

Conclusion

Both Test-Driven Development and Behavior-Driven Development offer valuable approaches to software testing and development. By understanding the strengths and differences of each methodology, development teams can make informed decisions about which approach best suits their projects. Whether you choose TDD, BDD, or a combination of both, implementing these methodologies can lead to higher quality software, improved team collaboration, and more satisfied end-users.

References:

Beck, K. (2002). Test-Driven Development: By Example. Addison-Wesley Professional.t ]]>

Enhancing Microservice Communication in AWS ECS with Service Discovery Techniques

Yusuf Adeyemo — Sun, 11 Feb 2024 08:48:00 +0000

Service discovery is a vital component of modern distributed systems, enabling seamless communication and dynamic scaling in environments where services frequently change IPs, ports, or even hosts. AWS Elastic Container Service (ECS) integrates seamlessly with service discovery mechanisms, simplifying the deployment and operation of microservices architectures. In this comprehensive guide, we delve into the intricacies of service discovery within ECS, ensuring your applications remain resilient and scalable.

Understanding Service Discovery

At its core, service discovery facilitates the dynamic detection and interaction among services in a distributed ecosystem. The challenge lies in the fluid nature of these services, which may traverse across different environments, necessitating a flexible approach to maintain connectivity. Service discovery transcends the limitations of static configurations, allowing services to communicate based on logical identifiers rather than hard-coded network addresses.

The Role of Service Discovery in ECS

Amazon ECS simplifies service discovery by leveraging AWS Cloud Map, a fully managed service registry that automates the discovery of ECS services. Cloud Map enables your applications to discover resources by name, eliminating the need for manual IP management or service configuration. This abstraction not only enhances flexibility but also significantly reduces the overhead associated with deploying and managing microservices.

Implementing Service Discovery in ECS

To utilize service discovery in ECS, you begin by registering your services with AWS Cloud Map. This process involves creating a namespace, which serves as a container for all service instances. Within this namespace, you register service names that correspond to your ECS services. Each service can then be discovered through its logical name, streamlining the interaction between different components of your application.

Practical Example: Integrating Service Discovery with ECS

Consider a scenario where you have a microservices architecture with a front-end service needing to communicate with a back-end service. Instead of hardcoding the back-end service's IP address, you register both services with Cloud Map under a common namespace, say myapp.local. The back-end service registers itself with the name backend.myapp.local. The front-end service, needing to send a request to the back-end, queries Cloud Map for backend.myapp.local and receives the current IP address and port of the back-end service. This mechanism ensures that even if the back-end service is redeployed or its IP changes, the front-end can always discover and communicate with it without any manual intervention.

Best Practices for Service Discovery with ECS

Automate Service Registration: Ensure your ECS services are automatically registered with Cloud Map upon deployment. This can be achieved through ECS task definitions or service configurations.
Health Checks: Utilize Cloud Map's health checking capabilities to automatically remove unhealthy service instances from the registry. This ensures your applications always connect to operational instances.
Security: Implement appropriate IAM policies to control access to the service discovery system, ensuring only authorized services can register or discover other services.

Conclusion

Service discovery is a cornerstone of modern distributed systems, ensuring applications remain resilient and adaptable to changing environments. By integrating AWS ECS with Cloud Map, developers can significantly simplify the discovery process, allowing services to dynamically interact regardless of their underlying infrastructure. This guide provides a foundation for leveraging service discovery within your ECS deployments, paving the way for more efficient and scalable applications.

Incorporating service discovery into your ECS strategy not only optimizes communication between services but also enhances overall application resilience. By following the practices outlined above, you can create a robust ecosystem where services seamlessly discover and interact with each other, driving efficiency and scalability across your deployments.

Thank you for reading this article! If you enjoyed it and would like to stay up to date on the latest technical articles and insights, I invite you to subscribe to my newsletter. By subscribing, you'll be the first to know when we publish new articles and you'll have access to exclusive content and resources.

NodeJS Graceful Shutdown: A Beginner's Guide

Yusuf Adeyemo — Tue, 23 May 2023 13:52:58 +0000

Imagine this scenario: Your Node.js application is happily running, processing requests, interacting with databases, and then suddenly, it gets terminated. The system administrator decided it was time to scale down, or perhaps a critical error forced the application to exit. In any case, the application was in the middle of processing requests, writing data to a file, and now all of that is abruptly stopped. What happens to that data? What happens to your users' requests? The consequences of an abrupt shutdown can range from minor inconveniences to significant data loss, and degraded user experience. To avoid these situations, it is important to shut down your applications gracefully. In this article, we'll discuss why graceful shutdowns are important, how to handle them in Node.js applications, particularly in the context of Docker, and the potential issues that could arise if not handled correctly.

Prerequisites

Before proceeding, you should have:

Basic knowledge of JavaScript and Node.js
Understanding of Express.js framework
Familiarity with Docker and its basic commands

What is a Graceful Shutdown?

A graceful shutdown involves carefully handling the shutdown signal, completing the in-progress tasks, closing the active connections, and then finally allowing the application to terminate. This ensures that the system resources are properly freed and that the application does not exit while it's in the middle of important tasks.

Why is a Graceful Shutdown Important?

Handling shutdown signals in your applications allows you to manage resources properly, provide a better user experience, and help your system degrade more gracefully. Not handling these signals can lead to issues like data loss or corruption, incomplete transactions, resource leaks, and unexpected behavior.

Implementing Graceful Shutdown in Node.js

In this section, we'll walk through the code required to listen for shutdown signals in a Node.js application and how to perform cleanup tasks before allowing the application to exit.

Listening for Shutdown Signals

In Node.js, we can listen to process-level signals, such as SIGINT and SIGTERM. These signals are emitted when the process is requested to shut down, whether by manual user interruption (SIGINT from Ctrl+C) or system-level termination (SIGTERM from Docker or another process manager). To listen for these signals, we can use the process.on method and provide a callback function that will be executed when these signals are received.



process.on('SIGTERM', () => {
  console.log('SIGTERM signal received.');
});

process.on('SIGINT', () => {
  console.log('SIGINT signal received.');
});

Performing Cleanup

Once a shutdown signal is received, it's important to perform necessary cleanup tasks to close any open resources, finish transactions, and prepare the application for a graceful exit. This may involve closing database connections, completing any in-progress writes to file systems, or other application-specific cleanup. This cleanup code should be placed inside the callback function provided to process.on.



process.on('SIGTERM', () => {
  console.log('SIGTERM signal received.');
  // Perform cleanup tasks here
});

process.on('SIGINT', () => {
  console.log('SIGINT signal received.');
  // Perform cleanup tasks here
});

Remember to handle asynchronous cleanup tasks correctly. If a cleanup task is asynchronous (like closing a database connection), you'll need to handle it with async/await or Promises to ensure it completes before the process exits.

Exiting the Process

After performing the necessary cleanup tasks, we must manually terminate the Node.js process by calling process.exit(). This signals to the system (or Docker) that our application has finished shutting down. We can provide an exit code to this method; a code of 0 indicates a successful exit, while any other number indicates an error occurred. Typically, if we've handled everything correctly in our cleanup, we'll want to exit with a code of 0.



process.on('SIGTERM', () => {
  console.log('SIGTERM signal received.');
  // Perform cleanup tasks here

  process.exit(0);
});

process.on('SIGINT', () => {
  console.log('SIGINT signal received.');
  // Perform cleanup tasks here

  process.exit(0);
});

Remember, the goal of all this is to allow your application to exit gracefully when it receives a shutdown signal. This helps reduce the risk of data corruption, loss of data, and other issues associated with an abrupt termination.

Handling Shutdown in Express.js Application

Express.js applications, in particular, have some unique considerations when it comes to graceful shutdowns. This section discusses how to handle shutdown signals in an Express.js application, including how to stop the server from accepting new connections and how to ensure all existing connections are closed before shutdown.

Creating and Starting the Server

First, we need to create an Express.js application and start a server. Once the server is created, we can use it to close existing connections when we're ready to shut down the application.



import express from 'express';

const app = express();
const server = app.listen(3000, () => {
  console.log('Server listening on port 3000');
});

Listening for Shutdown Signals

Just like in a basic Node.js application, we need to listen for SIGINT and SIGTERM signals. We can use the process.on method to add listeners for these signals. Inside the callback function for each listener, we'll call server.close() to stop the server from accepting new connections and to begin the process of shutting down.



process.on('SIGTERM', () => {
  console.log('SIGTERM signal received.');
  server.close(() => {
    console.log('Closed out remaining connections');
    // Additional cleanup tasks go here
  });
});

process.on('SIGINT', () => {
  console.log('SIGINT signal received.');
  server.close(() => {
    console.log('Closed out remaining connections');
    // Additional cleanup tasks go here
  });
});

Closing Existing Connections

When server.close() is called, the server stops accepting new connections and waits for all existing connections to close. The function that we pass to server.close() will be called once all connections are closed. This is where we can perform any additional cleanup tasks that need to happen before the application shuts down.

Performing Additional Cleanup

Depending on the needs of your application, you may have additional cleanup tasks that need to happen when your application shuts down. For example, if you have a database connection, you should close it before your application exits. This cleanup code should be placed inside the callback function that you pass to server.close().



process.on('SIGTERM', () => {
  console.log('SIGTERM signal received.');
  server.close(() => {
    console.log('Closed out remaining connections');
    // Additional cleanup tasks go here, e.g., close database connection
    process.exit(0);
  });
});

process.on('SIGINT', () => {
  console.log('SIGINT signal received.');
  server.close(() => {
    console.log('Closed out remaining connections');
    // Additional cleanup tasks go here, e.g., close database connection
    process.exit(0);
  });
});

Handling Shutdown in a Dockerized Node.js Application

When running a Node.js application in a Docker container, there are additional considerations to take into account. This section discusses how Docker sends shutdown signals and how to ensure your Node.js application can handle them correctly.

Understanding Docker Shutdown Signals

When Docker is asked to stop a running container, it sends a SIGTERM signal to the main process running in the container. This is Docker's way of asking the process to shut down gracefully, by finishing what it's currently doing, cleaning up as needed, and then terminating.

However, if the process does not terminate within a certain period (10 seconds by default), Docker will then send a SIGKILL signal to forcibly terminate the process. This is akin to pulling the plug on the application - it won't have a chance to finish what it's doing or clean up.

This is why our Node.js application needs to listen for and handle the SIGTERM signal, as we discussed in previous sections. By handling SIGTERM, our application can ensure it shuts down gracefully when Docker asks it to stop.

Adjusting Docker's Grace Period

Sometimes, our application may need more than 10 seconds to shut down gracefully. For example, it might need to finish processing a long-running request, or it might need to wait for a database transaction to commit.

In such cases, we can tell Docker to wait longer before it sends the SIGKILL signal, by using the --stop-timeout option when we run our Docker container. This option takes several seconds as its argument.

For example, to start a Docker container and give it 30 seconds to shut down gracefully before forcibly killing it, we would use a command like this:



docker run --stop-timeout 30 my-nodejs-app

Keep in mind that while extending the stop timeout can help in some situations, it's not a panacea. If your application consistently takes a long time to shut down, it may be a sign that it needs to be optimized or refactored.

Conclusion

Handling shutdown signals in your Node.js applications allows you to manage resources properly, reduce potential data loss or corruption, provide a better user experience, and more. By understanding how to handle these signals, you can make your applications more robust and reliable, both in development and in production.

Do Not Tolerate Flaky Tests. Fix Them (or Delete Them).

Yusuf Adeyemo — Mon, 17 Apr 2023 09:04:54 +0000

As a DevOps engineer, you know the importance of testing in ensuring the quality and reliability of your software. However, not all tests are created equal, and some tests are more reliable than others. Flaky tests are tests that fail intermittently, even though the code being tested is not broken. These tests can be frustrating to deal with and can waste valuable time and resources. In this article, we will discuss why you should not tolerate flaky tests and what you can do to fix or delete them.

Why You Should Not Tolerate Flaky Tests

Flaky tests can cause several problems that affect the quality and reliability of your software. Here are some reasons why you should not tolerate flaky tests:

Flaky tests can make it difficult to identify real bugs: Flaky tests can mask real bugs in your code, making it difficult to identify and fix them.
Flaky tests can waste valuable time and resources: Flaky tests can consume valuable time and resources that could be better spent on other tasks.
Flaky tests can erode confidence in your tests: Flaky tests can erode confidence in your tests and make it difficult to trust the results.
Flaky tests can lead to false positives: Flaky tests can lead to false positives, which can cause unnecessary rework and delays.

What You Can Do to Fix or Delete Flaky Tests

Fixing or deleting flaky tests can help you avoid the problems caused by flaky tests. Here are some things you can do to fix or delete flaky tests:

Identify the root cause of the flakiness: To fix flaky tests, you need to identify the root cause of the flakiness. This can be done by analyzing the test results and identifying patterns.
Fix the root cause of the flakiness: Once you have identified the root cause of the flakiness, you can fix it. This may involve modifying the test code, the test environment, or the application code.
Delete the flaky tests: If you are unable to fix the root cause of the flakiness, you may need to delete the flaky tests. This can help you avoid the problems caused by flaky tests.
Prioritize fixing flaky tests: Fixing flaky tests should be a priority. You should allocate the necessary time and resources to fix or delete flaky tests.

Conclusion

Flaky tests can cause several problems that affect the quality and reliability of your software. To avoid these problems, you should not tolerate flaky tests and should fix or delete them as soon as possible. By identifying the root cause of the flakiness and fixing it or deleting the flaky tests, you can improve the reliability and quality of your software. Remember, a reliable and trustworthy test suite is crucial for the success of your DevOps pipeline.

I hope this article has been helpful in understanding why you should not tolerate flaky tests and what you can do to fix or delete them. If you have any questions or comments, feel free to leave them below.

How to resolve AWS S3 CORS error

Yusuf Adeyemo — Fri, 31 Mar 2023 07:40:33 +0000

The error message you're seeing is due to the Cross-Origin Resource Sharing (CORS) policy on your AWS S3 bucket. This policy determines who can access your bucket's contents from a different domain. In my case, it seems the policy is not allowing the local server (http://localhost:3001) to access the resources.

To resolve this issue, you need to update the CORS policy for your S3 bucket. Here's a step-by-step guide:

Sign in to the AWS Management Console and open the Amazon S3 console at https://console.aws.amazon.com/s3/.
In the bucket list, choose the name of the bucket that you want to add a CORS policy to.
Choose the 'Permissions' tab.
Scroll down to the 'Cross-origin resource sharing (CORS)' section and choose 'Edit'.
In the CORS configuration editor, add a new CORS rule. For example:

[
    {
        "AllowedHeaders": ["*"],
        "AllowedMethods": ["GET", "PUT", "POST", "DELETE"],
        "AllowedOrigins": ["http://localhost:3001"],
        "ExposeHeaders": []
    }
]

Choose 'Save'.

This policy allows your local server (http://localhost:3001) to perform GET, PUT, POST, and DELETE operations on your S3 bucket. Please adjust the policy according to your needs.

Remember, CORS policies can pose a security risk if not configured properly. Only allow access to trusted domains and use the strictest settings that your application allows.