DEV Community: Esther Nnolum

Cut Costs by Automating EC2 Start/Stop with AWS Lambda

Esther Nnolum — Sun, 10 Aug 2025 13:11:29 +0000

Every hour an unused EC2 instance runs is money slipping through the cracks, but with a little smart automation, you can turn that waste into predictable savings. Managing cloud costs is important, especially when dev or test servers run outside of business hours. A great way to save is to automatically stop EC2 instances after work hours, and start them again in the morning.

In this article, you’ll learn how to do this using:

AWS Lambda
EC2 tags to control which instances are affected
EventBridge Scheduler
IAM roles

Prerequisites

Before you begin, make sure you have:

An AWS account
Basic understanding of EC2 and AWS Console
EC2 instances you want to automate

Step 1: Tag Your EC2 Instances

This step is important if you don’t want to stop every instance, but just the ones you choose.
Go to your EC2 instances (Select the instance, open the Tags tab, and click Manage Tags) and add a tag:

Key: AutoShutdown
Value: Yes

This is to tell our automation to only act on these tagged instances.

Step 2: Create Lambda Function to Stop EC2 Instances

Go to AWS Console → Lambda → Create Function

Name: StopEC2Instances
Runtime: Python 3.x
Permissions: Create a new role with basic Lambda permissions

Click on Create function and Replace the default code with:

import boto3

def lambda_handler(event, context):
    ec2 = boto3.client('ec2')

    # Get all running EC2 instances with AutoShutdown = Yes
    response = ec2.describe_instances(
        Filters=[
            {'Name': 'tag:AutoShutdown', 'Values': ['Yes']},
            {'Name': 'instance-state-name', 'Values': ['running']}
        ]
    )

    instances_to_stop = []
    for reservation in response['Reservations']:
        for instance in reservation['Instances']:
            instances_to_stop.append(instance['InstanceId'])

    if instances_to_stop:
        print(f"Stopping instances: {instances_to_stop}")
        ec2.stop_instances(InstanceIds=instances_to_stop)
    else:
        print("No instances to stop.")

Step 3: Update IAM Permissions

Go to IAM → Roles → find your Lambda role (StopEC2Instances-role-xxxx) → attach this policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Action": [
                "ec2:DescribeInstances",
                "ec2:StopInstances",
                "ec2:StartInstances"
            ],
            "Effect": "Allow",
            "Resource": "*"
        }
    ]
}

Step 4: Schedule the Stop Function with EventBridge Scheduler

Go to Amazon EventBridge → Create Schedule

Schedule name: StopEC2AtNight
Occurrence: Recurring schedule
Time zone: "select-your-timezone"
Schedule type: Cron-based schedule
Cron expression: cron(0 19 ? * MON-FRI *)

This means 7:00 PM of the specified timezone, Monday to Friday
N.B: Replace "select-your-timezone" with your preferred timezone

Step 5: Create Lambda Function to Start EC2 Instances

Now let’s create a second function to start the instances in the morning.
Create another Lambda function

Name: StartEC2Instances
Runtime: Python 3.x
Use the same IAM role

Click on Create function and Replace the default code with:

import boto3

def lambda_handler(event, context):
    ec2 = boto3.client('ec2')

    response = ec2.describe_instances(
        Filters=[
            {'Name': 'tag:AutoShutdown', 'Values': ['Yes']},
            {'Name': 'instance-state-name', 'Values': ['stopped']}
        ]
    )

    instances_to_start = []
    for reservation in response['Reservations']:
        for instance in reservation['Instances']:
            instances_to_start.append(instance['InstanceId'])

    if instances_to_start:
        print(f"Starting instances: {instances_to_start}")
        ec2.start_instances(InstanceIds=instances_to_start)
    else:
        print("No instances to start.")

Step 6: Schedule the Start Function

Go to Amazon EventBridge → Create Schedule

Schedule name: StartEC2InTheMorning
Occurrence: Recurring schedule
Time zone: "select-your-timezone"
Schedule type: Cron-based schedule
Cron expression: cron(0 7 ? * MON-FRI *) This means 7:00 AM of the specified timezone, Monday to Friday

Conclusion

This simple automation is proof that small changes can deliver big results. By letting AWS Lambda and EventBridge Scheduler handle your EC2 stop/start, you’re not just cutting costs, you’re building a culture of efficiency and smart resource usage.
The result: predictable savings, reduced human error, and an AWS environment that runs only when it needs to.

AWS CI/CD Services: CodeBuild, CodeDeploy, CodePipeline and What Replaces CodeCommit

Esther Nnolum — Sun, 29 Jun 2025 13:43:07 +0000

As DevOps teams continue to modernize their workflows, AWS provides a set of tools to automate software delivery through Continuous Integration and Continuous Deployment (CI/CD). These services support the reliable development, testing, and deployment of applications while scaling with your infrastructure.

The main AWS-native CI/CD tools will be described in this article:

AWS CodeBuild: Build automation
AWS CodeDeploy: Deployment automation
AWS CodePipeline: Workflow orchestration
[Deprecated] AWS CodeCommit: Git hosting (with migration guidance)

We’ll explore the key AWS services that power modern CI/CD pipelines, including how they work and how they’re billed.

1.0 CodeBuild: Build and Test in the Cloud

With AWS CodeBuild, you can build and test code with automatic scaling. It compiles your source code, runs tests, and generates artifacts (e.g., Docker images or JAR files).

Use Cases:

Remove the complexity of managing build servers
Build source code hosted on other git provides (e.g., GitHub or GitLab)

Pricing

AWS CodeBuild follows a pay-as-you-go pricing model with no upfront commitments. You only pay for the compute resources used during your build process, and charges are based on how long the build runs. The cost varies depending on the compute type you choose. Refer to the AWS CodeBuild pricing for more pricing detail .

2.0 CodeDeploy: Safe, Automated Deployments

AWS CodeDeploy automates application deployments to different targets like EC2, ECS, or Lambda. It supports blue/green, canary, and rolling deployments.

Use Cases:

Support various deployment strategies such as in-place, canary, and blue/green
Eliminate manual steps by fully automating deployments
Integrate alarms to trigger automatic rollbacks and halt deployments when issues are detected
Deploy applications across multiple hosts seamlessly

Pricing

This is dependent on the deployment target (eg EC2, ECS, or Lambda)
Free for EC2 and Lambda
You pay $0.02 per on-premises instance update using CodeDeploy. View CodeDeploy Pricing for more pricing detail.

3.0 CodePipeline: End-to-End CI/CD Automation

AWS CodePipeline orchestrates the full CI/CD flow — integrating your source, build, test, and deployment stages. AWS CodePipeline supports two types of pipelines: V1 and V2, which differ in features and billing.

V1 pipelines are created by default unless you explicitly specify type V2.
V2 pipelines offer additional capabilities, including support for triggers and variables.

Use Cases:

Trigger builds on PR merges from connected git providers like github.
Use declarative JSON templates to create and update pipelines,
Manage who can change and control your release workflow with IAM roles.
monitor pipeline activity via Amazon SNS notifications with event details and source links.

Pricing
V1 Pipelines:

$1.00/month per active pipeline
Free for the first 30 days
No charge if no code runs through the pipeline that month

V2 Pipelines:

$0.002 per minute of action execution time (rounded up)
Manual and custom actions are free View CodePipeline Pricing for more pricing detail.

4.0 CodeCommit: Deprecated Git Hosting

As of July 2024, AWS CodeCommit is no longer available to new customers. Existing users can continue using the service, which will still receive security, availability, and performance updates — but no new features are planned.

Migrating From CodeCommit:

AWS has provided an official migration guide to help move your repositories to other git providers (GitHub, GitLab or Bitbucket)

Note: You can still integrate GitHub or GitLab with CodePipeline using Webhooks or GitHub connections.

Conclusion

While AWS has deprecated CodeCommit for new customers, its other CI/CD services CodeBuild, CodeDeploy, and CodePipeline remain powerful, flexible, and cost-effective. By combining them with modern Git providers like GitHub, you can build fully automated pipelines that are secure, scalable, and AWS-native where it matters most.

Tip: Pricing can evolve over time. Always refer to the official AWS pricing documentation linked throughout this article as the source of truth for the most accurate and up-to-date cost information.

Deploying Dockerized Microservices to AWS ECS Fargate with CodePipeline

Esther Nnolum — Sun, 25 May 2025 16:20:05 +0000

Introduction

In modern DevOps workflows, teams often break applications into smaller parts called microservices and run them in containers. Deploying these containers in a way that’s easy to manage, cost-effective, and can grow with demand is really important. That's where Amazon ECS Fargate and AWS CodePipeline come in, they let you run and update these services automatically without managing servers. AWS Fargate is a serverless compute engine for containers, and when combined with CodePipeline, you get fully automated CI/CD. In this article, I’ll explain how it all works and show you step-by-step how to deploy your container-based services using these tools.

Prerequisites

Before we begin, ensure you have:

Basic knowledge of Docker and AWS
AWS CLI configured
A GitHub repository with a simple Dockerized microservices application.
An existing VPC and subnets (or allow Fargate to create one)

Architecture Overview

GitHub → CodePipeline → CodeBuild → ECR → ECS Fargate

GitHub: hosts the source code and Dockerfiles.
CodePipeline: detects changes and triggers deployment.
Build: CodeBuild (build Docker image and push to ECR)
ECS Fargate: deploys containers using the latest image.

Implementation Steps

STEP 1: Dockerize your app - Your microservice should include a Dockerfile like this.

FROM node:18-alpine
WORKDIR /app
COPY . .
RUN npm install
CMD ["node", "index.js"]

STEP 2: Push Your Code to GitHub - Push your microservice to a GitHub repo if you haven’t done so at this stage. This will be the source for your CodePipeline.

STEP 3: Create an Amazon ECR Repository - Before deploying your Dockerized microservice to ECS Fargate, you need a place to store your container images. Amazon Elastic Container Registry (ECR) is AWS's fully managed Docker container registry, and it's where you'll push your built images for ECS to pull from. You can do this manually using the AWS Management Console, Using the AWS CLI or via IaC (Terraform) during infrastructure provisioning.

STEP 4: Define Buildspec for CodeBuild - The buildspec.yml file is a configuration file that tells AWS CodeBuild exactly what to do when it runs. It defines:

Commands to run during each phase of the build process
How to build your Docker image
Where to push it (ECR, in our case)
What (if anything) to output as artifacts AWS CodeBuild reads this file at runtime to carry out your instructions step by step.

version: 0.2

phases:
  pre_build:
    commands:
      - echo Logging in to Amazon ECR...
      - aws ecr get-login-password | docker login --username AWS --password-stdin <your-ecr-uri>
  build:
    commands:
      - echo Building Docker image...
      - docker build -t my-service .
      - docker tag my-service:latest <your-ecr-uri>:latest
  post_build:
    commands:
      - echo Pushing image to ECR...
      - docker push <your-ecr-uri>:latest

artifacts:
  files: []

Replace with your actual ECR repo URI.

STEP 5: Create an ECS Fargate Cluster - To run your containerized microservices on AWS, you need an ECS (Elastic Container Service) cluster. When using Fargate, AWS handles the underlying server infrastructure, allowing you to focus only on your container and application code.

Go to ECS → Task Definitions → Create
Use Fargate launch type
Add container with image: :latest
Set port mappings (e.g., 3000)
Set memory/CPU (e.g., 512MB/256 CPU)

STEP 6: Create CodePipeline - AWS CodePipeline is a fully managed CI/CD (Continuous Integration and Delivery) service. It automates building, testing, and deploying code every time you make a change.

Go to CodePipeline → Create Pipeline
Source: GitHub, Choose Connect to GitHub, and follow the instructions to authenticate with GitHub.
Build: Add build stage page, for Build provider, choose CodeBuild. (create a project using the buildspec.yml in step 4)
Deploy: ECS → Fargate → Select your cluster and service created in steps 5

CONCLUSION

ECS Fargate + CodePipeline offers a modern, scalable foundation for deploying containerized microservices. Whether you're starting from scratch or modernizing legacy apps, this setup provides a future-proof, low-maintenance path to continuous delivery in the cloud. This approach is particularly well-suited for microservice architectures, where deploying and updating individual services frequently and independently is a key requirement. With the setup described in this article, each microservice can be connected to its own pipeline, reducing deployment risk and making rollback strategies easier to implement.

A Beginner’s Guide to Terraform on AWS

Esther Nnolum — Sun, 06 Apr 2025 14:19:35 +0000

Introduction

Terraform, developed by HashiCorp, is an Infrastructure as Code (IaC) tool that allows you to define and manage your infrastructure using easy-to-read, declarative configuration files. It automates the entire lifecycle of your infrastructure—from provisioning to updates and teardown.

Why Use Terraform for AWS?

Repeatable and consistent environments
Trackable via Version control for infrastructure
Automated infrastructure setup

Prerequisites

Terraform Installed: You need to install Terraform before you can start using it. Refer to the official Terraform documentation for detailed installation steps, whether you're using a package manager or installing manually based on your operating system.
The AWS CLI installed: Check the official AWS CLI installation guide for step-by-step instructions tailored to your operating system.

Getting Started

A. Create AWS account and associated credentials

If you don't already have one, create an AWS account, this serves as your entry point to Amazon Web Services, where you can manage cloud resources such as servers, storage, databases e.t.c for your applications. The associated credentials enable you to create and manage resources within your account.

B. Steps to Create AWS Credentials for Terraform

Login to AWS Console:
- Access the AWS Management Console and search for the IAM service.

Create a New User:
- Click on Users in the IAM dashboard.
- Click the Create user button and enter the required user details.

Configure User Access:
- Uncheck the option "Provide user access to the AWS Management Console" (this is not needed because Terraform is managed via code, not the UI).
Set Permissions:
- Click Next: To set user permissions.
- For testing, assign the "AdministratorAccess" policy to the user.

Review and Create User:
- Click Next: To review your configuration.
- Click the Create user button to create the Terraform user.
Access Keys Configuration:
- Once the user is created, click on the username.
- Scroll down to the Access keys section.
- Click Create access key.

Select Use Case:
- In the pop-up, choose Command Line Interface (CLI) as the use case.
- Check the confirmation box and click Next.
Set Description and Create Key:
- Optionally, set a description tag for the access key.
- Click Create access key to generate the credentials.
Store Access Keys Securely:
- Copy the Access Key and Secret Key immediately.
- Store them securely — this is the only time you can view or download the secret access key. If you lose it, you will need to create a new one.

C. Configure AWS CLI

Open a terminal and run the command aws configure. When prompted, enter the IAM user's Access Key ID and Secret Access Key that were generated in the previous step. Additionally, specify your default region (e.g., us-east-1) and output format (e.g., json).

Note: Always protect your access key and secret key, and periodically rotate them to minimize the risk of compromise.

D. Create a basic Terraform script

Now that our AWS account is set up and the CLI is configured, we can start working with Terraform to define our infrastructure. The files used to describe infrastructure in Terraform are called "Terraform configurations."

To get started, create a dedicated directory for your project, as each configuration must reside in its own working directory. This directory will hold all the necessary configuration files.

mkdir terraform-aws

Change into the directory

cd terraform-aws

Create a main.tf file and paste in the configuration below.

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
  required_version = ">= 1.4.0"
}

# Configure the AWS Provider
provider "aws" {
  region = "us-east-1"
}
resource "aws_instance" "app_server" {
  ami           = "ami-05238ab1443fdf48f" #Substitute this with the AMI ID you wish to use.
  instance_type = "t2.micro"
  tags = {
    Name = "MyServerInstance"
  }
}

This Terraform code sets up an AWS EC2 instance in the us-east-1 region, using the specified AMI (ami-05238ab1443fdf48f) and a t2.micro instance type, with a tag called MyServerInstance. It also specifies that Terraform version 1.4.0 or higher and AWS provider version 5.x.x are required.

E. Initialize, Plan, and Apply:

When you create or check out a configuration, run the following commands;

terraform init: Initializes the working directory and downloads the required providers (e.g., AWS).

$ terraform init
_Initializing the backend...
Initializing provider plugins...
- Finding hashicorp/aws versions matching "~> 5.0"...
- Installing hashicorp/aws v5.94.1...
- Installed hashicorp/aws v5.94.1 (signed by HashiCorp)
Terraform has created a lock file .terraform.lock.hcl to record the provider
selections it made above. Include this file in your version control repository
so that Terraform can guarantee to make the same selections by default when
you run "terraform init" in the future.

Terraform has been successfully initialized!

You may now begin working with Terraform. Try running "terraform plan" to see
any changes that are required for your infrastructure. All Terraform commands
should now work.

If you ever set or change modules or backend configuration for Terraform,
rerun this command to reinitialize your working directory. If you forget, other
commands will detect it and remind you to do so if necessary._

terraform fmt: Automatically formats configurations for improved readability.
terraform validate: Checks that the configuration is syntactically correct and internally consistent.

_$ terraform validate
Success! The configuration is valid._

terraform plan: Generates an execution plan, describing the actions Terraform will take to modify the infrastructure. It shows you what will happen but doesn't make any changes. I've shortened some of the output for brevity.

$ terraform plan
Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  + create

Terraform will perform the following actions:

  # aws_instance.app_server will be created
  + resource "aws_instance" "app_server" {
      + ami                                  = "ami-05238ab1443fdf48f"
      + arn                                  = (known after apply)
      + associate_public_ip_address          = (known after apply)
##...

Plan: 1 to add, 0 to change, 0 to destroy.

terraform apply: Applies the changes to your infrastructure based on the plan. It will ask for confirmation (e.g., typing "yes") before proceeding. I've shortened some of the output for brevity.

_$ terraform apply
##...

Plan: 1 to add, 0 to change, 0 to destroy.

Do you want to perform these actions?
  Terraform will perform the actions described above.
  Only 'yes' will be accepted to approve.

  Enter a value:_

Post-Execution: Once terraform apply completes, check the EC2 console for the newly created instance.
terraform destroy: Destroys all the infrastructure managed by the Terraform configuration. It will prompt for confirmation before deleting the resources, ensuring you can safely tear down the setup when no longer needed.

Conclusion

Using Terraform for Infrastructure as Code (IaC) on AWS simplifies and automates the provisioning and management of cloud resources. By defining your infrastructure in configuration files, you ensure consistency, repeatability, and version control. With Terraform's tools—like terraform init, terraform plan, terraform apply, and terraform destroy— you can easily create, modify, and delete resources, making it an essential tool for modern DevOps practices. Whether you’re just starting or scaling your infrastructure, Terraform helps streamline the process and enhances your ability to manage cloud environments efficiently.

Automating ECR Token Renewal for Private Registry Pulls in Kubernetes Using a CronJob.

Esther Nnolum — Mon, 09 Sep 2024 20:14:17 +0000

In Kubernetes, the authentication token for Amazon ECR (Elastic Container Registry) expires every 12 hours. This is due to ECR's use of the 'GetAuthorizationToken' API, which generates a base64-encoded token with a default validity period of 12 hours.

Why the Token Expires in 12 Hours:

Security: Short-lived tokens reduce the window of opportunity for unauthorized access if a token is compromised. Regularly rotating the token enhances security by ensuring that stale or leaked tokens are invalidated in a short time frame.
AWS Design: AWS designed the ECR token system to issue tokens with a 12-hour expiration as part of its security best practices. This time limit balances the need for frequent reauthentication with minimizing user disruption.

Handling Expiration in Kubernetes:

To maintain continuous access to ECR from within a Kubernetes cluster, it is common to automate the refresh process using a Kubernetes CronJob. This job would periodically refresh the authentication token and update the necessary secrets to ensure uninterrupted image pulls from ECR. This article walks through automating the process using a kubernetes CronJob to refresh ECR credentials.

Step 1: Create a Base Docker Image

You will need a Dockerfile that includes essential tools such as aws-cli, curl, wget, docker, and kubectl. This image will serve as the base for the CronJob to refresh the ECR secret.
Dockerfile

# create Dockerfile with this content
FROM alpine:latest
RUN apk --no-cache add aws-cli wget curl docker docker-compose \
    && wget https://storage.googleapis.com/kubernetes-release/release/v1.29.1/bin/linux/amd64/kubectl \
    && mv kubectl /usr/local/bin/kubectl \
    && chmod +x /usr/local/bin/kubectl \
    && apk del wget

After creating the Dockerfile:

Build the image.
Push it to a public container repository (e.g., Docker Hub or Amazon ECR).

Step 2: Create an AWS Credentials Secret

In this step, create a Kubernetes secret that contains AWS credentials with permission to access ECR. Ensure that the AWS credentials have the necessary ECR permissions.

kubectl create secret generic aws-credentials \
  --from-literal=aws_access_key_id=<AWS_ACCESS_KEY_ID> \
  --from-literal=aws_secret_access_key=<AWS_SECRET_ACCESS_KEY> \
  --namespace dev #replace with your namespace

Step 3: Create a Service Account

A dedicated Service Account will be used by CronJob to perform its tasks. This Service Account ensures proper security scoping for the refresh job.

cronjob-service-account.yaml

apiVersion: v1
kind: ServiceAccount
metadata:
  name: cronjob-sa
  namespace: dev #replace with your namespace

Apply file
kubectl apply -f cronjob-service-account.yaml

Step 4: Create a Role with Secret Permissions

Create a Role that allows the Service Account to manage secrets, as it will need to update the ECR secret with new credentials.

cronjob-role.yaml:

apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: cronjob-role
  namespace: dev
rules:
  - apiGroups: [""]
    resources: ["secrets"]
    verbs: ["get", "list", "create", "update", "delete"]

Apply the Role:
kubectl apply -f cronjob-role.yaml

Step 5: Bind the Role to the Service Account

Bind the Role to the Service Account using a RoleBinding to ensure the Service Account has the required permissions.

cronjob-rolebinding.yaml:

apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: cronjob-rolebinding
  namespace: dev
subjects:
- kind: ServiceAccount
  name: cronjob-sa
  namespace: dev
roleRef:
  kind: Role
  name: cronjob-role
  apiGroup: rbac.authorization.k8s.io

Apply the RoleBinding:
kubectl apply -f cronjob-rolebinding.yaml

Step 6: Create the Kubernetes CronJob

Now, set up a CronJob that will periodically refresh the ECR Docker registry credentials. In this example, the job runs every 11 hours.

The CronJob performs the following actions:

Retrieve the ECR token: Using the AWS CLI command aws ecr get-login-password, the CronJob retrieves the authentication token for the ECR registry.
Create/Update a Kubernetes secret: The token is then stored in a Kubernetes secret. This secret is used by Kubernetes to authenticate with the ECR registry whenever it needs to pull Docker images.

cronjob.yaml:

apiVersion: batch/v1
kind: CronJob
metadata:
  name: ecr-creds-refresh
  namespace: dev
spec:
  schedule: 0 */11 * * *
  jobTemplate:
    spec:
      template:
        spec:
          containers:
            - name: ecr-creds-refresh
              image: awsecr-kubectl:latest #Input the image from step 1
              command:
                - /bin/sh
                - '-c'
                - >-
                  aws --version 

                  aws ecr get-login-password --region <region>

                  echo "deleting imagepull secret..."

                  kubectl delete secret -n dev --ignore-not-found ${SECRET_NAME}

                  echo "recreating imagepull secret..."

                  kubectl create secret -n dev docker-registry ${SECRET_NAME} \

                  --docker-server=***.dkr.ecr.us-east-1.amazonaws.com \

                  --docker-username=AWS \

                  --docker-password="$(aws ecr get-login-password --region
                  us-east-1)" 
                  echo "secret recreated!!"
              env:
                - name: AWS_REGION
                  value: us-east-1
                - name: SECRET_NAME
                  value: aws-ecr # secret name
                - name: AWS_ACCESS_KEY_ID
                  valueFrom:
                    secretKeyRef:
                      name: aws-credentials
                      key: aws_access_key_id
                - name: AWS_SECRET_ACCESS_KEY
                  valueFrom:
                    secretKeyRef:
                      name: aws-credentials
                      key: aws_secret_access_key
              imagePullPolicy: IfNotPresent
          restartPolicy: OnFailure
          serviceAccountName: cronjob-sa
          serviceAccount: cronjob-sa

Deploy the CronJob:
kubectl apply -f cronjob.yaml

Purpose of the Secret:
Kubernetes uses this secret when pulling Docker images from ECR. Without this step, your cluster could lose access to ECR when the token expires, leading to failed deployments or pods unable to start due to image pull errors.

Conclusion

By automating the update of ECR Docker registry credentials with a Kubernetes CronJob, you can eliminate manual intervention and ensure your cluster always has valid credentials for pulling Docker images from ECR. This approach leverages Kubernetes-native tools like CronJob, RBAC, and Secrets to securely and efficiently manage credentials. The steps outlined, from creating a base Docker image to setting up the necessary roles and service accounts, provide a robust solution for maintaining a seamless container lifecycle in your Kubernetes environment. This automation not only improves security by regularly rotating credentials but also enhances operational efficiency, freeing your team to focus on more critical tasks.

Refrence
Amazon ECR AuthorizationData

Provisioning Kubernetes Clusters with Kubespray

Esther Nnolum — Sun, 12 May 2024 12:50:59 +0000

Kubernetes, an open-source orchestration system, automates the deployment and management of containerized applications. For beginners, the journey into Kubernetes can often start with the daunting question: "Where do I begin?"

In the early days, setting up and managing a Kubernetes cluster was a challenging and time-consuming task. However, with the evolution of Kubernetes, user-friendly solutions have emerged to simplify this process. Among these solutions, Kubespray shines as an invaluable tool.

Kubespray, an open-source solution, facilitates the automated deployment of Kubernetes clusters across nodes. Engineered to be highly customizable, efficient, and lightweight, Kubespray caters to a wide range of requirements, making Kubernetes cluster deployment accessible to all.

Overview of Kubespray
Kubespray is a composition of Ansible playbooks, inventory, provisioning tools, and generic Kubernetes cluster configuration management tasks. In this writeup, I'll demonstrate how to deploy a Kubernetes cluster on 3 nodes (1master and 2 worker nodes) using Kubespray.
While a basic understanding of Ansible and Kubernetes terminologies is assumed, the steps are simple enough for beginners to follow along.

Prerequisites
Before proceeding, ensure the following prerequisites are in place:

Provision Infrastructure: Set up computing resources, such as 3 servers, for the cluster.
Install Dependencies: Install the following dependencies on your Ansible server (can be one of your nodes or a dedicated server {recommended} for ansible):
Git
Python3
Pip3
Ansible

Setting Up the Cluster
Follow these steps to set up your Kubernetes cluster with Kubespray:
Step 1: Set Up SSH Keys
Generate SSH keys on the Ansible node and copy the key to all your cluster nodes:

ssh-keygen # Go with the defaults
ssh-copy-id <user>@<node-IP>

Step 2: Download and Configure Kubespray
Download the Kubespray GitHub repository and checkout the latest version:

git clone git@github.com:Kubernetes-sigs/Kubespray.git
cd Kubespray
git checkout release-2.xx #replace 'xx' with release number

Step 3: Install Python Dependencies
Install the required Python dependencies using pip:

pip3 install -r ./requirements.txt

Step 4: Update Ansible Inventory
Update the Ansible inventory file with the IP addresses of your nodes:

cp -rfp inventory/sample inventory/mycluster
declare -a IPS=(<node1-IP> <node2-IP> <node3-IP>)
CONFIG_FILE=inventory/mycluster/hosts.yaml python3 contrib/inventory_builder/inventory.py ${IPS[@]}

Further customize inventory/mycluster/hosts.yaml to specify your master, worker, and etcd nodes.

Step 5: Review and Customize Configuration
Review and customize parameters under inventory/mycluster/group_vars:

cat inventory/mycluster/group_vars/all/all.yml
cat inventory/mycluster/group_vars/k8s_cluster/k8s-cluster.yml

Step 6: Allow Kubernetes Ports
If behind a firewall, ensure all necessary Kubernetes ports are allowed. These ports include; api-server, etcd, DNS, kubelet, kube-controller-manager, calico ports, kube-scheduler e.t.c.

Step 7: Clean Up Old Kubernetes Cluster
Run the playbook to clean up any old Kubernetes cluster:

ansible-playbook -i inventory/mycluster/hosts.yaml --user=<your-user-with-sudo-access> --ask-become-pass --become reset.yml

Step 8: Deploy Kubernetes with Kubespray
Run the playbook to deploy Kubernetes:

ansible-playbook -i inventory/my-cluster/hosts.yml --user=<your-user-with-sudo-access> --ask-become-pass --become cluster.yml

Step 9: Access the Cluster
To access the cluster using kubectl commands, update your kubeconfig file to be in the ~/.kube directory:

mkdir .kube
cd .kube/
sudo cp /etc/kubernetes/admin.conf config
kubectl get nodes

Note: The playbook will take some time to complete, but once finished, you'll have a highly available and self-managed Kubernetes cluster at your disposal.
Kubespray enables you to manage Kubernetes clusters, you can add/remove nodes (playbooks/scale.yml|remove_node.yml), delete the entire cluster (playbooks/reset.yml), and performing other Kubernetes management tasks.

Troubleshooting

Issue with Python Packages Installation: When Ansible is already installed via system packages on the control node, Python packages installed using sudo pip install -r requirements.txt may end up in a different directory tree (e.g., /usr/local/lib/python2.7/dist-packages on Ubuntu) compared to Ansible's directory (e.g., /usr/lib/python2.7/dist-packages/ansible on Ubuntu). Consequently, the ansible-playbook command may fail with the following error:

ERROR! no action detected in task. This often indicates a misspelled module name, or incorrect module path.

This likely indicates that a task depends on a module present in requirements.txt.
2.Ensure Firewall Rules Allow Necessary Ports: Make sure that all necessary ports are allowed through the firewall to ensure proper communication between components.
3.Failure to Run Playbook without --become: The playbook will fail to run if the --become flag is not used. Ensure that you include --become to grant necessary privileges for the playbook to execute successfully.
4.For further troubleshooting on any encountered issue, please refer to the official Kubespray repository for comprehensive troubleshooting steps.

Provisioning Kubernetes Clusters with Kubespray

Esther Nnolum — Sun, 12 May 2024 12:50:56 +0000

Prerequisites
Before proceeding, ensure the following prerequisites are in place:

Provision Infrastructure: Set up computing resources, such as 3 nodes, for your cluster.
Install Dependencies: Install the following dependencies on your Ansible server:
Git
Python3
Pip3
Ansible

ssh-keygen # Go with the defaults
ssh-copy-id <user>@<node-IP>

Step 2: Download and Configure Kubespray
Download the Kubespray GitHub repository and checkout the latest version:

git clone git@github.com:Kubernetes-sigs/Kubespray.git
cd Kubespray
git checkout release-2.xx #replace 'xx' with release number

Step 3: Install Python Dependencies
Install the required Python dependencies using pip:

pip3 install -r ./requirements.txt

Step 4: Update Ansible Inventory
Update the Ansible inventory file with the IP addresses of your nodes:

cp -rfp inventory/sample inventory/mycluster
declare -a IPS=(<node1-IP> <node2-IP> <node3-IP>)
CONFIG_FILE=inventory/mycluster/hosts.yaml python3 contrib/inventory_builder/inventory.py ${IPS[@]}

Further customize inventory/mycluster/hosts.yaml to specify your master, worker, and etcd nodes.

Step 5: Review and Customize Configuration
Review and customize parameters under inventory/mycluster/group_vars for further customization:

cat inventory/mycluster/group_vars/all/all.yml
cat inventory/mycluster/group_vars/k8s_cluster/k8s-cluster.yml

Step 6: Allow Kubernetes Ports
If behind a firewall, ensure all necessary Kubernetes ports are allowed.

Step 7: Clean Up Old Kubernetes Cluster
Run the playbook to clean up the old Kubernetes cluster:

ansible-playbook -i inventory/mycluster/hosts.yaml --user=<your-user-with-sudo-access> --ask-become-pass --become reset.yml

Step 8: Deploy Kubernetes with Kubespray
Run the playbook to deploy Kubespray:

ansible-playbook -i inventory/my-cluster/hosts.yml --user=<your-user-with-sudo-access> --ask-become-pass --become cluster.yml

Step 9: Access the Cluster
Access the cluster using kubectl commands:

mkdir .kube
cd .kube/
sudo cp /etc/kubernetes/admin.conf config
kubectl get nodes

Note: The playbook will take some time to complete, but once finished, you'll have a highly available and self-managed Kubernetes cluster at your disposal.

Troubleshooting

Issue with Python Packages Installation: When Ansible is already installed via system packages on the control node, Python packages installed using sudo pip install -r requirements.txt may end up in a different directory tree (e.g., /usr/local/lib/python2.7/dist-packages on Ubuntu) compared to Ansible's directory (e.g., /usr/lib/python2.7/dist-packages/ansible on Ubuntu). Consequently, the ansible-playbook command may fail with the following error:

ERROR! no action detected in task. This often indicates a misspelled module name, or incorrect module path.

This likely indicates that a task depends on a module present in requirements.txt.

Ensure Firewall Rules Allow Necessary Ports: Make sure that all necessary ports are allowed through the firewall to ensure proper communication between components.
Failure to Run Playbook without --become: The playbook will fail to run if the --become flag is not used. Ensure that you include --become to grant necessary privileges for the playbook to execute successfully.
For further troubleshooting on any encountered issue, please refer to the official Kubespray repository for comprehensive troubleshooting steps.

Trigger Jenkins builds with Github Webhook Using Smee Client

Esther Nnolum — Sun, 12 May 2024 10:54:09 +0000

In line to automating a complete CICD pipeline where Jenkins pipelines are triggered by GitHub web-hook based on specific events; It has been argued that neither polling for updates nor exposing the Jenkins server is desirable in any environment. Both of these issues can be resolved by using the SMEE client, having things not addressable on the web, or locked down in some default way and not burn through API quotas in the polling method.

Smee is an open source project offered by GitHub that receives web-hook payloads (from Github) and sends them to listening clients(Jenkins).

USAGE
You’ll need a method for making localhost accessible to the internet if your application wants to respond to web-hooks. http://Smee.io is a small service that transmits web-hook payloads to your locally running application through the usage of Server-Sent Events.

SETUP
Step 1: Go to https://smee.io, you may create a new channel and acquire a special URL (which you should copy for later use) to send payloads to. The public website http://smee.io and the smee-client are the two parts of Smee that make it function. Through Server-Transmitted Events, a connection type that enables messages to be sent from a source to any clients listening, they communicate with one another.

Step 2: install the smee client where you have the Jenkins server running using the command:

npm install --global smee-client

This will enable the smee client to accept and deliver webhooks.
Step3: Start the smee client now, then direct it to the Jenkins server. I’m currently running it on port 8080. (change both the port and the smee URL as needed)

smee --url https://smee.io/GSm1B40******SjYS --path /github-webhook/ --port 8080

This says to connect to the smee service, and web-hooks should be forwarded to /github-webhook/ (the trailing slash is crucial; don’t forget it). You will notice that it is connected and forwarding web-hooks as soon as this is started. You can continue to receive web-hooks as long as you leave this command running.

You can also setup the service as a systemd service, by adding the smee.service file to /etc/systemd/system directory

[Unit]
Description=smee.io webhook delivery from GitHub
After=network.target
StartLimitIntervalSec=0

[Service]
Type=simple
Restart=always
RestartSec=1
User=jenkins
ExecStart=smee -u https://smee.io/z*****BvuEt --path /generic-webhook-trigger/invoke --port 8080
[Install]
WantedBy=multi-user.target

If you’re using github-webhook-trigger in your Jenkins then the ExecStart command should be;

ExecStart=smee -u https://smee.io/z*****BvuEt --path /generic-webhook-trigger/invoke --port 8080

Step 4: Configure a pipeline that makes use of github from Jenkins. Make sure to check the ‘GitHub hook trigger for GITScm polling’ or the ‘Generic Webhook Trigger’ depending on your previous choice.

Next, specify a repository. This will prepare the server to accept webhooks from GitHub. (It’s also OK if you already have a pipeline configured that uses GitHub as the SCM source.)

Step 5: The last step is to instruct GitHub to notify Smee of webhook events for that repository.
Go to the GitHub repository’s settings tab, select “webhooks,” and then “add webhook.”

Next, configure the webhook: It should resemble the following:

Enter the “smee” URL that you copied in the previous step.
Choose application/json as the content type
Choose what events you want to trigger the web-hook
Press Add Web-hook (or update)

The links from jenkins and github will help to further understand how it works.

Understanding the Intersection of DevOps: Where Dev and Ops Converge

Esther Nnolum — Sun, 12 May 2024 10:47:44 +0000

Within the field of software development and IT operations, the term “DevOps” has gained widespread usage in the current rapidly evolving technological world. However, what precisely is DevOps, and why is it important for contemporary software delivery?

Transitioning my career trajectory towards DevOps prompted extensive research to grasp the essence and comprehensive scope of DevOps to better understand what I was getting into, and simply put, at its fundamental core; DevOps is like an enhanced cooperative effort between the teams that create software (like developers) and the teams that ensure it runs smoothly (like IT operations). In the software industry, DevOps is about bringing these teams together, facilitating improved communication, utilizing tools to make their jobs easier, and operating in a manner that swiftly and flawlessly releases updates and new features to users. It all comes down to making the software development process faster, more dependable, and more seamless for all parties involved.

The Convergence of Development and Operations

Collaborative Approach: DevOps promotes a collaborative approach, cultivating a culture in which teams from development and operations collaborate throughout the software development lifecycle. DevOps teams work closely together to remove bottlenecks and streamline operations rather than operating alone.
Continuous Integration and Delivery: concepts of continuous integration (CI) and continuous delivery (CD) are central to the discussion. Code development, testing, and deployment are all automated via CI/CD pipelines, enabling regular and dependable software releases.
Automated Infrastructure: DevOps emphasizes infrastructure as code (IaC), where infrastructure provisioning and management are automated using code-based configurations. This enables consistent, scalable, and reproducible environments.
Monitoring and Feedback: Monitoring and feedback loops are integral to DevOps. Real-time monitoring of applications and systems aids in early issue detection, allowing swift responses and iterative enhancements.

By encouraging a culture of cooperation, automation, and continuous improvement, DevOps essentially signifies a fundamental change in software development and operations. Organizations may achieve faster, more dependable, and customer-focused software delivery and maintain their competitiveness in the ever-changing business landscape of today by breaking down traditional barriers between development and operations.

Essential Linux Commands for DevOps Beginners

Esther Nnolum — Sun, 12 May 2024 10:43:43 +0000

Linux, a widely used operating system, serves as the backbone for many renowned platforms. Renowned for its flexibility and efficiency, Linux stands out as a powerful operating system extensively employed in the DevOps landscape. The majority of DevOps tools necessitate a Linux environment and It is a prevalent practice that many DevOps tools undergo development and testing primarily on Linux systems before being adapted for use on Windows platforms.

For DevOps beginners, acquiring proficiency in essential Linux commands is very vital for DevOps tasks such as server management, issue troubleshooting, DevOps tool setup, task automation, e.t.c. In the following sections, we will delve into fundamental Linux commands that are essential for any DevOps enthusiast to grasp.

Navigating the File System:

cd: Change directory.
mkdir: Create a new directory in the current directory.
pwd: Print the current working directory.
touch: create a new, empty file.
ls: List files and directories.
cp: Copy files or directories.
mv: Move or rename files or directories.
rm: Remove files or directories.

mkdir linux-devops
cd /home/ennolum/linux-devops/
touch file1 file2
pwd
ls
cp file1 /home/ennolum/.
mv file2 file3
rm file1

Managing Processes:

ps: Display information about processes.
top: Monitor system processes in real-time.
kill: Send signals to linux process (SIGKILL/-9 is the signal to terminate a process).
nice: Used to manipulate the priority of a process. Processes with higher priority receive more CPU time compared to those with lower priority

ps aux
top
kill -9 <process_id>
nice -n <value> <process name>

User and Permissions:

useradd: Create a new user.
passwd: Change user password.
chmod: Change file permissions. Note the following
r (read): 4
w (write): 2
x (execute): 1
chown: change file owner

useradd <newuser>
passwd <newuser>
chmod 755 file1.sh #7=4+2+1=rwx 5=4+1=rx

Text Processing:

cat: Display the content of a file.
grep: Search for a pattern in files.
sort: Sort lines of text files
uniq: Remove duplicate lines from texts in stdin or files
sed: Stream editor for text manipulation.
head & tail: Display the first few lines and the last few lines of a file respectively

cat file1.txt
grep <pattern> file.log
sort file.txt # use -r to sort in reverse
sort file.txt | uniq
sed 's/old/new/' file.txt #replace <old> with <new> in file.txt
head file.txt ; tail file.txt

Network Commands:

ping: Check network connectivity.
ifconfig or ip: Display network configuration.
netstat: Display network status information.
dig: Used to query the DNS name server.
hostname: Used to view and set the hostname of a system

ping google.com
ifconfig
netstat -an
dig <domainName>

Package Management:

apt (Debian/Ubuntu) or yum (RHEL/CentOS): Install, update, or remove packages.

sudo apt-get update
sudo apt-get install <package_name>
sudo yum update
sudo yum install <package-name>

File Compression and Archiving:

tar: Create, view, or extract tar archives.
gzip and gunzip: Compress and decompress files.
zip: Creates a zip archive from one or more files or directories

tar -cvf archive.tar files/ # Combine tar and compression commands
gzip file.txt

SSH and Remote Access:

ssh: Securely connect to a remote server.
scp: Copy files securely between local and remote systems or between 2 remote systems.

ssh <user@remote_host>
scp local_file.txt user@remote_host:/path/to/destination/

System Information:

uname: Display system information.
df and du: Show disk space usage (df — filesystem and du disk usage).
uptime: Displays duration the system has been operational and the average system load.
free -m: Provides details on the utilization of the system’s memory

uname -a
df -h
du -sh /path/to/directory

Shell Scripting Basics: Shell scripting plays a vital role in automating processes on Linux systems. By creating a script — a file containing a series of commands — you can execute tasks without repeatedly typing commands. This approach enhances efficiency in performing routine tasks, enabling streamlined daily operations and the possibility of scheduling automated executions.

echo: Display a message.
chmod +x script.sh and ./script.sh: Make a script executable and run it respectively.

echo "Hello, DevOps"

Conclusion:
Mastering these fundamental Linux commands will provide a solid foundation for DevOps beginners. As you become more comfortable with these commands, you’ll be better equipped to navigate, manage, and automate tasks within a Linux environment.