DEV Community: VENKATA SRI HARI

Cut AWS EC2 Costs with Python: Automate Cost Optimization Using AWS Lambda

VENKATA SRI HARI — Mon, 17 Feb 2025 06:08:20 +0000

Introduction
Managing AWS EC2 costs efficiently is crucial for DevOps and cloud engineers. In this guide, we’ll explore how to automate EC2 instance optimization using AWS Lambda and Python (Boto3). You’ll learn how to:
✅ Automatically stop and start instances based on business hours
✅ Monitor CPU utilization and shut down underutilized instances
✅ Use tags and CloudWatch metrics to identify non-essential resources
✅ Schedule the Lambda function using EventBridge (CloudWatch Events)
✅ Improve cost efficiency while ensuring service availability.

With hands-on examples and best practices, this article will help you reduce EC2 costs without manual intervention.

Lambda Use Case
Create an IAM Role for Lambda
Write the Python Code for Lambda
Schedule the Lambda Function
Modify to Stop EC2 Instances Based on CPU Usage
Final Steps
Conclusion.
**

Lambda Use Case: -** Stop idle EC2 instances outside of business hours (e.g., stop from 8 PM to 8 AM). Start instances automatically during working hours. Monitor CPU usage and shut down underutilized instances. Tag-based filtering to exclude critical instances.

*2.Create an IAM Role for Lambda: -
*
Your Lambda function needs EC2 permissions to start/stop instances.
Create an IAM Role with the following permissions:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "ec2:DescribeInstances",
                "ec2:StartInstances",
                "ec2:StopInstances"
            ],
            "Resource": "*"
        }
    ]
}

Version: The policy language version.
Statement: The array of permissions statements.
Effect: This policy allows the listed actions.
Action: The specific EC2 actions allowed by this policy (DescribeInstances, StartInstances, StopInstances).
Resource: The resource scope for the actions (in this case, all resources).

3. Write the Python Code for Lambda
Auto-Stop EC2 Instances Based on Time:-

import boto3
import datetime

# Initialize EC2 client
ec2 = boto3.client('ec2')

# Define business hours (e.g., 8 AM - 8 PM)
START_HOUR = 8
STOP_HOUR = 20

def lambda_handler(event, context):
    now = datetime.datetime.now()
    current_hour = now.hour

    # Fetch all running instances with the tag `AutoStop=Yes`
    instances = ec2.describe_instances(Filters=[
        {'Name': 'instance-state-name', 'Values': ['running']},
        {'Name': 'tag:AutoStop', 'Values': ['Yes']}
    ])

    instance_ids = [i['InstanceId'] for r in instances['Reservations'] for i in r['Instances']]

    if current_hour >= STOP_HOUR or current_hour < START_HOUR:
        if instance_ids:
            ec2.stop_instances(InstanceIds=instance_ids)
            print(f"Stopped instances: {instance_ids}")
    else:
        print("Business hours - no action needed")

    return {
        'statusCode': 200,
        'body': 'Lambda execution completed'
    }

How It Works?
The function runs on a schedule (CloudWatch Event Trigger).
It checks if the current time is outside business hours (8 AM — 8 PM).
If instances have the tag AutoStop=Yes, they are stopped.

4. Schedule the Lambda Function
Use Amazon EventBridge (CloudWatch Rule) to trigger this Lambda:

Cron Expression: 0 20 * * ? * → Runs at 8 PM UTC every day.
Separate Function: Create a separate Lambda function to start instances at 8 AM, ensuring they are available during working hours.

5. Modify to Stop EC2 Instances Based on CPU Usage
If you want to stop EC2 instances with low CPU utilization, modify the function to check CloudWatch CPU metrics:

import boto3

ec2 = boto3.client('ec2')
cloudwatch = boto3.client('cloudwatch')

def get_cpu_utilization(instance_id):
    response = cloudwatch.get_metric_statistics(
        Namespace='AWS/EC2',
        MetricName='CPUUtilization',
        Dimensions=[{'Name': 'InstanceId', 'Value': instance_id}],
        StartTime=datetime.datetime.utcnow() - datetime.timedelta(minutes=30),
        EndTime=datetime.datetime.utcnow(),
        Period=300,
        Statistics=['Average']
    )
    return response['Datapoints'][0]['Average'] if response['Datapoints'] else 0

def lambda_handler(event, context):
    instances = ec2.describe_instances(Filters=[{'Name': 'instance-state-name', 'Values': ['running']}])
    for res in instances['Reservations']:
        for inst in res['Instances']:
            instance_id = inst['InstanceId']
            cpu_usage = get_cpu_utilization(instance_id)

            if cpu_usage < 10:  # Stop instances with <10% CPU utilization
                ec2.stop_instances(InstanceIds=[instance_id])
                print(f"Stopped instance {instance_id} due to low CPU usage")

    return {'statusCode': 200, 'body': 'Lambda executed successfully'}

6. Final Steps
Deploy the Lambda function in AWS Lambda.
Assign the IAM Role created earlier.
Set an EventBridge schedule (e.g., every hour) to trigger it.
Tag instances with AutoStop=Yes to include them in automation.
This Lambda function helps reduce EC2 costs by shutting down idle or low-utilization instances while keeping critical workloads running.

Conclusion:
Automating AWS EC2 cost optimization with Python and AWS Lambda is a smart way to reduce cloud expenses while maintaining operational efficiency. By implementing automated start/stop schedules, monitoring CPU utilization, and leveraging tags, you can ensure that only necessary instances are running — saving costs without compromising performance.

By integrating CloudWatch, EventBridge, and Boto3, you create a scalable and hands-free solution that continuously optimizes your cloud infrastructure. This approach not only enhances cost efficiency but also aligns with DevOps best practices for cloud automation.

Start implementing cost-saving Lambda functions today, and take control of your AWS billing! 🚀💡

………………………………………………………………………………………………………………………………………………………………………………………………

You’re welcome! Have a great time ahead! Enjoy your day!

Please Connect with me any doubts.

Mail: sriharimalapati6@gmail.com

LinkedIn: www.linkedin.com/in/

GitHub: https://github.com/Consultantsrihari

Medium: Sriharimalapati — Medium

Thanks for watching ##### %%%% Sri Hari %%%%
Python
AWS Lambda
AWS
DevOps
Automation

Mastering AWS DevOps: Automation & Scripting with Python

VENKATA SRI HARI — Thu, 13 Feb 2025 07:09:10 +0000

Unlock the power of AWS DevOps with Python scripting! In this we’ll explore how to automate cloud infrastructure, streamline CI/CD pipelines, and optimize deployments using Python. From AWS SDK (Boto3) to scripting EC2, S3, Lambda, and IAM automation, this article covers real-world examples and best practices for DevOps engineers. Whether you’re automating deployments, monitoring resources, or enhancing security, this guide has you covered!

💡 Key Takeaways:
✅ Automate AWS infrastructure with Python (Boto3)
✅ Integrate Python into CI/CD pipelines
✅ Optimize AWS services with automation scripts
✅ Best practices for DevOps scripting

Let’s dive into AWS DevOps automation!
**
What is AWS DevOps, and how does Python fit into it?**
AWS DevOps is a set of practices that automate and streamline software development, deployment, and infrastructure management on AWS. Python plays a crucial role by enabling automation through scripts using Boto3 (AWS SDK for Python), infrastructure-as-code (IaC) tools, and integration with CI/CD pipelines.

Q1: How can Python be used to automate EC2 instance management?

A: Python can automate EC2 operations such as instance creation, termination, and status checks.

Example: Launching an EC2 instance using Boto3

import boto3

ec2 = boto3.resource('ec2')
instance = ec2.create_instances(
    ImageId='ami-0abcdef1234567890',
    MinCount=1,
    MaxCount=1,
    InstanceType='t2.micro',
    KeyName='my-key'
)
print("EC2 Instance Created:", instance[0].id)

Q2: How do you use Boto3 to list all EC2 instances?

A: Boto3 is the AWS SDK for Python. You can list all EC2 instances using:

import boto3

ec2 = boto3.client('ec2')

response = ec2.describe_instances()
for reservation in response['Reservations']:
    for instance in reservation['Instances']:
        print(f"Instance ID: {instance['InstanceId']}, State: {instance['State']['Name']}")

Q3: How do you automate S3 bucket creation using Python?

import boto3

s3 = boto3.client('s3')
bucket_name = "my-unique-bucket-12345"

s3.create_bucket(Bucket=bucket_name)
print(f"Bucket {bucket_name} created successfully!")

Q4: How do you trigger a Lambda function using Python?

import boto3

lambda_client = boto3.client('lambda')
response = lambda_client.invoke(
    FunctionName='myLambdaFunction',
    InvocationType='Event'
)

print("Lambda triggered:", response)

Q5: How do you use Python to stop EC2 instances based on a tag?


import boto3

ec2 = boto3.client('ec2')

instances = ec2.describe_instances(Filters=[{'Name': 'tag:Environment', 'Values': ['dev']}])

for reservation in instances['Reservations']:
    for instance in reservation['Instances']:
        ec2.stop_instances(InstanceIds=[instance['InstanceId']])
        print(f"Stopping instance: {instance['InstanceId']}")

##### This stops all EC2 instances with the tag Environment=dev.

*Q6: How do you automate IAM user creation with Python?
*

import boto3

iam = boto3.client('iam')

response = iam.create_user(UserName='newuser')
print(f"User created: {response['User']['UserName']}")

#### This creates a new IAM user.

Q7: How do you automate Route 53 DNS record creation using Python?

import boto3

route53 = boto3.client('route53')

response = route53.change_resource_record_sets(
    HostedZoneId='ZXXXXXXXXXXXXX',
    ChangeBatch={
        'Changes': [{
            'Action': 'CREATE',
            'ResourceRecordSet': {
                'Name': 'example.mydomain.com',
                'Type': 'A',
                'TTL': 300,
                'ResourceRecords': [{'Value': '192.168.1.1'}]
            }
        }]
    }
)

print("DNS record created:", response)

This creates an A record in Route 53.

*Q8: How do you use Python to fetch AWS CloudWatch logs?
*

import boto3

logs = boto3.client('logs')

response = logs.describe_log_groups()
for log_group in response['logGroups']:
    print(f"Log Group: {log_group['logGroupName']}")

This fetches all available CloudWatch log groups.

*Q9: How do you automate AMI creation using Python?
*

import boto3

ec2 = boto3.client('ec2')
instance_id = "i-0abcd1234efgh5678"

response = ec2.create_image(
    InstanceId=instance_id,
    Name="MyBackupAMI",
    NoReboot=True
)

print("AMI ID:", response['ImageId'])

This creates an Amazon Machine Image (AMI) from an EC2 instance.

*Q10: How do you upload a file to an S3 bucket using Python?
*

import boto3

s3 = boto3.client('s3')

s3.upload_file("local_file.txt", "my-bucket", "uploaded_file.txt")
print("File uploaded successfully!")

This uploads local_file.txt to my-bucket.

Q11: How do you automate AWS Lambda deployment using Python?

import boto3

lambda_client = boto3.client('lambda')

with open('lambda_function.zip', 'rb') as f:
    zip_data = f.read()

response = lambda_client.update_function_code(
    FunctionName='myLambdaFunction',
    ZipFile=zip_data
)

print("Lambda function updated:", response)

This updates an AWS Lambda function with new code.

*Q12: How do you monitor AWS services using Python and CloudWatch?
*

import boto3

cloudwatch = boto3.client('cloudwatch')
response = cloudwatch.get_metric_statistics(
    Namespace='AWS/EC2',
    MetricName='CPUUtilization',
    Dimensions=[{'Name': 'InstanceId', 'Value': 'i-1234567890abcdef0'}],
    StartTime='2024-02-01T00:00:00Z',
    EndTime='2024-02-02T00:00:00Z',
    Period=3600,
    Statistics=['Average']
)
print("CPU Utilization Data:", response)

Q13: How do you integrate Python with CI/CD pipelines for AWS?

Python can be used in Jenkins, GitHub Actions, and AWS CodePipeline for tasks like:
🔹 Running automated tests with PyTest
🔹 Deploying infrastructure using Terraform with Python scripts
🔹 Triggering AWS Lambda functions for post-deployment automation

Example: A Python script to trigger an AWS Lambda function during deployment

import boto3

lambda_client = boto3.client('lambda')
response = lambda_client.invoke(
    FunctionName='myLambdaFunction',
    InvocationType='RequestResponse'
)
print("Lambda Function Triggered:", response)

Conclusion: Mastering AWS DevOps Automation with Python
Automating AWS with Python scripting is a game-changer for DevOps engineers, enabling efficient infrastructure management, CI/CD optimization, and security enhancements. With powerful tools like Boto3, AWS Lambda, and CloudWatch, Python simplifies tasks such as EC2 management, S3 automation, IAM security, and cost optimization.

By integrating Python into your DevOps workflows, you can:
✅ Reduce manual intervention and human errors
✅ Improve deployment speed and system reliability
✅ Optimize AWS resources for cost efficiency
✅ Enhance security through automated IAM policies

**You’re welcome! Have a great time ahead! Enjoy your day!

Please Connect with me any doubts.

Mail: sriharimalapati6@gmail.com

LinkedIn: www.linkedin.com/in/

GitHub: https://github.com/Consultantsrihari

Medium: Sriharimalapati — Medium

Thanks for watching ##### %%%% Sri Hari %%%%**

Building a Scalable 3-Tier Architecture on AWS Using Terraform: A Modular Approach

VENKATA SRI HARI — Wed, 05 Feb 2025 18:41:49 +0000

In this project, we will design and implement a scalable three-tier architecture on AWS using Terraform. The setup follows a modular approach, organizing infrastructure into separate layers: Core, Web, App, and Database. This ensures better manageability, reusability, and security while deploying cloud resources. By the end, you will have a fully automated, infrastructure-as-code (IaC) solution for hosting applications on AWS.

GitHub Repository: https://github.com/Consultantsrihari/3-Tier-Architecture-on-AWS-Using-Terraform

Prerequisites
Before starting this project, ensure you have the following:

✅ AWS Account — To provision cloud resources using Terraform.
✅ Terraform Installed — Download and install Terraform on your local machine.
✅ AWS CLI Installed & Configured — Install AWS CLI and run aws configure to set up credentials.
✅ IAM User with Required Permissions — Ensure your IAM user has permissions to create and manage VPCs, EC2, RDS, ALB, and other AWS resources.
✅ Basic Knowledge of Terraform — Familiarity with writing .tf files, providers, modules, and variables.
✅ Code Editor (VS Code Recommended) — Use VS Code with the Terraform extension for syntax highlighting and better development experience.
✅ Git Installed (Optional) — For version control and managing Terraform code in a GitHub repository.

Architecture Overview
We will deploy a three-tier architecture consisting of the following:

VPC (Core Layer): Contains public and private subnets.
Public Subnets: Bastion Host & NAT Gateway.
Private Subnets: Web/Application servers (EC2 instances) and Database (Amazon RDS).
Load Balancer (ALB): Distributes traffic.
Security Groups: Restricted access at different layers.
**Project Structure
**Root Module (main.tf)
The root module ties together the core network, web, app, and database modules.

module "core" {
  source = "./modules/core"

  vpc_cidr             = "10.0.0.0/16"
  public_subnet_cidrs  = ["10.0.1.0/24", "10.0.2.0/24"]
  private_subnet_cidrs = ["10.0.3.0/24", "10.0.4.0/24"]
  db_subnet_cidrs      = ["10.0.5.0/24", "10.0.6.0/24"]
  azs                  = ["us-east-1a", "us-east-1b"]
}

module "web" {
  source = "./modules/web"

  public_subnet_ids  = module.core.public_subnet_ids
  web_alb_sg_id      = module.core.web_alb_sg_id
  web_instance_sg_id = module.core.web_instance_sg_id
  web_ami            = "ami-0c55b159cbfafe1f0"
  web_instance_type  = "t2.micro"
}

module "app" {
  source = "./modules/app"

  private_subnet_ids = module.core.private_subnet_ids
  app_alb_sg_id      = module.core.app_alb_sg_id
  app_instance_sg_id = module.core.app_instance_sg_id
  app_ami            = "ami-0c55b159cbfafe1f0"
  app_instance_type  = "t2.micro"
}

module "database" {
  source = "./modules/database"

  db_subnet_ids = module.core.db_subnet_ids
  db_sg_id      = module.core.db_sg_id
  db_name       = "mydb"
  db_user       = "admin"
  db_password   = "securepassword123"
}

Security Group Setup (sg.tf)
Security groups are used to restrict traffic at different layers.

# Web ALB Security Group
resource "aws_security_group" "web_alb" {
  name        = "web-alb-sg"
  description = "Allow HTTP inbound traffic"
  vpc_id      = aws_vpc.main.id

  ingress {
    from_port   = 80
    to_port     = 80
    protocol    = "tcp"
    cidr_blocks = ["0.0.0.0/0"]
  }

  egress {
    from_port   = 0
    to_port     = 0
    protocol    = "-1"
    cidr_blocks = ["0.0.0.0/0"]
  }
}

# Database Security Group
resource "aws_security_group" "database" {
  name        = "db-sg"
  description = "Allow MySQL access from app tier"
  vpc_id      = aws_vpc.main.id

  ingress {
    from_port       = 3306
    to_port         = 3306
    protocol        = "tcp"
    security_groups = [module.core.app_instance_sg_id]
  }
}

Module Definitions

Core Network Module (modules/core/main.tf) This module creates the VPC, subnets, NAT Gateway, and route tables.

resource "aws_vpc" "main" {
  cidr_block = var.vpc_cidr
  tags = {
    Name = "3tier-vpc"
  }
}

# Public Subnets
resource "aws_subnet" "public" {
  count             = length(var.public_subnet_cidrs)
  vpc_id            = aws_vpc.main.id
  cidr_block        = var.public_subnet_cidrs[count.index]
  availability_zone = var.azs[count.index]
  tags = {
    Name = "public-subnet-${count.index}"
  }
}

# Private Subnets
resource "aws_subnet" "private" {
  count             = length(var.private_subnet_cidrs)
  vpc_id            = aws_vpc.main.id
  cidr_block        = var.private_subnet_cidrs[count.index]
  availability_zone = var.azs[count.index]
  tags = {
    Name = "private-subnet-${count.index}"
  }
}

# Internet Gateway
resource "aws_internet_gateway" "gw" {
  vpc_id = aws_vpc.main.id
}

# NAT Gateway
resource "aws_nat_gateway" "nat" {
  allocation_id = aws_eip.nat.id
  subnet_id     = aws_subnet.public[0].id
}

resource "aws_eip" "nat" {
  vpc = true
}

2. Web Tier Module (modules/web/main.tf)
This module defines the public-facing ALB, EC2 instances, and auto-scaling group for the web tier.

resource "aws_lb" "web" {
  name               = "web-alb"
  internal           = false
  load_balancer_type = "application"
  security_groups    = [var.web_alb_sg_id]
  subnets            = var.public_subnet_ids
}

resource "aws_launch_template" "web" {
  name_prefix   = "web-"
  image_id      = var.web_ami
  instance_type = var.web_instance_type
  key_name      = var.key_name

  network_interfaces {
    security_groups = [var.web_instance_sg_id]
  }

  user_data = base64encode(<<-EOF
              #!/bin/bash
              yum install -y nginx
              systemctl start nginx
              EOF
              )
}

resource "aws_autoscaling_group" "web" {
  desired_capacity   = 2
  max_size           = 4
  min_size           = 2
  vpc_zone_identifier = var.public_subnet_ids

  launch_template {
    id      = aws_launch_template.web.id
    version = "$Latest"
  }
}

3. Application Tier Module (modules/app/main.tf)
This module defines the internal ALB, EC2 instances, and auto-scaling group for the app tier.

resource "aws_lb" "app" {
  name               = "app-alb"
  internal           = true
  load_balancer_type = "application"
  security_groups    = [var.app_alb_sg_id]
  subnets            = var.private_subnet_ids
}

resource "aws_launch_template" "app" {
  name_prefix   = "app-"
  image_id      = var.app_ami
  instance_type = var.app_instance_type
  key_name      = var.key_name

  network_interfaces {
    security_groups = [var.app_instance_sg_id]
  }
}

resource "aws_autoscaling_group" "app" {
  desired_capacity   = 2
  max_size           = 4
  min_size           = 2
  vpc_zone_identifier = var.private_subnet_ids

  launch_template {
    id      = aws_launch_template.app.id
    version = "$Latest"
  }
}

4. Database Tier Module (modules/database/main.tf)
This module defines the MySQL RDS instance and associated subnet group.

resource "aws_db_subnet_group" "db" {
  name       = "db-subnet-group"
  subnet_ids = var.db_subnet_ids
}

resource "aws_db_instance" "main" {
  allocated_storage    = 20
  storage_type         = "gp2"
  engine               = "mysql"
  engine_version       = "5.7"
  instance_class       = "db.t2.micro"
  db_name              = var.db_name
  username             = var.db_user
  password             = var.db_password
  parameter_group_name = "default.mysql5.7"
  skip_final_snapshot  = true
  db_subnet_group_name = aws_db_subnet_group.db.name
  vpc_security_group_ids = [var.db_sg_id]
}

Deployment Steps
1️. Initialize Terrafom
terraform init
2️. Plan the Deployme
terraform plan
3️. Apply the Configuratin
terraform apply -auto-approve

Verify the Deployment 5️.Destroy Infrastructure terraform destroy -auto-approve Key Components: *Check AWS Console for created resources. Validate EC2 instances, ALB, and RDS. *

Conclusion:
This Terraform-driven 3-tier AWS deployment establishes a scalable,
secure, and highly available architecture, leveraging infrastructure-as-code(IaC) best practices. By modularizing components (web, app, database) and isolating network layers, it ensures fault tolerance, simplified maintenance,and controlled access between tiers. The use of auto-scaling groups, ALBs,and RDS guarantees resilience and performance, while security groups enforce least-privilege principles. This foundation supports seamless scaling, cost optimization, and compliance readiness, providing a robust blueprint for modern cloud-native applications. Future enhancements like HTTPS, WAF, or secrets management can further strengthen production readiness.

………………………………………………………………………………………………………………………………………
………………………………………………………
You’re welcome! Have a great time ahead! Enjoy your day!
Please Connect with me any doubts.
Mail: sriharimalapati6@gmail.com
LinkedIn: www.linkedin.com/in/
GitHub: https://github.com/Consultantsrihari
Medium: Sriharimalapati — Medium
Thanks for watching ##### %%%% Sri Hari %%%%

Top 50 Hands-On DevOps Projects: From Zero to Hero

VENKATA SRI HARI — Fri, 17 Jan 2025 12:05:10 +0000

Beginner Projects

Version Control with Git and GitHub

Collaborate on a project using branches, pull requests, and merge conflicts.

CI/CD with Jenkins

Set up a simple pipeline for a Java/Maven or Node.js application.

Dockerize an Application

Containerize a Node.js or Python application with Docker and manage images.

Basic AWS EC2 Setup

Launch and connect to an EC2 instance, and deploy a simple web app.

Infrastructure Automation with Terraform

Create a basic VPC, subnets, and an EC2 instance.

Automated Configuration with Ansible

Write a playbook to install and configure Apache or Nginx on multiple servers.

Linux Administration for DevOps

Manage users, groups, and permissions. Automate tasks with shell scripts.

Monitoring with Prometheus and Grafana

Monitor basic system metrics and create visual dashboards.

Static Code Analysis with SonarQube

Integrate SonarQube into Jenkins pipelines for code quality checks.

Container Orchestration with Docker Swarm

Deploy a multi-container app using Docker Compose and Docker Swarm.

Intermediate Projects

Multi-Region AWS Deployment

Deploy a web app with a load balancer and Auto Scaling across regions.

CI/CD Pipeline with GitHub Actions

Automate builds, tests, and deployments for a Node.js app.

Kubernetes Cluster Setup

Install Kubernetes (Minikube or k3s) and deploy a multi-tier application.

Infrastructure as Code with Terraform Modules

Create reusable Terraform modules for networking and compute resources.

Automated Backups with AWS S3

Automate EC2 snapshot backups and upload them to S3 using a script.

Blue-Green Deployment with Jenkins and Kubernetes

Implement a zero-downtime deployment strategy.

Log Aggregation with ELK Stack

Set up Elasticsearch, Logstash, and Kibana to centralize logs from multiple sources.

Ansible Roles for Multi-Tier Deployment

Use roles to configure web, app, and database servers.

Docker Networking Basics

Set up bridge, overlay, and host networks in Docker.

SSL/TLS Certificate Automation

Automate SSL certificate provisioning using Let's Encrypt and Certbot.

Advanced Projects

Hybrid Cloud Setup with AWS and Azure

Use Terraform to create resources across both AWS and Azure.

GitOps with Argo CD

Manage Kubernetes applications with GitOps principles.

Automated Testing Pipeline with Selenium and Jenkins

Integrate Selenium for automated UI testing in a CI/CD pipeline.

Dynamic Scaling with Kubernetes HPA

Deploy an app with Horizontal Pod Autoscaler based on CPU/memory metrics.

Multi-Cluster Kubernetes Management with Rancher

Manage multiple Kubernetes clusters using Rancher.

Serverless CI/CD with AWS Lambda

Automate deployments using AWS Lambda and CodePipeline.

Kubernetes Network Policies

Implement fine-grained network controls for microservices.

Building a Private Docker Registry

Set up and secure a private Docker registry for your team.

Implementing Service Mesh with Istio

Deploy Istio for traffic management and observability in a Kubernetes cluster.

Securing DevOps Pipelines

Integrate SAST and DAST tools for security scanning in CI/CD.

Distributed Tracing with Jaeger

Monitor microservices with distributed tracing in Kubernetes.

Infrastructure Monitoring with Datadog

Use Datadog to monitor and visualize cloud and server resources.

Multi-Environment CI/CD Pipelines

Implement pipelines for dev, staging, and production environments.

Advanced Terraform State Management

Use remote state backends with locking mechanisms (e.g., S3 + DynamoDB).

Custom Helm Chart Development

Create Helm charts for application deployment on Kubernetes.

Implementing Chaos Engineering

Use tools like Gremlin or Chaos Monkey to test system resilience.

CI/CD for Microservices with Jenkins and Docker

Create pipelines for multiple services with interdependencies.

Real-Time Monitoring with Prometheus Alertmanager

Configure alerting rules and notifications for system events.

Setting Up Centralized Authentication in Kubernetes

Implement RBAC with external authentication (e.g., Keycloak).

Disaster Recovery Strategy with AWS Backup

Automate backup and recovery for critical workloads.

Expert-Level Projects

Deploying a Kubernetes Operator

Develop and deploy a custom Kubernetes operator.

Cross-Cloud CI/CD Pipelines

Implement CI/CD for an app deployed across AWS and GCP.

Serverless Microservices with AWS Lambda and API Gateway

Build a scalable serverless architecture for an application.

Dynamic Secrets Management with HashiCorp Vault

Securely manage application secrets and credentials.

Monitoring Kubernetes with Thanos

Set up a highly available monitoring system for Kubernetes clusters.

Implementing Canary Deployments

Deploy updates incrementally and monitor performance with Kubernetes.

Pipeline as Code with Jenkinsfile

Create complex pipelines entirely using Jenkinsfile syntax.

Container Security with Aqua or Trivy

Scan container images for vulnerabilities and enforce compliance.

Self-Healing Infrastructure with Auto Remediation

Automate recovery from failures using AWS Lambda and CloudWatch.

Full CI/CD with End-to-End Security and Observability

Set up pipelines with advanced features like automated rollbacks, Slack alerts, Prometheus metrics, and Jaeger tracing.

AWS DevOps Project: Advanced Automated CI/CD Pipeline

VENKATA SRI HARI — Tue, 31 Dec 2024 17:13:21 +0000

AWS DevOps Project: Advanced Automated CI/CD Pipeline

This project demonstrates how to set up an advanced automated CI/CD pipeline with Infrastructure as Code (IaC), microservices, a service mesh, and monitoring using AWS. Tools like Terraform, Jenkins, Kubernetes (EKS), Istio, Prometheus, Grafana, and ArgoCD are utilized.

Github: https://github.com/Consultantsrihari/AWS-DevOps-Project-Advanced-Automated-CI-CD-Pipeline.git

Architecture Overview

Infrastructure as Code: AWS resources are provisioned using Terraform.
Containerized Microservices: Deployed on EKS.
Service Mesh: Managed by Istio for advanced traffic control and observability.
CI/CD Pipeline: Managed with Jenkins and ArgoCD.
Monitoring: Implemented using Prometheus and Grafana.

Pre-requisites

AWS Account.
Terraform installed locally.
Kubectl and Helm installed.
Jenkins installed and configured.
Docker installed.
Prometheus and Grafana configured for monitoring.

Step 1: Provision Infrastructure with Terraform

Terraform Configuration (main.tf)

provider "aws" {
  region = "us-east-1"
}

resource "aws_vpc" "dev_vpc" {
  cidr_block           = "10.0.0.0/16"
  enable_dns_support   = true
  enable_dns_hostnames = true

  tags = {
    Name = "dev-vpc"
  }
}

resource "aws_subnet" "public_subnet" {
  vpc_id                  = aws_vpc.dev_vpc.id
  cidr_block              = "10.0.1.0/24"
  map_public_ip_on_launch = true

  tags = {
    Name = "public-subnet"
  }
}

resource "aws_eks_cluster" "eks_cluster" {
  name     = "dev-cluster"
  role_arn = aws_iam_role.eks_role.arn

  vpc_config {
    subnet_ids = [aws_subnet.public_subnet.id]
  }
}

Run the following commands:

terraform init
terraform apply -auto-approve

Step 2: Deploy Jenkins for CI/CD

Helm Chart for Jenkins

helm repo add jenkinsci https://charts.jenkins.io
helm repo update
helm install jenkins jenkinsci/jenkins

Access Jenkins UI and install required plugins (e.g., Docker, Kubernetes, Git, Pipeline).

Step 3: Deploy Kubernetes Cluster and Microservices

Sample Deployment YAML (microservice-deployment.yaml)

apiVersion: apps/v1
kind: Deployment
metadata:
  name: sample-microservice
spec:
  replicas: 3
  selector:
    matchLabels:
      app: sample
  template:
    metadata:
      labels:
        app: sample
    spec:
      containers:
      - name: sample-app
        image: myregistry/sample-app:latest
        ports:
        - containerPort: 8080

Apply the configuration:

kubectl apply -f microservice-deployment.yaml

Step 4: Configure Istio Service Mesh

Install Istio

istioctl install --set profile=demo -y
kubectl label namespace default istio-injection=enabled

Sample VirtualService Configuration

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: sample-app
spec:
  hosts:
  - "*"
  gateways:
  - sample-gateway
  http:
  - route:
    - destination:
        host: sample-microservice
        port:
          number: 8080

Apply the configuration:

kubectl apply -f virtualservice.yaml

Step 5: Integrate Monitoring with Prometheus and Grafana

Install Prometheus and Grafana

helm install prometheus prometheus-community/prometheus
helm install grafana grafana/grafana

Access Grafana Dashboard and connect to Prometheus as a data source.

Step 6: Jenkins Pipeline Configuration

Jenkinsfile

pipeline {
  agent any
  stages {
    stage('Checkout Code') {
      steps {
        checkout scm
      }
    }
    stage('Build Docker Image') {
      steps {
        sh 'docker build -t myregistry/sample-app:latest .'
      }
    }
    stage('Push to Registry') {
      steps {
        sh 'docker push myregistry/sample-app:latest'
      }
    }
    stage('Deploy to Kubernetes') {
      steps {
        sh 'kubectl apply -f microservice-deployment.yaml'
      }
    }
  }
}

Step 7: Continuous Delivery with ArgoCD

Install ArgoCD

kubectl create namespace argocd
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml

The kubectl apply command is used to install ArgoCD in a dedicated namespace (argocd). ArgoCD manages the deployment and synchronization of applications in Kubernetes clusters using GitOps principles. Once installed, it acts as a continuous delivery tool that monitors a Git repository for changes and automatically applies them to the cluster.

ArgoCD Application YAML

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: sample-app
spec:
  destination:
    namespace: default
    server: https://kubernetes.default.svc
  source:
    path: sample-app
    repoURL: https://github.com/your-repo/sample-app.git
    targetRevision: HEAD

Apply the configuration:

kubectl apply -f argocd-application.yaml

Summary

Infrastructure: Provisioned with Terraform.
CI/CD: Managed by Jenkins and ArgoCD.
Service Mesh: Configured with Istio.
Monitoring: Handled by Prometheus and Grafana.

This setup ensures a robust and automated DevOps workflow, promoting scalability, observability, and reliability.

Create a Terraform project which Implements a VPC on AWS and deploys to EKS cluster.

VENKATA SRI HARI — Tue, 31 Dec 2024 16:37:27 +0000

In this Terraform project we will learn how to provisions a Virtual Private Cloud (VPC) on AWS, deploys an Amazon Elastic Kubernetes Service (EKS) cluster within that VPC, and provides associated networking and IAM resources.

In this I’ll create a well-organized Terraform configuration for an AWS VPC and EKS cluster, split into logical modules.

The complete project GitHub URl: https://github.com/Consultantsrihari/Create-a-Terraform-project-which-Implements-a-VPC-on-AWS-and-deploys-to-EKS-cluster..git

AWS VPC and EKS Terraform Configuration
**_

I. created a modular Terraform configuration that sets up:

_**Main.tf:

terraform {
  required_providers {
    aws = {
      source  = "hashicorp/aws"
      version = "~> 5.0"
    }
  }
}

provider "aws" {
  region = var.aws_region
}

module "vpc" {
  source = "./modules/vpc"

  vpc_cidr           = var.vpc_cidr
  availability_zones = var.availability_zones
  environment        = var.environment
}

module "eks" {
  source = "./modules/eks"

  environment         = var.environment
  subnet_ids         = module.vpc.private_subnet_ids
  kubernetes_version = "1.27"

  node_desired_size  = 2
  node_max_size      = 4
  node_min_size      = 1
  node_instance_types = ["t3.medium"]
}

Variables.tf:

variable "aws_region" {
  description = "AWS region"
  type        = string
  default     = "us-east-1"
}

variable "environment" {
  description = "Environment name"
  type        = string
  default     = "dev"
}

variable "vpc_cidr" {
  description = "CIDR block for VPC"
  type        = string
  default     = "10.0.0.0/16"
}

variable "availability_zones" {
  description = "Availability zones"
  type        = list(string)
  default     = ["us-east-1a", "us-east-1b", "us-east-1c"]
}

Outputs.tf

output "vpc_id" {
  value = module.vpc.vpc_id
}

output "cluster_endpoint" {
  value = module.eks.cluster_endpoint
}

output "cluster_name" {
  value = module.eks.cluster_name
}

**II. A VPC module (modules/vpc) that creates:

_
**VPC with DNS support
Public and private subnets across multiple AZs
Internet Gateway
NAT Gateways
Route tables

main.tf

# VPC
resource "aws_vpc" "main" {
  cidr_block           = var.vpc_cidr
  enable_dns_hostnames = true
  enable_dns_support   = true

  tags = {
    Name        = "${var.environment}-vpc"
    Environment = var.environment
  }
}

# Public Subnets
resource "aws_subnet" "public" {
  count             = length(var.availability_zones)
  vpc_id            = aws_vpc.main.id
  cidr_block        = cidrsubnet(var.vpc_cidr, 8, count.index)
  availability_zone = var.availability_zones[count.index]

  map_public_ip_on_launch = true

  tags = {
    Name                                           = "${var.environment}-public-${var.availability_zones[count.index]}"
    Environment                                    = var.environment
    "kubernetes.io/cluster/${var.environment}-eks" = "shared"
    "kubernetes.io/role/elb"                       = 1
  }
}

# Private Subnets
resource "aws_subnet" "private" {
  count             = length(var.availability_zones)
  vpc_id            = aws_vpc.main.id
  cidr_block        = cidrsubnet(var.vpc_cidr, 8, count.index + length(var.availability_zones))
  availability_zone = var.availability_zones[count.index]

  tags = {
    Name                                           = "${var.environment}-private-${var.availability_zones[count.index]}"
    Environment                                    = var.environment
    "kubernetes.io/cluster/${var.environment}-eks" = "shared"
    "kubernetes.io/role/internal-elb"              = 1
  }
}

# Internet Gateway
resource "aws_internet_gateway" "main" {
  vpc_id = aws_vpc.main.id

  tags = {
    Name        = "${var.environment}-igw"
    Environment = var.environment
  }
}

# Elastic IP for NAT Gateway
resource "aws_eip" "nat" {
  domain = "vpc"

  tags = {
    Name        = "${var.environment}-nat-eip"
    Environment = var.environment
  }
}

# NAT Gateway
resource "aws_nat_gateway" "main" {
  allocation_id = aws_eip.nat.id
  subnet_id     = aws_subnet.public[0].id

  tags = {
    Name        = "${var.environment}-nat"
    Environment = var.environment
  }
}

# Public Route Table
resource "aws_route_table" "public" {
  vpc_id = aws_vpc.main.id

  route {
    cidr_block = "0.0.0.0/0"
    gateway_id = aws_internet_gateway.main.id
  }

  tags = {
    Name        = "${var.environment}-public-rt"
    Environment = var.environment
  }
}

# Private Route Table
resource "aws_route_table" "private" {
  vpc_id = aws_vpc.main.id

  route {
    cidr_block     = "0.0.0.0/0"
    nat_gateway_id = aws_nat_gateway.main.id
  }

  tags = {
    Name        = "${var.environment}-private-rt"
    Environment = var.environment
  }
}

# Route Table Associations
resource "aws_route_table_association" "public" {
  count          = length(var.availability_zones)
  subnet_id      = aws_subnet.public[count.index].id
  route_table_id = aws_route_table.public.id
}

resource "aws_route_table_association" "private" {
  count          = length(var.availability_zones)
  subnet_id      = aws_subnet.private[count.index].id
  route_table_id = aws_route_table.private.id
}

variables.tf

variable "vpc_cidr" {
  description = "CIDR block for VPC"
  type        = string
}

variable "availability_zones" {
  description = "Availability zones"
  type        = list(string)
}

variable "environment" {
  description = "Environment name"
  type        = string
}

Outputs.tf


output "vpc_id" {
  value = aws_vpc.main.id
}

output "public_subnet_ids" {
  value = aws_subnet.public[*].id
}

output "private_subnet_ids" {
  value = aws_subnet.private[*].id
}

**_

III. An EKS module (modules/eks) that creates:

_**
EKS cluster
Node group with autoscaling
Required IAM roles and policies

main.tf

resource "aws_eks_cluster" "main" {
  name     = "${var.environment}-eks"
  role_arn = aws_iam_role.eks_cluster.arn
  version  = var.kubernetes_version

  vpc_config {
    subnet_ids              = var.subnet_ids
    endpoint_private_access = true
    endpoint_public_access  = true
  }

  depends_on = [
    aws_iam_role_policy_attachment.eks_cluster_policy,
    aws_iam_role_policy_attachment.eks_service_policy,
  ]
}

# EKS Cluster IAM Role
resource "aws_iam_role" "eks_cluster" {
  name = "${var.environment}-eks-cluster"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Action = "sts:AssumeRole"
        Effect = "Allow"
        Principal = {
          Service = "eks.amazonaws.com"
        }
      }
    ]
  })
}

resource "aws_iam_role_policy_attachment" "eks_cluster_policy" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKSClusterPolicy"
  role       = aws_iam_role.eks_cluster.name
}

resource "aws_iam_role_policy_attachment" "eks_service_policy" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKSServicePolicy"
  role       = aws_iam_role.eks_cluster.name
}

# Node Group
resource "aws_eks_node_group" "main" {
  cluster_name    = aws_eks_cluster.main.name
  node_group_name = "${var.environment}-node-group"
  node_role_arn   = aws_iam_role.eks_node_group.arn
  subnet_ids      = var.subnet_ids

  scaling_config {
    desired_size = var.node_desired_size
    max_size     = var.node_max_size
    min_size     = var.node_min_size
  }

  instance_types = var.node_instance_types

  depends_on = [
    aws_iam_role_policy_attachment.eks_node_policy,
    aws_iam_role_policy_attachment.eks_cni_policy,
    aws_iam_role_policy_attachment.eks_container_registry,
  ]
}

# Node Group IAM Role
resource "aws_iam_role" "eks_node_group" {
  name = "${var.environment}-eks-node-group"

  assume_role_policy = jsonencode({
    Version = "2012-10-17"
    Statement = [
      {
        Action = "sts:AssumeRole"
        Effect = "Allow"
        Principal = {
          Service = "ec2.amazonaws.com"
        }
      }
    ]
  })
}

resource "aws_iam_role_policy_attachment" "eks_node_policy" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKSWorkerNodePolicy"
  role       = aws_iam_role.eks_node_group.name
}

resource "aws_iam_role_policy_attachment" "eks_cni_policy" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEKS_CNI_Policy"
  role       = aws_iam_role.eks_node_group.name
}

resource "aws_iam_role_policy_attachment" "eks_container_registry" {
  policy_arn = "arn:aws:iam::aws:policy/AmazonEC2ContainerRegistryReadOnly"
  role       = aws_iam_role.eks_node_group.name
}

variables.tf


variable "environment" {
  description = "Environment name"
  type        = string
}

variable "subnet_ids" {
  description = "Subnet IDs for EKS cluster"
  type        = list(string)
}

variable "kubernetes_version" {
  description = "Kubernetes version"
  type        = string
  default     = "1.27"
}

variable "node_desired_size" {
  description = "Desired number of worker nodes"
  type        = number
  default     = 2
}

variable "node_max_size" {
  description = "Maximum number of worker nodes"
  type        = number
  default     = 4
}

variable "node_min_size" {
  description = "Minimum number of worker nodes"
  type        = number
  default     = 1
}

variable "node_instance_types" {
  description = "Instance types for worker nodes"
  type        = list(string)
  default     = ["t3.medium"]
}

Outputs.tf


output "cluster_endpoint" {
  value = aws_eks_cluster.main.endpoint
}

output "cluster_name" {
  value = aws_eks_cluster.main.name
}

output "cluster_certificate_authority_data" {
  value = aws_eks_cluster.main.certificate_authority[0].data
}

To use this configuration:
Initialize Terraform:

terraform init

Review the plan:

terraform plan
Apply the configuration:

terraform apply

The configuration uses variables with sensible defaults but can be customized through the terraform.tfvars file.

Key features:

Highly available setup across 3 AZs
Private subnets for EKS nodes
Public subnets for load balancers
Proper IAM roles and policies
Autoscaling node group
Proper tagging for EKS and AWS Load Balancer Controller
https://github.com/Consultantsrihari/Create-a-Terraform-project-which-Implements-a-VPC-on-AWS-and-deploys-to-EKS-cluster..git

You’re welcome! Have a great time ahead! Enjoy your day!

Please Connect with me any doubts.

Mail: sriharimalapati6@gmail.com

LinkedIn: www.linkedin.com/in/

GitHub: https://github.com/Consultantsrihari

Medium: Sriharimalapati — Medium

Thanks for watching ##### %%%% Sri Hari %%%%

Automated Node.js Deployment to AWS EC2 with Docker and GitHub Actions

VENKATA SRI HARI — Sun, 28 Jul 2024 15:00:03 +0000

In this project, we will learn how to deploy a Node.js application using GitHub Actions, Docker, and AWS EC2. This fully automated pipeline ensures that our application on the EC2 instance is always up-to-date with the latest changes.

GitHub Actions is a CI/CD platform designed to automate your build, test, and deployment processes. By configuring a .yml file, you can specify tasks to run in response to events like pull requests, issues, or commits.

In this guide, we will utilize GitHub Actions to build a Docker image and push it to Docker Hub. An AWS EC2 instance will serve as a self-hosted runner. The instance will then pull and run the Docker container, allowing us to access the application via its public IP.

Github link: NodeJS-Deployment

let’s create a Dockerfile in the project’s root directory.

You can use this same docker file in your project also….

Now I can create a .dockerignore file.

# dockerignore file 
node_modules/
package-lock.json

Log in to GitHub account and create a new repository and{My Repository Name is NodeJS-Deployment, in you case you can choose different name}.

click on create Repository.

Push your existing code to the GitHub repository.

Give this below cmds

git init
git add .
git commit -m "Initial commit"
git branch -M main
git remote add origin https://github.com/Consultantsrihari/NodeJS-Deployment
git push -u origin main

Now go toActions tab and select Docker image and click on Configure, then you can write the docker .yml file.

In your case write own docker image or you can use below code.

# Name of the GitHub Actions workflow
name: CI/CD for Node.js REST-API

# Trigger this workflow on push events to the main branch and manual dispatch
on:
  push: 
    branches: [main]
  workflow_dispatch:

# Permissions needed for this workflow
permissions:
  contents: write

# Define the jobs to be executed
jobs:
  # Build job
  Build:
    # Use the latest Ubuntu runner
    runs-on: ubuntu-latest

    # Steps to be executed in the Build job
    steps:
      # Checkout the repository
      - name: Checkout repository
        uses: actions/checkout@v3 

      # Login to DockerHub using secrets for credentials
      - name: Login to DockerHub
        env:
          DOCKER_USERNAME: ${{ secrets.DOCKER_USERNAME }}
          DOCKER_PASSWORD: ${{ secrets.DOCKER_PASSWORD }}
        run: echo $DOCKER_PASSWORD | docker login -u $DOCKER_USERNAME --password-stdin

      # Build Docker image with a specific tag
      - name: Build Docker Image
        run: docker build -t bishal5438/rest-api .

      # Push the built image to DockerHub
      - name: Push to DockerHub
        run: docker push bishal5438/rest-api:latest 

  # Deploy job
  Deploy:
    # Use a self-hosted runner for deployment
    runs-on: self-hosted

    # Steps to be executed in the Deploy job
    steps:
      # Pull the latest Docker image from DockerHub
      - name: Pull the Docker Image
        run: docker pull bishal5438/rest-api:latest 

      # Delete the old container if it exists
      - name: Delete Old Container
        run: |
          if [ "$(docker ps -q -f name=rest-api-Container)" ]; then
            sudo docker rm -f rest-api-Container
          fi

      # Run a new container from the pulled image
      - name: Run the Container
        run: docker run -d -p 80:80 --name rest-api-Container bishal5438/rest-api

Now, commit the changes, which will create a CI-CD docker-image.yml file inside .github/workflows

Now, let’s configure GitHub secrets to store sensitive information such as Docker Hub’s username and password.

For this, go to the repositorySettings

Click onSecrets and variables and select Actions.

To add a new secret, click on New repository secret.

Add your Docker Hub username and password.

Let’s configure AWS EC2.

Go to the AWS console and select EC2 service.
Provide an instance name NodeJS-Deployment
For the AMI option, select Ubuntu.
Select an instance type based on your requirements. I am usingt2.micro instance.
For the Key pair option, click on Create new key pair.
Provide a key pair name and select RSA as key pair type.
Select .pem as private key file format and click on Create key pair.
Save the key securely as we need it to connect to our instance.
Under Network Settings, click on Create security group.
Then, allow http, https and ssh traffic.
Select the required volume of storage.
Finally click on Launch Instance.
Let’s configure AWS EC2.
Go to the AWS console and select EC2 service.
Provide an instance name NodeJS-Deployment
For the AMI option, select Ubuntu.
Select an instance type based on your requirements. I am usingt2.micro instance.
For the Key pair option, click on Create new key pair.
Provide a key pair name and select RSA as key pair type.
Select .pem as private key file format and click on Create key pair.
Save the key securely as we need it to connect to our instance.
Under Network Settings, click on Create security group.
Then, allow http, https and ssh traffic.
Select the required volume of storage.
Finally click on Launch Instance.

Take the public IP of your instance and login.
Iam using Git bash, in your case use your own terminal.
Now succuesfully login our instance.

To configure and run docker as a non-root user, use the following commands.

sudo apt update 

sudo apt install docker.io -y 

#Add group named docker
sudo groupadd docker

#Add user to the docker group 
sudo usermod -aG docker $USER

#Reload group permissions  
newgrp docker

#Managing permission 
sudo chown -R $USER:docker /var/run/docker
sudo chown $USER:docker /var/run/docker.sock

#Restart docker service 
sudo systemctl restart docker 

#Auto start on boot 
 sudo systemctl enable docker

#To check docker status
 sudo systemctl status docker

Now, let’s set up a self-hosted runner for our GitHub repository.

Go to repository Settings.

Click on Actions and select Runners.

Next, click on New self-hosted runner.

Select Linux as our EC2 instance is based on Ubuntu.

GitHub will provide us the instructions to set up the runner.

Copy past all cmds in ubuntu server.

Download

# Create a folder
$ mkdir actions-runner && cd actions-runnerCopied!
# Download the latest runner package
$ curl -o actions-runner-linux-x64-2.317.0.tar.gz -L https://github.com/actions/runner/releases/download/v2.317.0/actions-runner-linux-x64-2.317.0.tar.gzCopied!
# Optional: Validate the hash
$ echo "9e883d210df8c6028aff475475a457d380353f9d01877d51cc01a17b2a91161d  actions-runner-linux-x64-2.317.0.tar.gz" | shasum -a 256 -cCopied!
# Extract the installer
$ tar xzf ./actions-runner-linux-x64-2.317.0.tar.gzCopied!

Configure

# Create the runner and start the configuration experience
$ ./config.sh --url https://github.com/Consultantsrihari/NodeJS-Deployment --token BGHFMQ3HYQU6CX2VVFG73E3GUUQJKCopied!
# Last step, run it!
$ ./run.shCopied! 

Using your self-hosted runner
# Use this YAML in your workflow file for each job
runs-on: self-hosted

If prompted for any inputs, you may press Enter for default values.

Once the setup is completed, we can see active machines under Runners tab on our github repository.

Now, pull the changes to the remote repo to test CI-CD pipeline, We can see successful and failed jobs under the same action tab.

Finally, we can access our application by visiting http://.

Conclusion
Hence, By integrating Github Actions with Docker and AWS, we can automate CI-CD steps and ensure a smooth and reliable workflow for Node.js application.

You’re welcome! Have a great time ahead! Enjoy your day!

Please Connect with me any doubts.

LinkedIn: www.linkedin.com/in/

Mail: sriharimalapati6@gmail.com

GitHub: https://github.com/Consultantsrihari

Thanks for watching ##### %%%% Sri Hari %%%%

Github Actions
Github
Docker
Devops Project
DevOps