AWS DevOps Project: Advanced Automated CI/CD Pipeline

This project demonstrates how to set up an advanced automated CI/CD pipeline with Infrastructure as Code (IaC), microservices, a service mesh, and monitoring using AWS. Tools like Terraform, Jenkins, Kubernetes (EKS), Istio, Prometheus, Grafana, and ArgoCD are utilized.

Github: https://github.com/Consultantsrihari/AWS-DevOps-Project-Advanced-Automated-CI-CD-Pipeline.git

Architecture Overview

• Infrastructure as Code: AWS resources are provisioned using Terraform.
• Containerized Microservices: Deployed on EKS.
• Service Mesh: Managed by Istio for advanced traffic control and observability.
• CI/CD Pipeline: Managed with Jenkins and ArgoCD.
• Monitoring: Implemented using Prometheus and Grafana.

---

Pre-requisites

1. AWS Account.
2. Terraform installed locally.
3. Kubectl and Helm installed.
4. Jenkins installed and configured.
5. Docker installed.
6. Prometheus and Grafana configured for monitoring.

---

Step 1: Provision Infrastructure with Terraform

Terraform Configuration (main.tf)

provider "aws" {
  region = "us-east-1"
}

resource "aws_vpc" "dev_vpc" {
  cidr_block           = "10.0.0.0/16"
  enable_dns_support   = true
  enable_dns_hostnames = true

  tags = {
    Name = "dev-vpc"
  }
}

resource "aws_subnet" "public_subnet" {
  vpc_id                  = aws_vpc.dev_vpc.id
  cidr_block              = "10.0.1.0/24"
  map_public_ip_on_launch = true

  tags = {
    Name = "public-subnet"
  }
}

resource "aws_eks_cluster" "eks_cluster" {
  name     = "dev-cluster"
  role_arn = aws_iam_role.eks_role.arn

  vpc_config {
    subnet_ids = [aws_subnet.public_subnet.id]
  }
}

Run the following commands:

terraform init
terraform apply -auto-approve

---

Step 2: Deploy Jenkins for CI/CD

Helm Chart for Jenkins

helm repo add jenkinsci https://charts.jenkins.io
helm repo update
helm install jenkins jenkinsci/jenkins

Access Jenkins UI and install required plugins (e.g., Docker, Kubernetes, Git, Pipeline).

---

Step 3: Deploy Kubernetes Cluster and Microservices

Sample Deployment YAML (microservice-deployment.yaml)

apiVersion: apps/v1
kind: Deployment
metadata:
  name: sample-microservice
spec:
  replicas: 3
  selector:
    matchLabels:
      app: sample
  template:
    metadata:
      labels:
        app: sample
    spec:
      containers:
      - name: sample-app
        image: myregistry/sample-app:latest
        ports:
        - containerPort: 8080

Apply the configuration:

kubectl apply -f microservice-deployment.yaml

---

Step 4: Configure Istio Service Mesh

Install Istio

istioctl install --set profile=demo -y
kubectl label namespace default istio-injection=enabled

Sample VirtualService Configuration

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: sample-app
spec:
  hosts:
  - "*"
  gateways:
  - sample-gateway
  http:
  - route:
    - destination:
        host: sample-microservice
        port:
          number: 8080

Apply the configuration:

kubectl apply -f virtualservice.yaml

---

Step 5: Integrate Monitoring with Prometheus and Grafana

Install Prometheus and Grafana

helm install prometheus prometheus-community/prometheus
helm install grafana grafana/grafana

Access Grafana Dashboard and connect to Prometheus as a data source.

---

Step 6: Jenkins Pipeline Configuration

Jenkinsfile

pipeline {
  agent any
  stages {
    stage('Checkout Code') {
      steps {
        checkout scm
      }
    }
    stage('Build Docker Image') {
      steps {
        sh 'docker build -t myregistry/sample-app:latest .'
      }
    }
    stage('Push to Registry') {
      steps {
        sh 'docker push myregistry/sample-app:latest'
      }
    }
    stage('Deploy to Kubernetes') {
      steps {
        sh 'kubectl apply -f microservice-deployment.yaml'
      }
    }
  }
}

---

Step 7: Continuous Delivery with ArgoCD

Install ArgoCD

kubectl create namespace argocd
kubectl apply -n argocd -f https://raw.githubusercontent.com/argoproj/argo-cd/stable/manifests/install.yaml

The kubectl apply command is used to install ArgoCD in a dedicated namespace (argocd). ArgoCD manages the deployment and synchronization of applications in Kubernetes clusters using GitOps principles. Once installed, it acts as a continuous delivery tool that monitors a Git repository for changes and automatically applies them to the cluster.

ArgoCD Application YAML

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: sample-app
spec:
  destination:
    namespace: default
    server: https://kubernetes.default.svc
  source:
    path: sample-app
    repoURL: https://github.com/your-repo/sample-app.git
    targetRevision: HEAD

Apply the configuration:

kubectl apply -f argocd-application.yaml

---

Summary

• Infrastructure: Provisioned with Terraform.
• CI/CD: Managed by Jenkins and ArgoCD.
• Service Mesh: Configured with Istio.
• Monitoring: Handled by Prometheus and Grafana.

This setup ensures a robust and automated DevOps workflow, promoting scalability, observability, and reliability.