A Sample Structure for Jenkins CI-CD: Stages&Tools

I'll design a comprehensive CI/CD pipeline for a three-tier microservice architecture using the specified tools.

Architecture Overview

Three-Tier Microservice Architecture:

• Presentation Tier: Web frontend, API gateways
• Application Tier: Business logic microservices
• Data Tier: Databases, caching, storage

CI/CD Pipeline Diagram

Developer → Git Repository (GitHub/GitLab/Bitbucket)
    │
    ▼
Jenkins Pipeline (Declarative Pipeline)
    │
    ├── Stage 1: Code Checkout & Dependency Resolution
    │   ├── Git checkout
    │   └── Maven dependency resolution
    │
    ├── Stage 2: Build & Unit Tests
    │   ├── Maven compile
    │   ├── Unit testing
    │   └── Code coverage
    │
    ├── Stage 3: Static Code Analysis
    │   ├── SonarQube analysis
    │   └── Quality gate check
    │
    ├── Stage 4: Build Docker Image
    │   ├── Docker build
    │   ├── Image tagging
    │   └── Security scanning
    │
    ├── Stage 5: Push to Nexus Repository
    │   ├── Push Docker image
    │   └── Push application artifacts
    │
    ├── Stage 6: Deployment to Staging
    │   ├── Kubernetes deployment
    │   ├── Service configuration
    │   └── Ingress setup
    │
    ├── Stage 7: Integration Testing
    │   ├── API testing
    │   ├── Load testing
    │   └── Performance testing
    │
    ├── Stage 8: Security Scanning
    │   ├── Vulnerability scanning
    │   └── Compliance checks
    │
    ├── Stage 9: Production Deployment (Blue-Green/Canary)
    │   ├── Kubernetes rollout
    │   ├── Service mesh configuration
    │   └── Traffic shifting
    │
    └── Stage 10: Monitoring & Observability
        ├── Prometheus metrics
        ├── Grafana dashboards
        ├── CloudWatch logs
        ├── ELK stack logging
        └── Slack notifications

Jenkins Pipeline Configuration

Jenkinsfile (Declarative Pipeline)

pipeline {
    agent any
    tools {
        maven 'M3'
        jdk 'JDK11'
    }
    environment {
        REGISTRY = "nexus.example.com"
        DOCKER_CREDENTIALS = credentials('docker-nexus-creds')
        KUBECONFIG = credentials('kubeconfig')
        SONARQUBE_SCANNER = tool 'SonarQubeScanner'
        SLACK_CHANNEL = '#ci-cd-notifications'
    }
    stages {
        stage('Checkout') {
            steps {
                git branch: 'main', 
                    url: 'https://github.com/your-org/your-microservice.git',
                    credentialsId: 'github-creds'
            }
        }

        stage('Build & Unit Tests') {
            steps {
                sh 'mvn clean compile test'
                junit '**/target/surefire-reports/*.xml'
            }
            post {
                always {
                    archiveArtifacts artifacts: '**/target/*.jar', fingerprint: true
                }
            }
        }

        stage('SonarQube Analysis') {
            steps {
                withSonarQubeEnv('sonarqube-server') {
                    sh 'mvn sonar:sonar -Dsonar.projectKey=my-microservice'
                }
            }
        }

        stage('Quality Gate') {
            steps {
                timeout(time: 1, unit: 'HOURS') {
                    waitForQualityGate abortPipeline: true
                }
            }
        }

        stage('Build Docker Image') {
            steps {
                script {
                    def dockerImage = docker.build("${REGISTRY}/my-microservice:${env.BUILD_ID}")
                }
            }
        }

        stage('Push to Nexus') {
            steps {
                script {
                    docker.withRegistry("https://${REGISTRY}", 'docker-nexus-creds') {
                        docker.image("${REGISTRY}/my-microservice:${env.BUILD_ID}").push()
                    }
                }
            }
        }

        stage('Deploy to Staging') {
            steps {
                script {
                    def k8sNamespace = 'staging'
                    sh "kubectl config use-context staging-cluster"
                    sh "kubectl apply -f k8s/deployment.yaml -n ${k8sNamespace}"
                    sh "kubectl apply -f k8s/service.yaml -n ${k8sNamespace}"
                    sh "kubectl apply -f k8s/ingress.yaml -n ${k8sNamespace}"
                }
            }
        }

        stage('Integration Tests') {
            steps {
                sh 'mvn verify -DskipUnitTests'
                // Additional integration test commands
            }
        }

        stage('Deploy to Production') {
            when {
                branch 'main'
            }
            steps {
                script {
                    sh "kubectl config use-context production-cluster"
                    // Blue-green deployment strategy
                    sh "kubectl apply -f k8s/production-deployment.yaml"
                    // Canary deployment script
                    sh "./scripts/canary-deployment.sh"
                }
            }
        }
    }
    post {
        success {
            slackSend channel: "${SLACK_CHANNEL}",
                     message: "SUCCESS: Job ${env.JOB_NAME} build ${env.BUILD_NUMBER}"
            // Update Grafana dashboard
            sh './scripts/update-deployment-metrics.sh success'
        }
        failure {
            slackSend channel: "${SLACK_CHANNEL}",
                     message: "FAILED: Job ${env.JOB_NAME} build ${env.BUILD_NUMBER}"
            // Send detailed logs to ELK
            sh './scripts/send-failure-logs.sh'
        }
        always {
            // Cleanup resources
            sh './scripts/cleanup-resources.sh'
        }
    }
}

Kubernetes Deployment Configuration

deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: my-microservice
  labels:
    app: my-microservice
spec:
  replicas: 3
  selector:
    matchLabels:
      app: my-microservice
  template:
    metadata:
      labels:
        app: my-microservice
      annotations:
        prometheus.io/scrape: "true"
        prometheus.io/port: "8080"
    spec:
      containers:
      - name: my-microservice
        image: nexus.example.com/my-microservice:latest
        ports:
        - containerPort: 8080
        env:
        - name: SPRING_PROFILES_ACTIVE
          value: "production"
        resources:
          requests:
            memory: "256Mi"
            cpu: "250m"
          limits:
            memory: "512Mi"
            cpu: "500m"
        livenessProbe:
          httpGet:
            path: /actuator/health
            port: 8080
          initialDelaySeconds: 30
          periodSeconds: 10
        readinessProbe:
          httpGet:
            path: /actuator/health
            port: 8080
          initialDelaySeconds: 5
          periodSeconds: 5
---
apiVersion: v1
kind: Service
metadata:
  name: my-microservice
spec:
  selector:
    app: my-microservice
  ports:
  - port: 80
    targetPort: 8080
  type: ClusterIP

Monitoring and Observability Setup

Prometheus Configuration (prometheus.yml)

global:
  scrape_interval: 15s

scrape_configs:
  - job_name: 'kubernetes-pods'
    kubernetes_sinks:
      - role: pod
    relabel_configs:
      - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_scrape]
        action: keep
        regex: true
      - source_labels: [__meta_kubernetes_pod_annotation_prometheus_io_path]
        action: replace
        target_label: __metrics_path__
        regex: (.+)
      - source_labels: [__address__, __meta_kubernetes_pod_annotation_prometheus_io_port]
        action: replace
        regex: ([^:]+)(?::\d+)?;(\d+)
        replacement: $1:$2
        target_label: __address__

  - job_name: 'kubernetes-services'
    kubernetes_sinks:
      - role: service
    metrics_path: /metrics
    relabel_configs:
      - source_labels: [__meta_kubernetes_service_annotation_prometheus_io_scrape]
        action: keep
        regex: true

Grafana Dashboard Configuration

JSON dashboard for microservice monitoring including:

• CPU/Memory usage
• Request rates and error rates
• Response times
• Database connection metrics
• Custom business metrics

CloudWatch Logs Configuration

{
  "awslogs": {
    "group": "/ecs/my-microservice",
    "streamPrefix": "ecs"
  }
}

ELK Stack Configuration

• Filebeat for log collection
• Logstash pipelines for log processing
• Elasticsearch indices for different log types
• Kibana dashboards for log analysis

Slack Integration Configuration

Jenkins Slack Notification

// In Jenkins configuration
slackSend channel: '#ci-cd-notifications',
          color: currentBuild.currentResult == 'SUCCESS' ? 'good' : 'danger',
          message: """
          *${env.JOB_NAME}* - #${env.BUILD_NUMBER} - ${currentBuild.currentResult}
          Branch: ${env.GIT_BRANCH}
          Commit: ${env.GIT_COMMIT}
          Duration: ${currentBuild.durationString}
          Build: ${env.BUILD_URL}
          """

Security and Best Practices

1. Secrets Management: Use HashiCorp Vault or AWS Secrets Manager
2. Network Policies: Kubernetes network policies for microservice communication
3. RBAC: Role-based access control in Kubernetes
4. Image Scanning: Trivy or Clair for vulnerability scanning
5. Infrastructure as Code: Terraform for AWS resource provisioning

Pipeline Optimization

1. Parallel Execution: Run tests and builds in parallel
2. Caching: Maven dependency caching, Docker layer caching
3. Test Optimization: Selective testing based on changes
4. Resource Optimization: Right-sized Kubernetes resources
5. Rollback Strategy: Automated rollback on deployment failures

This complete CI/CD pipeline ensures automated testing, quality checks, secure deployments, and comprehensive monitoring for your three-tier microservice architecture.