DEV Community

Cover image for Netflix Clone DevSecOps: CI/CD Pipeline with Jenkins, Docker, Kubernetes & Security
Anilkumar N
Anilkumar N

Posted on

Netflix Clone DevSecOps: CI/CD Pipeline with Jenkins, Docker, Kubernetes & Security

#devops #kubernetes #docker #devchallenge

In this guide, we will walk through the end-to-end deployment of a Netflix clone using a modern DevOps pipeline. We’ll leverage Jenkins for CI/CD automation, containerize the application with Docker, and orchestrate the deployment on a Kubernetes cluster. To ensure system reliability, we will implement a robust monitoring stack using Prometheus, Grafana, and Node Exporter to track real-time metrics across Jenkins and our K8s nodes.

CLICK HERE FOR GITHUB REPOSITORY :

Steps:-

Step 1 — Launch an Ubuntu(22.04) m7i-flex.large Instance

Step 2 — Install Jenkins, Docker and Trivy. Create a Sonarqube Container using Docker.

Step 3 — Create a TMDB API Key.

Step 4 — Install Prometheus and Grafana On the new Server.

Step 5 — Install the Prometheus Plugin and Integrate it with the Prometheus server.

Step 6 — Email Integration With Jenkins and Plugin setup.

Step 7 — Install Plugins like JDK, Sonarqube Scanner, Nodejs, and OWASP Dependency Check.

Step 8 — Create a Pipeline Project in Jenkins using a Declarative Pipeline

Step 9 — Install OWASP Dependency Check Plugins

Step 10 — Docker Image Build and Push

Step 11 — Deploy the image using Docker

Step 12 — Kubernetes master and slave setup on Ubuntu (20.04)

Step 13 — Access the Netflix app on the Browser.

Step 14 — Terminate the AWS EC2 Instances.

Now, let’s get started and dig deeper into each of these steps:-
Start by launching a m7i-flex.large instance on AWS using the Ubuntu AMI. You have the choice of generating a new key pair or selecting an existing one. In the Security Group configuration, ensure that HTTP and HTTPS traffic is enabled. Additionally, for experimental purposes, open all ports—though keep in mind this is only for learning and isn't a recommended security practice.

image

Step 2 — Install Jenkins, Docker and Trivy

Connect to your console, and enter these commands to Install Jenkins

vi jenkins.sh #make sure run in Root (or) add at userdata while ec2 launch
Enter fullscreen mode Exit fullscreen mode 
#!/bin/bash
#!/bin/bash
sudo apt update
sudo apt install fontconfig openjdk-21-jre -y
java -version

sudo wget -O /etc/apt/keyrings/jenkins-keyring.asc \
  https://pkg.jenkins.io/debian-stable/jenkins.io-2026.key
echo "deb [signed-by=/etc/apt/keyrings/jenkins-keyring.asc]" \
  https://pkg.jenkins.io/debian-stable binary/ | sudo tee \
  /etc/apt/sources.list.d/jenkins.list > /dev/null
sudo apt update
sudo apt install jenkins -y

sudo systemctl enable jenkins
sudo systemctl start jenkins
sudo systemctl status jenkins
Enter fullscreen mode Exit fullscreen mode 
sudo chmod 777 jenkins.sh
./jenkins.sh    # this will installl jenkins
Enter fullscreen mode Exit fullscreen mode

Once Jenkins is installed, you will need to go to your AWS EC2 Security Group and open Inbound Port 8080, since Jenkins works on Port 8080

<EC2 Public IP Address:8080>
sudo cat /var/lib/jenkins/secrets/initialAdminPassword
Enter fullscreen mode Exit fullscreen mode

image

Unlock Jenkins using an administrative password and install the suggested plugins.

img

Jenkins will now get installed and install all the libraries.

img

Create a user click on save and continue.

Jenkins Getting Started Screen.

image

2B — Install Docker 

sudo apt update
sudo apt install ca-certificates curl gnupg -y
sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
sudo chmod a+r /etc/apt/keyrings/docker.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt update
apt-cache policy docker-ce
sudo apt install docker-ce -y 
sudo systemctl status docker
Enter fullscreen mode Exit fullscreen mode

After the docker installation, we create a sonarqube container (Remember to add 9000 ports in the security group).
Now our sonarqube is up and running

docker run -d --name sonar -p 9000:9000 sonarqube:lts-community
Enter fullscreen mode Exit fullscreen mode

image

Now our sonarqube is up and running

image

Enter username and password, click on login and change password

username admin
password admin
Enter fullscreen mode Exit fullscreen mode

image

Update New password, This is Sonar Dashboard.

image

2C — Install Trivy 

vi trivy.sh
Enter fullscreen mode Exit fullscreen mode 
sudo apt-get install wget apt-transport-https gnupg lsb-release -y
wget -qO - https://aquasecurity.github.io/trivy-repo/deb/public.key | gpg --dearmor | sudo tee /usr/share/keyrings/trivy.gpg > /dev/null
echo "deb [signed-by=/usr/share/keyrings/trivy.gpg] https://aquasecurity.github.io/trivy-repo/deb $(lsb_release -sc) main" | sudo tee -a /etc/apt/sources.list.d/trivy.list
sudo apt-get update
sudo apt-get install trivy -y
Enter fullscreen mode Exit fullscreen mode

Step 3: Create a TMDB API Key

Next, we will create a TMDB API key

Open a new tab in the Browser and search for TMDB

image

Click on the first result, you will see this page.
image

Click on the Login on the top right. You will get this page.

You need to create an account here. click on click here. I have account that’s why i added my details there.

image

once you create an account you will see this page.

image

Let’s create an API key, By clicking on your profile and clicking settings.

fs

Now click on API from the left side panel.

dfed

Now click on create

swf

Provide basic details

image

SDA

Click on submit and you will get your API key.

SDVDS

Step 4 — Install Prometheus and Grafana On the new Server

First of all, let’s create a dedicated Linux user sometimes called a system account for Prometheus. Having individual users for each service serves two main purposes:

It is a security measure to reduce the impact in case of an incident with the service.

It simplifies administration as it becomes easier to track down what resources belong to which service.

To create a system user or system account, run the following command:

sudo useradd \
    --system \
    --no-create-home \
    --shell /bin/false prometheus
Enter fullscreen mode Exit fullscreen mode

image

–system – Will create a system account.
–no-create-home – We don’t need a home directory for Prometheus or any other system accounts in our case.
–shell /bin/false – It prevents logging in as a Prometheus user.
Prometheus – Will create a Prometheus user and a group with the same name.

Let’s check the latest version of Prometheus from the download page.

You can use the curl or wget command to download Prometheus.

wget https://github.com/prometheus/prometheus/releases/download/v2.47.1/prometheus-2.47.1.linux-amd64.tar.gz
Enter fullscreen mode Exit fullscreen mode

image

tar -xvf prometheus-2.47.1.linux-amd64.tar.gz
Enter fullscreen mode Exit fullscreen mode

image

Usually, you would have a disk mounted to the data directory. For this tutorial, I will simply create a /data directory. Also, you need a folder for Prometheus configuration files.

sudo mkdir -p /data /etc/prometheus
Enter fullscreen mode Exit fullscreen mode

image

Now, let’s change the directory to Prometheus and move some files.

cd prometheus-2.47.1.linux-amd64/
Enter fullscreen mode Exit fullscreen mode

image

First of all, let’s move the Prometheus binary and a promtool to the /usr/local/bin/. promtool is used to check configuration files and Prometheus rules.

sudo mv prometheus promtool /usr/local/bin/
Enter fullscreen mode Exit fullscreen mode

image

Optionally, we can move console libraries to the Prometheus configuration directory. Console templates allow for the creation of arbitrary consoles using the Go templating language. You don’t need to worry about it if you’re just getting started.

sudo mv consoles/ console_libraries/ /etc/prometheus/
Enter fullscreen mode Exit fullscreen mode

image

To avoid permission issues, you need to set the correct ownership for the /etc/prometheus/ and data directory.

image

cd
rm -rf prometheus-2.47.1.linux-amd64.tar.gz
Enter fullscreen mode Exit fullscreen mode

image

prometheus --help
Enter fullscreen mode Exit fullscreen mode

We’re going to use some of these options in the service definition.

We’re going to use Systemd, which is a system and service manager for Linux operating systems. For that, we need to create a Systemd unit configuration file.

Prometheus.service

[Unit]
Description=Prometheus
Wants=network-online.target
After=network-online.target
StartLimitIntervalSec=500
StartLimitBurst=5
[Service]
User=prometheus
Group=prometheus
Type=simple
Restart=on-failure
RestartSec=5s
ExecStart=/usr/local/bin/prometheus \
  --config.file=/etc/prometheus/prometheus.yml \
  --storage.tsdb.path=/data \
  --web.console.templates=/etc/prometheus/consoles \
  --web.console.libraries=/etc/prometheus/console_libraries \
  --web.listen-address=0.0.0.0:9090 \
  --web.enable-lifecycle
[Install]
WantedBy=multi-user.target
Enter fullscreen mode Exit fullscreen mode

****image

Let’s go over a few of the most important options related to Systemd and Prometheus. Restart – Configures whether the service shall be restarted when the service process exits, is killed, or a timeout is reached.
RestartSec – Configures the time to sleep before restarting a service.
User and Group – Are Linux user and a group to start a Prometheus process.
–config.file=/etc/prometheus/prometheus.yml – Path to the main Prometheus configuration file.
–storage.tsdb.path=/data – Location to store Prometheus data.
–web.listen-address=0.0.0.0:9090 – Configure to listen on all network interfaces. In some situations, you may have a proxy such as nginx to redirect requests to Prometheus. In that case, you would configure Prometheus to listen only on localhost.
–web.enable-lifecycle — Allows to manage Prometheus, for example, to reload configuration without restarting the service.

To automatically start the Prometheus after reboot, run enable.

sudo systemctl enable prometheus
Enter fullscreen mode Exit fullscreen mode

image

Then just start the Prometheus.

sudo systemctl start prometheus
Enter fullscreen mode Exit fullscreen mode

image

sudo systemctl status prometheus
Enter fullscreen mode Exit fullscreen mode

image

Suppose you encounter any issues with Prometheus or are unable to start it. The easiest way to find the problem is to use the journalctl command and search for errors.

journalctl -u prometheus -f --no-pager
Enter fullscreen mode Exit fullscreen mode

Now we can try to access it via the browser. I’m going to be using the IP address of the Ubuntu server. You need to append port 9090 to the IP.

image

If you go to targets, you should see only one – Prometheus target. It scrapes itself every 15 seconds by default.

Install Node Exporter on Ubuntu 22.04
Next, we’re going to set up and configure Node Exporter to collect Linux system metrics like CPU load and disk I/O. Node Exporter will expose these as Prometheus-style metrics. Since the installation process is very similar, I’m not going to cover as deep as Prometheus.

First, let’s create a system user for Node Exporter by running the following command:

sudo useradd \
    --system \
    --no-create-home \
    --shell /bin/false node_exporter
Enter fullscreen mode Exit fullscreen mode

image

You can download Node Exporter from the same page.

Use the wget command to download the binary.

wget https://github.com/prometheus/node_exporter/releases/download/v1.6.1/node_exporter-1.6.1.linux-amd64.tar.gz
Enter fullscreen mode Exit fullscreen mode

image

Extract the node exporter from the archive.

tar -xvf node_exporter-1.6.1.linux-amd64.tar.gz
Enter fullscreen mode Exit fullscreen mode

Move binary to the /usr/local/bin.

sudo mv \
  node_exporter-1.6.1.linux-amd64/node_exporter \
  /usr/local/bin/
Enter fullscreen mode Exit fullscreen mode

image

Clean up, and delete node_exporter archive and a folder.

rm -rf node_exporter*
Enter fullscreen mode Exit fullscreen mode

Verify that you can run the binary.

node_exporter --version
Enter fullscreen mode Exit fullscreen mode

image

Node Exporter has a lot of plugins that we can enable. If you run Node Exporter help you will get all the options.

node_exporter --help
Enter fullscreen mode Exit fullscreen mode

–collector.logind We’re going to enable the login controller, just for the demo.

Next, create a similar systemd unit file.

sudo vim /etc/systemd/system/node_exporter.service
Enter fullscreen mode Exit fullscreen mode

node_exporter.service 

[Unit]
Description=Node Exporter
Wants=network-online.target
After=network-online.target
StartLimitIntervalSec=500
StartLimitBurst=5
[Service]
User=node_exporter
Group=node_exporter
Type=simple
Restart=on-failure
RestartSec=5s
ExecStart=/usr/local/bin/node_exporter \
    --collector.logind
[Install]
WantedBy=multi-user.target
Enter fullscreen mode Exit fullscreen mode

image

sudo systemctl enable node_exporter
Enter fullscreen mode Exit fullscreen mode

Then start the Node Exporter.

sudo systemctl start node_exporter
Enter fullscreen mode Exit fullscreen mode

Check the status of Node Exporter with the following command:

sudo systemctl status node_exporter
Enter fullscreen mode Exit fullscreen mode

image

If you have any issues, check logs with journalctl

journalctl -u node_exporter -f --no-pager
Enter fullscreen mode Exit fullscreen mode

At this point, we have only a single target in our Prometheus. There are many different service discovery mechanisms built into Prometheus. For example, Prometheus can dynamically discover targets in AWS, GCP, and other clouds based on the labels. In the following tutorials, I’ll give you a few examples of deploying Prometheus in a cloud-specific environment. For this tutorial, let’s keep it simple and keep adding static targets. Also, I have a lesson on how to deploy and manage Prometheus in the Kubernetes cluster.

To create a static target, you need to add job_name with static_configs.

sudo vim /etc/prometheus/prometheus.yml
Enter fullscreen mode Exit fullscreen mode

image

prometheus.yml

image

By default, Node Exporter will be exposed on port 9100.

Since we enabled lifecycle management via API calls, we can reload the Prometheus config without restarting the service and causing downtime.

Before, restarting check if the config is valid.

promtool check config /etc/prometheus/prometheus.yml
Enter fullscreen mode Exit fullscreen mode

Then, you can use a POST request to reload the config.

curl -X POST http://localhost:9090/-/reload
Enter fullscreen mode Exit fullscreen mode

Check the targets section

http://<ip>:9090/targets
Enter fullscreen mode Exit fullscreen mode

image

Install Grafana on Ubuntu 22.04

To visualize metrics we can use Grafana. There are many different data sources that Grafana supports, one of them is Prometheus.

First, let’s make sure that all the dependencies are installed.

sudo apt-get install -y apt-transport-https software-properties-common
Enter fullscreen mode Exit fullscreen mode

image

Next, add the GPG key.

wget -q -O - https://packages.grafana.com/gpg.key | sudo apt-key add -
Enter fullscreen mode Exit fullscreen mode

image

echo "deb https://packages.grafana.com/oss/deb stable main" | sudo tee -a /etc/apt/sources.list.d/grafana.list
Enter fullscreen mode Exit fullscreen mode

After you add the repository, update and install Garafana.

sudo apt-get update
Enter fullscreen mode Exit fullscreen mode 
sudo apt-get -y install grafana
Enter fullscreen mode Exit fullscreen mode

image

To automatically start the Grafana after reboot, enable the service.

sudo systemctl enable grafana-server
Enter fullscreen mode Exit fullscreen mode

Then start the Grafana.

sudo systemctl start grafana-server
Enter fullscreen mode Exit fullscreen mode

image

To check the status of Grafana, run the following command:

sudo systemctl status grafana-server
Enter fullscreen mode Exit fullscreen mode

image

Go to http://:3000 and log in to the Grafana using default credentials. The username is admin, and the password is admin as well.

username admin
password admin
Enter fullscreen mode Exit fullscreen mode

image

When you log in for the first time, you get the option to change the password.

To visualize metrics, you need to add a data source first.

image

image

For the URL, enter localhost:9090 and click Save and test. You can see Data source is working.

<public-ip:9090>
Enter fullscreen mode Exit fullscreen mode

image

Click on Save and Test.

image

Let’s add Dashboard for a better view

image

Click on Import Dashboard paste this code 1860 and click on load

image

Select the Datasource and click on Import

image

You will see this output

image

Step 5 — Install the Prometheus Plugin and Integrate it with the Prometheus server
Let’s Monitor JENKINS SYSTEM

Need Jenkins up and running machine

Goto Manage Jenkins –> Plugins –> Available Plugins

Search for Prometheus and install it

Step 6 — Email Integration With Jenkins and Plugin Setup
Install Email Extension Plugin in Jenkins

image

Go to your Gmail and click on your profile

Then click on Manage Your Google Account –> click on the security tab on the left side panel you will get this page(provide mail password).

image

2-step verification should be enabled.

Search for the app in the search bar you will get app passwords like the below image

image

image

Click on other and provide your name and click on Generate and copy the password

image

In the new update, you will get a password like this

Once the plugin is installed in Jenkins, click on manage Jenkins –> configure system there under the E-mail Notification section configure the details as shown in the below image

image

image

Click on Apply and save.

Click on Manage Jenkins–> credentials and add your mail username and generated password

image

This is to just verify the mail configuration

Now under the Extended E-mail Notification section configure the details as shown in the below images

image

image

image

Click on Apply and save.

post {
     always {
        emailext attachLog: true,
            subject: "'${currentBuild.result}'",
            body: "Project: ${env.JOB_NAME}<br/>" +
                "Build Number: ${env.BUILD_NUMBER}<br/>" +
                "URL: ${env.BUILD_URL}<br/>",
            to: 'postbox.aj99@gmail.com',  #change Your mail
            attachmentsPattern: 'trivyfs.txt,trivyimage.txt'
        }
    }
Enter fullscreen mode Exit fullscreen mode

Next, we will log in to Jenkins and start to configure our Pipeline in Jenkins

Step 7 — Install Plugins like JDK, Sonarqube Scanner, NodeJs, OWASP Dependency Check
7A — Install Plugin
Goto Manage Jenkins →Plugins → Available Plugins →

Install below plugins

1 → Eclipse Temurin Installer (Install without restart)

2 → SonarQube Scanner (Install without restart)

3 → NodeJs Plugin (Install Without restart)

image

7B — Configure Java and Nodejs in Global Tool Configuration
Goto Manage Jenkins → Tools → Install JDK(17) and NodeJs(16)→ Click on Apply and Save

image

image

7C — Create a Job
create a job as Netflix Name, select pipeline and click on ok.

Step 8 — Configure Sonar Server in Manage Jenkins
Grab the Public IP Address of your EC2 Instance, Sonarqube works on Port 9000, so :9000. Goto your Sonarqube Server. Click on Administration → Security → Users → Click on Tokens and Update Token → Give it a name → and click on Generate Token

image

click on update Token

image

Create a token with a name and generate

image

copy Token

Goto Jenkins Dashboard → Manage Jenkins → Credentials → Add Secret Text. It should look like this

image

You will this page once you click on create

Now, go to Dashboard → Manage Jenkins → System and Add like the below image.

image

Click on Apply and Save

The Configure System option is used in Jenkins to configure different server

Global Tool Configuration is used to configure different tools that we install using Plugins

We will install a sonar scanner in the tools.

image

In the Sonarqube Dashboard add a quality gate also

Administration–> Configuration–>Webhooks

image

Click on Create

image

Add details

#in url section of quality gate
<http://jenkins-public-ip:8080>/sonarqube-webhook/
Enter fullscreen mode Exit fullscreen mode

Let’s go to our Pipeline and add the script in our Pipeline Script.

pipeline{
    agent any
    tools{
        jdk 'jdk17'
        nodejs 'node16'
    }
    environment {
        SCANNER_HOME=tool 'sonar-scanner'
    }
    stages {
        stage('clean workspace'){
            steps{
                cleanWs()
            }
        }
        stage('Checkout from Git'){
            steps{
                git branch: 'main', url: 'https://github.com/Aj7Ay/Netflix-clone.git'
            }
        }
        stage("Sonarqube Analysis "){
            steps{
                withSonarQubeEnv('sonar-server') {
                    sh ''' $SCANNER_HOME/bin/sonar-scanner -Dsonar.projectName=Netflix \
                    -Dsonar.projectKey=Netflix '''
                }
            }
        }
        stage("quality gate"){
           steps {
                script {
                    waitForQualityGate abortPipeline: false, credentialsId: 'Sonar-token'
                }
            }
        }
        stage('Install Dependencies') {
            steps {
                sh "npm install"
            }
        }
    }
    post {
     always {
        emailext attachLog: true,
            subject: "'${currentBuild.result}'",
            body: "Project: ${env.JOB_NAME}<br/>" +
                "Build Number: ${env.BUILD_NUMBER}<br/>" +
                "URL: ${env.BUILD_URL}<br/>",
            to: 'postbox.aj99@gmail.com',
            attachmentsPattern: 'trivyfs.txt,trivyimage.txt'
        }
    }
}
Enter fullscreen mode Exit fullscreen mode

Click on Build now, you will see the stage view like this

image

image

You can see the report has been generated and the status shows as passed. You can see that there are 3.2k lines it scanned. To see a detailed report, you can go to issues.

Step 9 — Install OWASP Dependency Check Plugins
GotoDashboard → Manage Jenkins → Plugins → OWASP Dependency-Check. Click on it and install it without restart.

image

First, we configured the Plugin and next, we had to configure the Tool

Goto Dashboard → Manage Jenkins → Tools →

image

Click on Apply and Save here.

Now go configure → Pipeline and add this stage to your pipeline and build.

stage('OWASP FS SCAN') {
            steps {
                dependencyCheck additionalArguments: '--scan ./ --disableYarnAudit --disableNodeAudit', odcInstallation: 'DP-Check'
                dependencyCheckPublisher pattern: '**/dependency-check-report.xml'
            }
        }
        stage('TRIVY FS SCAN') {
            steps {
                sh "trivy fs . > trivyfs.txt"
            }
        }
Enter fullscreen mode Exit fullscreen mode

The stage view would look like this,

image

You will see that in status, a graph will also be generated and Vulnerabilities.

image

Step 10 — Docker Image Build and Push
We need to install the Docker tool in our system, Goto Dashboard → Manage Plugins → Available plugins → Search for Docker and install these plugins

Docker

Docker Commons

Docker Pipeline

Docker API

docker-build-step

and click on install without restart

image

Now, goto Dashboard → Manage Jenkins → Tools →

image

Add DockerHub Username and Password under Global Credentials

image

Add this stage to Pipeline Script

stage("Docker Build & Push"){
            steps{
                script{
                   withDockerRegistry(credentialsId: 'docker', toolName: 'docker'){
                       sh "docker build --build-arg TMDB_V3_API_KEY=Aj7ay86fe14eca3e76869b92 -t netflix ."
                       sh "docker tag netflix anilkumar061/netflix:latest "
                       sh "docker push anilkumar061/netflix:latest "
                    }
                }
            }
        }
        stage("TRIVY"){
            steps{
                sh "trivy image anilkumar061/netflix:latest > trivyimage.txt"
            }

        stage('Deploy to container'){
            steps{
                sh 'docker run -d --name netflix -p 8081:80 sevenajay/netflix:latest'
              }
            }
        }
Enter fullscreen mode Exit fullscreen mode

You will see the output below, with a dependency trend.

image

image

When you log in to Dockerhub, you will see a new image is created

Jenkins-public-ip:8081

You will get this output

image

Step 11 : Deploy the image using Docker

  • Kuberenetes Setup
  • Connect your machines to Putty or Mobaxtreme
  • Take-Two Ubuntu 20.04 instances one for k8s master and the other one for worker.
  • Install Kubectl on Jenkins machine also.
  • Kubectl is to be installed on Jenkins also 
sudo apt update
sudo apt install curl
curl -LO https://dl.k8s.io/release/$(curl -L -s https://dl.k8s.io/release/stable.txt)/bin/linux/amd64/kubectl
sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl
kubectl version --client
Enter fullscreen mode Exit fullscreen mode
  • Part 1 ----------Master Node------------ 
sudo hostnamectl set-hostname K8s-Master
Enter fullscreen mode Exit fullscreen mode
  • ----------Worker Node------------ 
sudo hostnamectl set-hostname K8s-Worker
Enter fullscreen mode Exit fullscreen mode
  • Part 2 ------------Both Master & Node ------------ 
sudo apt-get update 

sudo apt-get install -y docker.io
sudo usermod –aG docker Ubuntu
newgrp docker
sudo chmod 777 /var/run/docker.sock

sudo curl -s https://packages.cloud.google.com/apt/doc/apt-key.gpg | sudo apt-key add -

sudo tee /etc/apt/sources.list.d/kubernetes.list <<EOF
deb https://apt.kubernetes.io/ kubernetes-xenial main
EOF

sudo apt-get update

sudo apt-get install -y kubelet kubeadm kubectl

sudo snap install kube-apiserver
Enter fullscreen mode Exit fullscreen mode
  • Part 3 --------------- Master --------------- 
sudo kubeadm init --pod-network-cidr=10.244.0.0/16
# in case your in root exit from it and run below commands
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config
kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
Enter fullscreen mode Exit fullscreen mode
  • ----------Worker Node------------ 
sudo kubeadm join <master-node-ip>:<master-node-port> --token <token> --discovery-token-ca-cert-hash <hash>
Enter fullscreen mode Exit fullscreen mode
  • Copy the config file to Jenkins master or the local file manager and save it
  • copy it and save it in documents or another folder save it as secret-file.txt
  • Note: create a secret-file.txt in your file explorer save the config in it and use this at the kubernetes credential section.
  • Install Kubernetes Plugin, Once it's installed successfully

  • goto manage Jenkins --> manage credentials --> Click on Jenkins global --> add credentials

  • Install Node_exporter on both master and worker

  • Let's add Node_exporter on Master and Worker to monitor the metrics

  • First, let's create a system user for Node Exporter by running the following command:

sudo useradd \
    --system \
    --no-create-home \
    --shell /bin/false node_exporter
Enter fullscreen mode Exit fullscreen mode
  • Use the wget command to download the binary. 
wget https://github.com/prometheus/node_exporter/releases/download/v1.6.1/node_exporter-1.6.1.linux-amd64.tar.gz
Enter fullscreen mode Exit fullscreen mode
  • Extract the node exporter from the archive. 
tar -xvf node_exporter-1.6.1.linux-amd64.tar.gz
Enter fullscreen mode Exit fullscreen mode
  • Move binary to the /usr/local/bin. 
sudo mv \
  node_exporter-1.6.1.linux-amd64/node_exporter \
  /usr/local/bin/
Enter fullscreen mode Exit fullscreen mode
  • Clean up, and delete node_exporter archive and a folder. 
rm -rf node_exporter*
Enter fullscreen mode Exit fullscreen mode
  • Next, create a similar systemd unit file. 
sudo vim /etc/systemd/system/node_exporter.service
Enter fullscreen mode Exit fullscreen mode
  • node_exporter.service 
[Unit]
Description=Node Exporter
Wants=network-online.target
After=network-online.target

StartLimitIntervalSec=500
StartLimitBurst=5

[Service]
User=node_exporter
Group=node_exporter
Type=simple
Restart=on-failure
RestartSec=5s
ExecStart=/usr/local/bin/node_exporter \
    --collector.logind

[Install]
WantedBy=multi-user.target
Enter fullscreen mode Exit fullscreen mode
  • Replace Prometheus user and group to node_exporter, and update the ExecStart command.
  • To automatically start the Node Exporter after reboot, enable the service. 
sudo systemctl enable node_exporter
sudo systemctl start node_exporter
sudo systemctl status node_exporter
Enter fullscreen mode Exit fullscreen mode
  • If you have any issues, check logs with journalctl 
journalctl -u node_exporter -f --no-pager
Enter fullscreen mode Exit fullscreen mode
  • At this point, we have only a single target in our Prometheus. There are many different service discovery mechanisms built into Prometheus. For example, Prometheus can dynamically discover targets in AWS, GCP, and other clouds based on the labels. In the following tutorials, I'll give you a few examples of deploying Prometheus in a cloud-specific environment. For this tutorial, let's keep it simple and keep adding static targets. Also, I have a lesson on how to deploy and manage Prometheus in the Kubernetes cluster.
  • To create a static target, you need to add job_name with static_configs. Go to Prometheus server 
sudo vim /etc/prometheus/prometheus.yml
Enter fullscreen mode Exit fullscreen mode
  • prometheus.yml 
  - job_name: node_export_masterk8s
    static_configs:
      - targets: ["<master-ip>:9100"]

  - job_name: node_export_workerk8s
    static_configs:
      - targets: ["<worker-ip>:9100"]
Enter fullscreen mode Exit fullscreen mode
  • By default, Node Exporter will be exposed on port 9100.
  • Since we enabled lifecycle management via API calls, we can reload the Prometheus config without restarting the service and causing downtime.
  • Before, restarting check if the config is valid. 
promtool check config /etc/prometheus/prometheus.yml
Enter fullscreen mode Exit fullscreen mode
  • Then, you can use a POST request to reload the config. 
curl -X POST http://localhost:9090/-/reload
Enter fullscreen mode Exit fullscreen mode
  • Check the targets section 
http://<ip>:9090/targets
Enter fullscreen mode Exit fullscreen mode
  • final step to deploy on the Kubernetes cluster 
stage('Deploy to kubernets'){
            steps{
                script{
                    dir('Kubernetes') {
                        withKubeConfig(caCertificate: '', clusterName: '', contextName: '', credentialsId: 'k8s', namespace: '', restrictKubeConfigAccess: false, serverUrl: '') {
                                sh 'kubectl apply -f deployment.yml'
                                sh 'kubectl apply -f service.yml'
                        }   
                    }
                }
            }
        }
Enter fullscreen mode Exit fullscreen mode
  • In the Kubernetes cluster(master) give this command 
kubectl get all 
kubectl get svc #use anyone
Enter fullscreen mode Exit fullscreen mode

Step 12: Kubernetes master and slave setup on Ubuntu (20.04)

  • Access from a Web browser with

Step 13: Access the Netflix app on the Browser

  • image
  • image
  • image
  • image
  • image
  • image
  • image
  • image
  • image

Step 14: Terminate instances

🛠️ Author & Community

This project is crafted by Anilkumar N 💡.

I’d love to hear your feedback! Feel free to share your thoughts.

📧 Connect with me:

⭐ Support the Project

If you found this helpful, consider starring ⭐ the repository and sharing it with your network! 🚀

📢 Stay Connected

Top comments (0)