Kubernetes on AWS
Dockerhub
/ \
/ \
Push/Image Pull\Image
/ \
Pull Code Copy Artifacts / Deploy Container \
Github--------->Jenkin-------------->Ansible--------------->Kubernetes |
|
| Build Code
\|/
Maven
Setup Kubernetes on Amazon EKS
You can follow same procedure in the official AWS document Getting started with Amazon EKS – eksctl
Pre-requisites:
- an EC2 Instance
- Install AWSCLI latest verison
Setup kubectl
- Download kubectl version 1.21
- Grant execution permissions to kubectl executable
- Move kubectl onto /usr/local/bin
- Test that your kubectl installation was successful
curl -o kubectl https://amazon-eks.s3.us-west-2.amazonaws.com/1.21.2/2021-07-05/bin/linux/amd64/kubectl
chmod +x ./kubectl
mv ./kubectl /usr/local/bin
kubectl version --short --client
Setup eksctl
- Download and extract the latest release
- Move the extracted binary to /usr/local/bin
- Test that your eksclt installation was successful
curl --silent --location "https://github.com/weaveworks/eksctl/releases/latest/download/eksctl_$(uname -s)_amd64.tar.gz" | tar xz -C /tmp
sudo mv /tmp/eksctl /usr/local/bin
eksctl version
Create an IAM Role and attache it to EC2 instance
Note: create IAM user with programmatic access if your bootstrap system is outside of AWS
IAM user should have access to
IAM
EC2
CloudFormation
Note: Check eksctl documentaiton for Minimum IAM policies
Create your cluster and nodes
eksctl create cluster --name cluster-name \
--region region-name \
--node-type instance-type \
--nodes-min 2 \
--nodes-max 2 \
--zones <AZ-1>,<AZ-2>
example:
eksctl create cluster --name valaxy-cluster \
--region ap-south-1 \
--node-type t2.small \
To delete the EKS clsuter
eksctl delete cluster cluster-name --region ap-south-1
Validate your cluster using by creating by checking nodes and by creating a pod
kubectl get nodes
kubectl run tomcat --image=tomcat
Deploying Nginx pods on Kubernetes
Deploying Nginx Container
kubectl create deployment demo-nginx --image=nginx --replicas=2 --port=80
# kubectl deployment regapp --image=valaxy/regapp --replicas=2 --port=8080
kubectl get all
kubectl get pod
Expose the deployment as service. This will create an ELB in front of those 2 containers and allow us to publicly access them.
kubectl expose deployment demo-nginx --port=80 --type=LoadBalancer
# kubectl expose deployment regapp --port=8080 --type=LoadBalancer
kubectl get services -o wide
Create Pod and Service using manifest file
Create pod manifest file
sudo su -
vim pod.yml
apiVersion: v1
kind: Pod
metadata:
name: demo-pod
labels:
app: demo-app
spec:
containers:
- name: demo-nginx
image: nginx
ports:
- name: demo-nginx
containerPort: 80
Create service manifest file
vim service.yml
apiVersion: v1
kind: Service
metadata:
name: demo-service
labels:
app: demo-app
spec:
ports:
- name: nginx-port
port: 80
targetPort: 80
selector:
app: demo-app
type: LoadBalancer
Run pod and service
kubectl apply -f pod.yml
kubectl get pod
kubectl apply -f service.yml
kubectl get service
kubectl get all
kubectl get pods -o wide
kubectl get services -o wide
kubectl describe pod podname
kubectl describe service servicename
Setup Pod and Service
selector: labels:
app: demo-app app: demo-app
External network-------------->Service------------------->Pod
As we know there will be large number of pods,When we send a request from External network to Service it looks for selector app: demo-app and forward the requset to the pods having Labels app: demo-app.That is why we use selector in service manifest file and labels in pod manifest file.
Integrating Kubernetes in CI/CD pipeline
Go to ansible machine
create a user and allow password authentication.
sudo su -
adduser ansadmin
id ansadmin
vim /etc/ssh/sshd_config
PasswordAuthentication yes
#PasswordAuthentication no
systemctl restart ssh
Modify visudo for created user.
vim /etc/sudoers
# User privilege specification
root ALL=(ALL:ALL) ALL
ansadmin ALL=(ALL:ALL) NOPASSWD: ALL
User Privilege Lines
root ALL=(ALL:ALL) ALL The first field indicates the username that the rule will apply to (root).
root ALL=(ALL:ALL) ALL The first “ALL” indicates that this rule applies to all hosts.
root ALL=(ALL:ALL) ALL This “ALL” indicates that the root user can run commands as all users.
root ALL=(ALL:ALL) ALL This “ALL” indicates that the root user can
run commands as all groups.
root ALL=(ALL:ALL) ALL The last “ALL” indicates these rules apply to all commands.
Generate ssh-key.
su - ansadmin
ssh-keygen
Go to Kubernetes machine
Create Deployment and Service using manifest file
Create deployment manifest file
sudo su -
vim deployment.yml
apiVersion: v1
kind: Deployment
metadata: --->Deployment name and label
name: demo-webapp
labels:
app: webapp
spec:
replicas: 2
selector: --->Create 2 pods from pod template
matchLabels:
app: webapp
template: |
metadata: ---> pod definition |
labels: |
app: webapp |
|
spec: containers |---> Template to create a pod |
- name: webapp |
image: nithinalias/mytomcat ---> container definition |
imagePullPolicy: Always |
ports: |
- containerPort: 8080 |
strategy:
type: RollingUpdate
rollingUpdate: --->make sure only one pod updated at a time
maxSugre: 1
maxUnavialable: 1
Create service manifest file
vim service.yml
apiVersion: v1
kind: Service ---> Resource Type
metadata:
name: demo-service ---> service name and label
labels:
app: webapp
spec:
ports:
- name: nginx-port
port: 8080 ---> port number exposed at cluster level
targetPort: 8080 ---> port that container listening
selector: ---> To which deployment it can send traffic
app: webapp
type: LoadBalancer ---> service type
labels in deployment manifest file need to match with selector in service manifest file.containerPort in deployment manifest file need to match with targetPort in service manifest file.
sudo su -
passwd
adduser ansadmin
id ansadmin
docker login
Allow password authentication.
vim /etc/ssh/sshd_config
PermitRootLogin yes
PasswordAuthentication yes
#PasswordAuthentication no
systemctl restart ssh
Modify visudo for created user.
vim /etc/sudoers
# User privilege specification
root ALL=(ALL:ALL) ALL
ansadmin ALL=(ALL:ALL) NOPASSWD: ALL
Go to ansible machine and add ip address of kubernetes machine and ansible machine(then only jenkin server can access ansible and kubernetes machine) inside ansible host file.
vim /etc/ansible/hosts
[ansible]
192.168.33.25
[kubernetes]
192.168.33.30
copy the ssh public key to kubernetes machine and ansible machine itself(localhost).
su - ansadmin
ssh-copy-id ansadmin@ip-of-ansiblemachine
ssh-copy-id ansadmin@ip-of-kubernetesmachine
ssh-copy-id root@ip-of-kubernetesmachine
ansible all -m ping
Now authorized key will be generated inside /home/ansadmin/.ssh/authorized_keys of kubernetes,ansible machine and /root/.ssh/authorized_keys of kubernetes machine.
create a directory,give ownership of ansadmin,add ansadmin to docker group and docker.sock permissions to all users.
sudo su -
mkdir /opt/docker
chown ansadmin:ansadmin /opt/docker
usermod -aG docker ansadmin
id ansadmin
chmod 777 /var/run/docker.sock
systemctl restart docker
su - ansadmin
docker login
Create dockerfile for tomcat image creation.
vim /opt/docker/dockerfile
FROM tomcat
RUN cp -R /usr/local/tomcat/webapps.dist/* /usr/local/tomcat/webapps
COPY ./*.war /usr/local/tomcat/webapps
Generate ansible-playbook for building tomcat image.
vim /opt/docker/build_tomcatimage.yml
---
- hosts: ansible
tasks:
- name: create docker image
command: docker build -t mytomcat .
args:
chdir: /opt/docker
- name: create tag to push image onto dockerhub
command: docker tag mytomcat nithinalias/mytomcat
- name: push docker image
command: docker push nithinalias/mytomcat
Generate ansible-playbook to run deployment and service manifest file.
vim /opt/docker/run_kube_manifestfile.yml
---
- hosts: kubernetes
user: root
tasks:
- name: run deployment manifestfile on kubernetes
command: kubectl apply -f deployment.yml
- name: run service manifestfile on kubernetes
command: kubectl apply -f service.yml
- name: update deployment with new pods if image updated in dockerhub
command: kubectl rollout restart deployment.apps/demo-webapp
# we don't want to mention exact path like /root/deployment.yml or /root/service.yml, since we are performing the task as root user.
Integrate Ansible with Jenkins
we need to add ansadmin ssh connection of ansible machine.
Manage Jenkins - Configure System - SSH servers - Add - Add name(ansible-server),Hostname(ip-of-ansible machine),username
(ansadmin) - Advanced - Enable Use password authentication or use a different key - Add password of user ansadmin(instead you can use ssh-key/path of ssh-key if present) - Test configuration - Apply - save.
Continuous Integration using Jenkins
New Item - Add item name(CI webapp),select Maven project - OK - Add description - Select Git and add Repositories URL,Branch - Build Triggers(Poll SCM=* * * * * means every minute check the repository and if there any update it will trigger build - Build periodically means it will trigger build periodically even if repository not updated) - Build(Add Root POM=pom.xml,Goals and options=clean install) - Add post-build Action - Send build artifacts over SSH - Select SSH server name(ansible-server),Add sourcefile(webapp/target/*.war),Add Remove prefix(webapp/target),Add Remote directory="//opt//docker"(if it is blank webapp.war will be in /home/ansadmin),
Exec command ="ansible-playbook /opt/docker/build_tomcatimage.yml" - Add post-build action - Build other projects - Go to post-build action(top) - Add Projects to build="CD webapp" - Enable Trigger only if build is stable - Apply - Save - Build Now - Console output.You can see all build outcomes in /var/lib/jenkins/workspace/CI webapp/webapp/target.surefire contain the reports of build and webapp.war is called artifacts.This webapp.war will be in ansible machine /opt/docker directory and copy this to tomcat image.The image created inside ansible machine is pushed to dockerhub.
Continuous Deployment using Jenkins
New Item - Add item name(CD webapp) - OK - Add description - Add post-build Action - Send build artifacts over SSH - Select SSH server name(ansible-server),Exec command =
"ansible-playbook /opt/docker/run_kube_manifestfile.yml" - Apply - Save.
Go to Kubernetes machine
kubectl get service
copy the EXTERNAL-IP of your service and paste it in browser.Now you get the webapp.
OR
You can go to LoadBalancer in AWS - Description - Copy DNS name - paste it in browser.Now you get the webapp.
Source: https://github.com/yankils
Top comments (0)