DEV Community: KodeKloud

CKS Challenges

KodeKloud — Mon, 09 May 2022 08:13:12 +0000

We have launched a special series to help you prepare for the Kubernetes CKS certifications. Check out the Certified Kubernetes Security Specialist Challenge Series, where you can put all your hardcore Kubernetes skills to the test.

This series consists of a set of complex challenges that will assist you in mastering Kubernetes Security concepts and getting ready for the coveted Certified Kubernetes Security Specialist Certification.

These challenges will test you on Kubernetes security concepts such as network policies, RBAC, seccomp, AppArmor, etc. To solve some of the tasks, you will also need to make use of third party security tools such as Aquasec Trivy, Kubesec, CIS Benchmarks and Falco from Sysdig open source.

The interface of these challenges is divided into two parts:

The top half contains the Quiz portal where the details related to the challenge is displayed and an interactive Architecture diagram.

Click on the icons and the arrow connectors in the architecture diagram and an associated task (if available) will be displayed on the quiz portal.

2.The bottom half of the interface contains the terminal to the Kubernetes controlplane which you would use to complete the tasks.

To validate if a task is complete, you can click on the “Check” button. If complete, the icons in the architecture diagram will be highlighted in green. If something is yet to be completed, it will be highlighted in red.

To complete the challenge, you must complete all the tasks.

How to submit your solution

Record your solutions and share on social media on your blog/video on YouTube.
And these are absolutely free for anyone to attempt.

Just follow the steps below to submit your solution and win exciting prizes.

Document your solution with a good explanation on your blog or GitHub
Or even better record a demo and upload on YouTube
Add a link to the CKS challenges on KodeKloud in your reference https://kodekloud.com/courses/cks-challenges/
Submit the details through the form: https://g0u7lypetlq.typeform.com/cks-challenge

Prices

The best solution with the best explanation wins the content.

We give away 2 Exam vouchers each month for top submissions.

So what are you waiting for? Get started right now.

Docker Certified Associate Exam Series (Part -2): Container Orchestration

KodeKloud — Wed, 13 Jan 2021 07:38:44 +0000

Introduction to Container Orchestration

An essential part of preparing for the Docker Certified Associate (DCA) exam is to familiarize yourself with Container Orchestration. Container Orchestration requires a set of tools and scripts that you can use to host, configure, and manage containers in a production environment.

Deploying in Docker typically involves running various applications on different hosts. Container orchestration will help you set up a large number of application instances using a single command. Container orchestration tools also help scale your application’s instances up or down in response to fluctuations in demand. With container orchestration tools, you can also provide advanced networking between various containers.

Three of the most popular Container Orchestration tools are Docker Swarm, Kubernetes, and MESOS.

Docker Swarm is hugely popular and easy to set up, yet has a few drawbacks when it comes to autoscaling and customizations.
MESOS is challenging to use, and is only recommended for advanced Cloud developers.
Kubernetes is a popular container orchestration solution that offers plenty of customization options and unmatched auto-scaling capabilities.

For this part of the study guide series, we shall cover Docker Swarm.

Docker Swarm

Docker Swarm helps you run applications on the Docker Engine seamlessly through multiple nodes that reside in the same containers. With Docker Swarm, you can always monitor the state, health, and performance of your containers and the hosts that run your applications.

As you prepare for the DCA exam, some of the topics of Docker Swarm that you’ll need an in-depth understanding of include:

Swarm Architecture
Setting up a 2-node cluster in Swarm
Creating a demo swarm cluster setup
Basic Swarm Operations
Swarm High Availability and the Importance of Quorum
Swam in High Availability Mode
Auto-lock and a classroom demo
Swarm Services
Rolling Updates, Rollbacks, and Scaling
Swarm Service Types
Placement in Swarm
Service in Swarm- Basic Operations
Service in Swarm- placements, global, parallelism, and replicated
Docker Config Objects
The Docker Overlay Network
MACVLan Networks
Swarm Service Discovery
Docker Stack

Let’s explore some of these areas in detail:

Swarm Architecture

As you study for your exam, you should develop a high level of familiarity with the structure and architecture of Docker Swarm.

Docker Swarm lets you integrate different Docker machines onto a single cluster. This helps with your application’s load balancing and also improves its availability. A Docker Cluster is made up of different instances called Nodes. Nodes can be categorized into two types: Manager and Worker nodes.

A Manager Node receives instructions from a user, turns these into service tasks, which are then assigned to one or more worker nodes. Such nodes also help maintain the desired state of the cluster in which it belongs. Managers can be configured to run production workloads too, when needed.

On the other hand, a Worker Node receives instructions from the manager nodes, and uses these instructions to deploy and run the necessary containers.

Some features of Docker Swarm Architecture include:

Swarm is easy to set up and maintain since all the features of Docker Swarm are embedded in the Docker Engine.
Docker Swarm deploys applications in a Declarative format.
The Swarm manager automatically scales and distributes application instances across worker nodes depending on demand.
Rolling updates reconfigure your application one-at-a-time for easier change management.
Docker Swarm performs desired state reconciliation for self-healing applications.
SSL/TLS certificates secure communication between nodes via authentication and encryption
Uses an external load balancer to distribute requests between nodes

Setting up a 2-Node Swarm Cluster

This lesson is a demonstration of how you can create a Docker Swarm cluster with 2-worker nodes and 1-manager.
The prerequisites needed for this session will include:

Machines (Nodes) deployed and designated as Manager, Worker-1, and Worker-2.
The machines should have the Docker Engine installed.
Each node should be assigned a static IP address.
The ports TCP 2377, TCP & UDP 7946 and UDP 4789 should be opened.

To initialize Docker Swarm on your manager node, use the following command while the manager is active:

$ docker swarm init

Swarm initialized: current node (whds9866c56gtgq3uf5jmfsip) is now a manager.

To add a worker to this swarm, run the following command:

    docker swarm join --token SWMTKN-1-19nlqoifkry03y8l5242zl6e2te2k9dvzebf5b70ihhpn7r4qh-aqtxt2sd0sh0hj2f8ceupj53g 172.17.0.27:2377

To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions.

This command initializes Swarm on the selected node, which is now a manager. The command also returns a script you will use to add a worker to this swarm, as indicated on the Command Line Interface.

To add the second worker to this Swarm, use the following command:

$ docker swarm join-token worker

To add a worker to this swarm, run the following command:

docker swarm join --token SWMTKN-1-19nlqoifkry03y8l5242zl6e2te2k9dvzebf5b70ihhpn7r4qh-aqtxt2sd0sh0hj2f8ceupj53g 172.17.0.27:2377

To display a list of your nodes and workers’ names and status, type the following command onto the CLI:

$ docker node ls

Swarm Operations

You will learn some of the common Swarm operations that involve promoting, draining, and deleting nodes.

To promote a node to manager, you will run the command:

$ docker node promote worker1
Node worker1 promoted to a manager in the swarm.

To demote a manager node to Worker, you will run the command:

$ docker node demote worker1
Manager worker1 demoted in the swarm.

When you want to perform upgrades and maintenance on your cluster, you might need to drain each node independently, one at a time. Let us assume that the current state of the cluster has the following nodes, as shown below:

To drain your node, use the command:

$ docker node update --availability drain worker1
worker1

This command brings down containers on worker1 and runs replica instances on another worker until it gets back up.

Once you are done patching or maintaining your node, you will run the update command but with the availability being active to bring it back up. The command to use is:

$ docker node update --availability active worker1
worker1

To delete a node from a cluster, drain it so that it’s workload is redistributed to another node, then run the command:

$ docker swarm leave
Node left the swarm.

Swarm High Availability- Quorum

Docker Swarm uses a RAFT algorithm to create distributed consensus when more than one manager node is running on a cluster. Having multiple managers in a cluster helps with fault tolerance.

The RAFT algorithm initiates requests at random times. The first manager to respond then requests other managers in the cluster to make it a Leader. If the managers respond positively, the leader assumes this role, sending notifications and updating a shared database on the state of the cluster.
This database is available to all managers in the cluster. Before the leader makes any changes to a cluster, he sends instructions to other managers. The managers should reach a Quorum and agree before changes are effected. If the leader manager loses connectivity, other managers initiate a process to elect a new leader.

The best practices for high availability in Swarm include:

Each cluster should have an odd number of managers for easier network segmentation.
Every decision should be agreed upon when the number of managers presents reach a Quorum. The quorum for a cluster with N managers is N/2+1
The number of failures a cluster can withstand is the Fault Tolerance and is calculated as 2*N -1

Distribute manager nodes equally over different data centres/availability zones so the cluster can withstand sitewide disruptions.

If more than the allowed number of managers fail, you cannot perform managerial duties on your cluster. The worker nodes will, however, continue to run normally with all the services and configuration settings still active.

To resolve a failed cluster, you could attempt to bring the failed managers back online. If this fails, you can create a new cluster using the force-create command. This will be a healthy, single-manager cluster. The command for force create is:

$ docker swarm init --force-new-cluster

Once this cluster has been created, you can promote other workers into manager nodes using the promote command.

Swarm Services

As you begin to deploy your clusters, you’ll need a way to run multiple instances of your application across several worker nodes to help with automation and load balancing. The Docker Service allows you to launch containers in a coordinated manner across several nodes.

To create 3 replicas of your application using the Docker service, run the command:

$ docker service create --replicas=3 App1

When you deploy your applications, an API server creates a service which is then divided into tasks by the orchestrator. The allocator then assigns each task an IP address, and a dispatcher assigns these tasks to individual workers. The scheduler then manages task handling by the workers.

Here are a few common service tasks and their Docker Commands.

Create an overlay network

$ docker network create --driver overlay my-overlay-network

Create a subnet

$ docker network create --driver overlay --subnet 10.15.0.0/16 my-overlay-network

Make a network attachable to external containers

$ docker network create --driver overlay --attachable my-overlay-network

Enable IPS Encryption

$ docker network create --driver overlay --opt encrypted my-overlay-network

Attach a service to a network

$ docker service create --network my-overlay-network my-web-service

Delete a newly created network

$ docker service create --network my-overlay-network my-web-service

Delete all networks

$ docker network prune

Here are some network ports and their purposes.

TCP 2377: Cluster Management Communications
TCP/UDP: Container Network Discovery/ Communication Among Nodes
UDP 4789: Overlay Network Traffic

To publish a host on port 80 pointing to a container on port 5000, use the command:

$ docker service create -p 80:5000 my-web-server

docker service create --publish published=80, target=5000 my-web-server

To include UDP:

$ docker service create -p 80:5000/UDP my-web-server

$ docker service create --publish published=80, target=5000, protocol=UDP my-web-server

Swarm Service Discovery

Containers and services in a node can communicate with each other directly using their names. To make sure that these containers can ‘see’ each other, you should create an overlay network in which you should place the application and the naming service. For instance:

Create an overlay network:

$ docker network create --driver=overlay app-network

Then create an API server within this network:

$ docker service create --name=api-server --replicas=2 --network=app-network api server

Create the web service task:

$ docker service create --name=web --network=app-network web

The services can now reach each other using their service names. The web server can now reach the api-server using the service name api-server.

Docker Stack

In Docker, a stack is a group of interrelated services that together form the functionality of an application. All of your application’s configuration settings and changes are stored in a docker configuration file, known as a Docker Stack or Docker Compose File. Docker Compose lets you create stack files in YAML, which makes your application easier to manage, highly distributed, and scalable. The docker stack file also lets you perform health checks on your container and set the grace period during which the health check stays inactive.

To create a docker stack file, run the command:

$ docker stack deploy --compose-file docker-compose.yml
Other Docker Stack Commands include:

Task & Command

Create a Stack

$ docker stack deploy

list active stacks

$ docker stack ls

list services created by a stack

$ docker stack ps

list tasks running in a stack

$ docker stack ps

Delete a Stack

$ docker stack rm

Docker Storage

To understand how container orchestration tools manage storage, it is important to know how docker manages storage in containers. Getting to know storage in docker will go a long way in helping you manage storage with Kubernetes. Storage in Docker is managed by two techniques: Storage Drivers and Volume Drivers.

Docker uses Storage Drivers to enable layered architecture. These are attached to containers by default and point file systems to the default path: /var/lib/docker where it stores files in subfolders such as aufs, containers, image and volumes.

Popular storage drivers include: AUFS, ZFS, BDRFS, Device Mapper, Overlay and Overlay2.

Volume Driver Plugins in Docker

Volume Drivers help create persistent volumes in Docker. By default, volumes are assigned a local driver that stores data on the host's volume directory.

There are third-party volume driver plugins that help with storage on various public cloud platforms. These include: Azure File Storage, Convoy, DigitalOcean BlockStorage, Flocker, gce-docker, GlusterFS, NetApp, RexRay, Portworx, VMware vSphere Storage, among others. Docker automatically assigns the appropriate volume driver depending on the operating system and application needs.

To create a volume on Amazon AWS ElasticBlockStorage, run the command:

$ docker run -it \
--name app1
    --volume driver rexray/ebs
    --mount src=ebs -vol,target=/var/lib/app1
    app1

This command creates a persistent volume storage on Amazon EBS at app1’s default file location.

Sample Questions:

Here is a quick quiz to help you assess your knowledge. Leave your answers in the comments below and tag us back.

Quick Tip - Questions below may include a mix of DOMC and MCQ types.

Which command can be used to remove a `kubeapp` stack?

[A] docker stack deploy kubeapp
[B] docker stack ls kubeapp
[C] docker stack services kubeapp
[D] docker stack rm kubeapp

Which command can be used to promote worker2 to a manager node? Select the right answer.

[A] docker promote node worker2
[B] docker node promote worker2
[C] docker swarm node promote worker2
[D docker swarm promote node worker2

What is the command to list the stacks in the Docker host?

[A] docker stack deploy
[B] docker stack ls
[C] docker stack services
[D] docker stack ps

What is the maximum number of managers possible in a swarm cluster?

[A] 3
[B] 5
[C] 7
[D] No limit

... are one or more instances of a single application that runs across the Swarm Cluster.

[A] docker stack
[B] services
[C] pods
[D] None of the above

Conclusion

Once you have understood Swarm architecture, set up a cluster, and ensured high availability, you would have developed enough familiarity to tackle real-world projects. Swarm services will help you automate the interaction between various nodes to help with load balancing for distributed containers. By following this guide, you will have enough knowledge on the importance of Container Orchestration and how Docker Swarm offers a simple, no-fuss framework that helps keep your containers healthy.

To test where you stand in your Docker certification journey take the DCA Readiness Test at dca.kodekloud.com. On KodeKloud, you also get a learning path with recommendations, sample questions and tips for clearing the DCA exam.

Docker Certified Associate (DCA) - The Ultimate Certification Guide for 2021

KodeKloud — Wed, 23 Dec 2020 09:22:08 +0000

Why Get a Docker Certification?

With a rising pattern of organizations adopting Cloud services, Docker continues to gain popularity. At its core, Docker aids in containerization, that is, for the packaging of applications into modules that can easily be replicated and scaled independently. Unarguably, as more applications are moving to the cloud, being an expert in Docker makes you a hot-skilled candidate in the modern IT world.

In this article today, we would run through the Docker Certified Associate (DCA) exam curriculum and helpful resources to crack the certification test.

Curriculum Covered in Docker Certification

The DCA Certification is awarded by Docker (proctored by Examity) that helps highlight your familiarity and expertise with application deployment using Docker. In the exam, apart from your knowledge of Container Orchestration, you are required to have working knowledge of Docker Enterprise Edition and Docker Swarm. On passing this exam, you’ll have proven that you can install and configure containerized applications using Docker, making it a useful benchmark of your IT skills.

Below are few of the key concepts based on their weightage that you should get hands-on while you start preparing for a DCA certification:

Container Orchestration

This part essentially covers Basics of Docker and Container Orchestration; and carries about 25% of the total mark of your DCA exam. Additionally, you are also required to learn various Container Orchestration Tools that help automate the process of managing containers.

Content break-up on container orchestration includes:

Setting up a Swarm Mode Cluster
Locking a Swarm Cluster
Deploying Applications into Stack Files
Running a Service vs. Running a Container
Manage a stack of running services
Replication of Services
Replicated and Global Services
Troubleshoot a non-deploying Service
Communication among Docker Applications and Legacy Systems
Service Templates

An essential part of this content ensures that you learn the basics of container orchestration, including tools that automate deployment of containerized applications, manage release updates, and configure failed containers. On successful completion of this part, it is expected that you’ll be able to create your first orchestrated, containerized application.

Docker Swarm/Kubernetes

Both Kubernetes and Docker Swarm are popular choices of container orchestration. In essence, Kubernetes focuses on open-source and modular orchestration, offering an efficient container orchestration solution for high-demand applications with complex configuration. On the other hand, Docker Swarm emphasizes ease of use, making it most suitable for simple applications that are quick to deploy and easy to manage. As you prepare for the DCA exam, you are required to have a working knowledge of both of these tools and should be aware about their most appropriate use-cases in different scenarios.

Image Creation, Registry and Management

Image Creation, Management and Registry carries about 20% weightage of your overall mark in a DCA test. All Docker containers are based on images, which are the building blocks of containerized applications. An image is, in fact, the executable package containing all components you’ll need to run your application.

For the Docker Certified Associate Exam, the content will include:

Dockerfile Options
Creating Images using Dockerfile
Image Management using CLI commands
Docker Image Layers
Deploying, Configuring and Logging into Registry
Pushing, Signing and Pulling Images from the Registry
Image Deletion
Tagging Images

Installation and Configuration

This is considered to be the most crucial part of the entire DCA learning. Though Installation and Configuration accounts for 15% of your total score, it should be noted that in a real-world a thorough knowledge of these concepts would come in handy almost regularly.

Content covered in Installation and Configuration includes:

Upgrading the Docker Engine
Installing the Docker Engine on Various Platforms
Logging Drivers
User and Team Creation, User Management
Sizing Requirements
Client-Server Authentication for Image Registry Access

As an essential part, you will also familiarize yourself with the Docker Universal Control Plane (UCP), Docker Daemon and the Docker Trusted Registry (DTR).

Security & Networking

Networking and Security each carry 15% of the total score weightage. Networking in Docker involves connecting containers using Network Drivers. To fully grasp networking for the DCA exam, you’ll have to understand concepts such as:

Building Docker Bridge Networks for developer use
Troubleshooting logs
Publishing application ports
Identifying container ports and IP addresses
Describing the various types of network drivers
Configuring the Docker engine to use an external DNS
Performing HTTP HTTPS load-balancing
Types of traffic on Docker Networks
Deploying services on Docker Networks.

The DCA security chapter explores all content relating to authentication, encryption and transport layer security. This chapter will include:

Ensuring images pass security scans
The process of signing images
Docker Content Trust
Docker Engine Security
Swarm Security
Distinguishing UCP workers from managers.
Mutual Transport Layer Security (MTLS)
Using External Certificates with the Docker Universal Control Plane

Storage and Volumes

This chapter carries about 10% of your total exam score. Volumes offer a way to store information in Docker. For the DCA Associate exam, it is expected that you develop an understanding of:

How to quickly create volumes
The differences between volumes and bind mounts
Volume drivers and their most suitable use-cases
Use of the devicemapper
Object storage vs. block storage
Filesystem layers
Persistent storage in Docker
Cleanup of unused images
Storage in cluster nodes.

Docker Enterprise Edition

The Docker Enterprise Edition (EE) is created for applications with mission-critical deployments. This gives you a managed solution, complete with advanced container management, security scanning and application logging & monitoring. This version can be deployed on all major Server operating systems, including Red Hat Enterprise Linux (RHEL), Ubuntu, Oracle Linux, Windows Server 2016 and SUSE Linux Enterprise Server (SLES). It is also available for major cloud providers, including Azure and AWS.

Exam Preparation

Certification Format

The Docker Certified Associate Exam is of one and a half hours and consists of 55 questions, including 44 Discrete Option Multiple-Choice (DOMC) and 11 Multiple-Choice (MCQ) questions. In DOMC, options are randomly shown at a time for the examinee to choose a YES or a NO. On the other hand, in a MCQ question there are multiple correct answers, all of which the examinee has to select discreetly. This exam is proctored by Examity, and you can register by clicking this link. While there are no prerequisites, it is recommended that you should have used Docker for 6-12 months to be fully prepared for the exam. The exam fee is $195 and there are no free retakes if you flunk. You may however reschedule the exam prior to taking the test, so don’t feel pressured to take the exam unless you are completely ready.

Docker Certified Associate (DCA) Study Plan

As you prepare to study for your DCA exam, it is best to plan well to make sure that you do not miss any important topics, while ensuring that you do not get overwhelmed with the amount of knowledge flowing in.

To plan well, you may divide the entire study into three parts:

The first is a lab setup that you can use for practise demos. This could either be a local Docker Command Line Interface (CLI), an on-cloud platform like AWS (if you have a subscription), or an Online Playground that emulates the Docker CLI.
Second consideration as part of your plan should include a set of practice exams that lets you acquaint yourself with Discrete Option Multiple Choice (DOMC) and Multiple Choice Question (MCQ) exam formats used in the exam. To help with this, KodeKloud provides research questions, practice tests and mock exams in both MCQ and DOMC format that helps you get familiarised with the certification exam.
Lastly, but most importantly, you should plan to consider helpful resources for meaningful research. Through KodeKloud’s practise lesson lectures, you can get in-depth understanding of the DCA curriculum in a structured schedule. Such lectures also act as great resources to understand various Docker commands, options and tips which often act handy for a thorough understanding of Docker. Ensure that your approach to learning is more practical than theoretical that helps solve real-world problems. As such, some questions in the DCA exam will check your knowledge on commands, command options, and shortcuts.

As the ultimate focus, your plan should be to gather knowledge that helps you gain working knowledge of Docker in a practical world.

Study Schedule

The Study Schedule above indicates various subject areas you’ll need to cover, categorized into sections with estimated study times based on studying speeds.

The topics - Docker Architecture, Docker Swarm, Kubernetes and Images, carry the bulk of the work, each taking over 20 hours to study. While the topics on Security, Networking, and Disaster Recovery roughly require eight hours for certification level expertise. As part of the Study Schedule, you will also take several mock exams covering each of the subjects, requiring up to twenty eight hours of your time.

In total, to gain expert level knowledge helping clear the DCA certification, we estimate that you’ll need to practice for 3 months if you’re learning for two hours a day, 1.5 months studying four hours a day and one month studying 6 hours a day.

Sample Questions

Here is a quick quiz to help you assess your knowledge. Leave your answers in the comments below and tag us back.

Quick Tip - Questions below include a mix of DOMC and MCQ types.

Which statement best describes Quorum?

[A] Quorum is the minimum number of nodes that must be available for the cluster to function properly.
[B] In the case of 3 manager nodes, the quorum is 3
[C] one of the best practises; maintain an odd number of managers in the swarm to support manager node failures.

Which of the below is a recommended best practice while taking backups of a swarm cluster?

[A] Perform the backup operations from a swarm manager node that is a leader
[B] Perform the backup operations from a swarm worker node
[C] Perform the backup operations from a swarm manager node that is not a leader

Which of the following steps are required to add a worker node in the UCP cluster?

[A] Provision a node and Install Docker enterprise engine on it.
[B] Run the docker swarm join command to join the new node to the cluster.
[C] Deploy an instance of the ucp-agent on the new node.
[D] ucp-agent then installs the necessary components on the worker node

What will happen if the container consumes more memory than its limit?

[A] the container will not be killed
[B] the container will be killed with an Out of Memory exception -[C] the container’s memory usage will be throttled

How does docker map a port on a container to a port on the host?

[A] Using an internal load balancer
[B] FirewallD Rules
[C] Using an external load balancer
[D] IPTables Rules

Which of the following solutions support network policies?

[A] kube-router
[B] Calico
[C] Flannel
[D] Weave-Net

Which command can be used to stop (only and not delete) the whole stack of containers created by compose file?

[A] docker-compose down
[B] docker-compose stop
[C] docker-compose destroy
[D] docker-compose halt

What is the command to run 3 instances of httpd on a swarm cluster?

[A] docker swarm service create --instances=3 httpd
[B] docker swarm service create --replicas=3 httpd
[C] docker service create --instances=3 httpd
[D] docker service create --replicas=3 httpd

In which service does the DTR image scanning occur?

[A] A service known as the dtr-jobrunner container
[B] A service known as the dtr-registry container
[C] A service known as the dtr-api container
[D]A service known as the dtr-runner container

Assume that you have 3 managers in your cluster, what will happen if 2 managers fail at the same time? Select the all right answers.

[A] The services hosted on the available worker nodes will continue to run.
[B] The services hosted on the available worker nodes will stop running.
[C] New services/workers can be created or added.
[D] New services/workers can’t be created or added.

Study Resource

KodeKloud's Docker Certified Associate Exam Course

Docker Certification Readiness Test

To assess where you stand in your certification journey, take this Docker Certification Readiness Test

PODS in Kubernetes for Developers

KodeKloud — Wed, 23 Sep 2020 06:08:56 +0000

Here, we will take a look at PODS.

POD introduction
How to deploy a pod?

Kubernetes doesn't deploy containers directly on the worker node.

The containers are encapsulated into a Kubernetes object called POD.
A POD is a single instance of an application.
A POD is the smallest object that you can create in Kubernetes

Here is a single-node Kubernetes cluster with a single instance of your application running in a single docker container encapsulated in the pod.

A Pod usually has a one-to-one relationship with containers running your application.

To scale up, you create a pod, and to scale down, you delete a pod.
You do not add additional containers to an existing POD to scale your application.

Multi-Container PODs

A single pod can have multiple containers except for the fact that they are usually not multiple containers of the same kind.
Sometimes you might have a scenario where helper containers that might be doing some kind of supporting tasks for a web application such as processing a user-entered data, processing a file uploaded by the user, etc. and you want these helper containers to live alongside your application container. In that case, you can have both of these containers part of the same POD.

How to deploy pods?

Let’s now take a look to create an Nginx pod using kubectl.

To deploy a docker container by creating a POD.
$ kubectl run nginx --image nginx

To get the list of pods
$ kubectl get pods

K8s Reference Docs:

https://kubernetes.io/docs/concepts/workloads/pods/pod/
https://kubernetes.io/docs/concepts/workloads/pods/pod-overview/
https://kubernetes.io/docs/tutorials/kubernetes-basics/explore/explore-intro/

Access certified Kubernetes administrator course with practice tests

DevOps: Git for Beginners

KodeKloud — Wed, 19 Aug 2020 11:25:58 +0000

Software development and related methodologies have come a long way with the advent of Agile, Lean, and DevOps. Now, it is all about Automation - frequent and faster releases in small chunks, so the features and software reach the target audience in a much quicker and more efficient way. DevOps has become a center stage, and every software developer wants to learn more about modern software development approaches and tools. One such tool is 'Git,' and is a must-know for every developer out there.

Git is a must for anyone who writes code or is involved in a DevOps Project. In this article, we will discuss what Git is and other concepts related to it.

What is Git?

Git was developed by Linus Torvalds, the creator of Linux Operating System. Git is a Version Control System (VCS), which is more commonly used than the alternatives. On a fundamental level, there are two remarkable things a VCS allows you to do:

Track changes you make to your files
It enhances collaboration by simplifying working on projects with multiple people and teams.

Git is a software that runs locally on the developers' machine. The files and history are stored on the developers' computer. Developers can also use online hosts such as GitHub and Bitbucket, to save a copy of their files and revision history. A central place to upload changes and download changes from others enables collaboration easily between the teams.

Git can automatically merge the changes; that way, two developers can work on different parts of the same file and later merge the changes without waiting for each other and losing each other's work.

Why GIT?

Software development involves a team of developers working together on the same code base. To avoid code conflict between these developers, we need a centralized version control system like Git. Git helps developers go back to older versions of the code base to make revisions such as fix bugs or revert code changes. With branches Git helps developers work on multiple feature implementations or bug fixes in parallel and later merge those changes in when ready..

With Git, you will be able to see what others are working on, review their code, view your previous changes, roll back to previous code, and do much more.

How to get Git?

Git usually is installed by default on many systems.

You can download git for any operating system here.
Try GitHub Desktop (for Windows and Mac) if you like to use a graphical user interface (GUI).
Well, if you want to download it from scratch, this link has details on installing Git on multiple operating systems.

Git key concepts

Git key terminologies include:

Version Control System:Git helps maintain various versions of the code base at different stages of the development lifecycle. It is also called a source code manager.
Commit: When a developer makes code changes, the changes are saved in the local repository. Every commit saves a copy of the changed/added files within Git.
Push - This sends the recent commits from developers' local repository to a remote server like GitHub, GitLab or BitBucket.
Pull - This downloads any changes made from the remote Git repository and merges them into the developers' local repository.
SHA (Secure Hash Algorithm): This is a unique ID given to each commit.
Branch: When you diverge from the main line of software development and continue to work/code without messing with the main/master development line.

Role of GIT in DevOps

The DevOps approach needs a version control system that will track all the changes. Git is a distributed version control system that allows developers to keep a local copy of the commits they make. GIT as a DevOps tool empowers collaboration and faster release cycles, and that is what the DevOps concept is based upon.

Version control is one of the best practices of DevOps. With version control, developers working on a project have the ability to version the software, share, collaborate, merge, and have backups.

When working in large organizations, where multiple teams work together on the same project, Git comes handy and makes it easy to track changes made by each team. It helps in tracking code, version control, and effective management of code.

Anyone willing to start or approach DevOps as a career should start from the basics, and GIT is the fundamental tool that overrides everything else.

Most popular Git solutions

Every company is now powered by software in one or the other ways. There are multiple projects handled by many developers in an organization, and they all need a means to track, upload, and receive changes to the code base. Effective repository management services are the key to fast and efficient software development. The most popular ones based on Git are GitHub, Bitbucket, and GitLab.

GitHub is a git-based repository host launched initially in 2008 by PJ Hyatt, Tom Preston-Werner, and Chris Wanstrath. As of now, GitHub is the largest repository hosting platform with more than 38 million projects. It authorizes to host and review code, manage projects, and build software.

Bitbucket was also launched in 2008 by an Australian startup and it initially supported only for Mercurial projects. In 2010 Bitbucket was smartly bought by Atlassian, and from the next year, it started supporting Git hosting, which is now its primary focus. Bitbucket has become a household name and provides free unlimited private repos, many powerful integrations like Jira and Trello, and has built-in continuous delivery.

GitLab started as a small project in 2011, aiming to provide an alternative to the available repository management solutions. The company was only incorporated in 2014. Gitlab now provides a complete DevOps setup for the organizations from continuous integration and delivery, agile development, security etc.

Here is a basic Git cheat sheet you would love to have.

Basic Git commands

Install git
yum install git
To see the version of git installed
git version
To initialize a git repository
git init
To check the status of the git repository
git status
To track all the files except notes.txt file
git add LICENSE README.md main.py ...
To stage changes
git add main.py
To commit changes
git commit -m "initial commit"
To copy a repository
git clone username@host:/path/to/repository
To set user-specific configuration values like email, username, file format, and so on
git config --global user.email youremail@example.com
To view all remote repositories
git remote -v
To create, or delete branches
git branch

Git is the most basic requirement in the software development and DevOps field.

This video will walk you through the basics of Git.

Difference between Git and GitHub

One of the most common questions we get asked are the differences between Git and Github. Git is the tool/technology that allows versioning of code and Github is the centrally hosted Git server where the Git repository lives. Developers use the remote repository hosted on Github to share their work among themselves. Github is just one of the many tools such as Gitlab, BitBucket etc.

Image source credits: TheServerSide

Myth: Git is only for Software Developers

Git is not just for developers. Git is for anyone working in IT - from Systems Administrators, to Solutions Architects, Software Engineers, Team leads, Project managers - should equally learn and understand the workflows in Git.

With the advent of Infrastructure as Code, System administrators and operations teams will be using Git to store code related to infrastructure such as Terraform Configuration files, Ansible Playbooks, Vagrant files, supporting shell scripts.

Myth: GitHub is just a code base/repository

Git is not just a code repository. Git is where new software is developed. Today most of the popular projects are developed on Github - Kubernetes, Ansible, Terraform, Tensorflow, Helm Charts being a few of the top repositories. All of these projects have extensive documentation built on Github. The Git workflow was built with collaboration between developers in mind. Code reviews and approvals happen on Github. Github has a Project dashboard that enables project management.

Checkout our new course on Git for Beginners here

Learn Shell Scripting

KodeKloud — Fri, 10 Jul 2020 07:14:15 +0000

We just launched our latest course on Shell Scripting for Beginners. If you want to increase productivity by automating daily repetitive tasks in Linux, then this course is for you. This is for those who have always wanted to learn shell scripting but didn't have sufficient coding or programming experience to get through it.

This course is a beginner level course and is for those who are absolute beginners to shell scripting or programming. System Administrators, Developers or IT engineers who do not have any prior programming experience can go through this course to gain basic knowledge of shell scripting. As part of this course we will explain the necessary programming concepts required for shell scripting.

Learn by doing!

And we learn in a fun way using examples of a space station launching missions to explore the universe. As well as our embedded hands-on labs that will make sure you gain enough hands-on practice right after you learn each concept. We will test your scripts to make sure you have written them correctly and also provide feedback.

This course is divided into different sections where we discuss:

What shell scripts are
Getting started with your first shell script.
Making your scripts executable.
Variables
Conditional statements
For loops
While loops
Arithmetic operations
Best practices

Project

Automate the Deployment of a 2 tier application just with Shell Scripts. Watch me do it following best practices.

And throughout the course we will also discuss best practices while scripting such as what to do and what not to do and how to develop a script that’s reusable. We will also see some tips and tricks such as some IDEs and utilities that can help you improve your scripting skills.
Check it out and let me know what your thoughts.

Shell Scripts for Beginners

Scripting for Beginners

KodeKloud — Tue, 07 Jul 2020 09:19:57 +0000

The scripting language is a language where instructions are provided for a run time environment. Scripting languages are an integral part of the engineering teams in enterprises. They are often used in many areas, other than the server-side and client-side applications; scripting languages are well suited in system administration. Some notable examples of scripts used in system administration include Shell, Perl, and Python.

With shell scripting, you can automate the tasks which consume a lot of time. Shell scripting is a way for you to save a lot of time and focus on the things that matter rather than wasting time on unnecessary time consuming repetitive tasks.

History of the Shell

There was a shell program by the name the V6 Shell starting with Unix in the 1970s developed by Ken Thomson. It lacked scripting ability for developers and was then followed by the Bourne Shell in 1977, and it is still in use as the default shell for the root account; this added scripting abilities that proved extremely useful for developers.

Further, there were many improvements made in the 1980s and gave rise to many popular shell variants like C-Shell and Korn Shell, the important ones. These Shell scripts had their own syntax, which was, in some instances, completely different from the original shell. Bash Shell is the most popular shell today, and Bash stands for Bourne-Again-Shell and is the finest improved variant of the original Bourne Shell.

What are Shell scripts?

Shells are interactive, they accept the command as an input from users and execute them. Shell can also take commands from a file; we can write these desired commands in a file and can easily execute them in shell to avoid repetitive work. Such files are called Shell Scripts or also called Shell Programs. Shell scripts are more or less similar to the batch file in MS-DOS. Every shell script is saved by .sh file extension. For example, myscript.sh
Just like any other programming language, a shell script has syntax. It would be very easy and effortless to get started if you have prior experience with programming languages like C/C++, Python, etc.
A shell script involves the following basic elements –
Shell Keywords – if, else, break, etc.
Shell commands – cd, ls, echo, pwd, touch etc.
Functions
Control flow – if..then..else, case and shell loops, etc.
When it comes to server-side, the scripting languages include JavaScript, Perl, PHP, etc. and client-side scripting languages include JavaScript, jQuery, AJAX, etc. Scripting languages are mainly used in system administration and by developers to automate their day to day repetitive tasks. In such cases, languages like Shell, Python scripts, and Perl, etc help them a lot.

Here are the few reasons why you should use the scripts:

To avoid manual work, to initialize something at boot time of the system, you can write the script, and it saves a lot of time.
To install pre-requisite and to build the code with user input to enable/disable some features, you can write a script that does all the work for you.
Anything from killing or starting multiple applications together, scripts come handy.
Even when you have to observe a large database of files, analyze and find some patterns out of it, you can use scripts.
To automate any mundane task in your day to day activities, you can write scripts.
According to the recent Stack Overflow survey, Bash/Shell/PowerShell is one of the languages that is associated with the highest salaries around the world.

Image credits: Stack Overflow survey
Bash/Shell/PowerShell is among one of the top 10 most popular technologies as per the Stackoverflow insight data derived from over 80K responses in the recent survey.

Image credits: Stack Overflow insights data

Scripting language use cases and examples

Scripting is widely used by developers to automate tasks within an operating system, to enhance web pages within browser software, etc.

Javascript is a scripting language that can be used to inject logic into the webpages and they don’t need to be compiled like Java or C.

Python is a scripting language that can be used to run any kind of automation with few lines of code and that is one reason why DevOps people think Python fits really well into the DevOps space.

These days, scripting languages are generally associated with web development, where they are extensively used to make dynamic web applications that result in smooth user experience.

WordPress sites are very good examples of where scripting languages come into action. A PHP script makes it simply possible to have your three or four latest blog posts automatically appear on a website’s homepage.

The visitor viewing the site will not have any clue and doesn’t see the script or its backend process; they just see the end result, a smooth user experience. Rather than hand-coding each and every single instance and outcome of a dynamic function, the site’s developer is able to implement such features with only a one-time set of instructions.

Another modern example can be, say there is an automated deploy task you want to run on a GitHub repository, bash script can be used inside GitHub actions. When a commit happens, it can ssh into a machine, fetch the latest code, and serves it.

Conclusion

Scripting languages are more crucial than ever. DevOps practitioners are required to know a scripting language to save themselves some valuable time. The automation world mostly relied on scripting languages and is heavily used in software development and making things fast.

Learning a scripting language is the best way to get your hands dirty in the coding world as well as in the DevOps industry. There is a scarcity of talented DevOps professionals who know the scripting languages. Just remember, the scripting language is a must and will be there until there is software running the world.

People might assume that they need an in-depth knowledge of languages, such as Python or C, or even Java, for higher functionalities, but that’s not fundamentally true. The Bash scripting language is sometimes more than enough, it is compelling. There is a lot to learn to maximize its worth and usefulness when it comes to system administration and DevOps.

Learn more about Shell Scripting in our ‘Shell Scripts for Beginners’ course!

CI/CD with Docker for Beginners

KodeKloud — Tue, 23 Jun 2020 10:01:33 +0000

Docker is a DevOps platform that is basically used to create, deploy, and run applications using the concept of containerization. With Docker, developers can pack all the dependencies and libraries of an application easily and ship it out as a single package.

This helps developers and operations teams mitigate the environmental issues that used to happen before. Developers can now be able to focus more on the features and deliverables than being concerned about the infrastructure compatibilities and configurations aspect of the platform. Further, this promotes the microservices architecture to help teams to build highly scalable applications.

Why Docker?

Docker is an open-source project that has changed how software is built and shipped by providing a feasible way to containerize applications. Over the last few years, this has resulted in a lot of enthusiasm around containers in all stages of the software delivery lifecycle, from development to testing to production. Docker has become a very mainstream platform in a short time since its debut in 2013. The big giants like Amazon, Cisco, Google, Microsoft, Red Hat, VMWare, and others have created the Open Container Initiative to develop a common standardization around it.

Here's an overview of a few commonly used commands.

Image source: Tania Rasciai

Here are some key benefits of using Docker

You get a high level of control over all the changes because they are made using Docker containers and images. Thus, you can return back to the previous version whenever you want to.
With Docker, you get a guarantee that if a feature is working in one environment, it will work in others as well.
Docker, when used with DevOps, simplifies the process of creating application topology embodying various interconnected components.
It makes the process of load balancing configuration easier with Ingress and built-in service concepts.
It enables you to run CI/CD using them, which is more comfortable to use when compared to just using it with Docker.

Read our full article on the same 'The role of Docker in DevOps'

Docker can be a common interface tool between developers and operations personnel, as stated in DevOps principles, eliminating a source of friction between the two teams. It also promotes the same image/binaries to be saved and used at every step of the pipeline throughout. Moreover, being able to deploy a thoroughly tested container without environment differences is the most significant advantage, and it ensures that no errors are introduced in the build process.

You can simply and seamlessly migrate applications into production, eliminating all the friction in between. Something that was once a tedious task can now be as simple as:

docker stop container-id; docker run new-image

And in case if something goes wrong when deploying a new version of the application, you can always roll-back quickly or change to other container:

docker stop container-id; docker start other-container-id

Now, we will learn about how Docker integrates with CI/CD pipeline. Let us now look at how Docker plays a key role in the CI/CD pipeline. To begin with, Docker is supported by a majority of the build systems like Jenkins, Bamboo, Travis, etc. So how it works typically is that each project has a Docker file checked into its code repository along with the rest of the code for the application. The Docker file as we learned before, has instructions on building the Docker image. Once checked-in to GitHub, Jenkins pulls the code, uses the Docker file part of the code to build the Docker image.

You may use a supported Docker plugin for this purpose. On building the new Docker image, Jenkins will tag the image with a new build number, in this case, 1.0. On successful build, this image can then be used to run tests.

Once the tests are successful, it can then be pushed to the image repositories known as Docker registries, either to a repository internal to the company or external on Docker Hub. The image repository can then be integrated into a container hosting platform like Amazon ECS to host our application. This entire cycle of automated actions from making a change in the application to building, testing, releasing and finally deploying in production, completes a CI/CD pipeline.

The next step is to deploy this image in production. Major cloud service providers like Amazon, Google, Azure, all support containers. Google Container Engine supports running containers in production on Kubernetes clusters.

Kubernetes is a container orchestration technology which is an alternate solution to Docker swarm that we learned earlier. AWS has ECS, which stands for EC2 Container Service, it provides another mechanism to run containers in production.

On-prem solutions like Pivotal Cloud Foundry have PKS which stands for pivotal container service, which again, uses Kubernetes underneath. Finally, Docker’s own container hosting platform, Docker cloud uses Docker swarm underneath to orchestrate containers. As you can see containers and Docker are supported everywhere and there are many options to host containers online.

Conclusion

Because of the efficiencies of virtualizing the OS, containerization allows for a much larger scale of applications in virtualized environments. In development and testing, the applications can be built, tested, and deployed much more quickly. The use of containers is definitely booming and has been adopted by many companies so far.

With the help of Docker containers, once an application is containerized, developers will be able to deploy the same container in a different environment. As the Container remains the same, the application will be running identical in all environments without causing any dependency confusion.

Containers and Docker provide developers the freedom they want, as well as ways to build scalable apps that respond and change quickly to the ever-changing business conditions. It is evident from its adoption by many huge and small startups that Docker will continue to gain more fans, grow, and become of greater importance in the DevOps space.

Get Your Free Docker Course Here

Kubernetes Concepts Explained for Developers

KodeKloud — Wed, 10 Jun 2020 07:06:19 +0000

Image source: Platform9

With Docker, you can run a single instance of the application with a simple Docker run command. In this case, to run a Node JS based application, you run the docker run nodejs command. But that's just one instance of your application on one Docker host. What happens when the number of users increase, and that instance is no longer able to handle the load, you deploy an additional instance of your application by running the Docker run command multiple times. So that's something you have to do yourself, you have to keep a close watch on the load and performance of your application and deploy additional instances yourself. And not just that, you have to keep a close watch on the health of these applications.

Container Orchestration

And if a container was to fail, you should be able to detect that and run the Docker run command again to deploy another instance of that application. What about the health of the Docker host itself? What if the host crashes and is inaccessible?

The containers hosted on that host become inaccessible too. So what do you do in order to solve these issues, you will need a dedicated engineer who can sit and monitor the state performance and health of the containers and take necessary actions to remediate the situation. But when you have large applications deployed with 10s of thousands of containers, that's, that's not a practical approach. So you can build your own scripts and that will help you tackle these issues to some extent. container orchestration is just a solution for that. It is a solution that consists of a set of tools and scripts that can help host containers in your production environment.

Typically, a container orchestration solution consists of multiple Docker hosts that can host containers. That way even if one fails, the application is still accessible through the others. The container orchestration solution easily allows you to deploy hundreds or thousands of instances of your application with a single command.

Some orchestration solutions can help you automatically scale up the number of instances when users increase and scale down the number of instances when the demand decreases. Some solutions can even help you in automatically adding additional hosts to support the user load. And, not just clustering and scaling, the container orchestration solutions also provide support for advanced networking between these containers across different hosts. As well as load balancing user requests across different hosts. They also provide support for sharing storage between the host, as well as support for configuration management and security within the cluster.

Orchestration solutions

There are multiple container orchestration solutions available today - Docker has Docker swarm, Kubernetes from Google and Mesos from Apache. Well, Docker swarm is really easy to set up and get started. It lacks some of the advanced auto-scaling features required for complex production-grade applications. Mesos on the other hand is quite difficult to set up and get started but supports many advanced features.

Kubernetes, arguably the most popular of all is a bit difficult to set up and get started but provides a lot of options to customize deployments and has support for many different vendors. Kubernetes is now supported on all public cloud service providers like GCP, Azure, and AWS, and the Kubernetes project is one of the top-ranked projects on GitHub. With Docker, you were able to run a single instance of an application using the Docker CLI by running the Docker run command, which is great, running and application has never been so easy before. With Kubernetes. Using the Kubernetes CLI, known as kubectl, you can run a 1000 instances of the same application with a single command.

Kubernetes can scale it up to 2000 with another command, Kubernetes can be even configured to do this automatically so that instances and the infrastructure itself can scale up and down based on user load. Kubernetes can upgrade these 2000 instances of the application in a rolling upgrade fashion, one at a time with a single command. If something goes wrong, it can help you roll back these images with a single command. Kubernetes can help you test new features of your application by only upgrading a percentage of these instances through AB testing methods.

The Kubernetes open architecture provides support for many different network and storage vendors. Any network or storage brand that you can think of has a plugin for Kubernetes. Kubernetes supports a variety of authentication and authorization mechanisms. All major cloud service providers have native support for Kubernetes.

The relation between Docker and Kubernetes

Kubernetes uses Docker host to host applications in the form of Docker containers. Well, it need not be Docker all the time. Kubernetes supports as native to Dockers as well such as rocket or a crier but let's take a quick look at the Kubernetes architecture. A Kubernetes cluster consists of a set of nodes.
Let us start with nodes. A node is a machine physical or virtual on which the Kubernetes software setup tools are installed. And node is a worker machine and that is where containers will be launched by Kubernetes. But what if the node on which the application is running fails. Well, obviously, our application goes down. So you need to have more than one node.

A cluster is a set of nodes grouped together. This way, even if one node fails, you have your application still accessible from the other nodes. Now, we have a cluster but who is responsible for managing this cluster? Where is the information about the members of the cluster stored? How are the nodes monitored? When a node fails? How do you move the workload of the failed nodes to another worker node? That's where the master comes in. The Master is a node with the Kubernetes control plane components installed.

The master watches over the nodes in the cluster and is responsible for the actual orchestration of containers on the worker nodes.

When you install Kubernetes on a system, you're actually installing the following components, an API server, Etcd server, a Kubelet service, contain runtime, engine like Docker, and a bunch of controllers and the scheduler.

The API server acts as the front end for Kubernetes. The users, management devices, command-line interfaces, all talk to the API server to interact with the Kubernetes cluster. Next is the Etcd key-value store. The Etcd is a distributed reliable key-value store used by Kubernetes to store all data used to manage the cluster. Think of it this way, when you have multiple nodes and multiple masters in your cluster. Etcd stores all that information on all the nodes in the cluster in a distributed manner. Etcd is responsible for implementing logs within the cluster to ensure there are no conflicts between the masters.

The scheduler is responsible for distributing work or containers across multiple nodes. It looks for newly created containers and assigns them to nodes. The controllers are the brain behind orchestration, they are responsible for noticing and responding when nodes containers or endpoints goes down. The controllers make decisions to bring up new containers in such cases.
The container runtime is the underlying software that is used to run containers, In our case, it happens to be Docker. And finally, Kubelet is the agent that runs on each node in the cluster. The agent is responsible for making sure that the containers are running on the nodes as expected.

And finally, we also need to learn a little bit about one of the command line utilities known as the Kube command-line tool or the Kube control tool or Kube cuddle as it is also called the Kube control tool is the Kubernetes CLI, which is used to deploy and manage applications on a Kubernetes cluster to get cluster related information, to get the status of the nodes in the cluster, and many other things. The kubectl run command is used to deploy an application on the cluster, the Kube control cluster info command is used to view information about the cluster. And the kubectl get nodes command is used to list all the nodes part of the cluster. So, to run hundreds of instances of your application across hundreds of nodes, all I need is a single Kubernetes command like this.

Well, that's all we have for now. A quick introduction to Kubernetes and its architecture. We currently have four courses on KodeKloud on Kubernetes. That will take you from an absolute beginner to a certified expert. So have a look at it.

Kubernetes for the absolute beginners

Certified Kubernetes Administrator(CKA)

Kubernetes - Absolute beginner to expert

Certified Kubernetes application developer(CKAD)

Docker For Absolute Beginners

KodeKloud — Sun, 31 May 2020 07:02:42 +0000

Every company is dependent on software for innovation and one of the biggest innovations in the software development field was the invention of containers. The containers have changed the way the software is built and shipped these days. The company Docker took this route and made containers available for everyone. It has made developers worry-free because of its amazing features. Docker also enables the adoption of DevOps in enterprises by eliminating the gap between Dev and Ops teams that often used to get into conflicts.
Today, we will go through some fundamentals of Docker and more.

Docker Containers

We're now going to look at a high-level overview of why you need Docker, and what it can do for you. Let me start by sharing how I got introduced to Docker. in one of my previous projects, I had this requirement to set up, an end to end stack including various different technologies, like a web server using node js, and database such as MongoDB, a messaging system like Redis, and an orchestration tool like Ansible. We had a lot of issues developing this application with all these different components. First, the thing to be taken care of was their compatibility with the underlying operating system, we had to ensure that all these different services were compatible with the version of the operating system we were planning to use.

There have been times when certain versions of these services were not compatible with the OS. And we've had to go back and look for another OS that was compatible with all these different services. Secondly, we had to check the compatibility between the services and the libraries and dependencies on the OS. We've had issues where one service requires one version of a dependent library, whereas another service requires another version.

The architecture of our application changed over time, we've had to upgrade to newer versions of these components or change the database, etc. And every time something changed, we had to go through the same process of checking compatibility between these various companies and the underlying infrastructure. This compatibility matrix issue is usually referred to as the matrix from hell.

Next, every time we had a new developer on board, we found it really difficult to set up a new environment, the new developers had to follow a large set of instructions and run hundreds of commands to finally set up their environment. They had to make sure they were using the right operating system, the right versions of each of these components. And each developer had to set all that up by himself each time. We also had different development tests and production environments.

One developer may be comfortable using one OS and the others may be using another one. And so we couldn't guarantee that the application that we were building would run the same way in different environments. And so all of this made our life really difficult.
So I needed something that could help us with the compatibility issue, something that will allow us to modify or change these components without affecting the other components and even modify the underlying operating system as required. And that search landed me on Docker.

With Docker, I was able to run each component in a separate container with its own libraries and its own dependencies, all on the same VM and the OS, but within separate environments or containers. We just had to build the Docker configuration once and all our developers could now get started with a simple Docker run command, irrespective of what the underlying operating system they run. All they needed to do was to make sure they had Docker installed on their systems.

So what are containers?

Containers are completely isolated environments. As in they can have their own processes or services, their own networking interfaces, their own mounts just like virtual machines, except they're all shared the same operating system kernel. We will look at what that means in a bit. But it's also important to note that containers are not new with Docker.

Containers have existed for about 10 years now. And some of the different types of containers are LXC, LXD, LXCFS, etc. Docker utilizes LXC containers. Setting up these container environments is hard as they are very low level. And that is where Docker offers a high-level tool with several powerful functions, making it really easy for end users like us.

How does Docker work?

To understand how Docker works, let us revisit some basic concepts of operating systems First. If you look at operating systems like Ubuntu, Fedora, CentOS, etc. They all consist of two things, an OS kernel and a set of software. The operating system kernel is responsible for interacting with the underlying hardware, while the OS kernel remains the same, which is Linux in this case, it's the software above it that makes these operating systems different.

The software may consist of a different user interface drivers, compilers, file managers, developer tools, etc. So you have a common Linux kernel shared across all operating systems and some custom software that differentiates operating systems from each other.
We said earlier that Docker containers share the underlying kernel. What does that actually mean, sharing the kernel? Let's say we have a system with an Ubuntu OS with Docker installed on it. Docker can run any flavor of OS on top of it. As long as they're all based on the same kernel, in this case Linux. if the underlying operating system is Ubuntu Docker can run a container based on another distribution like Debian, Fedora, Susi or CentOS. Each Docker container only has the additional software that we just talked about previously which makes these operating systems different.

And Docker utilizes the underlying kernel of Docker host, which works with all the operating systems above. So what is an OS that did not share the same kernel as these, windows. And so you won't be able to run a Windows-based container on a Docker host with Linux OS on it. For that you would require Docker on a Windows Server.

You might ask isn't that a disadvantage then? Not being able to run another kernel on the OS? The answer is no.
Because unlike hypervisors Docker is not meant to virtualize and run different operating systems and kernels on the same hardware. The main purpose of Docker is to containerize applications and to ship them and run them.

Differences between virtual machines and containers

So that brings us to the differences between virtual machines and containers, something that we tend to do, especially those from a virtualization background. As you can see on the right, in the case of Docker, we have the underlying hardware infrastructure, then the operating system and Docker installed on the OS. Docker can then manage the containers that run with libraries and dependencies alone.

In the case of a virtual machine, we have the OS on the underlying hardware, then the hypervisor like ESX or virtualization of some kind, and then the virtual machines. As you can see, each virtual machine has its own operating system inside it. Then the dependencies and then the application. This overhead causes higher utilization of underlying resources as there are multiple virtual operating systems and kernels running. The virtual machines also consume higher disk space as each VM is heavy, and it's usually in gigabytes in size, whereas Docker containers are lightweight, and they're usually in megabytes in size. This allows Docker containers to boot up faster, usually in a matter of seconds, whereas virtual machines as we know take minutes to boot up as it needs to boot up the entire operating system.

It is also important to note that Docker has less isolation as more resources are shared between containers like the kernel, whereas VMs have complete isolation from each other. Since VMs, don't rely on the underlying operating system or kernel. You can have different types of operating systems such as Linux based or Windows-based on the same hypervisor. Whereas It is not possible on a single Docker host. So these are some differences between the two.

So how is it done? There are a lot of containerized versions of applications readily available as of today. So most organizations have their products containerized and available in a public Docker registry called Docker Hub, or Docker store already. For example, you can find images of the most common operating systems, databases, and other services and tools. Once you identify the images you need, and you install Docker on your host. Bringing up an application stack is as easy as running a Docker run command with the name of the image.

In this case, running a Docker run Ansible command will run an instance of Ansible on the Docker host. Similarly run an instance of MongoDB Redis and node js using the Docker Hub command. When you run node js, just point to the location of the code repository on the host. If you need to run multiple instances of the web service, simply add as many instances as you need and configure a load balancer of some kind in the front. In case one of the instances wants to fail, simply destroy that instance and launch a new instance. There are other solutions available for handling such cases that we will look at later. We've been talking about images and containers.

Let's understand the difference between the two. An image is a package or a template just like a VM template that you might have worked with, in the virtualization world. It is used to create one or more containers. Containers are running instances of images that are isolated and have their own environments and set of processes.
As we have seen before, a lot of products have been Dockerized already. In case you cannot find what you're looking for, you could create an image yourself and push it to the Docker Hub repository, making it available for the public.

If you look at it, traditionally, developers developed applications. Then they hand it over to the Ops team to deploy and manage it in production environments. They do that by providing a set of instructions such as information about how the host must be set up, what prerequisites are to be installed on the host, and how the dependencies are to be configured etc.
The Ops team uses this guide to set up the application. Since the Ops team did not develop the application on their own, they struggle with setting it up. When they hit an issue, they work with the developers to resolve it. With Docker, a major portion of work involved in setting up the infrastructure is now in the hands of the developers in the form of a Dockerfile. The guide that the developers built previously to set up the infrastructure can now easily be put together into a Dockerfile to create an image for the applications.

This image can now run on any container platform, and it's guaranteed to run the same way everywhere. So the Ops team now can simply use the image to deploy the application. Since the image was already working, when the developer built it, and operations are not modifying it, it continues to work the same way when deployed in production. To learn more about containers, check out my other courses ‘Docker for the absolute beginners’

Watch the video on 'Docker for Beginners: Full Course' on YouTube.

What Makes Linux So Popular Among Developers

KodeKloud — Wed, 27 May 2020 07:18:05 +0000

Linux makes automation easy, hence, it has become an integral part of DevOps professionals. The best advice anybody can get while starting their journey in DevOps is to learn and understand the basics of Linux thoroughly. This makes the DevOps career path easy in the future. Linux is going to be there no matter what, you have to face it and work with it, to become a great DevOps engineer. Today, we will see why Linux is famous and reasons that make it so popular among developers and DevOps engineers.

What is Linux?

Linux is an operating system that is high performing and completely free and it closely resembles UNIX.
Linus Torvalds, a student from the University of Helsinki in Finland, started Linux in 1991 which was the inspiration of his dissatisfaction with MS-DOS and his strong desire to obtain a free version of UNIX for his new computer. Linux soon became a global project with so many developers supporting and contributing to the project via the internet. Slowly individual developers started using and experimenting with Linux and then the world saw heavy usage of Linux by corporations, educational institutions, and also governments.

Linux stats

In 2019, 100% of the world’s supercomputers ran on Linux.
Linux is once again the most loved platform for development in 2019 on StackOverflow developer survey results 2019.
Linux runs on all of the top 500 Supercomputers, again!.
96.3% of the world’s top 1 million web servers run on Linux.
Every Facebook post you make, every YouTube video you watch, every Google search you run, is done on Linux.
95% of the servers that run the world’s top 1 million domains are powered by Linux.
In 2018, Android dominated the mobile OS market with 75.16%.
85% of all smartphones are based on Linux.
Even the famous DevOps tools like Docker, Ansible and Kubernetes etc run with the help of Linux. For most of the years initially, Docker was only available on Linux based systems. A Linux system is required to be the Ansible controller. Also, Master nodes in Kubernetes can only be Linux systems. [Source: Hostingtribunal.com]

5 Reasons you should love Linux

1. Open source
The course code of Linux falls under FOSS (Free and Open Source Software) category and the developers, who are its members can always view and modify the source however they want. Around the world, several countries are developing their own version of Linux and this will help all such countries to strategically develop their own OSs for specialized and strategic areas such as defence, communications, government etc

2. Customization
With Linux, users get great flexibility in customizing the system as per their desire and requirements. The philosophy of Linux is based on several small and base programs, each of which does one task very well.
Linux provides a powerful command-line interface that helps system administrators to write shell scripts and automate routine and various repetitive tasks.

3. Free and easy to use
Linux is open-source and very simple to understand. This makes curious developers to go and see how this stuff works and experiment with it, this increases the developer adoption. Also, businesses can use this software free of cost and reduce their IT budgets considerably.
The GUI has developed and improved to such an extent that most of what typical users want can be done and executed on Linux.

4. Automation
Automation is the key for modern enterprises and that is one reason why companies hiring for DevOps professionals are asking for Linux knowledge.
Linux makes it much easier to efficiently carry the server-side work. Once the scripts are written to install server softwares ((MySQL, Apache, ssh, FTP, etc)), configure them, and maintain them, everything is taken care of automatically.

5. High endurance
The uptime and availability for Linux servers are very high. Linux has the highest number of servers running on the Internet. According to an article on the ZDNet website, 96.3 percent of the top 1 million Web servers are running on Linux. Twenty-three out of the Top twenty-five websites run on Linux. The two remaining websites in the top twenty-five are live.com and bing.com, which belong to Microsoft! [Source: Opensourceforu]

Why are companies eager to hire Linux people?

Linux continues to dominate employers’ needs. Today, Linux is the highest-ranked skill in software development and the job market. According to dice.com, the demand for Linux experts is so heavy that some companies are even allowing employees to write their cheques on their own.
The operating system capability is one of the great reasons why companies hire Linux experts. With the advancements in the cloud portfolio, firms are always in search of people that have skills with automation, private cloud, containers, orchestration, and server virtualization. We all know, these are the characters through which Linux is leading today's cloud industry.
DevOps is booming, and in DevOps, Linux knowledge is essential because it helps in automation. Firms look for such a combination of skills that fit well in the DevOps team.
IT professionals seeking job security, good pay, career advancement, or raises - Linux is leading the industry. Linux is a versatile, robust, and scalable solution for IT companies of any shapes and sizes.
Getting trained in Linux basics, tools, and implementation can guarantee your entry into the DevOps career, and Linux has been listed under the best-paying and most exciting jobs in the IT field right now.
Top corporates and governments use Linux
Over the past few years, there has been a massive growth in the number of Linux-based products that have had a significant impact on the cloud IT field:

Tools,

Kubernetes: Container orchestrating tool from Google, now part of CNCF
OpenStack: Software platform for infrastructure as a service cloud platform
OpenDaylight: Java-based project to help accelerate the adoption of SDNs and Network Functions Virtualization (NFV) by Linux Foundation.
Docker: Software containerization solution

Companies,

Amazon Web Service - A leading cloud platform that supports millions of enterprises power their infrastructure.
Sony PlayStation 4 runs on Orbis OS, and it is developed on a Linux-based kernel.
In 2019, IBM is in the process of acquiring Red Hat Enterprise Linux. An IDC report states that Red Hat is expected to contribute $10 trillion to the global economy. [Source: Hostingtribunal.com]

Linux is here to stay, and it has become a must-know skill for everyone who is working on DevOps. It is one of the tried and most trusted solutions in the history of computer software. Longevity, maturity, and high security make Linux one of the most advanced and trusted OSes available today. Linux is an ideal solution for enterprises that want to use it and its peripherals to customize their own network and data center infrastructure. Having Linux knowledge is a boon and makes it easy for people to enter the DevOps career and get paid well.

Our Linux Basics Course is now live

DEV Community: KodeKloud

CKS Challenges

Docker Certified Associate Exam Series (Part -2): Container Orchestration

Introduction to Container Orchestration

Three of the most popular Container Orchestration tools are Docker Swarm, Kubernetes, and MESOS.

For this part of the study guide series, we shall cover Docker Swarm.

Docker Swarm

Let’s explore some of these areas in detail:

Swarm Architecture

Some features of Docker Swarm Architecture include:

Setting up a 2-Node Swarm Cluster

Swarm Operations

Swarm High Availability- Quorum

The best practices for high availability in Swarm include:

Swarm Services

Here are some network ports and their purposes.

Swarm Service Discovery

Docker Stack

Task & Command

Docker Storage

Volume Driver Plugins in Docker

Sample Questions:

Which command can be used to remove a kubeapp stack?

Which command can be used to promote worker2 to a manager node? Select the right answer.

What is the command to list the stacks in the Docker host?

What is the maximum number of managers possible in a swarm cluster?

... are one or more instances of a single application that runs across the Swarm Cluster.

Conclusion

Docker Certified Associate (DCA) - The Ultimate Certification Guide for 2021

Why Get a Docker Certification?

Curriculum Covered in Docker Certification

Container Orchestration

Content break-up on container orchestration includes:

Docker Swarm/Kubernetes

Image Creation, Registry and Management

For the Docker Certified Associate Exam, the content will include:

Installation and Configuration

Content covered in Installation and Configuration includes:

Security & Networking

Storage and Volumes

Docker Enterprise Edition

Exam Preparation

Certification Format

Docker Certified Associate (DCA) Study Plan

Study Schedule

Sample Questions

Which statement best describes Quorum?

Which of the below is a recommended best practice while taking backups of a swarm cluster?

Which of the following steps are required to add a worker node in the UCP cluster?

What will happen if the container consumes more memory than its limit?

How does docker map a port on a container to a port on the host?

Which of the following solutions support network policies?

Which command can be used to stop (only and not delete) the whole stack of containers created by compose file?

What is the command to run 3 instances of httpd on a swarm cluster?

In which service does the DTR image scanning occur?

Assume that you have 3 managers in your cluster, what will happen if 2 managers fail at the same time? Select the all right answers.

Study Resource

Docker Certification Readiness Test

PODS in Kubernetes for Developers

Here, we will take a look at PODS.

Multi-Container PODs

How to deploy pods?

K8s Reference Docs:

Access certified Kubernetes administrator course with practice tests

Top DevOps Queries Answered

What is DevOps?

Does DevOps require coding?

What are the skills required to become a great DevOps engineer?

Is DevOps easy to learn?

How long does it take to learn DevOps?

Who can learn DevOps?

What are the stages in DevOps?

What is Continuous Integration?

What is Continuous Delivery?

What is Continuous Deployment?

What are the most common DevOps tools used?

What are the most popular DevOps tools that a beginner should know?

What is a pipeline in DevOps?

Where can I start my DevOps learning?

How can I get DevOps experience prior to the actual job?

Which command can be used to remove a `kubeapp` stack?