DEV Community: Srinivas karnati

ErrImagePull and ImagePullBackOff in Kubernetes

Srinivas karnati — Wed, 18 Jan 2023 06:15:25 +0000

When you create a pod in Kubernetes, it goes through a lot of steps. One of the important tasks is to pull the container image from the registry. While pulling those images from the registry, pods sometimes experience issues. If an error occurs, pods go into ErrImagePull, ImagePullBackOff errors.

ErrImagePull happens when the image that is specified in the pod spec can’t be fetched.
ImagePullBackOff is the grace period while the image pull is fixed. After each unsuccessful pull, the delay (grace period) increases.

Container Image and Image Pull Policy

A container image includes the binary data of the application and its dependencies. So you have to create the image for your application and push it to the registry before you refer that image in the Pod. The image registry by default is Docker unless you specified the registry.

Tags are also an important part of the image name, tags are used to identify different versions of the image.

Image pull policy defines when the kubelet tries to pull the image. the IfNotPresent value tells the kubelet to pull the image only if it is not present locally. Never tells the kubelet not to fetch the image. And Always tells kubelet to check the registry and pull the image for this pod. By default, the Image pull policy is set to Always.

ErrImagePull

This occurs when the kubelet tries to pull the image from the registry (based on its image pull policy). Sometimes things can go wrong, we might have messed up the image spec in the pod definition.

For example, I have specified my image as nginxxx . Let’s check the pod status by running kubectl get pods .

controlplane $ kubectl get pods
NAME     READY   STATUS         RESTARTS   AGE
my-pod   0/1     ErrImagePull   0          13s

ImagePullBackOff

ImagePullBackOff is the waiting status, a grace period with increasing time for every retry. Once this grace period completes, Kubernetes pulls the image again. Maximum backoff time is of five minutes.

So let’s try seeing the pod status again.

controlplane $ kubectl get pods
NAME     READY   STATUS             RESTARTS   AGE
my-pod   0/1     ImagePullBackOff   0          2m43s

We can get more information about the issues and retries by kubectl decsribe pod/<your-podname> .

controlplane $ kubectl describe pod/my-pod 
Name:             my-pod
Namespace:        default
--------------------------
Status:           Pending
IP:               192.168.1.3
IPs:
  IP:  192.168.1.3
Containers:
  my-pod:
    Container ID:   
    Image:          nginxxx
    Image ID:       
    Port:           <none>
    Host Port:      <none>
    State:          Waiting
      Reason:       ImagePullBackOff
    Ready:          False
---------------------------
Events:
  Type     Reason     Age                    From               Message
  ----     ------     ----                   ----               -------
  Normal   Scheduled  5m18s                  default-scheduler  Successfully assigned default/my-pod to node01
  Normal   Pulling    3m26s (x4 over 5m18s)  kubelet            Pulling image "nginxxx"
  Warning  Failed     3m20s (x4 over 5m12s)  kubelet            Failed to pull image "nginxxx": rpc error: code = Unknown desc = failed to pull and unpack image "docker.io/library/nginxxx:latest": failed to resolve reference "docker.io/library/nginxxx:latest": pull access denied, repository does not exist or may require authorization: server message: insufficient_scope: authorization failed
  Warning  Failed     3m20s (x4 over 5m12s)  kubelet            Error: ErrImagePull
  Warning  Failed     3m5s (x6 over 5m11s)   kubelet            Error: ImagePullBackOff
  Normal   BackOff    4s (x19 over 5m11s)    kubelet            Back-off pulling image "nginxxx"

The events section shows details about the issue that happened.

Common causes for ErrImagePull and ImagePullBackOff

Pod Spec provides the wrong registry name- resolve it by editing the pod specification and providing the correct registry
Pod Spec mentioned the wrong image name — we can resolve it by providing the correct image in the pod spec.
Invalid image tag in Pod Spec — we can resolve it by giving the correct image tag. If not provided, by default it will be labeled as the latest. If the image doesn’t contain the latest tag, you must provide a valid tag.
Authentication issue while accessing registry — Resolve it by providing correct secrets to the image registry. Add the secret reference in the Pod spec.
The container registry is not accessible- we can fix the issue by restoring the network connection and allowing the pod to retry pulling the image.
And those are just some of there might be more reasons why kubelet is not able to pull the image.

For troubleshooting these errors, your best friend is kubectl describe pod/<yourpodname> .

That’s a wrap!!!

Hope you learned something. Let me know your feedback and if you have any suggestions or questions, you can connect with me on Twitter.

Till next,

Happy kubectl’ing

What happens when you create a pod?

Srinivas karnati — Sun, 15 Jan 2023 06:04:58 +0000

A pod is the smallest deployable unit that you can create and manage in Kubernetes.

But how to create a pod, is just as simple as running kubectl run my-pod --image=nginx . But this simple task undergoes a complicated workflow that touches a lot of components in the cluster.

So what happens exactly,

First things first, you need to tell the API server that you need a pod to be created. So you use kubectl it to send YAML (pod manifest) to the API. API server receives the request and saves the definition to the database which is default etcd.
etcd is the key value store in Kubernetes that stores the entire state of the cluster. Now we have asked API Server to create a pod, so etcd will now store the Pod. At this point, Pod is stored in etcd. And it will be in the pending state now.
Now Scheduler sees the pending state and checks the nodes to make the decision to schedule the pod to the node. No pod is created until now, it just schedules the pod to a node.
On each worker node, we have a kubelet which is more of an interface between the control plane and components in the worker node. The kubelet is the scheduled node now retrieves the pod template and starts creating it.
The kubelet now asks CRI (container runtime interface) to create a container (one which our pod def contains).
The kubelet also asks CNI ( container network interface) to attach the container to the network. Now CNI assigns an IP address to the POD.
Also, kubelet checks any probes that are defined.
Now we have an IP address at which our pod is running. The last job of the kubelet is to report back to the Control plane. So Kubelet reports the IP address to the control Plane.
The POD is now created and is finally marked as running.

This shows the high-level workflow of what pod creation goes through, but please note these steps might have missed some minute details.

That's a wrap!!!

Hope you learned something. Let me know your feedback and if you have any suggestions or questions, you can connect with me on Twitter.

Till next,
Happy kubectl'ing

Kubernetes Objects

Srinivas karnati — Mon, 04 Apr 2022 04:35:24 +0000

Kubernetes objects are different types of entities provided by Kubernetes for deploying, maintaining, and scaling applications either on cloud or on-premise infrastructure. In this post we will go through different kubernetes objects that we'll use in our applcation deployment.

Pods

A pod is the most basic object that you can create in an Kubernetes cluster. It usually contains one or more running containers. Containers in a pod share the same network, storage, and lifecycle. What this means is that they can communicate with each other directly, and will both be stopped and started at the same time.

Pod Definition

apiVersion: v1
kind: Pod
metadata:
  name: my-first-pod
  labels:
    name: my-first-pod
    app: nginx-app
spec:
  containers:
     - image: nginx
        name: nginx-app
        ports:
           - containerPort: 80

Let’s break down the yaml definition we've just used, as we will be using similar definition in other objects definitions as well. And also try to understand this definition file will do.

apiVersion: This definies which version of the Kubernetes API you're using to create this object. There are several versions, and several objects are introduced with each version. Some common ones are v1, apps/v1, and extensions/v1beta1.

Kind: This is the type of Kubernetes object. In this example we’re creating a pod. (so Pod)

Metadata: The metadata is the information about the object. The metadata usually contains the name you want to give the object , the labels etc. For the labels, you can define as many labels as you want like name, app, owner, group, tier etc,

Spec: The spec section is where you define the specifications of your object (the desired state of the object). In the case of a pod, it’s where you describe the state of your container. Some options you can specify in the spec section are:

Container: Here you specify:

The image that you want to run in your pod
The name of the container that you’ll run in your pod.
The ports

Let's create the Pod with our pod-definition.yaml file:

Now go to your terminal, make sure you’re in the directory where you created your file.

kubectl apply -f pod-definition.yaml

The above command will create the pod with nginx image as defined in the pod-definition file. and you can view the Pods by using kubectl get pods.

ReplicaSet

The ReplicaSet is used to create multiple copies of the same pod in a Kubernetes cluster. It helps ensure that at any given time, the desired number of pods specified are in the running state.

ReplicaSet helps bring up a new instance of a Pod when the existing one fails, scale it up when the running instances are not up to the specified number, and scale down or delete Pods if another instance with the same label is created. A ReplicaSet ensures that a specified number of Pod replicas are running continuously and helps with load-balancing in case of an increase in resource usage.

ReplicaSet Definition

apiVersion: apps/v1
kind: ReplicaSet
metadata:
  name: myapp-replicaset
  labels:
    app: myapp
    tier: frontend
spec:
  replicas: 2
  selector:
    matchLabels:
      tier: frontend
  template:
    metadata:
      labels:
        tier: frontend
    spec:
      containers:
      - name: nginx-app
        image: nginx

To create ReplicaSet, go to your terminal and run the following command with your YAML Definition file.

kubectl apply -f ReplicaSet-definition.yaml

Once the ReplicatSet is running, you can check its status using kubectl get replicaset <your-replicaset-name>

Deployment

Deployments are Kubernetes objects that are used for managing pods. The first thing a Deployment does when it’s created is to create a replicaset. The replicaset creates pods according to the number specified in the replica option. If you look at the YAML definition for the Deployment below, you’ll notice that the spec section of the deployment looks just like the definition for a replicaset.

apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp-deployment
  labels:
    app: mongo
spec:
  replicas: 2
  selector:
    matchLabels:
      app: mongo
  template:
    metadata:
      labels:
        app: mongo
    spec:
      containers:
      - name: mongo
        image: mongo

So when you create a Deployment, it will create a Replicaset with the pods mentioned in definition.
Deployments can be used to scale your application by increasing the number of running pods, or update the running application.

kubectl apply -f Deployment-definition.yaml

You can view the Deployments with kubectl get deployments

Namespace

Namespaces are used to organize objects in a Kubernetes cluster. They enable you to group resources together and perform actions on those resources.

To see all the namespaces in you environment , use kubectl get namespaces

Creating a Namespace can be done with a single command. If you wanted to create a Namespace called ‘testing’ you would run:

kubectl create namespace testing

Or you can create a namespace using YAML Definition, as follows:

apiVersion: v1
kind: Namespace
metadata:
  name: testing

Now that we have created a namespace, but how do we create the objects in the newly created namespace. For that you need to reference the namespace in the definition file/command.

apiVersion: v1
kind: Pod
metadata:
  name: myPod
  namespace: testing
  labels:
    name: namespaced-pod
spec:
    containers:
      - image: hello-world
         name: firstpod

Service

Kubernetes Service is used to expose an application deployed on a set of pods using a single endpoint. Service enables network access to a set of Pods in Kubernetes.

A service in Kubernetes can be created via an API request by passing in a service definition such as:

apiVersion: v1
kind: Service
metadata:
  name: my-service
spec:
  selector:
    app: myApp
  ports:
    - protocol: TCP
      port: 80
      targetPort: 9376

In this example the service is named my-service and will target TCP port 9376 on any pod with the metadata label "app=MyApp" . Kubernetes constantly evaluates the service’s label selector to determine at any given moment which pods are included in the service. This means that a new service can include existing pods that already match the label selector.

Service Types

The default and simplest service type is ClusterIP . It exposes the service’s ClusterIP address internally to the Kubernetes cluster.

NodePort – exposes the service on the specified port number, on all nodes in the Kubernetes cluster. Meaning that an incoming request to a node’s IP on the specified port will get routed to the service’s ClusterIP .

LoadBalancer – service is exposed like in NodePort but creates a load balancer in the cloud where Kubernetes is running (if supported by the cloud provider) that receives external requests to the service. It then distributes them among the cluster nodes using NodePort. To specify this type add this line to the spec:

type: LoadBalancer

15 Docker Commands you should know

Srinivas karnati — Tue, 08 Mar 2022 06:19:44 +0000

Docker is a containerization system which packages and runs the application with its dependencies inside a container. In this article, I mentioned about 18 Docker Commands you need to know while working with Docker.

1. docker --version

One of the first things we all want to know is how to find the installed docker version.

2. docker pull

The 'docker pull' is a Docker command to download a Docker image or a repository locally on the host from a public or private registry.

3. docker images

List all the docker images pulled on the system with image details such as TAG/IMAGE ID/SIZE etc.

4. docker run

The docker run command creates a container from a given image and starts the container using a given command. It is one of the first commands you should become familiar with when starting to work with Docker.

If the image is not present locally, docker first pulls the image from the repository and runs it.

5. docker ps

docker ps command lists all the docker containers are running with container details. To list all the docker containers running/exited/stopped with container details, we use docker ps -a command

6. docker start/ stop/restart

docker start command in docker starts the docker container with container id mentioned in the command.

docker stop stops a container with container id mentioned in the command.

To restart the docker container, we use docker restart.

7. docker rm

Remove the docker container with container id mentioned in the command.

8. docker rmi

Remove the docker image with the docker image id mentioned in the command

9. docker commit

Create a new image from a container’s changes. It can be useful to commit a container’s file changes or settings into a new image.

10. docker login

I already had my credentials setup, so it didn't ask me to enter login id and password.
11. docker push

Upload a docker image with the image name mentioned in the command on the dockerhub.

Once it is successful, you would be able to see the docker image in your docker hub repo.

12. docker info

Get detailed information about docker installed on the system including the kernel version, number of containers and images, etc.

13. docker history

Shows the history of a docker image with the image name mentioned in the command.

14. docker search

Search for a docker image on dockerhub with the name mentioned in the command.

15. docker logout

Logging out from dockerhub.

Connect with me on

Getting Started with Azure CLI

Srinivas karnati — Fri, 04 Mar 2022 14:41:00 +0000

What is Azure CLI?

The Azure Command-Line Interface (CLI) is a command-line tool which connect to Azure and execute administrative commands on Azure resources. It allows the execution of commands through a terminal using interactive command-line prompts or a script.

How to install the Azure CLI on Macos?

Azure CLI is available to install in Windows, macOS and Linux environments. It can also be run in a Docker container and Azure Cloud Shell.

We can install Azure CLI on MacOS using Homebrew package manager. Homebrew makes it easy to keep your installation of the CLI update to date. It is always a best practice to update the homebrew first and install the packages.

brew update 
brew install azure-cli

Once the dependencies are installed, you can check the version of Azure CLI using

az --version

Sign into Azure CLI

To sign into the Azure, Run the login command

az login

If the CLI can open your default browser, it will do so and load an Azure sign-in page.

Otherwise, open https://aka.ms/devicelogin on your browser and enter the authorization code displayed in your terminal.

Azure CLI Examples

To see subscriptions in your Azure account, we can use account as follows:

az account list 

\\ We can also use account show

az account show

To see resources that are created in your azure account, we can use resource command:

az resource list

`find` and `--help`

To search for commands, use az find. For example, to search for command names containing network, use the following command:

az find network

Use the --help argument to get a complete list of commands and subgroups of a group. For example, to find the CLI commands for working with account we use:

az account --help

Create a Virtual Network using Azure CLI

Virtual networks (VNets) is something that allows Azure resources to communicate securely with each other. VNets can also be connected to other VNets if their address ranges don't overlap.

Use the az network vnet create command to create a virtual network named VNet1 with address prefix of 10.1.0.0/16 and a subnet named Subnet1 with address prefix of 10.1.1.0/24.

az network vnet create \
 --name "vnet1" \
--resource-group "my_learning" \
--address-prefixes 10.1.0.0/16 
--subnet-name "subnet1" \
--subnet-prefixes 10.1.1.0/24

You can also use shell variables to pass the values like as follows:


vnetName=vnet1
subnetName=subnet1
vnetAddressPrefix=10.1.0.0/16
subnetAddressPrefix=10.1.1.0/24

az network vnet create \
  --name $vnetName \
  --resource-group $resourceGroup \
  --address-prefixes $vnetAddress \
  --subnet-name $subnetName \
  --subnet-prefixes $subnetAddress

Once you run the commands mentioned earlier, upon successful execution you will be able to see the vnet created in the resources list.

In the Terminal :

In the Portal:

Create a Virtual Machine (VM) using CLI

We use the az vm create command to create a new virtual machine running Ubuntu, which uses SSH authentication for login, and is connected to the subnet and VNet we created just now.

az vm create --name "myVM1" --resource-group "my_learning" 
--image UbuntuLTS --vnet-name "vnet1" 
--subnet "subnet1" --generate-ssh-keys

Upon successfull execution, you should be able to see the following output which displays all information you need to access the VM.

Confirm that the VM is running by connecting over SSH.

ssh <public IP address>

Go ahead and log out from the VM by typing exit.

Note: Always Clean up the resources which are not in use to save billing costs.

**Connect with me on: **

Network Security Groups In Azure

Srinivas karnati — Mon, 28 Feb 2022 14:12:47 +0000

Network Security Group

An Azure Network Security Group (NSG) is a core component of Azure’s security. You can use an Azure network security group to filter network traffic to and from Azure resources in an Azure virtual network. Network (VNet).

A network security group contains security rules that allow or deny inbound network traffic to, or outbound network traffic from, several types of Azure resources. For each rule, you can specify source and destination, port, and protocol.

Each rule in a NSG specifies the following properties:

Rule	Property
Name	A unique name within the network security group.
Priority	A number between 100 and 4096. Rules are processed in priority order, with lower numbers processed before higher numbers, because lower numbers have higher priority. Once traffic matches a rule, processing stops. As a result, any rules that exist with lower priorities (higher numbers) that have the same attributes as rules with higher priorities are not processed.
Source/Destination	This setting defines the source or destination of the network traffic. It can be set to “Any” for traffic from anywhere, or you could lock it down to a single IP address or an IP range.
Protocol	TCP, UDP, ICMP, ESP, AH, or Any.
Direction	This setting defines the direction of the network traffic, and you can set it to either Inbound or Outbound.
Port Range	The port range setting describes the port or port range of the rule. You can specify a single port, e.g., 443, or a range of ports, e.g., 1000-2000.
Action	This setting defines what action needs to be done, "Allow" or "Deny"

Default Security rules:

When you create an Azure NSG, Azure creates the following default rules in each network security group.

Azure Network security groups execute rules in order of priority, with the lower numbered priorities processed before high numbers.

Create a Network security group

On the Azure Portal, select Create a resource -> Networking -> Network Security Group
In the Create network security group page, under the Basics tab, set values for the following settings:

Setting	Action
Subscription	Choose your subscription.
Resource group	Choose an existing resource group, or select Create new to create a new resource group.
Name	Enter a unique text string within a resource group.
Region	Choose the location you want.

Select Review + create.
After you see the Validation passed message, select Create. To view your network security groups. Search for and select Network security groups. The list of network security groups appears for your subscription.

Create a security rule

In Azure Portal, Search for and select Network security groups. Select the name of the network security group you want to add a security rule to. In the network security group's menu bar, choose Inbound security rules or Outbound security rules.
Select Add. Select or add values for the following settings, and then select OK:

*Source: MS Learn *

Getting started with Azure Policies

Srinivas karnati — Tue, 22 Feb 2022 12:16:49 +0000

Azure Policy

Azure Policy is a service in Azure which allows you create polices which enforce and control the properties of a resource. When these policies are used they enforce different rules and effects over your resources, so those resources stay compliant with your IT governance standard. Through its compliance dashboard, it provides an aggregated view to evaluate the overall state of the environment, with the ability to drill down to the per-resource, per-policy granularity.

Azure Policy uses a JSON format to form the logic the evaluation uses to determine whether a resource is compliant or not.

Policy definition is the conditions which you want controlled. There are built in definitions such as controlling what type of resources can be deployed to enforcing the use of tags on all resources. These are the business rules, described in JSON format.

Policy assignment is the scope of what the policy definition can take effect around. Scope of assignment can be assigned to a individual, resource, resource group or management group. Policy assignments are inherited by all child resources.

Azure Initiative

An Azure initiative is a collection of Azure policy definitions that are grouped together towards a specific goal or purpose in mind. Azure initiatives simplify management of your policies by grouping a set of policies together as one single item. Sometimes also called as Policyset

Assign a Policy

To enforce the Complaince with Azure Policies , we need to assign the policy. In this example, we assign the built-in policy definition called "Allowed locations" to ensure that resources are created in the specific region only.

Go to the Azure portal to assign policies. Search for and select Policy.

Select Assignments on the left side of the Azure Policy page. An assignment is a policy that has been assigned to take place within a specific scope.
Select Assign Policy from the top of the Policy - Assignments page.

On the Assign Policy page and Basics tab, select the Scope by selecting the ellipsis and selecting either a management group or subscription. Optionally, select a resource group. A scope determines what resources or grouping of resources the policy assignment gets enforced on. Then select Select at the bottom of the Scope page.
Resources can be excluded based on the Scope. Exclusions start at one level lower than the level of the Scope. Exclusions are optional.
Select the Policy definition ellipsis to open the list of available definitions. You can filter the policy definition Type to Built-in to view all and read their descriptions.
Select Allowed locations. If you it, type locations into the search box and then press ENTER or select out of the search box. Select Select at the bottom of the Available Definitions page once you have found and selected the policy definition.
The Assignment name is automatically populated with the policy name you selected, but you can change it. Leave Policy enforcement as Enabled.

Select the Parameters tab at the top of the wizard.
In the Allowed locations, you need to select the locations in which you want your resources to be created.
We will leave the rest of the values as default.
Select the Review + create tab at the top of the wizard.

Review your selections, then select Create at the bottom of the page.

To test our Policy, Let's create two logic apps one in US region and another in India.

To create a logic apps, click on Add resource -> Search for logic apps
On the Basics tab, select Your subscription, Resource group, Region (here I selected South India).

I will leave all the remaining details as default, We will review and create the Logic app.

As you can see the deployment is completed.

Repeat the same for the US region, the details of resource are as follows

Though it is not allowed in Policy Definition, validation will be successful.
Let's try to deploy the resource.

As you can see the Deployment is failed due to "Disallowed by the Policy".

Create a Virtual Network on Azure

Srinivas karnati — Sun, 20 Feb 2022 05:45:12 +0000

What is Azure Virtual Network?

Azure Virtual Network (VNet) is the bulding block for establishing your private network in Azure. VNet enables many types of Azure resources, such as Azure Virtual Machines (VM), to securely communicate with each other. VNet is similar to a traditional network that you'd operate in your own data center, but brings with it additional benefits of Azure's infrastructure such as scale, availability, and isolation.

Azure virtual network enables Azure resources to communicate with each other.

Key scenarios that you can accomplish with a virtual network include - communication of Azure resources with the internet, communication between Azure resources, communication with on-premises resources, filtering network traffic, routing network traffic, and integration with Azure services.

Create a Virtual Network using Azure Portal

In this post, we will create a VNet ( Virtual network), we will add two subnets, and create two VMs, one in each subnet.

Create a Virtual network

To access azure portal, go to https://portal.azure.com
Click on create resource -> Networking -> Virtual Network

In Create virtual network, enter or select this information in the Basics tab:
- Subscription - Select your subscription.
- Resource group - Select your Resource group. ( or create new)
- Instance details - enter Name and the Region for your Vnet

Select the IP Addresses tab, or select the Next: IP Addresses button at the bottom of the page.
In IPv4 address space, select the existing address space and change it to 10.1.0.0/16.
Select + Add subnet, then enter Subnet name and 10.1.1.0/24 for Subnet address range.
Select Add.
Repeat previous two steps to add one more subnet as follows:

We will leave the rest as default and select Review + Create. Once the validation is passed, click on Create.
Once the deployment is completed, you will see your deployed Virtual Network on your resources.

Create Virtual Machines

We will create two VMs in our virtual network:

Create the first VM

On the upper-left side of the portal, select Create a resource > Compute > Virtual machine.
In Create a virtual machine, type or select the values in the Basics tab:
- Subscription - Select your Azure subscription
- Resource Group - Select the Resource group, you've created.
- Instance details
- Virtual machine name Enter name for your VM
- Region Select the Region( I select South India)
- Availability Options Select No infrastructure redundancy required
- Image - I selected Ubuntu 20.04.
- Azure Spot instance Select No
- Size Choose VM size or take default setting
- Administrator account
- Username Enter a username
- Password Enter a password
- Confirm password Reenter password

Select the Disks tab, select the preferred disk type or leave defaults.
In the Networking tab, select or enter:
- Virtual network - Select Vnet1 ( the name of Virtual network you created).
- Subnet Select vnet1-subnet1 (The first subnet)
- Public IP Leave the defaults

Select Review + Create. You will be taken to Review page where Azure checks your VMs Configuration.
Once the validation is passed, Select Create. Once the deployment is completed, you will see the VM in your resources list. #### Create the Second VM Repeat the above steps to create a New Virtual Machine with following Networking Configuration.
Virtual Network - Vnet1
Subnet - select vnet1-subnet2 ( second subnet) Keep the rest as similar to first VM, and create the virtual machine.

Communicate between VMs

To Connect to your VM (vnet1-vm1), you can either use Connect button, or you can access using public IP via SSH

ssh username@IPaddress

enter the username and password you created for the virtual machine previously.
Once you are in the VM, enter ping vnet1-vm2 ( to check the second VM's reachability). You'll receive a message similar to this output:

Repeat the same steps for VM2 to test if VM1 is accessible or not.

OSI Model and it's layers

Srinivas karnati — Sun, 20 Feb 2022 05:43:54 +0000

A network both contains hardware and software that sends data from one location to another. The end goal of the networking is to allow two devices share the data.

The hardware part of networking, consists of physical equipment that carries signal from one point of the network to another. While the software consist of instructions sets that make possible the services that we expect from a network.

The task of sending data from one point in the another can be performed by different layers in the network.

There are two popular network models:

OSI Model
TCP/IP

In OSI Model, the rules for networking are divided into seven different layers.

OSI Model

The Open Systems Interconnection (OSI) Model is an ISO standard that describes the functions of networking system.

The OSI model is a layered architecture that allows communication between all types of computer systems regardless of their underlying architecture.

In the OSI model, the communications between a computing system are divided into seven different layers: Physical, Data Link, Network, Transport, Session, Presentation, and Application.

Physical Layer

The physical layer is the layer 1 of the OSI, responsible for sending bits from one computer to another through a physical medium. It deals with the setup of physical connection to the network and with transmission and reception of signals.

Functions of Physical Layer

Representation of Bits - defines how the stream of bits is encoded into signals for transmission i.e. how 0s and 1s are changes to signal.
Data Rate - defines the rate of transmission i.e. the number of bits sent per second.
Synchronization - does the synchronization of bits by providing a clock. The clock controls both sender and receiver thus providing synchronization at a bit level. One bit is transmitted or received during each clock cycle.
Interface - defines the transmission interface between devices and transmission medium. For example which electrical connectors and cables to use are defined by the physical layer.
Topologies - defines the topology to use - bus, star, mesh, or ring.
Transmission Modes - defines the direction of transmission between two devices: Simplex(One-way communication), Half Duplex(Two-way communication but one at a time), Full Duplex(Two-way simultaneous communication).

Data Link Layer (DLL)

The data link layer is responsible for the node to node(or hop to hope) delivery of the message. It makes sure data transfer is error-free from one node to another, over the physical layer. This is also where we introduce or first see MAC addresses.

Functions of the Data Link Layer:

*Framing *- The data link layer divides the stream of bits received from the network layer into manageable data units called frames.
Physical Addressing - After creating frames, DLL adds physical addresses(MAC address) of the sender and receiver in the header of each frame.
Error Control - The Datalink layer adds reliability to the physical layer by adding mechanisms to detect and retransmit damaged or lost frames. It also uses a mechanism to recognize duplicate frames.
Access Control - When two or more devices are connected to the same link, data link layer protocols are necessary to determine which device has control over the link at any given time.

Network layer

The network layer is responsible for receiving frames from the data link layer, and delivering them to their destinations among based on the addresses contained inside the frame.

The network layer finds the destination by using logical addresses, such as IP (internet protocol). At this layer, routers are a crucial component used to quite literally route information where it needs to go between networks.

Functions of Network Layer:

The network layer is responsible for the source to destination delivery of a packet across multiple networks.

Logical addressing - Network layers allocate a unique address called IP address to the header of the packet to distinguish each device.
Routing - The network layer determines which route is suitable from source to destination across multiple networks.

In Layer 2 MAC Address is introduced, but in Layer 3 IP address is introduced. The difference between MAC and IP address is that MAC Address is used to ensure the physical address of the computer. It uniquely identifies the devices on a network.
While IP addresses are used to uniquely identifies the connection of the network with that device takes part in a network.

Transport Layer

The transport layer is responsible for service to service delivery of the entire message.

A Service is an application program running on a host. The network layer oversees source to destination delivery of individual packets, it does not recognize any relationship between those packets and treats them individually as though each piece belonged to a separate application, whether or not it does. The transport layer, on the other hand, ensures that the whole message arrives intact.

Functions of Transport Layer:

Port Addressing - The transport layer header includes the port address(a 16-bit address allocated to every process) which delivers a message from a specific process on one computer to another process on another computer.
Segmentation and Reassembly - A message is divided into transmittable segments, with each segment containing a sequence number. The receiver transport layer uses these numbers to reassembles segments and replace packets that were lost in the transmission.
Connection Control - The transport layer can be either connectionless or connection-oriented.

Session Layer

The session later is responsible for dialog control and synchronization.

Functions of Session Layer -

** Session maintainence**: The session layer is respnsible to establish, use and terminate a connection.
Dialog Control - The session layer allows two systems to enter into a dialog. It allows the communication between two processes to take place in either half-duplex(one way at a time) or full-duplex(two ways at a time) mode.
Synchronization - The session layer allows a process to add synchronization points to a stream of data.

Presentation layer

The Presentation Layer deals with the syntax and semantics of the information transmitted between two systems. The Presentation Layer is responsible for translation, compression, and encryption.

Functions of Presentation Layer -

Translation - This Layer translates the exchanging information into a compatible format between the sender and receiver.
Encryption - The presentation Layer uses some algorithms for encrypting the information before sending it to the receiver, and decryption on the receiver side reverses the original process to transform the message back to its original form.
Compression - Presentation Layer compresses the number of bits contained in the information using some algorithms. It maybe of either lossy or lossless compression.

Application Layer

The application layer enables the user, whether human or software, to access the network.

Overview of layers

Automating user management with bash script

Srinivas karnati — Sun, 20 Feb 2022 05:42:50 +0000

As a System administrator, creating users on multiple servers maybe something that you would do on a daily basis. It maybe an easy task to create a user on single server but imagine that you need to add a user on 100+ different Linux servers. Isn't it a tedious task? so let's create a bash script that automates it.

First, let us create a text file that includes all the server hostnames or IP addresses where you wish to add the user.

Here in this example, I've created a file servers.txt that includes the servers names ( here I have two servers):

Now take a look at the following addusers.sh bash script:

#!/bin/bash
servers=$(cat servers.txt)
echo -n "Enter the username: "
read userName
echo -n "Enter the user id: "
read userID

for i in $servers; do
echo $i
ssh $i "sudo useradd -m -u $userID $userName"
if [ $? -eq 0 ]; then
echo "User $userName added on $i"
else
echo "Error on $i"
fi
done

When you run addusers.sh, the script will first ask you to enter the username and the user id of the user who you want to add; then, it will loop over and connect to all the servers in the servers.txt file via SSH and add the requested user.

Let's run the addusers.sh script and see how it executes:

If you haven't enabled password-less login via SSH, it will prompt to enter the password to gain access to server.

The script ran successfully and testuser2 account was created on the two servers.

You must have a valid account that has super user access on all the servers.

As a beginner to Bash Scripting, I found it really impressive to know that with bash scripting you can automate many boring, tedious task in Linux!

Basics Of Bash Scripting

Srinivas karnati — Sun, 20 Feb 2022 05:41:44 +0000

What is a Shell

The shell is a computer program, it takes each command and passes it to the operating system kernel to be executed. It then displays the results of this operation on your screen.
Different systems may use different shells. The most commonly available shells are:

Bourne shell (sh)
Bourne Again Shell (bash)
C shell (csh)
Z Shell (zsh)
Korn shell (ksh) by using cat /etc/shells, We can see the list of available shells in our Linux system.

Shell Script

A shell script is a file that contains a list of commands that are executed sequentially by the Unix shell. The main components of a Shell Script are :

Shebang ( #!/bin/bash)
Comments (starts with #)
Actual Commands of Linux( ls, cd, mkdir, chmod etc...)
Statements ( If, for , while etc.,)

In order to execute, the shell script file need to have executable permissions.

Sample Shell Script :

#!/bin/bash
# script to print - Hello World!
echo "Hello World!

It is a basic script, that prints hello world to the terminal. Here the first line is called as "shebang" which tells the shell to run the file as bash script. The second line is a comment which is ignored while execution. And the echo command which print the stdout to terminal screen.

We can execute the script either by Bash source, or execute it as a shell-script by making it executable (using chmod) from the path.

Variables

We all know, we need variables to store some value which we are gonna use repeatedly. Bash has variables using which you can store values. Let's create and use some variables in Bash.

#!/bin/bash
name = srinivas
echo $name

To create a variable, we simply write the variable name and assign the value to the variable. To access the variable add $ sign before the variable name.

Conditional statements

If Statement

if COMMAND_TO_TEST
then
  STATEMENTS
fi

The if statement starts with the 'if' keyword followed by the conditional expression and the then keyword. The statement ends with the fi keyword.

If the COMMAND_TO_TEST evaluates to True, the STATEMENTS gets executed. If COMMAND_TO_TESTreturns False, nothing happens; the STATEMENTS get ignored.

#!/bin/bash

echo -n "Enter a number: "
read number

if [[ $number -gt 100 ]]
then
  echo "The variable is greater than 100."
fi

The script will prompt you to enter a number. When you enter 120, the test command will evaluate to true because 120 is greater than 100, and the echo command inside the then clause will be executed.

The -gt in the script is numeric comparison - greater than. The following table lists comparison operators for both numbers and strings.

if...else Statement

The Bash if..else statement takes the following form:

if COMMAND_TO_TEST
then
  STATEMENTS1
else
  STATEMENTS2
fi

If the COMMAND_TO_TEST evaluates to True, the STATEMENTS1 will be executed. Otherwise, if COMMAND_TO_TEST returns False, the STATEMENTS2 will be executed. You can have only one else clause in the statement.

Let’s add an else clause to the previous example script:

When you enter 100, the test command will evaluate to false and else block executes.

if...elif...else Statement

if COMMAND_TO_TEST
then
  STATEMENTS1
elif COMMAND_TO_TEST
then
  STATEMENTS2
else
  STATEMENTS3
fi

The conditions are evaluated sequentially. Once a condition returns True, the remaining conditions are not performed, and program control moves to the end of the if statements.

BASH also allows you to nest if statements within if statements. You can place multiple if statements inside another if statement and you can also use logical operators like OR and AND operators allow you to use multiple conditions in the if statements.

Loops

Loops are useful when you want to run a series of commands repeatedly until a certain condition is reached.

There are three basic loop types in Bash scripting.

for loop
While loop
until loop ### for loop The basic for loop in Bash looks like:

for item in [LIST]
do
  [COMMANDS]
done

In the below example, the loop will iterate over a each number in the list.

for number in 0 1 2 4 5 6 7 8 
do
  echo "number : $number"
done

For looping over a number range, we can use {0..8}. In the latest versions of Bash, it is also possible to specify an increment when using ranges. The expression takes the following form:

for number in {0..10..2} 
do
  echo "number : $number"
done

We can also use the for loop as follows:

for ((INITIALIZATION; TEST; STEP))
do
  [COMMANDS]
done

`break` and `continue`

We all know, the break and continue statements can be used to control the loop execution.

The break statement terminates the current loop and passes program control to the statement that follows the terminated statement. It is usually used to terminate the loop when a certain condition is met.
The continue statement exits the current iteration of a loop and passes program control to the next iteration of the loop.

Bash `while` loop

The while loop is used to performs a given set of commands an number of times as long as the given condition evaluates to true.

The Bash while loop takes the following form:

while [CONDITION to TEST]
do
  [COMMANDS to EXECUTE]
done

In the following example, for each iteration, the current value of the variable i is printed and incremented by one.

number=0
while [ $number -le 5 ]
do
  echo "Number: $number"
  ((number++))
done

The above script loops as long as i is less than 5.

Bash `until` loop

The until loop is used to execute a given set of commands as long as the given condition evaluates to false.

The Bash until loop takes the following form:

until [CONDITION to TEST]
do
  [COMMANDS to EXECUTE]
done

In the following example, on each iteration the loop prints the current value of the number and increments the variable by one.

#!/bin/bash
number=0
until [ $number -gt 3 ]
do
  echo "Number: $number"
  ((number++))
done

The loop iterates until the variable has a value greater than three. The script gives the following output.

The while and until loops are similar to each other. The main difference is that the while loop iterates as long as the condition evaluates to true and the until loop iterates as long as the condition evaluates to false.

Functions in Bash

Function are part of any programming language , we can define the function as follows:

function_name () {
  commands to execute
}

Let's look at the following example which prints "hello world" to screen.

#!/bin/bash
helloworld () {
   echo "Hello_world"
}
helloworld

If you run the script, it will print Hello_world

We can also declare a function using the keyword function just like we do in most of the programming languages.

Unlike functions in “real” programming languages, Bash functions don’t allow you to return a value when called.
The return status can be specified by using the return keyword, and it is assigned to the variable $?. The return statement terminates the function. You can think of it as the function’s exit status .

#!/bin/bash
my_function () {
  echo "Test of Return"
  return 7
}

my_function
echo $?

A Bash function is a block of reusable code designed to perform a particular operation. Once defined, the function can be called multiple times within a script.

Connect with me on Linkedin, Twitter.

Setting up an Ubuntu 20.04 VPS

Srinivas karnati — Sun, 20 Feb 2022 05:40:55 +0000

Creating an VPS

VPS, or Virtual Private Server, is an virtual machine configured to operate as a physical computer under virtualization software on a dedicated server. It stores all of the data and files that are useful for your application .You would interact with a VPS just as you would a physical, dedicated server.

I use Digital Ocean for creating my VPS. Ofcourse you can use any of the other service providers. Once you login into your account, you will be presented with the below screen.

Click on Create and then Droplets (VPS) , you will get a screen where you can choose the Operating system, and plan size.

Choose a region which is closest to your target users. This is basically the physical location where the server will be located. Finally, choose a password and click create droplet.

Once the Creation of droplet is done, you will see the details of the Server like Public IP, Private IP and some hardware details.

Accessing the Server

We will use an SSH Client to get access to the server's terminal (our VPS), and once we have access to the terminal, we can start executing commands on this server.

To log in to our server, we need to know its public IP address ( which we already have).

Execute the following command on the terminal

ssh root@yourserver_PublicIP

Enter the password you have created while creating the droplet in DigitalOcean. Once you enter the password when prompted, you will be logged in as root user.

You an execute almost all the Linux Commands on your Ubuntu Server.

You can create non-root users, by useradd command.

useradd srinivas

You will be asked to enter some information and a password. Once you are provided the details required, It will create a non root user profile in the server.

The exit command will close the server's terminal.

You can configure as many servers as you want of your choice( but beware of Billing ). Hope you got basic understanding on how to setup a VPS and how to access it. :)