DEV Community: Dipanshu Torawane

KCD Chennai 2022 Blogathon Contest 🎉

Dipanshu Torawane — Mon, 28 Mar 2022 17:39:20 +0000

Blogathon

Either write something worth reading or do something worth writing. --Benjamin Franklin

As of 2021, there are more than 570 million blogs on the internet (and this number is constantly growing). Over 7 million blog posts are published daily. 77% of people regularly read blogs online. Blogging is a powerful means of sharing knowledge and promoting your thoughts and views.

Kubernetes Community Days (KCD) Chennai 2022 is hosting a Blogathon. If you are a seasoned blogger, or the occasional writer, or the one searching for the right reason to start blogging, we invite you to participate!! Unleash the writer in you.

What are the Prizes on Offer?

How do I Participate?

Register yourself here
Upon registration you'll receive an email containing further steps to be followed

Important Dates

29-March-2022: Registration begins
16-April-2022: Blogging begins
15-May-2022: Registration ends
31-May-2022: Blogging ends
4-June-2022: Winners announced during KCD Chennai 2022

What topics can I write about?

We invite Technical blogs on a wide variety of topics. You are allowed to write about any topic you are passionate about, as long as the content is technical.

Publishing your blog

When you create a new blog, under 'Create Post' choose 'Kubernetes Community Days Chennai' as shown below.

How is the score calculated?

Your score would be calculated using all these parameters:
- No. of views
- No. of reactions (likes, unicorns, bookmarks)
- No. of blogs
Blogs published during 16-April to 31-May alone would be considered for scoring. (Tip: Keep publishing as many good-quality blogs as possible.)
The person who publishes the max. no of blogs during 16-April to 31-May wins the Prolific Blogger award (Tip: Become a blogging factory!)

Score board

The kcd-chennai bot publishes the score board to the #kcd-chennai channel in CNCF slack. Sign-up for free at https://slack.cncf.io and join the #kcd-chennai channel to view the score board

Rules

You are allowed to publish any number of blogs; there is no restriction.
You are allowed to cross-post blogs you've already published in other blogging sites e.g. medium, hashode, wordpress etc.
You are allowed to publish blogs you've already published in DEV under your personal account.
Published blogs would be reviewed by our team. If the blog is offensive, violates copy rights or we detect plagiarism, such blogs would be removed.

Can I promote my blogs?

Of course yes! Feel free to promote your blogs via Social media, Emails, Newsletters, WhatsApp etc. This would help increase the views and reactions!!

Contact Us

If you have further questions or need help, post it in the #kcd-chennai channel in https://slack.cncf.io

Intro to Environment Variables and SSH...

Dipanshu Torawane — Tue, 22 Mar 2022 11:17:38 +0000

What are environment variables?

Each user has their own environment. Each user can configure their own environment/account by setting preferences.

These OS configurations should be isolated from other user environments.

Where does OS store all these configurations?

These are stored in environment variables which are just KEY = value pairs

Variables that store information. By convention, names are defined in UPPERCASE. For example, SHELL=/bin/bash (Default shell program of the user)

Users can change these environment variable values. For example, SHELL=/bin/zsh

Variables are variables, which means they can be changed.

Commands for environment variables

printenv => prints all the environment variables.

printenv <Env. variables> => prints the provided environment variable's name.

Referencing Environment Variable

Using the $ sign. For example, $USER

Use cases of Environment Variables

OS stores information about the environment.
We can create our own environment variables.

Creating Environment Variables

export (env.) variable_name=value -> This is available all over the environment and not unlike bash variables which are available in the script.

For example,

export DB_USERNAME=dbuser
export DB_PASSWORD=secretpwdvalue
export DB_NAME=mydb

Deleting Environment Variables

unset (env.) variable_name=value

For example,

unset DB_NAME

NOTE:

These are only available for the current session*(temporary env. variable) and* after exiting the terminal the env. variable created is removed.

Persisting Environment Variables

This is user-specific.

There are per-user shell-specific configuration files.
- For example, if you are using BASH, you can declare the variables in the ~/.bashrc file.
Variables set in this file are located whenever a bash login shell is entered.

To add these variables, open the .bashrc file and at the last entry

export DB_USERNAME=dbuser
export DB_PASSWORD=secretpwdvalue
export DB_NAME=mydb

Now, after reopening the terminal, the environment variables will be there as they are saved on the system.

To load the new environment variables into the current shell session

source ~/.bashrc

Persisting Environment Variables System-wide

To add for all users there is a configuration file in Linux in the home folder.

/etc/environment

PATH environment variable

These are lists of directories to executable files, separated by :(colon)
This tells the shell which directories to search for the executable in response to our executed command. PATH = $PATH:.....
You need to provide an absolute location to make that file available.
It adds our custom app/command to be available for the user(if configured in .bashrc) or for all users (if configured in /etc/environment)

SSH - Secure Shell

It is a network protocol that gives users a secure way to access a computer over the internet. SSH also refers to the suite of utilities that implement that protocol.

Some Use Cases

Copy the file to the remote server.
Install the software on a new server.

2 ways to authenticate with the remote server

Username and Password.

Admin creates a user on the remote server and the user can then connect with the username and password.

SSH key pair (more secure way).

The client creates an SSH key pair.

key-pair = Private key + Public key

Private Key => Secret key which is stored securely on the client machine.
Public Key => This can be shared with the remote server.
The client machine for that public key can safely connect.
The client can "unlock" the public key with his private key.

NOTE: If the public key of a person is not registered on the remote server, he/she will not be able to connect to it.

SSH for services

Services, like Jenkins, often need to connect to another server via SSH.

Create a Jenkins user on the application server.
Create SSH key pair on Jenkins server.
Add public SSH key to authorized_keys on the application server.

Connect via SSH(Password Authentication)

ssh username@SSHserver(IP address)

For example,

ssh root@159.89.14.94

Generate SSH key pair

mkdir .ssh/ => create a directory for storing keys*(if not there)*.

ssh-keygen -t rsa

This generates ssh key pairs.
Here, -t means type and rsa means the method of encryption.

~/.ssh => .ssh folder under the home directory is the default location for your ssh key pair.

id_rsa => Private key.

id_rsa.pub => Public key.

Copy files to the remote server

scp(secure copy) => allows you to securely copy files and directories.

For example,

scp test.sh root@159.89.14.94:/root

Basics of Networking...

Dipanshu Torawane — Mon, 21 Mar 2022 12:01:59 +0000

Basic Networking Terms

LAN(Local Area Network)

Collection of devices connected together in one physical location. Each device has a unique IP(Internet Protocol) address. Devices communicate via these IP addresses.

IP(Internet Protocol)

It's a 32-bit value.

1 bit: 1 or 0.
00000000: 0
11111111: 255

For example, 172.16.0.0

IP addresses can range from 0.0.0.0 to 255.255.255.255

IP address range

Switch

It sits within the LAN and facilitates the connection between all the devices within the LAN.

Gateway is the IP address of the router.

Router

It sits between LAN and outside networks (WAN: Wide Area Network).

It connects devices on LAN and WAN and allows networked devices to access the Internet.

A subnet is the logical subdivision of an IP network.

How to know whether the other device is inside or outside the LAN?

It knows because of the IP address of the target device.
Devices in the LAN belong to the same IP address range.

Subnetting is the process of dividing a network into two or more networks.

CIDR(Classless Inter-Domain Routing) block

It is a subnet mask that dictates how many bits are fixed.

255.255.0.0 => means that 16 bits are fixed
255.255.255.0 => means that 24 bits are fixed

Any device needs 3 pieces of data for communication:

IP address
Subnet
Gateway

NAT(Network Address Translation)

It's the IP address range chosen by an administrator.

Each device gets a unique IP from that range.

How to make sure that IP addresses don't overlap?

IP addresses within LAN are not visible to the outside network or internet.
Your laptop's private IP address is replaced by the router.

Benefits of NAT

Security and Protection of devices within LAN.
Reuse IP addresses.

There are a limited no. of IPv4 addresses available - 4,294,967,296 public addresses.

Firewall

A system that prevents unauthorized access from entering a private network.

Using Firewall Rules you can define, which requests are allowed.

What are these firewall rules?

It defines which IP address in your network is accessible and which IP address can access your server.
- For example, you can allow any device to access your server.
It allows the device's IP address at the port to be accessed.
This is also called Port Forwarding.

What is a Port?

Port => every device has a set of ports.

You can allow specific ports(doors).

You can allow specific ports(doors) AND specific IP addresses (guests).

Different Applications listen to specific ports.

Standard Ports for many applications

Port 80 => Web Servers
Port 3306 => MySQL DB
Port 5432 => PostgreSQL DB

For every application, you need a port!

Each port is unique on a device.

DNS or Domain Name Service

It translates domain names to IP addresses.

Why do we use names instead of IP addresses?

Humans are better at remembering words & names instead of numbers.

For example, www.facebook.com is converted to 69.63.176.13 (Mapping IP addresses to names).

How does DNS manage all these IP addresses?

Domain Names have a hierarchical structure.

Root Domains (there are 13 root domains named from a to m)

Top Level Domain

There are also geographical domains that were added later.

Who manages these names?

Who can keeps the track of the availability of names?

This is all regulated by a dedicated organization called Internet Corporation for Assigned Names & Numbers*(ICANN)*.

This organization manages the TLD development and architecture of the internet domain space and authorizes Domain Name Registrars, which register and assign domain names.

Each domain name can have a subdomain name with an IP address range of the domain name.

How does the DNS Resolution work?

DNS entries are cached for efficiency.

Networking Commands

ifconfig => shows all network details.
netstat => shows an active connection on your machine.
ps -aux => shows currently running applications on which port they are listening, computer resources they are consuming, etc.
nslookup => you can get the IP address of any domain name.
ping => checks whether a service/application is accessible.

These are used to troubleshoot your network.

And of course, there are parameters for all those commands.

Let's learn VIM and also User Management...

Dipanshu Torawane — Wed, 16 Mar 2022 13:27:08 +0000

There are different text editors like emacs, nano, vim, etc. But vim text editor has gained popularity between developers and sysadmins.

The main takeaway is we learn to insert text in vim and more specifically learn about quitting the vim editor🤣.

Working with VIM Editor

How do we edit files in CLI?

We have a built-in text editor in Linux like nano, vi/vim(Improved version of vi), etc.

Use cases to use the text editor in CLI :

small modifications can be faster, especially when you are currently working in CLI.
faster to create & edit at the same time.
supports multiple formats.
when working on a remote server it speeds up the workflow.

Important VIM commands

vim [filename] => opens the file with vim.

vim has two modes :

Command Mode

this is the default mode and you cannot edit the text.
whatever you type is interpreted as a command.
Navigate, Search, Delete, Undo, etc. can be done in this mode.

Insert Mode

allows you to enter text.
to switch from command mode to insert mode, press 'i'.
and from insert to command mode, press 'Esc'.

NOTE: All these below commands should be run in command mode.

:wq => writes and saves the file to disk and quits the vim editor.
:q! => quits the vim editor without saving the changes.
dd => delete an entire line.
d[number] => delete a bunch of lines.
- For example, d10 -> deletes the next 10 lines.
u => undo the changes.
A => jumps to the end of the line and switches to insert mode.
0 => jumps to the start of the line(but does not switch to insert mode).
$ => jumps to the end of the line(without switching to insert mode).
[number]G => go to the line [number]
- For example, 12G -> go to line 12.
/[pattern/word] => search for that pattern/word.
- For example, /nginx -> searches for the pattern/word 'nginx'
Type 'n' => jumps to the next match.
Type 'N' => jumps to the previous match/ in opposite direction.
:%s/[old]/[new] => replace the old word with the new word throughout the file.

Linux Accounts and Groups

There are three user categories :

i. superuser*(sudo)* account

this is the root user with unrestricted permissions.
used for administrative tasks*(No need to log in as a Root user or execute any commands as root*(sudo command)).

ii. user account

a regular user, we create to login.
- For example, username=dipanshu then it is found under /home/dipanshu.

iii. service account

relevant for Linux server distros as each service will get its own user.
- For example, a MySQL user will start a MySQL application(Best practice for security).

NOTE:

Don't run services with a root user
Always keep one root user per computer.

Can we have multiple regular users & service users and why multiple standard users?

Many companies use windows for their employees. Usually, employees can log in to their accounts on every computer.

How does this work?

Windows is able to centrally manage users. Admins add users to the system and all computers are connected to this system.
When someone tries to log in, OS checks it with the system*(Only have access to their* home folder)[completely isolated system](No access to system files or other user's home folder).
But Linux doesn't have this centrally managed system. This is the reason why Windows is preferred in companies or universities.

Multiple Users on Linux

User accounts are managed on that specific hardware*(cannot access from any other hardware)*

Multiple Users on a Server

For Linux having a multi-user is important for servers. Usually, teams administer a server.

Why not just use a shared user? & Why having a user for each team member is important?

They need a non-root user. Permissions can be assigned per team member.

Traceability => who did what on the system?

Admin creates a user with permissions.

Groups and Permissions

How to manage Permissions?

User Level => give permission to the user directly.

Group Level => group users into Linux groups and give permissions to the group.

This is the way to go is if you manage multiple users*(Best Practice)*.

User management in practice

Access Control Files :

/etc/passwd => stores user account information and everyone can read it but only the root user can change the file.

/etc/shadow => contains information about the system user's passwords.

/etc/group => contains a list of groups and the members belonging to each group.

Managing users :

Do not edit these access control files with a text editor. Instead, use the dedicated commands.

Commands for user/group creation and management

NOTE: *root* user privileges are required*.*

adduser [username] => creates a new user and automatically creates a home directory with skeletal configuration.

Where does the primary group come from?

Whenever we create a user using useradd , by default, it creates the same group named the user and sets that as the primary group ID of the user.

passwd [username] => changes the password of a user.

su - [username] => login as a username( su - short for substitute or switch user).

su - => login as root.

groupadd [groupname] => creates a new group and by default, the system assigns the next available GID from the range of group IDs specified in the login.defs file.

Different User and Group Commands

adduser addgroup

Interactive and more user-friendly*(easier to use).* It chooses conformant UID and GUID values for you.
Creates a home directory with skeletal config automatically or asks for input in interactive mode.

useradd groupadd

You need to provide the information. Low-level utilities.

Which one to use?

adduser addgroup => when executing it manually.

useradd groupadd => when executed in an automated way.

Same goes for deluser(deletes user), delgroup(deletes groups) and userdel groupdel.

Modifying user account

usermod [options] [username] => modify a user account.

for changing group => usermod -g [groupname] [username]

In addition to one primary group, the user can have multiple secondary groups and as a result, the user will get all the permissions that the groups have to which it belongs*(which means we can add one user to multiple groups)*.

sudo usermod -G [groupsname(separated with commas)] [username]

add a user to multiple groups and the option will overwrite the whole secondary group's list. So, it always sets a new list of groups for the user.
If you need to add a user to a new secondary group in addition to the existing ones it already belongs to we use sudo usermod -aG [groupname] [username]

groups [username] -> displays the groups to which the user is added.

More efficient way

useradd [options] [username] => creates a new user.

the low-level command compared to adduser

-G => create a user with multiple secondary groups

-d => custom home directory

sudo gpasswd -d [username] [groupname] => removes the user from the specified group.

File Permissions and Ownership

User permissions are related to reading, writing and executing files in Linux.

ls -l => print files in a long listing format.

Ownership => It means who owns the file/directory.

There are two levels of a file/directory.

Which user owns the file?
- The owner is the user who created the file.
Which group owns the file?
- The owning group is the primary group of that user.

(should be executed with root privileges)

chown [username]:[groupname] [filename] => changes the ownership*(in this command you can also change user by not adding* [groupname])

sudo chgrp [groupname] [filename] => changes the group

File Permissions

Modifying Permissions

[Symbolic values method]

sudo chmod [flags] [filename] => Changes the permission for that file and for all users.

Flags :

+r -> add read permission
+w -> add write permission
+x -> add execute permission
-r -> takes away read permission
-w -> takes away writing permission
-x -> takes away execute permission

Adjusting permissions :

Owner(u), Group(g), Others(o) & All(a)
- For example, sudo chmod g-w config.yaml => removes the write permission for the group.

Alternate ways to give permissions :

sudo chmod u/g/o/a=rwx [filename] => gives multiple permission to mentioned ones.

Type out the whole rwx block or r--/rw- etc.

[Numeric values method]

sudo chmod [numeric value] [filename]

Bonus Content :)

Input, Output and Pipes in Linux

Every program has input and output. The output from one program can become the input of another command.
The syntax for this is the "pipe" command |. It pipes the output of the previous command as an input to the next command.

For example, cat /var/log/syslog (outputs log in non-user friendly way) | less (helps to see the output page-by-page)(to jump to the next page press 'spacebar' and to jump to the previous page press 'b')

To filter commands for specific keywords we use grep [keyword] =>

grep stands for Globally Search for Regular expression and print-out. And it searches for a particular pattern of characters & displays all lines which contain that pattern.
To search for a phrase we have to enclose in (" ") double-quotes.

Redirects in Linux

It is used to save the output of another command into a file.

Redirection

> character is the redirect operator.
Takes the output from the previous command and sends it to a file that you mention. For example, history | grep sudo > sudo-commands.txt

In order to add new lines in the existing file by using redirect can be done using

>> => append text to the end of the file.

Standard Input and Standard Output

Every program has 3 built-in streams

STDIN(0) => Standard Input
STDOUT(1) => Standard Output
STDERR(2) => Standard Error

We pipe or redirect the standard output from one command to the standard output of another command. We cannot execute many commands in one line but can be done by separating them with ;(semi-colon).

For example, clear; sleep 2; echo "Hello, welcome back"

All you need to get started with Linux...

Dipanshu Torawane — Mon, 14 Mar 2022 13:25:25 +0000

NOTE: Everything in Linux is a file. For example, text documents, pictures, directories, and devices like printers, keyboards, USBs, CDs, etc.

So this article we will learn all the basics of Linux and commands to get us started to become a power-user of Linux.

File Manipulation, moving to different folders and removing files commands

pwd(print working directory) => Prints the current directory in which we are present onto the terminal.
cd [dir name] => Change Directory means moving from the current directory to the [dir name] directory.
cd .. => Move one level up or to the previous directory from the current directory.
mkdir [dir name] => Make a directory of [dir name] in the current directory.
ls => Lists all files and directories in the present working directory.
ls -R => Lists files in sub-directories as well.
ls -al => Lists files and directories with detailed information like permissions, size, owner, etc.
ls -a => Shows all files & folders*(including hidden)*
touch [filename] => creates [filename] with the provided extension.
rm [filename] => Deletes [filename]
rmdir or rm -d => Deletes a directory

NOTE: Here rm commands delete the file permanently.
rm -r(recursively) => Removes file or folder if it contains files inside it.

Displaying contents of the file, moving/copying the file from one folder to another

cat [filename] => Displays the file content.
cat file1 file2 > file3 => Joins two files (file1, file2) and stores the output in a new file (file3).
mv file "new file path" => Moves the file to the new provided file location.
mv filename new_file_name => Renames the file to a new filename.
cp -r [dirname] [new_filename] => Copy [dirname] to new_filename.

Superuser, manual, history and some common terminal commands

sudo => Allows regular users to run programs with the security privileges of the superuser or root.
man => Gives the whole documentation of commands
history => Gives a list of all past commands typed in the current terminal session.
history [number] => Gives the previous [numbers] of the command typed in the session.
clear => Clears the terminal.
Ctrl + C => Stops the current command completely instead of running in the background.
Ctrl + Shift + C => Copies the selected text from the terminal.
Ctrl + Shift + V => Pastes the copied text from other terminal session or browser.
Ctrl + R => Search the last history of command.
uname -a => Shows information about system and kernel.
lscpu => Shows information about the CPU in the machine.
lsmem => Shows information about memory in the machine.
su - [name of the user] =>Switchs the user to [name of the user].
cat /etc/os-release => Gives information about distribution, version, etc.
init 0 => To shutdown/poweroff machine.
init 6 => To restart/reboot machine.

Package Manager

In Windows, you download the installer and then installs it through the installation wizard. But in Linux, we don't use this way, we install most of the software/apps using the package manager tool.

What is a software package?

A compressed archive, containing all required files like the apps usually have some dependencies or required some other softwares to run.

For example, Firefox Browser requires dependencies that are not packaged into the archive. These dependencies also need to be installed.

What is a package manager?

Files are split across different folders. Not like Windows, where every program is installed in a single file in Program Files. In Windows, it is also easy to uninstall programs as they are in a single folder but in Linux, managing apps uninstalling everything completely is more difficult(as they are split).

So there are different package managers as per the Linux distros which

downloads, installs, or updates existing software from a repository.
ensures the integrity & authenticity of the package.
manages & resolves all required dependencies.
knows where to put all the files in the Linux file system.
easy upgrading of the software.

In Ubuntu, you have an APT(advance package tool) package manager available.

Commands used in package manager(commands should be used with sudo)

apt search [package_name] : Search for a package of package_name.
apt install [package_name] : Installs the package with package_name.
apt install [package_name1] [package_name2] : Install multiple packages with one command.
apt remove [package_name] : Completely removes installed package of that package_name.

The benefits of using a package manager are one central place to install, upgrade, configure and remove the software.

Difference b/w APT & APT-GET

APT:

more user-friendly, like it has a progress bar.
fewer, but sufficient command options in a more organized way(Use apt because recommended by Linux distributions).

APT-GET:

search command not available
you can achieve the same if you use additional command options.

Where do these Packages come from?

Package Manager fetches the packages from the Ubuntu registered/validated repositories.

Where do these files store?

Repository*(storage location)* containing thousands of repositories of programs.

Best practice

Always update the package index before upgrading or installing new packages.

apt update :

updates the package index
pulls the latest changes from the APT repositories
the APT package index is basically a database
holding records of available packages from the repositories.

add-apt-repository :

add the repository to the official list of repository
add repository to APT sources (into /etc/apt/sources.list)

Why add a repository?

When installing relatively new applications which are not in the official repository yet.

Repositories Types

Repositories are of type PPA(Personal Package Archive)

PPA's are provided by the community. Anybody can create this PPA-private repository to distribute the software. Usually used by developers to provide updates more quickly than in the official Ubuntu repositories.

Be Aware of possible risks before adding a PPA:

no guarantee of quality or security
like any other unofficial software package it can cause difficulties, for example, when upgrading to new Ubuntu release.

Linux and its way of storing files...

Dipanshu Torawane — Fri, 11 Mar 2022 12:26:54 +0000

What is the Linux File System?

The file system is generally a built-in layer of a Linux operating system used to handle the data management of the storage. It helps to arrange the file on disk storage and manages the file name, file size, creation date, and much more information about a file.

The Linux file system is a structured collection of files on a disk drive or a partition. A partition is a segment of memory that contains some specific data. In our machine, there can be various partitions of memory. Generally, every partition contains a file system.

The files are stored in the hierarchical tree structure. Root folder comes first and then files are under root respectively. Whereas in windows, files are stored in folders on different data drives like C: D: E:

Linux File System Overview

/ or /root => root user's home directory which is separate from other users

/home => contains the home directories of all non-root users.

/bin(stands for binaries) => executables for most essential user commands(all files are system-wide usable)

What is binary?

A binary is just a computer-readable format (a bunch of 0's & 1's)

/sbin(system binaries) => essential system binaries, programs that admin would use(need superuser privileges)

/lib, /lib32, /lib64, /libx32 => essential shared libraries that executables from /bin or /sbin use.

/usr(user) => this was used for user home or directories*(contains the same* /bin, /sbin, /lib folders).

Why do /usr contain these folders?

historic reasons(as UNIX also had those splits)
Because of storage limitations, it was split to root binary folders ad user binary folders.

NOTE: When we execute basic Linux commands they are executed from /usr/bin folder. In other distros, /bin or /sbin contains fewer commands and /usr/bin or /sbin contains all other commands.

Inside /usr, there is a folder /local that also contains the same files as /usr(/bin or /sbin or /lib).

/usr/local => programs that you install on the computer. Third-party applications like docker, minikube, etc go here. Programs installed here will be available for all users on the computer.

NOTE: If you want to install third-party applications and don't want to be accessed by all the other users then you install them on your own /home directory.

/boot(booting) => contains files required for booting.

/opt(optional) => third-party programs you install(all programs will be available system-wide).

Difference between /usr/local & /opt:-

/usr/local: programs that split into their components.
/opt: programs which not split into their components.

NOTE: Till now all folders are read-only folders.

/etc(et cetera) => a place where the configuration for system-wide applications is stored. Originally for etc*(means everything else goes there)*. It emerged into the main configuration location.

/dev(devices) => location of device files like webcam, keyboard, hard drives, etc. Apps and Drivers will access this, NOT the user.

/var(variable) => contains files to which the system writes data during the course of its operation.

/var/log => contains log files.

/var/cache => contains cached data from application programs.

/tmp(temporary) => temporary resources required for some processes kept here temporarily.

/media(removable media) => contains subdirectories where removable media devices inserted into the computer are mounted. For example, when you insert a CD, a directory will automatically be created and you can access the contents of the CD inside the directory.

/mnt(temporary mount points) => historically, sysadmins mounted temporary file systems there(or manually mounting folders).

Interacting with these folders

Usually, you are not interacting with these folders. Installing apps with a package manager is responsible to store the required files in the right corresponding folders.

Hidden Files

These are primarily used to help prevent important data from being accidentally deleted and are automatically generated by programs or operating system. The file name starts with .(dot). These are also called 'dotfiles'.

Why operating system and virtualization...

Dipanshu Torawane — Fri, 11 Mar 2022 12:24:13 +0000

We in this modern era are the power users of computers which contain an interface to interact and perform our day-to-day tasks. But really why is there an interface and why you should care about it?

Every computer is made up of hardware such as CPU, memory, storage, and I/O devices like a monitor, keyboard, and mouse. All the programs running on the computer need to use these hardware resources.

How do we run apps if there were no operating systems?

Let us understand what it takes to run your favourite browser on your machine. Use of browsers like Chrome*(pretty heavy duty)* will require some CPU, memory & I/O devices to interact and do the task (how to center a div)😅.

Some things to keep in mind are that

apps don't know how to talk to the hardware,
apps can't be installed directly on the hardware.

We need to write code for that application and configuration with the hardware in order to use the application on the hardware. Imagine doing this for every application we need to use, it will be so inefficient. Writing code and configs require knowledge of programming languages, converting to machine code*(0's and 1's)* and then optimising it for better performance. Ohh god, feels like a long TDS and inefficient process.

As to remove this inefficiency we use an intermediate to talk to the hardware which can handle all these for us which we call an Operating System. The operating system can interact with hardware & applications so it is like a middleman between the two.

What does an O.S. do to make our life easier?

We focus on performing day-to-day without caring about managing, allocating resources and so on. But there's a lot of stuff happening behind the scenes.

The above image shows the functions and tasks that are performed in order to run our next-level heavy-duty applications.

What is the operating system made up of?

The operating system contains layers of software that have specific tasks when a computer is powered on. It starts with the kernel, a program that consists of device drivers, dispatchers, schedulers, file systems, etc.

The kernel always loads first. It is the heart of every operating system responsible for managing hardware components. It starts the process for the app, allocates the required resources and also cleans up when not needed.

Linux Distros like Ubuntu, Mint, CentOS, etc, and Android are based on the Linux kernel. Linux Kernel is the most widely used. On the other hand, macOS & IOS are based on different kernels known as Darwin.

How to interact with the kernel?

Graphical User Interface*(GUI)* and Command Line Interface*(CLI)*, both are ways to interact with the kernel and have their own importance.

GUI uses a combination of technologies and devices to provide a platform that users can interact with, for the tasks of gathering and producing information. A series of elements conforming to a visual language have evolved to represent information stored in computers. This makes it easier for people with few computer skills to work with and use computer software. The most common combination of such elements in GUIs is the windows, icons, text fields, canvases, menus, and pointer (WIMP) paradigm, especially in personal computers.

The WIMP style of interaction uses a virtual input device to represent the position of a pointing device's interface, most often a mouse, and presents information organized in windows and represented with icons. Available commands are compiled together in menus, and actions are performed by making gestures with the pointing device. A window manager facilitates the interactions between windows, applications, and the windowing system. The windowing system handles hardware devices such as pointing devices, graphics hardware, and positioning of the pointer.

In personal computers, all these elements are modelled through a desktop metaphor to produce a simulation called a desktop environment in which the display represents a desktop, on which documents and folders of documents can be placed. Window managers and other software combine to simulate the desktop environment with varying degrees of realism.

There are three main operating systems for computers:-

Linux
Windows
macOS

Each OS has many versions. The kernel stays the same but the application layer is being developed & improved.

The operating system for servers (which are extensively used in Software Development to the apps reach the user with high availability, and no downtime). The servers are mostly Linux-based (more lightweight and performant).

The servers use the command-line interface (no GUI or other user applications). CLI is commonly used to debug apps that are accidentally stopped due to bugs, on the servers they are deployed on. This interface gives us the whole control of the OS with the use of commands and is also used to automate some stuff on the servers with the help of scripts*(are written using scripting languages that OS provides as Linux provides with bash as the terminal and bash scripting is used to write the scripts)*.

The servers use the concept of virtualization and virtual machines to make efficient use of available resources and these are the building blocks of cloud computing.

Virtualization and Virtual Machines.

What is a Virtual Machine(vm)?

A computer system is created using software on a physical computer in order to emulate the functionality of another separate physical computer. Virtual Machines are completely isolated means if something breaks inside the vm, it doesn't affect the host machine.

And if it breaks, you can delete that vm and create a new fresh one easily. This is achieved by using Hypervisor (a technology that allows multiple virtual computers on a physical computer host OS on top of the OS that is already installed).

VMware was the one to come up with virtualization. Nowadays people can add their own hypervisor called Oracle VirtualBox which is open-source, in order to test different operating systems and test the compatibility of their applications on different OS so that every user has a great experience while using the app.

VirtualBox takes hardware resources from the host OS and creates a virtual CPU, virtual RAM, and virtual storage for each virtual machine. You can only give resources that you actually have, which means hardware resources are shared.

The benefits of using a Virtual Machine are to learn and experiment with different operating systems as you don't need to buy a new computer for that. Mainly, you don't endanger the main OS.

There are two types of Hypervisors

Type 1 Hypervisor
- Type 1 Hypervisor use cases efficient usage of hardware resources. As to use all the resources of a performant big server and users can choose any resource combination.
- Some big cloud providers like AWS, Azure, GCP, etc use this type of Hypervisor.
Type 2 Hypervisor
- VMware and VirtualBox are examples of this type of Hypervisor which gives users flexibility for various use cases.

Why are companies adopting Virtualization?

The main benefit is the abstraction of OS from the hardware.

Without Virtualization =>

Companies have their own servers and are managed by themselves.
OS is tightly coupled to the hardware.
If any part of the hardware fails, the whole data is gone*(one-point failure)*.

With Virtualization =>

OS acts as a portable file(Virtual Machine Image includes the operating system & all applications on it).
You can have backups of your own OS*(known as* snapshots).
The OS is portable and not dependent on the physical server.
They can be secured very easily.

Thanks for reading and hope you would have now understood what is Operating System, and how it works. The main takeaway is Virtualization, the backbone of the servers to make efficient use of resources.

Please do share your thoughts in the comments and like the blog so that it reaches everyone who is stumbling across this topic and want to understand the idea behind it.

You can find my all socials at => bio.link/dipanshutorawane

All you need to know about DevOps...

Dipanshu Torawane — Wed, 09 Mar 2022 16:45:26 +0000

Introduction to DevOps

DevOps is a pretty hot topic since the evolution of the Software Development Life Cycle from just running one app per server to virtualization to containerised applications. Most people confuse it and assume this is a new field as ML, SDE, or others but it is a set of practices that help organizations innovate faster by automating and streamlining the software development and infrastructure management processes. Most of these practices are accomplished with proper tooling.

WATERFALL methodology

Traditionally, the WATERFALL method was a practice for developing new software which was a linear process and took more time to implement the idea.

Let's say a team had an idea of making a new social media platform like Facebook, or Instagram. The team will first have to plan everything about how the app should work, and the features it should have, and make a list of features, pages, events, etc.

Then the documentation of how the business team envisioned the app and the whole list of features, events, etc are given to the development team. This team will then develop one feature per time, and this will take some months to code features, pages, etc., and develop the complete app.

The next step is testing the developed app *(*Testing will be done only after everything has been developed) which is a very crucial process to implement a complex app like Facebook or Instagram. This is then hand-over to the testing department which will test every feature, app performance, load testing, etc. They might find some bugs or compatibility issues and it will send it back to the developer team to fix it, and then it will get tested again. This cycle would continue back & forth till everything is ready with no issues. This will take some months depending upon the complexity.

After development and testing it will be passed on to the Operations Team. They will require some preparations to set up the app on the server. There they will have to resolve dependencies, or some things wouldn't run or break, they will again send it to the developer's team to fix it, then to the testing team to test again and this cycle again would take a couple of months.

Maybe after a couple of years, the app is ready, deployed, and running.

This method sounds like a long and ineffective process. Over time new requirements may arise, and then they need to be resolved. There will be many places of failure and miscommunication as there is no fast feedback.

Agile methodology

A very effective method alternative to the WATERFALL method was developed called Agile. It is very popular among organizations and companies.

Agile methodology allows for a more dynamic, fast, and easily changeable development and deployment process. And it is just a methodology or framework that says what are the good practices and efficient ways to do things. It has specific implementations like Scrum and Kanban. Depending on company requirements they will use one of them.

Problems solved that were lacking in the WATERFALL method :

It increases the speed of development, testing, and deployment cycles. Instead of waiting for the whole application to get developed.
Here each feature gets tested and deployed.
There is immediate feedback if there are bugs or deployment doesn't work and so on.

The main takeaway is Agile at the core is the heart of the Continuous Integration and Continuous Delivery process which has become so popular that the company strives towards making its software development and delivery processes much more efficient.

DevOps

The introduction or adoption of Agile methodology introduction or adoption by companies has supported the need for DevOps tasks. Because DevOps is implementing the best practices or most of the practices that the Agile framework introduces.

DevOps is an approach or practice that combines the development team and the IT team to collaborate throughout the entire software development cycle so that all the teams will be on the same page. It helps in shortening the Software Development Life Cycle and ensures the delivery of high-quality software.

In simple words, DevOps is just a way of working between Developers & Operations.

The main aim of DevOps is CI/CD*(Continuous Integration & Continuous Delivery)* to develop and deploy applications/features as fast as possible and automate tasks.

Thanks for reading and hope you would have now understood the need for DevOps and what gained it so much popularity.

Please do share your thoughts in the comments and like the blog so that it reaches everyone who is stumbling across this topic and want to understand the idea behind it.

You can find my all socials at => https://bio.link/dipanshutorawane