DEV Community: Mahbub Ferdous Bijoy

YAML Syntax And So More

Mahbub Ferdous Bijoy — Mon, 07 Oct 2024 20:25:18 +0000

here is key : value

key : value
app : user-authentication


#this all about key value just remember to put indentation

there is all variable in YAML.

#string
app : user-authentication

#integer 
port : 8000

#float
version : 1.7

#boolean
deployed : true

here is all objects YAML

microservices : 
  app : payment-app
  port : 9001
  version : 1.9
  deployed : false

here is list

fruits : 
  - apple
  - kiwi
  - banana
  - orange
  - nuts : 
    - almond
    - cashew

here is all multiline yaml, list with objects

---
microservices : 
  - name : user-authentication
    port : 9001
    version : 1.9
    deployed : true
  - name : payment-app
    port : 9002
    version : 1.8
    deployed : false
  - name : db-app
    port : 9003
    version : 1.8
    deployed : true 

---

## schools --> name, principal --> students --> name,rollnumber,parents,number

schools:
- name : MHS
  principal : Gius
  students :
  - name: Mahbub
    roolnumber : 7
    parents : 
    - father : rahim
    - mother : josna
    number: 460

YAML - YAML Ain't Markup Language

Mahbub Ferdous Bijoy — Mon, 07 Oct 2024 20:19:44 +0000

What is YAML?

YAML is a human-readable data serialization language that is often used for writing configuration files. Depending on whom you ask, YAML stands for yet another markup language or YAML ain’t markup language (a recursive acronym), which emphasizes that YAML is for data, not documents.

YAML is a popular programming language because it is designed to be easy to read and understand. It can also be used in conjunction with other programming languages. Because of its flexibility, and accessibility, YAML is used for Docker-compose,Kubernetes,Ansible

YAML syntax

YAML files use a .yml or .yaml extension, and follow specific syntax rules.

YAML has features that come from Perl, C, XML, HTML, and other programming languages. YAML is also a superset of JSON, so JSON files are valid in YAML.

There are no usual format symbols, such as braces, square brackets, closing tags, or quotation marks. And YAML files are simpler to read as they use Python-style indentation to determine the structure and indicate nesting. Tab characters are not allowed by design, to maintain portability across systems, so whitespaces—literal space characters—are used instead.

Comments can be identified with a pound or hash symbol (#). It’s always a best practice to use comments, as they describe the intention of the code. YAML does not support multi-line comment, each line needs to be suffixed with the pound character.

A common question for YAML beginners is “What do the 3 dashes mean?” 3 dashes (---) are used to signal the start of a document, while each document ends with three dots (...).

indentation is more important for YAML

#Comment: This is a supermarket list using YAML
#Note that - character represents the list
---
food: 
  - vegetables: tomatoes #first list item
  - fruits: #second list item
      citrics: oranges 
      tropical: bananas
      nuts: peanuts
      sweets: raisins

Note that the structure of a YAML file is a map or a list, and it follows a hierarchy depending on the indentation, and how you define your key values. Maps allow you to associate key-value pairs. Each key must be unique, and the order doesn't matter. Think of a Python dictionary or a variable assignment in a Bash script.

A map in YAML needs to be resolved before it can be closed, and a new map is created. A new map can be created by either increasing the indentation level or by resolving the previous map and starting an adjacent map.

A list includes values listed in a specific order and may contain any number of items needed. A list sequence starts with a dash (-) and a space, while indentation separates it from the parent. You can think of a sequence as a Python list or an array in Bash or Perl. A list can be embedded into a map.

In the example provided above “vegetables” and “fruits” represent items that are part of the list named “food”.

YAML also contains scalars, which are arbitrary data (encoded in Unicode) that can be used as values such as strings, integers, dates, numbers, or booleans.

When creating a YAML file, you’ll need to ensure that you follow these syntax rules and that your file is valid. To achieve it, you can use a linter—an application that verifies the syntax of a file. The yamllint command can help to ensure you’ve created a valid YAML file before you hand it over to an application.

What is YAML used for?

One of the most common uses for YAML is to create configuration files. It's recommended that configuration files be written in YAML rather than JSON, even though they can be used interchangeably in most cases, because YAML has better readability and is more user-friendly.

In addition to its use in Ansible, YAML is used for Kubernetes resources and deployments.

A benefit of using YAML is that YAML files can be added to source control, such as Github, so that changes can be tracked and audited.

YAML in Ansible

Ansible Playbooks are used to orchestrate IT processes. A playbook is a YAML file containing 1 or more plays, and is used to define the desired state of a system.

Each play can run one or more tasks, and each task invokes an Ansible module. Modules are used to accomplish automation tasks in Ansible. Ansible modules can be written in any language that can return JSON, such as Ruby, Python, or bash.

An Ansible Playbook consists of maps and lists. To create a playbook, start a YAML list that names the play, and then lists tasks in a sequence. Remember that indentation is not an indication of logical inheritance. Think of each line as a YAML data type (a list or a map).

By using YAML templates, Ansible users can program repetitive tasks to happen automatically without having to learn an advanced programming language. Developers can also use the ansible-lint command, a YAML linter for Ansible Playbooks, to identify mistakes so errors don't occur during a critical stage of operation.

With the introduction of Ansible Lightspeed with IBM Watson Code Assistant, a generative AI service, developers can create Ansible automation content more efficiently. Users can enter a task request in plain English and get clean and compliant YAML code recommendations for automation tasks that are then used to create Ansible Playbooks.

YAML for Kubernetes

Kubernetes works based on defined state and actual state. Kubernetes objects represent the state of a cluster and tell Kubernetes what you want the workload to look like. Kubernetes resources, such as pods, objects, and deployments can be created using YAML files.

When creating a Kubernetes object, you’ll need to include specifications to define the object's desired state. The Kubernetes API can be used to create the object. The request to the API will include the object specifications in JSON, but most often you’ll provide the required information to kubectl as a YAML file. Kubectl will convert the file into YAML for you when it makes the API request.

Once an object has been created and defined, Kubernetes works to make sure that the object always exists.

Developers or sysadmins specify the defined state using the YAML or JSON files they submit to the Kubernetes API. Kubernetes uses a controller to analyze the difference between the new defined state and the actual state in the cluster.

you can check your YAML indentation by using yamllint.com

or you can install yamllint in your system:

$ sudo dnf install -y yamllint

Use yamllint command to validate your YAML file:

$ yamllint --strict dummy.yaml || echo “Fail”

the output will be JSON file .

Corn-jobs with some custom script

Mahbub Ferdous Bijoy — Fri, 04 Oct 2024 17:53:38 +0000

How to Automate Scripts by Scheduling via cron Jobs:

Cron is a job scheduling utility present in Unix like systems. You can schedule jobs to execute daily, weekly, monthly or in a specific time of the day. Automation in Linux heavily relies on cron jobs like network troubleshooting, process managment,cpu and memory management day to day same time.

Below is the syntax to schedule crons:

* * * * * sh /path/to/script.sh

sh represents that the script is a bash script and should be run from /bin/bash.
/path/to/script.sh specifies the path to script

Here, (*) represents minute(s) hour(s) day(s) month(s) weekday(s), respectively.

1th (*) for : minutes -> 0-59
2nd (*) for : hour -> 0-24
3rd (*) for : date -> 1-31
4th (*) for : month -> 1-12
5th (*) for : days -> 0-6

5 0 8 : at 00.05 in august
5 4 6 : at 4.05 on sunday
0 22 1-5 : At 22:00 on every day-of-week from Monday through Friday.

most important crontab command:

edits crontab entries to add, delete, or edit cron jobs: $ crontab -e
list all the cron jobs for the current user: $ crontab -l
list another user's crons: crontab -u userName -l
edit another user's crons: crontab -u username -e

Example:

#! /bin/bash

#make a date-script.sh file 

echo date >> date-output.txt

# make this file executabel:

chmod 774 date-script.sh

# add this in crontab for every minutes:

crontab -e 

#and paste this line:
*/1 * * * * sh date-script.shFilePath
#now do cat in date-output.txt you will see all the output

nodeHealth.sh

#!/bin/bash

#####################################################

# Author : Mahbub 
# Date : 04/10/24
# Tags: Check node health
# Description: This script outputs the node health
# Version : v1

######################################################

#debug mode for know which output which command:
set -x 

#print the disk space
df -h

#print the memory 
free -g

#print the resources like cpu
nproc

check_process_running.sh:

#!/bin/bash

#####################################################

# Author : Mahbub 
# Date : 07/10/24
# Tags: Check all processes
# Description: This script outputs the all runing process
# Version : v1

######################################################

#set debug mode for human readable
set -x
#when script get an error :
set -e    # exit the script when there is an error
set -o pipefail  # if we use pipe operator and its have problem without this line command will executed


#check process command:
ps -ef

# check human readable:
ps -h

#if you want to know selected PID(processID) using querythan we can use pipe(|) and (grep)

#ps -ef | grep "processName"
ps -ef | grep "zsh"
#or we can get output in a txt file: here we use backtic for commands 

# echo ps -h | grep "processId" >> ps-output.txt
echo ps -h | grep "zsh" >> zsh-output.txt
#must use backtick after echo command
# awk command for get info column by column :

ps -ef | grep zsh | awk -F" " '{print $2}'   # here $2 means 2nd column

get_error_from_logfile.sh:

#!/bin/bash

#####################################################

# Author : Mahbub 
# Date : 09/10/24
# Tags: Find ERROR in logfile from remote storage 
# Description: This script helps to get error log from remote logfile 
# Version : v1

######################################################

#add debug mode
set -x
set -e
set -o pipefail

# in general logfile are big and also important thats why every company put it on amazon s3 or google drive or github. i mean any remote storage service they use 

# to get error from remote log file we use curl :

curl https://www.ibm.com/docs/en/zos/2.4.0?topic=problems-example-log-file | grep WARNING

# here after grep we can use INFO,WARNING,ERROR,TRACE

Shell -If/elif/else and loops

Mahbub Ferdous Bijoy — Fri, 04 Oct 2024 17:38:07 +0000

Numeric Comparison logical operators:

Comparison is used to check if statements evaluate to true or false. We can use the below shown operators to compare two statements:

Equality :num1 -eq num2 --------> is num1 equal to num2
Greater than equal to:num1 -ge num2 --------> is num1 greater than equal to num2
Greater than :num1 -gt num2 --------> is num1 greater than num2
Less than equal to :num1 -le num2 --------> is num1 less than equal to num2
Less than :num1 -lt num2 --------> is num1 less than num2
Not Equal to :num1 -ne num2 --------> is num1 not equal to num2

if code structure:

if [ conditions ]
    then
         commands
fi

if else code structure:

if [[ condition ]]
then
    statement
elif [[ condition ]]; then
    statement 
fi

if elif else code structure:

if [[ condition ]]
then
    statement
elif [[ condition ]]; then
    statement 
else
    statement
fi

we can use AND -a and OR -o as well.

Example:

#! /bin/bash

read a
read b
read c


# if statement :

if [$a -e $b]
than
    echo a is equal to b
elif [$a -gt $b]
    than
    echo a is greater than b
else [$a -lt $b]
    than
    echo a is less than b
fi 



# we can use AND (-a) and OR (-o) as well.

if [$a -ne $b -o $a -e $b]
    than 
        echo "true"
    elif [$a -gt $b -a $b -ge $c]
        than 
            echo "elif true"
        else
            than
                echo "false"
fi

shell script for...while loops:

#! /bin/bash

# For loops allow you to execute statements a specific number of times:

for i in 0 1 2 3 4 5 6 7 8 9 
    do
        echo $i
done

# for loop for os file system :


for FILE in $HOME/.bash*
do
   echo $FILE
done

# for loop for strings :

for X in cyan magenta yellow  
do
    echo $X
done

# while loop :

i=1
while [[ $i -le 10 ]] ; do
   echo "$i"
  (( i += 1 ))
done

Shell - Variable,Arithmetic Operator, Input

Mahbub Ferdous Bijoy — Fri, 04 Oct 2024 16:50:42 +0000

How to define variables

We can define a variable by using the syntax variable_name=value. To get the value of the variable, add $ before the variable.

Example:

shebang added 
#! /bin/bash

#here our variable example

greetings1=Hello
echo "ENTER YOUR NAME: "
read name
greetings2="How are you?"
echo "$greetings1 $name.$greetings2"

Arithmetic Expressions

Below are the operators supported by bash for mathematical calculations:

1. +    = addition
2. -    = subtraction
3. /    = division
4. *    = exponentiation
5. %    = modulus

for arithmetic operations always make a variable like this:- varName=$((operations))>>> additons=$((10+20))

when you echo all arethmetic operations at a time its work first modulus than division than multiply than subtraction last it will works addition

For decimal calculations:

we can use bc command to get the output to a particular number of decimal places. bc (Bash Calculator) is a command line calculator that supports calculation up to a certain number of decimal points.

echo "scale=2;22/7" | bc

Where scale defines the number of decimal places required in the output.

Examples:

#! /bin/bash

#for arithmetic operations always make a variable like this:-  varName=$((operations))>>> additons=$((10+20))

addition=$((10+10))
subtraction=$((10-8))
division=$((16/4))
exponentiation=$((10*2))
modulus=$((11%2))

#here all arithmetic answer:

echo $addition      
#ans is 20
echo $subtraction   
#ans is 2
echo $division      
#ans is 4
echo $exponentiation  
#ans is 20
echo $modulus         
#ans is 1

here is all functionalities addition:

when you echo all arithmetic operations at a time its work first modulus than division than multiply than subtraction last it will works addition

echo $(($addition - $subtraction + $modulus * $exponentiation))

Ans is 38 >> 1st(*) than (-) than (+)

for decimal calculations:

echo "scale=2;22/7" | bc

Ans will be - 3.14

Where scale defines the number of decimal places required in the output.

How to read user input:

Sometimes you'll need to gather user input and perform relevant operations.

In bash, we can take user input using the read command.

read variable_name

To prompt the user with a custom message, use the -p flag.

read -p "Enter your age" variable_name

Examples:

#! /bin/bash

read -p "ENTER YOUR NAME: " name
read -p "ENTER YOUR AGE: " age

echo "Your Name is : $name and your age : $age."

Introduction to shell scripts

Mahbub Ferdous Bijoy — Fri, 04 Oct 2024 16:42:38 +0000

Introduction to Shell and shell scripting:

If we are using any major operating system, we are indirectly interacting with the shell. While running Ubuntu, Linux Mint, or any other Linux distribution, we are interacting with the shell by using the terminal. In this article we will discuss Linux shells and shell scripting so before understanding shell scripting we have to get familiar with the following terminologies:

Kernel
Shell
Terminal

What is Kernel?

The kernel is a computer program that is the core of a computer’s operating system, with complete control over everything in the system. It manages the following resources of the Linux system –

File management
Process management
I/O management
Memory management
Device management etc.

It is often mistaken that Linus Torvalds has developed Linux OS, but actually, he is only responsible for the development of the Linux kernel.

Complete Linux system = Kernel + GNU system utilities and libraries + other management scripts + installation scripts.

What is Shell?

A shell is a special user program that provides an interface for the user to use operating system services. Shell accepts human-readable commands from users and converts them into something which the kernel can understand. It is a command language interpreter that executes commands read from input devices such as keyboards or from files. The shell gets started when the user logs in or starts the terminal.

Shell is broadly classified into two categories –

Command Line Shell
Graphical shell

Command Line Shell

Shell can be accessed by users using a command line interface. A special program called Terminal in Linux/macOS, or Command Prompt in Windows OS is provided to type in the human-readable commands such as “cat”, “ls” etc. and then it is being executed. The result is then displayed on the terminal to the user.

Graphical Shells

Graphical shells provide means for manipulating programs based on the graphical user interface (GUI), by allowing for operations such as opening, closing, moving, and resizing windows, as well as switching focus between windows. Window OS or Ubuntu OS can be considered as a good example which provides GUI to the user for interacting with the program. Users do not need to type in commands for every action.

There are several shells are available for Linux systems like –

BASH (Bourne Again SHell) – It is the most widely used shell in Linux systems. It is used as default login shell in Linux systems and in macOS. It can also be installed on Windows OS.
CSH (C SHell) – The C shell’s syntax and its usage are very similar to the C programming language.
KSH (Korn SHell) – The Korn Shell was also the base for the POSIX Shell standard specifications etc.

Each shell does the same job but understands different commands and provides different built-in functions.

The Linux command line is provided by a program called the shell. Over the years, the shell program has evolved to cater to various options.

Different users can be configured to use different shells. But most users prefer to stick with the current default shell. The default shell for many Linux distros is the GNU Bourne-Again Shell (bash). Bash is succeeded by Bourne shell (sh).

When you first launch the shell, it uses a startup script located in the .bashrc or .bash_profile file which allows you to customize the behavior of the shell.

When a shell is used interactively, it displays a $ when it is waiting for a command from the user. This is called the shell prompt.

[username@host ~]$

If shell is running as root, the prompt is changed to #. The superuser shell prompt looks like this:

[root@host ~]#

Bash is very powerful as it can simplify certain operations that are hard to accomplish efficiently with a GUI. Remember that most servers do not have a GUI, and it is best to learn to use the powers of a command line interface (CLI).

What is a Bash Script?

A bash script is a series of commands written in a file. These are read and executed by the bash program. The program executes line by line.

For example, you can navigate to a certain path, create a folder and spawn a process inside it using the command line.

You can do the same sequence of steps by saving the commands in a bash script and running it. You can run the script any number of times.

How Do You Identify a Bash Script?

File extension of .sh.
By naming conventions, bash scripts end with a .sh. However, bash scripts can run perfectly fine without the sh extension.

Scripts start with a bash bang.

Scripts are also identified with a shebang. Shebang is a combination of bash # and bang ! followed the the bash shell path. This is the first line of the script. Shebang tells the shell to execute it via bash shell. Shebang is simply an absolute path to the bash interpreter.

Below is an example of the shebang statement.

#! /bin/bash

The path of the bash program can vary. We will see later how to identify it.

Execution rights

Scripts have execution rights for the user executing them.

An execution right is represented by x. rwx(read, write, execute)

File colour

Executable scripts appear in a different colour from rest of the files and folders.

In my case, the scripts with execution rights appear as green.

How to Create Your First Bash Script

Let's create a simple script in bash that outputs Hello World.

Create a file named hello_world.sh

touch hello_world.sh

Find the path to your bash shell.

which bash

In my case, the path is /bin/bash and I will include this in the shebang.

Write the command.

We will echo "hello world" to the console.

Our script will look something like this:

#! /usr/bin/bash
echo "Hello World"

Edit the file hello_world.sh using a text editor of your choice it is vim

Provide execution rights to your user.

Modify the file permissions and allow execution of the script by using the command below:

chmod u+x hello_world.sh

chmod modifies the existing rights of a file for a particular user. We are adding +x to user u.

Run the script.

You can run the script in the following ways:

./hello_world.sh

bash hello_world.sh.

and see the output

Two ways to run scripts

examples:

##### added shebang #####
#! /bin/bash               
### write our commands or scripts here:
echo "hello world"
echo "Enter your Name: "
read name
echo "$name"
pwd
ls
whoami

Day-6: Network Troubleshooting, Network config, Dns

Mahbub Ferdous Bijoy — Mon, 30 Sep 2024 16:52:21 +0000

copy and rsync command:

To copy a file over from local host to a remote host:

$ $ scp myfile.txt username@remotehost.com:/remote/directory

To copy a file from a remote host to your local host:

$ scp username@remotehost.com:/remote/directory/myfile.txt /local/directory

To copy over a directory from your local host to a remote host:

$ scp -r mydir username@remotehost.com:/remote/directory

rsync files on the same host:

$ rsync -zvr /my/local/directory/one /my/local/directory/two

rsync files to local host from a remote host:

$ rsync /local/directory username@remotehost.com:/remote/directory

rsync files to a remote host from a local host:

$ rsync username@remotehost.com:/remote/directory /local/directory

simple server command :

run simple python server command :

$ python -m SimpleHTTPServer

here is network routing comamnd:

Look at your machine's routing table:

$ route -n

Gateway

If we are sending a packet that is not on the same network, it will be sent to this Gateway address. Which is aptly named as being a Gateway to another network.

Genmask

This is the subnet mask, used to figure out what IP addresses match what destination.

Flags

UG - Network is Up and is a Gateway
U - Network is Up
Iface

This is the interface that our packet will be going out of, eth0 usually stands for the first Ethernet device on your system.

add a new route :

$ sudo route add -net ip_address/cidr gw ip_addresss
$ ip route add ip_address via ip_address

delete any route :

$ sudo route del -net 192.168.2.1/23 
$ ip route delete 192.168.2.1/23

network config command:

network config command :

$ ifconfig 
$ ipconfig
$ ifconfig -a              [to see more details]

To create an interface and bring it up:

$ ifconfig eth0 192.168.2.1 netmask 255.255.255.0 up

To bring up or down an interface:

$ ifup eth0
$ ifdown eth0

To show interface information for all interfaces:

$ ip link show

To show the statistics of an interface:

$ ip -s link show eth0

To show ip addresses allocated to interfaces:

$ ip address show

To bring interfaces up and down:

$ ip link set eth0 up
$ ip link set eth0 down

To add an IP address to an interface:

$ ip address add 192.168.1.1/24 dev eth0

To obtain a fresh IP:

$ sudo dhclient

network manager command :

There are also command-line tools to interact with NetworkManager:

$ nm-tool

The nmcli command allows you to control and modify NetworkManage:

$ nmcli

we lookup a MAC address with ARP command:

$ arp

You can also view your arp cache via the ip command::

$ ip neighbour show

network troubleshoting command :

One of the most simplest networking tools is ping

$ ping hostName or ip_address
$ ping google.com
$ ping -c google.com

The traceroute command is used to see how packets are getting routed. It works by sending packets with increasing TTL values, starting with 1:

$ traceroute hostName or Ip_address or google.com

An extremely useful tool to get detailed information about your network is netstat. Netstat displays various network related information such network connections, routing tables, information about network interfaces and more:

$ netstat
$ netstat -at

utility to debug TCP/UDP sockets:

$ nc

we can be used to test remote connectivity on ports by using telnet command:

$ telnet

mtr is a mix of ping and traceroute. It also provides additional information like intermediate hosts and responsiveness:

$ mtr

here all DNS related linux command:

The /etc/hosts file contains mappings of some hostnames to IP addresses:

$ cat /etc/hosts

The "name server lookup" tool is used to query name servers to find information about resource records:

$ nslookup DNSname or ip_address

Dig (domain information groper) is a powerful tool for getting information about DNS name servers:

$ dig DNSname or ip_address

Day-5: Processes Utilization, Devices, and Logging

Mahbub Ferdous Bijoy — Mon, 30 Sep 2024 16:51:36 +0000

Go ahead and run the ps command to see a list of running processes:

$ ps

PID: Process ID
TTY: Controlling terminal associated with the process (we'll go in detail about this later)
STAT: Process status code
TIME: Total CPU usage time
CMD: Name of executable/command

another details process running command :

$ ps aux

USER: The effective user (the one whose access we are using)
PID: Process ID
%CPU: CPU time used divided by the time the process has been running
%MEM: Ratio of the process's resident set size to the physical memory on the machine
VSZ: Virtual memory usage of the entire process
RSS: Resident set size, the non-swapped physical memory that a task has used
TTY: Controlling terminal associated with the process
STAT: Process status code
START: Start time of the process
TIME: Total CPU usage time
COMMAND: Name of executable/command

more details if we see the running process command:

$ ps l

what processes are taking up a lot of your resources command niceness level of process:

$ top

1st line: This is the same information you would see if you ran the uptime command (more to come)

The fields are from left to right:

Current time
How long the system has been running
How many users are currently logged on
System load average (more to come)

2nd line: Tasks that are running, sleeping, stopped and zombied

3rd line: Cpu information

us: user CPU time - Percentage of CPU time spent running users’ processes that aren’t niced.
sy: system CPU time - Percentage of CPU time spent running the kernel and kernel processes
ni: nice CPU time - Percentage of CPU time spent running niced processes
id: CPU idle time - Percentage of CPU time that is spent idle
wa: I/O wait - Percentage of CPU time that is spent waiting for I/O. If this value is low, the problem probably isn’t disk or network I/O
hi: hardware interrupts - Percentage of CPU time spent serving hardware interrupts
si: software interrupts - Percentage of CPU time spent serving software interrupts
st: steal time - If you are running virtual machines, this is the percentage of CPU time that was stolen from you for other tasks

4th and 5th line: Memory Usage and Swap Usage

Processes List that are Currently in Use

PID: Id of the process
USER: user that is the owner of the process
PR: Priority of process
NI: The nice value
VIRT: Virtual memory used by the process
RES: Physical memory used from the process
SHR: Shared memory of the process
S: Indicates the status of the process: S=sleep, R=running, Z=zombie,D=uninterruptible,T=stopped
%CPU - this is the percent of CPU used by this process
%MEM - percentage of RAM used by this process
TIME+ - total time of activity of this process
COMMAND - name of the process

you can kill some process by using kill command, first you run top than get the PID:

$ kill PID         [PID - process ID get when you run top command]

sending a job to background :

$ sleep 1000 &
$ jobs                     [to see the job id what you make before]
$ bg                       [too see all job in background]

a list of all the open files and their associated process:

$ lsof

To view process threads, you can use:

$ ps m

CPU monitoring command :

$ uptime

we can see CPU details with I/O command :

$ iostat

tps - Indicate the number of transfers per second that were issued to the device. A transfer is an I/O request to the device. Multiple logical requests can be combined into a single I/O request to the device. A transfer is of indeterminate size.
kB_read/s - Indicate the amount of data read from the device expressed in kilobytes per second.
kB_wrtn/s - Indicate the amount of data written to the device expressed in kilobytes per second.
kB_read - The total number of kilobytes read.
kB_wrtn - The total number of kilobytes written.

memory monitoring command :

$ vmstat

procs

r - Number of processes for run time
b - Number of processes in uninterruptible sleep


memory

swpd - Amount of virtual memory used
free - Amount of free memory
buff - Amount of memory used as buffers
cache - Amount of memory used as cache


swap

si - Amount of memory swapped in from disk
so - Amount of memory swapped out to disk

io

bi - Amount of blocks received in from a block device
bo - Amount of blocks sent out to a block device

system

in - Number of interrupts per second
cs - Number of context switches per second

cpu

us - Time spent in user time
sy - Time spent in kernel time
id - Time spent idle
wa - Time spent waiting for IO

make a cronjob every day :

$ 00 08 * * * home/mahbub/script/change_wallpaper.sh   [here is my script path]

here three * means everyday in every month and 00 means min and 08 means 08 am

to create a cronjobs just edit a crontab:

$ crontab -e

here is all device command :

listing usb devices:

$ lsusb

Listing PCI Devices:

$ lspci

Listing SCSI Devices:

$ lsscsi

here some monitoring command :

if you want to see background task:

$ bg

Gives free hard disk space on your system:

$ df
$ df -h         [ -h gives you a human readable file]
# du -h        [This shows you the disk usage of the current directory you are in]

Gives free RAM on your system:

$ free

this for cpu and ram yoou can use :

$ nproc

Day-4: Server management, download, checkOS and services

Mahbub Ferdous Bijoy — Mon, 30 Sep 2024 16:50:41 +0000

to start any service in linux there have 2 ways service and systemctl:

$ sudo service SERVICENAME start         [service name like httpd,mySQL etc]
$ sudo systemctl start SERVICENAME

to stop any service in linux :

$ sudo service SERVICENAME stop
# sudo systemctl stop SERVICENAME

restart any stop service again :

$ sudo service SERVICENAME restart
$ sudo systemctl restart SERVICENAME

to check any service up and running or not :

$ sudo service SERVICENAME status
$ sudo systemctl status SERVICENAME
$ sudo systemctl is-active SERVICENAME 
$ sudo systemctl show SERVICENAME

here is all download related linux command :

install some software we use apt and yum here all yum command for RPM like centOS, alpine etc:

$ sudo apt install AppName
$ sudo yum install appName

remove any app from your local linux machine :

$ sudo apt remove appName
$ sudo yum erase appName

get update and upgrade:

$ sudo apt update 
$ sudo apt upgrade
$ sudo yum update

get any install package info:

$ sudo apt show appName
$ sudo yum info appName

download something from remote server or internet we use curl and wget:

$ curl downloadLink        [curl https://www.baeldung.com/] {curl show output in console}
$ wget downloadLink        [wget https://www.baeldung.com/] {wget download it into a file}

to compress a file in linux by .gz extention:

$ gzip myCompressFile

to decompress a file in linux from .gz extended file:

$ gunzip myfileName.gz

tar file with .tar extention :

$ tar cvf mytarFile.tar myDestinyFile      [here cvf's c stand for create]
$ tar xvf mytarFile.tar                    [here xvf's x stand for extract]

dpkg and rpm:

$ dpkg -i some_deb_ubuntu_package.deb   [for install debian and ubuntu package file]
$ rpm -i some_centOs_alpine_rpm_package.rpm    [for install redhat package file]
$ dpkg -r some_deb_ubuntu_package.deb    [remove file]
$ rpm -e some_centOs_alpine_rpm_package.rpm   [remove file]

list install packages in ubuntu debian and redhat:

$ dpkg -l       [for ubuntu and debian]
$ rpm -qa       [for red hat centOS,alpine ]

Day-3: Linux user management and permissions

Mahbub Ferdous Bijoy — Mon, 30 Sep 2024 16:48:59 +0000

use sudo command to see some confidential or root user access file:

$ cat /var/etc/shadow         [you cant see access denied]
$ sudo cat /var/etc/shadow        [noew you can see]
$ sudo cat /var/etc/passwd
$ sudo cat /var/etc/group

add some user by using useradd command:

$ sudo useradd userName

delete some user by using userdel command:

$ sudo userdel userName

cahnging some user password :

$ sudo passwd userName

if we want to know our ssh key :

$ cat ~/.ssh/id_rsa.pub

connect your machine with remote machine via ssh :

$ ssh -i "key-pair_fileName_PATH ubuntu@ip_address    [ubuntu or any os]

something copy from remote server:

$ scp -i <keypair_PATH> < keypair_PATH> <user@ip_address>: filePath

here all linux permission command:

check file permission:

$ ls -l

to change any file permission by using chmod(rwx):

$ chmod +x fileName            [make executable file]
$ chmod +rwx fileName          [ make permission read,write, executable]
$ chmod -rwx fileName          [ remove all permission]
$ chmod -x fileName            [remove executable permission]

we can give permission by using numaric command like 777:

Description: for read - 4, write - 2, execute -1. all together 7. for 777 means 1st 7 for user, 2nd for group and 3rd for all user

$ chmod 777 fileName    [all permission for user,group and all]
$ chmod 742 fileName    [all permission for user, read for group, write for all]
$ chmod 124 fileName    [execute permission for user, write for group, read for all]
$ chmod 711 fileName    [ all permission for user, execute for groups and all]

modifying ownership permission :

$ sudo chown userName fileName    [it will change files user ownership]
$ sudo chgrp groupName fileName    [it will change files group ownership]
$ sudo chmod userName:groupName fileName     [it will change both user & group ownership]

if you want to change your user password:

$ passwd

to see our system name :

$ uname

to see our system name and details:

$ uname -a
$ uname -r     [to check only version numaric ]

add a user to a group :

$ sudo usermod -a -g groupName userName

remove a user from a group:

$ sudo deluser userName groupName

Shows information of all the users logged in:

$ finger

show information of a particular user:

$ finger userName

Day-2: Linux Text manipulation

Mahbub Ferdous Bijoy — Mon, 30 Sep 2024 16:48:00 +0000

write something on a text file using echo directly:

$ echo hello world > my_File.txt

you can copy a text file from another file using echo command:

$ echo < my_file.txt > > my_File2.txt

: environment variable by echo:

$ echo $VARIABLE
$ echo $USER
$ echo $HOME
$ env 
$ echo $PATH
$ VARIABLE_NAME = variable_value
$ unset                           [remove a variable]
$ export Variable = value         [set value of an environment variable]

cut something from text cut command:

$ cut -c 5-10 fileName.txt    [5-10 word index number mention]

using paste command to concatenate text file:

$ paste -s fileName.txt
$ paste -d ' ' -s fileName.txt [for space indentation]

to show any log file or file from head or sorting use head command:

$ head /var/logs/keybagd.log.0   [you see only the head part of the log file]
$ head -n 10 /var/logs/keybagd.log.0    [you can see exact 10th number of line just sorting by using -n]

to show any log file or file from tail or sorting use tail command:

$ tail /var/logs/keybagd.log.0   [you see only the tail part of the log file]
$ tail -n 10 /var/logs/keybagd.log.0    [you can see exact 10th number of line just sorting by using -n]
$ tail -f /var/logs/keybagd.log.0

make two file join by join command:

$ vim file1.txt
$ vim file2.txt
$ join file1.txt file2.txt

sorting some text file by using sort:

$ sort fileName.txt
$ sort -r fileName.txt   [ reverse sorting ]

upercase to lower case by using tr command :

$ tr A-Z a-z

sorting by using uniq command for unique word:

$ uniq fileNAme.txt

word and line count from a log or text file by using wc and nl command :

$ wc fileName.txt or PATH 
$ nl fileName.txt or PATH

to sort anything in any file or directory we use grep command:

$ grep fox sampole.txt   [grep find fox from this file]
$ grep -i FOX sample.txt   [ this -i is getting for case insensitive result]
$ env | grep -i Users         [you can use grep with pipe(|) for more sorting] [you get only User info]

here is all command and shortcuts for text editor vim:

go and write file with vim editor:

$ vi fileName.extention
$ vim fileName.extention

to write file in vim do insert mode by uing i in the keyboard:

$ press i button

if you search something unique just use / :

$ /searchWordName
$ go up to down press n
$ down to up press N

vim short cuts :

$ O -- for insert text on previous line
$ o -- for insert text on the next lien 
$ a -- for append text after the cursor 
$ A -- for append text at the end oif the line
$ x -- for cut the selected text also for deleting characters 
$ dd - for delete the current line 
$ y -- copy whatever is selected 
$ yy - copy the current line 
$ p -- paste the copied text before the cursor
$ :w - write and save file
$ :wq - write save and quite from vim
$ :q -- quite from vim
$ :q! - quite out from vim without saving
$ u -- undo your last action
$ ZZ - for :wq but more faster
$ Ctrl+r - redo your last action

Day-1.Linux Basic CLI(Command line interface)

Mahbub Ferdous Bijoy — Mon, 30 Sep 2024 16:46:53 +0000

Present working directory(pwd):

$ pwd

change directory(cd):

$ cd "dirName or path"

back one step directory(cd):

$ cd ..

back root directory(cd):

$ cd /

from user to direct jump to the file (cd~):

$ cd~

List file (ls):

$ ls

List file with hidden (ls):

$ ls -a

List file with file permission details (ls):

$ ls -l

List file with permission and hidden files (ls):

$ ls -al

Make directory(mkdir):

$ mkdir "dirName"

create subdirectory at a time(mkdir):

$ mkdir -p dirM/2ndDir/lastDir

make any file(touch):

$ touch fileName.txt

to check description of a file(file):

$ file fileName.jpg

show the content of a file(cat,less):

$ cat fileName
$ less fileName

show any long file with head part or tail part(head,tail):

$ head fileName
$ tail fileName

clear terminal (clear):

$ clear

to see last command you write(history):

$ history

copying file (cp):

$ cp fileNmae pathwithFilename
$ cp *.jpg /user/desktop/picture

rename file and directory (mv):

$ mv oldFile newFile
$ mv oldDir newDir

move file(mv):

$ mv fileName /home/desktop/files
$ mv file1 file2 /files

remove file and directory(rm):

$ rm fileName
$ rmdir dirName
$ rm -r dirName
$ rm *.text

know any commands manual(man):

$ man mkdir

find some file(find):

$ find /home -name fileName

see date and time:

$ date 
$ time

want to see my ssh key:

$ cat ~/.ssh/id_rsa.pub

to show a short description about your desire command(whatis):

$ whatis cat

for exit from terminal(exit):

$ exit 
$ logout

locate a program file from Users file using which command:

$ which bash

who is in the system to check by using whoami command:

$ whoami