Kedar Kodgire

Posted on Mar 21, 2020 • Edited on Mar 25, 2020

The Linux guide/cheatsheet/tutorial (Commands, Directories, etc)

#linux #bash #computerscience

This is a Linux guide, There might be some stuff that I can miss out but I will try my best to add all the important content in this post. This post might be a little bit long so you gotta bear with me

note that I have provided examples only where it's too necessary to understand commands because I want you to try it so that you can understand it better. in the end, I will share some websites which can quickly allow you to try commands in the browser itself.

Command Syntax
Teach Yourself To Fish
Piping
Redirecting
Basic Linux Commands
Creating or Removing Directories
Working with Directories
Sorting Data
Searching in files
Removing files
Compressing files
Directory Structure
Finding Things
Informational Commands
File and Directory Permissions
Editing Files
Scheduling
Shortcuts to Make it all Easier
Practice it in Browser

Command Syntax

This command syntax will be the same for every command that you will use in Linux.

command [-argument] [-argument] [--argument] [file]
Examples:
ls List files in the current directory
ls -l Lists files in “long” format
ls -l --color As above, with colorized output
cat filename Show contents of a file
cat -n filename Show contents of a file, with line nu
↑

Teach Yourself To Fish

The intention of starting with this - There should not be a need to refer something each time you forget some command because there are a lot of commands in Linux hence it's easy to forget sometimes.

The man (Manual) command will be used for this purpose. To get help for the man command type the letter h while viewing a manual page. Here's a concise version

Enter - Move down one line.
Space - Move down one page.
g - Move to the top of the page.
G - Move to the bottom of the page.
q - Quit.

for example

Now you got an Idea about this let's proceed.
↑

Piping

Linux has a huge number of small utilities. Programmers are smart, aren't they? Hence as a programmer, you need to make efficient use of all these utilities.

The pipe character, |, is used to chain two or more commands together. The output of the first command will be pushed to the next command, if there is the second pipe then the output of the second command will be pushed to next and so on.
Let's have a look at an example



[kedar@www.masswerk.at:2]$ ls -l ~ | grep '^.....w'
drwxrwxr-x 3 kedar users 4096 Jan 21 04:12 dropbox

This will identify all files in your home directory which the group has to write permission for.

↑

Redirecting

There will be many times when you want to save the output of the command to the file instead of displaying it to the screen. Suppose to create a file that will have a list of all the .py files in the directory.

ls -l /home/vic/pyproc/*.py > pyfiles.txt
(or)
ls -l /home/vic/pyproc/*.py >> pyfiles.txt

remember

> to write
>> for append

This goes very well with piping.

Basic Linux Commands

Here is a shortlist of basic, but essential commands. Remember that commands are case sensitive. Content in the brackets will be case sensitive.

ls - Lists directory contents. You will use ls to display information about files and directories.
cd [dir] - Changes the current directory to dir. If you execute cd without specifying a directory, cd changes the current directory to your home directory. This is how you navigate around the system.
pwd (Print Working Directory) - Displays the present working directory name. If you don't know what directory you are in, pwd will tell you.
cat [file] - Concatenates and displays files. This is the command you run to view the contents of a file.
echo [argument] - Displays arguments to the screen.
man command - Displays the online manual for command. Type q to quit viewing the manual page. The documentation provided by the man command is commonly called "man pages."
exit, logout, or Ctrl-d - Exits the shell or your current session.
clear - Clears the screen.
which [command name]- If you want to know exactly where a command is located you can use the which command
$PATH - The PATH environment variable contains a list of directories that contain executable commands. You can use this as [kedar@www.masswerk.at:2]$ echo $PATH you get the result as /bin /sbin /usr/bin ~ ↑

Creating or Removing Directories

Directories are simple folders that provide tree-like structures.

mkdir - make directory
mkdir [-p] directory - Create a directory. Use the -p (parents)
option to create intermediate directories.
rmdir - remove directory
rmdir [-p] directory - Remove a directory. Use the -p
(parents) option to remove all the specified directories. rmdir only
removes empty directories. To remove directories and their contents,
use rm.
rm -rf directory - r Recursively removes the directory and all files
and directories in that directory structure. Use with caution. There is no "trash" container to quickly restore your file from when using the command line. When you delete something, it is gone.
↑

Working with Directories

Usually, we use names to access the directories. In Linux we use symbols.

. This directory.
.. The parent directory.
/ Directory separator. Directories end in a forward slash and this is often assumed

the directory separator is optional i.e.
$ cd /bin/abc $ cd /bin/abc/ are same.

You can check the example below to understand it better

note that cd . doesn't do anything because it is just the current directory.
↑

Sorting Data

In the simplest form, it sorts lines of text alphabetically. SORT command is used to sort a file, arranging the records in a particular order. By default, the sort command sorts file assuming the contents are ASCII. Using options in sort command, it can also be used to sort numerically.

suppose we have a file with alphabets in it



[kedar@www.masswerk.at:2]$ cat > file.txt
a
c
s
r
n
d
h



[kedar@www.masswerk.at:2]$ sort file.txt

Output :
a
c
d
h
n
r
s

Let's look at some sorting options

sort -k F file - Sort by key. The F following -k is the field number.
sort -r file - Sort in reverse order.
sort -u file - Sort text in file, removing duplicate lines.
sort -n file - sort numerically ↑

searching in files

grep searches for PATTERNS in each FILE. PATTERNS is one or more patterns separated by newline characters, and grep prints each line that matches a pattern. Typically PATTERNS should be quoted when grep is used in a shell command.

example:
grep “nfs” /etc/services
This looks for any line that contains the string “nfs” in the file “/etc/services” and displays only those lines.

There are a few options too

c: This prints only a count of the lines that match a pattern
h: Display the matched lines, but do not display the filenames.
i: Ignores, the case for matching
l: Displays a list of filenames only.
n: Display the matched lines and their line numbers.
v: This prints out all the lines that do not match the pattern
e exp: Specifies expression with this option. It can use multiple times.
f file: Takes patterns from a file, one per line.
E: Treats pattern as an extended regular expression (ERE)
w: Match whole word
o: Print only the matched parts of a matching line, with each such part on a separate output line.

↑

Removing files

Well, sometimes you gotta get rid of some files and you can use rm command for these structures.

let's see the options which are used regularly

rm -r directory - Remove the directory and its contents
recursively. If you want to remove a directory with rm, you have to
supply the -r argument.
rm -f file - Force removal and never prompt for confirmation.
Search patterns can be used to delete multiple files at once. It's a good
idea to double-check what you are going to remove with ls before you
execute rm

Quick Remainder - We are only looking at the options which will be used regularly, there are several other options too which you can find with man command.

↑

Compressing files

We all know what is compressing files (if not please post in comments or google it). So let's just dive in commands.

gzip file - Compress file. The resulting compressed file is named file.gz.
gunzip file - Uncompress files.
gzcat or zcat - Concatenates and prints compressed files. You can use the command du to display how much space is used by a file.
bzip2 file - bzip2 compresses files using the Burrows-Wheeler block sorting text compression algorithm, and Huffman coding. Compression is generally considerably better than that achieved by bzip command (LZ77/LZ78-based compressors). Whenever possible, each file is replaced by one with the extension .bz2.
zip .zip-file file-to-compress - zip is a compression and file packaging utility for Unix/Linux. Each file is stored in single .zip {.zip-filename} file with the extension .zip
du - Estimates file usage.
du -k - Display sizes in Kilobytes.
du -h - Display
bzip2 -d .bz2-file or bunzip2 {.bz2-file} - Decompressed a file that is created using bzip2 command.
unzip .zip-file - Extract compressed files in a ZIP archive. time for a quick example



$ gzip data

$ du -k data.gz

$ du -k data

15360 data

$ zip mydata.zip mydata.doc

$ zip data.zip .doc

$ bzip2 mydata.doc

$ bzip2 .jpg

↑

Directory Structure

Understanding the directory structure will help you when you are searching for components on the system. It can help you answer questions like:

Where are files for programs located?
What about the configuration files?
And log files for some applications?

directory	description
/	The nameless base of the filesystem. All other directories, files, drives, and devices are attached to this root. Commonly (but incorrectly) referred to as the “slash” or “/” directory. The “/” is just a directory separator, not a directory itself.
/bin	Essential command binaries (programs) are stored here (bash, ls, mount, tar, etc.)
/boot	Static files of the boot loader.
/dev	Device files. In Linux, hardware devices are accessed just like other files, and they are kept under this directory.
/etc	Host-specific system configuration files.
/home	Location of users' personal home directories (e.g. /home/Susan).
/lib	Essential shared libraries and kernel modules.
/proc	Process information pseudo-filesystem. An interface to kernel data structures.
/root	The root (superuser) home directory.
/sbin	Essential system binaries (fdisk, fsck, init, etc).
/tmp	Temporary files. All users have permission to place temporary files here.
/usr	The base directory for most shareable, read-only data (programs, libraries, documentation, and much more).
/usr/bin	Most user programs are kept here (cc, find, du, etc.).
/usr/include	Header files for compiling C programs.
/usr/lib	Libraries for most binary programs.
/usr/local	“Locally” installed files. This directory only really matters in environments where files are stored on the network. Locally-installed files go in /usr/local/bin, /usr/local/lib, etc.). Also often used for software packages installed from source, or software not officially shipped with the distribution.
/usr/sbin	Non-vital system binaries (lpd, useradd, etc.)
/usr/share	architecture-independent data (icons, backgrounds, documentation, terminfo, man pages, etc.).
/usr/src	Program source code. E.g. The Linux Kernel, source RPMs, etc.
/usr/X11R6	The X Window System.
/var	Variable data: mail and printer spools, log files, lock files, etc.
/opt	Optional or third party software.

Okay, these were some common directories which you will come across if you use Linux. Let's have a look at some application directory structure now

Here is a sample directory structure of an application named apache installed in /usr/local.

directory	description
/usr/local/apache/bin	The application's binaries and other executable programs.
/usr/local/apache/etc	Configuration files for the application.
/usr/local/apache/lib	Application libraries.
/usr/local/apache/logs	Application log files.

and it might look like this if it were installed in /opt

directory	description
/opt/apache/bin	The application's binaries and other executable programs.
/opt/apache/etc	Configuration files for the application.
/opt/apache/lib	Application libraries.
/opt/apache/logs	Application log files

↑

Finding Things

The following commands will be used to find files in the system. ls is good for finding files if you already know approximately where they are, but sometimes you need more powerful tools such as these:

which: Shows the full path of shell commands found in your path. For example, if you want to know exactly where the “grep” command is located on the filesystem, you can type “which grep”. The output should be something like: /bin/grep
whereis : Locates the program, source code, and manual page for a command (if all information is available). For example, to find out where “ls” and its man page are, type: “whereis ls” The output will look something like:
ls: /bin/ls /usr/share/man/man1/ls.1.gz
locate A quick way to search for files anywhere on the filesystem. For example, you can find all files and directories that contain the name “Mozilla” by typing: locate Mozilla
find: A very powerful command, but sometimes tricky to use. It can be used to search for files matching certain patterns, as well as many other types of searches. A simple example is:
find. -name \*mp3
This example starts searching in the current directory “.” and all subdirectories, looking for files with “mp3” at the end of their names.

↑

Informational Commands

These commands are going to help if you want some information about the current user or the system. You might not use these commands every day, but they are useful.

ps Lists currently running process (programs).
w Show who is logged on and what they are doing.
id Print your user-id and group id's
df Report filesystem disk space usage (“Disk Free” is how I remember it)
du Disk Usage in a particular directory. “du -s” provides a summary for the current directory.
top Displays CPU processes in a full-screen GUI. A great way to see the activity on your computer in real-time. Type “Q” to quit.
free Displays the amount of free and used memory in the system.
cat /proc/cpuinfo Displays information about your CPU.
cat /proc/meminfo Display lots of information about current memory usage.
uname -a Prints system information to the screen (kernel version, machine type, etc.) ↑

File and Directory Permissions

Authorization is necessary whichever system or program it may be. It is a way to provide access to users or groups. If you quickly want to check the file permissions in your current working directory just use the command ls -l. Once you run this command you can see something like -rw-r--r-- in results, these are the permissions.

The first character in the permissions string reveals file type. There are three file types

- regular file
d directory
l symbolic link

you can look at the image below, you can understand a lot from this.

Let's look what r, w, x in case of file and directory

Permission	File Meaning	Directory Meaning
Read	Allows a file to be read.	Allows file names in the directory to be read.
Write	Allows a file to be modified.	Allows entries to be modified within the directory.
Execute	Allows the execution of a file.	Allows access to contents and metadata for entries in the directory.

now that you understood the permissions, let's look at manipulating them using chmod command

Numeric Way

Octal	Binary	String	Description
0	000	---	No permissions
1	001	--x	Execute only
2	010	-w-	Write only
3	011	-wx	Write and execute (2 + 1)
4	100	r--	Read only
5	101	r-x	Read and execute (4 + 1)
6	110	rw-	Read and write (4 + 2)
7	111	rwx	Read, write, and execute (4+2+1)

let's look at the example

chmod 754 myfile

7 is for the owner, 5 is for group and 4 for others.

	U	G	O
Symbolic	rwx	r-x	r--
Binary	111	101	100
Decimal	7	5	4

Symbolic mode

In the numeric mode, you change permissions for all 3 owners but in the symbolic mode, you can modify the permissions of a specific owner. It makes use of mathematical symbols to modify the file permissions.

Operator	Description
+
-
=

The various owners are represented as -

User	Denotations
u	user/owner
g	group
o	other
a	all

let's look at an example for this

[kedar@www.masswerk.at:2]$ chmod o=rwx myfile: setting permission to other users

[kedar@www.masswerk.at:2]$ chmod g+x myfile: adding execute permission to the group

and similarly,

[kedar@www.masswerk.at:2]$ chmod u-r myfile: removing read permissions for user

↑

Editing Files

vi is the most widely used editor in Linux. There are also other editors like nano but this has more powerful tools. Hence, instead of discussing all of them let's discuss the most widely used. There is a learning curve to using
these editors as they are not exactly intuitive. It will require a bit of a
time investment to become proficient. Let's get started.

vi [file] - Edit file.
vim [file] - Same as vi, but with more features.
view [file] - Starts vim in read-only mode. Use view when you want to examine a file but not make any changes.

Vim stands for "Vi IMproved." It is compatible with the commands found in vi. Some of the additional features of vim include syntax highlighting, the ability to edit files over the network, multi-level undo/redo. On many Linux distributions when you invoke vi, you are actually running vim. One advantage of knowing vi is that vi or a vi variant like vim is always available on the system.

vi has many modes such as insert, command, line modes. When vi starts you are placed into command mode To get back to command mode at any time hit the escape key (Esc). Letters typed while in command mode are not sent to the file, but are rather interpreted as commands. Command mode allows you to navigate about the file, perform searches, delete text, copy text, and paste the text.

some commonly used key bindings for navigation.

k - Up to one line.
j - Down one line.
h - Left one character.
l - Right one character.
w - Right one word.
b - Left one word.
^ - Go to the beginning of the line.
$ - Go to the end of the line

There are still so many of them which you can check Harvard's documentation.

Insert mode

To enter insert mode, press one of the following keys.

i - Insert at the cursor position.
I - Insert at the beginning of the line.
a - Append after the cursor position.
A - Append at the end of the line. After entering into insert mode, type the desired text. When you are finished, type Esc to return to command mode

Line mode

To enter line mode you must start from command mode and then type
a colon (:) character. If you are in insert mode, type Esc to get back to
command mode and then type a colon for line mode. Here are some of
the most common line mode commands you will want to know.

:w - Writes (saves) the file.
:w! - Forces the file to be saved even if the write permission is not set. This only works on files you own.
:q - Quit. This will only work if there have not been any modifications to the file.
:q! - Quit without saving changes made to the file.
:wq! - Write and quit. After modifying a file this command ensures it gets saved and closes vi.
:x - Same as :wq.
:n - Positions the cursor at line n. For example, :5 will place the cursor on the fifth line in the file.
:$ - Positions the cursor on the last line of the file

↑

Scheduling

If you need to repeat a task on a schedule, you can use the cron service. Every minute the cron service checks to see if there are any scheduled jobs to run and if so runs them. Cron jobs are often used to automate a process or perform routine maintenance. You can schedule cron jobs by using the crontab command.

cron - A time based job scheduling service. This service is typically started when the system boots.
crontab - A program to create, read, update, and delete your job schedules.

To edit or create your crontab file, type the following command at the Linux shell prompt:
$ crontab -e

Cron will examine the modification time on all crontabs and reload those which have changed. Thus cron need not be restarted whenever a crontab file is modified.

let's see an example

1 2 3 4 5 /path/to/command arg1 arg2 - runs command with arguments

(or)

1 2 3 4 5 /root/backup.sh - runs script

Where,

1: Minute (0-59)
2: Hours (0-23)
3: Day (0-31)
4: Month (0-12 [12 == December])
5: Day of the week(0-7 [7 or 0 == sunday])
/path/to/command – Script or command name to schedule

The format is

* * * * * command to be executed
- - - - -
| | | | |
| | | | ----- Day of week (0 - 7) (Sunday=0 or 7)
| | | ------- Month (1 - 12)
| | --------- Day of month (1 - 31)
| ----------- Hour (0 - 23)
------------- Minute (0 - 59)

You can play around with it in corn guru

↑

Shortcuts to Make it all Easier

When you start using Linux more often, you will appreciate these shortcuts that can save you very much typing time.

Shortcut	Description
Up/Down Arrow Keys	Scroll through your most recent commands. You can scroll back to an old command, hit ENTER, and execute the command without having to re-type it.
“history” command	Show your complete command history.
TAB Completion	If you type a partial command or filename that the shell recognizes, you can have it automatically completed for you if you press the TAB key. Try typing the first few characters of your favorite Linux command, then hit TAB a couple of times to see what happens.
Complete recent commands with “!”	Try this: Type “!” followed by the first couple of letters of a recent command and press ENTER! For example, type: find `/usr/bin -type f -name m\*` ...and now type: `!fi`
Search your command history with CTRL-R	Press CTRL-R and then type any portion of a recent command. It will search the commands for you, and once you find the command you want, just press ENTER.
Scrolling the screen with ShiftPageUp and Page Down	Scroll back and forward through your terminal.

↑

Practice it in Browser

I use masswerk all the time when I am on windows

You can find some more here

↑

For more updates on stuff like this, you can follow me on twitter and let me know if you feel I missed something in the comments, I will consider it as it will help others too.

Have a great day,

Happy Learning.

If my posts or resources I share has helped you in some way, please consider buying me a coffee. It will help me pay my tuition fee and encourage me to share amazing and useful stuff with you all.