INTRODUCTION
There are many reasons why you should learn about I/O redirection, shell filters, and shell expansion. These concepts are essential for anyone who wants to use the command line effectively. Without an understanding of how these features work, it will be difficult to fully utilize the potential of the command line. Additionally, learning about these topics will give you a better understanding of how Linux works under the hood. Finally, being knowledgeable about I/O redirection, shell filters, and shell expansion will make it easier for you to troubleshoot any issues that might arise when using the command line.
I/O Redirections
Redirection is a feature in Linux that allows you to change the standard input/output devices when executing a command. The basic workflow of any Linux command is that it takes input and produces output.
Each command performs all terminal-related activities with three files that the shell makes available to every command:
- Standard input (stdin): Represented by <, it is the information inputted into the terminal through the keyboard or input device.
- Standard output (stdout): Represented by >, it is the information outputted after a process is run.
- Standard error (stderr): It is an error message outputted by a failed process.
Output Redirection
The ">" symbol is used for output redirection.
root@537302f1e700:~/fruits# ls -l > listings
root@537302f1e700:~/fruits# cat listings
total 0
-rw-r--r-- 1 root root 0 Sep 7 13:46 apple
-rw-r--r-- 1 root root 0 Sep 7 13:46 Apple
-rw-r--r-- 1 root root 0 Sep 7 13:49 banana
-rw-r--r-- 1 root root 0 Sep 7 13:48 Banana
-rw-r--r-- 1 root root 0 Sep 7 13:47 cherry
The output of ls when redirected will be stored in the listings file. This is how you can redirect a command.
When you use the ">" operator to redirect to a file with the same name, the information in the file is overwritten.
root@537302f1e700:~/fruits# echo " welcome " > listings
root@537302f1e700:~/fruits# cat listings
welcome
In this example, echo "welcome" > listings has overwritten the existing file.
To prevent overwriting a file, you can append to it using the output append operator >>.
root@537302f1e700:~/fruits# echo " welcome " > listings
root@537302f1e700:~/fruits# cat listings
welcome
root@537302f1e700:~/fruits# echo "welcome again" >> listings
root@537302f1e700:~/fruits# cat listings
welcome
welcome again
Input Redirection
The "<" symbol is used for stdin redirection.
For example:
$ wc -l users
2 users
$
$ wc -l < users
2
$
Note that there is a difference in the output produced by the two forms of the wc command. In the first case, the name of the file users is listed with the line count; in the second case, it is not.
In the first case, wc knows that it is reading its input from the file users. In the second case, it only knows that it is reading its input from standard input, so it does not display the filename.
Filters in Linux
Filters are programs that take in plain text as standard input, transform it into a meaningful format, and then return it as standard output. These include:
1. cat
The cat command displays the text of the file line by line.
cat [path]
root@537302f1e700:~/fruits# cat list
apple
mango
banana
oranges
2. head
The head command displays the first n lines of a file. If the number of lines is not specified, it prints the first 10 lines by default.
head [lines-to-print] [path]
root@537302f1e700:~/fruits# head list
apple
mango
banana
oranges
Melon
mandarin
Avocado
coconut
Cherry
lime
root@537302f1e700:~/fruits# head -3 list
apple
mango
banana
root@537302f1e700:~/fruits#
3. tail
The tail command is the reverse of head, returning the lines from bottom to top.
root@537302f1e700:~/fruits# tail list
mango
banana
oranges
Melon
mandarin
Avocado
coconut
Cherry
lime
Longan
root@537302f1e700:~/fruits# tail -3 list
Cherry
lime
Longan
4. sort
The sort command is used to sort a file, arranging the records in a particular order.
sort [options] path
root@537302f1e700:~/fruits# sort list
1mango
20melon
3lime
apple
Avocado
banana
Cherry
coconut
Options with sort command:
- -r: Sorts in reverse order.
- -n: Sorts the file numerically.
- -nr: Sorts a file with numeric data in reverse order.
- -k: Sorts the table based on any column.
- -c: Checks if a given file is already sorted.
- -u: Sorts and removes duplicates.
- -M: Sorts a file on a monthly basis.
5. uniq
The uniq command is used to remove duplicate lines. For example, in this case, banana was duplicated, and the uniq command removed the duplicate item.
root@537302f1e700:~/fruits# cat list
apple
mango
banana
banana
oranges
Melon
mandarin
Avocado
coconut
Cherry
lime
Longan
1mango
3lime
20melo
apple
root@537302f1e700:~/fruits# uniq list
apple
mango
banana
oranges
6. wc
The wc command gives the number of lines, words, and characters in the data. By default, it displays four columnar outputs.
root@537302f1e700:~/fruits# wc list
17 17 112 list
wc [options] [path]
wc stands for word count.
With wc, you can pass more than one file.
Note: When more than one file is placed in the argument, it adds one more line that counts the total.
root@537302f1e700:~/fruits# wc list melon orange
17 17 112 list
0 0 0 melon
0 0 0 orange
17 17 112 total
wc options:
- -l: Prints the number of lines in a file.
- -w: Prints the number of words in a file.
- -m: Displays the count of characters in a file.
- -v: Displays the version of
wc.
Some applications of the wc command:
- To count all the files and folders present in a directory:
ls list | wc -l
root@537302f1e700:~/fruits# ls list | wc -l
1
- To display the number of word count only of a file:
wc -w list
root@537302f1e700:~/fruits# wc -w list
17 list
7. grep
The grep command is used to search for particular information in a text file.
grep [options] pattern [path]
root@537302f1e700:~/fruits# grep banana list
banana
Or:
root@537302f1e700:~/fruits# cat list | grep banana
banana
grep options:
- -v: Displays names not matching the specified word.
- -i: Filters output in a case-insensitive way.
- -A: Displays a line after the result.
- -B: Displays a line before the result.
- -C: Displays a line before and after the result.
8. nl
The nl command is used to print line numbers of the text file.
nl [file-path]
root@537302f1e700:~/fruits# nl list
1 apple
2 mango
3 banana
4 oranges
5 Melon
6 mandarin
root@537302f1e700:~/fruits# cat -n list
1 apple
2 mango
3 banana
4 oranges
5 Melon
6 mandarin
7 Avocado
8 coconut
9 Cherry
10 lime
11 Longan
12 1mango
13 3lime
14 20melon
15 apple
16 mango
17 apple
nl options:
- -n: To display the number and file.
- -s: To display number padding with space.
- -b: To display line numbers when the pattern matches.
Some applications of nl command:
-
To print a certain section of a file:
root@537302f1e700:~/fruits# head -5 list | nl -b a 1 apple 2 mango 3 banana 4 oranges 5 Melon root@537302f1e700:~/fruits# head -n 3 list | nl -b a 1 apple 2 mango 3 banana
Shell Expansions
Shell expansion is the process of simplifying commands in order to get a return value from a shell prompt. The shell reads the command line and analyzes it to make it easier to understand.
The types of shell expansion include:
1. Pathname Expansion
This is also known as "globbing." Pathname expansion is performed by the shell when it encounters a wildcard pattern (e.g., *, ?, []). These wildcards are replaced with a list of matching files and directories.
For example, ls *.txt would list all .txt files in the current directory.
$ ls *.txt
file1.txt file2.txt file3.txt
2. Tilde Expansion
When the shell encounters a tilde (~), it expands it to the current user's home directory.
For example, cd ~ would change the directory to the user's home directory.
$ echo ~
/home/username
3. Arithmetic Expansion
Arithmetic expansion allows the evaluation of an arithmetic expression and its substitution into the command.
For example, echo $((2 + 2)) would output 4.
$ echo $((2 + 2))
4
4. Command Substitution
Command substitution allows the output of a command to replace the command itself. This is done using backticks () or$(...)`.
For example, echo "Today is $(date)" would output the current date.
$ echo "Today is $(date)"
Today is Mon Aug 28 10:37:26 UTC 2024
5. Brace Expansion
Brace expansion is used to generate arbitrary strings. It is typically used for creating multiple similar files or directories at once.
For example, echo file{1..3}.txt would output file1.txt file2.txt file3.txt.
$ echo file{1..3}.txt
file1.txt file2.txt file3.txt
6. Variable Expansion
Variable expansion allows the value of a variable to replace the variable name.
For example, if NAME="John", then echo "Hello, $NAME" would output Hello, John.
$ NAME="John"
$ echo "Hello, $NAME"
Hello, John
Conclusion
Learning about I/O redirection, shell filters, and shell expansion is essential for effective command-line usage. These features allow for powerful text manipulation and automation, enabling users to work more efficiently with files, processes, and data in a Linux environment. By mastering these concepts, you will be better equipped to harness the full potential of the command line and streamline your workflow.
Top comments (0)