Another cheat sheet for Bash

Basics

Starting somewhere... Be careful, some command might require bash 4 (the vast majority of snippets are for bash 3, though).

Bash is an acronym

Bash stands for "Bourne Again Shell", which refers to "born again shell". Indeed, Bash has replaced the old Bourne shell initially released in 1979.

Bash is a "Unix-versal" scripting language

Bash is a crazy good language you can learn to automate various tasks whether it's for your personal use, for server maintenance, or in devOps actions. Bash scripting may consist of writing a series of command lines in a plain text file.

Note that you can type these commands on your terminal as well. You can use existing commands such as grep, sed, find but also additional commands from third-party libraries you might have installed.

However, it's often more convenient to group all instructions in files you can git and transfer from one machine to another or run in multiple servers. Bash scripts are usually extension-less files you can execute this way:

bash myscript

Alternatively, you can make your script executable and run it:

chmod +x myscript && ./myscript

Bash is not sh!

While Bash can run most sh scripts (the syntaxes are quite similar), sh cannot handle all Bash commands.
The following command lines will probably work but they are a bit misleading:

bash mysh.sh
sh myscript

Don't forget the Shebang

It's the #! at the top of bash files. "Bang" is for the exclamation point "!":

#!/usr/bin/env bash

The above line is not decorative at all! It links the script to the absolute path of the Bash interpreter. /usr/bin/env is usually a smart choice as it grabs the executable set in the user's $PATH variable.

Writing comments

Use a #:

# comment

Variables

Variables are easy to declare, use, and manipulate in Bash.

Built-in variables (not an exhaustive list)

echo "$1 $2" # positional arguments passed one by one, e.g., bash myscript arg1 arg2
echo "$#" # number of arguments passed
echo "$@" # all arguments passed
echo "You are here: $PWD" # $PWD is the current path

Assigning new variables

You assign variables with the = sign and no space around:

MyVar="My string"
MyArray=(all in one)

Using variables

You can use declared variables with the $ sign:

echo $Variable
echo "This is my var: $Variable" # double quotes are required for interpolation
echo "${MyArray[@]:0:3}" # "all in one", because it prints 3 elements starting from the first (0)

# Be careful the following does not display a value
echo ${#MyArray[2]} # "3" as there are 3 chars in the third element of the array, which is the word "one"

Modifying variables

You can modify declared variables with various signs enclosed with {}:

MyVar="My variable"
# Length
echo ${#MyVar}

# Substitution
echo ${MyVar//a/O} # "My vOriOble"

# Expansion
echo ${!BASH@} # all variable names than begin with "BASH"
echo ${!BASH*} # all variable names than begin with "BASH"

# Removals
MyFile="list.txt"
echo ${MyFile%.*} # the filename without extension
echo ${MyFile##*.} # the file's extension

# Default value
echo ${AnotherOne:-"Another one"} # displays "Another one" even if AnotherOne is not defined

Dictionaries

declare -A Movies

Movies[title]="The Dark Knight Rises"
Movies[bestActress]="Anne Hathaway"
Movies[director]="Christopher Nolan"

printf

printf is the C-like way to print preformatted text. It provides way more advanced features than a simple echo. It takes formats and parameters as arguments and prints the formatted text.

For example, you can specify the type and some formatting option in one line:

printf "%s\n" "Learn it" "Zip it" "Bash it" # it prints 3 lines
printf "%.*f\n" 2 3.1415926 # prints 3.14

Loop, loop, loop

For

for i in "${MyArray[@]}"; do
    echo "$i"
done

It works with strings too:

MyString="First Second Third"
for n in $MyString
do
   echo "$n line"
done

While

while [ true ]
do
    echo "We loop"
    break
done

Handy ranges

echo {1..7} # 1 2 3 4 5 6 7
echo {a..g} # a b c d e f g

Until

until [ $counter == 111 ]
do
    echo "Number is: $((counter++))"
done

Get user inputs

echo "6x7?"
read answer
echo answer # hopefully 42 but can be anything the user wants

Conditional statements

if [[ CONDITION1 || CONDITION2 ]]
then
    # code
elif [[ CONDITION3 && CONDITION4 ]]
then
    # code
else
    # code
fi

Alternatively, you might use a case statement:

case $COLOR in

  Red)
    echo -n "red"
    ;;

  Blue | Green)
    echo -n "blue or green"
    ;;

  Yellow | "Sunny" | "Gold" | "Fire")
    echo -n "Yellow"
    ;;

  *)
    echo -n "What?"
    ;;
esac

Errors & exit strategies

It's best if you can raise errors to prevent any misuse of your scripts.

Exit immediately if a command failed

#!/usr/bin/env bash
set -e

Exit N

You can exit the script and specify an exit code:

#!/usr/bin/env bash
if [ CONDITION ]; then
    exit 0 # 0 1 2 3 ... N
fi

0 indicates a success whereas {1,2,3, N} are errors.

Test errors

# $? is the exit status of last command
if [[ $? -ne 0 ]] ; then
    echo "You failed!"
    exit 1
else
    echo "You are the best!"
fi

Debug bash scripts

#!/usr/bin/env bash
set -x # set -n can also be used to check syntax errors
ARGS=("$@") # store all script args in a var

You can also execute the script with the -x option in the terminal:

bash -x myscript

Filesystem & directories

You can use all basic filesystem commands such as cp, rm, ls, mv or mkdir in your bash scripts.

To check if a file is executable before anything:

if [ ! -x "$PWD/myscript" ]; then
    echo "File is not executable!"
    exit 1
fi

Use -f for files, -d for directories, -e for existence, etc. There are tons of options, you can even test symbolic links with -L and compare files by date with the -nt (newer than) and -ot (older than) options.

A bit more advanced usages

Here are some tips I consider a little more complicated than the basic ones.

Leverage the benefits of the bash syntax

Bash is pretty convenient to build output fast, especially using brace expansions:

echo {1..10}{0,5}h # bash 3
echo {10..120..10}km # requires bash 4

Define functions

function test ()
{
    echo "$1 $2" # displays positional arguments passed to the function ("All" "In")
    return 0 # if you return a value, it !must be numerical!
}
test "All" "In"

Scopes

You don't have to declare variables in Bash. If you call a variable that does not exist, you don't get any error but an empty value.

Environment variables

Env vars are defined at the system level. They exist everywhere, in all processes.
To list them, use printenv:

printenv USER # if you don't specify a variable, all variables will be displayed

Shell variables

The following assignment defines a shell variable:

TEST="My test"

You can use it directly everywhere within the script, including in functions. However, they're not passed to child processes unless you use the export command to transmit such information.

The local keyword

The local keyword can only be used within functions:

function test ()
{
    local MyVar="My string" # local variable
}
test

You cannot export a local variable.

Best practices

As a general rule, it's better to use local variables and functions. Global variables should be used meaningfully. It prevents unwanted overriding and confusions.

Loop through the results of a command

for r in $(ls $PWD)
do
    # task using the result of the command => r
    echo "$r"
done

Use the output of a function

function hello() {
    echo "Hello Ethan"
}

echo "This is the result of the function : $(hello)"

Store the result of a command in a variable

Users=$(cat users.txt)

Capture yes/no inputs

read -p "Continue? [y/n]: " -n 1 -r
echo # extra line
if [[ $REPLY =~ ^[Yy]$ ]]
then
    echo "ok"
elif [[ $REPLY =~ ^[Nn]$ ]]
then
    echo "Too bad!"
else
    echo "Sorry, I did not understand your answer :("
fi

Capture user's selection

The following bash code prompts a selection to the user and capture the anwser:

select w in "zsh" "bash" "powershell" 
do
  echo "You prefer $w"
  break  
done

Aliasing and overriding commands

You can create special aliases:

alias lz='ls -alh'

You can also override existing commands simply by redefining them.

Chaining commands

wget -O /tmp/logo.jpg 'https://dev-to-uploads.s3.amazonaws.com/uploads/logos/resized_logo_UQww2soKuUsjaOGNB38o.png' && echo "echo only if the first hand is true"
wget -O /tmp/logo2.jpg 'https//example.com/logo.jpg' || echo "echo only if the first hand is wrong"
wget -O tmp/logo3.jpg 'https//example.com/logo.jpg' ; echo "echo whatever happens"

The hard mode

Here some advanced concepts in Bash you might find useful.

Execute last executed command

sudo !!

Redirections, stderr, stdout

COMMAND > out.txt   # write in out.txt 
COMMAND >> out.txt  # append in out.txt 
COMMAND 2> test.log # stderr to test.log
COMMAND 2>&1        # stderr to stdout
COMMAND &>/dev/null # stdout and stderr to (null)

Replace COMMAND with your command.

It's a Trap!

The trap command is an advanced approach of errors. The key to understand it is to forget about exceptions and similar concepts you might know in other languages.

The idea is to execute something only in specific conditions:

#!/usr/bin/env bash

trap 'catch' EXIT

catch() {
  echo "Exit game!"
}

echo "ok"
echo "---"

exit 0

The message "Exit game!" displays only because of the exit 0 at the end.

More pragmatically, it's not uncommon to see that kind of usage:

#!/usr/bin/env bash

trap "rm /tmp/scan.txt" EXIT

Execute scripts inside a Bash script

Use the source command:

#!/usr/bin/env bash

source ./otherScript

Subshells

A subshell is a separate instance of the command processor. A shell script can itself launch subprocesses. These subshells let the script do parallel processing, in effect executing multiple subtasks simultaneously.

Source: tldp.org

In other words, commands enclosed inside parenthesis execute in a subshell. It partly explains the use of the export command that passes environment variables on to sub-processes.

You sometimes read the term "forks" instead, but it's the same idea.

pipefail

You might use the following as a more advanced error handling:

#!/usr/bin/env bash

set -eu # u is to exit if using undefined variables
set -o pipefail

The above set commands not only enable "error mode" but also prevent any error in pipeline from occurring silently, which is the default behavior if you don't set the -o pipefail option.

My favorite tricks in the terminal

The following commands might be more Unix-based than pure bash scripting, but it's not completely off topic 🤞🏻.

Use bash

You might have install third-party solutions for terminal prompt or use zsh. Just type:

bash

If you press enter, you will now use the bash prompt. Then you might type man bash to display the help.

Use \

You can use \ at the end of long command lines to improve readability and not lose your focus.

Use &

nohup bash script &

The above allows executing bash scripts in the background and catch the hangup signal.

Misc

Some tricks I use frequently:

List sub-directories only in the current folder

SUBDIRS=$(ls -d */)
for sub in $SUBDIRS
do
    echo $sub
done

Quick rename a file

mv /project/file.txt{,.bak} # rename file.txt.bak

Execute local bash script on a remote server

ssh REMOTE_HOST 'bash -s' < myScript

Find heavy files (larger than 10MB)

find . -size +10M -print

Create multiple files quickly

You can type:

touch {script1,script2,script3,script4,script5,script6,script7}

But that's still a bit tedious, so use this:

touch script{1..7}

Bash for hackers

Hackers would bash for sure. While Python is amazingly convenient for pen-testing, bash can also help automate many tasks. It's not uncommon for professionals to create custom bash scripts to speed up analysis.

Hello world: scanning script

Instead of repeating the same command lines over and over, you can group them into specific files. Most security binaries are built like that.

You might use the read command to make your script a bit more friendly to use or just pass arguments. Let's name our script scanning

#!/usr/bin/env bash

echo "Welcome aboard"
echo "What's the targeted IP?"
read TargetIp

if [[ ! $TargetIp ]]
    then echo "Missing targeted IP!"
    exit 1
fi

echo "What port do you want to scan?"
read PortNumber

if [[ ! $PortNumber ]]
    then echo "Missing port number!"
    exit 1
fi

nmap $TargetIp -p $PortNumber >> scan.txt # append results in scan.txt

Then we can run bash scanning. Of course, there are existing tools to ease the use of nmap and it's a very incomplete example, but you see the idea.

Aliasing

Attackers would likely use aliases when using advanced commands in the terminal:

alias whatsMyIp="echo $(ifconfig -a | grep broadcast | awk '{print $2}')"
whatsMyIp # IP local

Pen-testing distribution may have better autocompletion features, though, but you don't always operate in such optimized environment, so aliases can save a lot of time and energy.

Hacking bash

Hackers can hack bash like any other language. Just read this post by hacktricks.

Investigate

Start here

Comments

[Mayank Raj]

Note: In zsh arrays are 1-indexed

echo ${#MyArray[2]} # This will display 2 = size("in") in zsh.