Shellscripting

Variables and Shebang

---

Small intro to shell programming

The Shell is the standard interface to every Unix and Linux system; users and administrators alike have experience with the shell, and combining commands into scripts is a natural progression. However, this is just a tip of the iceberg.

I've spent some time lately learning about shell and writing scripts, and I've realized that the shell is actually a full programming language with variables and functions, and also more advanced structures such as arrays (including associative arrays) and being so directly linked to the kernel it has native IO primitives built into it's very syntax, as well as process and job control.

I've planned this as a series of a few posts, and I'll try to be concise and on the point.

So, what is shellscripting?

Shellscripting is writing a series of commands for the shell to execute. It can combine lengthy and repetitive sequences of commands into a single and simple script, which can be stored and executed anytime, which is great for automating tasks. This reduces the effort required by the end user.

Script's commands are being executed by the interpreter (shell), one by one, and everything you can type in the command line you can also put in the script.

Before running scripts, we need to set up permissions for execution with: chmod 755 script.sh

We can then run the script with ./script.sh via command line.

---

#!

#! (shebang) specifies binary of the shell (interpreter) we want to execute the script, for example:
#! /bin/bash, #! /bin/zsh or for the best portability #! /bin/sh (this will run system shell).

Note that the most of code in this series is tested only with bash and zsh shell, most sh scripts can be run by Bash without modification, but some stuff wont work.

#! /bin/sh
sleep 90

When we execute the script that contains #! what actually happens is that interpreter is executed and the path used to call the script is passed as an argument. To confirm that, let's say we have sleepy.sh script, then we can run the script with ./sleepy.sh &, where & is used (it seems) to return the PID of the script execution process, and then we can run ps -fp [PID] to see process info:

UID PID PPID C STIME TTY TIME CMD
505 65418 59985 0 7:09PM ttys000 0:00.01 /bin/zsh ./sleepy.sh

We can see here that ./sleepy.sh is passed to my /bin/zsh binary as an argument.

If a script doesn't contain #! commands are executed with default shell, but it's the best practice to be explicit as different shells have slightly varying syntax.

Also, we don't have to use only shells as interpreters for scripts. We can also use other binaries like python:

#! /usr/bin/python

print "This is a Python script"

chmod 755 hi.py
./hi.py
This is a Python script

---

Variables

Variables are storage locations that have a name, and you can think of them as name-value pairs.
Syntax used to create a variable is: VARIABLE_NAME="Value". It's important to note that variable names are case sensitive, and that, by convention, variable names should be all in uppercase. Also make sure to not use spaces after and before = sign, when declaring a variable.

By default all variables are global, also they have to be defined before used.
Variables can be defined in the functions (we'll talk about them eventually), but we cannot access them before a function is called.

function var(){
    FUNC_VAR=1
}
# FUNC_VAR is not defined at this point and this will not return anything
echo $FUNC_VAR
var # This is how we call a function in the shell
# FUNC_VAR is now available because the function has been called
echo $FUNC_VAR # Output: 1

Valid variable names can consist of letters, numbers, and underscores, except that number cannot be the first char in the name.

# Valid names
DARK_JEDI="Vader"
GR4Y_J3DI="Ahsoka"
Regular_Jedi="Obi-Wan"

# Invalid names
3DARK_LORDS="Vader Sidius Plagueis"
TWO-REBELS="Solo Leia"
ONE@SHIP="Ebon Hawk"

#! /bin/bash
MY_SHELL="zsh"
echo "I like the $MY_SHELL shell" # Output: I like the zsh shell

We can also enclose the variable name in curly braces:

MY_SHELL="zsh"
echo "I like the ${MY_SHELL} shell" # Output: I like the zsh shell

Curly braces syntax is optional unless you need to precede or follow up the variable with additional data, like so:

MY_SHELL="bash"
echo "I'm ${MY_SHELL}ing on my keyboard!" # Output: I'm bashing on my keyboard.

Without curly braces this wouldn't work as the interpreter will take that ing following the name variable as a part of the variable name.

Another best practice is to enclose variables in quotes, when working with them, to prevent some unexpected side effects.

We can also assign the output of the command to a variable:

SERVER_NAME=$(hostname)
echo "You are running this script on ${SERVER_NAME}"

Local variables

Local vars are created with local keyword, and only functions can have the local variables, so they can only be accessed within the function where they're declared.

function myFunc(){
    local LOCAL_VAR=" I'm locally scoped"
}

It's the best practice to use only local variables inside functions.

---

That's it for now, I'll write a bit about tests and loops in the next one. Thanks for reading, and if you have any question, just ask away!

Comments

[Ben Sinclair]

by convention, variable names should be all in uppercase.

A lot of scripts do that, so you could call it a convention, but generally you should only use ALL_CAPS_NAMES for environment variables or other stuff that's part of the shell's world. Variables local to scripts should be lower_snake_case to avoid collisions and hint that they're not going to have any effects outside the script.

[Darkø Tasevski]

That sounds reasonable, thanks for the tip!

[Chris Leonard]

Good article! One comment: you should quote your variables, like this: "${myvar}". Google "bash quoted variables" for lots of discussion on this topic.

As an aside, unless I'm distributing something where I know someone might not have bash, I think it's fine to use bash for scripts. For example, everyone at work should have bash installed, so if I'm scripting there, I'll use bash. If it's intended for Linux and not Unix, I'll use bash, and let people either translate it or temporarily install bash if they don't have it installed for some reason (I think they should - it's a de facto community standard). I even installed bash on my router because not having its extensions was driving me insane.

But that's just me ... YMMV.

[Darkø Tasevski]

Thanks! I thought that only variables without curly braces should be wrapped in quotes. I'll update the article later to include this, thanks for the advice.

I agree on using bash for #! as most of my scripts, for now, are just for mine own usage, and I guess that all my friends at least have the bash shell on theirs PC-s if not actively using it.

Another thing that I've noticed is that with #! /bin/bash, if the user doesn't have it installed, the error will be something like: cannot find command: /bin/bash and the user can install it in mere minutes instead of getting a cryptic message I got because of using #! /bin/sh when I've experimented with shell functions and spent another 15 minutes trying to resolve the issue.

Shellscripts that will be used in production env, it may be a bit better to use #! /bin/sh but that can fail too, so there is no right answer I guess.

[Martin Schwarzer]

If you're using #! /bin/sh you have to be POSIX compliant, otherwise you might destroy something along the way, because you cannot know what shell you are running on.
If you are relying on bash built-in, obviously you have to specify that, otherwise you might again destroy stuff.

[Chris Leonard]

For example, regarding scripting:

stackoverflow.com/questions/100672...

Lots of other discussions around, but that should get you started.

[tux0r]

Addendum: It's best practice to write POSIX-compliant shell scripts. Never assume that your users have your shell.

#! /bin/sh

That's all you should need for shell scripts.

[Darkø Tasevski]

Yeah, I would agree that setting zsh as default shell would be a bad idea, it's here just for example purposes.

On the other hand, I've already encountered an issue when I've set shebang to be /bin/sh and after googling for a while I've found that preferred shell for scripts should be bash as basically bash is sh, but with more features and better syntax.
So, I would say that:

#!/usr/bin/env bash

is maybe a better solution.

source

[tux0r]

Unless your audience doesn't have bash...

Do you really need the additional features for your scripts?

[Darkø Tasevski]

Well, if you put it that way :D I guess you're right. Nevertheless, I don't think that I will write scripts for other users than me anytime soon.

[Evaldas Buinauskas]

That & after a command literally means to run a command in background (or you can call it async). Quite useful to trigger long running tasks.

[Darkø Tasevski]

Thanks for clearing that up for me!

[tux0r]

The Shell is the standard interface to every Unix and Linux system

Starting from Unix V8, this is increasingly less true.