Shell Scripts Matter

Thibaut Rousseau on March 12, 2017

The shell is an odd beast. Although it goes against every current trend in software engineering (strong typing, compil... [Read Full]
markdown cheatsheet

While the advice given here is indisputably great and worthwile, there's one more important thing IMO - most of the times you should not write long/complicated shell scripts. Among other people, Bourne himself stated that in relatively recent talks - it's not built for that really.

Really great advice.

One other thing I'd promote is Google's Shell Style Guide: google.github.io/styleguide/shell.xml. It's the nearest thing I've found to a universally-agreed style guide for Bash scripting.

Of course, it's actually only mandated for code written internally at Google, but it seems to me to be worthy of wider adoption.

Great article, thanks for the tips. I did run into one problem, I copied your template to use it and just the base template is failing a shellcheck test:

^-- SC2145: Argument mixes string and array. Use * or separate argument.

github.com/koalaman/shellcheck/wik...

I took that advice and changed from $@ to $*, the example still works, so I thought I would point it out.

HI! Good article.

On "It constitutes a library. Shell scripts can be hard to write. If there's a reference for something difficult somewhere, your coworkers will thank you when they need it. You should setup a "shell-scripts" repository somewhere as soon as possible."

There is SparrowHub - a repository of useful scripts, so welcome to contribution. If one has a useful Bash script - please upload it to SparrowHub and share with others. It's pretty easy.

Great article!

Recently I wrote an article about testing bash scripts. I made a small comparison between different tools. Link: medium.com/wemake-services/testing...

So, my tool of choice is bats. It is really good. Here's how it looks: github.com/sobolevn/git-secret/tre...

Most of this is good stuff I agree with, but depending on "$0" to be a usable path to the executing script isn't the most reliable technique, and when it fails it will be pretty confusing.

unix.stackexchange.com/questions/1...

I tend to use a big multiline string for my usage information, instead.

Thanks for the comment!

I like to say perfect is the enemy of good. $0 is not perfect but good enough in most situations. I think it's a good trade-off compared to the 15 lines better solution :)

Knowing it's not perfect is important though, I won't mention it in the post because I believe comments are an actual part of the article itself, so your comment on this point is good!

You're right on multiline strings, they're probably better everyday. I wanted to show off weird stuff with this example. In practice though I have noticed people are more prone to update the #/ comments than a usage string, I have no idea why.

Excellent advice all around.

shellcheck is a winner.

I'd also add, split your script into functions and do most of the work inside "main" or similar func. That makes it cleaner which parts of the code are synchronously executed and which are functionality to be called later (an alternative to the subshell () advice above).

i don't always use the trap builtin command, but i will start using it in my script. You should have made provision for type checking using the type builtin command.

for example to check if a command is a function or an alias

    s() {
       printf "bash is good\n"
    }

    checkType=$(type -t s)

   [[ "$s" != "function" ]] && {
      printf "$s is not a function"
      exit 1 
   }

Overall a good article. I cannot say that I agree with everything, but that does not really matter. All information we find should be taken with a grain of salt.

I would maybe add a really good resource, mywiki.wooledge.org/FullBashGuide and related sites (Pitfalls, FAQ), and also #bash channel on Freenode.

Also one good advice I have learned during my journey, it is useful know that some commands, syntax and also a lot of advice on Internet are outdated or obsolete. wiki.bash-hackers.org/scripting/ob...

I really enjoyed this article and it's got some great tips in it. I've seen countless examples of shell scripts being treated as second-class code citizens and given how important they are it's so unhelpful to do that.

I totally agree that they should be version controlled - as all code should - but I think encouraging a 'shell-scripts' repo doesn't help to elevate them up to the level of tools written in other languages. For example, I wouldn't have a 'python-scripts' repo or 'java-apps' repo - I don't care what language a tool is written in, I want to know what it does.

StackExchange/blackbox is all shell but it's barely mentioned on the project page - it's a tool and you shouldn't need to know how it's implemented to be able to use it.

Great article! I am getting an error for shUnit2.
Inside program I set IFS=$'\n\t' and when I source that file inside test file I get the following error:
`/usr/local/bin/shunit2: line 105: eval __shunit_lineno=""; if [ "${1:-}" = "--lineno" ]; then [ -n "$2" ] && _
shunit_lineno="[$2] "; shift 2; fi: command not found`
When I override IFS back to default inside _test file everything works as expected.

Another great tool worth mentioning is shfmt which can automatically pretty-print shell code. There is also an Atom editor plugin I wrote.

shfmt seems like a great tool, thanks for sharing!

Any thoughts on how to include a general functions.bash file in all scripts? should there be required SCRIPT_DIR env var that all scripts depend on, or should they all us relative paths?

Hear hear ; good article, I'll have to add IFS to my habits :) bash scripting is somewhat underrated ...!

Probably the biggest problem I find with bash is that you re-write everything often with every new script and keeping your monolithic scripts up to date regularly is a chore.

I've been maintaining a little tool, itself written in bash, for managing bash snippets and "include"-ing external files (along search paths, or just from current working dir)... just like "real" programming languages! It makes working with bash so much easier...

Use Bash by default, Sh if you have to. Try to forget about Ksh, Zsh or Fish unless there are really good reasons to use them.

If you want your scripts to be truly cross-platform /bin/sh should be the default choice. FreeBSD does not even include bash unless it's installed from ports.

If someone is interested in a really lightweight bash testing framework, I've built a KISS unit testing script a while back. You can simply Drop it in your repo and it searches and executes tests once you run it. Just try it out, feedback appreciated: github.com/meonlol/t-bash

Thank you. Does your mini logger logs time for each entry? Could help with all those failing crons...

No it doesn't. But it can easily be added with the date command with a pattern like +%Y-%m-%d %H:%M:%S

Ironically, UnixToolTip tweeted a link: The Most Brutal Man Page. That man page is that of Bash's.

TBH I've added this page to my bookmarks as a cheat sheet. It's become very useful for me and thank you for that.

Don't forget to give them the devops full treatment. Drop snapshots that can be deployed though your promotion process.

As a Bash shell scripting veteran of ~15 years, I'm amazed at how much I didn't know!

Excellent article.

Thank you.

Thank you for excellent article!!!

Great article!

Being a beginner in Bash Scripting, this article has provided a lot of valuable insight along with how to use them in the shell scripts we write. For me, the cleanup trap looks like a winner.

code of conduct - report abuse