DEV Community

Shell Scripts Matter

Thibaut Rousseau on March 12, 2017

The shell is an odd beast. Although it goes against every current trend in software engineering (strong typing, compile checks over runtime check...
Read full post
Collapse
 
philosophicles profile image
Stuart Cuthbertson • • Edited

Really great advice.

One other thing I'd promote is Google's Shell Style Guide: google.github.io/styleguide/shell.xml. It's the nearest thing I've found to a universally-agreed style guide for Bash scripting.

Of course, it's actually only mandated for code written internally at Google, but it seems to me to be worthy of wider adoption.

Collapse
 
ben profile image
Ben Halpern •

Good call

Collapse
 
siziyman profile image
Denis Chikanov •

While the advice given here is indisputably great and worthwile, there's one more important thing IMO - most of the times you should not write long/complicated shell scripts. Among other people, Bourne himself stated that in relatively recent talks - it's not built for that really.

Collapse
 
ben profile image
Ben Halpern •

Agreed

Collapse
 
jasondewitt profile image
Jason DeWitt •

Great article, thanks for the tips. I did run into one problem, I copied your template to use it and just the base template is failing a shellcheck test:

^-- SC2145: Argument mixes string and array. Use * or separate argument.

github.com/koalaman/shellcheck/wik...

I took that advice and changed from $@ to $*, the example still works, so I thought I would point it out.

Collapse
 
thiht profile image
Thibaut Rousseau •

That's fixed!

Collapse
 
melezhik profile image
Alexey Melezhik • • Edited

HI! Good article.

On "It constitutes a library. Shell scripts can be hard to write. If there's a reference for something difficult somewhere, your coworkers will thank you when they need it. You should setup a "shell-scripts" repository somewhere as soon as possible."

There is SparrowHub - a repository of useful scripts, so welcome to contribution. If one has a useful Bash script - please upload it to SparrowHub and share with others. It's pretty easy.

Collapse
 
ikirker profile image
Ian Kirker •

Most of this is good stuff I agree with, but depending on "$0" to be a usable path to the executing script isn't the most reliable technique, and when it fails it will be pretty confusing.

unix.stackexchange.com/questions/1...

I tend to use a big multiline string for my usage information, instead.

Collapse
 
thiht profile image
Thibaut Rousseau •

Thanks for the comment!

I like to say perfect is the enemy of good. $0 is not perfect but good enough in most situations. I think it's a good trade-off compared to the 15 lines better solution :)

Knowing it's not perfect is important though, I won't mention it in the post because I believe comments are an actual part of the article itself, so your comment on this point is good!

You're right on multiline strings, they're probably better everyday. I wanted to show off weird stuff with this example. In practice though I have noticed people are more prone to update the #/ comments than a usage string, I have no idea why.

Collapse
 
camelcaseguy profile image
Shubhendra Singh Chauhan •

Hey @thiht , These are really some good practices that one should follow while writing shell scripts. 👏
I'd also like to make an addition to the static code analysis tool for Shell - DeepSource's Shell analyzer which detects issues in sh, bash, dash and ksh and lets you automatically fix most of them. It covers ShellCheck issues, and it's free for Open-Source projects.
Do give it a try and share your thoughts. 😉

Collapse
 
skontar profile image
Stanislav Kontár •

Overall a good article. I cannot say that I agree with everything, but that does not really matter. All information we find should be taken with a grain of salt.

I would maybe add a really good resource, mywiki.wooledge.org/FullBashGuide and related sites (Pitfalls, FAQ), and also #bash channel on Freenode.

Also one good advice I have learned during my journey, it is useful know that some commands, syntax and also a lot of advice on Internet are outdated or obsolete. wiki.bash-hackers.org/scripting/ob...

Collapse
 
vosikwemhe profile image
Victory Osikwemhe •

i don't always use the trap builtin command, but i will start using it in my script. You should have made provision for type checking using the type builtin command.

for example to check if a command is a function or an alias


    s() {
       printf "bash is good\n"
    }

    checkType=$(type -t s)

   [[ "$s" != "function" ]] && {
      printf "$s is not a function"
      exit 1 
   }
Enter fullscreen mode Exit fullscreen mode
Collapse
 
vandot profile image
Ivan Vandot •

Great article! I am getting an error for shUnit2.
Inside program I set IFS=$'\n\t' and when I source that file inside _test file I get the following error:
/usr/local/bin/shunit2: line 105: eval __shunit_lineno=""; if [ "${1:-}" = "--lineno" ]; then [ -n "$2" ] && __shunit_lineno="[$2] "; shift 2; fi: command not found
When I override IFS back to default inside _test file everything works as expected.

Collapse
 
grahamlyons profile image
Graham Lyons •

I really enjoyed this article and it's got some great tips in it. I've seen countless examples of shell scripts being treated as second-class code citizens and given how important they are it's so unhelpful to do that.

I totally agree that they should be version controlled - as all code should - but I think encouraging a 'shell-scripts' repo doesn't help to elevate them up to the level of tools written in other languages. For example, I wouldn't have a 'python-scripts' repo or 'java-apps' repo - I don't care what language a tool is written in, I want to know what it does.

StackExchange/blackbox is all shell but it's barely mentioned on the project page - it's a tool and you shouldn't need to know how it's implemented to be able to use it.

Collapse
 
panta82 profile image
panta82 •

Excellent advice all around.

shellcheck is a winner.

I'd also add, split your script into functions and do most of the work inside "main" or similar func. That makes it cleaner which parts of the code are synchronously executed and which are functionality to be called later (an alternative to the subshell () advice above).

Collapse
 
sobolevn profile image
Nikita Sobolev • • Edited

Great article!

Recently I wrote an article about testing bash scripts. I made a small comparison between different tools. Link: medium.com/wemake-services/testing...

So, my tool of choice is bats. It is really good. Here's how it looks: github.com/sobolevn/git-secret/tre...

Collapse
 
blackknight36 profile image
Michael •

Use Bash by default, Sh if you have to. Try to forget about Ksh, Zsh or Fish unless there are really good reasons to use them.

If you want your scripts to be truly cross-platform /bin/sh should be the default choice. FreeBSD does not even include bash unless it's installed from ports.

Collapse
 
lovepreetkaul profile image
Lovepreet Singh •

Great article!

Being a beginner in Bash Scripting, this article has provided a lot of valuable insight along with how to use them in the shell scripts we write. For me, the cleanup trap looks like a winner.

Collapse
 
meonlol profile image
meonlol •

If someone is interested in a really lightweight bash testing framework, I've built a KISS unit testing script a while back. You can simply Drop it in your repo and it searches and executes tests once you run it. Just try it out, feedback appreciated: github.com/meonlol/t-bash

Collapse
 
taikedz profile image
Tai Kedzierski •

Hear hear ; good article, I'll have to add IFS to my habits :) bash scripting is somewhat underrated ...!

Probably the biggest problem I find with bash is that you re-write everything often with every new script and keeping your monolithic scripts up to date regularly is a chore.

I've been maintaining a little tool, itself written in bash, for managing bash snippets and "include"-ing external files (along search paths, or just from current working dir)... just like "real" programming languages! It makes working with bash so much easier...

Collapse
 
bf4 profile image
Benjamin Fleischer •

Any thoughts on how to include a general functions.bash file in all scripts? should there be required SCRIPT_DIR env var that all scripts depend on, or should they all us relative paths?

Collapse
 
antonfrattaroli profile image
Anton Frattaroli •

Don't forget to give them the devops full treatment. Drop snapshots that can be deployed though your promotion process.

Collapse
 
cjoemoleculerun profile image
CJoeMoleculeRun •

I just want to say that this is a really great article and I've forced my entire team to read it.

CJ

Collapse
 
jimmydburrell profile image
Jimmy Burrell •

As a Bash shell scripting veteran of ~15 years, I'm amazed at how much I didn't know!

Excellent article.

Thank you.

Collapse
 
lyma profile image
João Lyma •

Thank you for excellent article!!!

Collapse
 
focusaurus profile image
Peter Lyons •

Another great tool worth mentioning is shfmt which can automatically pretty-print shell code. There is also an Atom editor plugin I wrote.

Collapse
 
thiht profile image
Thibaut Rousseau •

shfmt seems like a great tool, thanks for sharing!

Collapse
 
wigwam profile image
OnyxSage •

Ironically, UnixToolTip tweeted a link: The Most Brutal Man Page. That man page is that of Bash's.

Collapse
 
piotrzelasko profile image
Piotr Żelasko •

TBH I've added this page to my bookmarks as a cheat sheet. It's become very useful for me and thank you for that.

Collapse
 
dyw972 profile image
Yohan D. •

Thank you for your article! Very useful for a novice. 🖖

Collapse
 
asafjaffi profile image
Asaf Jaffi •

Thank you. Does your mini logger logs time for each entry? Could help with all those failing crons...

Collapse
 
thiht profile image
Thibaut Rousseau •

No it doesn't. But it can easily be added with the date command with a pattern like +%Y-%m-%d %H:%M:%S