Don't Use Bash for Scripting (All the Time)

Niko Heikkilä on May 06, 2019

Writing scripts is a subset of coding we sometimes can't avoid nor should be afraid of. The standard tool for writing scripts is Bash for UNIX envi... [Read Full]
markdown guide
 

The strong points of bash (ksh, and sh) are the ubiquity. I am a strong proponent of using the right tool for the job, and many of the other scripting languages out there (python, node, etc) don't come close to the ubiquity of shell scripts.
I can agree that to the uninitiated there is a lot of syntax that looks really odd, but with well structured scripts (making proper use of functions, built-ins, etc), bash scripts can be quite understandable and powerful.
To take full advantage of shell scripting (which can be quite performant and portable), script authors should learn more about the shell and take advantage of shell-isms. Bash, for example, has native support for REGEX (in version 4.x up, I'm not sure how far back it goes) that negates the need for many grep operations. In addition one should seldom use grep piped to sed or awk; those programs (nay, programming languages) contain all that's necessary to do the same work without spawning another executable.
Using IFS and array variables is a much cleaner solution than using cut. Default values, variable splitting, and no-op operations are indispensable for moderate to advanced scripts.
Against your point, however, bash/shell scripts are nice because they are concise. The "hello world" example you contrived is a good point: the shell script is 2 lines. The python code is 2 lines plus the setup for argparse. Once one is familiar with the syntax, scripts like those are easier to read in bash because they're shorter.
I also like the points about using shellcheck and strict-mode. My experience is also that posix-mode is a good tool to use in scripts where either the bash version isn't guaranteed, or where bash may not be used at all. There are also some really nice testing frameworks (like BATS) that will help with code correctness. Any sufficiently advanced scripts should also be modular (and make use of the source built-in), with perhaps a build that will make a single executable script for deployment. Bash scripts, like any other code, should also be source managed unless they are sufficiently small, straightforward, and localized, that source management just doesn't make sense.
I, unfortunately, see many places where heavier scripting languages like python or node are shoehorned in to do a job where one or more bash scripts would do the job with less effort. We are entering a time when fewer and fewer people are as familiar with the basics of shell scripting, yet there are far more tools and libraries available to make shell scripts easier to write.
In conclusion: I agree with the premise that bash/shell scripting should not be used everywhere for every task, but I disagree with you on where to draw that line. Each and every task ought to be evaluated to determine which tool is best for that job, and many times a simple (or moderately complex) shell script will get the job done just as fast (or faster) with less effort than a heavyweight scripting language will (because shell scripts are designed to do one thing very well, and other scripting languages may be designed with other goals).

 

Good points. An analogy could be made with text editors. On the other side we have Vim which is super powerful editor in the hands of a wizard but many still prefer to use eg. VS Code for getting job done nice and easy. I use both, by the way.

 
 

Let's do the same in Python.

name = args.name or "World"
print(f"Hello, {name}")

😕

% python -c 'name="World"; print(f"Hello, {name}")'
  File "<string>", line 1
    name="World"; print(f"Hello, {name}")
                                       ^
SyntaxError: invalid syntax

% python3 -c 'name="World"; print(f"Hello, {name}")'
  File "<string>", line 1
    name="World"; print(f"Hello, {name}")
                                       ^
SyntaxError: invalid syntax
 

You're using Python 3.5 or earlier? f-strings are quite delicious as of 3.6. :)

 

Yep! It's just the example of many assumptions underpinning python code for scripting, and so on

Yes, for maximum compatibility the old % way of concatenating would be safest, of course.

 

I started reading this with the intention of sharing "all the counter points". Admittedly, not a good way to start reading something. After reading through it though, your points are well thought-out and communicated. So kudos on the great post!

This points that I would make (on both sides) have mostly already been made.

Point: Bash is everywhere.
Counter point: Same with Python 2.7 (for the most part). Precompiled Go binaries can run anywhere.

Point: Bash do almost everything you need in a script.
Counter point: Ruby, Python, Go, etc. can do it better and simpler.

Point: Bash doesn't require external dependancies.
Counter point: openssl, libcurl, etc. are external dependancies, they're just typically already installed. Python, Ruby, etc. can also be written without external dependancies.

I could go on.

As Aaron Cripps mentions in his comment, "I am a strong proponent of using the right tool for the job."

All that said, personally, I'm a huge fan of Bash. With solid programming practices and the use of a linter, it can be clean, concise, portable and readable.

 

Thanks for reading the article thoroughly. 👍🏽

 

What are your thoughts on using binary compiled software versus scripting languages?

One thing to consider is that when the tools you write are regularly used there is a point at which a software application becomes a more suitable tool than a script, no matter how convenient the tool or script was when first created.

Thoughts?

 

Some of the larger scripts I create with different languages I actually put in Docker images so they are quite like binaries. Helpful when moving those between platforms.

I'm trying to learn Golang better to do this without containers as well.

 

Do developers use C or C++ anymore for anything other than operating system code?

I used to be an application software maintainer for a commercial UNIX operating system group. The classic old utilities were written in C. The desktop utilities (Motif, CDE, etc.) we're written in C++.

That was in the mid to late 1990s.

Most newer Linux distributions have C code for core applications and a somewhat greater variety of coding languages for applications.

Golang is an example of a comparatively newer application language.

 

Do you see application development being done frequently with interpretive languages?

 

tl;dr "Shell should only be used for small utilities or simple wrapper scripts." google.github.io/styleguide/shell.xml

It is all I need to know.

 

That's a nice resource that I haven't seen before, thanks for that!

... definitely going to stick to those guidelines in future scripts.

 

I have to agree. The more complex the task, the more tedious bash seems. Especially when I need to pair it with .bat for cross platform projects, python or node make a lot more sense. My preference is coffeescript. I find Cakefile to be very helpfull.

 

Task runners are great tools for running maintenance scripts on projects. My usual pick is pyinvoke but, naturally, these should be selected depending on the project language.

 

I personally use python scripts for anything that stands alone or anything that benefits from internal classes. Anything running in my build server and needing system utilities like scp and ssh to push data around goes in a bash script for ease. This came about after spending a great deal of time chaining bash commands together into one liners that did a great deal of heavy lifting for me regularly. I stepped from one liners into bash scripts, and eventually decided that a good bit of the work that I was doing would be more easily written in python, and for the most part I've been able to keep with python for all things that are not deploying my java applications.

 

codeproject.com/?cat=2 has quite a few projects that use C++ in conjunction with the libraries and API's for a wide variety of system, application and programming interfaces.

Admittedly many of them are close to the system level as opposed to rapidly developed tools and conveniences.

 
Sloan, the sloth mascot Comment marked as low quality/non-constructive by the community View code of conduct

"Bash shines in small and efficient scripts which are designed mainly for one thing: repeating instructions you would otherwise type manually from top to bottom. "
Did you make any resources or know at all bash to write any post about it? Bash this bash that..
Check first Google result about bash complexity like ex. unix.stackexchange.com/questions/2...
Read more before posting.
Fix this post as it's full off lies.

 

Hi, I've written far too many Bash scripts to understand its downsides. The link you provided doesn't really change my views, would you care to elaborate where I have made mistakes?

Furthermore, your so-called "lies" are just opinions. It's not too hard to have some respect while commenting. Remember this and I don't have to report you.

 

Ah sorry. I forgot that this is opinion here. Not the facts. Then if there are no facts then I didn't get at all what this articles is about. Title should be in this case 'in my opinion don't use old non readable bash scripts as it have limits, you cannot extend it with new functions and commands and it's just for repeating what you writing in cli'
Lies:

  • "Bash shines in small and efficient scripts which are designed mainly for one thing: repeating instructions you would otherwise type manually from top to bottom" Bash main purpose is to access results of commands and do something with it. Also there is hundreds of complex programs written in it like ex installations scripts
  • "Syntax of Bash is ugly and has a steep learning curve." Step learning curve because or, args and modules? None English learner will need to learn the syntax this way or that way and for them it is no difference.
  • "Writing good Bash scripts takes years of practice" In this case writing javascript take 100 years..
  • "There might come a day Bash has a good (nested) dependency system and friendlier syntax." How usage of command can be more friendly and better than is now?
  • "complexity grows too large to handle reasonably within the limits of Bash" Limits?

Instead of saying 'hey guys you can run javascript, php, python, go etc from cli directly' you blaming that bash is crap.

Eh.. Don't like negative post.
And if this is your blog and this should be just opinion about not using bash as its shit then sorry for saying lies.
I used to devs who using facts when saying anything about language syntax etc.

Thanks for your input. Since you're just obviously misreading intentionally I'm signing this discussion off with a notice that opinionated blog posts and research articles are two different things. Learn that.

  1. Opinionated articles should have subjective title.
  2. The content of article is strictly not opinionated as there are no words (about bash) like:

Introductory Words and Phrases:
I think
I believe
I feel
In my opinion
My favorite
The best
I strongly believe
From my point of view
It’s my belief
Based on what I know
I am convinced
Speaking for myself
I know you will have to
agree that
I am confident that

Transitions:
Opinion Clues
First/second/third
First of all
Next
After that
Additionally
Equally important
Consequently
Besides
Further/furthermore
Clearly
Obviously
In addition
For all these reasons
Finally
In conclusion

Opinion Clues:
Always/Never
Awful/Wonderful
Beautiful/Ugly
Better/Best/Worst
Delicious/Disgusting
Definitely
Enjoyable/Horrible
Favorite
For/Against
Good/Bad
Inferior/Superior
Oppose/Support
Terrible
Unfair
Worthwhile

Why I don't like this post, scenario: I will have some junior/mid devs at work who will write cli scripts in js instead of bash saying that js:

  1. is easier
  2. have no limits
  3. is more readable
  4. have modules etc.
  5. bash is hard to learn
  6. bash is only for repeating command in cli

When I will ask them from where they have this information, they will point this article/post (as this is not a discussion neither opinionated content about bash)
After that there will be a big conversation why we cannot use that, why we don't have node on server (to run this little script of them which should be written in bash), after that chat with manager to explain everything and half of my day will disappear.

This is a facts. Fact when 'so called' opinionated articles/post around internet making discussion with junior/mid devs at work hell.

Sharing knowledge of using js and other language in cli is amazing Mate.
Saying that bash is shit and saying to young devs that they should move to 'more advance/better/more readable' like js is more than evil.

Your scenario highlights a work culture where junior dev skills are fundamentally underestimated and undervalued. If you get kicks from lecturing to people, then show us a better example and write a post here why (in your opinion) Bash should be used for scripting. After all, DEV is a free platform.

 

This is very hostile.

I'm guessing that you identify with Bash, and since Bash is part of your identity, you took the original post as a personal attack.

There is no need for this.

There are better ways to disagree with someone than to attack someone's competence and integrity.

 

I identify with everything. Probably you don't, probably other people also don't. Probably if somebody will pop in to:

  • your garage fixing cars for ages as mechanic
  • your building site where you putting second floor for house as builder

and say that all your tools are limited, old, crap, you making such a mistake using it, you will be ok.

I'm not. There are better ways of writing articles, better ways of dealing with facts what was written. There are better ways to disagree? Not for stuff like that Mate.

You cannot stand criticism based on facts - good luck being good dev.

You are just trolling here, friend.

No one in this discussion is hating Bash, no matter how much you like to throw that around. No one is telling it's a mistake to use Bash; there are just alternative tools that might serve you better depending on the use case. No one is coming to delete your Bash scripts.

Like I said, you are intentionally drawing a straw man argument to this discussion which is not welcome here nor does it contribute to anything. Let it go, take a walk, and chill.

Being a good developer or not isn't the issue. It's your choice of language.

Calling out statements as being opinion is fine. Disagreeing with them and providing a better option is fine. But calling them lies and insulting the intelligence of the author? That's not fine.

I'm guessing from the quality of your English that it's not your native language. Perhaps there's a misunderstanding here in terminology. For example, "lie" is not the same as "falsehood." The former implies malicious intent, whereas the latter does not.

Sloan, the sloth mascot Comment marked as low quality/non-constructive by the community View code of conduct

dictionary.com/browse/lie

  1. an inaccurate or false statement; a falsehood.

falsehood is synonymy of lie.

True, English is not my first language.
Apologize Niko for using lies instead of falsehood. Didn't mean to use lies as Ben mention.

"insulting the intelligence of the author" ?

  1. Is not my fault that meaning of lie looks like that
  2. straw man is wrong usage Niko. I'm not talking about you, I'm talking about your articles and facts. thebestschools.org/magazine/15-log...
  3. Niko, you trolling bash Mate. Not me. As I've mention, why not write nice post about usage of javascript in cli instead of trolling bash first ?
code of conduct - report abuse