It saddens, and surprises me the amount of manual work that we developers put up with. It is paradoxical that we are paid to automate other people’s work, yet fail do so the same with our own.
I am not just talking about automating tasks that can take you hours or days to do. A five second task that is repeated often enough may well be worth automating.
Automation saves time, enables repeatability, avoids mental overhead, and can be rewarding.
One off tasks
Your attitude before starting any task, no matter how small, should be: can I write a program to do this?
A word of caution, though: set a time limit for the task, as you can really get lost down the rabbit hole. Once that time limit has elapsed, you have to be willing to let it go and do the task manually. This requires a surprising amount of discipline.
Beware the GUI
GUIs, like the Sirens, lure you with an easy first time experience, but then enslave you in their own isolated world, away from other programs.
You want to use tools that compose. You want to use programs that you can glue together with other programs. Programs that can be inside a for loop.
GUIs do not compose. Free yourself.
Automation Tools
For quick and easy things, there is nothing like a good old Bash shell. It is the only tool that has remained constantly useful in my 17 years of working experience.
For more complex tasks, just use whatever your production programming language is. It probably has tons of libraries available which you are already very proficient with.
Or maybe you could use that “forbidden” language, the one that you actually love, thus making the task doubly enjoyable.
Remember that you are a developer. So develop. Write programs. Free yourself.
Oldest comments (13)
Great post. I had to scrub some production data last night that required a lot of manual checking before moving to the next step. I really wanted to just write a script, but there wasn't enough data to merit that, nor could I really work out how to automate the whole process that late in the day.
Definitely had those thoughts and feelings when I used my two simple methods over and over again.
Agree on most of your thoughts. Two links to add here:
xkcd.com/1205/ -- for an idea of estimating how much effort should be spent on automating tasks. I've always considered this a pretty good set of numbers.
automatetheboringstuff.com/ -- I tend to use Python for automation, and this gives a good crash course into right these things.
I always considered automation to be actually pretty interesting because it either enforces or requires not extremely deep skills in a particular language or technology but rather quite a wide range of technical skills including working with different applications, network protocols, file formats and the like. Where software development then and now has the chance to be "greenfield", automation is always and per se "brownfield". ;)
I have NEVER been productive with a Git GUI like SourceTree or Tower. I can only seem to get stuff done in terminal.
I can only say... Congrats! :)
Baby steps that saves me time.
This is exactly what I was referring to :). I see far too many people clicking around Putty.
I agree and interrestingly, you want to automate, gain time, but not to ide all complexity neither.
Otherwise you may not even know how to do the stuff at all anymore. A GUI tool will end up like that. You don't know anymore how it works and the time it is not available or it doesn't support what you need... You are lost.
While the command line and writting script shell and program compose well and you can always check back what the script/program is doing if needed.
Another huge producitivity boost is to ensure you have all theses programs/script and also all that knowledge correctly organized and available when you need it.
You are very right!
Maybe I will write a blog titled "Efficient developers version control all their stuff" :)
Thanks,
Dan
Next step: Remove the need to SSH into servers. ;)
Always be proficient in all of the base scripting languages available in your OSes in use. Only being able to automate something in 1 or two languages severely limits your automation capabilities. I have had to write "quicky" scripts in DOS CMD, Bash, KSH, SH, XML, XSL, Java and C. Being able to choose the one that will give the most bang for the least time is extremely freeing because if you don't have your favorite available on the OS you'll be doomed to using something you're not very good at and taking a lot longer at the task. Guru level in everything is not a requirement, but knowing the basics of the options so you can sit down and type away for 10 minutes without having to lookup syntax every 2 minutes can absolutely save your entire day.
I have a number of DOS cmd and Bash files that fire off C and Java programs to chain to a result. Some are connected to hotkeys because the task is so frequent. I've been doing this for 30 years and the only thing that has changed is the mixture of tools that I use for automation. Perl, Ruby and Python can be wonderful... when available. Remember to not bloat an OS installation just because you want a certain tool/language unless absolutely needed.
Free yourselves from being a monkey!!
Love it. Thanks a lot for the comment.
I agree with a caveat
I would pause and reflect whether you should actually Use Bash?.
I have a rule that if it can't be done trivially done with Bash, then it shouldn't be done with Bash.
Leverage $favoriteProgrammingLanguage instead
That is also my preference: "For quick and easy things, ... Bash".
Of course you run into the risk that the "quick and easy" starts to become a "complex" and you have to rewrite it to a "proper" language, but in theory your bash script is still very small and simple at that point.
In any case, I always recommend to learn Bash as it is the only thing that is installed in most servers, and as a backend developer, I find it irreplaceable.
Thanks for the comment!