I was recently tasked with abstracting a database into an API. The database had a lot of confusing data formatting functions which should also be abstracted in my API. Since it was a generic market system, the documentation for the database was available through a humongous Excel file explaining how each field should receive data.
Initially, I worked with the tools from my text editor to have the maximum productivity I could get, but apparently, there was no end to these database tables. Each table had around 300 fields, and I needed to create files for data formatting, payload validation and unit tests.
After a week, I was far away from finishing my first task, demotivated and my productivity was way smaller than I'd like. All I did was simply bash my head against the keyboard calling formatting functions for each field in a table the whole day, and even after a good night's rest, every day became a huge fight against falling asleep in front of the computer from boredom.
It was right on a Monday that a question popped in my mind
Why should I type all this?
I started my "new" task the same day: creating someone to type all that for me.
I used one full day and a bit over 300 lines of code, and created a script that parsed the Excel file, filtered which was the table I was trying to abstract and generated the payload validation function. Then, all I had to do was to dump the generated file inside my original project and check to see if everything was working. The entire file generation process took around 350~400ms.
I was in love with my new baby, and a week later, it was already generating every file and function that needed any heavy typing. Everyone got something out of this adventure. I had the oportunity to work on an exciting task, and the task was finished way quicker than I ever thought possible.
I know this is a very isolated metaprogramming example, but my real goal with this article is to make you question yourself if you can't fit some of that in your current task. Do you feel bored with what you need to do? Are you tired of banging your keyboard mindlessly? So create "someone" to do it for you!
The best part of being a programmer is to see the possibility of improving a situation, and to have the knowledge to actually improve it, so embrace this power!
Okay, but what about this metaprogramming?
The "meta-" prefix in this case means "reflection upon itself". Metaprogramming is the programming you do make some actual programming task easier :)
This idea came from stretching out the concepts from Tim Ferriss' book The 4-Hour Workweek. The original concept is that we should automate any task that takes us too long, and there are quite a few reasons to actually do this.
First off, speed. A computer is much, much faster to execute repetitive tasks than any human. You can leave your computer working for hours or days and it will not feel tired, bored nor slow down.
Second off, precision. A computer doesn't write typos or reads a text from the wrong place. The computer will always do what it was programmed to do. This is a double edged sword, since you'll need to know what is it that you want to do and how the computer should do it. But once you've got your code straight, you can simply execute the code and go do something else entirely.
Last but not least, fun! Making a robot that automates tasks is challenging, and it is way more fun than typing repetitive stuff forever. Not to mention that it is super fun to see your script running and fixing a problem for you (boredom) and a problem for your boss (time).
Bill Gates once said that he prefers to hire lazy people to do hard tasks because they will always find simple solutions for those problems.
So here goes a controversial advice: Be lazier!
Not in the sense of slacking at your job, or in the sense of procrastinating the things you need to do. But in the way that makes you avoid repetitive work.
The biggest value a programmer brings to the table is exactly the ability to abstract complex tasks and automate repetitive tasks, and not our typing speed. It's part of the job to find those tasks and automate them.
Have your eyes sharp for any menial tasks that pop up and eliminate them. This will free you team's time to focus on what really matters, the complex problems.
Do you have some report that takes hours to set up? Automate it.
Do you have bureaucracy that hinders your work? Automate it.
Do you have a task that pops up every single month? Automate it!
Cover image: Foto de Drew Coffman na Unsplash
This article is also available in Portuguese here
Article originally published (in portuguese) at Holyscript
Top comments (21)
When I was 21 ( way back in the 1990s now ) I said to my boss at the time, "You know I'm just very, very lazy, why would I do stuff manually when i can code it up and get systems to work for themselves.".
She jokingly went nuts at me, "You are NOT lazy and you are never, ever to say or think that ever again! It's not lazy, it's working efficiently and more people need to be like you and get this kit to work harder and smarter.".
To this day 30 years later I never use the word lazy to describe smart corner-cutting or efficiency. She was right, 'cos "lazy" makes you think that you're not good enough, "effiency" means you're thinking about working smarter and making best use of your skills and time.
damn right.
That's the theoretical ideal of programming, popularized by the 1980s generation and maybe obsolete when machine learning will take that part of metaprogramming away from humans. Then we can go beyond metaprogramming and rediscover real creativity again!
I was in a similar situation yesterday (Jan 23), I wanted to contact some phone numbers through whatsapp for marketing purposes and I didn't want to populate my phone's contact book unless they showed interest.
So, I decided to make a link out of each number.
I did this by creating a new html file and wrapping some text in an
atag and setting the number as thehrefvalue.After doing the first one it struck me that I could automate this.
I thought of using JavaScript since I was fond of it but used Python instead thinking on the simplicity and filesystem access.
I wrote a python script not more than ten lines to automate the task for me. 💯
I didn't want to populate phone's contact book unless they showed interest.What does this means? Not yet work with WhatsApp marketing, but it seems to be one point that I must be aware of in future?!?
Would love if you could share some more information about this (e.g. some stuff to read or a link, or an explaination).
Thanks.
Great article!
This is something that every developer should do. This can save you tons of hours of work. It's a skill and also is really fun. You can use these automations to learn something new too. For instance, if you're trying to learn a new programming language, you can use that new language to create an automation script. The script doesn't need to be the most optmized or clean, just need to save you some time.
Again, that was a great post and can inspire lots of developers to do the same!
Lazy developers found the better DX solutions, totally agree!
I have this age old saying: "A good developer will go to extreme lengths to be lazy".
As described in the post, you are better off automating boring tasks by building some temporary intermediate tool than by doing the monkey work. So the "go to extreme lengths" here is not trying to get away and not do the work, but to be passionate and innovative to build something so the work gets done for you.
If you consequently think about this deeper and deeper you will arrive at a system which had have automated your entire own life for you ... no more Sudoku, no more crossword puzzles, no more cooking, ...
And yes ... it's usual everyday business to create a programming environment and then create another one and then port the programming done in the first to the second one. This will keep you busy ... creating another meta-language to solve the problem of coping with multiple languages you won't run into if not creating the issue in first place. The only way out of this trap I see is to make the English language work as programming language ... I call it the "oOo way" of approaching computing.
There is definitely a limit to automation, you should not fly too close to the sun, so to speak.
The main focus of automation is to free your time that's taken up from repetitive tasks so that you can do stuff that you actually enjoy
That exactly the point. There is also a trap of over-engineering.
One kind of over-engineering is to put hours of work into a task that don't repeat and would be solved in same (or less) time.
Also seen this kind of contra-productive automation some times :)
So be careful ;)
Great post ! I am actively trying to be lazier
Thanks! Automation will give you time to laze around a lot :D
Nope, During that automation you will get another task :D
Used to do this to build a "data dictionary" from SQL Server. There is a hidden or little-known "gem" called "Extended Properties". You can set them at the Database, Table, Column, View, Procedure levels (almost anywhere). So you can create a self-documenting database. This can then be exported via a script into Excel or you can use T4 Templates or something to generate anything you want (classes with comments on properties auto-generated for example).
Good article
Thank you!
Thanks for the read I enjoyed it. Now my head is filling up with all types of ideas on how to automate some of my work.
Thanks again!!!
Please feel free to share your automation stories! I'd love to hear how I was able to inspire you :)
Honestly a great read, throughly enjoyed; Got the gears in my head turning now!⚙️🤯 I want to hear the next thing somebody decides to automate!
I'm really glad you enjoyed it! Feel free to share your automation stories as well! :D
Some comments may only be visible to logged-in visitors. Sign in to view all comments.