How to create an automated code cleaner with pre commit hook

We had a repo with code, some parts of which were not formatted properly. So this is the story of how we kick started our mission to clean-up our codebase, eventually, using git pre-commit hook. Maybe we won’t need these anymore (as nowadays, IDEs have these in-built) but my motto is to demonstrate how we can automate & improve our day to day development process, with no to very little efforts.

So the other day, I was going through the pull request of my teammate and I noticed that several minor code formatting have been overlooked. Those very ignorable (as per normal human being) so I felt a bit kinda stupid of myself to explain the logic behind my thoughts and comment to fix them. I wish we had a Development Style Guide defined and mutually agreed upon also but unfortunately, we didn’t. Something like this https://thermondo.github.io/style-guide/

So I thought why don’t we automate this whole code formatting process. So for that we need to agree upon a code style format and figure out a way to format in common way.

Lucky for Python we have a very well-defined PEP-8 https://www.python.org/dev/peps/pep-0008/ which I really miss in few other languages. So with this our first checklist was DONE.

Secondly, we had to figure out a common way to format our code. We discovered this jewel, black - https://github.com/psf/black.

Now the human behaviour come into play, we had everything in place. A style guide and a formatter but sometimes we were not conscious enough to format our code before committing. This part was not in the initial plan, typical example of - why software projects are not delivered on time. Surprises everywhere!
So we took this as a challenge and thought about automating this part too. Maybe integrate it with something that we won’t miss.

All of us had one thing in common and we can’t miss that - Committing. We were using git as our version control system thus all of us will certainly use git commit. How can we integrate this to commits itself - git pre-commit hook came to our rescue.

We designed this tiny very simply pre-commit hook to format and verify our code before committing it.

for file in $(git diff --cached --name-only | grep -E '\.(py)$')
do
  black "$file"
  $(git add "$file")
done

for file in $(git diff --cached --name-only | grep -E '\.(py)$')
do
  flake8 "$file"
  if [ $? -ne 0 ]; then
    echo "flake8 failed on staged file '$file'. Please check your code and try again. To check output manually run flake8 $file"
    exit 1 # exit with failure status
  fi
Done

So with this no manual intervention was required. We only needed to committ as usual and it will auto-format the code and verify it - sometimes black fails or something goes wrong, so it’ll fail in verification stage and we’ll need to manually fix it.

So just for the sake of clarity, in pre-commit hook we’re iterating through the files staged to be committed and black(ening, auto-formatting them) and then again verifying them same way - exit with non-error code just to indicate error occurred. We need to place this in our repo at .git/hooks/pre-commit For this hook to work we need to have black and flake8 pre-installed in our system.

Learning:

1. We started to not worry about formatting our code.
2. We didn’t need to worry about missing a line or two.
3. WE WERE CLEANING OUR REPO - best part was unconsciously. If we opened a file after a while, we felt like a magic, when did we format this file ? Awesome!
4. We were not bloating the commits with irrelevant changes - by only formatting those files which we were committing anyways. Passive code clean-up drive.
5. Lots of developers time were saved, arguing whether & why to have curly braces at the end of line or in the next line.

References:

• We discussed this pre-commit hook :

  https://gist.github.com/umr55766/ece111837cfb468211ccacb9e65fb5be
  

• We used this : The uncompromising Python code formatter https://black.readthedocs.io/en/stable/
• Tool For Style Guide Enforcement : https://github.com/pycqa/flake8