Monthly Insights - Automation, Ambiguity and Agile

Automation

Everything boring that can be automated, should probably be automated. Whether others know about that automation, depends upon how much it is valued over looking busy.

Image by magnific

I've been on a self undertaken journey at work for the past couple of months - the automation of our build process.

I learnt a lot about how Jenkins works, how interactions happen between GitHub, Jenkins, Artifactory, Docker, Ansible, etc. I started slow - one build pipeline that creates and pushes Docker images, and I kept adding pipelines as I felt the need.

Today, I have a suite of pipelines that run tests, code coverage, build, deploy, cleanup, and run security scans across x86 and s390x. Some highlights of this suite -

1. A multi architecture build - UI built on an x86 agent and build folder sent over to an s390x agent. This agent then builds the backend and the final image
2. An end-to-end .jar updater - Separate java repository whose .jar files were imported into the main repository to be called. The pipeline built these .jars and automatically created a PR on GitHub.

This has freed up a lot of dev hours for my team and myself. It's also helped keep the systems (and me) sane with the insane amount of work that gets done nowadays.

I keep looking for things I can automate now, especially the small, mundane tasks since the time saved really does compound up. To anyone reading this, or future me - "Automation is like getting regular exercise; you might not see immediate results, but your systems will thank you later."

Ambiguity

The biggest blocker of them all is often the difference in understanding of the same words

Image by starline on Magnific
A couple of years ago, when I just started working as a software engineer, I struggled with ambiguity. Before this, the requirements were straightforward assignments with most of them written down.

Now, I hold the opinion that dealing with ambiguity and sifting through it is a large part of my job. There's multiple stakeholders, ranging across the technical to non technical scale - Product, UX, leads, CS; all with a range of opinions and areas of expertise.

Especially today with how coding assistants work, I believe its imperative to identify and resolve ambiguities even before the first prompt is entered. I've personally found far greater success in reducing the amount of code generated (noise) this way.

Every large story I pick up always starts the same way now -

• Look at the summary, acceptance criteria and description and note down the questions that come to my mind while reading these.
• Figure out who are the best people to ask these questions, and reach out to them.
• Edit the ticket itself to reflect the answers I get.
• Only after steps 1-3 do I start making any changes.

Agile

Agile as a theory stands tall, but its difficult to judge a theory in isolation of its practitioners

Image by redgreystock on Magnific

Note - I have only experienced Scrum, maybe other types of Agile might solves some of these or maybe they don't, I do not know yet.

Agile was introduced as a solution to the problem that was waterfall. The intent was to move fast and break things. Agility is required, especially in today's fast moving world, and the theory behind Agile has always promoted empowering teams to function independently. In practice, however I have some issues with it.

1) The estimation problem - Story points: "Assign a random number to a task that you or someone else has never done before so we can know how difficult it is and how much effort it would take?". For a given ticket, how are the story points the same for a person who has worked previously on similar problems vs someone who hasn't? How is a junior expected to work with 2 story points for the same task when he/she has never seen that codebase before? How are we fitting story points into a unit of time (sprints), yet claiming that they are unrelated to the time required to complete a task?

2) The responsibility problem - Scope creep: Agile magically assumes that everyone in the team is magically responsible. What happens when the people say don't want to write documentation or more often, don't have the time to? What if people don't say no to scope increases and sacrifice quality for it? You end up adding features without any spec, scope creeps upwards and timelines shrink, further increasing the mess. A couple of years later, nobody knows why certain decisions were taken.

3) If the whole point is Agility, how do strict cadenced release cycles play into this? Isn't this exactly what we are avoiding?

4) Meetings: Need I say more?

To sum up, I believe most of the problems with Agile today lie in its monstrous interpretations that we have implemented. Maybe there will be a new theory that becomes the industry standard that eases the problems that occur.

About this series

Monthly learnings will hopefully turn out to be a journal of my life in the software engineer journey. I started this series when I was a full time SWE for 2 years, rather than as a fresh grad because it felt more appropriate. It will help future me - to look back at when I need to as a record of how far I've come and as a refresher. Some of these will be technical, some non-technical and some of these ramblings might not be related to the job at all.