Journey Through DevOps - Part 2: The Awakening

This is 3 part series documenting my journey through Software development. For Journey Through DevOps - Part 1: World without DevOps, click here

After a turbulent initiation into software development, one thing was certain, I was a total novice and had to up my game. It was at this junction that I'd taken my first Cloud computing course, AWS Fundamentals: Going Cloud-Native. Now, this course taught me multiple concepts such as Failover, Availability zones and Load Balancers.

But there's were a few issues. As the complexity of the infrastructure grew, with it grew:

1. Necessity to manage the infrastructure.
2. Need for developing a software development methodology that would allow changes in applications to be easily integrated with existing infrastructure.

The former issue was reflected in the event where I had lost track of an AWS Global Accelerator(networking service that improves the performance of your users’ traffic) and it went on to cost us credits for a few months. I had manually search for the resource across all regions, which is less than ideal. The latter issue was reflected when there were multiple Natural Language Processing models and they had to be manually updated in multiple instances.

Enter DevOps

My dad is a software engineer so I'd always hear words and technologies that they were using in their workflow. And upon looking up said words, I'd find out they solve a problem that I am facing in my workflow. Although explaining DevOps is something that beyond the scope of both me and a blogpost. These are the tools on my journey that have enabled me to solve problems and implement practices of DevOps.

CI/CD(Problem 2)

Jenkins

It's a versatile tool to execute both Continuous integration and Continuous Delivery(CI/CD). Jenkins is a server based tool which can define CI/CD processes as Pipeline as a Code. So that means there's an extra overhead in installing Jenkins master node and agent nodes. So for a small scale project as ours, its versatility was the bane.

We needed something lightweight.

GitHub Actions

GitHub Actions is a relatively new feature in GitHub where we can declare build and test processes in GitHub as YAML files. These are then executed either on GitHub runners(basically virtual machines hosted by GitHub) or self hosted runners. These are a game changer because they allow many simple processes to be executed simply by pushing code or creating a pull request.

However, they only solve half the puzzle. GitHub actions are extremely easy to incorporate with CI processes however, the CD part remains a problem. Apart from basic support to Kubernetes clusters(Only cloud provided) there are very few CD options.

However until our second release, GitHub actions sufficed, so we used them.

Hashicorp(Problem 1)

After our first project, it was apparent we were scaling up infrastructure wise. Enter, Terraform

Terraform

My love affair with Hashicorp began with Terraform. In all honesty there may not have been that desperate a need for it, I used it because it seemed cool, although it is impossible to separate Terraform from my current role.

To check out more about Terraform and its uses checkout my post on medium.

My misconception about Terraform was that it was to deploy infrastructure like we would to an application. This would mean I write code for the infrastructure needed and I would deploy it on Day 0, and then install my application on relevant resources. However, that turned out to not be feasible. Terraform is a tool that is a part of your daily workflow. Incremental changes to infrastructure are logged and created by Terraform allowing you to track and visualise infrastructure you are using and how it fits together, like a jigsaw puzzle.

Although there are tools such as Pulumi which offer IaC(Infrastructure as Code) in programming languages like Python or Node, I prefer Terraform because of it's simplicity and ease of use.

Conclusion

Granted these were major leaps from the previous experience(i.e described in previous post), the point of writing these posts to hopefully let people know that what you build at first will be horrible and you will cringe at it when you look back. However as one of my favourite movies said about cooking, we can safely bet "Not everyone can be a great developer, but a great developer can come from anywhere."