A DevOps Primer

#devops

DevOps has been around with us for a pretty long time now, and while a lot of companies, organizations, and teams have made efforts to move their release model towards DevOps or its closely related cousin, Agile, there are still many who are firmly entrenched in the old cycle of "Have the programmers write the software, send it to QA for testing, and then send it to operations for maintaining it after its release" and that's the end of the road for that application.

In particular, this model makes it difficult to ship updates because that process is treated the same as shipping a brand new software, without taking into consideration the quirks of the old software. That's why you sometimes see programs performing worse after major updates because development "copied and pasted" the update onto the program and operations don't get the opportunity to upgrade their infrastructure to handle the update. On the contrary, many times they are stuck with the same infrastructure they used for the initial release even if the update is fairly major.

What I hope to accomplish in this series is show how you can enforce shared responsibilities on the development and the operations team so that they work together to maintain the software after it ships.

DevOps trivia

A few important terms will be defined here.

Development consists of the interrelated disciplines of programming the software, testing for bugs, and running quality assurance (QA) tests that the solution delivers the expected amount of value and performance to end-users. It also includes creating bug fixes and feature updates for it post-deployment.
Operations consists of the disciplines of deploying the software and updates to it, maintaining its reliability during its lifetime, rolling back to a previous build of the deployment if necessary, and the creation of scripts to aid with these processes (Infrastructure as Code).
Agile is a methodology that unifies the programmers, testers, and QA staff into one team. Traditionally each of these belongs to different departments, and combining them all into one brings many benefits including less managerial overhead such that communication and cooperation between programmers and testers become smoother. This topic will be covered in a future post.
DevOps, this post's topic, unifies the shared development department with operations. You will find that forming a DevOps team is much easier if you form the Agile team from the development department beforehand.

The five stages of software engineering

The lifecycle of an application consists of five stages, three of which are relevant to development and two to operations. Most importantly some stages are shared by both development and operations, and there's one stage that is not the responsibility of either of those departments, which is the planning stage.

Idea: All software projects start with a problem to solve. I put this as number 0 because in a company this step isn't done by the development of operations departments. The idea is usually made by upper management, possibly as a result of market trends or their past revenue from other ventures.
Planning: The idea is handed to the R&D (Research and Development - not as in programming) department who then researches the idea to assess if it is technically viable. The idea is also handed to marketing to assess if it's financially viable for the company to pursue.
Specification: If both R&D and Marketing both conclude that the idea is viable then Project Managers are given the assignment to construct a specification out of it with detailed requirements, taking input and insights from these two departments.
- We don't task development with an engineering task without a specification because sometimes unexpected behavior is coded into the software as a result of requirements being fuzzy.
Development: The specification is then handed to Development to program software out of it, and to test it after its creation. They can consult Project Management for clarification about any unclear requirements. It is the stage that is streamlined by Agile.
Deployment: In this step, Operations deploys the software and contacts Development staff for guidance if necessary. Sales and Marketing kick in to advertise the software to customers.
Maintenance: Operations ensures that the software runs without failures and contacts Development for resolution should any bugs or performance issues are found post-deployment.

Below is a diagram of the lifecycle of software. Notice how Operations and Development are closely interconnected in stages 4 and 5 - Along with stage 3, these are the stages that can be optimized by following a DevOps methodology.

Diagram of software lifecycles

Why do we need DevOps?

DevOps solves the problem of development and operations conflicting with each other about key issues. Suppose a production application fails, then operations will blame development for overlooking the fault that caused the failure, while development could blame operations for not controlling the fault so that it wouldn't cause a failure. All this is both toxic and prevents or delays the actual issue, the fault, from being solved.

In the DevOps model, operations works with development to provision the hardware and system environment necessary to run the application. The application dependencies are also deployed instantaneously, eliminating the possibility of a mismatch of running environments of testing and production causing faults to be missed.

Whereas before, operations accommodated their OS configurations of the production servers to run applications from several different projects, so-called horizontal scaling, DevOps makes them dynamically create several different OS configurations, one for each application. In 2021 it's not difficult to see that this can be accomplished using containers or virtual machines. This isolation of the operating environments of each application brings tremendous benefit when finding the causes of faults; irrelevant configurations aren't intertwined with the problem application and can be ruled out, causing issues to be resolved faster.

Core facets of DevOps

Below is a partial list of the most important aspects of a DevOps project.

Deployment scripts, build scripts and unit testing are fully automated.
Issue trackers are connected to both development and operation workstations. Each of them can see the bug reports related to the software.
Both the source code and Infrastructure as Code scripts and input files with their parameters are stored inside version control. This allows development and operations to create heterogeneous environments suitable for the task they each are trying to accomplish. Development for example may want a debug environment, while Operations may need to create a release environment.
There are metrics defined by the team which measure the quality of the software, and
There are specific criteria in place which must be fulfilled to ship a release or a release candidate. A release is never shipped on the consensus that the software "might work".