DevOps Concepts

DevOps is an IT mindset that encourages communication, collaboration, integration, and automation among software developers and IT operations in order to improve the speed and quality of delivering software.

Tools Used in this methodology:

1. Source code repository - Eg. Git, Subversion, Cloudforce, Bitbucket, and TFS. Here, we will be mostly discussing about Git.
2. Build server - Eg. Jenkins, SonarQube, and Artifactory.
3. Configuration management - Eg. Puppet and Chef.
4. Virtual infrastructure - Eg. Amazon Web Services and Microsoft Azure. Private virtual clouds can also be considered here.
5. Test automation - Eg. Selenium and Water.
6. Pipeline orchestration - Kubernates, Openshift, Docker Swarm, Google Container Engine, Google Cloud Run, AWS Elastic Kubernetes Service (EKS), Amazon EC2 Container Service (ECS), Azure Container Instances, etc.

Comments

[Ahmed A. Elkhalifa]

Since I've joined my company as a DevOps Engineer I've started working on implementing CI/CD Pipelines as fast as we could, because our team was having a hard time publishing updates and patches. We are using ASP.NET Core and the old process of publishing updates was as follows:

1. Build the app locally and do dotnet publish to a folder
2. Transfare the files to the remote server over FTP
3. Restart the apache server over SSH

This process was painful and took a lot of time, not to mention how tedious it was.
Now the new process is much simpler (from the developer experience): developers will only have to push their code to the branch that corresponds with the desired environment (testing, staging, production) and the CI/CD Pipeline will take care of the rest!

Later on we implemented containerization and kubernetes and that came with a whole set of pros.

[Brian Richardson]

DevOps is a critical discipline as applications grow in complexity. Distributed systems have multiple components in the deployment with different targets. Here are the main benefits I see to our organization after migrating to Azure DevOps from doing manual deployments:

1. Improved velocity of releases: As the application grew and became distributed, the sheer number of components * server count to be deployed became prohibitive of a manual deployment.
2. Vastly improved visibility and audit trail: Workflows enforce policies. A good DevOps practice will prevent direct merges to production branches, require reviews and approvals. This allows you to find the unicorn that is "developers never touch production".
3. More time for QA: a consequence of 1. If it takes less time to review and deploy test branches, QA has more time.

How did we get there? We already had the code in Git, so it was a matter of creating a build server and a build pipeline and a release pipeline. Everything starts humbly. As time went on, the server count grew and application count grew, and the number of pipelines increased. More experience with the platform led to further efficiencies in terms of running additional parallel tasks and so forth.

The need to deploy the same code to production and test requires some kind of configuration management. Our approach was surprisingly simple to achieve without any additional tools. Our deployment application configuration files are tokenized, then the "Tokenize Files in Archive" job is used to modify the configuration prior to deployment. Azure DevOps provides Libraries of configuration variables that can be assigned to individual pipelines/stages.

The number of servers required by our application makes physical servers prohibitive. There's room for a number of racks, and I think that half of them would be filled by blades for this application alone. So, the solution is a cloud provider like Azure. I've used smaller ones like Digital Ocean too with some success.

The above was the first year or so: ditch the private cloud server, and move to the public cloud, and distribute the load between multiple servers. While this involved code changes, this was the first actual need for an automated deployment. And, necessity being the mother of invention...

As more time and resources could be dedicated to QA, test automation began. Now, our application is tested by a number of Selenium tests, as well as basic unit tests. Test automation relieves QA of tedious and numerous regression tests, and ensures consistent execution.

The final stage is not yet complete. We are going to automate the infrastructure now. Instead of building multiple servers in advance, move to Kubernetes and configure the application for autoscaling. Azure provides Kubernetes service as well. Moving to containers has a number of other advantages, too:

Easier testing, as multiple jobs can be run simultaneously. More reliable testing, as the environment is distributed with the code. Most notably, more efficient use of resources as Kubernetes support microCPU-scale resource requests.

In short, DevOps is a critical component in a modern cloud architecture based on microservices.