The great thing about the definition of DevOps is: there are so many to choose from. Many articles, books, and courses have been written about our Industry's latest favorite buzzword. Websites, conferences, and entire companies have been founded around it.
We have thought leaders, practitioners, and people trying to catch on to the wave but the one thing we don't have is a clear definition of what it is. This article should give you a better idea.
This is the first in a series of articles explaining DevOps practices, and we'll start at a very high level and dig down from there.
So What is DevOps Anyway?
DevOps is not software, or just a methodology, it's a culture. You can't buy it or download it, but you can learn it. Most definitions start with the idea that Operations and Development must work together and throw in automation for good measure. But it's much more that that.
DevOps is NOT:
- Merely automating everything
- Moving to containers / microservices
- Just continuous integration
There are a lot of myths and misconceptions surrounding this. You might here things like this:
"Throw some Kubernetes in there so we can be DevOps!"
You can't just implement containers, some automation, and CI/CD and say "we are DevOps now". These things enable DevOps but it's more about the people than the technology.
"Developers are now Operations people, and vice versa"
While this is true, it's usually exaggerated. There is more overlap between these groups than before, but roles are still clearly defined. Just telling your engineers they have to provision their own machines now isn't adopting DevOps.
DevOps is just releasing things every hour like big companies do.
While this may be a goal, it's just a small part of the overall intention of DevOps culture. Releasing bad code every hour isn't an improvement.
DevOps is:
A mindset and methodology that:
- Fosters communication and collaboration
- Focuses on rapid IT delivery
- Leverages tools to move quickly
DevOps is focused on shortening the Software Development Lifecycle (SDLC) using feedback loops to develop features and fix bugs at an increase rate.
A large part of DevOps is focusing on smaller, more frequent releases. In the past we released new versions of software over months or years, we now do it in days or hours.
I’ll focus on the concept of batch size and WIP because they're very important to this process.
The overall goal of DevOps is to increase speed and quality without burning out your engineers.
The Main Principles of DevOps
The main principles of DevOps are outlined clearly in the three ways of DevOps.
Way #1: Systems Thinking
The first way is systems thinking. Rather than focusing on a fast IT department or fast developer group, they both worth together as a system. Their goal is to go from idea to feature as quickly as possible. This is your value stream. The idea behind it is simple: once it's decided a feature needs to be made, developers create it and IT deploys it as quickly as humanly possible.
Automation plays a big part here because it enables you to push code through the system as quickly as possible.
When successful you're committing code, that's automatically pushed, tested, and integrated to the customer quickly and efficiently.
Way #2: Amplify Feedback Loops
The 2nd way has to do with creating a feedback loop. Developers are pushing code to the Ops team, and the Ops team is giving feedback about that software. Then developers change or improve the software based on that feedback. It continues in a loop.
Some examples of IT feedback:
- Speed
- Functionality
- Test Results
Way #3: Culture of Continual Experimentation and Learning
The third way builds on that feedback loop and creates more self reinforcing feedback loops. It encourages continual experimentation in short iterations. Learning from mistakes and benefiting from improvements.
The experimentation component here is huge. DevOps is a learning process. It's about coming up with ideas, implementing them, and learning from them as quickly as possible. You can't do that on six-month release cycles. You can't do that if you're burning time moving software from place to place.
DevOps creates a culture of learning that works in small, efficient workloads and using automation to free up people for meaningful work.
So let's break down some problems and see why we bother with DevOps in the first place.
What Problems Does DevOps Solve?
I've been in IT and Development for nearly 2 decades. To be more precise, I'm "I remember when IT was a cost center" years old. I still remember the first time I heard the term "DevOps" around 2011. One of my enthusiastic tech friends started ranting on about how IT operations and Developers would have to merge their duties and it was going to change the world.
My first thought was "SWEET! I love IT operations and I'm a developer. I get to provision machines now? awesome". No sarcasm intended, I've always loved the IT side of things, years after leaving it. The idea that I would get to do IT stuff as a developer was appealing. That meant low resistance for me personally, and likely why I pursued it so hard.
I now have a better understanding of DevOps than I did then, and I still don't feel like an expert. It's ever evolving and changing by nature. But I have seen a lot of changes come directly from adopting a DevOps mindset.
One of the greatest intentions of DevOps is removing bottlenecks
There have always been bottlenecks in development and there always will be. It may be a person, it may be a technology, it may be bureaucracy. DevOps cultures try to minimize all bottlenecks if possible.
1. Release Cadence
How many times have you heard this phrase?
We must wait until the release goes out to focus on that
Having large batch sizes and huge manual deployments means each software release is a huge event. Bugs and features have been carefully triaged by the team and we've decided what will be included in the next release. Everything else is sidelined.
That's fine unless something crucial comes up in the six months you're spending developing the new software. Security and compliant issues come to mind here. If you have a big security flaw, you have to build a patch and get it fixed now, not months from now. So being a good engineer you find a way to work it into the software for now.
As more of these pile up you watch your deadlines get pushed further out while you work on hot fixes and implementations, at the same including those fixes into the new software.
How DevOps solves this problem: by making software releases a less remarkable event. We aren't bringing pizzas and a cake to the team for a software deployment anymore. It's happening every week, every day, or even every hour.
By breaking the work into smaller chunks and continually integrating you no longer have to wait months to implement changes.
2. The Overworked Engineer
How about this phrase?
We must wait for Jane to fix that. It will be awhile, she has a lot on her plate right now.
If you've worked in this industry longer than a week, you know this engineer. You've probably been this engineer. The person who knows so much about one particular area or technology that they take ownership of it completely.
They're smart, driven, and crucial to the success of the project, maybe even the whole company. They might be an infrastructure engineer, a developer, or any number of roles and act as a "gatekeeper" for software.
This type of bottleneck creates a host of problems for your team. Software can't move quickly without their intervention. More importantly, they're working too hard. They're sacrificing personal time with family or friends, and constantly in a state of pressure. They're vulnerable to health issues, or leaving the organization.
How DevOps solves this problem: By breaking down silos and spreading responsibility among more members of the team. By breaking work down into smaller tasks to improve flow. This relieves the pressure on any one individual to enable work flow.
3. Unplanned Infrastructure Requests
I need a server for this, and we can't move forward until we get it
Early in my career I did provisioning. This was my usual process:
- Gather requirements
- Design a server
- Order the parts (if not on hand)
- Assemble the machine
- Put an Operating System on it
- Configure Software
- Find space on the rack
- Plug it in
- Establish access for the person needing it
I did this over and over for people day after day. Don't get me wrong, I loved it but I'm also glad this isn't how we do things anymore. This stuff took time, and I can remember being able to do about 5 machines concurrently before I started mixing things up and missing details.
Last year, my process for getting a server was much different.
- Gather requirements
- Load up Jenkins, select a profile
- Throw some parameters in if necessary
- Click a button
- Get IP address of server
The process wasn't much different if I had to log into Azure and deploy one there.
Guess which one of these takes longer? Now this problem is being solved by virtualization and cloud platforms, but DevOps is a big driver of it.
How DevOps Solves This Problem: DevOps culture pushes the idea of automating everything, including server configuration or container deployment. We used to name our servers like pets, now we number them like cattle. They should be instances of a common design rather than a special entity.
4. Testing Phase Slowdowns
We won't have the fix out for another week or two, it's still going through testing.
Testing and QA used to be an insanely bad process. I can remember developing software that didn't see the light of day for months. This was in part to the old school waterfall cadence of releasing large chunks of software at a time, and partially due to testing and how long it took.
It wasn't the fault of the testers. We dumped piles of software on them at once with a big list of features and bugs we fixed... probably. The iterations were so large that by the time a tester had found a bug we'd already piled a bunch of software on top of it. So the regressions were widespread by then.
How DevOps Solves This Problem: Two crucial changes here:
Batch size: With smaller batches of work you have a lower risk, and a single small feature or bug to check. Testing completes much faster
Automation: With DevOps, so much of testing is automated now that you can write code, commit it and watch it go through the testing processes. People testing are not obsolete, however as they are building the test automation and spot checking things as they go.
DevOps frees testers to be more productive.
Big Picture
DevOps can be a huge cultural shift for your organization. However, it's crucial in this day and age.
- Companies are innovating faster than ever before
- The technical barrier to entry is lower than it's ever been
- You can improve speed without sacrificing quality
These are the reasons everyone is talking about DevOps. This is why people are scrambling to implement new techniques.
This is the first of a series where I will explain DevOps principles, and how you can apply them. We'll explore some use cases and practices to help you as a developer, operations person, or manager to get the most out of this cultural movement.
If you want to learn more about DevOps in the meantime,
- Read The DevOps Handbook
- Check out Pluralsight's DevOps Course Path
- Follow me on Dev.To to be notified of future articles
NoteL In December I'll be doing a Webinar titled "Enabling Low Risk Releases" where I'll talk about how you can use DevOps principles to improve your release velocity. Sign up here if you want to join me!
Top comments (4)
Pretty informative and detailed post on DevOps, loved your writing style. Simple yet effective.
Looking forward to more such articles from you. Thank you for sharing.
Awesome post!
Also, some helpful resources as well:
Hidden Benefits of Automated Deployment You Should Know About
How Deployment Tools Save Time in Software Development
Not completely relevant to this post, but the major challenge is that your client/consumer of the software should also be ready for shorter release cycle. I have worked for financial software products for over 12 years and I always faced resistance for accepting frequent releases (unless it is a production bug)
Thank you for sharing useful information! In my opinion, this article romexsoft.com/blog/what-is-aws-dev... will complement your post