The DevOps function in an organization is responsible for deploying, securing and maintaining applications in the cloud. Let’s dissect the DevOps costs and the factors that influence them to understand the question, “Just how much should I spend on DevOps?” We also look at approaches the organizations have taken to reduce the cost.
Cost Categories: People and Tools
People and tools are the two cost categories in the DevOps budget. Our survey of 500 customers tells us that in most companies people account for 80% of the cost versus 20% spent on DevOps tools. One DevOps engineer is needed to operate an average of 50 virtual machines or an application stack of 10 microservices. The reason for this lopsided split is that DevOps is still largely DIY (Do-It-Yourself), i.e. DevOps engineers spend extensive staff hours writing scripts tying various point solutions and configurations together. This is rather ironic given that DevOps automation is probably the most crowded among the cloud software categories.
People
An average DevOps engineer in the United States costs in the range of $180K to $250K a year. The DevOps profile requires the same individual to have an expertise in operations and security, as well as programming (to write IaC). These three skills have never traditionally been unified in a single role. From college education to certifications to job profiles these have always been three independent silos. Now we are in a state where the current industry demand for DevOps engineers outstrips supply.
Tools
DevOps tools can be categorized into Automation, Security and Developer Productivity. In an early stage company the cost of these tools relative to the people’s cost is rather small. In fact mainstream automation tools like Terraform and AWS CloudFormation are free. Foundational security software like AWS Security groups, AWS CloudTrail, AWS Security Hub, AWS WAF, Amazon GuardDuty, and AWS KMS are either free or have minimal cost. Examples of software that improve developer productivity are code editors, CI/CD and observability tools. CI/CD tools like GitHub Actions, Jenkins, AWS CodeBuild, and others also have minimal cost. Observability tools like Datadog, Splunk, and New Relic can get expensive but then there are open source alternatives like Elasticsearch, Grafana and Prometheus. Larger organizations have larger budgets for tooling, especially security, but the cost of their DevOps and IT workforce is proportionately higher. As a rule of thumb, the cost of DevOps tools should not exceed 10-15% of the cloud infrastructure spend and would map to about 20% of the overall DevOps budget.
Factors that Influence DevOps Spend
The complexity of the DevOps function can vary significantly from one organization to another. Following are the key factors that influence how much one would spend in DevOps:
Compliance Standard
Operating in non-regulated industries is far easier as one can skip the work involved to meet the compliance requirements. Some standards are also easier than the others. SOC 2 and ISO are the easiest, while PCI-DSS and HIPAA are much harder. If you are required to have PCI-DSS or HIPAA prepare to have additional headcount focussed on security and compliance which are separate roles under the DevOps category and often called SecOps. If you are selling to the government and need to abide by FedRAMP, that is a multi-year process and would require you to even engage third parties with the cost running into several hundred thousand dollars.
Microservices and Kubernetes
Most modern application stacks today are adopting Microservices as there are clear advantages to the quality of the product. But that puts an additional burden on the DevOps team. Kubernetes, the industry’s de facto container orchestration tool, is a management beast by itself. Kubernetes experts are yet another niche within DevOps. Prepare to pay an additional 30% to the DevOps engineers if you have an application stack that is heavy on Kubernetes and microservices.
Developer Self-Service
While in most organizations Developers have little to no access to cloud infrastructure, some want to operate at great agility and hence require their Developers to have the maximum extent of self-service possible without compromising the security and other operational best practices. This requires fast response time to their change requests and can be achieved either by having a higher degree of automation or simply by having more DevOps engineers to serve the developer needs. Higher degrees of automation mean more seasoned (read expensive) expert DevOps engineers. In fact, DevOps engineers who can build an automation “platform” are called infrastructure engineers. We have seen them only in large modern organizations flush with cash like Uber, Intuit, Airbnb, LinkedIn and others. In most small organizations developers wait on DevOps engineers to run scripts for infrastructure change requests.
Infrastructure Size
While automation is the talk of the town, DevOps workloads never seem to decrease. The size of the DevOps team largely grows linearly with the size of the infrastructure and is a simple measure of total virtual machines and/or the count of microservices.
Approaches to Reducing DevOps Spend
There have been two main approaches organizations have taken:
External PaaS
Smaller organizations have deployed their application on Platform-as-a-Service (PaaS) solutions which have substantially reduced or almost eliminated DevOps efforts. Heroku is probably the most popular PaaS. The key limitation of this approach is that users don’t have access to native cloud services like S3, SQS, Kubernetes etc and are limited to the abstractions exposed by the PaaS. These work only for very small organizations and most of them migrate out of these PaaS to run directly on public cloud by hiring Devops engineers.
Build Internal Developer Platform (IDP)
Very large organizations have dedicated an experienced team of engineers often called the Platform team who over multiple years custom build and maintain an inhouse automation platform aimed at developer self-service. Uber, Netflix, Spotify are examples of a few companies that built this. The main disadvantage of this approach is this investment payoff only if the infrastructure is really big in organizations that generate hundreds of millions of dollars in revenue to be able to justify this investment.
At DuploCloud we have built a general purpose automation platform that can be used across the board as a Desecops-as-a-service platform in SMBs and an IDP for larger organizations. We are a team of engineers who were the original inventors of public cloud or Infrastructure-as-a-service at Azure and AWS. We repurposed that vast experience and expertise to build DuploCloud.
Conclusion
DevOps is a niche skill set that is hard to find and expensive to hire. Automation is still in its infancy and is largely about engineers taking a DIY approach to stitch together point solutions. About 80% of the DevOps cost is people cost and 20% is tools. The rule of thumb is for every 50 virtual machines or 10 microservices, an organization would need one DevOps engineer. The cost of tools is about 10% of overall cloud infrastructure spend. If you need to operate in compliance with standards like HIPAA, PCI-DSS or HITRUST, one needs to staff additional engineers focused on security operations.
We anticipate the DevOps automation space to evolve from point solutions to an end-to-end DevOps-as-a-Service platform that eliminates 90% of DIY. “as-a-service” platforms have revolutionized most aspects of software from infrastructure and data to observability. It is inevitable that DevOps-as-a-Service and IDPs are the future but they don’t have to be built in house.
Top comments (0)