Hiring a DevOps Engineer for the first time? Knowing what to look for in a talented engineer can be a challenge. In this article, I discuss what you can expect from a DevOps Engineer in today's marketplace. I share some of my own experiences hiring DevOps Engineers in today's competitive labor market. Finally, I talk about cheaper alternatives to hiring a full-time DevOps Engineer.
In my past articles, I've discussed DevOps release pipelines, stacks, and stages in-depth. A release pipeline is a software-driven process that development teams use to promote application changes from development into production. The pipeline creates multiple stacks - full versions of your application - across multiple stages of deployment.
A development team usually starts a pipeline automatically via a push to a source code control system, such as Git. The team then pushes the change set gradually through each stage (dev->test->staging->prod), testing and validating their changes along the way.
What I haven't discussed (directly, at least) is how complicated this process is. A DevOps release pipeline is itself a piece of software. It requires code to run - and that code needs to be tested, debugged, and maintained.
Many teams and small development shops get started without a dedicated DevOps engineer. Yours may be one of them! In these situations, a few team members generally own pieces of the pipeline and keep it running. Pipelines at this point are usually a mix of automated promotion and old-school manual deployment.
However, as your application and requests from your customers grow, you may realize the lack of a dedicated DevOps engineer is slowing your team down. Some of the signs include:
- Your team's velocity slows under the weight of its current (mostly manual) deployment processes.
- You have a somewhat automated deployment process but maintaining it is consuming more and more of the team's time.
- You realize after a high-profile failure that your release procedures need professional help.
- You know you should improve your deployment process but your team is so crushed with feature work that no one has time to spend on it.
If you're facing down one or more of these issues, it may be time to hire a part-time or full-time DevOps Engineer.
A DevOps Engineer's role will likely look slightly different at every company. However, the following broad-based responsibilities tend to be common and consistent.
A good release pipeline eliminates unnecessary manual steps and reduces the time required to deploy changes to your application. Building and maintaining this pipeline is the DevOps Engineer's primary job.
DevOps Engineers usually craft release pipelines using a Continuous Integration/Continuous Development tool. Tools such as Jenkins, Atlassian, GitLab, and Azure DevOps integrate with source code control tools (usually Git) and handle triggering automated actions in response to repository check-ins. If your team already uses such a tool and is committed to it, you'll want to find someone proficient in your specific CI/CD toolset.
For cloud-deployed software, a DevOps Engineer is also responsible for setting up the entire stack on which the application runs using Infrastructure as Code. A DevOps Engineer should be able to design and implement a stack deployment that can be deployed multiple times to any stage of your release pipeline.
Some engineers implement Infrastructure as Code using a scripting language such as Python. However, it's more common to use a templating language, such as CloudFormation on AWS or Azure Resource Manager (ARM) Templates on Azure.
As part of setting up the build and release pipeline, your DevOps guru will also define best practices for coding and validation of changes. In other words, they're the point person for your team's change management approval process.
For example, a DevOps Engineer may work with their team to devise the best way to manage the overall work process. For most teams, this usually means adopting an Agile approach to software development such as Scrum or Kanban. It could also mean defining a code review process and teaching the team how to conduct good reviews.
The DevOps Engineer is responsible for ensuring the continued health of the team's CI/CD pipeline. This includes:
- Monitoring build progress and logs from your team's CI/CD tool
- Moving quickly to resolve broken builds and keep changes flowing through the pipeline
- Observing dashboard metrics as new instances of the application come online
- Staying alert for errors as your deployment shifts more users over to the new version of your application
Monitoring should occur in all stages of the pipeline. As Atlassian points out, pre-production monitoring means you can stomp out critical errors before they ever reach customers.
Depending on the size of your organization, the DevOps Engineer may supervise all of this themselves. They may also work in conjunction with a Sustained Engineering or Support team that's ultimately responsible for maintaining application health. In either case, your DevOps Engineer should take the lead in defining what the team needs to monitor.
Ahhh, Git. The free source code control system is a marvelous invention. You can't be a developer nowadays and not know at least the basics of Git. And yet even seasoned developers will sometimes find themselves mired in Merge Conflict Hell.
A team's DevOps Engineer should know Git inside and out. They should understand, for example, the difference between a merge and a rebase - and which one to use when. They are the person primarily responsible for defining the team's branching and merging strategy - and maintaining quality internal documentation for other team members.
As an engineering manager, I've hired multiple DevOps engineers. During the interview process, my loops focus on validation a combination of technical and soft skills:
Does the candidate have the basics of CI/CD down pat? What successes have they accumulated in developing successful pipelines? What setbacks have they encountered - and how have they overcome them?
Cloud platform and DevOps tools
In what DevOps tools is your candidate most experienced? Do they know the tools your team is already using?
A DevOps Engineer will also need to make numerous decisions on whether to buy or build certain parts of the DevOps process. For example, does your team roll its own artifact storage features? Or does it leverage a tool like Artifactory? DevOps Engineers need to remain up to speed on the tools marketplace so they can make these critical buy vs. build decisions.
A DevOps Engineer needs to do more than build a pipeline. They need to convince a (sometimes reluctant) team of engineers and stakeholders to change the way they develop software. Does your candidate have experience talking a tough crowd into adopting new processes?
As a manager, I like to use STAR (Situation-Task-Action-Result) questions to determine a candidate's experience with being a technical leader. So I might ask something like, "Tell me about a time when you received pushback from your team on a process change. What was it and how did you resolve it?"
The DevOps and cloud spaces are changing constantly. So it's important that a DevOps Engineer not get overly set in their ways.
I also like to use STAR questions to gauge a candidate's willingness to grow. For example, what's the last thing that they learned just because it looked interesting? Did they end up using it on the job? If so, what was the result?
Alternatively, I may ask when was the last time they received critical feedback from their manager. What was it? And how did they use that feedback to improve their job performance?
You've determined that you need more DevOps savvy in your org. But that doesn't mean you need to start off with a full-time position out of the gate. Maybe you can't afford a full-time position at the moment. Or perhaps you'd just like to test the waters before diving in with both feet.
Fortunately, there are a couple of alternatives to hiring someone full-time.
You may not need (nor even desire) a full-time team member. It may be enough to hire someone on a part-time basis to construct and maintain your build and release pipeline.
In this scenario, you'd want to find a DevOps Engineer who's good at building self-service solutions. Your team should be able to kick off builds, perform releases, and monitor rollouts without having a full-time DevOps Engineer on call to oversee a successful outcome.
Another option? Forego the engineer! You can potentially save both time and money by adopting a DevOps tool that essentially provides you "DevOps as a service".
TinyStacks is one such tool. Built by a team with deep experience building out the Amazon Web Services console, TinyStacks provides an automated approach to DevOps. Using a simple UI Web interface, your team can migrate its application into a full release pipeline - complete with AWS cloud architecture - within the week.
Read a little more on what TinyStacks can do for you and contact us below to start a discussion!
Article by Jay Allen