DEV Community

Rob Seaver (He/him)
Rob Seaver (He/him)

Posted on

Moving to CI/CD

Last week I authored a quick post about moving my team towards CI/CD. I was looking for some feedback from folks, but I think as a neophyte to this community, I should have posted it with a "#help" tag to indicate more clearly that I was soliciting feedback.

To sum up, I'm getting some resistance in moving to CI/CD, and I was looking for feedback any of you who have made the transition. If you've faced resistance from your teammates, how did you help them get to a place where they felt more comfortable? For some more context, here's the original post:

Thanks in advance for any advice!

Top comments (4)

evanoman profile image
Evan Oman • Edited

We recently introduced CI/CD on one of our projects and fortunately didn't meet much resistance. In our case the benefits to the devs were very clear: we were able to build, test, and deploy a C++ app across multiple architectures and platforms in parallel automatically on each push. We even measured the time savings and everyone was able to see the clear benefits.

In your case I would try to make the benefits clear (e.g. measure the time savings of implementing CI/CD) and then work to address colleague concerns with some sort of compromise. Some possible solutions for the specific concern you mentioned include:

  • Commits which introduce small breaking changes could be pushed with CI-skip flag
  • Set a policy of only pushing working versions of the code and set CI to only build the HEAD of the pushed state
  • Use feature flags to disable new features until they are ready
  • Only run CI on merge requests which represent a completed feature

Admit that CI/CD is not cost free but work with your colleagues to come up with a workflow which provides a net gain.

rbseaver profile image
Rob Seaver (He/him)

Thanks, Evan, those are all great pieces of advice, and the CI-skip is actually new to me! We currently have a rudimentary feature toggle in place that keeps things turned off, per environment, until they're ready for prime time. Eventually, I think something like LaunchDarkly would be a useful next step. I'm going to run a couple of these by my teammate who has raised the concerns. He's our resident Git expert as it is and has a lot of interest in automation/DevOps, so I think he could really help us come up with something that works well for us. I appreciate the tips; I'm going to email them to my work address now and start digging in a bit more in the morning. Now, it's time to go forth and stop thinking about work :)

dmfay profile image
Dian Fay

His concern is that when we introduce many small commits over time, it becomes difficult to review the feature in its entirety to make sure the code is consistent and works with all of the previous pieces of code that were introduced. My position is that automated testing directly addresses his concern. It will become apparent pretty quickly if new code is incompatible with previous code, based on the automated test results.

Your colleague is correct to be concerned about this, and you're talking past him. Automated testing helps you catch unintended consequences of changes. He is worried that breaking up a complex change into multiple commits to meet an arbitrary length limit, especially if those commits are integrated and released individually, will make it more difficult to ensure that the change itself has been implemented correctly and completely -- which it absolutely will. When you say "automated tests will save us!" he smells bullshit, and, to be blunt, he's right.

Bite-sized commits are much easier for a reviewer to digest, and allow the reviewer to focus on style, readability, etc..., instead of whether the code wors properly.

Keeping commits small is a worthy goal, but not always a reasonably attainable one. A better criterion: one commit = one change, whatever the length of the diff. Rebasing and squashing make managing this as easy as rebasing and squashing ever are.

You will at some point, despite the absolute best of intentions, have to land a very large, complex change. It's going to be a pain to review. Your workflow needs to be able to accommodate it. And if you're going to sell your team on CI/CD being part of that workflow, there needs to be less busywork for them with CI/CD than without.

rbseaver profile image
Rob Seaver (He/him)

First, let me correct any impressions that I'm going to force this change on my team without taking into account any of their concerns. We had a long discussion about this and I absolutely said his concerns are valid and I that would do some research into how we can find some kind of middle ground that would work for us all. I actually make it a point to listen to and empathize with my teammates, because every concern they have, everything they are feeling as a developer, I've also been there.

I've also never been one to say that automated tests are some kind of magic bullet, and am still a vocal proponent of having a QA person on-hand to test for correct application behavior or regressions.

I can see how in my original post, I may have come across as some authoritarian who imposes policies as I please without considering the concerns of others. That's not the case, but I regret that it may have come across that way in my attempt to be concise.

Having said all that, the feedback is appreciated.