A great article, thanks. I'm working on a project where agile has become "we do one release every two weeks" which always gets delayed because everything gets tested the day of the release or the day before. It's actually painful and not agile at all. I often don't even remember what has been waiting in the staging pipeline for 2 weeks 🤔 I feel like we can do better but the biggest hurdle is that agile has been translated in 2 weeks sprint and nothing else. It's funny that we became rigid with a methodology that's supposed to be flexible and adaptable.
It can be tough that's why I felt it was important to mention it's a cultural challenge just as much as a technical one. You have to try and convince the right people that releasing more often with less manual process is less risky and scales better.
There have been lots of anti agile post lately. One good bit of advice is to simply decouple project processes and technical processes as much as you can get away with. Some organizations just are going to reinvent waterfall no matter what label you slap on the process.
Just because you are doing scrum to plan what you do doesn't mean you can't do CD. If you want to stop time once every two weeks and celebrate all the stuff that went live over the past two weeks, fine. But don't block deployments on some poor overworked PM staring at an issue tracker. That's the opposite of agile. Instead, it ships when it is better than what you had; if it isn't don't merge it. If what you merged is not good enough or can be improved, change it some more. That's called iterating; it needs to happen more often than once every two weeks to get good results.
@jillesvangurp
I have nothing against agile :-D just that in this particular case of this particular project it has just become another buzz word that didn't change the status quo much.
But don't block deployments on some poor overworked PM staring at an issue tracker. That's the opposite of agile. Instead, it ships when it is better than what you had; if it isn't don't merge it. If what you merged is not good enough or can be improved, change it some more. That's called iterating; it needs to happen more often than once every two weeks to get good results.
I agree, but in this case it's the PM that's also doing the tests that's blocking the release cycle, not the devs :-D It's a combined problem of lack of resources (no QA/test team) and not applying agile at all. Anyhow we can stil try to do CD, it's a cultural problem as Chris said.
There's your problem: you have a human in the loop. You can't do CD with humans in the loop. The whole point of CD is to rely 100% on your automated tests. Your PM is part of the problem, not the solution. If he insists on signing off manually on everything before it ships that means there's a lack of trust for people to do the right things. That's bad; call it out.
When it comes to acceptance testing, your PM should be doing that in production and keep on insisting on incremental improvements until the feature can be signed off on. Likewise, the pressure should be on developers to not merge things that aren't production ready and to keep production functioning correctly. If unacceptable/poorly tested changes are landing in production regularly, that means your PM messed up planning the feature or the team messed up building the feature. Either way, that needs to be fixed if that happens regularly but not by crippling CD. If that happens, you have a competence problem, not a process problem.
The longer PMs drag their heels, the more expensive deployments get. The cost of a deployment grows exponential to the size of the delta. The price of shipping a 1 line change: almost zero. The price of shipping a delta that touches double digit percentages of your code base: weeks of testing; integration headaches; haggling; over time, etc. It gets ugly very quickly. Add to that the cost of not having features that were ready to go weeks ago adding to your revenue. The economics here are brutal. Not doing CD is very expensive. You end up wasting lots of time and money.
The opposite is also true, the cost of fixing issues resulting from small deltas is very low. So, shipping lots of small things ASAP lowers costs by decreasing testing effort, maximizing time to market, and decreasing the cost of defect fixing.
Cant upvote this enough. I tried to argue that a lot of what CD advocates is what agile is meant to be. You know, being able to respond to business needs as simply and easily as possible!
Either way, decoupling technical and business processes is a good first step. Does your PO/Scrum master/whatever need to know when you copy code from one computer to another?
Testing day before release makes assumption there are no issues to fix.
Our team got around it by adding +50% time to sprint for fixes or adjustments. If there are no issues to fix, we have this already booked time to deal with tech debt or fix punch of smaller issues that never get highest priority on their own.
In our case what happens is that the "release day" is not a fixed day because there was too much to test and this is a problem that speaks for itself :D
We struggled too, as our tester was part of another department, hardly ever available. Once we managed to explain management it's a bottle neck, and it didn't happen over night, eventually we got dedicated tester for our team. Think it was around two years ago. It has made a massive difference, we get our feedback very quickly, no slipping deadlines, everyone's happy :)
For further actions, you may consider blocking this person and/or reporting abuse
We're a place where coders share, stay up-to-date and grow their careers.
A great article, thanks. I'm working on a project where agile has become "we do one release every two weeks" which always gets delayed because everything gets tested the day of the release or the day before. It's actually painful and not agile at all. I often don't even remember what has been waiting in the staging pipeline for 2 weeks 🤔 I feel like we can do better but the biggest hurdle is that agile has been translated in 2 weeks sprint and nothing else. It's funny that we became rigid with a methodology that's supposed to be flexible and adaptable.
At least we have continuous integration...
Thanks for the kind words.
It can be tough that's why I felt it was important to mention it's a cultural challenge just as much as a technical one. You have to try and convince the right people that releasing more often with less manual process is less risky and scales better.
Good luck!
I agree, the hard part is to find a better release schedule
thanks
There have been lots of anti agile post lately. One good bit of advice is to simply decouple project processes and technical processes as much as you can get away with. Some organizations just are going to reinvent waterfall no matter what label you slap on the process.
Just because you are doing scrum to plan what you do doesn't mean you can't do CD. If you want to stop time once every two weeks and celebrate all the stuff that went live over the past two weeks, fine. But don't block deployments on some poor overworked PM staring at an issue tracker. That's the opposite of agile. Instead, it ships when it is better than what you had; if it isn't don't merge it. If what you merged is not good enough or can be improved, change it some more. That's called iterating; it needs to happen more often than once every two weeks to get good results.
@jillesvangurp I have nothing against agile :-D just that in this particular case of this particular project it has just become another buzz word that didn't change the status quo much.
I agree, but in this case it's the PM that's also doing the tests that's blocking the release cycle, not the devs :-D It's a combined problem of lack of resources (no QA/test team) and not applying agile at all. Anyhow we can stil try to do CD, it's a cultural problem as Chris said.
There's your problem: you have a human in the loop. You can't do CD with humans in the loop. The whole point of CD is to rely 100% on your automated tests. Your PM is part of the problem, not the solution. If he insists on signing off manually on everything before it ships that means there's a lack of trust for people to do the right things. That's bad; call it out.
When it comes to acceptance testing, your PM should be doing that in production and keep on insisting on incremental improvements until the feature can be signed off on. Likewise, the pressure should be on developers to not merge things that aren't production ready and to keep production functioning correctly. If unacceptable/poorly tested changes are landing in production regularly, that means your PM messed up planning the feature or the team messed up building the feature. Either way, that needs to be fixed if that happens regularly but not by crippling CD. If that happens, you have a competence problem, not a process problem.
The longer PMs drag their heels, the more expensive deployments get. The cost of a deployment grows exponential to the size of the delta. The price of shipping a 1 line change: almost zero. The price of shipping a delta that touches double digit percentages of your code base: weeks of testing; integration headaches; haggling; over time, etc. It gets ugly very quickly. Add to that the cost of not having features that were ready to go weeks ago adding to your revenue. The economics here are brutal. Not doing CD is very expensive. You end up wasting lots of time and money.
The opposite is also true, the cost of fixing issues resulting from small deltas is very low. So, shipping lots of small things ASAP lowers costs by decreasing testing effort, maximizing time to market, and decreasing the cost of defect fixing.
Thank you!
Cant upvote this enough. I tried to argue that a lot of what CD advocates is what agile is meant to be. You know, being able to respond to business needs as simply and easily as possible!
Either way, decoupling technical and business processes is a good first step. Does your PO/Scrum master/whatever need to know when you copy code from one computer to another?
Testing day before release makes assumption there are no issues to fix.
Our team got around it by adding +50% time to sprint for fixes or adjustments. If there are no issues to fix, we have this already booked time to deal with tech debt or fix punch of smaller issues that never get highest priority on their own.
In our case what happens is that the "release day" is not a fixed day because there was too much to test and this is a problem that speaks for itself :D
I'll work on changing the approach!
We struggled too, as our tester was part of another department, hardly ever available. Once we managed to explain management it's a bottle neck, and it didn't happen over night, eventually we got dedicated tester for our team. Think it was around two years ago. It has made a massive difference, we get our feedback very quickly, no slipping deadlines, everyone's happy :)