Preface
This isn't intended to be an attack on SCRUM. Just a thought experiment of a team trying to be agile and lets assume practicing continuous improvement. There are lots of problems with how they're working and they'll no doubt be brought up during the next retrospective.
Think of it as a snapshot in time on any agile team. There will always be flaws in the process as they constantly experiment and improve on ideas.
Personally I've seen this scenario play out a lot during my career and I think it highlights some pretty common problems that arise while we try our best to build software.
I'll follow up with another article but my goal here is to highlight the inefficiencies that exist while trying to distribute work to individuals rather than working together as a group (mob programming).
Let's build an app!
So you're building some software and it's pretty involved so you make sure that there's at least 3 or 4 engineers on the project (maybe much more).
You do a bit of planning and break up all the work into small deliverables. Everyone knows what they need to work on for the next two weeks and they start the sprint.
So you go to Jira and grab a ticket with your name on it and it's estimated with 3 story points. You're not supposed to map story points to time, but you know from experience 3 points means you can probably get the task done in about half a day.
Takes you about an hour to complete the task and submit a PR. Great! Ahead of schedule 😊 Good start to the sprint right? So you go look at your assigned tickets and grab the next one. This ticket is also a 3 pointer so you think the same amount of time (an hour to half a day or so).
You're team doesn't like to have long living branches and likes to keep PR's small and concise so the normal thing to do here is to cut a new feature branch off main rather than cutting a new branch off the one you submitted for code review. This works for you because your next ticket has no dependency on your previous work.
So you cut a new branch off main and begin your work. About an hour in, you're still not done but there's some feedback on your PR. Normally you'd wait to go look at the review because you know the cost of task switching but your PR is actually blocking another team member from getting their work in. Usually your team tries to avoid this from happening but you missed a detail during Sprint Planning.
Like a good team player you decide unblocking one of your teammates is worth the cost of task switching.
So now you need to switch back to the other branch. But there's a difference in migrations plus you were playing around with some data locally that you don't want to lose. So you back that up, switch branches, rebuild your containers and reinstall dependencies and rerun migrations etc...
You make the requested change from the code review and push up your work for review. The person who requested the change is now in a meeting so they can't check your updates right now. So now you don't want to just sit idle because that would be unproductive right?
You make the context switch again to your other task and rebuild containers, reinstall deps, rerun migrations and import your data backup etc. By the time you remember what you were even doing the other SWE gets out of their meeting and approves your PR. Which is great!
But now your PR is out of sync with the main branch and you have to rebase. But there's merge conflicts... so now you have to switch branches again to do the rebase and resolve conflicts. You rebuild, reinstall, run migrations etc...
You resolve the conflicts and push up the changes but you still can't merge because you have a 2 approval requirement. The other SWE who was blocked decides to help you out by reviewing too. While he's doing that you go on to work on your other task.
By the time you switch branches, and switch context all over again it's about time for lunch. After lunch you finally get some progress and you complete the task and submit a PR and add some reviewers.
Meanwhile the other SWE is struggling to get your first PR running on his machine. Some crap with Docker or something... So you decide to pair up and help them sort out the situation so they can do the review.
A couple hours later and after pulling other SWE's in to help you get the problem resolved and the code review continues. While you wait you decide to get some of your own work done. Before picking up a new ticket (creating more WIP) you check the code reviews and you're assigned to one.
So you do some more branch switching and context changing to start a code review but it takes you a bit to get up to speed with what the task even was. By the time you are fully understanding what you're even looking at you have a scheduled 1:1 and right after that there's a design review meeting you're required to attend.
The meetings are done and you have 30 minutes left until the end of the day. You try to wrap up the code review you're in the middle of but you just can't wrap your head around it after the last meeting. And now that you've sat there thinking and trying to switch contexts it's about quarter to and you figure F-it lets just go home 🤷♂️
Next morning at standup it gets brought up that there's a lot of code sitting in review so the team decides to focus on code reviews first thing in the morning before continuing on with new work. That's great!
You decide to continue on from you code review the day before since that's the branch you already have checked out. But there's also comments from yesterday sitting there unresolved on your first PR (that ticket that was supposed to take half a day to complete, remember?).
By the time you finish your review and submit your comments it's already a couple hours into the morning but you can finally resolve the comments made on your first PR. Pretty easy changes but you still have to reset your local environment to work on them. And by the time you get the updates pushed up the branch has fallen way out of sync with main.
Now you need to rebase, but yet again, there's more merge conflicts to sift through and you can't resolve these ones on your own. You want to pull another SWE in to your desk to help but they say they're too busy sorting out another issue on another PR and ask if you mind waiting until after lunch.
No more reviews left for you to help with but you still have two PRs waiting to go in. Your second PR is STILL waiting for review because no one has had time to look at it yet. You're feeling pretty blocked on all your tasks and there isn't much you can do until after lunch.
You're reluctant to pull more work into "in progress" with so much in the pipeline but you don't want to sit idle either so you pick up another ticket.
Cut a new branch and get going on that work but then you realise you need some stuff from another branch that's still sitting in code review that you didn't think about during planning. By the time you get that far it's lunch time anyway so you just forget about it for now since it's looking like your afternoon is going to be pretty busy with other stuff anyway.
Get back from lunch and that other SWE helps you resolve those merge conflicts in that first PR and get through them all but then they start taking issue with some of the changes in your work. This becomes a debate that lasts about an hour until you both agree that some of the suggested changes actually do make a lot of sense. "Ok fine." you say to yourself. "These are actually good points".
Before you start the new changes you recall that someone is still blocked by some of this work getting merged in, so you pair up with them and spend a bit of time isolating the changes they need so that they can cherry-pick those changes into their branch.
By this time, there's finally some comments on your 2nd PR and it also needs to be rebased. You decide since you're already in "rebase" mode you might as well just take care of it now. Switch branches, and contexts etc again, resolve the conflicts, resolve the comments, and push the updates...
Now you can finally focus on making those changes to that first PR but there's an all-hands meeting. You get back, you only have 30 minutes left in the day. You try to get as much as you can but it takes you 15 minutes just to fully context switch and by the time you feel productive someone wheels their chair over to your desk and asks for help with something.
Now it's 5 and time to go home and that first 3 point ticket you pulled in yesterday is still incomplete, the second 3 point ticket is still in review, you have another task that you started but haven't touched all day, and your name has been added to 2 more PRs that you haven't looked at yet and every other SWE on the project is complaining that code reviews are taking too long.
Top comments (6)
Nice rant for your first post, Dan 😊
Thinking about code review and your story above, I was surprised about engineers trying to check out and run each of their colleagues PRs locally.
IMHO, a CR should check for:
In most CR processes I have encountered, the peer reviewer is only expected to eyeball the diff in GitHub/Azure DevOps and sign off that it doesn't look wrong wrt business process. The rest can be covered by a mature CI process.
Secondly, since this is kind of a character attack on SCRUM itself (😉), I'd suggest that the retrospective should shake out a lot of the worries above:
If the process isn't working (it sounds like it isn't) then retro is the team's opportunity to rant about and solve those problems. And the solution seems pretty straightforward:
Anyway, thanks for your post, I hope you enjoy writing for the community and we see more from you. Perhaps a follow up? 😁
I have a bit more time to properly reply to this now :)
Thanks again for your feedback!
Re:
I think these are great points but they don't completely eliminate the inefficiencies and interruptions of code reviews. Even if you don't have to pull and build branches because you have the right quality control measures in place, they're still massive context switches.
You also still run into all sorts of other problems that I didn't cover in that little story.
What if you completely disagree with the approach someone took? There's a dozen ways to deal with that, some better than others, but I think it's common to allow things upstream even if the quality isn't up to everyone's standards.
It's also a lot harder to catch issues or bugs when it's not your code. Even more so if you don't actually pull it down on your machine and run it (or at least test it out on a deployable preview branch which some companies do). The more bugs that slip past the development phase of a feature, the more work QA has to do.
Under the assumption of a 100% perfect test suite, tests can't prevent mistakes in implementation, misinterpretations or all scenarios. It's why we like to have QA right?
But the more work QA has to do the more things get kicked back for rework causing more context switching among devs. Kicking work back can also quickly jam up the release process depending on what the deployment strategies are.
So it's a pretty good idea to include devs in the quality process. There's simply things a human is better at verifying than a computer, and if that human is a dev and they see a mistake, they can fix it more efficiently while it's "in development" than waiting for QA to catch it and kick it back.
I'm not making a claim that I know best, but after a year of giving mob programming a serious shot, I've drastically changed the way I think about delivering software.
There's no silver bullet, but mob programming eliminates a lot of these problems.
Hey, thanks!
Yeah there are a lot of things I could suggest be done differently in this fabricated story! I imagine this team probably would try the things you suggested.
Maybe I need to change the title, as I'm not meaning to attack SCRUM itself.
I'm trying to figure out how to express a lot of the problems that arise when you distribute work to individual SWEs instead of practicing pair/mob programming.
Not sure if this is the best way to do it yet ;)
Will definitely have a follow up post for this soon!
Thanks for the feedback 😄
Hi @danjfletcher
You make a few points, by example, that I'd like to address:
A) Sprint plans are not very flexible
B) Too many meetings
C) ROI on code reviews is not always there
A) causes many of the problems you talk about because when stories are blocked or need review your inflexible Sprint plan forces you to address everything immediately even if it ultimately wastes time context switching. Similarly being afraid to pull a story into the Sprint is again an artifact of sticking to a plan made a week or two ago.
B) - you mention a design review meeting as well as several resources you need stuck in meetings. Again as with A) stories in Scrum don't "pause" so everything is immediate and it's difficult to stay unblocked. But on top of that a face to face design review isn't a good use of time. Design is complicated and sticking it into a real time meeting isn't likely to help anything.
C) Where's the heat on these code review meetings really coming from? I'd guess there aren't many ways to get other's opinions on implementation before coding (other than the problematic design review meeting) so the code review becomes the whole shebang. That's unfortunate because if anything really wrong is found that late in the game it's a huge waste of time re-code.
All of A-C is a result of Scrum being designed when meetings were the only way to solve anything. I work on software that challenges meeting driven process and would love to see if it can help your team.
Hey David, thanks for the thoughtful comment!
This is all hypothetical of course 😊
There are so many things this team could try to improve on but what I think is interesting is that a lot of the issues really stem from this idea that work needs to be broken up to be worked on in parallel by multiple individuals as apposed to a group.
I still have to finish the follow up post to this (in the middle of getting ready for a move right now so I'm pretty busy 😅) but my idea is to try and communicate the benefits of mob programming.
That fits in well with the meetup I am trying to organize on team structure meetup.com/san-jose-collaboration-... If you have time pop in and we can discuss your idea.