At TNP we worked using story points to estimate and control the amount of work everybody on the team was doing, but we decided to ditch it, here is why.
We discovered several problems:
People were stressed to finish their points, and they couldn’t focus on helping each other because they wanted to end their user stories. This issue was pushing individualism rather than quality and ship feature focus.
Having story points wasn’t pointing us to build faster or with more quality. User story points weren’t focusing the work on value but rather on risk, time, and complexity of the user story and features. 1-week sprints were making everything worse due to this fact when we finished features that weren’t good enough to put in production and were coming back to haunt us.
We always like to iterate on processes to improve it and reduce friction at maximum, so everybody is comfortable working and focusing on delivering as much value as possible. With this idea, we decided to focus much more on delivery, following some rules.
We have two-week sprints, and we organize teams by features, so everybody is on board to ship the feature and responsible for it. We don’t have departments, but feature teams with different roles. I haven’t invented this. It’s from The Nature of Software Development book that @stanete recommended some time ago. Every user story or feature has a team behind, design, product, development, and everybody is needed to accomplish it.
We had to improve user story definition, make them more shippable and self-contained so we could deploy each of them with value for the client. That’s why we decided to be much more product-oriented. Make those minimum viable features to adjust the value and time consumption working on them. This idea is probably the most difficult to achieve. Still, we think it is better to work in this direction rather than trying to estimate better, which won’t focus on deliverability but internal needs.
This new way of working makes it more natural to have other advantages for the team. For example, when someone finished their stories early, he/she has to help the rest of the team on the features to complete for this sprint so we can ship as a team. We have time every week to learn in the process because we’re not stressed, and the time is much less tight. Improve accomplishment managing scope rather than improving estimates. We plan to do as much as in the last iteration to do good work at a consistent pace.
In the case of engineering tasks and reducing technical debt, we follow the rule ‘leave the campground better than you found it.’ I have adopted that rule for years since I read Clean Code, but all the team must stay in tune, so every time we do a feature, we can clean up the area where we are going to work. We’re exhaustive on testing and good quality code to avoid technical debt. We also try to move the QA phase of the feature as soon as possible and not only when it’s done. Once we ship, the feature shouldn’t come back with defects in it; otherwise, It disrupts our development pace.
We haven’t invented this way of working. Some of the tips are from the Influence of the Nature of Software Development and Clean Code , among other books. We’re always iterating to improve our processes and learning new things so that these changes will vary in the future.
Cover Photo by Franck V. on Unsplash
Top comments (17)
I read the post through twice and I still don't think I clearly understand why your company has chosen to drop story points.
It seems like they've just moved to working on what I'd see as an 'epic' level feature rather than the tasks to make up the epic - and doing so with no metric or indication of how long it'll take beyond 'it ideally needs be done in 2 weeks so we can ship it'
The post also mentions that the company has changed several things at once ie; sprint duration from 1 week to 2 weeks and refocusing on value rather than workloads/estimates among any other things, which (in my experience) just means they'll be unable to measure or exactly pinpoint what has made the actual benefit (if any occurs at all) as each change muddies any potential metrics of the other changes.
Don't get me wrong - I do applaud your company for realizing that something isn't working and taking steps/risks to make changes for your employees!
I just personally think that they're barking up the wrong tree.
Did they consider using another style of work such as kanban or changing the existing process - such as by empowering the teams to decide how they broke the work up so they could determine workloads themselves?
I think there were other, far simpler and measurable ways to tackle the issues you mentioned than what you've described has been done.
Would be happy to discuss further and I hope the changes do have some positive results for y'all 👍
Hello Alex! You're right, my article is a bit messy, I have to put more time on it, but it's difficult for me. :-)
We have two main goals in the company regarding product delivery:
In these two main objectives story points were something pulling us back in our opinion. We are a software agency, BTW.
Not really, we still have epics and features, what we want to do there is to make all the features more or less "atomic" in the sense that we will spend more or less the same time on each one of them and they need to have a common goal, all the features or user stories need to give value to the client. When we deploy, the client needs to see something he/she can use.
We have two-week sprints, but we deliver when it's ready, usually at least two deploys per sprint, we have CI/CD, so that's quite easy if we focus on feature QA and testing, because when we ship we need to be sure that feature is right and stable. If the features are "atomic", we can deploy each of them separately, although we all know that's sometimes difficult, that's something we want to achieve more and more.
We decided this together with the team (as we usually do), we set goals, and we agreed that according to our goals, which is delivering more and with more quality, story points are not a metric giving as anything.
The human being is terrible at estimating, and in our case, and how we work, it's not a first-class metric. we do estimate when doing proposals for clients, of course.
We do use Kanban and Scrum, most of their ideas, of course almost nobody follows completely, neither do we. :-)
We are simplifying the way we measure this. Our new metrics are the number of features deployed, code quality (duplicity, complexity, code coverage, coding standards issues), frequency of deployment. Story points wasn't a good metric for us to know if we're healthy and we're delivering fast.
Thank you, Alex! Thank you for making the effort of writing your ideas, and I do appreciate having good constructive criticism. We can talk about it further, of course!
Thank you for the detailed reply and more insight into the things that influenced your decisions 😃
For context on where I work and where my thinking comes from - currently I full time with a small team of 3 developers (myself included). Previously I've worked at larger companies with 100+ developers working on the same project at once so the last 6 months have been quiet the change. I also do a small amount of contract work on the side for clients.
Currently at work we're tackling the some of the same issues/goals you've mentioned:
Our approach to achieve these goals has been to focus on breaking down our requirements into stories that are only the value-add/feature they provide and then create tasks within them, each feature as you said needs to be as 'atomic' as possible. We then point the feature with the effective 'cost' it will have on our team to complete it (based on rough time, complexity and risk).
We don't find that this process holds us back as it only consumes 2-3 hours every fortnight (~45 minutes planning on sprint start, and 2x 1 hour sessions on Wednesday's to discuss where we're at and what needs to come in to the backlog, be refined for next sprint and then pointing what we have 'certainty' on) and we are able to estimate reasonably effectively how much we'll be able to deliver each sprint.
So long as we remain focused on completing the tickets in the order we bought them into the sprint, we've found that we remain effective and keep delivering value quickly via our processes (and the CI improvements we've made in the last few months have enabled this further)
While not official, if I were to write down our sprint health metrics they're probably along the lines of;
and these are bought up and discussed at every retrospective we've had.
I think the best way to sum up how I'm thinking after reading your reply is that I think we both (and probably everyone else in the industry too!) are tackling the same issues and trying to find solutions to them - and I really appreciate reading other posts about people going through the same/similar things and trying different ways to solve them!
However, I still do not think that these new metrics you're deploying are any better than story points in themselves;
number of features deployedas the same metric asnumber of points completed- in the end they'll be about the same. Because if you do more features you'll see an increase in the number just like if you complete more points you'll see an increase (and vice versa)code qualitydoesn't add value to the customer directly (they only care whether they get x feature or not this week) and from what I've learned in the past I'd say that its notorious for being an arguably, poor indicator of software qualityfrequency of deploymentdoesn't strike me as a measurable metric but that could just be the wording or my interpretation of it! 😉I think I'd rephrase this now to say something more like; It would seem that you could continue to measure the things you want and get the metrics out by continuing to use story points - as they don't seem to have been the issue for you team(s) in my opinion. Rather, I believe that you could make small and incremental process changes through actions. Reading the agile manifesto thoroughly and applying it better than you admit that you have been - that may be more likely to provide you with the benefits and improvements you're looking for!
However, something I omitted in my original reply and removed before hitting submit was that I'll be the first to admit that I don't work within your organisation and I don't know the complexities or other challenges your teams face day to day. You all need to do whats best for you and as long as you (and your teams) can learn from any mistakes, make changes and continually improve - eventually you'll find what works right for y'all.
Hello again! :-)
For me, those are pretty reasonable goals. The only difference in our approach is we want to have two simple main goals (delivery more and with more quality), and only the last one you said is in our top list. We also take care of the other goals you wrote, but what I want is to simplify our primary goals.
I think we share many metrics and approaches, what I wanted to clarify with this post is we tried to change our primary goals and metrics, so we improve the company direction. It wasn't bad, but focusing on delivery and shipping made us more product-oriented. I'm not saying the other metrics are wrong. I say the other metrics fulfill other goals that are less important for us right now.
Yes! I totally agree with this one. :-)
I got into my trap here! 😁 Because what I can say with the indicators we have (We use Codacy for it) is estimate if we're going to spend more or less time on something and what is the health of the code (less healthy, more risk and uncertainty), but it's NOT a delivery metric.
I agree with you again.
It's true, but It says what's the delivery health, if we want to have 'atomic' features and iterate fast we need to deploy fast, and many times. It says something of the project and delivery, although I have to say we're still thinking on what metrics we want to focus on here.
What we're doing right now is following more the shippable features that don't go back approach rather than checking all these metrics. Still, in the following months, we need to have a good overview of what's happening regarding our features and delivery, that's for sure. :-)
Thank you for your comment again! I appreciate your effort responding, and I've really learned from our feedback. 💯
Thank you for being open to discussing and receiving feedback - I've certainly been able to learn from your replies as well 👍
Do a follow up post in a few months - would love to know how it's getting on!
That’s a great idea! I’ll do it! 😁
Ditto
I think my thoughts echo a couple of other comments; have you considered switching to kanban?
Certainly moving to 2 week iterations was a good move, it would be interesting to see now if reintroducing story points makes a difference given you've time to breathe.
Hello Jacob, we follow Kanban and many Scrum ideas too, but we want to focus more on delivery and quality, and we changed our most important metrics to achieve that. Once that happened, we realized story points weren't giving us positive advantages.
Thank you Jacob!
It is true to say that we've one project where we don't use story points, but we use kanban rather than scrum.
Do you work on your own product(s) or external clients?
We have both!
Interesting! I've yet to have a client give carte blanche to just get on and build their product without some form of estimate, as much as that estimate is usually (read: always) wrong one way or the other, and usually I use story points after breaking down stories to guesstimate rough time and cost at that point in the project. True to say this is constantly revised as the project goes on with each iteration.
How do you go about this with clients?
Don't get me wrong, I really like the idea of delivering (and billing) quality and value over time but the business world doesn't seem to be there quite yet in my experience.
I understand you. We have the same problems. In our case it's more about scope rather than carte blanche, in software having closed budget projects is usually sad for all the parts, the expectations are challenging to fulfill from the client and our point of view. So what we say is let's see what features we want to build and make an estimation in sprints, we need two developers full time and a product owner part-time (for example) per sprint and we charge you that.
Of course, this estimation won't be written in stone, but we'll charge you only for that sprint and not a big release, so it's more flexible for the client too, and the client can decide to prioritize other features depending on us finishing faster or not. Sometimes they want to prioritize different features depending on new metrics, so this approach is also more convenient.
When we start working with someone we explain we use user story mapping, we split features and stories and estimate from there, we prioritize and usually have a Sprint Zero to deliver the first product backlog.
These discussions are great to improve! Let me know if you want to discuss more about it. :-)
Thank you for your comment!
I suppose how story points are used varies greatly.
My team uses story points as a gauge of effort of a story. Although Fibonacci-esque values are not supposed to represent time, for my team they do roughly correspond to:
For my project, the team* takes on stories, not individuals. The stories are broken down into tasks. Tasks don't have story points; tasks have a time guesstimate in hours. Tasks are guesstimated by the person taking on the task for the sprint.
Stories are not assigned to people; stories are assigned to teams. (Or rather, each team has its own Product Backlog. Each team has its own Product Owner. Product Backlog Items do sometimes migrate from one team to another, usually due to horse-trading by the Product Owners.)
My project has about a dozen Scrum teams. Story points are not interchangeable between Scrum teams.
We do not use Story Points to calculate velocity. Velocity is something that can be utilized by the Product Owner to gauge how much can get done by when, so if the Product Owner was interested in that kind of forecast it could be done. But in my project's situation, the Story Points is more used by the Product Owner as a weighting factor to balance business value against effort to help in prioritization.
* Unless your Scrum teams have only 1 person per team. At a prior company, our Scrum teams operated in that mode.
We use Jira as our tool. I think it is an okay tool. I'm not aware of a better tool. My biggest gripe is that it commingles the Product Owner's Product Backlog with the Development Team's Sprint Backlog. The Dev Team shouldn't be futzing with the Product Owner's Product Backlog. Likewise, the Product Owner should be futzing with the Dev Team's Sprint Backlog. (The commingling may very well be our fault in how we are using Jira.)
Thank you for the comment!
We used to do this too! But my main goal ditching story points is product-oriented. What we want to achieve is to focus more on delivery metrics and quality, so our metrics go more in that direction (Codacy health, number of deploys, number of features deployed, etc.)
We want to go in this direction, having fully autonomous feature-driven teams. :-)
I think that makes total sense, we wanted to have less metrics so we can focus on the work and value, and for us estimating wasn't giving us much value, also because even you're experienced, calculating velocity is hard and not a value-driven metric.
We use Jira too and I have the same impression. :-)
Thank you!
Nice.
I find too many times, that Scrum meetings are run like interrogations for the hint of missing a deadline. It doesn't matter if a fibonacci number is assigned or not because ultimately someone way up has already planned a time for delivery.
The accountants and CIOs are fully in charge and they don't like late deliveries ever.
Excellent post! Thank you very much, certainly will help me and my team to get better