Estimation... Often terrible sometimes harmful

tldr; Estimation can be useful when choosing between items of similar business value. Using story points as a proxy to forecast when a feature or epic might be finished? Utterly useless and a waste of time.

I feel as thought I don't even have the energy to get into this particular debate but since writing this post has been on my to-do list for at least three months... here goes!

This is my understanding of how 'classic' story point estimation for items should be used. In the early days of your 'agile transformation' you'll all be sitting around a table, playing a game with cards whereby everyone talks about an item, flips over their cards and then the consensus view is adopted with some room for challenge (based mainly on the inter-personal dynamics of the group). Rinse, Repeat. There's room for debate, knowledge sharing and discussion.

However this soon becomes a PITA which is time consuming and this devolves to someone 'giving it a number'. Any debate or learning that would have happened about part 'x' of the code base has now vanished.

Now the focus becomes how many 'story points' a particular team can get done or might be able to 'commit to' in a sprint. Let's just isolate that 'commit to' phrase for a moment because it's essentially meaningless. This month I "committed to" getting some exercise every day. I can have all the commitment in the world but there's really nothing underpinning that statement which will ensure I do as I say. Any number of things might come up.

Similarly "committing" to 30 story points in a sprint and then continually failing to meet that commitment, as well as being harangued by the PM or scrum master for failing to hit some arbitrary number, soon leads to a very toxic work environment.

Estimating single items (as story points as some measure of 'size') and then grouping them together as a way of trying to freeze the number of tasks in a week is ridiculously counter productive. Each item you are presented with will likely turn into a task list, removing all critique and creative potential that might mean the problem is addressed in a better way.

Estimates of single items are riddled with the following issues:

• People just inherently are terrible at estimating task time in isolation
• Huge tendency to low-ball estimates in order to please PMs/Managers
• Unknown unknowns, especially around code, are always present
• Failing to consider the time needed to write documentation
• Failing even more to consider the time needed to write good tests
• Peer pressure when making estimates

Management will talk about 'getting better estimates' or having more frequent reviews of items progress during the sprint but neither of these are really effective uses of time.

Where estimates do have legitimacy (and the only place this applies) is when looking at a list of possible items to do. It will be pretty clear, because humans are actually good at this, when items will require more work versus the others. This can be used to inform any discussion around feature, bug or epic prioritization since the value will be apparent and then 'cost' relative to the other items on the list can be used to inform a judgement as to what to do next and why.

Let's take a step back from story points and scrum to see why they exist in the context of time and whether there's a better way. Ideally there are only two frameworks for consistently shipping software: -

1. Fixed date - "we will ship on a regular cadence" e.g. monthly

2. Fixed scope - "we have a roadmap and the next release will contain these features"...

I much prefer option 1 and the inevitable response from Sales, marketing and the CEO to option 2 will be... "So, when is the next release coming out?". You should immediately be able to see the issue with option 2. not just because it's a throwback to how software was planned, delivered and sold back in 1998...

Now you've got some sort of 'sprint velocity' which has been painstakingly assembled over time through building a ziggurat of Story Points.

This is now the measuring stick you're going to use to try to map 'n' features which happen to be in a fixed scope release against time to answer the "when is it coming out?" question. And you're inevitably going to get this wrong and developers will feel bad even if you're supper supportive!

No-one likes to miss a target, even if having one is super ill-conceived in the first place.

Fixed date releasing is FAR better than fixed scope for so many reasons...

1. You don't need story points, you can focus on customer value
2. You get better at shipping software 'cause you're doing in more often
3. Requires you to invest in tooling an automation to make your life pleasant
4. The business knows releases happen on a regular basis and understand why
5. Testing 3-4 weeks of changes is SO MUCH SAFER than testing 6-12 weeks of changes!

Story points, as required by management, exist for only one reason. An attempt to answer the question from the business of "when?". They're also an attempt to replace communication around the progress of an item (or not) with a number that, at the start of a sprint, everyone can happily agree is "true".

Rather than teach engineers and developers how to communicate better about the state of an item their working on the inventors of this methodology decided to let them pick numbers for tasks, because that's easier.

Let's say that an item is a three... but half way through the sprint some god awful new implementation detail appears which means it's now almost the same amount of work again...

Does the item still have business value? Should it still even be worked on? Well, in the case of fixed date releases it's all about doing the correct, most valuable thing. Time is expansive and it's a real discussion to be had.

In the case of fixed scope (but now everyone wants a date for the next release) it's all about shoving the item in, under pressure, possibly badly or creating technical debt, and still feeling bad about it!.

As someone a lot wiser than me pointed out...

Happiness is all about your expectations.

Adopting a methodology which is effectively guaranteed to fail, where you or others are always painfully aware of your own shortcomings, is a terrible environment to work in.

Let's spend the time we would have spent in countless estimation debates as to whether something is a 3, 5 or 8 learning rather how to communicate and manage people's expectations in the moment when something inevitably slips.

There's a huge benefit in educating the rest of the business and others as to how GOOD software is written. Yes, use a number for relative size of items (and risk) versus customer value BUT never, ever use velocity as a proxy for time.

While we're here, let's not call them sprints. If development is all about naming things, let's rename this. Sprint 114 is daft and self-flagellatory.

Let's say Iteration 114 'cause that's better. It also tells non-developers something about how software is built. It's not a sprint, it's a marathon.