Software development is unpredictable but people waste hours every week trying to guess how long tasks will take.
Estimation can provide confidence in future changes, but this doesn’t apply to software development because:
- If you can predict work then you should automate it instead of repeating it.
- If you can’t predict work then attempting to do so is a waste of time.
Automation Instead of Accurate Estimation
Repetition increases predictability. It isn’t possible to accurately estimate how long a task will take if you have never done it. In other words, the more you repeat a task the more predictable it becomes.
Repeated work should be automated. In software development the ability to re-use libraries, frameworks, and APIs means that repeatable work can be automated. The more you repeat it the more effort would be saved with code re-use and automation.
Accurate Estimates Correlate with Wasted Effort. As repetitions increase the value of automating and the predictability both go up. Therefore …
Inaccurate Estimation is Harmful
If the purpose of estimation is to plan around the future then inaccurate estimates can only lead to problems. This is compounded by the cost of estimating work in time and effort.
Instead of spending time and effort on estimating work that can’t be predicted or should be automated we should find alternatives. One of the most agile approaches is to work without estimates.
Working Without Estimates
Working without estimates is critical to good software development and goes hand in hand with continuous delivery. The simplest process to work without estimates is:
- Identify the most valuable task.
- Work on that task.
- Repeat.
It is important to consider the time spent on estimating work and the value it provides. This minimal process may be too strict for you but next time you are estimating your work consider the value it provides.
If you would like to discuss this further please contact me on Twitter @BenTorvo or Email ben@torvo.com.au
Latest comments (39)
Still my favourite piece on planning: secure.phabricator.com/w/planning/... see the "Maximizing Throughput" section.
Clint may not agree with this approach.
Estimation of work which you are doing for the first time leads to frustration and finally waste of time.
The word automation is used a lot in this article but is never actually elaborated on. The only detail I could find is reusing code as a form of automation. What other forms of automation are you referring to?
Automation through code-reuse with packages and automation of processes through code and APIs. There should be a strong focus in reducing manual work to maximise the speed to deliver software.
Hello Ben,
Thank you for the read, was interesting and our team had similar thoughts on estimation as well! But one thing I currently am suffering from is a bad estimate could go wrong as an agency, clients that are not well versed with agile and old-fashioned would return any further discussion on planned timelines with only negativity.
Other than culling out such clients, what are some of the ways we can prevent this from happening? It is indeed unpleasant on both sides, let me know if anyone have a fair share of this!
Thanks :)
I don't think it there is an easy solution to change culture.
A good start would be to engage with clients in smaller pieces of work more frequently. So instead of a large contract, try delivering a smaller set of deliverables sooner with renewal to continue the work. That should reduce risk while still allowing for the same or better long term results.
This is the same way we implement continuous delivery but at the end of the day it relies on building trust and responsibility to make sure that we are providing value to customers.
We all like to work without a rush. But every project has a budget.
If a developer is not asked about the estimation:
As practice shows that time&material approach is better anyway, because customers always do not know what they want. But anyway some planning is necessary (hello to Sprints).
Most of the time managers do not need the precise estimation. They are working with bigger horizon. Often rough estimations like "2 months", "this quarter" would be enough.
I have a lot of problems with this.
Some not automatable work is estimatable. For example an investigation into a software behaviour. If you do it enough you can estimate it. But you can't automate it.
Another example is the research for choosing a third party library , you can estimate that but you can't automate it. Also for some work, for example adding a new report or endpoint there are many aspect of that work that are the same each time you do it, like making a test plan, writing the documentation, security testing, performance testing. There are parts of the work that have higher confidence values and can set a "minimum time" for a peice of word.
Also there is some work that's automatable but 1. It doesn't happen often enough to warrant the work. 2. It's is critical enough or risky enough that code error handling is requisite but makes the work large.
In the case of number 2. You should probably get an SOP, standard operating process , for the manual work and then consider automation.
I also think there is some valuable organizational self discovery that can happen when you make estimates. Often organizations want the value good estimates can bring but don't want to the work of managment that process. The result is just bad morale.
Olympics runners don't just run many times and record how fast they finished the race. They collect all sorts of data, they reflect, they ask questions, they experiment. Most leadership gives me "ain't no body got time for that shit" . And then try to foist blame on me about the estimate inaccuracies.
We use agile and story points in Jira but it's still not easy to have a 100% estimate on how long something will take. There could be blockers, environmental issues, colleagues who are on leave etc...
So many variables to take into account.
Agreed, the past few companies I worked for wasted a considerable amount of time on story refinement and estimates. Their velocity was always really low compared to other projects I worked on. Personally I don't give a rats ass if the story I'm working on is 2 points or 5 points, I still have to do the work. From a management point of view it has no value either... What they really ask for is a global estimate: When can we go into production with this product.
Somehow people have convinced themselves that they can give a more accurate global estimate if they take the total amount points estimated for stories ( which get added onto constantly) and divide it by the teams velocity (that is never the same, due to people getting sick, holidays, devs having a bad day, massive stories that carried over from previous sprints...) Its utter madness really 😛
In the 16 years since I started in this industry, I've only worked in one place where estimation was even slightly accurate, and that was in 2006 at a place that did Extreme Programming and used a vague notion of story points (mostly inaccurate) and "ideal hours" for tasks which tended to be more meaningful but still not very useful.
Since then, I've worked in a few teams where management thought Fibonacci points estimation was a good idea. It never was, as far as I can tell. Planning poker invariably led to long debates that were unproductive and led to tension and frustration:
T-Shirt sizing sounds more promising but I've seen it used to map each size onto a time period (say 2 days for small, 5 for medium, 10 for large) which then feeds into a Gannt chart of predicted work sequencing and targets, which is dangerous, but if your management thinks in terms of "failure to meet targets" instead of "targets were incorrect", then you have problems either way.
On teams where we did detailed estimation and sprint planning, I've tried to gently point out that I've almost never seen any team actually finish a sprint because they always feel pressured to load it with more and more tasks, making bigger commitments to avoid being seen as bad team players. This isn't their fault, but rather a natural consequence of detailed estimation and sprint planning. However I've rarely made the argument successfully and at least once suffered for trying to make the case. Sometimes there is an almost religious commitment to a particular model of "Agile" which is definitely not agile.
Speaking of the pressure to commit and deliver based on uninformed predictions, I've also spoken to good devs who got bad performance reviews from their manager because it took them 4 days to do a task that had only been estimated 1 point by the team. Imagine how unfair that would feel, and the negative effects it would have on morale and trust.
There was only one benefit I've seen from estimation, and that's simply thinking and talking about tasks with the team. Someone would often raise a really important point that saves us wasted effort later.
If teams would just spend a few minutes discussing upcoming work, looking for gotchas or proposing an approach to difficult problems, then THROW AWAY any estimates, they might get more value and less pain from the whole thing. Also, throw away your sprint board and try Kanban for the same reason. Sprints don't really work and they just lead to a regular feeling that the team has failed to meet targets.