Software development is unpredictable but people waste hours every week trying to guess how long tasks will take.
Estimation can provide confidence in future changes, but this doesn’t apply to software development because:
- If you can predict work then you should automate it instead of repeating it.
- If you can’t predict work then attempting to do so is a waste of time.
Automation Instead of Accurate Estimation
Repetition increases predictability. It isn’t possible to accurately estimate how long a task will take if you have never done it. In other words, the more you repeat a task the more predictable it becomes.
Repeated work should be automated. In software development the ability to re-use libraries, frameworks, and APIs means that repeatable work can be automated. The more you repeat it the more effort would be saved with code re-use and automation.
Accurate Estimates Correlate with Wasted Effort. As repetitions increase the value of automating and the predictability both go up. Therefore …
Inaccurate Estimation is Harmful
If the purpose of estimation is to plan around the future then inaccurate estimates can only lead to problems. This is compounded by the cost of estimating work in time and effort.
Instead of spending time and effort on estimating work that can’t be predicted or should be automated we should find alternatives. One of the most agile approaches is to work without estimates.
Working Without Estimates
Working without estimates is critical to good software development and goes hand in hand with continuous delivery. The simplest process to work without estimates is:
- Identify the most valuable task.
- Work on that task.
- Repeat.
It is important to consider the time spent on estimating work and the value it provides. This minimal process may be too strict for you but next time you are estimating your work consider the value it provides.
If you would like to discuss this further please contact me on Twitter @BenTorvo or Email ben@torvo.com.au
Oldest comments (39)
I agree with this one. One thing to consider though, is that, for consulting companies, like the one I work at, we need to create bugets/cost estimations for certain developments. So estimating is kinda required, otherwise we don't know how much to charge them. But it's really difficult to calculate, so we usually double or even tripple our initial estimations, and even then we sometimes take longer to build.
Totally agree. I try to push for no estimates wherever I am, and refuse to do them if asked. I've never seen any benefit in them at all... they usually just lead to: broken promises, rushed work, massive over-estimates to avoid missed deadlines, and generally a bad atmosphere.
We estimate on perceived size and complexity using tshirt sizes (S...XL) or Fibonacci if the PM needs to do some analytics. Time estimates never work and put undue stress and anxiety on all involved.
What analytics would they be doing? I imagine it would be something to do with estimating how much work can get done in a sprint.
We'd do this for Weighted Shortest Job First - so asking a dev: "is this a much bigger job than that? Because if it is I'll rank the one you think is easier first", and of course, possibly never do the really really hard job as it might not be valuable enough. We don't really do this for sprints - just for epics/features etc.
We also insist that those putting a value on a thing also make T shirt sizes for the value.
I think this is a false dichotomy. Not all work that you can estimate can be automated.
Case in point:
If I was given a new project to work on with requirements and mockups that have been fine-tuned through deliberation, then I would immediately break down the entire project in terms of stories.
Then, I would verify the project breakdown with co-workers and get story points estimated for every story, making any adjustments as needed through this collaborative process.
By the end, I'll have an entire project represented in story points. Using either a sensible average velocity of the team on a previous project or a gut estimate of a velocity, you can pretty accurrately predict the length of a project, providing some padding for a low and high estimate.
Now, a project to, let's say, integrate a UI based on some mockups with new APIs that are being built cannot be automated--otherwise, our jobs wouldn't exist.
Is the estimate going to be perfect? Not always, but I have found the process I have described to get you very close. If it's off, was it a waste of time? By no means! We would have an accurate roadmap of the work that needs to be done, and often, the generated stories will have included valuable implementation hints/pointers by having gone through the process. It also helps set an appropriate expectation for the team even if it can flex.
Upfront detailed breakdowns rarely, if ever, reflect the reality of how things go - and TBH are about as futile as estimates. Thought processes change, approaches change, requirements change, simpler solutions are identified, etc. Only extremely high level breakdowns serve any kind of purpose in my experience.
Development (as opposed to simple ’building') is a very organic process, and should be allowed to proceed as such.
I agree. What I'm saying is it's you're not stuck between no estimates and a perfect crystal-ball vision of the project.
A detailed breakdown will give you a more accurate picture than no breakdown at all, even if things change as time goes on.
Estimations have no value for development; they are a necessity of management in order to be able to plan. However, neither an estimation nor a plan ever survived the battle with reality.
The main reason is that estimations are one-dimensional, while reality knows the dimensions known effort, unknown effort (risk) and external dependencies. Only the first dimension can be estimated to near precision with enough experience, everything else is guesswork.
I can provide 100% accurate estimates, as long as I can provide them in arrears of after having done the work.
I can meet any deadline my boss sets for me, as long as the software doesn't have to work properly.
Jesting aside...
For the actual work, I look at things as "small, medium, or big". And then for the small bucket, I divvy them up as "smallest (1), smaller (2), small (3)". The medium bucket as "smaller medium (5), average medium (8), larger medium (13)". The big bucket as "big (20), bigger (40), biggest (100)".
The pseudo-fibonacci numbers in parens are what I enter for the story points in Jira. It's just used by the PM to order things when factoring in the "effort" guesstimate. The PM doesn't calculate or use velocity, but that would be another possible use case.
Accurate estimates are valuable to management. I don't think accurate estimates are feasible in good software engineering and a managers job is prioritization over planning.
Over a long career I came to the cynical conclusion that in many large organisations the primary job of a manager is to pass good news upwards.
Have been working on software for 30 years
Never seen an accurate estimator/estimation
Great 3 points algorithm
After 30 years, the best estimates come from a gut feeling. Intuition gets more accurate with experience but it's often hard to explain why you 'feel' it should be 3 months rather than 3 weeks.
If you can't estimate with some degree of certainty, that you can often achieve within 10% of that estimate, you're not in a great environment. All my teams in the last 12 years have been able to do this. I've been in the industry for 22 years, and in the last 12 years, I'm referring to 10 teams in 3 different environments.
The reason you can't estimate is because your environment has not been set up for you to be able to do so. You need technical and architectural standards, an excellent team and support structure, a rock solid process that empowers you to decompose complex domains methodically and systematically, methods of implementing and addressing true domain complexity, and last and not least, the right people.
I work in digital health, we have crazy regulatory requirements and deal with super complex domains, and we consistently estimate to within 10% of our initial guestimate. These are multi-million dollar projects that run for 9-18 months to reach v1.0. They often continue well beyond this.
It's much, easier than you think. It's just that creating the environment that makes it possible is the hard part. This is about leadership and responsibility. I don't blame you for thinking it's a load of crap if you've not been empowered this way. What is sad is that it's the case more often than not, and that is not good. Not for you or your project sponsor or customer. What a shit-show.
A great leadership and an empowering environment won't make the task any less unpredictable. If it's the first time you do something, you can't put an estimate on it. Being able to make good estimates means your work isn't really creative. You can live your entire life selling the same thing, though. Which is ok by the way.
We certainly don’t do the same thing project to project, but we don’t introduce new tech or change our architectural preferences unnecessarily either. That then just leaves the domain, which is far more predictable if you can decompose it responsibly. The idea is to create an environment that eliminates most technical risk, leaving people to solve for the domain with a great process and support structure in place.
This leaves every project in an 80/20 state from really early on, where 80% is what we’re pretty certain we’ll be able to figure out without any real heavy risk. Risk is then buried in that 20%, which is what we’ll focus on very early on to identify and address. Honestly, it’s just a recipe that works and I know can be replicated anywhere. Unfortunately as I said, it’s just really rare to find environments like this. Which is nuts.
Hi Ben,
That is an interesting point of view regarding software estimation, thanks for that.
Actually, this topic becomes very controversial when you mixed software developers with managers, as it all depends on the point of view. From business perspective, software estimation are basic, from software development a estimation it's just that .. an estimation. So, I would recommend (if you haven't read it yet) "Software Estimation: Demystifying the Black Art" amzn.to/39SiAYW
Because... What is an estimation? What's the difference between an estimation, a commitment or target? Actually, that's the first chapter of the book, and the author provides with an example that most of software engineers have faced at some point: A manager asks for an estimate, when they are really looking for a commitment or a target.
So from my software engineer point of view I agree with your focus, but from companies point of view probably an estimate (or commitment or targtet) is mandatory. Therefore, I think a balance between both trends is mandatory.
What do you think? Do you agree?
Thanks for the feedback.
I think companies that try to plan too far ahead can't respond quickly and effectively to changing technology and markets. This might or might not impact them.
I think we should focus on prioritisation and delivery over estimation and planning. We don't need to worry about planning if we are sure we are working effectively on the most important thing.
Now you're talking more the language of LEAN agile practices of which I'm engaging with more nowadays. Throw out waste, work on the most important thing, be confident in the next week and less so in following weeks, and go through predictable cycles of planning for things a month out and for this quarter.
Planning anything beyond this quarter will change a ton. Beyond 2 quarters is just business prioritisation/roadmap planning of what to engage in after resources finish up current projects.
Very thought provoking! It seems to be that the managers have to be agile trained in order for this to be facilitated smoothly throughout?
// not speaking from experience, never been in a full agile team