loading...
Cover image for Why developers are SO sh*t at estimating!

Why developers are SO sh*t at estimating!

lukegarrigan profile image Luke Garrigan Updated on ・4 min read

The 10x striver: A short story, about you.

Chapter 1 - Planning

You're a programmer, you're in the middle of sprint planning with your team. A new feature has been requested by a customer and you've been given the go-ahead. The feature seems pretty simple: add a nice popup to tell the user they're being a silly bugger putting a string in the age picker. You think of the app, you know there are places popups have been used, you've not seen the code, but you know its there.

You take a momentary glance at the other developers who are watching expectantly, waiting for the question. "So, how long do you think this will take you?". This is your moment, this is your time to shine, nothing can stop you, you're a 10x. You let out a loud sigh to ensure you have everyone's undivided attention. "Half a day".

Chapter 2 - Half a day

Half a day, half a day, half a day. This is now your mantra.

It has been a week since you uttered those words. Every morning you have had to drag yourself to your morning stand up and repeat, "I'm still working on...". You make up excuses about the code in the area being hard to understand, unreadable or an insufficient amount of tests. You wish you could rewind time and give a more reasonable estimation.

It is not YOUR fault

So many of us have been there, we've completely underestimated a task we thought was trivial and it took far longer than we could ever have expected. This is not your fault, this is something humans are inherently bad at.

It is fortune-telling.

Image of a fortune teller with hands around a fortune tellers ball

This is commonly known as the planning fallacy which is a phenomenon where predictions on how much time a particular task will take to complete are exceedingly optimistic, often disregarding the knowledge of prior, similar tasks.

The planning fallacy only occurs when planning for our own tasks, interestingly, us humans take a much more pessimistic approach when estimating for other people.

And also, the more knowledge you have around a certain area the smaller that optimism bias gets. So that explains why the senior developer who has been working on the app since the beginning was shaking his head at you when you said half a day.

We want to impress

Everyone wants to be decent at what they do, everyone wants to be an asset to the team. You want to be the developer who can get sh*t done.

And in our want to impress we only increase the amount we underestimate tasks (As shown by the brilliant story above🤣). Unfortunately, as programmers, a lot of our sprint-planning meetings are done in groups so that need to impress is often multiplied.

Pressure from above

Often it is not just the need to impress which can skew our estimations, it could also come from your team leader, or your project manager, or the CEO of the bloody company.

We all love Elon Musk, but blimey does he make some crazy deadlines. I can only imagine the pressure the engineers must feel working for him!

It's not all bad (Parkinson's Law)

Parkinson's law is the notion that work expands so as to fill the time available for its completion. So basically, you know that time when you had to finish a school assignment in 2 days and you managed to get your shit together and get it done? That's Parkinson's Law.

You underestimating the time it's going to take you to implement a feature may make you work harder at hitting the deadline, but that's only if you place a high degree of importance on getting it done. If you aren't all that bothered if your work takes longer than estimated then Parkinsons Law probably won't have much of an effect.

This is likely why Elon Musk makes crazy deadlines for his engineers, putting them in a perpetual state of Parkinsons Law forcing them to work their asses off.

So how do I get better at estimating?

In Software Estimation by Steve McConnell he recommends:

  • Breaking the problem into sub-problems or sub-tasks
  • Estimate using ranges rather than giving an explicit date (This depends on how your company estimates)
  • Find similar completed tasks, so how long it took you to implement a feature in the same area. Essentially use your prior knowledge.
  • Understand your margins - if you're estimating something completely new you might need to take a 100% or even 400% factor.
  • Communicate with the appropriate people about your estimation as you refine it.

Finally

I wrote this blog because as I said, humans are inherently bad at estimating, but I myself am probably even worse than the average human. Even just this week I've underestimated a ticket I thought would take me a couple hours, probably prompting me to write this blog.

The moral of the story:
if you're shit at estimating, don't worry about it, we all are.

I'd love to hear your thoughts and or stories of times you've underestimated a task or even your own tips for making estimations. Equally, I'd like to hear from the 10x developers who estimate bang on every time 😄.

This blog is sponsored by Code Canvases

Make your room come alive with the coolest programming/coding canvases on the market. codecanvases.com is the number 1 seller for programming prints with 100% exclusively designed canvases. Get them now whilst they're 20% off!!
[https://codecanvases.com/](https://codecanvases.com/)

Discussion

pic
Editor guide
Collapse
bootcode profile image
Robin Palotai

We also estimate the happy-path, solving if everything goes smooth. When was the last time you said "this should take 5 minutes" and there you are 2 hours later, installing drivers and reading arcane config syntax..

Collapse
lukegarrigan profile image
Luke Garrigan Author

Yeah that’s so true, I’ll be honest, it happens far too often in my case 😂

Collapse
bootcode profile image
Robin Palotai

Let's break it down a bit. The process looks like this:

  1. Oh, this looks exactly what I want, and sounds easy.
  2. Start reading docs or installing package. At this point we are hooked, and likely to go on.
  3. First hurdle comes, but now we have invested. Why leave now, success is so near. Like when waiting for bus - I won't leave now, I have waited for long, it surely will arrive soon.

The first sign of barrier should make us stop and think. When we branch out to 3 Chrome tabs, following how to configure various deps. Or get a result from a call that is not expected according to the docs.

I'm not saying we should throw up the white flag, but we have to ask ourself: This seems rather 4 hours instead the 10 minutes I hoped. Is it worth the investment, or should I stop now?

Collapse
jorgecc profile image
Jorge Castro
  • Customer: We want this modification in 6 months, can you do it? (in fact, he wants for yesterday)
  • IT: We could do it in 8 months.
  • Seller: Nah, we could get it in 4 months, top!.
  • Customer: Ok, let's start.

Finally, the project is delivered in a year.

Collapse
firozansari profile image
Firoz Ansari

Alternative:

...
The project is actually delivered in 4 months.

  • Customer: But it's a sh*ty deliverable.
  • Seller: It's a sh*ty deliverable.
  • IT: But you asked to deliver 8 months work in 4 months.
  • Seller: Can you fix it?
  • IT: Sure, we should fix it in 4 months.
  • Seller: Nah, we could do it in 1 month
  • Customer: Ok, let's start.
Collapse
lukegarrigan profile image
Collapse
lukegarrigan profile image
Collapse
jaloplo profile image
Jaime López

You are right, totally right with your article. I'm currently in a position where I have to estimate a lot of projects/solutions and estimating the time is something that makes me crazy. Overall because I could be the person in charge of that project and it's completely possible that I forgot some tasks or constraints.

In my company, we only estimate the effort for coding/configuring phase and then complete the rest of the phases (analysis, design, testing, ...) as a percentage of it. For example, I estimate implementing an API could be 5 days of coding, the testing phase will be 80% of it so we will spend 4 days. The whole task will be the sum of all these phases getting a final time of 9 days for this case.

In addition to this, we set a lot of assumptions to avoid forgetting anything. Think of a web application you estimated creating a simple user interface but that doesn't mean anything to your customer. You need to add some assumptions for limiting the scope like: no CSS rules will be developed. This is something the customer will understand or, at the end of the day, you will have as a shield to defend your position.

Collapse
lukegarrigan profile image
Luke Garrigan Author

Thank you for this, I love hearing how people take different approaches to estimate tasks. Your approach is very interesting, we do a relatively similar thing at work although we have Quality Assurance engineers who estimate the effort for testing and that gets bundled on top of the overall estimate.

Collapse
jaloplo profile image
Jaime López

Thanks for this. QA team is a good thing to test the whole product but there are a lot of tests in a project lifecycle. You have to include some effort for unit testing, integration tests, smoke test, user acceptance tests, etc. So, there should be something in the estimation to cover testing saying what kind of test are included in the estimation so you can adjust the percentage of the effort of the team.

Do your estimations include the time QA team will spend in the project? This is something you should consider to include inside or outside your estimation and you should co-work with your QA mate.

Collapse
dillieo profile image
Sean Patterson

We take this approach too, but also add another batch of time for iterations. How many times does the mock-up not do justice to the task or a fringe case wasn't caught in the original API design? Those things we take into account for as well when we estimate.

Collapse
jaloplo profile image
Jaime López

I learned that estimations are a value we said based on experience, calculus or any other technique we used. So, if a customer requires an API implementation for their business logic, we will estimate the implementation of a number of services and their complexity, let's say 5 services with medium complexity, and not the final solution - a description of what each of these 5 services will do like a black and white box. In the assumptions section (or any other place), we will define what means "medium complexity" so fringe cases will be considered out of the scope of the current project and we should include a note telling need an assessment to give a proper estimation for that issue, problem, requirement or system.

We, as estimators (that exists?), need to limit what we are estimating and considering. We cannot give the final solution in the estimation so we are not able to include fringe cases if we don't study them.

Collapse
bootcode profile image
Robin Palotai

I liked Making Things Happen (aka The Art of Project Management), which despite the title, is a totally BS-free book about overseeing a project. Has great advice about estimation. You can pop it open anywhere your current interests are.

Collapse
vimmer9 profile image
Damir Franusic

In my case, it's because I'm a perfectionist most of the time, and refuse to do ad-hoc solutions. That being said, I tend to always under-estimate the time needed to finish the project.

On the other hand, when I'm laser focused and free of distractions, I seem to be able to summon a huge amount of coding mantra, and produce more quality code that I ever expected to be possible. To paraphrase the last sentence; I sometimes still manage to surprise myself, even after 20 years of coding.

Collapse
lukegarrigan profile image
Luke Garrigan Author

I love that feeling, somedays you feel like you could just take any ticket and code it up in a few minutes, embracing that inner 10x.

Collapse
vimmer9 profile image
Damir Franusic
Thread Thread
jaloplo profile image
Thread Thread
zkriszti profile image
Krisztina Závecz

Same here! Trance, deep house, lowkey tech... 😎🎵

Collapse
jeikabu profile image
jeikabu

Back when I was a wee CS student a professor I respect a great deal told us to estimate the time and then just double it. At the time I think a lot of us only half-listened to all these "old guys". Now that I'm approaching old guy status I accept most of what they said. 🤣

That said, there's a reason why many companies/projects don't have precise, external roadmaps as to what's coming when. It's easier for things to slip than predict a hard date. Better to delay than "ship" something not ready.

I'm convinced it's more witchcraft than science, really.

Collapse
lukegarrigan profile image
Luke Garrigan Author

Haha the older you get the more you realise that these old buggers actually made some sense 😄

Collapse
elasticrash profile image
Stefanos Kouroupis

When it comes to frontend nowadays I feel like I am really good at estimate tasks. We estimate a task and it's risk separately. For example this might be a half a day task but it has a big risk (two days). So it's fine if that task spans for 2.5 days. Also modern frameworks like angular made predictions even easier due to their structured nature

Collapse
lukegarrigan profile image
Luke Garrigan Author

Yeah, I agree with this, to be honest, a lot of my dodgy estimations are because of the unknown of the backend, but sometimes the frontend does catch me out!

Collapse
phizzard profile image
Phil Tietjen

At a previous job, we used to estimate time and log time on our sprint tasks. I am very guilty of underestimating tasks from optimism, pressure, and trying to impress. Eventually I got better by just adding a padding of time from my actual estimates, but if I went over that then I would feel super poopy.

These days where I am uses points and so far it's been far less stressful being constraint to an approximate amount of work than being constraint to a time to be done.

Not that points is the answer but it's been better for me!

Collapse
lukegarrigan profile image
Luke Garrigan Author

I've used a point system also but we end up converting the points into days anyway. Like 1 point would be half a day, 2 points is a day, 3 points is 2 days, etc. I don't know whether that's just company-specific, but I like the idea of using points without the conversion to time!

Collapse
phizzard profile image
Phil Tietjen

A lot of it I'd imagine is company-specific. We don't do straight time conversions because points have to account for initial work, work post PR to address comments, and QA.

It's been an adjustment but it's been pretty good so far.

Collapse
stealthmusic profile image
Jan Wedel

It’s important, that when you need to fix estimates in a team, most probably the root cause lies in management. Estimates are estimates, but this is hard to understand for managers. So, the most important part is to acknowledge that even if estimates get better, the work won’t be done faster.

If that step has reached, based on my experience it works pretty good to use points and fixed length iterations. Once team (and it should alway estimate the whole team) has a good common sense on how many points for a comparable task, you just change the meaning of what a point is, based on how many points have been successfully delivered in the iteration. You dont‘t try to improve the estimations itself.
There will still be times where you will still need a week longer than planned. But that’s the way it is.

Ah, and breaking something up in subtasks can help, but it’s not worth to do that upfront if it’s not clear that the task will be done at all. In that case, simple team complexity estimation is sufficient.

Collapse
lukegarrigan profile image
Luke Garrigan Author

I particularly like the part about even if estimates get better, the work won't be done faster, I'll have to use that next time when I'm trying to blame the bad code for me taking so long😄

Collapse
stealthmusic profile image
Jan Wedel

Maybe you have to work on your communication, too. If it’s late, than only because we need to meet our company’s quality goals. 😉

Collapse
lankydandev profile image
Dan Newton

Are you trying to describe me??

I need to improve on this as well. I once estimated a 5 month project could be completed in 1, maybe 2 months tops. Yeah, that didn't work out very well.

Since then I have tried to be more pessimistic when estimating.

I have not got much better at it, but I haven't got worse as least 😊😊

Collapse
lukegarrigan profile image
Luke Garrigan Author

😅 that's brilliant, I've never had to estimate an entire project but I can only imagine that's 10x worse, so, so many unknowns!

Collapse
christiangroeber profile image
Christian Gröber

I honestly don't feel too much like what you described - I know that I have the tendency to underestimate how long something might take me, so I just always go wa, way higher than that, so that I can be sure that I'll be done in the time I promised. Now, that might sound great, but I honestly believe that that's what cost me my last web development job. They went with someone else, and even though it took them just as long as I estimated it would take me, me saying it would actually take me that long tipped them off.

Collapse
codemouse92 profile image
Jason C. McDonald

The planning fallacy only occurs when planning for our own tasks, interestingly, us humans take a much more pessimistic approach when estimating for other people.

Not always. I've known a number of cases where bosses, clients, or even co-workers set unrealistic deadlines for others. (You even mentioned a few such cases.)

By the way, I've written about this phenomenon as well, and broken it down into several root causes and how to fix them.

Collapse
lukegarrigan profile image
Luke Garrigan Author

Yeah of course, as I mentioned with Elon Musk setting unrealistic deadlines!

What I meant is that if someone from a 3rd party standpoint were to make an honest estimation of time for a particular person to do a task it would likely be more pessimistic.

Collapse
leoat12 profile image
Leonardo Teteo

This week I was given the task of solving a bug on the application front-end. It looked simple enough so I said when I asked that you should be solving by tomorrow (Monday). Well, I still have no idea how I'm going to solve it. It is basically some bug around Samsung smartphones' keyboard predictive text that makes the user input act like crazy. I tried a lot of things already, but it seems Samsung was very crafty as to how to turn developer's life like hell. u.u

Collapse
kayis profile image
K

Biggest problem is usually changing goal posts.

I think I estimate pretty well, but the stuff we agree on at the start is never 100% of what I'm actually end up doing.

But I could imagine, if you're a dev for a corp das does everything waterfall style, and not for a startup, you could get the impression that estimation isn't that of a problem.

Collapse
s_aitchison profile image
Suzanne Aitchison (she/her)

I think we also often fail to take into account "coordination overhead" - e.g. when my task depends partially on understanding or agreeing an approach taken by someone else working on another task.. or liaison back and forth with a client or designer. Sometimes these chats and slack convos end up taking a decent chunk of time and I think we're pretty bad at factoring it in!

Collapse
lukegarrigan profile image
Luke Garrigan Author

Completely, that's something that didn't even cross my mind! Thank you Suzanne.

Collapse
william1530 profile image
william1530

I rather overestimate than underestimate. We use points to estimate in the team. If the estimates are far apart then the lowest and the highest explain their position, after which we estimate again. Of course some stories can take longer based on uncertainty but I don't feel I have to estimate low in order to impress my team.

Collapse
lukegarrigan profile image
Luke Garrigan Author

You're an exception William, maybe you're one of those 10x's 😄

Collapse
alexparra profile image
Alex Parra

Hey Luke!
Nice write up. I’ve faced this so many times and it’s nice to know I’m not the only one.

Also thought you might Michael Wolfe’s answer on this:
quora.com/Why-are-software-develop...

Collapse
lukegarrigan profile image
Luke Garrigan Author

Thank you, yeah you're not alone! 😅

haha, I loved that write up, thank you for sharing!

Collapse
eekayonline profile image
Edwin Klesman

This is a solid topic. Thanks for writing it! 🙌🏻

I've created a 3 point estimate Excel.
When I need to estimate effort for a development project I use it to get a more realistic estimation.
It even has a risk indicator that adds an extra percentage depending on the risk level, etc.

It basically allows me to write down my best case, realistic, and worst case estimation for a (sub)task.
This is less overdone than a X times multiplier but it doesn't overcome the fact that experience/historical metrics, and the need for objective estimations still remain very important.

Collapse
_craigmurray profile image
Craig murray

So true, and not just for developers, this is the case for many many day to day tasks!!

Collapse
lukegarrigan profile image
Luke Garrigan Author

Thank you, yeah it's definitely a universal thing! 😃

Collapse
adam_cyclones profile image
Adam Crockett

3pts ... Oh

Collapse
remast profile image
Jan Stamer

I am all for not estimating at all. See Beyond Estimates by Woody Zhill.

Collapse
lukegarrigan profile image
Luke Garrigan Author

Oh me too, many companies enforce it though.

But yeah, If I had it my way I'd never estimate again.

Collapse
jjtriff profile image
jjtriff

Uf, I'm so glad to have read this. It happens to me all the time.

The tips are great, btw.

Thank you so much ;)

Collapse
lukegarrigan profile image
Luke Garrigan Author

Haha that's good to hear, we're all in the same boat!

You're welcome 😃

Collapse
lukegarrigan profile image
Luke Garrigan Author

Yeah, I agree!

I do wonder though if, for instance, the person in charge of the project used to be a programmer and they think to themselves: I could do it in half a day.

Collapse
asifnaeem1 profile image
Asif Naeem

Absolutely right, Im still struggling with that problem. If I plan to complete a task in one day, it takes 3 days. Even I try my best to follow the plan

Collapse
darkes profile image
Victor Darkes

That's why I'm glad at my work we have the full team saying how long a story should take and agree on the work required.

Collapse
theodesp profile image
Theofanis Despoudis

One rule I use is, whenever I give a hardcoded estimate, for example half-day work, I double that to be sure, so it becomes a days work.