Surely the answer isn't "good judgment", since that opens the door to bias?
Even if you applied a statistic of some sort, you would still have to apply your own judgement to make sure that it is a statistic that does not introduce its own bias, and that you are applying it correctly.
It would be great if we could determine everything by seemingly objective and data-driven numbers, but the reality is that even interpreting numbers still requires judgement. You can see examples of this all around, for example in politics you can see folks using the same numbers to draw different conclusions and support opposing stances.
If you look at the article that Joe shared below (very interesting, by the way, Joe, thanks for sharing!), it is revealing that the metrics in that study looked at the programmers "over a handful of hours" and measured things like "program size" and "time to debug". Maybe these metrics can tell us something about an engineer's ability to write code, but they tell us very little about their ability to do actual software engineering. As Jack points out in the article, they didn't look at the things that, in my opinion, truly distinguish stronger engineers: maintainability of the resulting code, ability to deal with ambiguous requirements, etc. They also never once addressed anything relating to collaboration, which is essential in a modern software engineering team.
So yeah, the answer is, in fact, "good judgement". As for bias, if you have managers that are not working to counteract their prejudices, no set of statistics is going to save you.
I ran the engineering team for epos, a point of sale company in Singapore. We grew the business from 0 to S$4.5 million annual revenue over the course of three years. Now starting something.
Even if you applied a statistic of some sort, you would still have to apply your own judgement to make sure that it is a statistic that does not introduce its own bias, and that you are applying it correctly.
It would be great if we could determine everything by seemingly objective and data-driven numbers, but the reality is that even interpreting numbers still requires judgement. You can see examples of this all around, for example in politics you can see folks using the same numbers to draw different conclusions and support opposing stances.
If you look at the article that Joe shared below (very interesting, by the way, Joe, thanks for sharing!), it is revealing that the metrics in that study looked at the programmers "over a handful of hours" and measured things like "program size" and "time to debug". Maybe these metrics can tell us something about an engineer's ability to write code, but they tell us very little about their ability to do actual software engineering. As Jack points out in the article, they didn't look at the things that, in my opinion, truly distinguish stronger engineers: maintainability of the resulting code, ability to deal with ambiguous requirements, etc. They also never once addressed anything relating to collaboration, which is essential in a modern software engineering team.
So yeah, the answer is, in fact, "good judgement". As for bias, if you have managers that are not working to counteract their prejudices, no set of statistics is going to save you.
Thank you for this reply!