I ran the engineering team for epos, a point of sale company in Singapore. We grew the business from 0 to S$4.5 million annual revenue over the course of three years. Now starting something.
I'm curious to know if you use any quantifiable metrics at all, or if you could recommend some in addition to qualitative measurements. Sure, there might not be a 'good set of statistics' one could use, but surely the answer isn't "good judgment", since that opens the door to bias?
Surely the answer isn't "good judgment", since that opens the door to bias?
Even if you applied a statistic of some sort, you would still have to apply your own judgement to make sure that it is a statistic that does not introduce its own bias, and that you are applying it correctly.
It would be great if we could determine everything by seemingly objective and data-driven numbers, but the reality is that even interpreting numbers still requires judgement. You can see examples of this all around, for example in politics you can see folks using the same numbers to draw different conclusions and support opposing stances.
If you look at the article that Joe shared below (very interesting, by the way, Joe, thanks for sharing!), it is revealing that the metrics in that study looked at the programmers "over a handful of hours" and measured things like "program size" and "time to debug". Maybe these metrics can tell us something about an engineer's ability to write code, but they tell us very little about their ability to do actual software engineering. As Jack points out in the article, they didn't look at the things that, in my opinion, truly distinguish stronger engineers: maintainability of the resulting code, ability to deal with ambiguous requirements, etc. They also never once addressed anything relating to collaboration, which is essential in a modern software engineering team.
So yeah, the answer is, in fact, "good judgement". As for bias, if you have managers that are not working to counteract their prejudices, no set of statistics is going to save you.
I ran the engineering team for epos, a point of sale company in Singapore. We grew the business from 0 to S$4.5 million annual revenue over the course of three years. Now starting something.
There are a bunch of studies that have tried to figure that out, specifically regarding software development.
The results are highly controversial though. I wrote an article about it a while back. The article references what they measured and has links to some of those studies at the bottom: codingblocks.net/practice/four-rea...
For the most part, these studies looked at how programmers solved the same problems in isolation and the results are still highly controversial. As for pulling metrics from actual "real world" work where people are working on different problems, and have different specialties, and different responsibilities...well, good luck! :)
For further actions, you may consider blocking this person and/or reporting abuse
We're a place where coders share, stay up-to-date and grow their careers.
I'm curious to know if you use any quantifiable metrics at all, or if you could recommend some in addition to qualitative measurements. Sure, there might not be a 'good set of statistics' one could use, but surely the answer isn't "good judgment", since that opens the door to bias?
What's a good mix?
Even if you applied a statistic of some sort, you would still have to apply your own judgement to make sure that it is a statistic that does not introduce its own bias, and that you are applying it correctly.
It would be great if we could determine everything by seemingly objective and data-driven numbers, but the reality is that even interpreting numbers still requires judgement. You can see examples of this all around, for example in politics you can see folks using the same numbers to draw different conclusions and support opposing stances.
If you look at the article that Joe shared below (very interesting, by the way, Joe, thanks for sharing!), it is revealing that the metrics in that study looked at the programmers "over a handful of hours" and measured things like "program size" and "time to debug". Maybe these metrics can tell us something about an engineer's ability to write code, but they tell us very little about their ability to do actual software engineering. As Jack points out in the article, they didn't look at the things that, in my opinion, truly distinguish stronger engineers: maintainability of the resulting code, ability to deal with ambiguous requirements, etc. They also never once addressed anything relating to collaboration, which is essential in a modern software engineering team.
So yeah, the answer is, in fact, "good judgement". As for bias, if you have managers that are not working to counteract their prejudices, no set of statistics is going to save you.
Thank you for this reply!
There are a bunch of studies that have tried to figure that out, specifically regarding software development.
The results are highly controversial though. I wrote an article about it a while back. The article references what they measured and has links to some of those studies at the bottom:
codingblocks.net/practice/four-rea...
For the most part, these studies looked at how programmers solved the same problems in isolation and the results are still highly controversial. As for pulling metrics from actual "real world" work where people are working on different problems, and have different specialties, and different responsibilities...well, good luck! :)