My name is Anton Omelianenko and I’m Head of Software Development at EXANTE.
A manager runs a team so that it delivers results for the business. To judge how well people are handling their work, a manager needs data and a system for assessing it. In software development this is harder than it looks.
This article covers three questions:
- Why classic metrics fail when assessing developers.
- How to measure performance properly.
- What to do when an employee may not be working only for you.
Lines of Code, Commits and Story Points: Why They Don’t Work
The question of how to measure developer performance is as old as software development itself. The industry has tried almost everything in that time. We tried to rely on the same data. Here is what didn’t work, both across the industry and for us.
Lines of Code
This is the most intuitive method and the least reliable. On a simple section of code a developer writes more lines in a day than on a complex one. Complex work means long consideration of every decision.
The best-known illustration comes from Facebook, where a developer reportedly spent a week on a single line that later earned the company a million dollars.
We looked at our own statistics on lines of code as well. The chart was chaotic and gave us nothing to build on.
Commits
Productive developers do commit several times a day. The reverse doesn’t hold.
Committing less often doesn’t mean a person is working less well. Tasks differ in complexity, volume and purpose. Counting commits rewards people for breaking work into pieces to improve a number.
Story Points
At first glance this looks fair. Whoever closes more points is more productive.
Teams that assess people this way soon meet a problem. People start taking easy tasks and avoiding difficult ones, and real work gives way to the appearance of work.
A race for story points also removes the incentive to build something that lasts. The only motivation left is to close tasks faster, and the quality of the work suffers for it.
All of these metrics count volume rather than quality.
We tried them at EXANTE and found that they reflect actual performance poorly. The business still needs to know that development is working effectively, so we kept looking for a solution.
AI Opens a Route to Objective Measurement
That’s how DevMind came about, our internal tool based on AI. It collects data from GitLab and Jira and builds a report on each developer.
It analyses three things:
- Commits. DevMind rates their complexity and quality on a ten-point scale. Where the score is low, it explains why.
- Tasks. From Jira it takes the number of tasks a developer closed, how often those tasks returned from testing because of bugs and which story points were closed during the sprint.
- Meetings. DevMind assesses participation in daily stand-ups: what a person says, and how closely it reflects the real picture.
Two observations on accuracy, drawn from our own use of the tool.
The commit score is correct in roughly 80% of cases. We have seen the AI rate a commit below average where the tech lead judged the work sound and well reasoned.
The scores for tasks and meetings have proved more reliable.
Even a complete report isn’t an assessment.
DevMind assembles an objective picture and a person draws the conclusions. Not an outsider, but the tech lead who works with these people every day.
Only the tech lead can say whether the AI judged the complexity of a commit and the quality of the code correctly.
For the tech lead this isn’t a reporting exercise.
The tech lead joins the stand-ups and speaks to the team directly. That shows who’s strong where, who needs help and who’s ready for a harder task.
The DevMind report adds figures to that view.
The final decision stays with the tech lead and the product owner. We work from their feedback and from overall progress against the product plan.
The result is an objective assessment from DevMind alongside a subjective one from the people who work with the team every day.
Together they are more accurate than any single metric.
We will describe how DevMind works in more detail in a later article.
Performance Isn’t a Constant
Performance also varies over time.
Events in a person's life affect productivity, and that is normal. An employee in that position still knows the system, still helps the team and still holds experience that remains valuable.
Performance is therefore best assessed over a long period, without hasty conclusions.
If productivity drops, the tech lead doesn’t need to arrive with complaints the same day.
Start with open questions.
Are there difficulties, and what would help?
That is often enough.
If performance hasn’t recovered after some time, a direct conversation is reasonable:
You used to deliver X, now it is half of that. What has changed?
Sometimes the reason is an unexpected one.
What to Do When Polyworking Is the Cause
Sometimes the reason is simple.
The employee has taken on a second job alongside the first.
This is polyworking.
The problem isn’t the second job itself. The problem is that the person doesn’t mention it and, in order to keep up in both places, works at half capacity in each.
In a remote environment this happens more often than people care to admit.
One of our leads once came across a forum post by a developer on the team.
The developer proudly described a project delivered "at work" and attached screenshots.
The Jira in those screenshots wasn’t ours.
That’s how we learned the person had a second job, and by the look of it a higher-priority one.
I don’t believe a company should forbid employees from taking on additional work.
If a specialist handles their core tasks well, I have no questions for them.
They are free to run their own project or take on extra work.
The problem begins when the second job becomes the priority and the person stops delivering on the first.
The manager's response matters here.
Don’t dismiss anyone on the spot and don’t look for someone to blame.
The best available step is to state the expected result as clearly as possible.
For example:
"I need you to close a certain volume of tasks in a sprint. Do you consider that workload reasonable?"
"Yes."
"Then we agree on this. Delivering less on a single occasion because of something unforeseen is fine. If it happens for two sprints in a row, I won’t be able to keep you on the project, because the work can’t be held up."
After a conversation of that kind the decision belongs to the employee, who weighs up how much they need the second source of income.
One point has to land:
Keeping the role while working at half capacity isn’t possible.
That’s also the more honest position for everyone, for the team and for the person.
Conclusion: Watch the Trend and Trust Both Assessments
Bringing it together:
- Classic metrics don’t reflect actual performance on their own. Lines of code, commits and story points count volume rather than quality.
- Measuring properly means combining an objective assessment (DevMind, Jira data, story points) with a subjective one (feedback from the tech lead and product owner, plus 360-degree reviews). Neither gives the full picture alone.
- Watch the trend rather than the moment. Performance at a single point in time is an unreliable indicator.
- If an employee appears to be underperforming, set out your expectations first and allow some time. They may be working perfectly well. Just not only for you.
Top comments (0)