Measuring Engineering Velocity on a Software Team in 2024

The concept of "velocity" has become a cornerstone metric for modern software development teams, aimed at quantifying the rate at which a team completes work. However, defining and accurately measuring velocity is not without its challenges. Below is a guide to understanding and effectively measuring engineering velocity on a software team.

1. What is Engineering Velocity?

Productivity management has been a part of “knowledge work” for a long time. The history of productivity management, particularly in the context of project management in non-manufacturing sectors, has evolved significantly over time, influenced by various theories, methodologies, and technologies. Here's a brief overview:

• Early 20th Century - Scientific Management: The roots of productivity management can be traced back to the early 20th century with Frederick Taylor's Scientific Management theory. Taylor's approach focused on improving industrial efficiency through the analysis of workflow and labor processes. Although initially applied to manufacturing, these principles later influenced non-manufacturing sectors in terms of structuring work and managing projects.
• Mid-20th Century - Project Management Emergence: The concept of modern project management began to take shape in the mid-20th century. Two notable methodologies, the Critical Path Method (CPM) and the Program Evaluation and Review Technique (PERT), were developed in the 1950s. These were initially used for large-scale government and defense projects, but they later found applications in various non-manufacturing sectors like construction, IT, and services.
• 1960s and 1970s - Management by Objectives (MBO): Developed by Peter Drucker in the 1950s, MBO became popular in the 1960s and 1970s. It focused on setting clear, measurable objectives and provided a framework for organizations to align their goals with the performance and development of their employees, including in non-manufacturing industries.
• 1980s and 1990s - Agile and Lean Methodologies: The emergence of Agile methodology in the late 20th century marked a significant shift in project management. Initially developed for software development, Agile's principles of flexibility, iterative progress, and cross-functional teams have been widely adopted in various non-manufacturing sectors. Similarly, Lean methodology, with its focus on reducing waste and adding value, also began to influence non-manufacturing sectors.
• 21st Century - Digital Transformation and AI: The advent of digital technologies has profoundly impacted productivity management. Project management software, collaboration tools, and AI-driven analytics have enabled more efficient workflow management, real-time communication, and data-driven decision-making in non-manufacturing sectors.
• Recent Trends - Remote Work and Sustainability: The increase in remote work, accelerated by the COVID-19 pandemic, has led to new challenges and innovations in managing productivity and projects outside traditional office settings. Additionally, there's an increasing focus on sustainability and social responsibility in project management, reflecting broader societal and environmental concerns.

Throughout its history, productivity management in non-manufacturing sectors has continuously evolved, adapting to technological advancements, changing market demands, and shifting organizational priorities. The field remains dynamic, with ongoing innovations and methodologies emerging to address new challenges and opportunities.

Pre-Agile Era: The Waterfall Model

More specific to software development, the important history starts with waterfall. Before the rise of Agile methodologies, the predominant software development approach was the Waterfall model. This linear, phase-based model did not emphasize iterative development, and thus, the concept of velocity as we know it today wasn't relevant. Of course, deadlines and speed have always been part of any project - I’m sure they had some kind of scrum meetings when building pyramids, castles, or railroads - but the focus on how to measure and increase iteration speed for a project team was less focused on the metrics we will discuss below.

Birth of Agile

The seeds for the concept of velocity were sown with the rise of Agile methodologies in the late 1990s and early 2000s. Agile's emphasis on iterative and incremental development created a need for metrics that could gauge progress within short development cycles.

Scrum and the Introduction of Velocity

Scrum, one of the most popular Agile frameworks, introduced the term "velocity" to represent the amount of work a team could complete in a sprint (typically a 2-4 week period). This measure was used to help teams estimate how much work they could commit to in the next sprint. Over time, tracking velocity also helped teams predict their capacity for future work, allowing for better planning and stakeholder communication.

XP and the Emphasis on Estimation

Extreme Programming (XP), another Agile methodology, placed significant emphasis on continuous feedback and estimation. While XP didn't explicitly coin the term "velocity," its practices like "Planning Game" and "Iteration Planning" contributed to the broader Agile community's focus on adaptive planning and metrics, indirectly influencing the adoption and refinement of velocity as a concept.

Broader Adoption and Critiques

As Agile methodologies gained traction, so did the use of velocity as a measure. However, with broader adoption came critiques:

• Misuse as a Performance Metric: While velocity was intended as a planning tool, some organizations began using it as a performance metric, putting undue pressure on teams to increase their velocity without regard for quality.
• Variability Across Teams: The lack of standardization in story point estimation led to significant variability in velocity metrics across teams, making it hard to compare or aggregate velocities.

Modern Iterations and Extensions

As the software development community recognized the limitations and potential pitfalls of velocity, various adaptations and extensions emerged:

• Throughput and Cycle Time: These are metrics focused on the actual output (e.g., features shipped or bugs fixed) and the time taken for a task to move from start to finish, respectively. They can serve as complementary or alternative metrics to velocity.
• Commit-to-Complete Ratio: Measures how frequently teams meet their sprint commitments, providing an alternative view of consistency and reliability.

The history of measuring engineering velocity is intertwined with the evolution of Agile methodologies. As with many concepts in software development, velocity has seen its share of praise and critique. Its history reflects the software industry's ongoing quest for better ways to plan, measure, and improve the development process. The key lies in understanding its purpose, benefits, and limitations, ensuring it's applied thoughtfully and judiciously.

At its core, engineering velocity measures the amount of work a software team can complete within a given time frame, typically a sprint or iteration. It is expressed in terms of "story points" or other units that the team assigns to represent the complexity or effort required for a task.

DORA and SPACE are two frameworks that bring these concepts into the year 2023. Most of you probably have some exposure to teams that measure these stats - maybe alongside story points or maybe on their own. In our conversations at Coherence, we hear about these new frameworks a lot more than about agile terms like sprint velocity.

2. Why is it Important?

Velocity offers several benefits:

• Predictability: It helps in forecasting how much work the team can tackle in future sprints.
• Transparency: It provides a clear view of team progress to stakeholders.
• Continuous Improvement: Tracking changes in velocity over time can highlight areas for optimization.

3. How to Measure Velocity

There are many different metrics monitored around velocity by different teams and the common theme is that there is no industry standard for measuring velocity. Below are some of the common metrics we see teams using. Some of these are related to planning and task management, and others are more output-focused and help to quantify a team’s performance and shipping speed.

Throughput

This metric counts the number of work items (user stories, tasks, features, bugs) completed in a specific time period, like a sprint or week. It offers a tangible count of output without resorting to the abstraction of story points.

PR Size

PR Size can be measured in lines of code per changeset. Smaller PRs generally move faster and have less defects.

Cycle Time

Cycle time measures the amount of time it takes for a work item to move from the beginning to the end of a process. For software teams, this could be the time taken for a user story to move from "Ready for Development" to "Done." Lower cycle times usually indicate higher velocity.

Lead Time

Lead time calculates the duration from when a new work item is created until it's completed. It encompasses the entire process, from the moment a new requirement is identified until it's fully implemented.

Commit-to-Complete Ratio

This ratio measures how often a team fulfills its commitments. For example, if a team commits to completing ten stories and finishes eight by the end of the sprint, their commit-to-complete ratio is 80%.

Work in Progress (WIP)

Though not a direct measure of velocity, WIP helps teams understand their capacity. By limiting WIP, teams can focus on completing tasks and potentially increase their velocity.

Burndown Chart

While this is a visual tool, the slope of a burndown chart (showing the amount of work remaining over time) can be indicative of a team's velocity. A steeper slope might indicate a faster pace of work.

Burnup Chart

This is the opposite of a burndown chart. It represents the cumulative work done over time. The slope of a burnup chart provides insights into the team's pace.

Escaped Defects

This metric counts the number of defects or bugs reported by customers after a release. Although not a direct measure of velocity, a high number of escaped defects might indicate that a team is prioritizing speed over quality.

Technical Debt

A measure of how much "quick and dirty" work is in the code, which will need to be paid back later for maintaining the software. Like escaped defects, rising technical debt might be a sign that velocity is coming at the expense of code quality.

Customer Satisfaction

While more qualitative, customer feedback can offer insights into whether the pace of delivering features and fixes aligns with user needs and expectations.

Story Points

• During sprint planning, the team assigns a point value to each user story based on its estimated complexity.
• At the end of the sprint, add up the points for all completed stories. This sum is the team's velocity for that sprint.
• Velocity can be volatile. To get a more consistent measure, average the velocity over the last three to five sprints.

4. Common Mistakes

One of the most important parts of looking at velocity on your engineering team is understanding why you are tracking velocity and what you are hoping to optimize for. There is no “universal standard” of objective velocity across the industry to benchmark against, since codebases, tech stacks, and businesses are so different. Some things to keep in mind are:

• Comparing Teams: Every team is unique. Avoid comparing velocities between teams as it can lead to incorrect assumptions and unhealthy competition.
• Overemphasis on Velocity: Velocity is a diagnostic metric, not a performance metric. Pushing a team to increase velocity without addressing underlying issues can lead to burnout and decreased quality.
• Inconsistent Point Estimation: From a planning POV, ensure that the team has a shared understanding of what each point value represents.

5. Best Practices

Measuring velocity can certainly help your team move faster, if you do it right. Keep these best practices in mind when you are looking at your team.

A. Regularly Refine Estimations and Metrics:

• Hold estimation sessions to ensure alignment in understanding the size and complexity of tasks or of code shipped.

B. Factor in Non-Development Work:

• Include activities like code reviews, bug fixes, and meetings. They consume time and should be accounted for.
• Also account for infrastructure toil and coordination. Often this is some of the lowest-hanging fruit in increasing velocity. If you want to get some help, feel free to chat with us at Coherence!

C. Review and Adjust:

• Continuously monitor velocity and discuss anomalies or changes during retrospectives.
• Getting input from your team is the best way to improve your metrics and your understanding of them.

6. Remember the Bigger Picture

Velocity is just one metric among many. While it can provide insights into a team's capacity and performance, it shouldn't be used in isolation. Other qualitative measures, like code quality, customer satisfaction, and team morale, are also important.

Conclusion

Measuring engineering velocity is a valuable exercise for software teams aiming to improve their processes and deliver value consistently. By understanding its nuances and potential pitfalls, teams can use velocity as a tool for growth and continuous improvement rather than a blunt instrument of judgment.

We’ve been speaking with a ton of people as we look at our velocity metrics feature set at Coherence. The most important thing we’ve learned is that teams are hungry for more metrics around velocity, and they often still rely on manual scripting or cobbling together metrics from github and spreadsheets. If you're interested in measuring your team's efficiency check out our new PR Efficiency Scorecard which we just released.