DEV Community: George Guimarães

3 better Engineering Metrics uses for Managers

George Guimarães — Fri, 12 Mar 2021 18:30:55 +0000

Many managers use engineering metrics to set goals. They measure the product delivery process and then use these numbers to set OKRs, attach to a pay raise or promotion, or micromanage. All the ways lead to pressure for achieving a specific number.

I’ve worked in teams in which managers would make leaderboards and control every individual’s output. It didn’t solve the problem, though. Micromanaging only increases illusory productivity, not the performance itself.

At the cultural level, it changes how the team dynamic works. Individualism increases and the team starts to make bad engineering decisions. The focus becomes to be at the top of the leaderboard, not delivering an excellent product.

There are other ways to use metrics. They serve multiple purposes, and managers can use them on different occasions. In this article, I listed three better ways to use metrics within engineering teams.

Metrics to motivate the team

Nobody likes to have their work wasted because of misalignments. Engineers want to work on what’s important and relevant. Add challenges to the equation, and engineers will have fun doing their jobs.

In this sense, managers need to communicate what’s most important clearly. If the team understands the plan, it is easier for them to see how they fit.

Some kinds of work are fast, and the team stays motivated from the beginning to the end. However, when an activity takes too much time, the team may lack the feeling of achievement. When it happens, morale and motivation tend to drop. Managers can avoid it by establishing smaller, tangible, and measurable milestones in the long run.

If a team has 30 technical debts that need to be addressed, managers can break it into a doable plan. It can be fixing one technical debt by a week or fixing ten up to the end of the quarter. The tactics don’t matter.

The crucial thing here is to enable the team to sense progress. Every week the leader can discuss the progress. In this case, metrics are not micromanaging anyone. They’re an indicator of progress. If the improvement is not satisfactory, that’s the right moment to change the plans, not blame.

Regularly looking to a metric towards a plan brings a sense of progress and a frequent feeling of achievement. That’s great. Besides, don’t hesitate to celebrate small and big wins. Both are essential.

Metrics as indicators

Metrics don’t tell the story; the context does. Metrics are merely a quantitative view of the facts. That’s important to have in mind when measuring, especially for those metrics that managers take as productivity.

The usage of indexes differs from traditional goals. Goals are future-centric and represent where the team should be from now. Indexes, on the other hand, show the present, like an indicator.

Therefore, a better approach is using Engineering Metrics as indicators. The hardest part is to find the appropriate set of metrics to compose the index.

For example, the Lead Time for Change, which measures from commit to deployment, is an excellent index of how fast the team works. It includes many outputs — commits, pull requests, deployments — and practices — coding, testing, DevOps, Product.

When it comes to quality, the problem can be more complicated. As quality is subjective, engineering managers may diverge.

Choosing the correct set of metrics depends on many aspects, such as team maturity, the company’s strategic goals, and product positioning. It could be the number of bugs found in production or the meantime to a system to recover from failure.

There are many metrics that we could use. Collaboration, culture, quality are all abstract terms that need a measurement. Once it’s covered, tracking indexes’ evolution over time can point to real-life changes.

I recommend starting with what seems obvious, then polish it as you learn.

Experimentation and innovation

In the last years, lots of frameworks, practices, and tools came out. Some of them changed the way we develop software — Spotify Model, to cite a famous one. They usually come from big companies that have a culture of experimenting.

Not every experiment needs to be that big and successful, though. Small improvements in the system are enough for our daily activities. But how can we tell whether an experiment has worked or not? Yes, metrics.

If managers measure outcomes instead of outputs, metrics become a powerful tool for empowering engineering teams with autonomy and responsibility.

The teams themselves can look at the metrics as indicators and tell whether their initiatives are working or not. They can try new practices, change the process, and dive into innovation. The only restriction would be not changing metrics for the bad.

Growth Marketers use this dynamic in daily operations. And I believe engineering teams should benefit from it as well.

The secret of enabling innovation is an instrumented system and strategic experimentation. There must be a rationale behind each initiative. And metrics will tell how successful they are.

To make it useful, it requires a fast feedback loop. It would be costly if every experiment had to run for months to determine its success. Early feedback — looking at how metrics behave — enables changing directions when things are not going well.

In short

Metrics shouldn’t be villains. They are a vital tool for modern software development. There is no space for micromanaging anymore because it just doesn’t work.

I listed in this article three better uses for metrics than comparing individuals against their peers.

In summary, we should look for metrics as indicators. They can motivate the team, tell how the system responds to daily activities, and enable innovation through experimentation.

SourceLevel provides a set of metrics that managers can use as indicators. If you’re unsure about what to measure, apply for a Free Strategy Session with me. If you’re wondering how to implement pragmatic metrics in your team, Schedule a Demo, and I will be happy to show.

The post 3 better Engineering Metrics uses for Managers appeared first on SourceLevel.

Three signs your team needs Engineering Metrics ASAP

George Guimarães — Thu, 28 Jan 2021 17:36:25 +0000

I’ve been using Engineering Metrics for a while, and I am amazed by the significant impacts on engineering teams. Metrics is a topic I genuinely care about. During my days at Plataformatec, we used to discuss a lot about them. At the time, we used to focus on an Agile perspective.

Our team was not satisfied in measuring Velocity in Story Points, nor we thought that estimating was helpful. So, we dug into Kanban philosophy and learned new statics techniques.

We found out that measuring the cards’ flow through the board was more accurate and useful and that forecasting using Monte Carlo could give us more realistic estimations.

Despite having many customers, we discovered that most of them had similar problems: lack of visibility and alignment. Once we introduced our metrics, the product teams had a north star.

Besides, we also noticed that many CTOs had no metric to show to the board. The CMO could prove they were bringing qualified leads, the CFO had data to show their team reduced costs, but CTOs usually had no concrete metrics.

Well, in summary, that’s the story behind SourceLevel. Over time, we received lots of valuable feedback, and we’re confident our product is in the right direction. We are helping companies to accelerate their deliveries sustainably and effectively.

For this article, I separated three signs that you need Engineering Metrics in your teams. Check if you identify with any.

#1 — You don’t know in which step of the product delivery bottleneck is.

For a commit to reach production, the code goes through a lot of steps. Many of them rely on human activities, like coding and peer-reviewing. The nature of this kind of work varies in quantity, time spent, and many other intrinsic variables.

When you analyze the big picture, engineers in all levels of experience collaborate to solve a diverse type of demand, from bugs to refactorings, from typos to new releases, and so on.

Systems are more complex than the sum of the parts. That’s why measuring individuals make no sense. The outputs produced vary too much in sizes, shapes, and purposes.

Besides, overseeing everything is very difficult. It increases significantly when the team scales up. So, you need to instrument each step by gathering data from commits, pull requests, and deploys — as a start —, so that you can inspect them to find improvement opportunities.

If you think there’s no bottleneck in your process or don’t know where it is, I strongly recommend instrumenting your system with Engineering Metrics.

#2 — You don’t have any metric to report upwards.

Every area has its standard metrics. You work for many companies, but the CMO or CFO, for instance, will always report pretty much the same metrics. However, the engineering area still doesn’t have a consolidated metric to report upwards.

I’ve seen many CTO fail because they couldn’t prove the team was making progress or that demand was over the capacity. Many executives still rely on gut feelings and outdated estimating methods.

By measuring the whole process and then being able to drill down on specific parts of the system, CTOs and managers can understand their initiatives’ impact and create better reports.

#3 — You don’t know whether the team is making progress towards desired outcomes.

When you focus on outputs, the results usually don’t go along with the outcomes. If the team’s productivity is measured by how many Pull Requests get merged, smart engineers will close as many Pull Requests as possible. It doesn’t matter whether the team reviewed it properly.

The organizational culture is abstract and changes frequently. Metrics are a crucial factor. Measuring the outputs, as the number of Merged Pull Requests, isn’t enough. Leaders need to measure the outcomes.

The engineering outcome is not only “code in production.” As a delivery system, we want engineering to deliver clean, well-tested, and reliable code to production faster in a sustainable — and predictable — way.

When the leaders spread this message across all teams and give the team autonomy to do whatever change they need, we have the benefits of measuring outcomes: teams are not attached to outputs to experiment and innovate. The chosen set of metrics provides the alignment.

For example, a way to deliver faster would be moving from a Pull Request approach to a trunk-based one. It can accelerate deliveries, but on the other hand, would the code still be well-tested and reliable? If the team is not mature enough, there are great chances it would not.

That said, after defining the outcomes for your engineering success, I strongly recommend looking for a set of metrics that reflects the results.

Do you want to know more about Engineering Metrics?

If you want to read more about Engineering Metrics, I’ve written a lot about them recently. You can find them under this category.

Feel free to schedule a 15-min chat with me to talk about the challenges at your company. Maybe, I can handle more resources for you.

The post Three signs your team needs Engineering Metrics ASAP appeared first on SourceLevel.

3 Benefits of DevOps Metrics within Engineering Teams

George Guimarães — Thu, 14 Jan 2021 18:33:44 +0000

DevOps metrics is a trending topic among software engineering managers. These metrics became very popular after the publication of Accelerate — The Science of Lean Software and DevOps: Building and Scaling High Performing Technology Organizations and the yearly The State of DevOps reports.

If you’re not familiar, here’s are the 4 DevOps Metrics proposed by the work:

Delivery Lead Time, which is also known by Lead Time for Change or Cycle Time
Deployment Frequency, also known as Delivery Throughput
Time to Restore Service
Change Fail Rate

If you search for “DevOps metrics,” you’ll quickly find many articles listing much more than four of them. Although I wouldn’t say they’re wrong, that’s something we need to pay attention to. Overseeing too many metrics — or DevOps KPIs, to cite a common term — at once can bring more problems than solutions.

For this article, I prepared a brief list with three benefits of using those metrics so that you can accelerate engineering deliveries at your place.

Benefit #1 — They consider only the engineering scope.

In the article A primer on engineering delivery metrics — Scaling software & systems, Buritica states that the product conception and design phase varies too much and then defends measuring what the engineering team is responsible for: from commit to deploy.

It makes sense. The product design phase defines what should be built, and considering it as engineering work distorts its efficiency. On the other hand, focusing on the product delivery phase allows managers to understand precisely how shipping code to production works.

In short, DevOps metrics show the big picture of your whole engineering process.

Benefit #2 — It promotes collaboration and alignment.

I don’t believe in developer or engineer productivity. Metrics such as the number of Lines of Code added, commits, and user story points delivered usually bring anxiety, depression, and burnout — to cite a few of them mentioned in Isaac Lyman’s article Can developer productivity be measured?

Modern Software Development requires collaboration. More than that, it relies on teamwork. It’s like a complex system that needs a hint of chaos to work appropriately. Because of that, it’s useless to measure individual productivity. If you measure each individual by inputs or outputs, the system will adapt to it, and then you’ll see a local improvement.

However, what about the other vital aspects of engineering success, such as quality, stability, and maintainability? The team probably won’t take them into account during the daily activities because what matters at the end of the day is to reach a certain number of inputs or outputs done.

In my opinion, this is a management anti-pattern. If we want engineering teams to collaborate, then we need to see how they work together. Every institutionalized practice, process, or automation changes how the system behaves altogether.

That’s why you need metrics that fully represent the big picture to understand if they have improved or worsened the team’s dynamic. As a result, teams work with high autonomy guided by the alignment provided by metrics that tell what’s expected from them.

Benefit #3 — Improving the system is more effective.

If you compare two individuals, given they’re humans, it very likely they won’t perform the same. Depending on the type of demand — whether it’s a bug, a new feature, or PoC —, their motivation, how clear the task, the related experiences they had, and many other aspects can cause divergences in the numbers.

In a Leaderboard, there will always have someone at the bottom. Even if you start a mentorship program to leverage individual productivity, it doesn’t mean all engineers will produce the same.

However, you can find whether a mentorship program is working by looking at the system. If the system is delivering more, with quality and reliability, it’s working. It doesn’t matter who is at the bottom or the top of the Leaderboard.

Daniel Markovitz wrote in an article for the Harvard Business Review called Productivity Is About Your Systems, Not Your People:

As legendary statistician and management consultant W. Edwards Deming argued in his book Out of the Crisis, 94% of most problems and possibilities for improvement belong to the system, not the individual. I would argue that most productivity improvements belong there as well. Personal solutions can be useful, but the most effective antidote to low productivity and inefficiency must be implemented at the system level, not the individual level.

Understanding the impact of engineering initiatives, projects, and daily activities on the system outcomes makes it much more likely to managers drive their teams to high-performance.

You’ll probably need more metrics later.

DevOps metrics are enough for having the big picture. But sometimes, we need to zoom in and see how the system behaves in a specific part of the process. In this case, you’ll need more metrics.

You may have a better idea of what I’m talking about reading Smruti Patel’s article Debugging engineering velocity and leading high-performing teams — Productivity & Eng velocity. Breaking the process into smaller steps is fundamental to understand where bottlenecks and issues are. Then, managers can complement their view by instrumenting those steps.

These complementary metrics give managers visibility and allow the teams to detach from inputs and outputs. So, they can elaborate on sophisticated solutions that impact the whole system.

However, the team should align every action with those 4 DevOps Metrics.

In short

The 4 DevOps Metrics are excellent to give managers a big view. However, to debug the process, they need to instrument the system using other metrics. It’s like Observability, but for the process.

For more ideas of how and what to measure, I strongly recommend our latest e-book, The Engineering Manager’s Play for debuggable and observable processes. It has quality content with examples of what to measure in each delivery phase.

See Engineering Metrics Live: Click here to get a full demo of SourceLevel Engineering Metrics and how they can help your team.

The post 3 Benefits of DevOps Metrics within Engineering Teams appeared first on SourceLevel.

Why you should adopt a community-maintained javascript style guide

George Guimarães — Tue, 01 Dec 2020 11:35:00 +0000

Airbnb’s JavaScript style guide is the most popular set of rules for code standardization available. The repository is open for contributions on GitHub. Yet, other style guides follow the same approach. They’re well maintained and available for free, such as Google‘s, Standard‘s, and Idiomatic‘s.

If you search more deeply, you can find even more style guides used by tech companies or individuals. There’s not a single way to write and format code, and that’s good and healthy.

However, even with lots of conventions documented out there, many companies prefer to write their version, which usually leads to a waste of time and effectiveness, as gathering the team to discuss aesthetic aspects of the code can be both time-consuming and exhausting.

In this article, I list reasons to enforce my opinion that choosing a community-maintained javascript style guide is the best thing for everyone — with few exceptions.

Maintaining a style guide is time-consuming

Each engineer develops a preference over time. Some people prefer tabs, others space; some prefer single quotes, others double quotes; and so on. Does the preference affect the quality of the code? I don’t think so. Does the preference affect the end-users value perception? I don’t believe it.

The fact is that quality and value aren’t expressed on coding preferences. They derive from solid processes and tools that support engineering activities. So, instead of focusing on discussing preferences, it’s more productive to focus on designing robust and well-crafted systems.

It’s fundamental to say that engineering teams and product final users benefit from a standardized codebase. The more a codebase looks like written by a single person, the faster and less buggy the development tends to be.

In this case, writing a document that states how code should look is not enough. It’s necessary to adopt and institutionalize as a practice, and the best tools available for that are linters. Linters use static code analysis to find mismatches. Examples include eslint and prettier for JavaScript, pylint for python, and rubocop for Ruby.

Community-maintained projects run this discussion with a broad reach, and detach those discussions to engineering daily activities. Maintenance is more frequent and peer-reviewed than in-house solutions.

Maintaining the linter’s configuration files is laborious

Keeping a document with the company’s rules requires dedication as new versions of the programming language or framework comes out. What engineering teams always neglect is that style guides get obsolete as people leave and join the company as well.

In small or mid-sized teams, when you hire opinionated engineers or fire senior developers, there are great chances that old discussions emerge again. As changing documentation is harder, linters’ configuration files differ from the predefined styles.

The same occurs with code documentation, which always ends up in the question: is it outdated documentation or an implementation bug?

Taking as an example the Airbnb JavaScript style guide, you can notice the documentation ships with some linters files already configured, which saves time and enforces consistency between the style guide documentation and how it performs the checks.

Faster ramp up of new-hires

If your company adopts a specific style guide with lots of custom rules for linters, then the ramp-up of new-hires is slower. The same may apply if different teams adopt different code conventions — which sounds against the idea of standardization of the code. The uniqueness of the settings increases the cognitive demand for new-hires during onboarding.

It means that adopting a community-maintained style guide helps scaling companies. In fact, I strongly recommend choosing a style guide before growing fast, as the coordination efforts of doing it afterward is significantly higher.

When you use the Airbnb, Standard, Idiomatic or whatever JavaScript style guide, the adherence to them will probably be higher than understanding and promoting your company’s own. Many of the hired engineers may already know the rules by heart, which accelerates the ramp up.

Feedback comes from the community

Prefer a community-based solution over a custom implementation. That’s what I believe is the best for companies I worked for, and what I always foster as part of SourceLevel’s culture.

Besides all the open-source benefits, the strongest point of adopting an open-source style guide is getting external feedback. Otherwise, the team relies only on their experience. The external view brings more value to the discussion.

If you the team really has strong arguments on a certain preference, the repository is open-source, and will accept a pull request or issue for discussing the matter. Of course, you can’t take the change as granted. But you definitively can collaborate with the community by sharing your opinions. And this is the kind of environment I want to promote.

Few exceptions

I think there are few exceptions to the rule. If your company is big enough, or maybe has created a framework that became very popular, then you might consider creating a public style guide. The community can help in the process, and it is a win-win.

This rule applies to big open-source projects as well. After establishing the rules for that project, it can be used by other people and companies around the world. However, if you don’t intend to put effort into maintaining it, I’d simply adopt an existing style guide.

The post Why you should adopt a community-maintained javascript style guide appeared first on SourceLevel.

On the Four Key DevOps Metrics, and why I measure them differently

George Guimarães — Fri, 06 Nov 2020 18:21:03 +0000

The four key DevOps metrics are an exciting set of measurements. They’re getting more and more relevant since the book Accelerate has been published. I firmly believe they’re essential for engineering teams seeking effectiveness and efficiency.

The Four key DevOps metrics are:

Lead Time for Change
Deploy Frequency
Change Failure Rate
Median Time to Restore (MTTR)

You can learn more about measuring it in Paul Duvall’s article or reading the 2019 Accelerate State of DevOps.

In this article, I want to explore why I like them and why we measure it slightly differently at Source Level. I’ll focus on the Lead Time for Change and Deploy Frequency, as I have more experience working with them.

Why are the Four Key DevOps Metrics essential for engineering teams?

The main goal of an engineering team is to add value for the customers through technology. Delivering value is abstract, and describing what it means is hard.

Sometimes value translates into a single line of code or even in a configuration change. But at other times, value translates into three months of 8-sized-team work. It happens because value perception is not attached to the work itself. It’s bound to what end-users extract from the running product.

That’s why measuring Story Points is useless. The team attributes a numeric value based on how much effort a given User Story or task requires. So, Story Points measures engineering performance based on their effort. And, as stated, it’s not related to the end-user perception of value.

However, Modern Software Development requires agility, and it doesn’t come from the nature of the work, such as painful features or slippery bugs. Agility comes from a smooth and sustainable process.

Some years ago, using agility and process in the same sentence would be hilarious. But agile’s core is to respond to change appropriately, and delivering more doing less is a consequence, not the goal.

That said, the Four Key DevOps metrics become the best allies of Tech Leads, Engineering Managers, and VPs of Engineering looking for efficiency. These roles require full attention to the big picture, and DevOps metrics give a perfect overview of how the system responds to daily activities.

Lead Time for Change and Deploy Frequency summarize the development pipeline, and any change in the process reflects on them. Managers should have them in their dashboards or any other accessible and accurate place.

Considerations on Lead Time for Change

Lead Time for Change measures how much time it takes for a code change to reach production. It’s crucial to measure up to production because staging or any other non-official environment won’t provide any value.

However, it raises an interesting discussion on feature toggles (or feature flags). Deploying a new feature is not the final step of the value chain if your team uses feature toggles or feature flags. Features must be enabled to add value, and so, I’d consider measuring not only Lead Time for Change but also the Lead Time to Value.

Another aspect I’d like to point is statistics. The recommendation is to calculate the Median Lead Time for Change. It outputs the value at the midpoint of a frequency distribution. In other words, half of the changes are equal or less the median. But how about the other half of the changes? They could take considerably more.

At SourceLevel, we’ve been more likely to use the 75th percentile as the default metric and keeping track of the 95th percentile as well. The 75th percentile is more comprehensive and realistic. In contrast, the 95th percentile gives us an idea of the small portion of work left out of the count and excludes the top 5% most time-consuming changes, as we consider them outliers.

Considerations on Deploy Frequency

Deploy Frequency measures how many times code changes reached production in a period. It can be measure as deploy per hour, per day, per week, per month, and so on.

It’s crucial to adjust the period properly. Saying a team deploys four times in a month sounds like a team deploys once in a week. But it may not be accurate. If the team deploys four times at the end of a month, it means the team has a different pace than deploying once a week.

If it’s usual for your team to have one or zero deploys per day, don’t measure it in days. Start measuring in weeks. Once the team starts deploying more than seven times a week, you can switch for days. Switch to shorter timeframes whenever possible, but always respecting the accuracy of the information.

In short

The Four Key DevOps Metrics are useful and give an overall idea of how the system responds to daily activities. Lead Time for Change and Deploy Frequency are much more price than Story Points and should be accessible and accurate.

Tech Leads, Engineering Managers, and VP of Engineering benefit from these metrics. Although more metrics are necessary to investigate problems, they are excellent indicators of the engineering process healthiness.

The post On the Four Key DevOps Metrics, and why I measure them differently appeared first on SourceLevel.

How do you measure engineering performance and process healthiness?

George Guimarães — Thu, 29 Oct 2020 19:07:59 +0000

Hey, folks!

The State of DevOps lists 2 metrics for Software Development: Deploy Frequency (Throughput), and Lead Time for Change. They are great for measuring the expected outcome of an Engineering Team.

I've seen managers attaching it to OKRs or used as the area KPIs. They are excellent indicators of performance for companies.

However, as an Engineering Manager, I don't see they're so useful for daily activities. They provide a high-level overview, but little visibility on what's going on in daily activities.

Managers usually need a more deep analysis of the process to improve the Lead Time and Deploy Frequency themselves.

For that task, I would go break Lead Time into smaller steps of the work, such as the Time from Commit to Open a Pull Request (assuming the company is doing code review), Time to Merge (from Opening to merging a Pull Request), and so on.

How do you track process healthiness and effectiveness in your company?

OKRs vs. KPIs: explanation with examples for Engineering Teams.

George Guimarães — Tue, 20 Oct 2020 22:04:32 +0000

OKRs vs. KPIs, what are the differences? That’s a common question I hear from managers of Engineering Teams. KPIs are more straightforward to explain than OKRs, which can be tricky and more complex. They don’t mean the same, although they are connected.

What are KPIs?

KPI stands for Key Performance Indicators. In other words, KPIs are a set of metrics that should give you an overview of the area or team’s performance. They need to be measurable and comparable.

If you look at many KPIs, they’re not fulfilling their purpose. Organizations should select as few as possible so that tracking the progress is possible. Besides, all the significant changes in process and company goals should impact the numbers.

Then, managing an area or a team becomes more tangible: when the KPIs show poor performance, it’s time to act. The actions taken should reflect better numbers; otherwise, managers need to find another strategy and act differently.

What are OKRs?

OKRs stands for Objective Key Results. It’s a managing methodology that is popular in Silicon Valey. It’s been widely adopted by companies and startups at scale. It’s also seen as an alternative or complement for strategic planning.

OKRs’ primary purpose is to define Objectives that must align with the business and a set of Key Results. Each Key Result needs to have a measurable number as the goal and a limit date. The limit date means that the goal should be achieved within that time frame.

What’s the difference between KPIs and OKRs?

KPIs provide an overview of the area or team for managers, whereas OKRs must focus on its future. That said, it becomes clear that KPIs are a controlling tool, and OKRs intend to change an organization’s status quo and keep track of its progress by periodic assessment meetings.

It’s crucial to have that in mind when choosing the KPIs and the OKRs. Otherwise, they become useless and can even play against business goals. Let’s see some examples of them.

Examples of OKRs and KPIs for Engineering Teams

Below there is a list of KPIs examples. However, you can find more examples in my article Software Engineering KPIs: how to choose the best fitting metrics.

Deploy Frequency : it’s a sum of the number of deploys made by day, week, or month, depending on the organization’s need. It shows how frequently the team — or area — delivers value to the final user.
Time to Merge : it measures the number of days a Pull Request remains open or under review. You can find an average or, as I prefer, see the 75th percentile of all pull requests closed in a given period. It shows how performant is the team.
Time to Recover : how much time does the application is inaccessible after an error? This metric tells a lot about how engineering teams respond to failures.

When thinking of OKRs, Engineering Teams may consider Objectives such as:

Improving the Time to Market of new features
Lessen the Churn rate of the product

Here are some possible Key Results for Improving the Time to Market of new features :

Increase the Deploy Frequency to 37/week until . Assuming you deploy less than 37 times a week currently, increasing the frequency means you’re delivering more. Delivering more is crucial to achieving a more competitive Time to Market.
Reduce the 75th percentile of Time to Merge to 3 days until . Instead of the Median or Average, I prefer using the 75th percentile for Key Results. In other words, it means that 75% of the Pull Requests must be merged up to 3 days after they were opened. The team can achieve it by opening lighter pull requests or engaging in collaborators’ pull requests instead of working on a new work item.

For the second object, Lessen the Churn rate of the product , let’s assume you know the primary source of churn comes due to a high number of errors in the application. Then, the Key Results could include:

Reduce the number of Technical Debts to 15 until . Let’s say you have 30 Technical Debts currently. It’s a 50% improvement. There are many chances that improving the codebase will positively affect the churn rate and help in the KR of the previous Objective. The cleaner the code, the better it embraces the changes. It means you can reduce the churn and also achieve better Time to Market by gardening your codebase.
Reduce the Mean Time to Recover to Recover (MTTR) to 3 minutes until . The team needs to monitor application outages, connectivity problems, 503 errors, and other failures to ensure end-users’ experience is not impacted with more than 3 minutes of instability.

In short,

KPIs and OKRs are not the same. They have different purposes. KPIs aim to give managers an overview of how the team or area is working, whereas OKRs focus on providing the team a direction and then tracking its progress.

I presented some examples of KPIs and OKRs for Engineering Teams to illustrate the difference. SourceLevel provides lots of metrics, which may include your KPIs. Check out our Analytics feature, or schedule a demo with me.

If you need to define KPIs for your team, I am giving away a 30-min consultation meeting. Schedule a demo so that we can discuss your specific needs.

The post OKRs vs. KPIs: explanation with examples for Engineering Teams. appeared first on SourceLevel.

Succeeding in a Tech Leadership position — The first 90 days

George Guimarães — Mon, 21 Sep 2020 17:01:08 +0000

Recently, I read a blog post titled VPE and CTO — The first 90 days. It’s a brief article in which James Turnbull shows a mind map with four areas that “every new technical leader needs to, at least, think about and explore when starting at a new organization.”

Although the author doesn’t detail how to tackle each of them, the mind map itself is very valuable. It organizes information, and make explicit the many responsibilities of tech leadership positions.

The areas are Company, Product, Technical, and Humans. For each one of them, he proposes to categorize the subtopics in:

No action needed — everything is cool.
This needs work — let’s work out what we need to do and when.
There’s nothing here, and should that worry me?

In my experience, many Software Engineers feel unassisted in the process of dealing with a Tech Lead position, or when becoming a CTO. Therefore, the exercise sounded like an excellent idea to me.

In addition to James’ contribution, I brought some pieces of advice for who is transitioning or starting to a new tech leadership position.

I condensed a bit of my experience and some useful resources I’ve shared recently. I am sure they will help you somehow to succeed in a tech leadership position, at least in the first 90 days.

The Company Area

The company area subdivides into the following items:

What are we committing to?
Our metrics
Company metrics
Mission
Budget/Runway

The mission and the answer to “what are we committing to?” are crucial pieces of information. If you have a clear understanding of them, designing an engineering roadmap gets more natural, and less risky.

The engineering roadmap is an outcome of what I call “establishing a technical vision” for the team. It gives people a north. It must align with the company metrics, and fit in the budget.

It’s part of your job to define a technical vision, and manage the team to comply with it. It applies to all tech leaders, from experienced Developers to CTO.

If you’re a Tech Lead, there are chances you don’t have a budget for you to manage on your own. However, it’s as essential as it is for a higher position to understand your limits. I’ve already written some advice for Software Engineers transitioning to Tech Lead position. It may complement this article.

The technical decisions in the engineering roadmap may require external tools, like SourceLevel, software licenses, or equipment, to cite a few. They cost money. Even though you’re not passing the credit card, you’re an active participant in the process.

The Product Area

Heads of Product are concerned about what engineers develop, whereas Heads of Engineering’s sphere of responsibility is on “how”.

However, CTOs and Tech Leads need to be aware of what’s coming in the future. Otherwise, they won’t be able to provide an effective infrastructure to achieve the business goals.

Jame’s chart lists in the Planning, Input, and Output topics some interesting aspects that will lead to a better understanding of the product, and consecutively bringing more results.

The last three topics can be more challenging for new leaders. They require abilities yet not well developed, like reporting to senior management and dealing with processes.

If you have no idea what to report to senior management, check out what Lucas Colucci wrote a cunning article on the matter: How to report metrics to C-Level executives?

In addition to Throughput by type of work (bug, feature, chore, etc), you may add the Time from Commit to Deploy, which measures how long a commit takes to reach production.

You may have other metrics to drill down that numbers. The chart cites Cycle Time, although I prefer to measure the Time to Review, Time to First Commit, and many other metrics I explained in an article called 50 Shades of Lead Time.

The Human Area

The human area is probably the most difficult area of the chart to deal with. It’s complex and very comprehensive. As a recent leader, it’s vital to pay attention to the people that make up your team.

If you look thoroughly, you probably will figure out very simple decisions you can make to add value to their daily activities. Those quick wins help you to build social capital and respectful leadership.

I consider 1-on-1’s and Mentoring the two main topics that the chart relates to the human area. In the webinar 1:1s e Gestão de Equipes de Tecnologia (currently only available in Portuguese), I talked with Know Your Team’s CTO Daniel Lopes about both subjects.

The Technical Area

Managing the technical aspects of your new job probably is the part you’re most comfortable with, although it doesn’t mean it will be easy.

Scaling the codebase at a sustainable pace is hard. Sooner or later there will be a pile of technical debts that needs prioritization.

It’s fundamental to keep the track and tackle using an appropriate strategy. Otherwise, the team may get demotivated, the system gets messy and delivers take more than usual.

To avoid that kind of problem, you can use static analysis tools, also known as linters. Linters are great. They can enforce style guides, avoid security issues, and find code smells, among other things.

SourceLevel offers code review automation and metrics. Check how you can help tech leads and engineer managers / CTOs to deliver faster with quality.

How Metrics Lead to Effective Sprint Retrospectives

George Guimarães — Fri, 04 Sep 2020 18:12:54 +0000

In the Atlassian’s playbook, it states that Sprint Retrospective’s goal is to identify how to improve teamwork by reflecting on what worked, what didn’t, and why. Usually, the meeting consists of brainstorming what the team did well and what the team needs to do better.

During my journey, I’ve been in many of these meetings, sometimes participating, sometimes facilitating. I’ve noticed that writing down post-its is painful for some people. When everybody is speaking, engineers get stuck and can’t think of anything else during the meeting.

Then, right after the meeting is over, many situations and problems emerge. They would like to discuss those topics. However, they wait a couple of weeks for the next meeting. Rarely they remember the issue. I don’t think the meeting gets as effective as it could.

To increase effectiveness, I think managers should promote a culture in which people raise their hands when the problems occur. They should discuss as soon and fresh in their mind as possible.

If the issue needs a more extended discussion, the team can add it to Evernote, Notion, GitHub Gist, or any other shareable document. A couple of sentences telling the problem with a little context would do the job.

The team can do the same as what did well. More post-its may pop up during the meeting. Then, the Scrum Master or facilitator can read the document.

When the meeting starts, the team already has a list of essential items to discuss. It saves time and makes it more productive.

So, how can teams make Sprint Retrospective meetings even more effective?

Using Pull Request Lead Time & Throughput in Sprint Retrospectives

I’ve been talking about Pull Request Lead Time and Throughput for a while. If you’re familiar with the term Cycle Time, I recommend reading my article on why I rather use Lead Time instead of Cycle Time.

To be explicit, in this article, Pull Request Lead Time measures how many days a Pull Request takes to be merged, and Throughput is the number of merged Pull Requests.

These two metrics can truly help teams to understand how they are performing. Besides, teams benefit from them when they look for the variability along the weeks. The data bring valuable insights and allow teams to question their decisions.

Let me explain through an example.

Lead Time and Throughput – Chart provided by SourceLevel

I took this chart as an example. Let’s say we’re back in January 2020. We’ve started the Sprint Retrospective creating post-its for the team’s issues and appraisals during the last weeks of December 2019.

We’ve discussed a bit, and now someone shares the chart above to analyze the Pull Request Lead Time and Throughput.

If the sprint is two-weeks long, we could use the metrics to ask ourselves questions like this:

Why did we merge less Pull Requests in the week of Dec 2nd compared to Nov 25th?
What did we do in the week of Nov 25th that allowed us to merge more Pull Requests than the last few Sprints?
Why did the Lead Time increase in the week of Dec 2nd?

Asking this kind of question is fundamental. However, nobody should point fingers or blame other people. That’s not healthy nor effective.

I would expect answers like this:

We merged less Pull Requests in the last week because we had a massive refactoring going on.
In the Week of Nov 25th, we fixed several small bug fixes that popped up after deploying a problematic feature in the week before.
The Lead Time increased because everyone held their reviews, and we made a task force to merge the refactoring.

The numbers themselves don’t matter as much as the story behind them. Now it’s time to ask what are learnings from the last iteration. Here are some alternatives to address those issues:

Should we avoid massive refactorings because they slow down everyone? Should we make it part of the team’s guidelines?
Deploying complex features may lead to a higher escaped bugs rate. How to minimize the impact? Should we invest more effort into testing? Should we deliver smaller changes? Should we adopt a more incremental approach? Should we consider a gradual rollout?
Was the task force an effective strategy? Maybe fewer people working on the refactoring at the same time ships it faster.

As you can see, the data allow teams to derive relevant aspects for the Retrospective. The discussion is a tremendous source of knowledge and should generate insights for improving the development flow.

In short

Sprint Retrospectives can be more effective using metrics. The team can extract valuable knowledge by questioning the Pull Request Lead Time and Throughput. Indeed, other metrics are useful, as well.

The crucial point is to use the metrics to distill tacit knowledge and formalize it. What went well and what went wrong should promote changes in the process, feed guidelines, and motivate new practices.

That’s what an effective Sprint Retrospective is about, in my opinion.

The post How Metrics Lead to Effective Sprint Retrospectives appeared first on SourceLevel.

3 Classic Books for Tech Leads (or those aspiring to be)

George Guimarães — Wed, 12 Aug 2020 18:04:01 +0000

Books are the best resource for sharing knowledge in a not-assisted way. They go deep into a topic, or more briefly over a bunch of them. Although, as a Software Engineer, I learned a lot from blog posts, tweets, and conference talks, it was books that prepared me for the Tech Lead role.

For this article, I created a list of 3 classic books I consider essential for whose transitioning — or desires to transition — to a leadership position in technology.

Books can teach us many things. It’s familiar to us to feel like there’re too many things to learn and too little time to absorb it. We tend to get anxious and overloaded with the content easily. Relax.

Take your time on reading, and take your time practicing it. Theory and practice converge at some point, and you realize you’ve been a leader for quite some time.

It’s been quite a while since their publication. So, you may notice some old stories during your reading. However, I think they are classic, and as any other book in this category, they aged very well.

That said, let’s go to the list.

1. Peopleware: Productive Projects and Teams

The book was first published in 1987. The authors, Tom DeMarco and Timothy R. Lister talk about managing people through lots of stories. The stories may sound outdated and unlikely to happen nowadays, but their essence is still valid.

Many things stated by the two authors are kind of obvious, like how expensive the turnover is, that we should include the team in the hiring process and that methodologies are restrictive.

It’s a fantastic opportunity to recapitulate those topics; some were written more than 30 years ago.
Peopleware was the first book that I recall to mention the need for a sociological view, not only a technical one over projects and teams. As such, Tom and Timothy’s theories are still valid.

Among the theories, I how they state the importance of peer view (or Code Review, in our case), and that knowledge workers, like software engineers, are continuously working their soft and hard skills.

The theories go through the traps of software engineering productivity, how office influences how we work, the impact of quality in our work, and much more.

2. Driving technical change: Why People on Your Team Don’t Act on Good Ideas, and How to Convince Them They Should

If you’re new on persuasion and influence, the Driving Technical Change book is an excellent start. It’s full of anecdotes that surface how to gain skeptics buy-in for your ideas. It was published in 2010 and written by Terrence Ryan.

The content works pretty well for an introductory approach on how to expose your ideas, find allies, and, most importantly, check whether your proposals are plausible.

The book also lists some categories of skepticism, which was very valuable to me. I could better understand how the team responded to my inclinations and how I responded to others.
This awareness is an excellent start for those wanting to manage large teams.

I confess the book is not very practical, and that may be a disadvantage against it. However, it was very relevant when I read it for the first time, and I strongly suggest you read this short (~200 pages) book even though it won’t answer all your questions or tell you exactly what to do.

3. The Mythical Man-Month: Essays on Software Engineering

If you thought Peopleware was old because it was published in the late 80s, well, my next recommendation is 15 years earlier.

The Mythical Man-Month by Fred Brooks was first published in 1975 (almost 50 years ago!) and covers software engineering and management topics.

The author creates the concept of man-month. It’s the unit for the amount of work of a person in a month. Then, he defends that adding more people to a late project makes it later. It’s known as Brook’s law.

It relates to the latest and hottest discussions in the agile world about “coordination costs”: when you increase the number of people in a group, organizing work gets harder.
Thus, it requires more meetings and management efforts to align everybody, which delays the software development even more.

It’s an inspiring book, a classic one, which, in a certain way, is still modern and worth the reading.

Software Engineering KPIs: how to choose the best fitting metrics

George Guimarães — Fri, 17 Jul 2020 16:07:15 +0000

Software Engineering KPIs (Key Performance Indicators) are measurable values that indicate the progress of engineering teams’ performance towards business objectives. Therefore, they need to be consistent, broad enough to consider everyone’s effort, and, most importantly, measurable.

As they should represent the team’s or area’s work, it’s crucial to pick the right metrics to measure. Otherwise, metrics are useless.

It’s a common mistake I’ve seen in the industry to measure productivity by the number of lines of code (LOC), the number of commits, or even the number of deploys. Don’t get me wrong, measuring the number of deploys may be a good fit, but it should fulfill a purpose (which may not relate to productivity).

Usually, blog posts about KPIs concentrate lots of metrics, but few of those correlate the metrics with real objectives. So, in this article, I tried something different. I selected 5 Engineering KPIs metrics and then listed candidate objectives for them.

Time from Commit to Deploy

Finding the elapsed time from an engineer’s first commit in a branch to when that very same commit reaches production is the easiest way to measure the whole development flow. It’s easy because it can be automated by looking to git’s commits history.

Many managers look for User Story Lead Time, which means they look for the time spent from adding a card to the “backlog” to the time the card reached “done.”

Besides, just because a card is in the “backlog” column of a Kanban board doesn’t mean it’s ready for development. It may miss requirements, layout specifications, and test instructions, for instance.

That’s why they give you an overview of the product’s flow. But, as engineers, we are usually more concerned about improving the development flow, which starts in “doing” and ends in “done” (deployed in production).

So, this metric can relate to many objectives, here I name a few:

Reduce the time to market of new features
Reduce waste and rework
Reduce the Cost of Delay
Maximize engineering efficiency

Deploy Frequency

The deployment step is at the end of the value perceived by the customer (be it an internal ). So, it is indeed related to productivity, as each release concludes a job.

However, the number of delivered features doesn’t relate precisely with business objectives. The goals established for a quarter are usually more abstract, for instance, reducing the time to market for new features.

That said, instead of using Deploy Frequency for measuring the team’s productivity, it could take part in measuring objectives as such:

Reduce the time to market of new features
Improve the responsiveness to failures and outages due to bugs
Mitigate security issues

Code Coverage

Code Coverage measures the portion of code sustained under automated tests. The higher the coverage, the better.

It is an excellent tool for indicating the code’s quality. Also, it could report the progress of the following objectives:

Increase the product/platform stability
Reduce churn (in case there is evidence churn relates to a buggy product)
Scale the technology area
Reduce the time to market (as performing manual tests take more time)
Reduce costs in the long run

Pull Request metrics

Collaborations review the code of their peer before it gets merged into the main branch. The conversation ignited by this practice is a piece of vital information for measuring collaboration and engagement.

Pull Request metrics are a set of measurements extracted from pull requests. Below I picked a few of them:

Time to Review: how much time do engineers take to open and merge a pull request?
Time to First Comment: how much time do pull requests take to receive the first comment?
Number of Comments: how many comments do pull requests receive?
Cross Team Collaboration: do teams review other team’s pull requests?

And here is a list of objectives pull request metrics can assist in measuring:

Spread knowledge through the team
Introduce a Code Ownership Culture
Lessen the ramp-up curve for junior engineers
Create a harmonious and ever-learning environment

Focus on urgent bugs

I’ve seen many companies implementing the objective of achieving “zero bugs” for the next quarter. The KPI would be the number of bugs found in production (by the team or by the end-user).

This approach is flawed, first of all, because it’s near impossible to eliminate all bugs. We’re humans, and bugs happen. Secondly, we can’t compare a cosmetic bug with a bug preventing the user from paying an order. In third place, if you only measure the number of bugs quantitatively, people tend not to register them. No bugs filled means goal achieved.

Measuring if the focus is on working on critical bugs can prevent such misunderstandings. For that purpose, you can combine metrics, like the number of urgent bugs (that are more severe and impactful than an agreed level), the time the team took to fix, and if the fix was definitive.

Of course, you can add more metrics. I strongly suggest having complementary metrics to ensure that the team takes no shortcuts to mask metrics values.

Some objectives focus on bugs benefit:

Improve the product value perception by the final-user
Reduce cost by focusing on adding more features (instead of fixing cosmetic or banal bugs)
Foster a harmonious and effective culture
Foster a collaborative culture between engineers and Quality Assurance personnel

In short

There are plenty of articles out there that list relevant Engineering KPIs with metrics. However, it’s prevalent to see managers picking the wrong engineering metrics, which ends up in an undesired behavior of the team members.

That’s why Engineering KPIs must align with business goals. It gives direction to the team to game on it. So, managers must wisely choose which of the KPI measurements make sense for the context of their team.

In practice, managers define KPIs for already-established objectives. In this article, I listed possible objectives for each KPI metric to exercise in the opposite direction.

The idea behind the article is to help you to check whether a KPI measurement is a good fit or not.

This practice is particularly useful while reading an article with lots of Engineering KPIs metrics. It helps me a lot, and I hope It can help you as well.

If you’re looking for automated Code Review or Pull Request Metrics, check out SourceLevel features for more information, or start the 14-days trial. Our tool is free for teams up to 5 members.

The post Software Engineering KPIs: how to choose the best fitting metrics appeared first on SourceLevel.

How to use Engineering Metrics in Daily Stand-up Meetings

George Guimarães — Wed, 10 Jun 2020 22:07:14 +0000

Software engineering metrics help daily stand-up meetings to be more productive for the team. They can become tedious or irrelevant for many developers when they frequently exceed the fifteen minutes time box or even sound like a work report.

Engineers that work together usually know what each one is working at the moment. There’s no need to go over through details. However, some topics should surface during the meeting.

After reading the 5 Strategies to Boost Software Deliveries in Any Company, a free e-book we released this week in a partnership with Wesley Zapellini, I realized I hadn’t shared my thoughts on such an important meeting as the daily stand-up.

However, it’s been a while since I got myself distant from the Scrum’s script for the meeting. In my experience, I learned that daily stand-ups should are crucial for tactical coordination for the day.

To achieve that, when I worked with Plataformatec’s team, I used to organize the meeting in three different moments.

Firstly, I would share the screen and have everyone looking to the Kanban board and reading it from right to left. The more at right the work items are, the closer they are from bringing value to the business. In other words, I would focus the tactical coordination on moving the cards forward in the board as much as possible, starting by the ones at the end of the pipeline.

Wesley explores this subject in a more depth way in the e-book. Besides, he brings studies showing how limiting the work in progress impacts the deploy frequency.

In a second moment, I would ask for blocks or other relevant issues for the day. After some discussion, we’d rearrange the plan if needed.

Finally, we’d also look at some engineering metrics. It was essential practice for improving the process continuously. As we were aware of the metric’s history, we could take action before raising any considerable problem.

When measuring the pull request lead time daily, it is easy to see its variation over time. By plotting a control chart, long-living pull requests become visible, and the team should focus on finding alternatives to merge (or even close) it.

Pull Request Control Chart

The screenshot is from SourceLevel, which integrates with GitHub and GitLab and visually organizes the information. However, you can use any tool that supports plotting a control chart. By looking at the yellow dots, the team can spot pull requests that are taking more time than desired. If this particular pull request is taking more time than desired, the engineers can discuss how to handle it.

Here are examples of actions that can emerge from discussions:

close the pull request because the change is not relevant anymore
ask for a review for the team because there was none so far
merge done but neglected pull requests
schedule a meeting with someone from another team to discuss technical details that are holding the pull request

If the team needs further investigation of a specific pull request, it would be nice to have a single table with work in progress from all repositories.

List of Pull Requests needing attention

In SourceLevel, you can quickly find Pull Requests needing attention from all monitored repositories. It provides a digested version of the chart. The list prioritizes by the pull request lead time, idleness, and size/impact of the changes. It’s useful for technical and non-technical people to find what needs to be done.

The benefits include a decrease in Lead Time and an increase of Throughput. Besides, it may increase the deploy frequency, although it depends on other variables, like how automated is the CI/CD, or internal policies.

My main point is that engineering metrics can benefit the whole team, not only the manager. And it’s crucial to keep an eye on the charts regularly. Otherwise, it doesn’t help.

Looking for the Lead Time (or Time to Review) of each Pull Request is the most critical practice for daily stand-ups. The quantitative, easy to understand, and actionable. Many healthy discussions emerge, and at the same time, all derivated tasks are very straightforward.

Have you tried to use engineering metrics in stand-up meetings? Share your ideas in the comments below!

The post How to use Engineering Metrics in Daily Stand-up Meetings appeared first on SourceLevel.

DEV Community: George Guimarães

3 better Engineering Metrics uses for Managers

Metrics to motivate the team

Metrics as indicators

Experimentation and innovation

In short

Three signs your team needs Engineering Metrics ASAP

#1 — You don’t know in which step of the product delivery bottleneck is.

#2 — You don’t have any metric to report upwards.

#3 — You don’t know whether the team is making progress towards desired outcomes.

Do you want to know more about Engineering Metrics?

3 Benefits of DevOps Metrics within Engineering Teams

Benefit #1 — They consider only the engineering scope.

Benefit #2 — It promotes collaboration and alignment.

Benefit #3 — Improving the system is more effective.

You’ll probably need more metrics later.

In short

Why you should adopt a community-maintained javascript style guide

Maintaining a style guide is time-consuming

Maintaining the linter’s configuration files is laborious

Faster ramp up of new-hires

Feedback comes from the community

Few exceptions

On the Four Key DevOps Metrics, and why I measure them differently

Why are the Four Key DevOps Metrics essential for engineering teams?

Considerations on Lead Time for Change

Considerations on Deploy Frequency

In short

How do you measure engineering performance and process healthiness?

OKRs vs. KPIs: explanation with examples for Engineering Teams.

What are KPIs?

What are OKRs?

What’s the difference between KPIs and OKRs?

Examples of OKRs and KPIs for Engineering Teams

In short,

Succeeding in a Tech Leadership position — The first 90 days

The Company Area

The Product Area

The Human Area

The Technical Area

Further Readings

How Metrics Lead to Effective Sprint Retrospectives

Using Pull Request Lead Time & Throughput in Sprint Retrospectives

In short

3 Classic Books for Tech Leads (or those aspiring to be)

1. Peopleware: Productive Projects and Teams

2. Driving technical change: Why People on Your Team Don’t Act on Good Ideas, and How to Convince Them They Should

3. The Mythical Man-Month: Essays on Software Engineering

Software Engineering KPIs: how to choose the best fitting metrics

Time from Commit to Deploy

Deploy Frequency

Code Coverage

Pull Request metrics

Focus on urgent bugs

In short

How to use Engineering Metrics in Daily Stand-up Meetings