DEV Community: CarmoBCosta

Engineering Efficiency: A Guide to Decision-Making During The Recession

CarmoBCosta — Thu, 16 Mar 2023 13:44:35 +0000

It’s a hard pill to swallow, but any engineering leader who’s been through it knows: Growth hurts efficiency.

As a former VP of Engineering to a team of 200+, I’m familiar with the growing pains and the twists and turns on the road to success.

In the beginning, having limited resources can actually be a good thing. It forces you to focus and prioritize, and software engineers are experts at this.

However, as the business booms, your org grows, and you gain more resources, staying focused and being effective doesn’t come as easy.

At that point, to increase productivity within your organization, there are two options:

Hire more people
Use existing resources more efficiently.

Today we’re going with option 2 by looking into ways engineering leaders can do more with less, maintain focus as their organization grows, and continuously improve and operate autonomously.

This is a blog post I’ve been meaning to write for a while, but now, more than ever, it’s time to put my ideas on the page.

TL;DR:

To foster autonomous improvement, team leaders should focus on honing processes and measuring engineering velocity to identify bottlenecks.

For managers of managers, the focus should be on coordinating investment and validating goals to ensure alignment across the organization.

It’s also important to have a mix of T-shaped and I-shaped skillsets in teams and reliable data to communicate goals and achieve alignment.

Two Ways Team Leads Can Foster Autonomous Improvement

First, let’s look at a few things Team Leads can do to maximize focus and alignment.

1. Focus on What Matters Most

Assess where you’re investing your time.

If you were a woodcutter, you’d spend some of your time sharpening your axe, and the rest of your time chopping down trees. Chopping down trees is how you make your living. But you’ll be much better at chopping wood if you keep your axe nice and sharp.

It’s the same for software engineering leaders. Spending time delivering your product is how your organization keeps the lights on. But don’t forget to sharpen your axe. Invest some of your time in honing your processes, and your whole team can become more efficient.

For example, you could commit to investing 20% of your team’s time on getting rid of tech debt, which will greatly benefit your throughput.

Setting goals is important at this stage. What do you expect to achieve through investing time into making your team more effective? For instance, you might decide that you want your cycle time to be 20% faster. And to work towards this goal, you need to start measuring your engineering velocity.

👉 Three Ways to Measure Velocity

The more traditional one, use Jira to analyze your ticket velocity, focusing on story points and burn down charts.
Rely on your gut feeling. You might understand your team and your data better than anyone. Some things won’t “feel” right to you. Focus on these areas, and you might find some bottlenecks.
Use specialized engineering software, like Athenian. Read our full guide to choosing the right engineering metrics platform for you.

2. Look For Bottlenecks

No matter how you measure velocity, your end goal should be to identify what you can do to help your team accelerate.

As the team lead, you’re responsible for maximizing acceleration. Once you’ve measured your engineering velocity, you need to start looking for bottlenecks.

👉 You can do this by:

Using dedicated engineering software to get total visibility of your data.
Surveying your staff, asking where they encounter problems and barriers.

💡 Ideally, you’ll use a combination of software and surveys to find your bottlenecks. But you can still run a survey if you don’t have access to the software.

How to Survey Your Engineering Team:

1. Ask the team where they feel they’re getting blocked. Ask each member of the team, use 1-1s for this, to bring two or three concrete examples, validate if they are having recency bias.
2. The team can then consolidate this feedback into an impact list. This is a rundown of common problems, arranged in order of what has the greatest impact on engineering velocity.

As soon as you have your impact list, arrange a team meeting in which to talk about the top three items on the list. The goal of this meeting is to ensure everyone is aligned on those top three priorities.
Finally, pick the one activity that the whole team agrees is the most feasible, and estimate the potential impact this will have on improving engineering velocity.

💡 When you’re comparing feasibility (effort) to potential impact, you can start drilling down into what you really need to do to achieve your goals, and on the activities that will help your team get there. You are doing technical roadmap here, treat it as roadmap and plan it (you can use a framework like RICE).

In short, you need to treat the development process as if you’re dealing with a fresh initiative or project:

Find opportunities.
Define your priorities.
Figure out how to achieve them.

Two Ways Managers of Managers Can Foster Autonomous Improvement

Managers of managers can maximize focus and alignment in much the same way as team leads. But their role should be more about calibrating and challenging processes.

They need to ask team leads what they’re willing to do and why. They need to promote platforms or enablement clusters.

1. Coordinate and Approve the Teams’ Investment

Imagine you’re managing two teams. One team tells you that they need to invest 30% in improving engineering velocity. The other team claims they need to invest 40% in improving engineering velocity.

The problem is that if everyone is investing time in improving engineering velocity, then who’s going to deliver the software?

Sharpening your axe is important, but only until a certain point (forgive my pun).

When you’re tweaking effectiveness, you’re not delivering. And the opposite can be true, too – teams can spend so much time on delivering that their tech debt will pile up, and their velocity will suffer.

So pay attention to where your teams are investing their time, and work on balancing their investments.

You need to coordinate this investment. Your teams should prioritize the organization's velocity as a whole rather than their own individual engineering velocities.

👉 Managers of managers need to foster autonomy among their teams. The team leads should define the investment, but their managers should coordinate between them to ensure that the whole organization’s working towards a shared goal.

💡 Let me stop you here!! Yes, I’m saying your leaders are responsibile for delivering value and optimizing the process, and that you need to help them understand that they are responsible for that, not you.

2. Validate Goals

As I explained above, software engineering teams should define their own velocity improvement goals. Then, the manager of managers ensures that these goals are for the good of the whole organization rather than just for the team.

How can you do this? By asking team leads to come back to you with their drafted objective by a specific date. That way, you’ll have time to discuss synergies to ensure alignment.

👉 Schedule a meeting where you and the team leaders will discuss everything together – opportunities, synergies, and alignment.

Managers of managers will have unique insights into what other people in the organization are doing. So look for synergies not only within your group but also within other groups. Remember that you’re responsible for managing the org, not just your teams.

Following this meeting, you might find that you need to set up a dedicated platform/devex group to help coordinate your goals (namely when you get to +80 engineers). Alternatively, you might establish a transversal guild.

💡 Some software engineering organizations have an in-house guild of Dev Experience. Rather than acting as an official team in itself, it instead features multiple people from multiple teams who help work towards goal alignment.

Coordinating across your organization helps remove bottlenecks. For example, you might end up with one team working on improving the build or testing system, which will actively benefit another team in improving their throughput. So in coordinating your teams, you’re ultimately avoiding waste.

A Note on Letting People Go

It’s the unavoidable elephant in the room. 🐘

So let’s address it.

When you take this transversal approach to maximizing throughput, you might start to recognize different mindsets across your teams. And in the current economic environment (I’m writing this in March 2023), letting people go is an unavoidable discussion.

Some team members will have specialized skills best suited for performing specific tasks. Others will have a more broad skillset. They’re good generalists who’ll work well at any task they’re set.

Think back to your startup mentality. Ask yourself: “Who are the people in my teams who will help across multiple areas, and who are the people who can only help me in one area?

You want your teams to be full of high-energy, committed people with T-shaped skillsets rather than I-shaped skillsets. This is extremely important the smaller your company is.

But there’s still room for people with highly specialized I-shaped skill sets. For example, that one extremely talented engineer who works in teams across your entire organization, helping every team they work with improve their efficiency? They have space in your organization, as they’re positively impacting everyone else’s work.

The key point here is that you need to carefully re-think your organization over this new reality, and see who helps you the most to deliver value and sharpen the axe.

Final Thoughts

Too many people seem to forget that autonomy comes with responsibility.

Team leads are responsible for making decisions locally. Managers are responsible for coordinating efforts globally.

Fostering autonomous improvement is a continuous job. During recessions, you need to be laser-focused when you have to do more with less. But staying lean, addressing bottlenecks, and reducing waste should always be top of mind.

To guarantee alignment across your entire organization, you need to be able to sell people on your goals. For this, you’ll need reliable data and visible insights that will help you communicate what you’re trying to achieve and why in a consistent way.

Athenian can give you end-to-end visibility of your software delivery pipeline. So you’ll know what you need to prioritize, improve velocity and align teams with company goals.Request a demo today!

Scaling Your Team From 5 to 250 Engineers: A Complete Guide

CarmoBCosta — Thu, 06 Oct 2022 14:44:15 +0000

‍Picture a map of a city. If it’s zoomed in enough, you’ll have fine-grained visibility of street names, shops, traffic, and restaurants.

But as you zoom out, the more challenging it becomes to find helpful information.

As an engineering leader, the same thing happens when your organization grows.

When your engineering team is small, you have visibility into all the nitty-gritty details of your software delivery pipeline. You know which tickets are moving, you know the CI runs, and you probably even know which PRs are pending.

But, as your business and team grow, the challenges you face evolve. And you’re pulled further and further away from those valuable details.

The more you zoom out on the map, the less visibility you have. Fortunately, you can always zoom in when needed.

The trick is to know where to zoom in.

Today we will delve into the challenges you will face as your organization scales from 5 to 250 engineers by focusing on three dimensions of your software development pipeline:

🏎 Velocity
⭐️ Quality
🎯 Outcome

We will also cover:

👉 The engineering metrics you should look at within these three dimensions

👉 Important considerations before adopting engineering metrics tools

👉 Tips for making engineering metrics a success

Click here to download the printable one-pager of this guide!

Why Do We Look At Velocity, Quality, and Outcome?

Visibility is essential for organizations of all sizes.

But how do you go from that high-level visibility of a larger team, back to the details you had when your org. was smaller?

If there’s a challenge or a bottleneck, would you know where to zoom in to find the information you need?

Maybe it’s the CI system that’s slowing you down. Or maybe it’s the bug backlog that you’re struggling to work through. It could even be that your code base is slowing you down, and you need to pay down some tech debt.

You need to know where your challenges are, so you know what you have to do to fix them.

As engineering leaders, we have to zoom in and zoom out all the time.

If you have access to the right data, you’ll know exactly where to zoom in.

There are three dimensions that every engineering organization should focus on:

🏎 Velocity: How fast are we moving? Where are the bottlenecks in the team and process?

⭐️ Quality: What’s the quality of the software you deliver? How do you respond to quality issues? When we’re talking about “quality”, we’re referring specifically to quality that impacts the end user – think of bugs, incidents, downtime, etc.

🎯 Outcome: What are we shipping? Where are we investing our efforts when it comes to building new features, paying down tech debt, fixing bugs, maintenance, and R&D? How are we actually dividing our efforts? Most importantly, how is the impacting our customers?

Let’s look at some of the patterns organizations face as they scale from 5 to 250 engineers from the lens of velocity, quality, and outcome.

🏎 Velocity

Many engineering leaders believe that when you double your team, you will double the output. But the truth is that velocity - or the speed of shipping code - doesn’t increase linearly as a team scales.

If you’re growing fast, you’ll have a constant onboarding challenge, which takes a toll on speed. At the same time, your newly promoted leaders might lack the needed experience to support your organization's growth.

Here’s what actually happens to your velocity as you scale:

🌱 From 5 to 20 Engineers

At this stage, companies are trying to get to Product-market fit (PMF), which puts a lot of pressure on delivery. This “move/scale fast, break things” mentality can lead to companies keeping a steady speed but accumulating a lot of tech debt.

When the team is small, we must build a proper foundation.

💡 Recommendations:

Avoid tech debt on everything that doesn’t allow you to scale in the future.
Create some core processes to fix the developer experience continuously, but focus on market fit.
Bring in a Head of Engineering who talks with their team leaders and understands blockers. They also needs to start thinking about the product and organization architecture to support the next growth level.

🪴 From 20 to 100 Engineers

When you start zooming out on the map, you lose visibility of the details.

Managers start having managers, and the new people you bring in will slow you down because it takes time to grasp the context of your product and organization.

If you didn’t set the right foundation you can lose 25-50% of effectiveness when your team gets this big.

You'll also begin to have misalignments, especially as your lines of communication increase:

💡 Recommendations:

Bring expertise from the outside and promote internal people simultaneously so you can pair them.
Create a team dedicated to developer experience while you keep teams on their toes.
Double down on processes and goal definition - this is where you want to build the foundations for a 1000 dev team.
Thoughtfully plan the structure of product teams (value delivery) and platform teams (product foundations and enablement) for what's coming afterward [Team Topologies]

🌳 From 100 to 250+ Engineers

As your company scales, your velocity may stagger due to a lack of visibility on bottlenecks and constant onboarding.

You'll continuously reshape the organization but lack clarity and visibility on where to act.

You are looking at the city 10.000 km in the air, which means you can’t see where bottlenecks are. This will slow you down.

💡 Recommendations:

Train managers and tech leads to delegate so they can focus on team performance and people management.
Systematically monitor organizational bottlenecks, gather with your leaders to discuss potential solutions, and define explicit goals for improving velocity.
Push responsibility to the teams. They are the ones that can solve problems while you focus on minimizing interactions and communication lines between groups.

⭐️ Quality

Most of us fall into “the build trap”.

As we grow, we focus on delivering new features, eventually reaching a point where tech debt is too big for us to tame, which leads to a product rewrite.

If you’re growing fast, you’ll have to constantly deal with tech debt. Granted, most engineers will complain about tech debt, but they will never bring the consequential data to the table. This makes it hard to know where you are.

🌱 From 5 to 20 Engineers

When your organization's just starting out, there are not enough customers that a bug or outage could impact. So you focus on speed instead of quality. This is essential as you race to PMF.

However, as your customer base increases, you need to start digging into the quality of the product.

💡 Recommendations:

Define a systematic process to capture the issues you’re facing and keep critical problems under control.
Prioritize for bugs that block any potential growth. Your engineers will deal with a lot of bugs. It comes with the territory. But rather than getting bogged down in fixing them all at once, you should prioritize the bugs that might directly influence your product's growth.
Promote good traceability and logging standards from the get-go. This way you'll always have a good idea of the sort of problems your engineers are facing, so you'll always know where to make improvements.

🪴 From 20 to 100 Engineers

PMF is evident and the size/value of your customer base increases - which means you’ll rapidly increase your nº of bugs and critical incidents, making the teams feel underwater.

If you lose focus, the product you built might no longer work for this market size.

💡 Recommendations:

Have a strong bug backlog management, ensuring you fix all critical and high issues quickly.
Remember that issue priority is based on frequency and severity, so you need to fix issues you know will hit you multiple times early on.

🌳 From 100 to 250 Engineers

All the challenges you faced when scaling from 20 to 100 engineers will continue to grow in line with your organization.

If you didn’t address issues as they came, some of your bugs will now come from the accumulation of tech debt, negatively impacting your MTTR.

In addition, teams will start pushing bugs to one another if you don't put good teams and architectural structures in place (Conway's law).

“Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization's communication structure.” — Melvin E. Conway

💡 Recommendations:

Create solid team structures to avoid teams pushing bugs between each other and slowing things down.
Platform teams should be taming MTTR. A low MTTR is what allows you to keep moving fast.
Another critical metric you need to pay attention to is the change failure rate. 4. Commit to regular reviews to ensure you don’t just track these metrics - you act on them too.

🎯 Outcome

As engineering leaders, we’re constantly making trade-offs.

Seasoned leaders are masters at deciding where to invest efforts, and, as your organization grows, becoming more opinionated on the trade-offs between velocity, quality, and customer impact, is key.

If you’re growing fast, you’ll have to constantly deal with the features vs. quality tradeoff, which takes a toll on the capacity to deliver on the product. Again, being intentional in making these decisions will play a critical role in your success.

🌱 From 5 to 20 Engineers

Shipping is king at this stage.

Your roadmap is directly impacted by the conversations you’re having with your customers, so you’ll invest most of your efforts in new features.

However, the technical decisions made when you were <5 engineers will start to slow you down.

💡 Recommendations:

Focus on new features until you achieve PMF, but pay attention to investment in bugs, developer efficiency, and architecture decisions that block the growth of your product.
As for everything else, let it burn, but keep an eye on it.

🪴 From 20 to 100 Engineers

You now have a live product with a mature client base. It's time to move your efforts from primarily creating new features to honing in on internal processes.

Start being pragmatic about where you invest your time.

Remember that business value is everything that creates value for the business. So you will focus more and more on fixing bugs and cleaning tech debt, and less on new features.

💡 Recommendations:

Ensure teams define and report their level of investment.
Standardize processes across teams for you to understand effort investment at a high level.
Don’t be punitive if teams don’t meet the investment level you were expecting. Understand why and help course-correct.

🌳 From 100 to 250 Engineers

Your visibility will continue to decrease at this stage, so trust is critical.

You might realize one quarter later, that the team wasn’t focused on delivering the features you were expecting. Teams will start to invest more time on bugs, and you'll get frustrated when the push for new features slows down.

Investment is the mastermind behind outcomes. Of course, you'll need to commit to internal processes and work quality, but you need to be decisive on time allocation and push a lot of these decisions to the teams.

💡 Recommendations:

Think of investment as a strategic component, define quarterly/yearly goals, and adapt based on insights from regular meetings.
Support and educate the teams on investment decisions. You also need to coordinate investment happening across the org and promote transparency with the other directors.
Now that we’ve seen the challenges of scaling your engineering org, let’s dive into what engineering leaders can do to prevent these scenarios.

👉 It starts with data!

Velocity, Quality, and Outcome Metrics

We’ve seen what happens to your velocity, quality, and outcome as your team grows. What can you do to keep these things in check?

The key is to understand how different parts of your software delivery pipeline function to identify the levers you can pull for smooth sailing as you scale.

Modern software engineering organizations focus on delivering value through continuous delivery, integration, and improvement.

To make sure that this is happening, you need to keep a close eye on some key metrics.

🏎 Velocity Metrics

Velocity metrics help you access the agility and leanness of your team so you can understand how to improve the speed and direction of developing software across all the stages of the software delivery pipeline (WIP → Review → Merge → Release → Deploy).

Lead Time

Lead Time tells you how long it takes for your team to go from a ticket being in progress to a released PR, allowing you to detect early bottlenecks and resolve them on the spot.

Release Frequency

Release Frequency tells you how often you release value to your customers. Together with Lead Time, Release Frequency measures software delivery performance tempo.

Released PRs

The number of pull requests released during a period is an important metric to consider in combination with your PR Cycle Time. A decreasing Lead Time might be linked to a decrease in the number of released PRs.

PR Cycle Time

The elapsed time between the 1st commit of a PR and the code being used in production. Compared to the lead time, the PR cycle time focuses on just the code pipeline section.

⭐️ Quality Metrics

Quality metrics should be as close as possible to the end-user experience of your software.

Quality needs to be proxied by metrics that give an indication of the quality of the software delivered, such as no. of bugs by priority and bug fixing ratio.

Without quality metrics, it’s easy to find yourself going at a faster speed, or increasing volume, for the sake of your end-user experience.

Bugs Fixing Ratio

The Bugs Fixing Ratio is the ratio of bugs fixed to resolved during the time period selected.

Bugs Raised by Priority

Bugs Raised by Priority helps you get more granular information into the bugs raised by severity level. Did the team raise a lot of bugs last month? Were they all critical issues? Correlating this with other metrics, might help you identify the root-cause of the issue.

Mean Time to Restore by Priority

As an engineering leader, you want to make sure that your team is fixing customer-impacting issues the fastest.

🎯 Outcome Metrics

Outcome links engineering work to business goals, and it helps align the whole organization.

When building software, it's easy to lose sight of the business objectives we impact.

For instance, refactoring a major component without changing its behavior - how does that relate to our end user?

Other times we’re delivering a new feature in an application, such as the ability to buy a book online with one click instead of four - which can be mapped to the business objectives.

Throughput by work type

A throughput by work type can be looked at from the number of tickets fixed or the number of PRs released. It helps you understand if you’re truly aligning your engineering org with business goals.

Consider This Before Adopting Engineering Metrics

End-to-end visibility of your software delivery pipeline is key. By linking ticketing (like Jira) and code (Github), you can truly understand the leading indicator of a KPI and the root cause of an issue.
DORA metrics are great, but they are flat. Imagine you see an increase in PR Cycle Time: What do you do? How do you understand the root causes so you can solve the issue? You need to dig deeper.
Context matters: DORA metrics and fancy benchmarks show that high-performing teams should release >X times a day. But in our example above, the PR cycle time issue was linked to engagement, so it might be positive. Here’s a deep dive into why content matters.
Trying to continuously improve by tracking individual metrics will have the opposite effect and hurt your developer culture. If you’re monitoring throughput per developer, you will have a high throughput per developer. But will it have an impact on your business? Probably not. 👉 Instead, focus on getting visibility on your teams and coaching your engineering managers and team leads to build high-performing teams.

You Have Data, Now What?

There are two kinds of engineering leaders: the ones that have continuous improvement processes in place (like OKRs), and the ones that are starting on this journey.

Whether you have these processes in place or not, to continuously improve you have to have clear goals and visibility. You also need to know where you will invest your time.

If you already have a clear Northstar in place, we suggest you:

Identify one metric to improve (don’t make it the most critical metric, but one that could be easily actionable, for example, review time.
Add engineering metrics to your current processes, consistently look at data, and discuss how to improve
Once you’ve made this a habit, look into more meaningful, even if challenging, metrics.
Create a continuous feedback and improvement loop that cascades down to the teams.

If you don't use OKRs or other goal-setting methods, we recommend a more expansive look into these steps.

Engineering Leaders have two responsibilities:

👉 Improve the developer experience

👉 Deliver impact to the end-user

These two are infinitely linked, and, when done successfully, will help you create a culture of continuous improvement. We believe this can be done with the right metrics and data.

If you’re a new engineering leader, ask for help by reaching out to more experienced leaders. You’ll find that the great ones are those who are constantly questioning their own methods and looking for ways to improve.

Finally, don’t be afraid of the challenges ahead. You can feel lost when the map is zoomed out, but with the right tools, the right people, and the right mindset, you can take your product and company to new heights, and have a happy team while doing so.

Athenian can help you scale your engineering organization by giving you end-to-end visibility of your software delivery pipeline. So you’ll know where to zoom in to improve velocity and quality and align teams with company priorities. Find out more!

What is Engineering Productivity & How Do We Measure It?

CarmoBCosta — Tue, 16 Aug 2022 15:28:09 +0000

In the world of software engineering, it seems like nobody can agree on what productivity means. I’ve seen many engineering leaders struggle with its definition, many with conflicting opinions on what engineering productivity means and what metrics to look at.

But understanding productivity is key to unlocking the potential of many engineering organizations, and can ultimately help us ship better products.

Today I’ll go through the reasons why I believe we’ve struggled to define productivity in our industry. I’ll also offer my views on it, who owns it, how to measure it, and how engineering teams can improve it.

Here’s my quick and dirty definition of engineering productivity:

💡 Engineering productivity is the capacity to measure if your engineering teams are doing the right things, with the right investment, to achieve a specific business outcome.

It’s important to note that productivity is often driven by expectations, and its meaning might depend on where you sit in the company, as well as the maturity stage of the company.

For example, the board and CEO are typically more concerned with time to market (velocity), while the CTO knows that there are other important indicators of productivity, such as quality and maintainability of the product and the developer experience.

Understanding The Different Components of Productivity

Before we dive in 👉 Since there seems to be a recurring theme in our industry where different meanings are given to the same word, here are my definitions of some of the terms you’ll see throughout the article:

🛠 Throughput: The nº of items you ship to production.

🛠 Velocity: The speed at which those items go through the pipeline. Velocity helps your to predict the time it takes to deliver something.

🛠 Quality: How satisfying your delivery is based on customer expectations. There are multiple criteria you can have to understand product quality: nº of bugs, ease of use, performance, etc.

🛠 Capacity: The nº of devs available to work on a specific project or initiative.

🛠 Outputs: Activities you’ve been executing. 👉 (e.g. deploying a feature)

🛠 Outcomes: The impact of those activities. 👉 (e.g. I want this feature to increase my user NPS by 20%)

🛠 Investment: The money you want to put into delivering a specific outcome to the market or specific output to the market. 👉 (nº of people/salary) x time to deliver

🛠 Business Value: The value for the business of delivering a specific outcome. Ideally measured in $. (eg. If I get a SOC2 compliance certificate I’m able to open a market that is worth x millions of dollars.)

🛠 Effectiveness: How fast you’re shipping things and how their quality is perceived. In this case, it’s not about technical quality, it’s about having the right impact on the market. Effectiveness combines quality, investment, and business value.

To me, effectiveness is deeply connected to productivity. Understanding this helps to answer: “Am I able to deliver the right thing at the right time, with the right investment?”

Ok, now that we have those definitions out of the way, let’s dive in.

Defining Software Engineering Productivity

Engineering productivity is hard to define because there is no set framework for it. And when you don’t have a framework, meaning will stem from expectations.

Typically in the early days of an organization, it feels like you’re moving fast. But as you scale, the distance between leaders and developers increases. Like zooming out on a map, you lose visibility, and your communication channels, dependencies, and issues increase.

Your team grows, and things start moving slower.

Slower compared to what?

If you are comparing a team with no customers, 10 devs, and no live product, to a team with 100 devs and 2,000 customers, it may seem like things have slowed down. But this is normal. As soon as you have more customers, you have more incidents and responsibilities. You need to care about value delivered to customers, not only about shipping new features, so you slow down.

“But if I added 10 engineers to my teams, shouldn’t I be moving 10x faster?”

Unless you’ve perfectly set up your organization for the smoothest onboarding of new hires, you have the best engineering managers, and your customers love everything about your product, no, you won’t be moving faster.

🚘 You buy a car. It’s a good car, and you’re a good driver, so it takes you 20min to get to the beach. One day, you have a kid, and your kid sits in the backseat as you drive to the beach. Suddenly you’re moving slower. But it’s not the engine, it’s not the car, it’s not that you suddenly became a bad driver: it’s the kid. The kid requires more attention and safety. You still get to the beach, but as you adapt to having a miniature version of you in the back seat, your ride there will take longer than those blissful 20 minutes. Your customers are the kid in the car, they increase your business value, but they also demand attention and caution.

Why & How Productivity Expectations Change

We’ve established that slowing down as you scale is normal - but you have a board to report to, and the pressure coming from the top is all about time to market.

But if you only look at velocity, you’re not capturing productivity. If I’m running around in circles super fast, am I being productive?

As you scale, the CEO gets further away from the CTO, and the CTO gets further away from the engineering teams. And you start to see something across the multiple levels of the organization:

🏎 The higher-up you are, the more productivity is connected to speed and revenue.

💪 The lower-down you are, the more productivity is connected to your effectiveness as a developer (which is inherently linked to developer experience).

TL;DR: The bigger the org, the harder it is to define productivity.

But it gets more complex. The expectations of productivity change as you grow, where you sit in the organization, and where the market is.

If the market has shifted its focus from growing, to cost optimization, the first thing that’s going to come from the board is “how can we make our engineering team more effective?”.

Effectiveness is a synonym for productivity. But because of the current market, the board/CEO will look at effectiveness differently. To them, it’s about having more output with fewer people.

This translates to 👉 “How can I maintain my effectiveness or productivity while cutting costs?”

This is different from the growth mindset of “how can I add more people to the team to do more, even if we go a little bit slower per contributor”.

The first statement is about reducing costs, the second is about improving the overall throughput.

In sum, how you measure productivity also depends a lot on your goals.

How to Measure Engineering Productivity

One of the first things you have to do as an organization is to define your productivity framework - this will allow you to consistently observe increases and decreases in productivity.

Productivity is the throughput of the engineering organization, and it’s measured by how much business value the engineering org can achieve in a given timeframe.

💡 Your framework is the consistent system by which you measure these productivity indicators, so they can be easily compared throughout your reporting periods (i.e., translating all values to dollars and repeating the process every quarter).

Whenever the market changes, the team grows, or the conditions change, you want to have a metric that allows you to compare how you were six months ago, or 16 months ago, from today.

Across industries and organizations, people, contexts, and cultures are completely different. So the way we define and measure productivity has to be too.

Productivity is one of the pulses of the organization. 🫀 Measuring it allows engineering organizations to course correct their processes and organization as they adapt and grow.

‍But how do you measure it?

The way that I measure productivity is by aligning business outcomes with the product team and attributing value to those outcomes.

This is sometimes called Business Value Framework. Based on this framework, the Product organization estimates the potential value for the company of an outcome/output and engineering estimates how much effort and time it would take to get there.

One key thing is to understand what the organization is prioritizing at the moment, which will be dependent on where the market moves.

A great way to think about it is by looking at the Business Value Matrix:

Once the high-level priority is clear, specific business goals are created.

With that in mind, we can see productivity as “how much effort/investment did I exercise to achieve a specific business goal”.

Initially, you will measure this by looking back, but as you start to gather more knowledge, patterns will arise, and you’ll be able to make predictions on future investments and potential impact. It will always be hard to be precise, but you want to use comparisons and have a common understanding across the org to generate alignment.

Once you’ve defined your metrics for productivity, you need to measure consistently (I suggest doing this quarterly). At that point, you can start planning ahead by asking the right questions, such as:

What are the outcomes we want to achieve in the next quarter?
What capacity do the multiple teams have/need to achieve them?
What is the business value of each of those outcomes?
What initiatives/activities will lead to these outcomes?

By looking at those four things: Outcomes, capacity, business value, and activities - we have the elements needed to measure productivity and keep an eye on it as the quarter goes by.

Each quarter we can look back and see if our productivity has improved, and adjust accordingly. We can also tweak for velocity and quality since everything is aligned with our business goals.

An Example of Measuring Productivity

To me, the easiest way to measure productivity with the metrics like the ones we have at Athenian is to:

1. Identify the quarterly initiatives or the epics that the teams are working on and their potential value.

2. Understand the overall lead time of your initiatives

3. Dig into the data to understand what impacted your productivity

Here’s a great example of how to do this 👉 How to Use Data to Improve MTTR of customer-facing bugs.

Once you have a clear understanding of all this, you can take action in terms of being effective in keeping lead time under check or reducing lead time for epics.

I like this method because you can measure whether you’re getting better or worse in terms of productivity without impacting quality.

To be clear: Athenian can help you measure the effectiveness of your engineering teams, but to get a complete understanding of productivity you will need to link this to your business goals, get customer feedback, and more.

There is an art to building organizations, and lot’s of luck. When you are able to combine the two, you will have a product that customers love, and that your teams are proud to be building.

In the end as a CTO or VP of Engineering, you’re responsible for bringing this up to the executive team and senior leadership team, to ensure you create alignment on how to measure the impact of your org.

‍

Ready to dig into the data? Find out more about how Athenian can help!

Hiring Software Engineers? Engineering Leaders Share Their Biggest Lessons

CarmoBCosta — Mon, 08 Aug 2022 15:41:00 +0000

‍“If you look at an organization and study the root cause of everything that’s good and bad, it’s going to be the people. You have to hire good people in order to be successful.” - Erik Bernhardsson‍

We’ve spent many hours talking to engineering leaders on our Developing Leadership podcast. They are industry experts who have built high-performing teams and generously share their knowledge with other engineering leaders.

It’s rare to go by an episode where our guests don’t mention the importance of hiring. So we’ve gathered some of the key lessons on building engineering teams, such as:

💡 Key mindsets to adopt and key questions to ask during interviews.

💡 The sort of skill sets you should prioritize when recruiting.

💡 Moments when it might be best not to hire new people (and what you should do instead).

This is not a step-by-step guide to hiring software engineers, but if you keep these nuggets of advice close by, you’ll improve your chances of success as you scale your engineering organization.

Set expectations, quickly.

“What does it mean to successfully bring someone on board into my organization? It means that people come in with the right set of expectations. They know the job they’re getting into, and there won’t be any surprises.” - Dana Lawson, SVP of Engineering at Netlify

“I tell engineering candidates early on in the process: This is matchmaking. We want you to feel comfortable with the organization you’re joining. So we tell candidates who we are, how we operate, and how we think.” - Andrew Conner, Co-Founder and Engineering Leader at Levels

Hiring is like speed-dating. You’ll meet many potential matches in a short period of time, and you’ll only spend an hour (at best) with each candidate.

A short interview will never tell you everything about a person, but the opposite is also true. Most interviewees have no idea what sort of organization they’re joining. You expect nothing but honesty from candidates, and they should expect the same from you.

It pays to be transparent as possible to find the perfect match for your organization. Be open and direct – tell them exactly who you are, how you work, and what you expect from them in this role. Be specific. As you build your engineering teams, this will set your organization up for success.

Invest in technical conversations.

“I love a technical exercise that’s also a technical conversation. The most important thing for any software development team to have is collaboration and cooperation. How are they going to demonstrate how they contributed to a project effectively? And can they effectively communicate any impediments they encountered?”- Dana Lawson
‍
Technical exercises are an integral part of most software engineering interviews. They help candidates prove that they can walk the walk and talk the talk.

But does the technical exercise in your job interview accurately reflect the job you’re hiring for? Because if it doesn’t, you might be testing for the wrong set of skills, and your new hire will feel disillusioned about their new job.

So, make your technical exercises technical conversations by:

Deciding which skills you’re looking for and what you wish to understand about your candidates.
Asking the right questions – perhaps about exciting projects they’ve been involved in – and inviting candidates to detail how they overcame challenges.

Technical conversations will help you hire a person rather than a skillset. As Dana Lawson says: “I’d rather have a strong engineer that everybody wants to work with than the most badass engineer of all time who everybody thinks is a jerk.”

So what qualities should you look for in your candidates? That brings us to our next lesson.

Look for people with a “default-to-action” mentality.

“Software engineering organizations need people who see something and say, ‘this is probably not right. Maybe I can do something about it’. Hiring is critical for this.” - Andrew Conner

If your team is full of people who always do things by the book, your organization won’t reach its full potential. Inefficient or ineffective ways of doing things become habitual, and people become unhappy.

You individuals who take initiative.

Look for people with a default-to-action mentality. Skilled at seeing the bigger picture and unafraid of trying new things, their drive and initiative can take your organization to the next level. These will be the future leaders of your org, so watch out for them.

Of course, not everyone on your team will have this default-to-action mindset. So what other skill sets should you look out for?

Build diverse teams.

“I wanted a team of some junior people, some senior people, and some people from different backgrounds. A diverse team.” - Dana Lawson

“We wanted to work with people who we’ve never worked with before when we started putting together our founding team.”- Sam Alba, Co-founder of Dagger

Building a successful software engineering team requires diversity in skill sets and backgrounds.

It helps to have individuals who take initiative, but you also need good communicators. You want people who can spot and communicate bottlenecks, so engineering leaders can help the whole team move forward. And, of course, you want highly technical engineers.

You can get this essential mix of insight, instinct, collaboration, and communication by aiming to hire as diverse and inclusive a team as possible. So look for candidates from a range of backgrounds and with different levels of experience.

Create a great system, and you’ll have great engineers.

“Great engineers don’t make great teams. It’s the other way around. You become a great engineer by working as part of a great team. Being part of a great team allows you to absorb the best techniques learned from hundreds of thousands of engineering hours. You become a high-performing engineer just by being around other amazing engineers.” - Charity Majors, Co-founder, and CTO of honeycomb.io
‍
Software engineers work in incredibly complex systems. If any part of that system is slow or understaffed, or if anyone’s spread too thin or working overtime, you’ll have a working environment where it’s difficult for anyone to flourish.

Hiring a talented new engineer won’t magically make an underperforming team productive. You need to start by building an engineering-first company culture.

Create a system in which any software engineer you hire is bound to thrive, and your organization will thrive too.

Ask yourself if hiring more people is the best solution.

“I’m a fan of throwing money at problems. If you can throw money at a scaling problem to make it go away, great! Do it. Money is cheaper than time. But we’re not even doing that intelligently. It’s easier for you to get approval to hire ten engineers for $2 million than it is to get approval to spend $500k on a tool that would replace ten engineers.” - Charity Majors

Scaling is hard. You want your organization to grow cost-effectively and sustainably. And you might think that hiring more software engineers will solve all your problems, but adding more people won’t make your organization more productive.

A lot of organizations get to the point where they're hiring new software engineers just to fix specific problems. Unfortunately, this will lead to individuals working on a small selection of unrewarding tasks that give them no opportunity to be creative.

You’ll get the worst of both worlds: a stressed and frustrated workforce with no corresponding rise in output.

Take the time to assess the problems you’re facing, and explore all possible solutions before you resort to hiring someone new. Sometimes investing in the right tools can make your scaling much easier to manage than simply expanding your workforce.

Spend More Time Hiring.

"Unless you're spending like 40, 50%, of your time on it, don't tell me it's hard to recruit, just do more of it. To me, this is the number one lesson in recruiting: You just have to put a lot of time into it." - Erik Bernhardsson, Founder, Modal Labs

Why would we tell you to "spend more time hiring" right after we advised you only to hire when you need to? Because when it comes to scaling your engineering org, you need to give it your all.

Modal Labs founder Erik Bernhardsson speaks to many engineering leaders who've struggled with recruiting. The one thing all of these leaders have in common? They spend 5% or less of their time recruiting. Which is not enough.

Software engineering leaders serious about building a fantastic team should spend 40 – 50% of their time recruiting.

But wouldn't it be great if you already had a handful of supremely talented individuals in mind for every role you need to fill?

Adopt an “always recruiting” mindset.

"I don't particularly like to use Twitter. But it's an excellent funnel for a certain set of things. For every person I end up talking to, I ask myself: 'I'm slowly recruiting this person. What are they excellent at?' For nearly every single person I interact with, that's how I think." - Jason Warner

Some engineering leaders will build relationships with people – on social media, networking, consulting, and so on. And they adopt a mindset where any of these individuals could be the person to fast-track their organization's growth.

"I may never hire that person," Jason says, "But that's how you have to look at life when you're interacting with people."

We love this mindset, and as an engineering leader, you should too!

If it’s not working out, move on as soon as you can.

“You should hire great people. But you should also ensure that you correct the problem quickly when you hire the wrong person. That’s tough. It’s not easy to fire people, but it’s an essential part of the job.”- Sam Lambert, CEO of PlanetScale

What sort of people do you want working with your organization? You want people who share your vision and are committed to working towards your company’s goals.

It might take some time for your new hires to find their feet. But you’ll learn pretty soon what sort of individuals they are. Are they the type who always has an excuse for why they underperform? Or are they the type who constantly tries new things, gives their best, and works tirelessly to overcome an issue?

The latter are keepers. The former? Not so much. And if they linger around for too long, they could bring your whole team down.

As an engineering leader, a massive part of your job is to hold people accountable and revitalize the parts of your organization that aren't doing well.

And when the time comes to let someone go, remember: it’s on you. Because it usually means your hiring process wasn’t foolproof.

Learn from the best engineering leaders

Every two weeks we bring industry experts to our podcast to talk about the ins and outs of managing software teams at different stages of their organization. Leaders from GitHub, DataDog, honeycomb.io, neo4j, and more!

We’re passionate about developing exceptional software engineering leaders, and our discussions with the best and brightest can help you on your journey.

If that sounds cool, subscribe to the Developing Leadership podcast!

If you want to find out how the right visibility tool can help scale your organization, let’s chat!

How Engineering Metrics Can Bridge the Gap Between Developers & Leaders

CarmoBCosta — Mon, 18 Jul 2022 09:45:37 +0000

As a software developer, the products you build can have world-changing potential, which makes it particularly frustrating when you hit a bump in the road – when something stands in the way of getting your product to market.

Bottlenecks and roadblocks are not always easy to identify, and communicating these challenges to engineering leads and other teams can be even more challenging when you’re in the trenches.

Developers are technical wizards committed to being the best at what they do. However, communication is one thing that many struggle with.

I’ve been a software developer, and I’ve managed software teams. So as a member of both tribes, I have a good understanding of this tension between builders and managers.

As software developers, our work might make sense to us. But how can we make it make sense to those in leadership who aren’t spending their day building?

Through visibility.

Data & Metrics Help Leadership Understand How Best to Help You

Making your software development processes visible through data and metrics doesn’t mean giving engineering leaders total control over your work. It means giving them the insights they need to make better decisions, so you can work together to address any bottlenecks you might be facing.

There are two ways to bring this sort of visibility to your work:

You can build an in-house dashboard to gather data and highlight challenges. But this takes time and needs to be continuously cared for.
You can use a tool that gives you end-to-end visibility, through data and metrics, of your entire software pipeline. (such as Athenian)

You can focus on doing what you do best with a visibility tool: building the product. Meanwhile, your manager can focus on doing what they do best: removing blockers to ensure the team is happy and maximizing impact on the end customers.

Too Much Data: A Code Coverage Cautionary Tale

Remember when code coverage was the very latest in software testing? I do. At first, it felt like many software engineers didn’t trust it. But eventually, it became an indispensable tool for all developers. Too indispensable...

Code coverage gave teams more information than they knew what to do with. It gave managers far more data than is necessary to make better business decisions. We were overusing it and fixating too much on benchmarks - until we finally found a balance.

I don’t want visibility tools to have the same journey. Managers should not demand “100% visibility” from their development teams. Instead, they should aim to gather the data and metrics that will help them answer and ask the right questions, without asking the team to spend time doing manual reports.

Visibility = The Right Context & The Right Questions

For visibility tools to have an impact, leaders must consider all the insights they gather through the lens and context of their business.

Ultimately, visibility tools are just that – tools. They enable managers to highlight the key metrics in their development processes, but it’s up to you how you use these metrics.

Software engineers sometimes feel like unsung heroes. When a lot of the work we do is invisible, it’s easy to feel underappreciated, and lack of visibility leads to questions like:

“Can’t we build that feature faster?”

This, of course, is the wrong question to ask.

Better questions might be: “Is my team facing any bottlenecks? What are they? What can I do to remove them? And how much time and money should I invest in removing these bottlenecks?”

Great Engineering Leaders take context into account when analyzing engineering metrics, and ask the right questions. This way, teams can dedicate their time to value-adding, impactful activities.

👉 Be extremely hard on the questions, and remember that you are not questioning people, you are questioning workflows, so that you can have meaningful conversation to improve (Pixar calls this braintrust).

With the right data and metrics, engineering leaders can identify patterns, communicate effectively with leaders in other functions, and work towards a culture of continuous improvement.

Ultimately, visibility tools can make software teams more productive, work more rewarding, and deliver more impact to end-users.

Want to talk about how visibility can make software engineering more productive? Let’s chat!

How to Improve Knowledge Sharing In Your Engineering Org. (and beyond)

CarmoBCosta — Fri, 08 Jul 2022 13:30:54 +0000

"Any organization that designs a system will produce a design whose structure is a copy of the organization's communication structure." - Melvin E. Conway.

As an engineering leader, a critical element of smooth scaling is having strong communication lines within your team and other departments.

Suppose we apply Conway's Law to the organizational behavior of engineering orgs. In that case, we can say that a product will result from how teams are structured and communicate. This means that the more effective you are at knowledge sharing, the better your business outcomes.

Let's look into why knowledge sharing is so critical, and which tools and tactics you can implement in your engineering org (and beyond) to improve it.

Why You Need To Nail Knowledge Sharing

Knowledge sharing is critical for engineering and other teams to continuously learn and improve. It also helps to:

👉 Drive Alignment: Sharing knowledge empowers teams to work towards achieving their goals according to the business mission and vision. This information is crucial to keep your teams working on impactful activities.

👉 Create Visibility: As your team and the company grow, it's not unusual to start losing visibility on the details. Ensuring a good knowledge exchange can prevent this from getting out of hand.

👉 Scale Smoothly: The number of communication lines increases as your team grows. Setting up processes and frameworks from the start helps you scale with ease.

The Risks of Flawed Knowledge Sharing

As an engineering leader, you have to understand the long-term consequences of not tackling something today. Putting your efforts into improvements takes away time from delivering things. But deprioritizing the dissemination of knowledge across teams can lead to complicated risks, which sooner or later will catch up with you.

The Bus Factor: If only one person or a set of people in your entire organization know about something, what will happen if this person is hit by a bus? Who will be able to perform what only that person knew? Of course, we are exaggerating here, but people leave, go on vacations, and the moment can be critical.

Working in Silos: Frontend, data, backend, QA teams, etc., all work towards the same goal: get the product to the end-user. At the same time, they all depend on each other to do their part. If there's a lack of communication at any step of the way, it can impact the final product and user experience.

Not Addressing Bottlenecks: When you know there's a bottleneck, you can't expect to solve the problem if you don't know what it is and why it's happening. Communication is key here to understand the whys and create strategies to solve them.

How to Improve Knowledge Sharing

So how can you mitigate these risks?

I've put together some best knowledge sharing practices between engineering teams, people, and other departments in your organization. Some might be familiar. Some might be new. I recommend you follow at least 5 of these regularly.

How to improve knowledge sharing within engineering teams

👉 Documentation: I can't say this enough: Document your decisions and discussions. They can serve you well in the future when trying to solve a similar problem. I recommend creating decision records, using comments on Jira, and sharing code on slack channels.

👉 Retrospectives: Set up regular meetings to reflect on what worked and didn't. What was the impact on your organization? To facilitate, share your Athenian metrics dashboards with everyone so they can analyze and bring questions to the table.

A quick way of doing so is by sharing a unique link to the dashboards you want to discuss. Each link will save the information for the specific metrics and timeline so you and your team can always quickly go back to it.

👉 Symposiums: Set up an event with a regular cadence where engineers can bring a problem they face(d) to discuss potential solutions.

👉 Stand-ups: Have engineers share accomplishments and the challenges and blockers for the day.

👉 Request for Proposal (RFP): Allow anyone to create a proposal to change something: process, code, architecture. It invites your team to be creative.

👉 Code Review and Pair Programming: Encourage your team to submit their codes under review or even pair them from time to time to push discussions.

👉 Meetups: Set up meetings with people in the same role or domain but from different teams. This way, they can discuss common topics that might be a pain for others.

👉 Post Mortem: Document the team's reflection on what went wrong, how it could've been avoided, and how to improve for the future if the situation happens again. [more]

How to improve knowledge sharing with other teams

I recently asked engineering leaders what they do to improve communication with other departments. Here are some of my favorites answers:

👉 Technical Demos: They're fun and effectively spread information between teams.

👉 Weekly Tech Deep-dives: This helps team members learn about new domains, onboard team members faster, and create a space to share knowledge and learn about specific or obscure systems parts.

👉 "Ask an Engineer": Engineering leaders have regular conversations with the customer support and sales teams to answer their questions about the product and what's happening in engineering.

👉 Create internal release notes with release notes for all stakeholders. Notes are sent via email + slack channel so people can reply in a thread if they have questions.

No matter what stage of the company you're at, nailing your communication and knowledge sharing will ultimately impact your business outcomes, scaling abilities, and the happiness of your teams.

Hopefully, you can walk away today with a sense of how impactful this can be and with a handful of tools that will set your organization up for success!

‍
Find out how Athenian can cultivate better communication within your engineering organization. Let's chat!

7 Fatal Flaws in The Software Engineering Industry

CarmoBCosta — Tue, 24 May 2022 16:02:28 +0000

Sometimes Jason Warner and I like to get on our podcast and vent about things that irk us in our industry, organizational structures, and engineering leadership. It’s frustrating to see some of the pattern behaviors in the tech industry and watch as they hurt future generations of engineering leaders.

The lack of pro engineering leaders, our obsession with cargo culting, and the wrong way of doing OKRs are some of the things we continuously talk about on Developing Leadership. So, I’ve decided to put some of these flaws on paper. To challenge engineering leaders to look inward at themselves, their teams, and their companies, and try to imagine a different way of doing things - without fear of experimentation.

1. The Lack of Professional Leadership in Tech

If we put engineering leaders into amateur, pro-ams, and pro categories, we have plenty of amateurs and pro-ams in our industry - but very few pros. There is a lack of professional engineering leadership throughout engineering organizations.

Why? The rate at which tech is growing leads to a massive demand for new engineering leadership positions. And since we have more amateurs than pros, we will have more amateurs training the next generation of engineering leaders.

If we assume that 80% of the current generation are pro-ams, and only 20% are pros, that means 80% are going to train the next generation. It’s unlikely that they’ll have a quality education, and this is what will hurt our industry the most.

2. The Way Engineering Leaders Learn

I’ve asked hundreds of engineering leaders, “what did you read and learn to get to where you’re at today?” And most will mention three books, and no matter how good they are, these books rarely apply to every company stage or type of leader.

The truth is that it’s easy to write content for amateurs or pro-ams, but writing for pros is highly nuanced. So there’s a point where we should be immersing ourselves in content that might not be specific to engineering leaders.

A book I highly recommend is “The Score Takes Care of Itself” by Bill Walsh, the famous 49ers football coach. It’s an excellent book. A key takeaway is that there are so many times when you need to make a decision, and it’s the art of the decision, not the science and the mechanics of the situation, that matter. That’s where the pros and the excellent leaders are made.

Another book we often mention on the podcast is "Simple Sabotage" by Rober M. Galford, Bon Frisch, and Cary Greene.

3. The Lack of Experimentation with Organizational Structures

No matter how mature an organization is, someone will slip in a new project or a new library, and sometimes it gets us into trouble, but it's what moves tech companies forward. We have a huge culture of experimenting with technology.

Sadly, however, there’s a lack of experimenting with organizational models, and by the time we decide to try something different in our organization, it often comes from cargo culting.

In software, we encourage an experimental mindset. We talk about short feedback loops and failing QA tests, and experiments happening all the time. But it’s the exact opposite with leadership. In leadership, you’re typically not allowed to make mistakes. You have to have all the answers.

This goes back to amateurs, pro-ams, and pros. Pros will never act like they have the answer. Pro-ams do. They say, “This is the way. This is the way because Google did it, or I read a blog post.” But a pro will say, “Our goal is to achieve this. We will try it this way because we need to optimize X, Y, and Z to achieve these goals. And if it doesn’t work, we throw it out.”

I shared some decision-making frameworks in a recent blog post which can help break away from this pattern.

4. OKRs

There are many conceptual things in OKRs that make sense, and we can call them alignment tools. Alignment tools work well in engineering, but traditional OKRs don’t. We all believe in setting goals, but OKRs and engineering don't go hand in hand. There’s also a lot time wasted around OKRs because they often lead to lengthy, pedantic discussions.

We can get pedantic over OKRs because we often don’t understand the philosophy and the soul of what this alignment tool is supposed to be doing. It should allow us to be on the same page about where we’re going, why we’re going there, who we’re doing it for and what time of horizon we’re doing it under. Instead, we argue about a K being worded like an O.

OKRs work pretty well for Sales and Marketing. But for Product, Design, and Engineering, not so much - because they take away the soul of these projects.

It’s tough to translate an OKR into software engineering, so some companies have adopted methods like Twilio’s “Draw The Owl,” where we have goals and time horizons, but we allow the middle to be messy and creative.

Not Being Able to Link Engineering Work to Business Impact

Engineering is where the rubber meets the road. It’s where most of the value in a company gets created. Still, so many great engineers and engineering leaders can’t see the impact of what they’re doing. If I could wave a magic wand, everyone in engineering would be able to see the effect of their work on the business.

KEEP READING

Why Context Matters When Analyzing Engineering Metrics

CarmoBCosta — Fri, 20 May 2022 13:10:09 +0000

Since the introduction of DORA metrics, engineering organizations have come a long way in understanding the value of data and professionalizing their work. However, engineering leaders still need to make the best decisions for their business and teams according to their context. Yes, there are industry benchmarks, but they don't cover your organization's unique context. And this makes all the difference, especially when you are scaling.

Today we will explore why context is critical when making decisions in growing organizations, even when you have metrics at your disposal. We will also look through the lens of three examples to help you on your journey.

What is Context?

Context is the frame of the organization, the set of unique circumstances that influence a business. For engineering leaders, your context is a combination of team structure knowledge, product knowledge and business goals knowledge.

Why does context matter?

If we think about DORA metrics, for a team to be considered an Elite Team, they must deploy multiple times a day. But what if you have a mobile team? They can't deploy several times a day because of app store limitations. Does it mean they are low performers?

If engineering leaders don't consider this context, they might push their team to be a "DORA high-performer" when the circumstances do not allow them to. This situation is far from being related to the team's capabilities. They will not get there and get frustrated with the demands. Or worse, you might ignore data and metrics that matter in their context.

Let's look at more consequences of not taking context into account when analyzing engineering metrics by looking at three practical examples.

Context I: The (normal) pain of growing

👉 Indicator: PR Cycle Time has increased in the last month.
💭 Without context: As an engineering leader, if your PR Cycle Time increases, you will agonize over it because it means you are slowing down.
💡 With context: Since the organization is growing, you are onboarding more engineers, meaning they're still adapting to the workflows and the product itself. They need time to ramp and feel comfortable on the codebase and stack.
🚀 Actions: Review the data in a month. Your PR cycle time should go back to normal if onboarding went well.

Context II: The grass is not (always) greener on the other side

👉 Indicator: Your Review Time is 2hrs higher than some “industry benchmarks.”
💭 Without context: You might think your organization is underperforming.
💡 With context: Your team is mostly made up of junior engineers, so your senior engineers are investing a lot of their time in code reviews to train the junior devs properly.
🚀 Actions: Understand the Review Time that works best for your team. There are plenty of reasons a longer code review process could benefit your context.

Context III: Fast, but not furious

KEEP READING

The 5 Things You Need to Measure in Your Software Delivery Pipeline

CarmoBCosta — Mon, 02 May 2022 16:58:33 +0000

If you're an engineering leader, in one way or another, you're building a product that's changing the world, and you're doing it with a team that continuously strives to be better at what they do.

But engineering is not a linear process. So, how do you ensure that you're not stumping growth with faulty processes?

The key is to understand how different parts of your software delivery pipeline (and beyond) are connected and the levers you can fix to harness that full potential.

Velocity, Volume, Engagement, Quality & Outcome: I believe these are the five things you need to measure in your software delivery pipeline. By zooming in on each one, you will be able to identify those critical levers for success. Let’s explore each of these metrics and what role they play.

🏎️ Velocity

Velocity is always a measure of time, usually an average, that allows you to assess the agility and leanness of your team across all the stages of the software delivery pipeline (WIP → Review → Merge → Release → Deploy).

Lead Time is a good example of a metric that tells you how fast you're moving from starting to work on code to releasing it in production.

Lead Time is a metric borrowed from lean thinking and manufacturing disciplines. It’s defined as the amount of time between the moment work begins until it’s delivered to the end-customer.

How shorter lead times impacts the team’s output performance:
🚀 Faster feedback loop with your users
🚀 Beating competition to market
🚀 Reducing risk in the deployment process by working in smaller batches

💡 In Google's 2019 DevOps research, the authors revealed that High-Performance Organisations have a Lead Time of under a week, and Elite organizations under a day.

Athenian allows you to quickly spot which segment of the pipeline adds the most time to total lead time. Generally speaking, this segment should then be tackled first.

📦 Volume

Volume helps you understand how much was done.

Volume is a difficult metric because a single unit of work, for example a pull request or a review, cannot be compared to one another. Maybe a one line bug fix was highly complex and took 10 hours, while 500 lines of code were easily added to a low risk/complexity component. But without aggregate volumes it is hard to understand the other type of metrics.

💡 If my Lead Time is 5 hours on average for 3 pull requests, that doesn't tell me much about the speed of our team, but if it is 5 hours for 300 pull requests, it starts painting a picture.

Volume is also key to understanding your processes. If a team releases 100 Pull Requests in a sprint and I see that only 20 were reviewed, volume is again important to help me understand how we work.

🔬 Engagement

Engagement is a measure of how many contributors are involved.

My favorite example is a growing team. As we add additional members to the team, does our volume of work increase while maintaining the same speed and quality? These are the kind of questions that can only be answered by knowing how many contributors were active during a period.

Understand how engaged developers are in the code review process.

⭐ Quality

Quality is a metric that needs to be proxied by metrics like # of bugs by priority that give an indication of the quality of the software delivered, you can also look at the Bugs Fixing Ratio and other subsequent metrics (like MTTR by bug priority).

💡 Quality metrics should be as close as possible to the end-user experience of your software.

Without quality metrics, it is easy to find yourself going at a faster speed, or increasing volume but if that is at the sake of delivering a worse user experience to your end-users. The trade-off between Quality, Velocity and Customer Impact is something I’ve written about here.

Ensure bugs are resolved in no time and set quantitative goals depending on the levels of priority.

🎯 Outcome

Outcome measures the business goals we had for the software we are delivering.

Often when we are building software it is easy to be removed from the business objectives we are impacting. For instance, refactoring a major component without changing its behavior, how does that really relate to our end user? Other times we are delivering a new feature in an application, for example, the ability to buy a book online with 1 click instead of 4, that can clearly map to the business objectives.

If you want specific teams to focus on shipping new features instead of paying tech debt, Athenian provides granular visibility.

💡 Outcome is sometimes hard to measure, but when you can, it helps align everyone.

Athenian uses the Athenian Insights Graph to automatically correlate output to outcome across the full delivery pipeline. Our platform is made for teams and doesn’t provide metrics on individual engineers.

With Athenian, organizations of all sizes can get end-to-end visibility into their software delivery pipeline, improve velocity and quality, and align teams with company priorities. Find out more!

Developing An Executive Mindset As An Engineering Leader

CarmoBCosta — Wed, 13 Apr 2022 15:22:19 +0000

There is a piece of advice that I share with every engineering leader I speak to. It changed my view on engineering leadership when I heard it from my podcast co-host and former GitHub CTO, Jason Warner:

💡 You have to think of yourself as your department and your department’s efficiencies, but it’s more important to think of yourself as an executive first. ‍

Thinking of yourself as an executive first means:

You see other executives in the company as your primary team.
You are not competing with different functions and departments.
You regularly meet with other executives (or managers) at your level.

Working together towards the same goals creates organizational alignment, so you can run your function incredibly well without being competitive with the other departments.

Ultimately, you'll create a healthier company culture through continuous alignment and open communication. Everybody wins.

The key is to start early on. Let’s explore how.
‍

Be an Executive First, and a Functional Leader Second

If you’re thinking about yourself as an executive first, you’re thinking about the business, the new LTV/CAC ratio, sales efficiencies, and all that’s going on in finance.

You don’t have to understand these things deeply, but you are interested when the CFO talks about cash flows at the exec meeting, your mind doesn’t wander to “we need to talk about database tuning.” Instead, think about those things second. Then you’ll care about the entirety of the business.

Almost everyone thinks about their division before they think about the company. As a result, their primary role is downward-looking, their secondary role is upward-looking, and their third role is a sideways looking role – at their exec teammates.

Try to invert that. Think about your partners and your teammates as the primary relationship. You have to operate in first team mode.

‍

Operate in “First Team Mode”

The healthiest exec groups and companies operate in first team mode.

Operating in first team or second team mode is all about a leader's mindset concerning their team, function, and executive group. It's all about alignment and knowing which team you are playing for first.

Your first team needs to be the exec group.

That means everyone is on the same page on things like:

What are the five things we need to get done this year?
What are the sub optimizations?
What are the things we likely won’t get done, and what do we worry about?

If you’re operating in second team mode, you’re not aligned. You’re out there working on your metrics. As a result, you don’t care exclusively about your organization, and tension will come through.

Operating in a first team mode is one way to create a healthier organizational culture.

‍

Run Your Function Well, But Don’t Make it Competitive

The next step is that your function runs incredibly well.

Running well does not mean that everything is green. Running well means that everyone in that room understands what’s happening – The risks, the trade-offs, the good, and the bad. You could have five red projects, but you’re fixing them, and two green projects that are critically important, so sales are still happy, and everything feels like it’s under control.

But you don’t make it competitive. You don’t say, “we’re better than sales or marketing.” That’s a well-run organization because everyone knows what’s happening. If everyone is operating in the same way, there can be no finger-pointing.

Adopt This Mindset Early On

If you’re an exec, you have to have one-on-ones with everyone on the exec staff – and you have to start early on.

It all comes back to company culture. This “executive first” mentality needs to be adopted as early as possible – before you have to fight things like politics or survival mechanisms that happen inside larger organizations.

For example, implementing this type of behavioral change in a company like Microsoft would be complex. There’s a reason it doesn’t happen inside places like that: information is power. And a lot of people are afraid of losing that power. So don’t let your company get to that point, do it early.

‍

I credit this advice to Jason Warner, an exemplary engineering leader who has been in companies that didn’t operate with a “first team” mentality and in companies that did. He has implemented this mindset in cultures that were at times unhealthy and saw the positive transformation it can yield.

Navigating both the exec room and your function as an engineering leader can be incredibly challenging, but something as simple as knowing who your primary team is can make a huge difference for you and the business.

Join Eiso Kant on April 28th for a LIVE webinar - "8 Mental Models for Exceptional Engineering Leadership"!

6 Decision-Making Frameworks for Engineering Leaders

CarmoBCosta — Tue, 12 Apr 2022 08:59:28 +0000

Decision-making is an indispensable skill for engineering leaders. In any company building software, engineering is where the rubber meets the road, and your decisions (short, medium, and long-term) will have the biggest impact on the business.

We often ask ourselves, "how can we be deliberate about the culture of our organization, so we don't end up pushing for outcomes without caring about how we achieve them?" – that's why I believe that "how we make decisions" should be one of the greatest building blocks of company culture.

I've argued that engineering leaders need to adapt their leadership approach as their company grows. So shouldn't decision-making also be flexible and easy to adapt?

Yes.

In this article, I will present a few practical frameworks and tools that you can use to help guide your decision-making.

These frameworks will help you understand:

Who should make certain decisions, and how fast.
What's the approach at different stages of the company.
Which questions should always be top of mind for engineering leaders.

Most of these frameworks come from the fantastic conversations I’ve had with Jason Warner (former CTO of GitHub) and other Engineering Leaders on our podcast.

This is not a magical guide to decision-making, but a simple toolkit from which you can pull out different frameworks. Ideally, these tools are woven into your company culture, and become daily, weekly, and monthly knee-jerk reactions to some of the questions that inevitably come along.

As you read through this list, remember: There are many times when you need to make a decision. But it's the art of the decision, not the science and mechanics of the situation, that matters. That's where excellent leaders are made.

1. The Jason Warner Diamond

Use for 👉 Creating a decision-making structure within your org. that fosters transparency and trust by assigning different decision-making parameters based on job levels.

One of the most important aspects of any function is decision-making. Jason Warner’s Diamond (widely discussed in Episode 3 of Developing Leadership) is a framework that can help bring more clarity to organizations by showing us what kind of decisions are being made at the different job levels.

The Diamond challenges the notion of “tops down” and “bottoms up” decision-making:

A set of all decisions will always originate from the top (one-way door, strategic, exec only) and bottom (majority of tactical decisions, ephemeral or transient choices, experimental mindset decisions).

The decisions in the middle are many that translate the tops-down decisions into action and make sure the bottom-up decisions map to the overall strategic direction of the company. If there are mismatches, the diamond middle is where it would likely originate.

It can be hard to accept a structure where decision-making isn’t always coming from the top, but if there is an immense amount of trust, this can be genuinely liberating as it leads to quicker decision-making processes, which, you guessed it, will help your company scale.

Jason believes that companies should not rely on an all tops-down approach or all bottoms-up approach. Instead, each organization needs to find a balance between the two approaches in order to achieve its goals.

TL;DR: Most decisions in a company originate from the top and bottom of the org, but most of the actual decision-making happens in the middle of the organization. Embrace this and you will be able to scale faster.

2. The Eisenhower Matrix

Use for 👉 Deciding how much time and how many people need to be involved in a decision before embarking on it.

The Eisenhower Matrix is a decision-making tool created by Dwight D. Eisenhower, who believed that there were two aspects that define decisions: important/non-important or urgent/non-urgent. Adopting the practice of using this matrix, or variations of it, for work decisions is a great way to avoid making bad decisions and wasting time.

The way we use this matrix at Athenian is a little different. Instead of categorizing decisions as urgent vs important, we categorize them on reversibility vs impact. This is a great first step into a decision-making process, as you can quickly decide how long you should spend on this decision and who needs to be involved.

TL;DR: If a decision is easy to reverse and low impact, make the decision fast – the higher impact and harder to reverse it gets, the more time and people you need to bring in.

3. The Inverted Bell Curve of Defence

Use for 👉 Understanding if you should be defensive or offensive in your decisions based on the stage of the company you’re at.

The Inverted Bell Curve of Defence illustrates the offensive vs defensive fronts companies should take, based on the stage that they are at. Understanding this will help you set the intention for your decision-making based on stage.

When you start building a product and the tech associated with it, at the beginning of the journey you are on the offense because that's what you have to do to succeed.

Then you become more defensive as an organization grows because you want to be structured architecturally, so the infrastructure works and it can’t be brought down.

But then at some point, if you just stay system and stability-oriented, you are going to be outcompeted, so you have to go on the offense again.

This framework comes from Ep. 6 of Developing Leadership “Deep Dive into Engineering Leadership Roles.

TL;DR: You should adopt an offensive front at the early stage of the company, become defensive when you have a structured architecture, and then offensive again so you don’t get out-competed.

4. The Product Delivery Organization: EPD

Use for 👉 Creating alignment between Engineering, Product, and Design team leads.
EPD is a cross-functional team structure that enables all three elements of this triad (Engineering, Product, and Design) to have equal importance to the business. This is essential for companies that are shifting their mindset from building and shipping software to delivering a service. Something which all tech companies should consider.

Each EPD squad will have a Product Manager, Development Lead, and Design lead. The success of EPD relies on the health of the relationships among these three leaders. The clear alignment of their goals is key for decision-making.

The Product Delivery Organization is a term coined by LaunchDarkly, but EPD has been around for a couple of years.

TL;DR: Consider having Engineering, Product, and Design as one integrated team, where the leaders of each team collaborate in their goal setting and decision-making. This is a mindset shift from building and shipping software to delivering a service.

Explore the remaining decision-making frameworks + get a free cheat sheet here!

8 Mental Models for Exceptional Engineering Leadership

CarmoBCosta — Mon, 11 Apr 2022 07:55:15 +0000

What makes a great engineering leader? And what does it look like at different stages of your company’s growth?

Eiso Kant (Founder & CEO Athenian) shares the essential building blocks of exceptional engineering leadership through 8 mental models - to get you thinking strategically about how you can be a better engineering leader and stay on track when things start to change.

Tune in on Wednesday, April 27th at 7:30 PM CEST!

Understand your responsibilities as an Engineering Leader.
Reflect on where you spend your time at different stages of the company.
Learn how to create a culture of continuous improvement.

Get your free spot here!