DEV Community: Arnaud Porterie

Echoes: a one-year update on our vision

Arnaud Porterie — Mon, 12 Sep 2022 12:27:31 +0000

Roughly one year ago, an eternity in startup time, we shared what we were building at Echoes HQ. We went through YCombinator during the summer of 2021, and launched our first version of the product the following September. This post is an update to our vision, one year into the life of the product.

How our experiences shaped Echoes

In 2014, blown away by Docker's potential for wide-scale developer empowerment, I left my job as software engineer in the banking industry in France to lead the Docker Engine team and its associated open source project. The mission spoke to me deeply: giving power to the builders, creating tools of mass innovation. I cannot understate how revolutionary this sounded as an outsider to tech companies and Silicon Valley: a world where engineers were at the center of value creation existed.

I later decided to take these learnings to Veepee which I joined as VP Engineering. My personal goal was to create the environment in which I would have wanted to evolve as an engineer. I tried to instill a culture of engineering empowerment, taking heavy inspiration from open source as to how code is maintained and how teams interact. In this journey, I learned the challenges of leading a successful engineering organization at scale.

The state of engineering management tools

Despite the variety of organizations I joined or advised, the same problems keep coming up. From one point of view: too many topics at once, and a difficulty to assess and communicate how we are succeeding (or not) as a tech organization. From another point of view: a perceived opacity of tech, and that nothing ever goes fast enough¹. As widespread as these problems are, none of the solutions on the market are appealing to me.

On one end of the spectrum are top-down management tools, with a bias for control. They go against the engineers' best interest in several ways: operationally, by imposing a unique view of how software development should be done; ethically, by imposing a form of surveillance which later feeds into misguided performance management.

On the other end of the spectrum are engineering productivity tools, measuring the efficiency of the development process through git and tickets analytics. Their insights are actionable to the teams but fail to inform management decisions²; when used as success metrics, they perpetuate the outdated idea that an engineering team's responsibility is solely about shipping code.

What all these tools appear to have in common is an underlying assumption that an engineering manager's role is to get more out of the engineers. They ignore the decades of learnings which have repeatedly proven how software development is a creative, poorly predictable, team effort. They define success in terms of outputs, but succeeding as an engineering organization takes a lot more than productivity. Arguably, engineering efficiency is not the bottleneck to most organizations' performance³.

Echoes approach

Echoes' mission is to empower developers everywhere, by helping engineering managers create the best conditions for success. Our first step in this journey is to offer managers a value-centric and developer-friendly way to follow and report on activity.

Value-centric

What we mean by value-centricity is that Echoes focuses on the purpose of the work over the velocity alone. It surfaces the determinants of an organization's performance which a manager can directly act upon: the clarity of goals, the alignment to the strategy, the focus, and challenges faced by the teams. The question Echoes aims to answer is: does my team have what it needs to succeed?

This is not to say that engineering efficiency is irrelevant to the performance of an organization, but that it should not be an engineering manager's primary concern, and of lesser and lesser significance as one goes up the management chain. Measures of efficiency belong where they can be acted upon: in the hands of the individual contributors.

An interesting side-effect of focusing on value is that it makes for a great communication medium. By connecting engineering day-to-day efforts to company impact, Echoes presents activity in a way that anybody can understand, including a non-technical CEO, CFO, or HR partner.

Developer-friendly

What we mean by developer-friendliness takes many different forms. At the most fundamental level, Echoes does not and will never surface any individual metrics, nor anything which could be misinterpreted as a proxy for individual performance. Echoes shifts the focus away from developer efficiency as the main lever to performance, and toward organizational effectiveness.

Echoes is designed to let the builders pick the tools and processes which make them most productive. This is no small feat, as aggregation across teams traditionally mandates a level of standardization incompatible with developer empowerment⁴. Echoes is designed around the smallest possible contract: code changes are to be linked to an intent; how each team choses to create this link is at their own discretion. Finally, Echoes knows to stay out of the way: engineers’ interactions with the platform can happen entirely through the tools they already use, such as GitHub and GitLab.

The results, one year in

What we've built with Echoes is different. It may look like a goal setting tool, but truly connects work to purpose. It may look like a measure of developer productivity, but truly says a lot more about the quality of management.

Echoes is different, and it shows in our user base. We naturally see Echoes being used by managers from the EM up to the CTO; but we’re also seeing it adopted by product managers, CPOs, Staff Engineers, and TPMs (Technical Program Managers). We see Echoes chosen by organizations of hundreds of engineers, where the pain of following activity is more acute; but we’re also seeing it in organizations under 10 engineers, where it helps focus efforts where they matter most.

We have learned a lot along the way, but as expected when trying something different, we are often left with more questions than we have answers. What is certain is that we are determined to continue exploring this path, because developers deserve it.

We wrote about the common disconnect between planning and execution in this post. ↩
We wrote about the value of engineering metrics in this post. ↩
We wrote about the disproportionate focus on developer productivity in this post. ↩
We wrote about the tension between management and developer needs in this post. ↩

Transitioning from CTO to founder

Arnaud Porterie — Fri, 05 Aug 2022 09:22:00 +0000

Roughly 18 months ago I took the jump of leaving my CTO job managing 600 engineers to start Echoes HQ, my first company as a founder. This post is a personal reflection of what I've learned and what surprised me in that transition, in no particular order.

For context, at the time of writing Echoes is 3 people: Arnaud (founder), Adrien, and Lucas (founding engineers, who both joined roughly one year ago).

Emotional stability
Emotional transparency
Goal setting
Communities
Prioritizing what hurts most

Emotional stability

Managing a large team inevitably involves a daily dose of bad news. Each day may bring a key person's resignation, a major production incident, or that budget cut you're going to have to handle.

Startups, especially at their early stage, are way bigger rollercoasters. But despite the frequency and relative amplitude of events being stronger, I feel like my time as CTO was a great preparation for it. One of the most important skill I gained is the ability to separate how I may feel about a particular event and how I'm going to act on it. That doesn't mean you should bury emotions, we're humans!, but that you should not react on emotions.

Startups are built on learnings and every event, good and bad, is an opportunity to learn. Be disappointed, upset, sad even about that customer you lost, but accept that fact and don't let emotions cloud your analysis of the situation.

Emotional transparency

I believe a founder at an early stage startup has more room, but also more duty, to be transparent about their feelings.

Your doubts as a CTO can be detrimental to your organization's success. Teams tend to mimic and through their size amplify the sentiment of their leaders. This can in some cases turn into self-fulfilling prophecies, when for example that project you say might get cancelled inevitably loses steam.

Startups on the other hand cannot succeed without embracing their doubts. The whole journey is a thug of war between the confidence in your vision and the reality of what works and doesn't (which in the beginning is roughly everything).

This cognitive dissonance is uncomfortable but healthy, and isolating the founding team from it is not doing anyone a favor. Early startup employees have a different aversion to risk than members of large organizations: doubt mustn't be an issue.

Goal setting

Setting goals is something that most CTOs would be very familiar with. I soon realized how different an exercise that was for an early stage startup, as our first attempts felt very much like wishful thinking. Indeed, while you may now be in full control of your goals, being pre-product market fit means that you have much less ability to predict their outcomes.

This required doing the opposite of what goal setting typically looks like in a mature company. Instead of setting goals on outcomes and largely leaving the solution space open, I was left setting goals on outputs. For example a revenue target becomes a number of demos, and an increase in conversion rate becomes a number of shipped experiments.

Communities

What was most unexpected to me in this transition is how founder (and even solo founder in my particular case) feels surprisingly less lonely than CTO.

Part of the reason is tied to the previous section on doubts: running a large organization forces you to constantly censor your thoughts, and you may not always have room to share things transparently.

Part of the reason is that while communities of CTO do exist, I personally find the ones I participated in to be quite mechanical. What I mean by that is that conversations are very much practical: discussing processes, organizations, tools, etc. That doesn't make them useless, but I can't say I've met many friends in these places (again, my personal experience!).

There's something very unintuitive here, but I believe that the little practical overlap between early stage startups makes for deeper conversations between founders. We each play such different games that there's little to discuss about practicalities: the personal journey is where we have the most overlap. It's been heart-warming to discover how other founders genuinely want to help, and have mostly no issues being vulnerable in conversations.

We had the chance with Echoes to be part of YCombinator S21 batch, which put us from day 1 in one the best communities we could hope for.

Prioritizing what hurts most

My time as a CTO taught me (sometimes painfully) that problems don't fix themselves but only tend to get worse. I've therefore developed a habit of doing what I hated most first, which proved very useful as a founder.

There are so many aspects to the role that some of them will inevitably suck, although different people will naturally have an aversion to different things. For me, it's administrative work (surprise!), such as legal and accounting. And I've been doing it flawlessly, because I know that if I don't do it immediately I'm only going to get a double dose of it.

We're building with Echoes HQ the product I wish existed during my time as an engineering leader. Check us out and reach out if you'd like to learn more!

Beyond developer productivity

Arnaud Porterie — Tue, 14 Jun 2022 08:24:33 +0000

Developer productivity receives disproportionate attention considering the share of organizations where it happens to be a bottleneck.

With tech being at the center of value creation engine for an increasing number of businesses, it's natural to look into every opportunity to be more efficient. There is however a misleading assumption that developer productivity is a universal leverage to an organization's performance.

Scaling productivity

A single team has little collaboration overhead and, constrained by its limited capacity, tends to have few goals and strong focus. Delivery performance in that context can therefore be directly attributed to developer productivity.

This ideal however tends to break down as early as having two or three teams. As an organization scales, the influence of individual developer productivity on the performance of the whole diminishes. A group of locally-efficient teams does not make an efficient organization: there is a point where collaboration, dependencies, and team cognitive load become the bottlenecks.

One of the reasons for this is that as an organization grows, the number of independently taken decisions increases. A fraction of these decisions result in the fascinating phenomenon of ✨ creating work ✨, for example by prioritizing the development of a new feature, or planning a technical migration.

The ability to make design and product decisions is, as Andy Budd pointed out in this excellent thread, a lot of fun. It's also one the most consequential decisions one can make because it spends an organization's crucially valuable resource: focus.

From developer productivity to organizational effectiveness

Developer productivity matters at all stages of a company, if only because it contributes to no small part to the broader developer experience. But developer productivity is, and should remain, a concern to the builders first and foremost.

While comfortable to management, it is a trap past a certain size to continue considering developer productivity as the main leverage to the organization's performance. First, there's an upper limit to efficiency which an untamed ability to create work will eventually exceed. Second, there are inevitably initiatives which contribution to goals does not justify their cost in engineering focus. Or as Peter Drucker puts it:

“There is nothing quite so useless, as doing with great efficiency, something that should not be done at all.”

Engineering managers who disproportionately focus on measuring and improving developer productivity are not only doing their team a disservice, they are also failing on their mission to bring clarity of goals, alignment, and focus. As Pedro Franceschi pointed out, the difference between the best and worst projects is not found in their individual contributors skills, but in their leadership.

For all these reasons, Echoes does not pretend to measure or improve developer productivity, or at least not in the commonly-understood sense. How Echoes makes developers more productive is by helping management create the right context for them to deliver their best work.

Echoes shines a light on the reality of the day-to-day, so that the complexity of execution informs planning decisions. Echoes shifts focus away from local efficiency and individual developer productivity to the bigger goal: the organization's collective ability to succeed.

Interested to learn more? Check out https://www.echoeshq.com.

Deliberate software developmemt

Arnaud Porterie — Wed, 08 Jun 2022 12:47:19 +0000

Introduction

Discourse on how to organize work has taken a lot of space: whether discussing methodologies (e.g., OKRs, Agile, Scrum), roles and responsibilities (e.g., Product Management), or organizational models (e.g., "Spotify").

While important, these topics tend to confuse startup founders and engineering leaders in more established companies alike, who struggle to figure out what is "right" for them. Countless time have I heard these questions: should we adopt OKRs? Should we move to a micro-services architecture? How should we set and communicate priorities?

On being deliberate

Discussions on the how can sometimes overshadow the one thing they should be in support of: the purpose of work. Putting aside sprints, objectives, tickets, and roadmaps for a minute:

We do things for a reason.
That reason is to have observable impact on some behaviors.
These behaviors should align with our definition of success.

Methodologies, organizational models, architecture choices are in service of the purpose. OKRs are a popular way to align teams behind a priority with well-defined success criteria at a fixed cadence. The North Star Framework is a popular way to capture the longer term strategy and provide a stable frame for goal settings. Micro-services architecture are a popular way to decouple the deployment of independent parts of the codebase. None of them can help if you are not sure which problem you're solving in the first place.

Back to basics

The most important thing you can do to succeed as an engineering organization is to define what success means for your particular context and frame activities with their purpose. Which goal setting framework, day-to-day execution methodology, and architecture you chose is ever going to be in support of that.

‍

Building with the GitHub API

Arnaud Porterie — Fri, 22 Apr 2022 09:29:55 +0000

I have a long history of building things with the GitHub API.

Many years ago when running the Docker open source project, where we needed tools for GitHub tasks automation (icecrime/poule) and project health metrics (icecrime/vossibility-stack).
Today with Echoes, which integrates with GitHub (among others) to help engineering leaders create high performing organizations.

We will look in this post at an unordered list of things that I've learned along the way.

GitHub Apps or OAuth Apps

There are two ways to integrate with GitHub: as a GitHub App or as an OAuth App. While the difference is explained in GitHub documentation, it's worth reinforcing how neat GitHub Apps are.

GitHub App use their own identity, not that of a user. In other words, actions performed by the app will appear as originating from the app itself, not "whoever lended a token".
GitHub Apps have fine grained permissions. Considering the sensitivity of GitHub data, this is crucial in order to build integrations which follow the principle of least privilege. In the example of Echoes, this is how we drastically reduce our security requirements by not requesting access to the content of the git repository.
GitHub Apps provide an excellent installation experience, which includes: installing on all or selected repositories, reviewing permissions and permissions scope updates.

If you're building in Go, bradleyfalzon/ghinstallation is an implementation of http.RoundTripper which authenticates as a GitHub App, for example to use in combination with google/go-github.

GitHub v3 or GitHub v4

GitHub has two active versions of its API: a v3 REST API, and a v4 GraphQL API. Which one you should build against is partly a matter of personal preference, but chances are high that you will end up using both.

The entire API surface is not available in both versions. For example, the Webhook Deliveries API referenced later in this post is only available in v3.

Another, trickier, aspect is that while GitHub Apps permissions naturally map to v3 endpoints, things are not as clear when using the v4. A GraphQL query requires the union of all permissions of its individual fields to succeed, or will result in a confusing “resource not accessible by integration” error. It is not always obvious which part of the query causes the error, especially when a seemingly equivalent call using the v3 REST API succeeds under the same permissions.

In our particular codebase, the interactions with the GitHub API appear to converge to using v4 GraphQL API for data retrieval, and to using v3 REST API for mutations (e.g., adding a label to an issue) or situations where v4 didn't behave the way we expected.

Defending against GitHub-side issues

GitHub, like any other complex system, is imperfect. Two issues we have seen are minor API downtimes (with 502 responses), and occasional authentication issues (with 401 responses) using a freshly-issued access token.

Both cases can be mitigated with automated retries. We went with hashicorp/go-retryablehttp as Echoes is built in Go. The library's default retry policy rightly doesn't consider a 401 response as a recoverable error, but this can be changed using the CheckRetry hook.

Defending against integration-side issues

Chances are that your integration won't be perfect either. What will happen to GitHub events when your app is unresponsive or fails to handle a message?

GitHub has a UI for inspecting recent deliveries and manually triggering new deliveries. This is immensely useful for development and debugging, but cannot help if you need to recover from any significant number of missed messages.

At Echoes, like we did at Docker, we're using our own durable queues to handle message processing. However, GitHub has since introduced the Webhook Deliveries API which allows listing past deliveries and triggering redeliveries.

Learn about secondary rate limits

GitHub API has two rate limiting mechanisms: one based on a number of requests per hour (5,000 in the case of GitHub Apps for github.com), and one based on the nature of the interactions themselves. The former is quite typical and easy to account for by examining GitHub API’s response headers. Complying with the latter, however, requires to be thoughtful and review how the system as a whole interacts with the GitHub API, including all services, tasks, and jobs in combination with various retry-policies.

Two of the guidelines to avoid hitting the secondary rate limits deserve particular attention:

"Do not make requests for a single user or client ID concurrently."
"If you're making a large number of POST, PATCH, PUT, or DELETE requests for a single user or client ID, wait at least one second between each request."

As a general rule of thumb, we would recommend against multiplying systems that will have to share the API budget.

Conclusion

GitHub is the place where most development work happens, and building tools for it can be extremely rewarding. GitHub puts a lot of effort in its platform, as seen by the introduction of GitHub Apps, and how the API continues to evolve.

We hope this post will help you with some of the less-obvious lessons of interacting with the GitHub API. If you curious to learn more about what we’ve built with it and how we believe it can help every engineering organization out there have more focus, more clarity, and more success, check us out at https://echoeshq.com!