Right now, here in early 2024, we seem to be experiencing YAPP (Yet Another Productivity Philosophy), and that philosophy is converging on developer experience or ‘DevEx.’ The “revelation” that better developer experience leads to better productivity outcomes is not a revelation at all, but a truth that’s been, perhaps bafflingly, deprioritized in favor of post-CI software delivery improvement initiatives like DORA that have all but ignored the role of the human in the loop.
It seems that with every new method we invent for the delivery of products, whether physical or virtual, we reinvent productivity philosophies to go alongside them. From lean principles and “just-in-time” manufacturing in the 1970s to DevOps and platform engineering today, process stakeholders update their respective buzzwords and slang to match what they observe in their surrounding culture, but the cross-functional behaviors don’t shift much. Conclusively, physics is physics, and the way we work is subject to those universal constraints.
GitHub’s recent highlight of the Microsoft/DX study of the productivity outcomes of improved Developer Experience is being lauded as “finally” presenting the much-needed hard data behind the productivity outcomes of improved DevEx. Their article begins with: “The wait is over: we finally have data to back up the benefits of developer experience (DevEx).”
A highlight of my own work and study has been some of the frequent and fantastic exchanges I've been lucky to have with Peggy Storey and Abi Noda, so they know this is coming, and there is excellent data in this study, everyone absolutely should read it, but…
Finally? … really? ;)
Veterans of the developer productivity space may immediately be reminded, for instance, of “The Coding War Games,” an ongoing study from the mid-1980s to the mid-aughts which provided the same conclusions and led to the publication of the book ‘Peopleware.’
Without a lot of centralization of knowledge available, almost everyone else will agree with GitHub, unperturbed, that the data is the first of its kind. But we already have:
- The Coding War Games - Taking place over several years across almost a hundred different software organizations, this study provides ample data over years that developer experience metrics are a great predictor of organizational productivity
- The Code Red Study (CodeScene) - A study completed in March of 2022 demonstrating conclusively that time wasted by low-quality code led to 9x delays in cycle time
- The Only Way to Measure Developer Productivity Without Causing a Revolt - A comprehensive response to Dan North’s “The Worst Programmer I Know” blog, illustrating the inefficacy of many supposed productivity metrics
- Cat Hicks on Developer Thriving - A perspective that draws conclusions that subjective developer experience impacts overall productivity
Why do these cycles repeat? Why do we spend our energy discovering and rediscovering that the experience provided for the worker will almost universally predict better productivity outcomes when instead we could invest that energy in the cultural and environmental shifts necessary to enshrine those outcomes universally? More on what those investments would look like later.
What failed before will probably fail again
Chomsky said “If you are teaching today what you were teaching five years ago, either the field is dead or you are,” and this wisdom has reached anthem-scale for cultures of continuous improvement. This guidance shouldn’t direct us to forget the lessons of the past, but remind us that we stand on the shoulders of giants. I think the majority will forget, though, and I think that’s why we see clear indications that, up to this point, DevEx efforts have largely failed:
- Leaders invest in software delivery, not software creativity - Everyone loves to talk about Developer Experience improvement, but forty years after Coding War Games, decision makers are still wary of ROI. Perhaps we just haven’t seen comprehensive enough solutions, it’s possible that the ROI doesn’t exist because the right solutions aren’t being invested in.
- DevOps has descended into Dev ← Ops, where Developers have taken on additional responsibility for Ops, while the opposite has not been true. Developers have been expected to learn Docker and Kubernetes, SREs have not seen similar expectations. Developer self-service solutions have not kept pace with this change.
- Release Engineering and SRE has become Production Support - When software quality is negatively impacted because of increased pressure and a lack of self-service options for developers, that leads to a culture where supposedly proactive resources become almost fully reactive. SREs spend time answering tickets instead of improving systems. This can be considered a cascading failure, where poor developer experience leads to failed adoption of proactive SRE/DevOps measures.
- Everyone is waiting for Generative AI to save the day - While LLMs show great promise, they are unproven as of yet, nevertheless we are already abandoning the current state of the art in favor of the next shiny thing, and we’re not even sure the cavalry is coming.
- DevEx Improvement Adoption Rates Are Abysmally Low - The critical cultural shift following adoption of better DevEx practices such as implementing Backstage often fail with a lack of organizational acceptance, with most teams never realizing more than a 10% adoption rate. Typically, the most elite developers in the organization are responsible for curating the portal, and it may not align to what will actually be consumed by the more mainstream engineers in the organization. These failures may even further drive divides between these classes of engineers.
At some point, the tension will snap, and the data suggests that we are inching towards this milestone. A 2021 study by JL Partners observed that a significant percentage of developer burnout can be blamed squarely on flawed productivity metrics, and that these flawed metrics are responsible for perpetuating a vicious cycle. When goals such as release dates are calculated using incomplete metrics, the accuracy of those calculations is negatively affected. This leads to missed deadlines, which turn into efforts by the business to “fix productivity” by trying to find new ways for developers to ship faster. Unsurprisingly the end result is generally longer hours for developers, complemented with eroded trust because of continually perceived “failure.”
To complicate matters further, the data shows us that we still haven’t aligned on the proper way to communicate about productivity and developer experience, despite the work that has been done over the decades. A criminally overlooked April 2022 study, seriously please go read it, by many of the same researchers who worked on SPACE concluded that developers and their leadership tend not to prioritize the same metrics when thinking about productivity, often creating tension between developer experience and company velocity.
This is dizzyingly ironic in a world where developer salaries increasingly exceed $500k annually and digital transformation is a universal driver for nearly every business. In almost every other industry, the environment is curated for workers and bound by open and enforceable standards such as the ones provided by OSHA.
A perfect storm may be brewing where:
- As predicted by IDC, 65% of the Global GDP is software driven, a huge responsibility for a single workforce
- Developer salary continues to drive higher business cost while developer outputs are not improving
- Workforce fatigue leads to unprecedented economic waste and lack of innovation
None of these indicators show any significant signs of improvement, despite billions of dollars in DevEx investment in 2023. And can we even define “improvement,” when we still have not done a thorough job of aligning on what exactly those indicators even are? Digital transformation is only accelerating with an industry most recently invigorated with the promise of breakthroughs in AI and augmented or extended reality. Developer salaries will continue to surge with demand. Without real breakthroughs in developer productivity, cognitive fatigue and burnout will only become more ingrained.
We need productivity engineering, not productivity management
There is good news emerging out of the platform engineering and developer productivity engineering (DPE) circles, in that many who have been spending time thinking about better ways to measure and improve productivity have discovered long-hidden sources of bottlenecks and friction in the developer experience, and are starting to take action to deal with them. When we move past speculation and use solutions that let us gather quantitative data about developer workflow, the parts of the workflow deleterious to developer experience show up in high resolution. With this a new set of metrics are emerging, ones that more closely resemble the efficiency of a developer’s workflow. With more accurate metrics, we can drive real improvements.
These metrics deal with pains felt acutely by developers, and more chronically by organizations. They come from dozens of sources and represent the galaxy of tools relied upon by developers in their daily workflows. Build tools like Maven and NPM, testing frameworks like Cucumber, security and quality scanners like Snyk and Checkmarx, deployment substrates like Kubernetes, requirements management tools like JIRA and even on-call management frameworks like PagerDuty, all capable of creating new frustrations and sources of context switches for developers.
These revelations are proving what many of us have known for decades, and what countless studies have proven time and time again. There is no one metric, or even one set of metrics, that can even hope to paint an accurate picture of developer productivity, and so no one metric or set of metrics can fix it. As the researchers on the SPACE framework put it, we need to capture a “constellation of metrics in tension.”
The right solutions prioritize integration… and they already exist
We need to invest in tools that can act as predictive engineering systems of record and analyze the full spectrum of metrics, not just ones that are evident in CI/CD. A fair criticism of DORA is that it only captures metrics post-CI, and Conway’s Law would then lead to tools which only look at data generated in CI. Those data are important to understand parts of delivery, but to see the full picture, we need solutions that allow us to look at data generated by every part of the developer workflow, from the laptop all the way to the deployment substrate, and they need to adapt quickly to changes in the infrastructure.
The right solution for this problem will prioritize gathering data easily and up-to-the-microsecond from the full bloom of developer tools used by engineers in the organization. The solution must also use these data to drive new effects and behaviors in the organization, through gamification and other proven techniques. These tools should not overburden teams and should require little administration past the point of integration and data processing.
Lucky for us, “The future already exists, it’s just not evenly distributed yet.” We have no need to reinvent another productivity approach when effective solutions such as internal developer portals already exist, and already prioritize the correct mechanisms for facilitating successful productivity engineering initiatives. As business leaders, we just need to invest in and create urgency behind these solutions today, so that we don’t find ourselves having the same discussion again in a decade.
Top comments (3)
Such a nice and deep post my friend, thank you.
I will have to read it several time to grasp everything, especially the external links. For now, i do not agree with everything, but that is the beauty of diverse views.
Developer Experience around me have been my profound will for the past 18 years, and the main problem is poor management afaik.
A few things come to mind, that I'm not sure you addressed... And you seem so knowledgeable in the field that I might look ridiculous, but I will try anyway.
1) DORA is indeed technical delivery metrics at the core, which can lead to burn-outs if misused. But it can serve different purposes. I has helped me advocate to managers that "quality ultimately means more revenue on top of nice quality of life".
2) Shipping fast with good tools is a first step for more interesting talks, and is wanting by lots of (good) developers in my experience.
3) is there really a need of metrics for DevEx ? It is a matter of people, could we just ask them what they think ?
4) Shipping fast will never surpass shipping the right things, no matter the DevEx. Ingeneers tend to be happy to solve a wrong problem.
5) Burn-outs is also a matter of technical leadership. It is the role of the trusted ingeneering leaders to give boundaries to managers. Ultimately human problems have to be solved too, not just tech ones.
What do you think ?
Thanks Benoit, I really appreciate your thoughts on this and I'm glad you are finding these posts to be insightful -- I think disagreement is really important right now too, as I think the entire industry is seeking thought leadership and the "right" way forward for platform, devex, etc, so we need to start answering these questions accurately. I'll do my best to respond, I think these are very thoughtful observations!
1) DORA is indeed technical delivery metrics at the core, which can lead to burn-outs if misused. But it can serve different purposes. I has helped me advocate to managers that "quality ultimately means more revenue on top of nice quality of life".
2) Shipping fast with good tools is a first step for more interesting talks, and is wanting by lots of (good) developers in my experience.
3) is there really a need of metrics for DevEx ? It is a matter of people, could we just ask them what they think ?
4) Shipping fast will never surpass shipping the right things, no matter the DevEx. Engineers tend to be happy to solve a wrong problem.
5) Burn-outs is also a matter of technical leadership. It is the role of the trusted engineering leaders to give boundaries to managers. Ultimately human problems have to be solved too, not just tech ones.
Thanks again Benoit for your thoughts and your kind words, I really appreciate this dialogue, we have to solve these problems, so much is riding on our workforce!
Great read