DEV Community: Nnamdi Iregbulem

Six Trends Shaping Developer Productivity

Nnamdi Iregbulem — Tue, 14 Jul 2020 19:20:44 +0000

Developer productivity is undergoing a tectonic shift. New software development paradigms and tooling have accelerated the pace and productivity of modern software teams, quickening the "shipping speed" of new software.

To dissect these trends, my good friend and colleague, Clio Smurro, and I interviewed founders and executives at next-generation software and infrastructure startups pushing the developer productivity frontier to get their thoughts and insights. They shared their views on:

major industry trends (you are here),
top strategic priorities (coming soon), and
biggest challenges and pain points (coming soon)

In this first chapter, we share our findings on the important trends shaping developer productivity, including:

Trend #1: Developers have the power... and the purse
Trend #2: Application security is "shifting left"
Trend #3: The distributed cloud is having its COVID moment
Trend #4: Remote software development is here to stay
Trend #5: The growth of Python, Spark, and Big Data
Trend #6: Transfer learning from DevOps to data science and data engineering

Want to be notified when we publish part two of our findings? Subscribe below, and we'll also send you a nicely formatted PDF of our research!

Trend #1: Developers have the power... and the purse

" Buying products that save developer time is no longer an argument you need to explain. People get it." — Executive, Developer Tools Startup

Software engineers continue to be a scarce resource in most organizations. Companies are increasingly focused on enhancing the productivity of developers. In doing so, power and autonomy flow to developers, and the dollars are quick to follow.

Vendors are reflecting this new reality in their go-to-market positioning and sales efforts:

"In our sales conversation, we frame things in terms of productivity and developer time saved... You're comparing the cost of the product against engineering time saved." — Executive, Developer Tools Startup

Even before sales gets involved, developers are adopting software they need to get their work done on their own, often without the involvement or permission of procurement or upper management. Developers know what they want, the tools they love, and the technologies that enable their ideal architectures and designs.

" We're seeing lots of self-serve. Developers are getting more autonomy as buyers. Most of our sign-ups are via bottoms-up — people signing themselves up, after which our sales team eventually reaches out to them." — Executive, Developer Tools Startup

Unlike many other domains of enterprise software, where features are built to appease higher and higher levels of management rather than end users themselves, the core developer experience remains hugely important. If anything, analogous trends in observability and monitoring are developing in a symbiotic rather than antagonistic fashion with developer productivity.

As engineering teams scale, the need for agile workflows becomes apparent, drawing many toward the burgeoning DevOps paradigm and its associated ecosystem of tools. DevOps enables ongoing operation of and rapid iteration on software via Git-based version control, continuous integration, continuous deployment, security testing, and more.

"DevOps hasn’t been around for long, but more companies are realizing the need/value for it." — Developer Advocate, Application Infrastructure Startup

To the extent that vendors are attempting to appease management, they are doing so by building unified product ecosystems. These enable customers to purchase multiple component tools of the overall software development lifecycle in a single package. While these do carry some benefits for end users, most still prefer to purchase the best solution for each task, again reflecting in influence and clout of individual software developers:

"GitHub, Atlassian, Microsoft... They’re trying to get everyone to adopt a unified tool system. But most people still go with best-of-breed, as far as tools go. The idea though is that some people will eventually go with more of a “you can’t get fired for buying IBM” approach, where you buy everything from a single vendor." — Executive, Application Security Startup

Trend #2: Application security is "shifting left"

In past eras, application security was oftentimes dealt with after the fact. Software would be largely complete by the time security analysts had a chance to examine and poke holes in its defenses. In main cases, this might not even happen until after code is already running in production, where any vulnerabilities may have already been exploited by nefarious actors:

"Once the code is built, the artifact goes into a registry. The security team wants to know... what are the risks? Is this meeting my policies?" — Executive, Application Security Startup

No longer. Application security is now a tier one priority in many organizations:

"Security used to be an afterthought. In the past, someone would write code, someone would deploy code, and then someone else would handle security. That boundary doesn’t exist anymore." — Executive, Developer Tools Startup

Spurred by the rash of high-profile security incidents and gaffes at major corporations around the world, organizations are challenging their development teams to take on more of the security burden upfront, well before software is even ready for production or artifacts have been built. "DevSecOps" is born:

" Shifting left means move everything towards the developer. It doesn't have to be a security person's responsibility to ensure secrets are secure — the developer can do this now too. The more tooling you give, earlier on in the process of writing the application, the easier this is." — Director, Developer Tools Startup

Security vendors continue to sell mostly to security teams but realize their tools are increasingly landing directly in the hands of developers themselves:

"We don't sell to developers, we sell to security teams...but at the end of the day, it’s the developers who need to take more upfront responsibility for security." — Executive, Application Security Startup

It's easy to think these new tools are only valuable to large enterprises paranoid about breaches, hacks, and other threats to application security. Not true, say some of the security leaders we spoke to, who emphasized that the heightened focus on security is reverberating through the software development industry, at both large organization and small:

"These security initiatives are not just for big companies... every company needs them." — Executive, Developer Tools Startup

Then there's the concept of "low trust" or even "no trust", where applications do not give each other the benefit of the doubt and every app must prove its credentials in order to send and receive requests and data from other apps and microservices. This adds new complexity to software development, heightening the important of thinking through security implications early on in development:

"In low trust or no trust environments, how do you make sure applications can talk to each other?" — Executive, Developer Tools Startup

These complexities are inevitable, but vendors also know there are limits to developer patience. They are keen to insert security tooling into workflows as seamlessly as possible. Usability drives usage — if a tool is to difficult to use or increases cycle times too dramatically, developers won't use it, defeating the purpose entirely:

"DevOps folks are often not security experts. They’re looking for usability. How easy is this to access? Does it fit in our existing workflows? Can it plug into Jenkins? I don't want my developers having to use a new tool." — Executive, Application Security Startup

As development and security increasingly merge, buying patterns and processes will incorporate the needs of both stakeholders, and vendors will need to adjust their tactics appropriately:

" DevOps needs to like it, SecOps needs to buy it. Both are involved in the purchase process." — Executive, Application Security Startup

Trend #3: The distributed cloud is having its COVID moment

Demand for distributed computing is growing and that demand has only surged in the COVID era.

" COVID has caused a 2-3 year acceleration in everyone's journey to the cloud. The companies who have survived and thrived are the ones which evolved sooner." — Executive, Developer Tools Startup

Elastic scaling of compute, storage, and other resources is important in a divergent set of scenarios. Some businesses (Zoom, Fastly, Amazon, Instacart, etc.) have seen demand for their services surge, requiring rapid scale up of existing deployments, assisted by prescient decisions to factor applications into microservices.

On the other hand, certain companies have seen business dry up overnight. This makes the ability to scale down equally important, allowing costs to flex proportionately with revenue:

"Do only what's needed. That's key for dynamic shifts in needs. You need to design software for scaling up, but the infrastructure also needs to be able to scale down." — Executive, Developer Tools Startup

But what if your business runs on legacy applications or infrastructure technologies? You're in a much tougher position, which is prompting many organizations to ask the question — can you accelerate developer productivity within legacy application development?

There's a lot of legacy .NET applications from 10 years ago that weren't originally built with containers. Organizations want to get those into containers. A question people are asking — can you get developer productivity with legacy applications too? — Executive, Application Security Startup

Increasingly, the answer is — yes. Legacy applications are being refactored from monolithic to microservices-based architectures, leveraging the power of containers and other associated advancements:

"We are seeing a movement from cloud "greenfield", only using the cloud for new applications, to "lift and shift", getting legacy applications into containers and into the cloud. We've started working with Windows-based containers." — Executive, Application Security Startup

While old-school organizations and engineering teams play catch up, next-gen startups and technology companies aren't waiting around.

" The biggest trend right now is the move to serverless — functions as a service, hiding more complexity from developers. Serverless is a way for developers to just focus on stateless applications, to just focus on what they use most directly." — Executive, Application Infrastructure Startup

Serverless has emerged as a major trend. Leveraging the "infinite" scalability of the major cloud providers, serverless promises to decompose applications into a series of function calls — without regard to the underlying infrastructure.

While the cloud providers have always had a clear incentive to push such a regime, it seems like development teams themselves are warming up to the idea, thought it's full realization is still years away:

"At our company, we talk about nodes and clusters today. We think of elastic pools of storage and compute that you can just use. But, a few years from now, we won't even be worried about literal nodes and clusters." — Executive, Application Infrastructure Startup

Trend #4: Remote software development is here to stay

In conducting our interviews, we were struck at how little COVID had affected the productivity of most software development teams. Perhaps we had a biased sample (we mostly spoke with startups), but the near-universal response was that the move to mandatory remote work had been relatively smooth. Many already had significant portions of their development teams working remotely, which helped to ease the transition.

Further, many had seen little disruption in their go-to-market efforts, given their focus on selling to developers who had themselves seen little interruption in their work.

If it is true that at least some teams and organizations can be nearly as productive working remotely as in-person, we may see some aspects of this new work paradigm stick around long after COVID subsides.

"The shift to working remote is happening... It's month three of the new normal, and I think there will be long-term changes as a result of this." — Executive, Developer Tools Startup

Software developers who enjoy this new style of work may have other reasons to rejoice too. As I've previously written about, developers who work remotely earn up to 22% more than developers who don't. Of course, as with many aspects of a forced transition to remote work, it's not clear whether this results applies when organizations had no choice about the move.

Trend #5: The growth of Python, Spark, and Big Data

"From our perspective, Python is winning , and we see that trend continuing." — Executive, Data Science Startup

Teams love Python for its ease-of-use and rapid time-to-value, even for relatively untechnical individuals, who can quickly get up to speed with the language and generate value output:

" It’s easy to learn….people can learn it in 8 weeks and then be useful in a Fortune 1000 company." — Executive, Data Science Startup

In addition to its user-friendliness, Python is revered for its ecosystem of packages for tackling difficult data science challenges and processing large data sets.

Speaking of large data sets, after much hype and suspense, Big Data has finally arrived and is a major driver of Python's massive popularity:

"Big Data is not just trendy anymore, but it’s actually happening now. Not like 5 years ago when it was first hyped." — Executive, Data Science Startup

These large datasets are increasingly shifting to the cloud, where various storage offerings from S3 to Snowflake and more have proliferated, offering no shortage of options at competitive prices for various performance levels and data access frequencies. Leaders agree that Python is the preeminent language for handling data in the cloud:

"We are seeing a shift to the cloud, and Python is dominant for data in the cloud." — Executive, Data Science Startup

However, having been conceived well before the microservices revolution or the cloud generally, Python out-of-the-box does not come with much distributed computing functionality, though multiple vendors and technologies have arisen in recent years to cover this gap (Dask, Ray, etc.):

"Python... is the fastest growing language and very popular, but Python has no distributed functionality." — Founder, Application Infrastructure Startup

Have no fear though, Spark is here!

Spark bills itself as "a unified analytics engine for large-scale data processing", which in layman's terms mean it's very, very fast — fast enough to handle large datasets at high throughput. Spark is built from the ground up with distributed computing in mind, enabling it to take advantage of the advancements in cloud computing we discussed above. Spark also offers a number of high-level operators that enable interoperability with more popular languages like Python, SQL, Java and more.

"Big data development cycles used to take forever, not fast enough for software developers. Spark has meaningfully sped up the cycles." — Founder, Data Science Startup

Trend #6: Transfer learning from DevOps to data science and data engineering

Historically, data science has not had the emphasis on fast iteration and development cycles that DevOps has had. When merged with traditional software engineering, this is less of a problem. But as data science as a practice gains clout, data science professionals are being carved out into their own teams. While this comes with certain benefits, it often comes at the cost of speed:

"Data science is now being carved out of development teams, and the data-related development cycles have elongated." — Founder, Data Science Startup

Additionally, data science is only becoming more complex. Data engineering has emerged as a key component of the overall data science lifecycle and, while less sexy than building and training the latest deep learning models, is often the phase that takes the longest in any given project. Many consider it to be an entirely different skill set from core data science. ETL (Extract, Transform, Load) tooling is just one example:

"Let's look at the problem of managing ETL pipelines. There are actually two pieces — the ETL pipeline that transforms the data, and the pipeline that manages the ETL pipeline itself." — Executive, Data Science Startup

DataOps and MLOps are the response to these complexities and speed bumps, bringing best practices from DevOps to the realm of data science and machine learning:

"DataOps and MLOps bring DevOps principles such as agile development to data and machine learning." — Founder, Data Science Startup

Instead of being crushed under the weight of increasingly intractable datasets and data engineering puzzles, DataOps and MLOps help data scientists and engineers better wrangle the data development process itself and achieve business outcomes with the same agility as traditional software development:

"We're big on DevOps and the empowerment of the data scientist and data engineer. They should have control over the end-to-end process. Whoever is the creator is also the person who’s responsible for the ongoing success of the artifact." — Executive, Data Science Startup

DataOps and MLOps are opening new possibilities for data and model version control, machine learning feature engineering and storage, and more. Think unit or regression tests, except for datasets and machine learning models:

"In GitOps, the continuous integration / continuous delivery / continuous deployment pipeline checks the code and deploys the application to a staging environment, eventually ending up in production. Most tech-forward companies have adopted that now. We want to bring that to the data scientist." — Founder, Data Science Startup

The grand vision? Driving efficiencies that will enable teams and organizations to keep up with the growth of Big Data:

" The trend toward DataOps will be big. It'll make organizations more efficient." — Executive, Data Science Startup

Conclusion

Developers are increasingly considered one of the most important constituencies within organizations. Organizations both old and young are generating and consuming greater amounts of software, and developers remain the basic economic unit of software production.

As the scarce input of the software production function, software engineers don't come cheap, and it's therefore critical to maximize their productivity and output. As software grows only more powerful, valuable, and essential, anything that makes software engineers more productive will be similarly potent and relevant.

Do College Degrees Matter for Software Engineers? Maybe

Nnamdi Iregbulem — Wed, 20 May 2020 15:39:14 +0000

This post is an excerpt from my 2020 software developer pay analysis. Read the rest of the analysis here.

Coding bootcamps say the traditional model of education is broken and that anyone can become a software developer with the proper effort.

Further, they claim that traditional institutions aren’t incentivized to ensure their graduates get hired post-school.

Regardless of your views, these claims ignore an important question: let’s say you get the job — will you earn less as a software developer without a college degree?

Yes, but it’s not as bad as you might think

Developers with college degrees earn 7.5% more than developers with no college education.

As in previous analyses, I additionally control for various observable factors like age, experience, hours worked, size of employer, programming languages and more. Interestingly, accounting for these factors doesn’t change the college wage premium.

Advanced degrees carry greater benefits — master’s degree holders earn 13.3% more than college degree holders, while those with doctoral degrees (i.e. PhDs) earn 20.5% more. These numbers decrease meaningfully though once I add in the controls I mentioned above, falling to 4.1% and 7.8% respectively.

Is this a big deal?

Now, a 7.5% college degree advantage is nothing to scoff at.

But 7.5% is tiny relative to the college wage premium earned by the public at large.

A review of the evidence around higher education shows that the college wage premium is much higher for the overall population. In my favorite study on this topic, researchers at the Federal Reserve Bank of St. Louis show that, controlling for age and level of degree (college vs. graduate), college graduates earn ~50% more than non-graduates (the data is similar for other ethnicities):

So the college degree wage premium for software developers is much smaller than the broader population benefit. This is not that surprising — income variation within careers is typically much narrower than across careers.

Getting the gig

Of course, this all assumes you get the job in the first place.

It doesn’t matter how much a college degree impacts one’s earnings as a software developer if not having one effectively locks you out of the career.

It’s hard to judge how true this is. If you believe the hype around Lambda School and other bootcamp programs, a college degree is not necessarily a barrier to becoming a software developer. Realistically, few people without a college degree bother to pursue software engineering, but that doesn’t mean they couldn’t if they put their mind to it.

If bootcamps and other alternative forms of education can successfully train and place students in legitimate software engineering roles, then a college degree might not be required.

I think the jury is still out. Recent media reports have not been kind to coding bootcamps like Lambda. Then again, anecdotes may not represent the full story. For now, only the schools themselves have the data to prove their own efficacy.

That said, we can finally put some numbers behind the claim that college degrees don’t matter for software engineers. They matter, but not much, assuming you can get hired as a developer. Once in the seat, your degree matters much less than other factors, like on the job performance, competence, etc.

Remote Software Developers Earn 22% More Than Non-Remote Developers

Nnamdi Iregbulem — Tue, 24 Mar 2020 16:07:39 +0000

This post is an excerpt from my 2020 software developer pay analysis. Read the rest of the analysis here.

Organizations around the world have pivoted to remote work to shield themselves against the novel coronavirus / COVID-19 threat.

The sudden change prompts numerous questions. To start, how will this new (for some but not all) paradigm affect the way we work and collaborate?

One piece of good news among all the dread—remote work could lead to higher take-home pay, at least for software engineers.

Remote work pays

Fully-remote software developers earn 21.9% more than developers who never or rarely work remote.

Even once I control for various observable factors (including age, experience, hours worked, size of employer, programming languages, and more), fully-remote software developers earn 9.4% more than developers who never or only rarely work remotely.

Further, the remote work pay advantage increases proportionately with the time spent working remotely. Therefore, fully-remote developers earn more than

those who work remotely more than half the time, who earn more than
those who work remotely roughly half the time, who earn more than
those who work remotely less than half the time.

Even working only a few days each month yields a substantial pay advantage of 15.5% on average and 4.9% controlling for observable factors, and the estimates are quite precise.

In other words, half or more of the benefit comes from being able to work remotely at least a small portion of the time (vs. not at all), while the remainder comes from increased remote time, maxing out at fully-remote.

This is an impressive result.

While I can’t be sure this is causal, the large apparent effects (even after controlling for other variables) paired with the clear upward trend as remote work increases gives me confidence there is something real here.

Wait but why?

The obvious next question is… why?

Hard to say—by definition the adjusted pay premium already controls for any other data I had access to.

However, I can show how the other factors contribute to the explainable premium (i.e. the gap between the unadjusted and adjusted pay premium):

Here we can see some of the other factors contributing to the pay advantage for fully-remote developers:

years of professional coding experience,
age,
and a variable that proxies for the influence a developer has within their organization (their decision making power over new technology purchases)

Combined with other confounding variables, these account for 12.5 percentage points of the 21.9 percentage point pay advantage.

Much of the apparent premium earned by remote developers is in fact driven by seniority and tenure. These are older, more experienced developers who either prefer to work remote or whose organizations grant them that privilege.

That said, the remaining 9.4% adjusted pay advantage for fully-remote developers is nothing to scoff at. The other variables cannot explain this meaningful earnings bump.

This one weird trick

I find an economically and statistically significant pay premium for remote developers, even those who are only away from the office a few days each month. In fact, most of the earnings gains come from this initial foray into remote work, while the rest come with additional hours spent away from the office.

Would this hold in a world where remote work is more the norm than the exception? No idea.

Does this apply to non-software developers? Again, can’t say.

But it’s an interesting result nonetheless, and one that deserves further investigation as more knowledge workers shift their work online and away from the corporate office—virus or no virus.

How Age, Race, and Gender Affect Software Engineering Pay

Nnamdi Iregbulem — Mon, 24 Feb 2020 18:11:43 +0000

This is part 1 of my 2020 analysis of software developer earnings. You can find the rest of the results here.

Age. Race. Gender. Sexual orientation.

In an ideal world, none of these factors would matter for what a software engineer earns. As characteristics, they shouldn’t necessarily influence the productivity or value of a developer, and hence shouldn’t affect pay.

Turns out, however, they do — though not always in the way or to the degree you might expect.

In this first part of my 2020 analysis of software engineering pay, I explore how these demographic characteristics match with developer pay, tease apart correlation from causation, and explain the confounding factors driving some of the more surprising results.

Key findings:

Earnings peak around 45-50 years of age
Racial minorities make up both the highest and lowest-paid developers
Female software engineers earn ~10% lower salaries than their male counterparts, but professional experience explains most of the gap
Gay and lesbian engineers earn more than straight engineers after adjusting for observable factors
Parents earn significantly more that non-parents, but this is explained by other factors

If you haven’t already, please check out the methodology behind the analysis, otherwise the numbers below might be difficult to interpret.

Sign-up for my newsletter to receive a report with the full results

Age

Earnings peak in the late 40s

Developer earnings rise fairly steadily from the early 20s through the late 40s. The late 40s represent the highest earnings of a developer’s life , where the average developer earns 28.7% more than the typical 26-30 year old (the most common age range in the data), after which pay stabilizes before finally beginning to decline in the early 60s.

Age doesn’t matter much after 35

However, adjusting for other characteristics, age is most “advantageous” in the 31-35 range, when a developer can expect to earn 4.7% more than an equivalent developer five years their junior. This advantage is highly statistically significant.

The pay bump quickly dissipates with additional years however, losing statistical significance and even turning somewhat negative by the 51 to 55 range, though this is not precisely estimated.

The key point is that the additional income earned by older developers is entirely explained by factors unrelated to age. When we control for other factors, pay does not vary much by age after 35.

Little causal impact of age

Why does the correlation between age and income disappear after controlling for other factors? Let’s dig in:

The analytics demonstrate that years of professional coding experience matters much more than age itself, which is what you’d hope to see. These additional years of professional experience effectively explain the entire earnings premium for older software engineers.

Additionally important variables include self-rated competence, having influence over technology purchases in their organization, and working remotely (which older workers are more likely to do), and having dependents.

Race

Minorities are both the highest and lowest-paid software developers

As in last year’s analysis, the largest pay gaps are between minority groups, which make up both the highest and lowest-paid developers.

East and South Asians see the most statistically significant pay premiums relative to white developers with or without controls in the regression. In the case of East Asians, their pay premium increases after controlling for various factors.

These premiums are large and meaningful — East Asians earn 7.4% more than white developers and 13.9% more controlling for observable characteristics, while South Asians earn 13.1% unadjusted and 8.1% adjusted more than whites respectively.

Examining the East Asian pay advantage

Breaking down the explainable East Asian earnings premium yields some interesting findings:

First — years of professional coding experience is far and away the biggest factor holding down the pay of East Asian software engineers. East Asian developers typically have less work experience than whites, which holds down their earnings.

My calculations suggest that East Asian developer would earn 8.0% more if they had similar amounts of professional experience as whites
This rises to 9.0% if we add in additional, non-professional, coding experience.

Age also holds back East Asian developers, as they are typically younger than white developers. This amounts to a 2.0% pay disadvantage.

Lastly, the sheer magnitude of the earnings premium enjoyed by Asian developers should be noted. That the premium remains so large even after controlling for various factors is puzzling.

Good and bad news for black software developers

Let’s look at one more — the pay gap for black developers. The unadjusted gap — which again simply compares average earnings of black and white developers — is -7.6% , while the adjusted gap contracts meaningfully to -0.3% , which is not statistically significant:

Breaking down the explainable gap for blacks reveals similar drivers as we saw in the East Asian case. Years of professional coding experience is the main contributor to lower pay for black software engineers relative to whites, in total driving 5.8 percentage points of the overall 7.6% gap. Nothing else matters nearly as much.

In a sense, this is heartening. Assuming a black engineer gets as much out of an additional year of work experience as anyone else, purely closing the gap there would bring black pay nearly in line with white pay.

On the other hand, that the unadjusted gap is explainable via years of experience also means that the gap is unlikely to close anytime soon.

Why? The age structure of the workplace is slow to change — it takes decades to see sizable shifts

Additionally, as the industry diversifies, by definition most black professionals entering the software development career track start off at the bottom rung of the ladder
Thus, the diversification of the industry in fact depresses average black earnings, as fresh out of bootcamp developers don’t earn nearly as much as seasoned veterans

This is not a bad thing — but it does mean I can’t conclude something nefarious is behind the slow convergence of black and white wages without other evidence.

The other factor worth touching on is ImpSyn which is a variable representing a respondent’s own confidence in their skills as a software developer. More confident developers earn more, and there appears to be a confidence gap between black and white developers driving 1.1% of the earnings gap.

Gender

Young women entering the software development workforce pull down average female earnings

Women earn 10.0% less than male software engineers on average, a sizable difference. However, this gap is effectively eliminated once we adjust for controllable factors, falling to only 1.4% , which is not statistically significant.

In diagnosing the unadjusted 10.0% pay gap for women, years of experience pops up once again as a dominant factor explaining most of the gap:

5.7 percentage points of the gender pay gap can be explained by the fact that female developers have less professional experience than male developers on average. Adding in overall coding experience explains 7.1 total percentage points of the overall gap.

While women are only 1.7 years younger than men on average in the dataset, they have 3.3 fewer years of professional coding experience (7.1 years for women vs. 10.4 for men). We can see this in the histogram comparing the respective distributions of years of professional coding experience of men and women, where the distribution for women is shifted and more clustered to the left. This meaningful difference explains why professional experience is such as major driver of the gender wage gap.

Confidence (ImpSyn) comes up again as a factor pulling down female wages. Here, the confidence gap explains 1.0% of the overall female-male gender gap, very similar in magnitude to that of black developers vs. white developers.

Small contribution from other factors

In line with other research, women are also less confident about their own programming skills than men are (who for all we know might be overconfident), which explains another 1.0% of the total gap (because higher confidence leads to higher pay, as I cover in a later post).

Experience and confidence collectively explain 8.1% of the gender pay gap among software engineers, leaving only a 1.9% gap, including the 1.4 percentage point difference that we cannot explain.

Sign-up for my newsletter to receive a report with the full results

Sexual orientation

Earnings penalty for non-straight developers disappears after controlling for other factors

Unadjusted pay gaps among non-straight software engineers ranges from 2.5% for gay and lesbian developers to 9.6% for bisexual developers, which simply means these individuals earn less on average than straight engineers.

In the case of gay and lesbian developers however, this gap closes and actually reverses once I add controls. The gap becomes a pay advantage of 3.4%.

Starting at the adjusted / unexplained 3.4% premium, the lower average unadjusted earnings is largely accounted for by years of professional coding experience (a recurring theme) and age, suggesting that gay and lesbian developers are simply earlier in their careers than straight developers, on average.

The decomposition of the difference in adjusted and unadjusted pay gaps is quite similar for bisexual developers. Professional experience and age continue to the do the most explanatory work here:

Parenthood

Parents earn more, but this is largely explained by other factors

Software engineers with dependents (typically children) earn 17.2% more than those without, but this earnings premium can be almost entirely accounted for via the other factors described in this analysis.

Controlling for these variables reduces the parenthood earnings premium to 1.5% , which is small but statistically significant.

Perhaps the data reflects that these developers earn extra income in order to take care of their dependents
Such a small bump in earnings like does not cover the additional expense of a child or other dependent, however

Unsurprisingly, years of software development experience and age account for most of the earnings premium among parents. These are largely mid and late career engineers.

Working remote explains another 1.1% of the pay premium among parents, as they are more likely to work from home, which makes sense given they may have a child to take care of.

Conclusion: Significant progress on pay

Not to editorialize, but I was encouraged by many of the results here. In general, along most dimensions, discrimination in software developer earnings appears small once various factors are controlled for. In most cases, the biggest factors were some combination of years of coding experience and age which are both “problems” that will largely fix themselves as the industry diversifies.

With the exception of race, most of the gaps are no more than a few percentage points in magnitude. In the case of race, the gaps are meaningful in some cases but difference is in fact in favor of minority groups like East Asian, South Asian, and Middle Eastern software developers. Their pay advantages are substantial, and the data from the Stack Overflow survey fail to fully explain these gaps.

The usual caveats to pay analyses apply here as well. Finding little discrimination on pay, after controlling for factors such as job title, does not disprove discrimination in its entirety.

For example, if female software engineers face discrimination in the form of reduced upward career mobility, that will not show up in this analysis, even though it depresses their earnings
The same would apply to older software developers, who may be pressured out of their organizations to make room for cheaper, younger developers.

Analogous statistical caveats abound.

That said, the data suggest the software development industry is well on its way to pay parity across the important dimensions of age, race, gender, and sexuality.

Thanks for reading part 1 of my 2020 analysis of software developer pay. You can find the rest of the analysis here.

I analyzed the earnings of 11,000 software developers. Here's what I found.

Nnamdi Iregbulem — Tue, 15 Jan 2019 11:48:50 +0000

Meet Dev.

Dev fell in love with code at a young age.

He graduated from college with an engineering degree and then joined the Navy as a cyber warfare engineer. Pressured by his highly educated Persian parents, Dev returned to school after military service to complete a master's in computer science. He has spent his entire career since in software development, mostly coding in C++.

He realized early on that workloads were moving to the cloud and spent years retooling and upgrading his skill set, familiarizing himself with each of the major cloud vendors. He championed his team's transition from an on-premise Oracle database to a next-gen database hosted in the public cloud.

Always on the cutting edge, Dev regularly attends employer-sponsored training sessions and hackathons.

Dev worked with all of the major programming methodologies over the years but eventually settled on Agile, becoming a big fan.

Now a manager at a large, 15,000+ employee technology giant, he takes time to work out 3-4 times per week at the free company gym. Even after completing an agile sprint he keeps moving - jogging home after work most days.

After promotion, Dev gave his second monitor to one of his younger direct reports. He wasn't spending as many hours furiously coding as he used to, so the extra real estate felt unnecessary.

A big proponent of healthy work habits, last year he procured a full set of standing desks for his entire team and required that all team members eat lunch away from their workstations. Rarely rushed himself, he never skips meals.

During the week, Dev wakes up just after 11am. Every. Single. Day.

Must be nice right?

However, at 50 years of age, Dev worries if his career has already peaked. He has no plans to retire any time soon, but whispers of ageism in tech and seeing his older colleagues being laid off have him wondering if he's next.

That said, he knows he could make up lost income working independently as a freelancer. He has a hunch he'd make even more running solo.

A proud, loving father, Dev hopes his daughter follows in his footsteps. He sees a bright, better future for her. As far as he's concerned, the world is her oyster.

Who is this guy?

If it's not clear already, Dev's not a real person.

But his personal and professional characteristics really do correlate with higher income.

Age. Gender. Race.

Education. Professional experience. Programming languages.

Methodology. Hackathons. Even how many monitors he uses.

The traits that correspond with higher pay are not always what we would expect - nor necessarily what we'd hope.

The economics of developer pay is an important topic for me. Developers are the key input in the production of software. You cannot understand tech unless you understand how developers get paid— you just can't.

However, the publicly available analysis of this important issue is quite poor.

The problem with most pay analyses

As covered in my last post, Stack Overflow conducts an annual survey of software developers, asking about various aspects of their careers, like income, job title, etc.

One disappointing feature of most pay analyses is that they show the data without controlling for any other variables. Avoiding any math more complex than simple averages, the typical analyses of pay have no way to understand a multivariate world.

Stack Overflow’s own write-up of the survey results does this at times, taking the simple average of income across all full-stack developers, for example, and comparing the same metric for DevOps specialists:

Statements like "Engineering managers, DevOps specialists, and data scientists command the highest salaries” are then made, which, though technically true, tend to mislead readers who are not well-versed in statistics into thinking that these are ceteris paribus (all things equal) comparisons ("X makes more than Y, all else equal"), which they are not.

To their credit, Stack Overflow at least takes the additional step of comparing these salaries against years of professional experience, but that is just one of many possible controls. In fact, the entire survey provides a rich set of potential controls to “hold equal” and thereby generate more intuitively accurate statements about potential relationships (“X developers make more than Ys who are otherwise similar”, which is a qualitatively and likely quantitatively different statement than “Xs make more than Ys").

Unfortunately, most public discussion of pay does not control for even a single variable. This isn’t lying with statistics so much as it is misleading with them.

Said simply—we don’t only want to know “how much more or less do 45-year-old developers make compared to 25-year-old developers.” More importantly:

“How much does a 45-year-old developer earn relative to a 25-year-old developer who is otherwise equivalent (in skills, experience, age, company size, country etc.)?”

Both are important questions, but to me, the latter is much more interesting and intuitive. It also happens to be much more difficult to answer—hence it is rarely attempted.

This is an attempt do better. Not perfect, but much better.

Results

I break down the results by survey question, with a chart displaying the controlled effect of each trait on income, in addition to 95% confidence intervals. Correspondingly, any references to statistical significance represent p-values < 0.05.

The results represent a subset of approximately 11,000 U.S-based developers from the Stack Overflow survey.

If you’d like a nicely formatted report with the full set of results, enter your email below and you’ll receive an email with a download link when ready.

For more detail on the methodology, please see the appendix at the end.

Demographics

Initially, age increases earnings. That said, the annualized compound gain is quite meager—about 0.2% through 45-54 years of age—with most of that happening before a developer turns 25.

More interesting is how the impact of age declines and turns negative (relative to a developer of 18-24 years of age) as a developer approaches 65:

At this level of precision, I cannot distinguish the pay of a developer 55 or older from an otherwise similar one younger than 25
I can say with reasonable certainty that a developer over the age of 65 makes less than one of 18-25 years of age who is otherwise similar.

Ageism could be playing a role. Tech companies, especially startups, are perceived to have a preference for younger employees. Evidence of ageism is often anecdotal, but this data is at least suggestive that there may be something real behind this concern.

Unfortunately, ageism is notoriously difficult to prove or disprove , exactly because it is exceedingly rare to find a 25-year-old who is, in every way other than age, the same as a 65-year-old.

For starters, a 25-year-old developer cannot possibly have much more than 5 years of professional development experience, while a 65-year-old developer almost certainly does. And that is just one variable.

This is certainly an issue worth exploring further.

Asian and middle eastern developers are paid much more than similar white developers, and the pay premium is likely larger than the pay discount faced by other minorities.

Black developers appear to earn 0.8% less than white developers with similar traits, though with a wide enough confidence interval to lose statistical significance.

The lack of precision (plus or minus 5%) is frustrating but inevitable given the low proportion of black software developers in the population and the dataset
Note: Only 1.5% of developers in my dataset are black (not significantly different than the overall developer population)

Hispanic or latino/a developers earn 1.5% less than similar whites. Again, we see reasonably large confidence intervals due to lack of sufficient data.

That there is such a large pay premium for asian and middle eastern developers is an interesting factoid in itself and one that warrants further exploration

The estimates range from 5% to 10%, and all are statistically significant.

The size of these effects are especially impressive given the small proportion of the dataset these minority groups respectively represent:

East Asian—2.8%
Middle Eastern—0.6%
South Asian—3.6%

That they are statistically significant even with few data points suggests these effects are quite real.

Pay discrimination is a serious issue that warrants rigorous analysis, much more than what I’ve done here.

All in all, the results suggest that attempts to level the “paying” field should also focus on equalizing pay across various racial minority groups, not simply between minority groups and the majority

The pay gap for female software developers is similar in magnitude and low statistical significance to that of black developers—roughly 1.3% plus or minus 2.4%.

Gender non-binary / non-conforming developers face a large pay discount of 10% relative to male developers.

Given very few observations in the dataset, the pay effect for transgender developers cannot be estimated precisely enough to conclude anything meaningful.

Gay and lesbian software developers appear to make 2.5% more than straight / heterosexual developers, but this estimate is not statistically significant.

In fact, given the confidence intervals above, none of the categories of sexual orientation are statistically significantly different from heterosexual.

Again, this is in part due to limited data—only 2.4% of developers in the sample are gay or lesbian, for example

Good news for parents—developers with dependents (children, grandparents etc.) earn 3.7% more than those who don’t, controlling for other factors.

There are a number of potential explanations for this

Workers with dependents likely want / need the extra income and hence seek out jobs that pay more
I do not have evidence to conclude that employers are specifically choosing to pay those with dependents more than others, though this could also be the case

Former and current U.S. military service members earn 3% more than those without prior military service.

As with other traits, there are any number of potential drivers of the veteran pay premium, and it's inherently difficult to pinpoint the most likely explanation.

Takeaways

Drawing justifiable conclusions around the impact of demographic characteristics on income is necessarily tricky given the highly imbalanced nature of the developer population
Caution is advisable when positing causal connections, and these should ideally be accompanied by some theoretical mechanism
That said, the results are interesting and should hopefully serve as a starting point for deeper and more rigorous analysis.

Education

Advanced, non-professional degrees drive higher earnings. A masters or doctoral degree (PhD etc.) drives statistically significant gains of 3% and 10%, respectively, relative to an equivalent developer with only a bachelor’s.

Going to college but not completing is associated with a 3% cut in earnings, while graduating with an associate degree decreases earning by about 8% vs. a bachelor’s. Again both are statistically significant.

The effect of professional degrees like JDs and MDs is not statistically significant due to wide confidence intervals.

Very few people with these degrees are working as software developers, so establishing a precise estimate is difficult.

Not much can be said for developers that never reached the college level—due to lack of data, the confidence intervals are too wide to draw meaningful conclusions. However, these folks are almost certainly worse off than someone with an advanced degree.

Interestingly, computer science is not the best-paid college major.

In fact, other engineering disciplines (civil, mechanical, electrical, etc.) ( 5% increase ), business degrees (accounting, finance, etc.) ( 4% increase ), and math / stats majors ( 3.4% increase ) all earn more, all else equal.

Outside of these areas, college major is largely irrelevant to developer pay. The meme of the underpaid, over-educated sociology major doesn’t apply.

This could be encouraging in that it shows you can be paid for good dev work regardless of your field of study
With dev schools and bootcamps like Lambda popping up all over, this is great news if you are considering a career change and worried you might be at a disadvantage.
On the other hand, it does raise the question—why aren’t developers who studied the most apparently relevant field paid more than all others?
Does this say anything about undergraduate computer science teaching, which is often criticized for being irrelevant and out of touch with professional software development work out in the wild?

The exceptions are information systems / technology and web development / design degrees, which bear a pay discount of 2.4% and 10% respectively.

Why? My take is that these degrees are often associated with private, for-profit schools not often known for their quality

Your parent’s level of education matters as a software developer.

Developers with at least one parent holding a professional degree like a JD or MD will earn 5.3% more than a developer whose parents never earned more than a bachelor’s degree. Masters and doctoral degrees among parents are associated with a 2.2% increase in pay for the children, though this relationship narrowly misses statistical significance for doctoral degrees.

Associate degrees, some college, and high school all have significantly negative impacts of 2.8-4%.

Primary school and no formal education were too rare in the dataset to generate precise effect estimates.

It’s interesting to consider how these effects manifest themselves throughout the professional life of a developer

We can all imagine ways in which parental education could have meaningful follow-on effects in the lives of children
It is also possible that, with more controls, these effects would dissipate or lose significance
For example, the effect may channel itself through income or other variables that correlate with (or are influenced by) education, rather than schooling itself

Participation in hackathons is clearly associated with higher pay—more than 4.3% higher, as are full-time developer training programs (bootcamps), which provide a 3% bump.

We’ve witnessed an explosion in the number of coding bootcamps over the past decade. The pay bump is equivalent in magnitude to a master’s degree, which is incredible given bootcamps take months to complete, not years, and are generally cheaper in tuition than an advanced degree.

Contributing to open source software, or OSS (certainly an educational experience, in a sense), has a positive but not statistically significant bump of 1.6%

Open-source penetration of the typical software stack has only increased over the years—it would make sense if active contribution to OSS is rewarded with higher pay.

Industry certifications get a big fat zero. No effect.

On the other hand, online programming courses, MOOCs, etc. are associated with a significant drop of slightly more than 2.4%. Given the proliferation of online software development courses over the years, this is an inconvenient finding.

That all said, I do worry about potentially confounding or omitted variables that have not been controlled for
I can’t imagine a strong causal link between taking an online course and lower pay, but I can certainly imagine that the "type of person” who takes an online course might also be the type of person that generally earns less

Likewise, the type of person to participate in a hackathon likely has other, unobserved traits that drive higher income

Certain companies, which may be higher paying on average, also host hackathons in their offices

In other words, correlation may not be causation here, even after controlling for many variables.

As an avid self-learner myself, I am disappointed to see that no form of self-learning seems to have a statistically discernible impact on earnings.

Books and e-books were close to achieving statistical significance, but even so, the point estimate is only 1%.

Self-learning is a great way to, in targeted fashion, learn exactly what you need to know about a certain topic or area of knowledge. For many, it is more effective than traditional teaching methods.

However, it doesn’t seem to impact pay in and of itself.

Takeaways

Education matters, largely in the ways one would guess
If you’ve already graduated from college, advanced degrees, hackathons, bootcamps, and open source projects can be smart ways to increase your value as a developer in ways that show up in your paycheck
If you haven’t graduated yet, know that the exact major you pick matters less than showing interest or commitment to the craft, at least when it comes to pay

Professional Characteristics

Professional development experience matters much more than casual coding. The gains from more years of general coding experience tend to plateau after 15 years, while professional coding experience continues to pay dividends well into the 30 year range.

The benefit from experience with casual coding (relative to 0-2 years) maxes out at 10-15% and actually begins to decline after 26 years.

Again, this is holding other factors equal, including professional experience, so this may make sense
Someone who learned to code 30 years ago and has little more to show for it than someone with only 10 years of coding experience might raise flags.
We must be careful however—taking the year one learned to code as given, years since learning to code correlates perfectly with age, suggesting that ageism may be leaking in here. I explore age specifically elsewhere in this post.

The fastest gains from professional experience come in the first ten years.

A developer with 9-11 years of professional experience can expect to earn 30% more than a newbie, all else equal, which translates to about 2.5 percent year-over-year gains.
Of course, “all else” is likely not equal for any given developer across a ten-year span, suggesting even greater potential annual raises for a given developer in the real world.

After the first decade, the line is nearly straight, aside from a mid-career slump at the 25-year mark.

Do not be fooled, however—constant absolute gains in fact represent declining growth
In other words, each year of professional development experience provides a nearly-constant dollar raise but a declining percentage raise. I explore this phenomenon further in a upcoming post

With that caveat, it is still encouraging to see that developer pay does not fully plateau or reverse course with additional professional experience over time.

No surprise here—working part-time does not help your earnings.

More interestingly, developers working as freelancers or independent contractors make more than those employed full-time

This may be compensated for by what is almost surely more variable income
In general, stable, full-time employment does come at a cost, not only in software development but in many other areas of the economy

Reverse causation could also be at play here—someone who knows they could make more as a freelancer is exactly the kind of person who would become one.

(Read: “DevOps specialists make 2% more than non-DevOps specialists, all else equal”)

It’s no surprise that managers and executives earn more. This includes product managers, who might not write significant amounts of code themselves.

Engineering managers and C-suite execs make 10% more, while product managers make 5.8% more, all else equal.

The only non-manager role that makes meaningfully more than its counterparts is DevOps:

DevOps specialists are about 2.2 percentage points higher on the pay scale than the non-DevOps average for similar developers.

Conversely, there are a number of roles that make noticeably less despite similar levels of experience, education etc.

Database admins (DBAs), sysadmins, designers, QA / testing developers make 2.5-7.5% less than developers who don’t fall under these categories
With the exception of designers, these roles are commonly seen as the “back office” of IT, though one could easily find counter examples depending on the specific team or organization
Perhaps needless to say—academics and students earn meaningfully less than those outside of academia

Takeaways

Professional experience matters and never stops mattering
Casual development experience helps too—but only so much, measured both by the amount of experience and the earnings impact
Among those working in industry, role matters positively for managers and DevOps professionals and negatively for system and database administrators, as well as designers

Languages, Frameworks, Databases, Platforms, and Tools

(Read: “Developers who have done extensive development work involving Golang earn more than those who haven’t”)

The main implication of these results—languages largely don’t matter to pay. The vast majority of languages do not provide a statistically significant pay bump or discount relative to jobs that don’t require that language. The pay scale for programming languages is quite flat across the universe of languages.

Hack is a clear outlier—working with Hack yields a 25% bump. However, don’t drop everything to go learn Hack right now

Hack is not a widely used language
It was developed by Facebook, and unlike other languages and frameworks that come out of the big tech giants, Hack has not had the same marketing push placed behind it.
The effect we see here likely reflects developers who work at Facebook itself—there’s no sign of a robust hiring market for Hack talent.

Outside of Hack, on the positive side we find Objective-C, C++, Go, VBA, and TypeScript with statistically significant income effects, ranging from 2.1 to 3.9 percent.

On the negative side we have JavaScript, VB.NET, CSS, and PHP

JS, CSS, and PHP likely reflect the lower pay of certain web development jobs
These languages represent fundamental technologies of web development
For that same reason however, they are considered table stakes
Hence, using these technologies on the job doesn’t provide much of a pay bump, though you may be quite employable

Please note that the results above are not saying “knowing language X increases / decreases pay.” Knowing a language is not likely to ever harm one’s pay, but working a dev job that uses it might (relative to other development work).

React—what needs to be said?

The cutting-edge JavaScript framework originated and maintained by Facebook is widely and wildly popular, especially among front-end developers looking for an easy-to-use but powerful way to craft compelling user interfaces.
React is associated with a 4.1% pay increase and is the only framework with a statistically significant positive impact.

Working with .NET Core and PyTorch negatively impacts income

I do not have a hypothesis about .NET Core, but I do know that PyTorch was historically used much more among researchers than industry practitioners, likely driving lower pay
Usage has since branched out as the framework has added production-grade capabilities and tooling.

One word—cloud. Almost all of the databases with positive income effects are hosted in the cloud, often exclusively as a managed service by one of the major public cloud service providers (Google, Amazon, Facebook).

Google’s BigQuery data warehouse tops the list, followed closely by Amazon’s competing Redshift. Apache Hbase and Hive come next. Afterwards it’s Amazon’s DynamoDB and Microsoft’s Azure Tables, Cosmos DB, and SQL. The list continues with Memcached, Redis, Elasticsearch, Google Cloud Storage, and Amazon RDS and Aurora—all positive and statistically significant.

Older database technologies from legacy vendors dominate the negative end. This includes Oracle’s once ubiquitous databases, MySQL, SQL Server, MariaDB, and IBM’s Db2.

It’s fascinating to see the impact of the cloud revolution laid out in stark relief. There are meaningful income gains associated with working with next-gen cloud-enabled databases. This fact should be top of mind for developers looking to upgrade their skills and pay.

I’ll say it again—cloud. The various public clouds are the only platforms aside from Windows Phone with positive, statistically significant effects on developer earnings. Azure is associated with 4.1% higher pay, GCP 2.7%, AWS 1.7%.

And Windows Phone? Not going to try to explain that one.

Android, Arduino and Drupal had meaningful negative association with income—2.4%, 4.6%, and 8.5% respectively.

Android’s ubiquitous operating system needs no introduction, though it’s negative pay relationship might need an explanation. Unfortunately, I am stumped here.

Arduino is an open-source prototyping platform with associated hardware microcontrollers that eases development of electronic devices. Though used for serious development work, Arduino is also heavily used by engineering students as well, which likely drives the negative value.

Drupal is a PHP-based content management system whose popularity peaked in 2011 but has since been on a slow decline.

You might not think that a developer’s choice of IDE or text editors wouldn’t matter—and you’d be (largely) correct! The associated effects here are generally quite small.

However, IntelliJ stands out from the pack

Billed by JetBrains, its creator, as the “Java IDE for professional developers”, IntelliJ users make 3.1% more than non-users
Of course, this is not an entirely useful comparison for someone who doesn’t use Java in their development work

Notepad++ and Atom are associated with slightly lower earnings, on the order of 2%, which is statistically significant. Notably, both are popular but are generally viewed more as text editors than fully-fledged IDEs.

PhpStorm, Xcode, and Coda also had negative effects that were statistically significant.

The incredibly popular Agile leads the pack among project management methodologies. Devs who use Agile in their development work earn 3.9% more than those who don’t.

The less common extreme programming methodology is associated with 2.7% higher pay.

Lean comes in at 2.5%, while Scrum developers earn 1.5% more, all else equal.

Prince2 appears to be at outlier, but this is mostly due to limited data—only 0.1% of developers in the sample use the methodology.

For the most part, the exact form of version control used does not matter.

The key takeaway here—any kind of version control is better than no version control, but please, please avoid copying and pasting files to a network share.

The best practice appears to be the best paid

Checking in code multiple times per day
It’s clearly important to check in code on some kind of cadence—at least monthly, ideally
If you are checking in code less than once per month or not at all, your pay is likely being impacted as a result.

Not to sound like a broken record, but the reverse causation caveat applies here as well. It is completely plausible that checking in code more often increases a developers pay. It’s also possible that companies that pay more are more likely to follow and mandate development best practices, such as frequent code check-ins.

It seems too convenient that in a survey run by Stack Overflow that its own enterprise product would be associated with significantly higher developer pay, but I can only go where the data takes me. Developers using Stack Overflow Enterprise earn 9.4% more than those who don’t.

This likely picks up some effect of simply working at a company that uses Stack Overflow Enterprise, which may be higher paying on average.

Other communication tools had smaller positive impacts like the incredibly popular Slack, Atlassian’s Confluence, HipChat, and internal intranet sites (wikis, Google sites etc), which all have earnings effects in the 2-3% range.

Trello and Facebook are negative enough to be statistically significant.

Surprisingly, developers who use two monitors earn 2.8% less than those who use only a single monitor, all else equal.

At this time, it’s not clear why more monitors would negatively impact earnings. The potential for enhanced productivity may be swamped by the temptation to use the additional screen real estate for non-work activities.

No other monitor count had a statistically significant effect on income relative to one.

Takeaways

The tools associated with higher developer pay are quite interesting and not necessarily what one might expect
In some cases, the most popular tools also pay the most
In other cases, more obscure tools appear to have an advantage
Check in your code (at least every once in a while)!
One clear trend is the impact the move to the cloud is having on developers—the effects of the public cloud on developer pay are large and consistently statistically significant across the big 3 U.S. clouds
Knowing how to use and leverage these next-generation computing environments and finding a job that employs those skills can drive meaningful pay improvements for the average developer

Work Life, Health, and Wellness

These results are admittedly difficult to interpret. There is no clear linear time trend for the impact of wake up time on earnings.

After 7am, later does appear to be somewhat better—up to a point. Strangely, developers between 11am and 12pm see 15.4% higher pay than those who are up before 5am.

The most important takeaway is this— have a set schedule. This will do more good than optimizing for a specific wake-up time. Not having a regular wake up time was associated with 7.7% lower pay for software developers.

Exercise is strongly associated with earnings. While only exercising once or twice a week does not appear impactful, exercising 3 times or more per week is associated with 2-2.9% higher pay than a similar developer who does not exercise at all.

It is possible however that reverse causation could cause developers who earn more to work out more—perhaps because they have more time or can more easily afford a gym membership. Alternatively, higher paying companies often have gyms on-premises, making it easier to work out more.

Don’t skip meals. Any amount of meal skipping was associated with lower pay, though never statistically significant. Developers are not rewarded for this unhealthy work habit.

Among ergonomic devices, standing desks were associated with 3.2% higher pay, which is statistically significant.

Again, higher paying companies are potentially more like to provide employees with standing desks, so the direction of causality here is questionable.

Time spent at the computer each day does not have a meaningful relationship with pay. Handcuffing yourself to your laptop is not going to earn you higher pay as a developer.

Unlike time spent at the computer, time spent outside does have an impact on pay, with the ideal amount being 1-2 hours. Spending fewer than 30 minutes outside is associated with 2.4% lower pay than 30 minutes to an hour.

Spending more than 4 hours outside was associated with lower pay, but there were not enough developers who do this regularly to generate a precise estimate.

Takeaways

The data suggest that common best practices are often the best way to go
Spend at least a small amount of time outside each day
Skipping meals will only grow your bank balance to the extent you save money on lunch
Exercising a few times per week is better than never hitting the gym

Conclusion

There are many important takeaways from the study and the charts above.

This analysis is a first attempt at exploring the various factors that affect developer pay. To that end, I hope that it is illuminating and informative.

However, as with many attempts to answer difficult questions, the analysis raises as many questions as it answers.

I am publishing the full code to reproduce this analysis because I believe open source, replicable research is the key to the robust advancement of knowledge

I would love for this analysis to serve as starting point for others who wish to elevate the state of knowledge on this important topic
If you find an error or disagree with some aspect of the analysis, feel free to submit edits (pull requests) to my GitLab or GitHub repositories

I care deeply about the technology industry. But solving its issues and compounding its strengths demands a rigorous understanding of its component elements. Developers are a critical piece of the tech puzzle, and they deserve our attention.

Appendix

Data

As in my last post, I leverage data from Stack Overflow’s annual software developer survey, which asks about income, in addition to many other questions of interest.

Examples include

Which of the following best describes the highest level of formal education that you’ve completed?
Approximately how many people are employed by the company or organization you work for?
Which of the following programming, scripting, and markup languages have you done extensive development work in over the past year?

I’m interested in how answers to these questions affect income. While it’s impossible to completely avoid issues of reverse causality or correlation rather than causation, by using regression augmented with machine learning techniques, described here, we can have greater confidence that our results accurately represent the true relationship income.

Essentially, we’ll analyze how each possible answer affects income, holding all other answers constant.

I limit the dataset to only non-retired US respondents above the age of 18 with income between $10,000-250,000. Responses above $250K have a higher tendency to be troll (i.e. made up) responses, which I’d like to exclude, and few answers come in above this threshold regardless.

This leaves us with a dataset of approximately 11,000 developers.

Methodology

I estimate the following equation on the data:

Where T is our trait of interest, B1 is the effect of that trait on income relative to the base category, X is a set of controls (in our case, the respondent's answers to other questions in the survey), B2 is the set of effects for each respective control, and e is the irreducible error in our estimate.

Assuming we’ve included a “complete” and “correct" set of controls, this should provide a reasonably accurate estimate of the relationship of the trait of interest, T, with income. The log transformation of income means our results will be in roughly percentage terms, implying that trait T is associated with an increase in income of B1*100% relative to the “base" category, all else equal. The base category will vary by trait.

Selecting the right set of controls is non-trivial. One has any number of degrees of freedom to select a subset of controls among all those available, opening the doors to “p-hacking” and other infamous behavior, which can lead to incorrect (biased) estimates of the parameters of interest.

To avoid this, I leverage a powerful machine learning technique, Double Selection (specifically Double Lasso Selection), to do principled covariate selection for each trait T, rerunning the regression for every trait. As described in Belloni, Chernozhukov, Hansen (2011), this should provide a more accurate estimate of the income effect of each trait than simply using all the covariates or attempting to manually select a subset. I won’t cover the method in detail here, but refer to the original paper for more information. This post provides a relatively intuitive explanation, as is also where I originally learned of the technique.

Long story short, Double Selection makes us much more confident that the results represent the accurate effects.

The Growth-Share Matrix of Software Development

Nnamdi Iregbulem — Thu, 22 Nov 2018 23:45:00 +0000

Human capital is our greatest asset.

Like financial capital, the all-powerful force of compound growth means that a small difference in the rate of skill acquisition over time can lead to massive differences in career outcomes.

Choosing which skills to hone is therefore one of the most important keys to professional growth and success in any arena.

Among various forms of human capital, technical aptitude is quickly becoming the mission-critical skill for 21st century knowledge work.

However, there are any number of technologies that one could dive deep into and attempt to master, with varying usefulness and practical applicability.

So how does one decide where to "invest" among a sea of options?

A mental model I've found surprisingly helpful for this task is the growth-share matrix, a framework concocted 50 years ago by the Boston Consulting Group.

The framework was originally conceived as a tool to help executives prioritize different business units based on their respective relative market shares and growth. The two dimensions separate the market landscape into quadrants, each with certain characteristics:

Stars (High Growth / High Share)
Cash cows (Low Growth / High Share)
Question marks (High Growth / Low Share)
Dogs (Low Growth / Low Share)

BCG advised clients to invest in the stars, exploit the cows for their cash flow, evaluate the potential of the question marks, and exit or sell the dogs ASAP.

The growth-share matrix was originally intended to apply to product lines or business units - an asset a corporation could own. In that respect, you might imagine this framework has limited applicability to programming languages, given no single person "owns" any given language.

Not so fast!

I argue that we each do own a little piece of a programming language, not in the form of equity or stock, but in the form of human capital.

Through careful curation of a "portfolio" of useful skills, we earn a return on our learning efforts - rewards for our time and effort.

Learning a programming language is a perfect example of this.

But what makes a programming language useful?

The value of any given programming language is a direct derivative of the number of other individuals who know that language and the number of companies using that language to develop and ship products.

That may sound obvious to some, but it is in fact quite counter-intuitive.

In most of life's skills we seek to master the rare things - the things no one else can do. We think that by differentiating ourselves through a unique set of talents we will shine brighter in an increasingly competitive world. Learning that which is rare will pay meaningful dividends, so the thinking goes.

While yes, knowing an obscure language that few others have familiarity with might carve out a nice niche in the market for you to charge highly for your rare talents, I would argue that, for most, it is actually more valuable to know a language that lots of other people also know , rather than one only a few have ever worked with.

Most people think that programming is how you speak to computers. Really, it's how you speak to other developers.

Due to network effects and the increasing size, scale, and scope of software development projects and teams, knowing the "lingua franca" is much more valuable than being an expert in some endangered language, soon to be discarded to the trash bin on the desktop of history.

Paradoxically, having low personal market share in a high market share language is actually not such a bad thing.

Building the Matrix

To construct the growth-share matrix for programming languages, we will leverage StackOverflow's annual developer survey. For the 2018 edition, they surveyed over 100,000 developers from around the world, covering a wide range of topics from job satisfaction to salary. Here we'll focus on US-based developers.

The key question for our purposes is:

"Which of the following programming, scripting, and markup languages have you done extensive development work in over the past year?”

Answers to this question should give us a rough proxy of the popularity of any given language, as defined by the proportion of developers who have worked with a particular language.

For growth, we can compare the answers to this question across 2017 and 2018 to come up with an estimate of the growth of each language. We'll define growth as the % growth rate of the proportion of respondents who've worked with the language in the past year. So, a language that went from 10% coverage to 13% would be considered to have grown 30% (rather than 3 percentage points).

Quadrant boundaries will be set at the median growth rate and relative market share.

One final piece - as is convention with the growth-share matrix, we will show market share relative to the language with the most market share. Therefore, the axis will end at 100% (representing the most popular language). We will also show this on a log scale to better showcase the distribution, which tends to be quite crowded below 10% relative market share.

We now have all we need to build our growth-share matrix!

Inside the Matrix

Take a look at the results:

One striking feature that immediately jumps out - very few languages saw a net decline in popularity. Almost every language grew , which by definition implies that the average developer is using an increasingly wide array of languages in their work.

Let’s spin through each quadrant and discuss some of the highlights.

Stars

Python - Python has existed for decades but only recently hit its stride as a go-to language for data analytics and machine learning use cases. Python is widely-regarded as one of the best languages for data-driven analysis given its relative ease of use and massive set of open source libraries that simplify and accelerate analytics. Python's syntax is quite simple compared to other languages. Ease of use and speed are especially important in data science, as data scientists often run and re-run numerous iterations of a model before settling on a preferred specification. The growing popularity of interactive and replicable computing environments like Jupyter notebooks dovetails nicely with Python's surging share among developers. I'm personally quite pleased to see Python's high popularity given I've spent the past 2 years self-teaching myself the language!

Ruby - Ruby has historically been known for its extreme ease of use and strength within web development. Many a web developer wrote their first web app in Ruby. The Ruby on Rails framework only extended this user-friendliness further, making Ruby incredibly popular among developers who want a no-frills way to quickly develop and deploy functional web applications. For several reasons however, Ruby's growth is slowing and has been for a few years now. No, Ruby is not “dead”, but it will likely migrate to the cash cow zone soon as the initial fanfare wears off. Ruby continues to be a great language that serves developers well.

Go - A new language seeing rapid adoption among developers, Go simplifies the process of writing code, thereby making developers more efficient. Go was initially birthed at Google, where technical teams were trying to solve engineering problems that only seemed to be multiplying in an era of increasingly large codebases, multicore processors, and network-aware applications. Go is built with concurrency in mind, making it relatively easy to build multi-threaded applications. Outside of Google, major companies making use of Go include Uber, Netflix, Adobe, IBM, Intel, Dropbox, CloudFlare, and more.

TypeScript - I debated including TypeScript as its own language here given its strong similarities to and overlap with JavaScript, but developers with experience in the language seem to be a distinct group worth highlighting. The language has also seen a surge of growth in the past few years. The fundamental goal of TypeScript is to ease development of large-scale applications that would otherwise be written in vanilla JavaScript. Accordingly, TypeScript is a superset of JavaScript that also compiles to simple JavaScript. Why the distinction then? Typescript adds a number of features to core JavaScript common to other languages, such as classes and modules, in addition to strong typing, generics and interfaces. TypeScript is developed and maintained by Microsoft.

Takeaway - These languages are highly popular and would constitute a solid foundation any budding developer or product manager. If you don't already have basic proficiency in at least some of the stars - I implore you: learn these growing tools of the trade.

Cash Cows

JavaScript - JavaScript has become the go-to language for modern web development, with a number of spin-off frameworks that leverage its core elements. JavaScript is more popular than Java today, due to the ubiquity of web applications today and the move SaaS and other web-based models for application consumption. Here, JavaScript is leading the charge, and the numbers reflect that. However, it should be noted that, despite similar nomenclature, Java and JavaScript are not closely related (it's a long story). Both are object-oriented, but the similarities end there.

Java - Java has long been a popular language for cross-platform development, and this flexibility has continued as new platforms have emerged, such as mobile. One of Java's many conventions is the idea of "write once, run anywhere", meaning that code written in Java can be run on any other platform that supports Java with no recompiling. When complete, Java applications are compiled into bytecode which runs on a Java Virtual Machine. Originally built by Sun Microsystems, through acquisitions it's ended up in the hands of Oracle today.

SQL - SQL (Structured Query Language) is an old workhorse that needs no introduction. It has existed for quite some time and is the main means by which analysts query and pull data from relational databases and data warehouses. Despite the popularity of “NoSQL” and other non-relational frameworks, SQL remains king, and in recent years many of these other frameworks have bolted on SQL-like interfaces in order to ease data extraction and transformation. As companies collect data from a greater range of diverse sources and continue to store this information in central databases, SQL will only increase in importance.

The C Family - No big surprise here - the extended family of C languages has held a strong position within the software development community for some time and continues to serve as the backbone for many critical applications we know and love today. Further, C has found its way into other languages as well. For example, the reference implementation of Python, CPython, is written in C and Python, and significant chunks of the core Python codebase are actually written in C due to it being a compiled (rather than interpreted) language and thus having faster performance at runtime. C is a hugely influential language that will not be going away any time soon.

PHP - PHP lands squarely in the cash cow category. PHP is a server-side scripting language primarily suited for web development, as evidenced by its original meaning of “personal home page". Numerous popular websites and web applications are built on PHP, including, perhaps mostly famously, WordPress. However, the language has stagnated in terms of popularity, in part to due to its clunkiness and security vulnerabilities, where PHP has historically suffered from a number of severe exploits (ex: SQL injection). That said, this is another language with incredible market share that will continue to see broad use for quite some time.

Swift - The popularity of Swift derives directly from the underlying popularity of macOS and iOS devices which, though a minority of overall smartphone shipments, represents a massive install base, especially among more affluent western populations. Launched in 2014, Swift initially saw massive growth, becoming one of the fastest growing languages in history. Swift is heavily influenced by Objective-C, another cash cow, which it recently surpassed in popularity. As the brainchild of Apple, Swift will live or die by Apple's own success, so plan accordingly.

Takeaway - These languages really pay the bills. If you are already proficient in any of the above languages, great, leverage that saved time to pick up some skills in the rising stars. If you do not know these languages well today, evaluate how practical / necessary they are for the specific set of projects you want to work on now or in the near future.

Question marks

Rust - Rust is a relatively new programming language that only appeared on the scene in the last decade. While Rust is technically a general-purpose language, due to its low-level nature, it is best used for embedded systems running close to bare metal. Comparisons are often made between Rust and C++, in part driven by their syntactic similarities. Rust is often known to create enthusiastic fans among its users. Though far from being one of the more popular languages, it is truly loved by the people who use it most. Development on Rust is quite active today, ensuring the language will stay on the bleeding edge for the foreseeable future.

Scala - Like Go, Scala is language oriented towards improving developer productivity. The name Scala is a portmanteau of "scalable" and "language", which hints at original intent of the language to enable high performance of large-scale applications and userbases. Scala is built on JVM and JavaScript runtimes and combines elements of object-oriented and functional programming. Due to these strong connections, Scala is often seen as "next-gen" Java. Scala is uniquely suited for parallel and distributed computing, providing a level of future-proofing that many legacy languages lack. Though popular among a certain subset of developers, its growth appears to have prematurely slowed relative to languages like Go or Rust. Its boosters hope that Scala may one day overtake Java, but this won't happen for some time, if ever.

R - R slightly missed the cutoff for star status, but given its incredible ~40% growth rate the language will easily cross the boundary next year. R is exploding in popularity for the same reasons as Python, though most consider Python to be relative winner in terms of speed, ease of use and general applicability. R's historical strength in data science and statistical analysis is now powering a major renaissance for R. Enthusiasts celebrated R's 25th anniversary earlier this year, and with the helpful tailwind of data science, the language shows no signs of slowing down.

Haskell - Function over form, or in the case of Haskell, have both. Haskell is a purely functional programming language, meaning that the language focuses on functions that take immutable values as input and produce the exact same output every single time. It's also lazy, which simply means results are not evaluated until absolutely necessary. These and other features make Haskell a very powerful and efficient language in the right hands but also potentially limit its applicability. Haskell's cult following is growing rapidly from its small base, but it's hard to say how long this will continue.

Takeaway - They're called question marks for a reason. No one really knows how the future will play out for these emerging technologies. They are probably not worth betting the farm on today, but they are also prime candidates for becoming the next "must-know" tools among forward-looking dev teams. Keep an eye on them.

Dogs

Visual Basic (All Flavors) - VB.NET, VBA, VB6 - whichever your flavor, the Visual Basic ecosystem has clearly fallen from grace. VB.NET is one of only two languages in the growth-share matrix to actually lose share in 2018. Significant chunks of VB's functionality exist in C# now, and Microsoft's stance towards the languages has not been 100% clear, having gone from originally planning to end support for the language in 2008 to recently declaring that Windows 10 will support the VB runtime for the lifetime of the OS. This is great for legacy applications built using Visual Basic, but these will inevitably need to be rewritten in a modern language or be end-of-lifed.

Takeaway - Unlike real dogs, dogs within the growth-share matrix are bound to be controversial. Developers and development teams need to seriously grapple with the current state of affairs these languages face and whether or not it's advisable to spend significant time and resources building applications powered by these less popular languages.

(Human) Capital Allocation

The moral of the story - think critically about where to invest your time.

To be clear - it’s not the end of the world if you pick the “wrong” language. In fact, there really aren’t any wrong choices here, even the "dogs". Use the best tool for the job. However, it certainly helps to avoid the transition costs inherent in trying to reposition oneself or play catch up later on.

If anything, don’t try to reposition yourself per se, but rather, seek to enhance your overall value and breadth of capabilities by acquiring at least intermediate mastery in several different languages. Again, similar to spoken languages, people who can converse in multiple valuable languages often gain disproportionate value from their learning efforts, which tend to compound on one another, especially when learning the basic features which form the building blocks of many dialects.

Remember, this mental model, though imperfect, is arguably flexible enough to accommodate many skills, not just programming.

I hope this framework is useful to you as you decide where to grow your human capital as a technically savvy individual.

You can find the full backup to this analysis in both Jupyter notebook and .py script format at my GitLab or GitHub.