Capacity of the pipeline
The US federal government has been gathering and publishing statistics about education annually since… forever? There’s a useful collection of data out at https://nces.ed.gov/programs/digest/current_tables.asp. This includes diversity data about the pipeline that’s producing trained software engineers for our industry.
Check this out: We’re currently awarding an all-time high number of CS degrees. Awesome!
Over time, we’ve gone through a couple of peaks and valleys. There was a decline in the early 2000s – probably related to the dot-com bust. Not surprising. But what’s that peak in 1986? Answering that question took me down an unexpected rabbit-hole, and the answer is fairly fascinating!
A pretty credible-sounding explanation is offered by Dr. Eric Roberts of Stanford, here: http://cs.stanford.edu/people/eroberts/CSCapacity.pdf The declines in 1986 and 2004 both trail events in 1984 and 2002, due to the lag-time as students filter through the pipeline. The 2002 “event” was indeed the dot-com bust, and even though the industry itself recovered fairly quickly, student interest in CS cooled as the prevailing (but incorrect) perception came to be that jobs were hard to get and being shipped overseas. (I observed this myself first-hand, when I visited local high schools in Atlanta around 2006 and spoke to the students about software careers. They all thought there were no jobs, when the reality was quite the opposite!)
In 1984, the events were entirely different. In the time leading up to 1984, the new wave of personal computing, and especially the advent of the Mac, drove excitement in this new field. The field itself was growing quickly, and demand for software engineers outstripped the supply. Meanwhile, fairly understandably, academic departments moved slower than industry. Professors and teaching capacity were not added fast enough to keep up with the growing pool of interested students. Added to that, demand for industry professionals was so strong, companies hired away many of the professors who were teaching those students. The academic capacity to teach students collapsed, and departments had no choice but to institute stricter limits on the CS student population. Competition went up as capacity to produce CS graduates went down.
Dr. Roberts also issues another dire warning: the current growth curve is likewise un-sustainable. Once again, our academic departments are not growing as fast as student demand. Something is going to have to give: either we need to grow CS departments faster (increase capacity), or go through another industry decline (decrease demand), or eventually physics dictates that we’ll hit another collapse in production of CS students.
Diversity in the pipeline
Of course, I am interested in how diversity fared over this time. The NCES data had a breakdown by sex but not race/ethnicity. So I graphed the same data set, split by gender, and it blew my mind!
Those peaks and valleys did bad things for our female population. In the beginning, the percentage of women in CS steadily increased for 13 years, until 1984, where it hit its all-time high at an amazing 37%! I never knew we had that level of female participation in the past! When the first CS drop happened at the capacity collapse, the percentage of women declined from 37% to a level around 27% for another decade. Then when the second CS drop happened after the dot-com bust, the percentage declined again to a steady-state around 18% for another decade. (And this is where I start getting especially angry, because I entered the work-force in 1999, and can’t help but feel like some of it happened on my watch!) Even if you ignore the percentages and look just at the pure number of degrees being awarded to women, despite the record number of CS degrees being awarded today, we haven’t yet reached the numbers we awarded to women in 1986 and 2003. We’re still off by about 30% from our peak female degree production.
Curiously, the first percentage-drop started in 1984, when CS-education capacity started collapsing, not 1986, when its effect became apparent on total awarded degrees. The number of women getting degrees was still going up for a couple of years, but the percentage dropped during that time. Women were a leading indicator of the collapse. Those women who never got their degrees in 1985/1986 might have been casualties to the increased competition for CS program space. But that is pure conjecture. There was no female lead on the drop during the dot-com bust.
I would love to understand why the percentages of women dropped at both events. As I mention above, the decline that started in 1984 may have been due to increased competition in CS degree-programs. Maybe it was a side-effect of implicit bias in an imperfect meritocracy; maybe women either self-evaluated themselves as unable to compete or decided they had no appetite for the competition, and opted out. Likely a mix of all of these. The decline starting in 2003 did not involve real competition; in fact degree programs were dropping in size so likely easing in competitiveness. However there was a public perception that software jobs were scarce and being sent overseas. So there was still a strong perceived competition in the software industry. That does not sound entirely satisfactory, either; I could believe competition driving women out during the contraction phases of CS programs, but it should not have kept them out during the growth phases.
One thing I do know, though. If we do go through another contraction in CS degree production, we can’t afford to let the percentage of women slide again with it!
I’ve been trying to decide how worried to be about this. On one hand, if I’m right that increases in competition drive women away, then there’s a risk that we’ll slide further. On the other hand, we could have bottomed-out to the bare minimum number of women who don’t shy from competition (which means these are a pretty premium group of employees – scratch that, CEOs – y’all!). The industry is also different than it was in 1984 and even 2002. In 1984, we were not only ignorant of imbalances, but open and flagrant unfairnesses were everywhere and widely accepted. In 2002, the flagrant unfairnesses were gone, but there was an awful lot of ignorance and indifference still. I truly believe that now in 2018 people are a lot more aware and care more. During another academic contraction, there’d be a lot more attention to preventing another backslide in gender equality. So overall, I’m cautiously optimistic that at least we won’t backslide.
Clawing our way back to 37% and beyond… Well, we have a lot of work to do.
Top comments (0)