At FOSDEM this year I gave a talk about contributing.today, a live streamed fireside chat on anything open source (licensing, funding for open source, the different language communities, …) that for many (many) months, in semi-regular intervals, I have been one of the organizers of.
Beyond the projects maintained by the proverbial single individual in Nebraska in their free time narrative, there are other reasons for projects "going bad". I'd like to share some stories, straight from the horse's mouth, share some of the give-aways of unhealthy projects, and maybe together support a more sustainable ecosystem.
Recognizing undope projects
A project is only as healthy as its community. Community is at the core of successful projects. Without community, a project is only technically open source because of its license and won’t reap the benefits of rapid innovation, consistent release cycles, and faster response to current trends and market needs.
Rain Leander, while they were on the show (I think).
My introduction to open source was a very positive one. I had decided to learn Ruby on Rails and had started following the Rails Girls guides to that end. I’d get stuck sometimes because my specific setup required an extra something-something that wasn’t explained in the guide. After a bit of googling I was able to fix the issue and continue my learning. And I could edit the guide to include notes on what to do when others would encounter the same issue. Others that I didn’t know, from all over the world. I could make sure they wouldn’t get stuck like I did - I was sold.
While that was a very positive experience, not all my endeavors in open source were followed by that same dopamine-triggering response. In fact, many were absolutely dreadful.
But I learned and I’d triage a new project, and a new employer alike, by looking at the Signs:
- Does the project have a Code of Conduct at the root of their repository? Can I reach out to a person(s), or does it only list a generic email address?
- Does their contributing file inform me of what contributions they’re looking for, does it set me and my contribution up for success? Is there any way to find support - like in a Discord server - or will I only be able to get feedback at the PR-stage?
- What’s the number of contributors and what’s the makeup of that group?
- A helpful README is a good sign
- What’s the amount of Issues and PRs that are open, or: “responsiveness”
- ... and what’s the general tone?
As I got to know more people in open source I would find out whether projects are friendly from hallway conversations. Making your project inclusive and safe can take these forms:
- Public recognition for contribution. Extra highlight for first time contribution. Possibly via an Author.md or Contributors.md list on at the root level of a repo.
- A mentorship program, either formal or informal, to help people complete their first contribution.
- The promotion of trusted contributors to maintainers with commit access.
In at least two other talks at FOSDEM “succession planning” was a topic of discussion (Abigail Cabunoc Mayes' "Sustaining Free and Open Source Software", and Melissa Mendonça's "Contributor Experience - Supporting social infrastructure for FOSS projects"
).
What does the data tell us (about healthy communities)?
The 2017 Open Source Survey that sadly never got repeated or maybe was thought to be replaced by the State of the Octoverse, collected responses from 5,500 randomly sampled respondents sourced from over 3,800 open source repositories on github.com, and over 500 responses from a non-random sample of communities that work on other platforms.
The main take-aways:
- Documentation is highly valued, frequently overlooked, and a means for establishing inclusive and accessible communities.
- Negative interactions are infrequent but highly visible, with consequences for project activity.
- Open source is used by the whole world, but its contributors don't yet reflect its broad audience.
- Using and contributing to open source often happens on the job.
- Open source is the default when choosing software.
I want to focus on the bit about interactions, and accessibility, not negating the importance of the other points by any means.
18% of respondents indicate that they have personally experienced a negative interaction with another user in open source, 50% have witnessed one between other people.
Negative experiences have real consequences for project health. 21% of people who experienced or witnessed a negative behavior said they stopped contributing to a project because of it, and 8% started working in private channels more often.
By far, the most frequently encountered bad behavior is rudeness (45% witnessed, 16% experienced), followed by name calling (20% witnessed, 5% experienced) and stereotyping (11% witnessed, 3% experienced). More serious incidents, such as sexual harassment, stalking, or doxxing are each encountered by less than 5% of respondents and experienced by less than 2% (but grouped together witnessed by 14%, and experienced by 3%).
In case you go “surely a lot has improved since 2017!!?”, a report by the Linux Foundation from 2021 that looks at diversity, equity and inclusion in open source, doesn’t paint a more favorable picture.
What do the tools (help) track?
Let’s look at what some of the initiatives in this space define as community health.
In Developer Relations and community management I worked with tools like Orbit and Common Room, I also looked at Savannah CMS, which is an open source solution. I even contributed to Orbit, but all aforementioned tools are better at measuring sentiment, and identifying influencers, than at evaluating community health.
CHAOSS (Community Health Analytics in Open Source Software) is a Linux Foundation project focused on creating metrics, metrics models, and software to better understand open source community health on a global scale. CHAOSS understands Open Source Community Health as “The potential that an open-source software community continues developing quality software”.
Bitergia is another company in the community health space. According to their website, they “Improve decision making and reporting by analyzing software development community, activity, and performance of open source projects” - and they do so across patches, meetings, forums, etc.
A Bitergia report from August 2022 looks at the Kubernetes open source ecosystem. I wasn’t so much interested in how well the Kubernetes community is doing, health-wise, although SIDESTEP I do love a quote from Sarah Novotny from the time when she led the Kubernetes Community Program at Google about how the “chop wood and carry water” isn’t just a thing they say in the Kubernetes community, it’s a core tenet of what makes the community work.
In Mary Thengvall’s book The Business Value of Developer Relations (the DevRel bible), she (Sarah) said that:
“In order to avoid some of the bad behavior we’ve seen in other communities, we wanted to make sure that when companies were coming into the project, they came in humbly and with the intent to help the project, as well as helping themselves. One of our core values is ‘community over company’, which means people are here to contribute to a good project first and foremost, and then have their company benefit from the work, rather than the other way around.”
And: “We’ve spend a lot of time and effort rewarding people for the things that are unglamorous, thanking individuals for a variety of contributions instead of only when they include code (...)”
When someone does drop a 10k line PR they should calmly be made aware of Google’s Developer Relations Manager Aja Hammerly’s ‘We don’t do that here’.
But back to the report!! Bitergia looks at:
- Responsiveness as the time to solve issues and pull requests or merge requests, as well as average commits per day. A lower average response time indicates a higher performance.
- The Complexity as determined by the number of code repositories and people involved. An increasing number of repositories indicates an increasing complexity.
- Activity Diversity as the total number of commits, issues, patches requests, and lines of code added/removed over a period of time. When and where the activity occurs indicates how it is distributed.
- They look at how talent is managed within an Open Source Ecosystem.
- By analyzing different aspects of code contributor growth, they can identify the contributor retention rate (contributors that remain engaged) and bounce rate (contributors leaving).
- The bus factor is the minimum number of team members that have to suddenly disappear from a project before a project stalls due to lack of knowledgeable or competent contributors. A low bus factor can be a sustainability risk.
- And they look at Community Footprint - the presence and influence of organizations within the open source ecosystem.
- Gamma Factor - Amount of code contributors using Gmail accounts for their commits, and the ratio of commits done by them. The bigger the number, the higher probability the ecosystem is a non-company driven project.
- Elephant Factor - Minimum number of email domains whose employees perform 50% of the total contributions.
- Contribution patterns - Activity during regular work hours may come from contributors who are paid by their job to participate. Activity after hours or on weekends may come from unpaid volunteer contributors. They note that distribution of paid and unpaid contributions may work as indicative of developer burnout.
Expectation setting
The ‘we don’t do that here’ from my Kubernetes community side story is a way to model behavior. If problematic behavior like described in the Open Source Survey takes place, it’s paramount to address that swiftly, politely, and publicly, to send a signal to potential contributors that such behavior isn’t tolerated.
Setting expectations. And automate stuff that can be automated - that includes pull requests and issue templates, when you find yourself asking contributors to elaborate on their contributions more than once.
A look at licenses
Licenses are a way of setting expectations. In one contributing.today episode we talked about licenses with folks from the OSI, from the Ethical Source Movement, Tidelift, and ClearlyDefined.
Is a project open source, source available, open governance? I learned a few things.
- Like that combining licenses for 1 project (so not a different license for a project’s core and IDK its documentation) is possible but is mostly great for lawyers
- You will want to choose a license that is standard in your ecosystem to optimize for adoption
- And if you’re thinking of building a business around a project ultimately, you’ll need to choose wisely
… Unless you’ve been using CLAs (Contributor License Agreements), relicensing is a huge pain for even just a mildly popular project. Contributors are hard to contact, may have moved on from the project or the platform, passed away… you’ll risk having to rewrite parts of the project, and you’ll need to do a major release, since you'll introduce breaking changes to your API. That can cause loss of community, especially when the release before the change gets forked and further developed under a community-friendlier license.
I wouldn’t have the license you adopt depend on whether or not you secure funding either - and this was in the Tech press recently - people will think twice about spending their time on a project that might change just how open they are.
Friendliness factor
Originally an academic project, Postgres’ open source stewards were interested in its extensibility, the ability to merge it into any project and with any codebase. The lesson Postgres can teach us is: invest in connectors. Java is also “insanely” backwards compatible. This is different from developing swiss army knife-like programs. A narrower scope is better, as long as it “plays well with others”. Interoperability.
Some organizations that are involved in open source have great policies internally - or an OSPO, which typically means that:
employees can make contributions to the upstream project during working hours, ensuring activity on the project
They’re professionally involved, which usually means a professional attitude as well.
Hopefully that doesn’t mean they come with whats-good-for-the-business-goggles on
Try to stay far away from the situation where you have a single vendor support . Most all projects accept help and contributions from service providers - some of them are at least on paper champion supporters of open source - but their feature requests can’t outweigh what the community is looking for in the project.
Side-tangent, and I promise there aren't many of these: recognize red flags when companies want to hire you for your open source work. Businesses operate on a different timeline, make super sure you’ll not have to subject yourself to the OKRs they set for the rest of Engineering.
Contributor experience
Sticking with Postgres, my colleague at Aiven Gregory Stark, recalls his first contribution to PostgreSQL well, and how it was a great experience - which can make all the difference in terms of gaining long term contributors to a project.
Remember that 2017 Open Source Survey and how it said that “Documentation is highly valued, frequently overlooked, and a means for establishing inclusive and accessible communities”?
From the contributing.today meetup panels on Docs often the conclusions were:
- Using a broad format like markdown allows for more people to be involved in contributing to / reviewing the docs.
- At minimum the setup or install needs to be documented. After that people can tinker around, but they need to know how to even get started, and if they can consider your project to get their job done.
- Be clear and concise, avoid jargon (or: make sure the definition can be found in a glossary).
I love this quote about non-code contributions:
Triaging and labeling is a contribution too. Projects that think about the different reasons for people to contribute to open source, can address those different motivations with appropriate work.
Marit van Dijk, Developer Advocate at Jetbrains & open source contributor
Long term sustainability
A 2021 Tidelift survey of 400 open source maintainers found that 46% of maintainers are not paid at all, and only 26% earn more than a 1,000 dollar per year for maintenance work. Over half (59% to be precise) have quit or considered quitting maintaining a project, and almost half of respondents listed lack of financial compensation as their top reason for disliking being a maintainer.
We’ve had several contributing.today sessions around funding open source work.
Companies making money from people's work without paying them is a huge liability.
Tobie Langel (UnlockOpen) in 2022.
… and he continued that "People don't understand what it takes to run a business (or: an open source project for that matter). Nobody scrutinizes companies providing healthcare, or bathrooms for their workers. Why would we be harder on open source maintainers?"
"The smaller projects may need your help more than Kubernetes, even if you heavily rely on Kubernetes." Phoebe Quincy is a Senior Community Relations Manager at Digital Ocean, adds that Hacktoberfest in recent years highlights projects that don't get a lot of attention (money) to support using Open Collective or GitHub Sponsors.
"Most sponsoring programs are small pockets of money, just enough to cover server costs".
Estelle Weyl, Sr. Technical Writer / Community Engineer at Open Web Docs:
Phoebe: "FOSS Funds award lump sum, large amounts of money. The next challenge is doing that consistently." Estelle adds that not having to spend the amount within the year or risk losing it, would help with budgeting.
In 2021 we had a very similar conversation and Babel maintainer Henry Zhu told us people got upset that he spends time on podcasts and marketing to get more donations, instead of writing code to sustain the project. And it’s wild how the industry is paying ridiculous amounts of money on software engineers' salaries, but we think awarding 10.000 Dollar on a project is this amazing thing.
Per Ploug, Open source Program Office Lead at Spotify, suggested that the ASF (Apache Software Foundation) and the Eclipse Foundation support many of the "plumbing" projects. But the donating a project to a foundation route isn't for everyone. There needs to be the option for people to create a business for managed services or support around their project, as well as other avenues.
While a “normal” company would like to have runway, when you're running an open source project that might actually send the wrong signal: you don't need the money, you’re still maintaining it without, aren’t you. This ties into the whole toxic notion that open source should not be paid ever, that only when you suffer in your mom’s basement, deprived of sunlight, will you be a true, unspoiled hacker.
When you would like to pay your mortgage, here’s a hot tip for links to organizations that can fund consistently: oss.fund
The FOSS Contributor Survey from 2020 by the Linux Foundation summarizes the results of a survey of free/open source software (FOSS) developers with the goal to identify key issues in improving the security and sustainability of FOSS. The survey was a collaboration between the Linux Foundation’s Core Infrastructure Initiative and the Laboratory for Innovation Science at Harvard. This work has been recently incorporated into the OpenSSF working group on securing critical projects.
To capture a cross-section of the FOSS community, the research team distributed the survey to contributors to the most widely used open source projects and also invited the wider FOSS contributor community through an open invitation. A total of 1,196 respondents filled out the demographic section and at least one question about current FOSS contributions, of whom 603 went through the entire survey.
The vast majority of respondents, nearly 75%, are employed full-time. The survey found that over half of all respondents are paid to contribute to FOSS.
The report suggests:
- To allay concerns over corporate involvement in FOSS projects through greater transparency and clear commitments to support FOSS in general and specific FOSS projects for several years.
- Incentivize paid contributors to dedicate time to mentoring new volunteer contributors.
- Transfer FOSS projects to a foundation with neutral governance to ensure diversity of organizations and control - but the Linux Foundation would say that, wouldn’t they? ## Safety first
The previously mentioned report found that all types of contributors reported they spend very little of their time responding to security issues (an average of 2.27% of their total contribution time) and reported that they do not desire to increase this significantly. Bug/security fixes, free security audits, and simplified ways to add security-related tools to their CI were mentioned as areas where respondents would love to see more contributions - from others.
I’d be remiss to not talk about the solidity of our supply chains. Your transitive dependencies are part of your community garden too. It’s imperative that you at least observe the health of your components for they are quite literally your project’s foundation.
In conclusion
I wouldn’t dare to suggest that open source is sick, but we can all do better in taking our vitamines. Collaboration between strangers is one of open source's most remarkable aspects, and we can all act as stewards for our communities. Set expectations, upgrade your onboarding, improve your contributor experience, check your dependencies, floss, and prepare to repeat all that for the long haul.
Photo by Brett Jordan on Unsplash.
Top comments (1)
Great post! Thank you 🎉