As we continue to watch the attempted xz utils backdoor hack unfold, I’ve been following several conversations where questions are being raised about what this type of hack means for the software supply chain, and for security, identity, and trust.
At Tidelift, for years our rallying cry has for years been to “pay the maintainers.” We believe this is an essential step in avoiding situations like xz where a volunteer maintainer who described themselves as an unpaid hobbyist was tasked with more work than they had the time or space to do.
Yet some of this discussion reveals conflicting opinions about the effectiveness of paying maintainers for their work. Some have said that money is not the answer, or even part of the answer. A few examples of exchanges that caught my eye:
My colleague Luis Villa pointed out in his post about xz earlier this week that paying maintainers should not be viewed as a magic bullet, but instead a cornerstone of efforts to improve the security and resilience of open source.
I’d like to outline the guaranteed, measurable benefits that Tidelift’s customers receive from using our software throughout their entire development lifecycle. Customers using Tidelift’s software to implement unified software supply chain policy and practices get the result of better software, and the maintainers developing their software are certified and paid. You can read more about what our software does, and how it works here.
Paid, contracted secure development work
Here is what is in our agreement with every single maintainer partner. These are all explicit, contractual promises that maintainers make about their projects that customers can count on, when they are paid by Tidelift.
- Multi-factor authentication is turned on.
- Security policies and procedures are in place to ensure maintainers are notified of vulnerability reports before they are made public.
- When a security vulnerability is reported, it is reviewed, a fixed release will be made (unless it’s determined to be a false positive), and a post-mortem review will be provided with context-specific details, available workarounds, and other critical data that saves engineering teams time and money.
- Maintainers review release managers every time a new release is made to ensure that those on the project with access to make releases are verified.
- There is a security maintenance plan in place, so customers can plan for the end of software’s life and know when security fixes will no longer be available. In many cases, our maintainers are willing and able to backport security fixes to older release streams—because they take the security of our customers seriously.
- Maintainers commit to giving us at least 30 days notice when they decide to stop their work on a project.
Multiplicative benefits
Maintainers receiving reliable income often are able to go above and beyond the list above to add even more security to their projects, and to the ecosystem.
These are just a few examples:
- Catching critical invisible security issues like leaked passwords, tokens, and other secrets due to attacks on the toolchains in use.
- Adding co-maintainers to projects to ensure they have backup and improve health, stability, and longevity.
- Expressing willingness to take on emerging security practices like SLSA—with commensurate income for that additional labor.
- Giving secure development templates back to the community to make secure development practices easier to adopt.
- Increasing the diversity of contributors and ideas by writing clear documentation and giving mentorship to the next generation of maintainers that we need to drive the global economy.
Why is this worth investment?
I’m going to make this really simple:
As Ashley Williams points out in their comments above, when it comes to the open source software we use to build our businesses, we’ve become “entitled AF.”
We expect volunteer open source maintainers to deliver as enterprise suppliers, but it is a contract that maintainers neither signed nor agreed to. Our collective actions are speaking louder than our words.
We’ve rated their performance against secure development standards without considering what resources it takes to meet those standards. We show up in their issues demanding fixes. We repackage those fixes and sell them as secured open source, without compensating the creators.
In place of actual contractual agreements with maintainers to pay them for work delivered, we’ve built up an entire industry around scanning and remediating vulnerabilities in their code.
The chart here shows the number of CVEs published in the National Vulnerability Database each week since 2006. Since 2017, the number of reported vulnerabilities trend is clear.
If the name of the game was fixing vulnerabilities, perhaps you could see this as a good thing. There are certainly people benefitting from more CVEs—but it isn’t open source maintainers.
Let me be clear: the current model of running scans to look for known CVEs in open source code is not working. The idea that we should just continue patching things until we burn out the people that are creating the patches in the first place, and then just move on to another replacement package is an embarrassing, and inefficient, strategy in 2024.
Don’t we ultimately want more secure, more reliable software? When software is more secure and reliable, it has less risk.
If you are building software on top of open source that delivers revenue-generating value to customers without investing time, energy, or money to ensure that your end-to-end code is safe and reliable — you are taking unnecessary chances with your customers’ data and adding risk to your business.
If your take is that any open source maintainer seeking reliable income should re-license or abandon their project, I would encourage you to review the data Github provides on maintainers seeking sponsorship, and hold that list up against any enterprise application dependency graph. All maintainers seeking income for their work essentially walking off the job is not a realistic solution.
Investing in open source doesn’t need to be complicated. If you need to unify your approach to software supply chain security throughout the development lifecycle, CI/CD, and internal organizational policies — Tidelift’s software does this. If you are running an operation that could benefit from data to comply with attestation requirements, Tidelift’s software does this— and we’ll pay for that data that you won’t acquire otherwise. All of our customers receive maintainer impact reports on what their money is achieving, and they are seeing results in their application risk posture.
A recent Harvard Business School paper estimated the demand-side value of the open source software ecosystem at $8.8 trillion. By comparison, the entire U.S. electrical grid is valued at 1.5- 2 trillion dollars, and the U.S. interstate highway system is valued at 750 billion dollars.
Open source software is an exceptionally valuable resource, and we shouldn’t take it—or its creators—for granted.
Technology problems and data risks are accelerating around us, and we will not meet these challenges without open source maintainers. They are providing the bedrock of maintenance needed to continue our progress as a civilization.
Watch a demo of how our partnered maintainers opt into your software supply chain, and how our superior resulting data drives better open source software supply chain security, and come join us for a conversation about new ideas for how to improve open source software security and resilience at Upstream on June 5.
Top comments (1)
Thank you for your insights. However, I disagree with the notion that paying maintainers will inherently improve the security and resilience of open source software (OSS). While companies may offer paid support for the OSS they have released, this model does not necessarily apply to all OSS projects.
It's important to note that the majority of OSS projects do not originate with monetary gains in mind. A significant portion of OSS is utilized by both individuals and enterprises, but very few contribute back to the projects. The solution, in my opinion, lies in fostering a sense of moral obligation among users to contribute to the OSS they find useful.
The very tenet of Open Source software dictates that all users of an OSS project should share the responsibility of supporting, improving, and maintaining it, rather than leaving the burden solely on the maintainers. Instead of relying on paid maintainers, the emphasis should be on encouraging active participation and contributions from the broader user community, aligning with the collaborative spirit of open source development.