The Need For Open Source Infrastructure To Be Open

Victor Bjelkholm — Thu, 23 May 2019 10:59:23 +0000

Software development today is infused with open source on mostly every level.

From your operating system, to the icons you use, open source is better than ever, keeps growing and enables more and more developers to get involved with creating something for the whole ecosystem and sometimes the whole world.

Open Source today is also a marketing tool for many companies. It's used to attract developers who normally wouldn't pay attention to your service, but since it's open source, can now not just see the source, but also help contribute new features and fixes.

That's great and all, but what does it mean when open source developers rely on closed source platforms for doing their development?

GitHub is a good example of this. GitHub is seen as the bastion of open source, where developers can collaborate and since today, even get paid to work on open source.

But long-term, I'm not sure if using GitHub as a bastion of open source, is a good idea or not.

We all want the same thing. Tools and services we can use to make our lives as developers better. But as everyone on this planet, we sometimes rush to get a short-term benefits over a long-term ones, even though working on long-term solutions is better.

The alignment between the open source ecosystem and companies who develop tools for open source, are not always as well aligned as we should demand them to be. We mostly solved this already with tools of the trade, but services are harder to make open and seems to have been left behind in favour of letting for-profit companies handle it. A recent example is the Apache Foundation dropping their own Git hosting to use GitHub instead.

But a for-profit company doesn't always have the users interest at heart. If they can improve profits while loosing just a small percent of the user-base, it might make sense for them to do it. And I don't blame them, it's a for-profit company, they have to act that way.

The services we sometimes rely on, hosting for git, our shared packages and alike, are all core infrastructure projects that when they don't operate in our favour or work how we want them to, they make our work harder to perform.

And sometimes these services for open source start out great but as their VC funding starts to run out (or the parent company need more profits to survive), they start to turn their back on their users, as the incentives are not aligned in the users favour.

With the recent additions of the GitHub Package Registry and "Sponsors", I ask all developers who rely on Open Source to tread carefully through these murky waters.

The issues of open source funding and infrastructure, are extremely important. Way too important for for-profit companies to own it. And the more power we give the companies, the less power we have to actually create open source infrastructure that people can use.

The everlong quest for the perfect package registry

Victor Bjelkholm — Sat, 11 May 2019 15:33:08 +0000

Today there is an interesting landscape when it comes to package registries. From basic, barebones registries like Homebrew that pulls down build instructions locally to more advanced, general-purpose registries like what GitHub just launched.

We as a community have mostly been focused around the functionality that the registries provide and less about how the registries are funded and structured in terms of community. I think we're missing a big and important point here.

You can take all the package registries currently running and divide them into two buckets. One bucket for the ones run by for-profit companies (such as GitHub or npm Inc) and another bucket for registries run by a community of people wanting nothing else than providing an excellent service to the community. Registries like Homebrew, PyPI and Pacman fall into this bucket.

What really is the difference? We got used to hypergrowth and for-profit startups developing excellent services with great user experience. We tend to forget that they either disappear trying to find a market fit or that they find a market fit but the product is not the same anymore after finding it.

I think we as a developer community deserve a service that we can rely on, knowing it won't be bought, it won't compromise the service for chasing a profit, or disappear because it couldn't be profitable in the first place.

Maybe, similar to how public utilities are run in most places, the benefit provided to users shouldn't depend on the utility surviving. Some things we need to have smoothly running no matter what, and I think package management falls into that category.

So what would the perfect package registry look like?

Always put user benefit in front of profits

The main issue with a for-profit company running a public utility is that they constantly need to balance earning a profit against providing value for the users.

If they focus only on providing user value but not earning a profit, the company has a few options: They would have to squeeze out more profit-per-user, subsidize the running of the registry with other add-on services, or run other unrelated services to cover the cost. Basically, it needs to spread itself thin and lose core focus on the package management side.

If they focus solely on earning as much profit as possible, the users will be left as a second thought and the service won't be as good as it can and should be.

Have the community own the project

For-profit companies usually don't provide a lot of insight into their decision making process. In the cases they do, you as a user often can't influence it directly.

Instead, a package registry should be owned by its users and contributors.

How would this work in practice? First, the project should be funded by its users to remain independent and to setup a guard against the registry implementing not-so-good features for the userbase. The idea is for people to fund the project if they like it and believe in the direction. If the direction is going somewhere they didn't expect and don't like, they can pull the funding and the project either self-corrects or dies trying.

Permanence and exit-strategy

Again, in the constant balance between profit and user benefit; a for-profit company needs to be easy to start using, but hard to leave and lock the user in. That way, they can make sure you continue using it.

A public utility would not have any problems offering all the data and migration tools for leaving the utility, as it doesn't impact the bottom-line. Instead, a public utility MUST ensure that the users can easily leave whenever they want.

No need to eat the world

In order to avoid being left behind, companies like GitHub are ever-expanding to be able to capture the entire market of some ecosystem. They need to "eat the world", otherwise it's not worth pursuing.

A public utility doesn't have to conquer anything. It simply has to provide user value and be made sustainable by itself.

Lots of words, lots of complaints. Stop bitching and provide a solution? Yes, ok.

Introducing Open-Registry

Open-Registry is a Open Service (TBD) which implements what I think are the core features of an Open Source Public Utility. This means:

Compatible with the registries being replaced
Only focuses on being a Package Registry
Full Transparency
Funded/Governed/Developed by the community

With these goals in mind, Open-Registry always considers the user first when implementing changes. You have our guarantee that it will never be sold to another entity, we will never focus on selling any data (in fact, we don't even collect enough information for that to work) and everything we do is done in the open.

And the best part is, you can participate in many different ways. From donating funds each month to becoming a leader in the project and deciding the direction, as long as you have other members support behind you.

Best way to get started is to checkout https://open-registry.dev/ which goes into more details about what Open-Registry is and how it's managed.

Or if you're feeling brave, scan through the repository and see where you can help: https://github.com/open-services/open-registry

Lastly, you can always reach out to me on Twitter (@VictorBjelkholm) or shoot me an email to victor@open-registry.dev if you have any questions or just want to leave some feedback for us to improve and learn from.

Open-Registry is an experiment

Words of caution: Open-Registry is an early experiment. It's an experiment to see if we can get the open source community to self-fund crucial open source infrastructure projects without compromising on the user benefits.

Either this works out, and we can manage to make it survive with just user funding, or the costs will grow too much and the project won't survive.

Thank you for taking the time to read and think about our open source infrastructure future.

(Thanks to Andrés Berríos Jiménez, Francisco Baio Dias and Alfonso Garcia for proofreading)

DEV Community: Victor Bjelkholm