Internal Open Source

OpenSource software has been out there for quite a long time, the idea of people around the world volunteering part of their time to work on free software is very exciting, but also maintaining those projects is very hard. Keeping everyone on the same page so that there's no duplicated work, agreeing on the road-map and what's going or needs to be done, prioritizing bugs, reviewing code, merging and then, everyone updating their local repos or forks. OpenSource software development is a huge human distributed system which, most of the time, uses async communication due to the inability to get every single developer working on the software in a room or, at least, a virtual call for obvious reasons: timezone, everyone works on different schedules so everyone is free at different times and most of the people also work full time for software companies, most of the time on other projects.

OpenSource projects and free software development has tons of limitations in the way they can coordinate and communicate with the entire team, and yet OpenSource projects has higher standards and a higher bar than most close-source projects or corporate-internal software built in-house. This is why, whenever I need to do something at work, whenever I need to research for best practices in how to manage a project, how to manage or coordinate a team, how to do anything with tech, I refer to how the team behind my favorite OpenSource project
at the moment is doing things and borrow as many ideas as possible and see how to adapt them to what I'm working on or the team I'm working with. OpenSource has very complex people problems, team members are scattered all around the globe, if their strategies, policies and practices help them solve their problems, they should also work at a smaller scale in private companies building software.

The cathedral and the bazaar

The Cathedral and the Bazaar is an essay written by Eric Raymond, in this essay he examines two different approaches to Software Development by two different OpenSource projects:

• The Cathedral: where the source code is available for each release, this is only the stable code, all the code in between releases and all the work done on top of it is private to the contributors only. In the essay the author cites GCC and GNU Emacs as examples for this approach.

• The Bazaar: where all the code and the development process is public to all of the internet, in the essay, Raymond cites Linux Torvalds as the inventor of this approach and the Linux Kernel Development as an example of a project built under this method as well as the Fetchmail project.

I won't go deep into the essay as I think it is a must read for every Software Engineer, so, I'll leave some links so that everyone can have a look at it:

• TL;DR; you can check the main points on the essay in its Wikipedia page
• You can also find the full version at the official website along with other interesting writings by Eric Raymond

Key points

The key highlights from the article, also mentioned in the Wikipedia page, are summarized in the following 19 lessons:

1. Every good work of software starts by scratching a developer's personal itch.
2. Good programmers know what to write. Great ones know what to rewrite (and reuse).
3. Plan to throw one [version] away; you will, anyhow. (Copied from Frederick Brooks' The Mythical Man-Month)
4. If you have the right attitude, interesting problems will find you.
5. When you lose interest in a program, your last duty to it is to hand it off to a competent successor.
6. Treating your users as co-developers is your least-hassle route to rapid code improvement and effective debugging.
7. Release early. Release often. And listen to your customers.
8. Given a large enough beta-tester and co-developer base, almost every problem will be characterized quickly and the fix obvious to someone.
9. Smart data structures and dumb code works a lot better than the other way around.
10. If you treat your beta-testers as if they're your most valuable resource, they will respond by becoming your most valuable resource.
11. The next best thing to having good ideas is recognizing good ideas from your users. Sometimes the latter is better.
12. Often, the most striking and innovative solutions come from realizing that your concept of the problem was wrong.
13. Perfection (in design) is achieved not when there is nothing more to add, but rather when there is nothing more to take away. (Attributed to Antoine de Saint-Exupéry)
14. Any tool should be useful in the expected way, but a truly great tool lends itself to uses you never expected.
15. When writing gateway software of any kind, take pains to disturb the data stream as little as possible—and never throw away information unless the recipient forces you to!
16. When your language is nowhere near Turing-complete, syntactic sugar can be your friend.
17. A security system is only as secure as its secret. Beware of pseudo-secrets.
18. To solve an interesting problem, start by finding a problem that is interesting to you.
19. Provided the development coordinator has a communications medium at least as good as the Internet, and knows how to lead without coercion, many heads are inevitably better than one.

My own views

While Eric Raymond's essay covers different approaches to open source development which can be also used in the industry, inside software companies,
I see something very common within open source projects, they all produce high quality documentation, not only because they want people to use it
but also because they want people to be able to collaborate and contribute without taking too much time from active developers who are volunteering
to work on the project. This approach to on-boarding new contributors is very powerful and can benefit a lot any team working on software projects
in any company.

Having high quality documentation can help mainly in two things:

On-boarding new team members

How many times have you repeated yourself over and over and over again when a new engineer joins your team, walking them through
the architecture, the code, setup the local development environment and then helping them ramp up on the tools used in the team.

This walk-through can be easily delegated to documentation pages maintained and updates by the team, just the same way OpenSource projects
do it, they have a section on their docs specialized in helping new contributors getting started, it also explains how to update the docs, so
that if any new contributor finds a mistake, something that's not up to date or something that needs to be documented, they can just edit
the wiki and fix it or add whatever is needed. This way the next person to be on-boarded won't face those issues.

Empower other teams to contribute

It's a common case to have internal services which are only used by other services maintained by other teams, these are called platform services,
they commonly abstract underlying resources needed by public-facing services, also, it's common to have public services relying on each other or
exposing endpoints so that other services request or update transactional data, which means, other teams might require features to be
built in other services by other teams for internal consumption.

In a normal scenario, both teams will have to coordinate and align on deliverable, prioritize and iterate over and over again, often, priorities
change and everything needs to be reevaluated. If you follow any inner source pattern, whether you give access to the stable code or all of it,
plus high quality documentation for new contributors, the other team can easily contribute to the service in question and build the feature they need
by themselves. Of course, you need to also make it easier for them by providing clear guidelines and policies on how to contribute, the code
conventions used in your team's project and making sure the team knows them and enforce them, after all they'll be the ones maintaining that code
after it gets merged.

I've implemented this in some of my teams and the relieve of flipping the responsibilities to support internal use cases to the team requesting the
features is amazing. We've only had to give them links to our docs, help them update it wherever it was not and have them add whatever they thought
could be useful, then, during development it only requires code review, 80% of the effort relies on the team building the feature they need on the
service your team maintains while you only support them with code review and, maybe, QA, but if the feature is with them their QA team should be thew
one having more clarity on the use case and, hence, the test cases needed to be covered.

It's very convenient, it unblocks other teams which need custom features for internal consumption to be built as they don't need to wait for your team
to be free to do it, risking their deadlines on priority changes on your end while also lets your team focus on whatever priorities are there for
them. It only requires some time to put the documentation in place and review the code being pull requested by the contributing team.

Does this sound like something you might want to try?