Tribal Knowledge
One of the most important realizations I have made in my early years of working as a Software Engineer is how big of a role that "tribal knowledge" plays within an organization. I use the term "tribal knowledge" to mean important information that is passed from person to person through impromptu meetings or instant messaging. This information can range from complex implementation details of a microservice to even just invoking that microservice's endpoints. It took me a solid 3 months on the job to even learn how to hit the happy path of the ecosystem I worked on. How did I learn? Someone shared me a private Postman Collection they had stored away on their computer, hidden from any new engineer that joins.
As time went on, I interacted with more and more engineers who were way more experienced than me. I learned who explained concepts in the most easy to understand way, and made the extra effort to absorb as much information from them as possible. I also learned who didn't explain concepts super well, and never really went back to them with more questions when I realized that I never got the answer I needed. And that's the issue with tribal knowledge - you only can learn as much as those around you are able to teach you. And this can be extremely limiting.
Once I reached a more tenured status within the organization, I started interacting with more individuals across the team boundaries. After various interactions with a particular team, I began to realize that the junior engineers barely knew their own systems, and the senior engineers were terrible at explaining the systems. This isn't to say that these senior engineers had no clue about their own systems, they very clearly did. The issue is that they could not effectively communicate their knowledge to anyone else. Therefore, all of the information they have accumulated, and all of the information passed to them from previous engineers will never be effectively passed on to the next "generation" of engineers.
The downfall of an organization
The problem with this word of mouth approach of information sharing is that if at some point the chain of communication breaks, all of the information passed down through the "generations" of engineers before will ultimately be lost. It could be an incredibly smart person who cannot explain concepts to a newcomer, or someone who has a hard time understanding the system no matter who explains it to them. In either case, the moment information stops flowing is the moment a team or organization can get into trouble.
The point where this becomes a major problem is when the number of these communication breaks reaches a critical mass. Suddenly, it feels like the general population of engineers simply does not know as much as it used to. Basic employee turnover has experienced engineers moving on and new engineers coming in. But because there are more broken links in the delicate chain of tribal knowledge, there are less people who can effectively train and teach the new engineers. Repeat this a few times, and the effect compounds. Suddenly you have a organization flushed with plenty of new faces who never got the proper training they needed to thrive. This is how an organization dies.
This is not something to be pinned on individuals, but rather on the greater system. You could be the smartest engineer alive, but if you are new to the team and have no one who can teach you all the organization specific secrets, how will you ever learn the organization's ecosystem? This system of passing tribal knowledge from person to person is easy to execute and flexible. When you have the right pairings of people, it works really well. But ultimately it is fragile, and is able to destroy an organization frighteningly fast. Rather than completely replacing the system, supplementing it with clear concise documentation that permanently records important information about an organization can help keep it from collapsing.
Invest in Documentation
The main reason an organization falls victim to the poison that is tribal knowledge is the notion that documentation takes too long to write - it takes time away from developing features that contribute to business value. Therefore, writing documentation gets deprioritized and ultimately rarely ends up happening. The truth is that writing documentation does not need to be difficult or involved. Even a couple of hastily thrown together paragraphs that explains a piece of business logic in code can be shockingly helpful. Rather than requiring a word of mouth knowledge transfer, it is now out there in the open for anyone to read. If someone reads this documentation and still needs further clarification, then scheduling a knowledge transfer can be helpful. Bonus points if the engineers update the documentation after their KT session.
If that is not convincing enough, consider the practice of writing Unit Tests. We could use the same excuses against writing Unit Tests as we do against writing documentation: they take too long to write and don't directly contribute business value. But, we still write tests because we know it will help us in the long run. Automatically testing our code helps ensure that we maintain a clean, bug free codebase. In the same vein, writing documentation helps us in the long run as well - it ensures that our transfer of knowledge is clear and consistent.
If your team already includes Unit Testing as part of its feature development process, then implementing the practice of writing documentation is easy. Just as you make sure to include Unit Tests before you submit code for review, make it a point to also write documentation before you submit your code for review as well. Better yet, make sure to submit documentation for review in addition to submitting code and Unit Tests. This way, not only are you writing documentation as you create a new feature, but the team also plays a part in contributing to that documentation as well.
The Takeaway
Documentation is key to the maintaining a healthy transfer of knowledge within an organization. By taking thoughts in our heads and putting them in writing, knowledge transfers are no longer a temporary event - they become a permanent fixture in the organizations ecosystem. An engineer that joins years after another quits can still get a glimpse into the thought process of that ex-engineer. It becomes infinitely more valuable and sustainable than word-of-mouth sharing of tribal knowledge.
If you've enjoyed this content, consider subscribing to my mailing list to be notified whenever I release new content.
Top comments (2)
There's a saying for API development & DevOps. You should have a single source of truth that is easy to understand so that anyone can pick it up and run with it.
If your unable to have it written as a documentation that is easy to understand and in a accessible area. You are not able to scale yourself or organisation as a whole.
There will be times word of mouth is useful for codify ethos and stories so they will remember the lessons like "don't deploy on Friday", "deploy once a day", " leave no man behind" or "every marine is a rifleman"
100% agree here. In an ideal world there would be some central location for documentation that can serve as the source of truth for the entire organization.
Unfortunately creating and maintaining that central place doesn't seem to be a practice that organizations follow. At my last role I started this central repository for documentation that I had hoped people would contribute to, but no one ever did. They always push it off for "later", but "later" never comes.