When Their Stuff Doesn't Work With Your Stuff

Here's the situation: There is an open source library that you love to use that works with an open source platform that your solution is built on. You use this library all the time and it's part of your standard set of tools for operating and managing the platform. Now you move the solution to a new cloud vendor which provides you the platform as a managed service. But, unfortunately the library you know and love no longer works in this new environment!

Let's explore this very real situation that unfortunately comes up too regularly. At the heart of this common problem is the risk that is part of any non-trivial software system.

There are 3 parts to this puzzle.

1. The Open Source Software or Platform itself.
2. The Cloud Flavor of this Open Source Software or Platform.
3. The Ecosystem of tools and libraries around the Open Source Software or Platform.

Note that I wont cite any specific examples to protect the names of the innocent :).

Cloud Flavors of Open Source

Modern software architecture today means using lots of different solutions from a lot of different sources. We use cloud services, 3rd party applications and open-source software. Making things more complicated is that many cloud providers have their own flavors of popular open-source software. For instance: AWS MySQL vs GCP MySQL vs Azure MySQL. You maybe thinking: 'So what? MySQL is MySQL is MySQL.' And, like-wise, 'Kubernetes is just Kubernetes.' Yes, but, and there is a big BUT: each platform brings its own features and integrates them into their own offerings.

These different Cloud Flavors often require specific adjustments, tweaking and testing. Usually it is not the core workload that varies among these Cloud Flavors. Rather, it is in the operationalization (observability, security, fault tolerance, etc.) where there can be lots of differences.

Open Source Ecosystem around Open Source.

Popular Open Source platforms, especially Open Source platforms, often have an ecosystem of other Open Source tools around them. Tools like libraries, plugins and connectors that provide additional features and integrations are widely used by the community. Think of WordPress and the tens of thousands of plugins available for it.

So cloud vendors have their own implementations of open source software. That open source software has its own ecosystem of open source tools. Everything should all work together, right? Ideally, yes, but that is not always the case. And when things don't work as expected who is responsible for fixing it?

Look at the image above. Suppose you want to use "Tool 1" with Open Source Software on Cloud Vendor C and it doesn't work as expected.

Who is Responsible?

Who is responsible for making sure the tool works?

• The Open Source Tool 1 maintainers?
• The Open Source Software maintainers?
• The Cloud Vendor C engineers?

We know it's a problem for you when you want to get your solution working. Who are you likely to expect to fix it?

My experience tells me that customers believe it is the Cloud Vendor who should be responsible. After all, the tool works with the platform in the non-managed service. The only thing different is the cloud vendor's implementation. But it is not that simple.

Usually the cloud vendor is building a specific implementation to simplify management of the software, and that can come with some trade-offs or lack of support for some integrations. Like WordPress, popular open source software and platforms can have an ecosystem of dozens, hundreds or thousands of libraries and plugins. How is the Cloud Vendor supposed to test, let alone support them all?

Look to the Docs!

Often you will find documentation and support in various places and that might begin to help us figure out who should be responsible.

• The tool maintainers may provide guidance for installing and running the tool on different cloud vendors. If they do that, then they should encourage the community to open bugs when problems are discovered.
• The cloud vendor may also provide specific documentation on how to use its service with popular 3rd party plugins and tools. If they provide this documentation, you can assume it's supported and customers should be able to submit support tickets when things don't work as expected.
• Rarely would the Open Source platform itself be responsible for fixing how 3rd party solutions integrate or operationalize the software.

Where is the Risk?

If the cloud vendor doesn't provide specific documentation, you may be using that tool at your own risk. How much risk is there?

• Does the tool project website say they specifically support the cloud vendor? That's good.
• Does the cloud vendor say they support the tool and provide documentation? That's even better.
• If the cloud vendor doesn't support it, things may work today, but may not work tomorrow. When the cloud vendor makes a change, they wont test it against unsupported 3rd party tools. If a change breaks the tool, you'll have to wait for the tool maintainers to update it (or submit a pull request with your own fix!).
• If the tool is focused more on the core workload of the platform, there is generally less risk.
• if the tool is focused on the operationalization of the platform there is generally more risk.

Final Thoughts

Always try to use supported libraries and tools when building solutions. The open source ecosystem combined with open source platforms and a cloud flavor, can create infinite variations. It may be hard to always build a solution with tools that are 100% supported in all these environments. Make explicit decisions and document what tools you use and for what reasons. Calculate the risk and have a mitigation plan.

Finally, as a Cloud Architect, I look at the world of opinionated software development to get a perspective on this. Think of Java Spring Boot and how it lets you get productive with "minimal fuss." If you choose a Cloud Flavor of an open source platform, why not also choose the built-in capabilities and supported tools? You're choosing a managed service to reduce complexity and ultimately save costs, so try to embrace it's "opinion" on what tools you should use with it.

Let me know what you think!