Pros and Cons of Using Internal Development Platform in the Cloud Computing World
Introduction
Internal Development Platform (IDP) as a brand new word in the DevOps and product development. In this article, I'm explaining:
- What exactly IDP is
- How to involve it in your organization
- An overview of the existing IDP platforms on the market, their pros, and cons
- An example of IDP Architecture based on the Azure Cloud
What is an internal developer platform?
As a software architect, I design the DevOps processes and spend much time selecting the DevOps toolset, building pipelines, and communicating with the DevOps team to implement some procedures for the development team. We can solve this with the internal development platform (IDP).
An internal developer platform(IDP) is an additional layer between development and DevOps teams. The main idea behind the IDP is to remove the dependency between DevOps and developers.
Let's look at what functions and components you should include in the IDP. The IDP platform should provide the following:
RBAC management
Infrastructure management
Configuration management
Deployment management
Let's have a look at what is the IDP from insights and how it is connected with DevOps tools and cloud providers.
IDP and DevOps
As seen in the previous example, IDPs are like frameworks built on top of DevOps tools, resources, and cloud providers. Some developer platforms may include an option to work with numerous DevOps tools and cloud providers, like in the diagram above. Other IDPs focused on specific ones.
The diagram below shows the example of the IDP components and how they combine DevOps tools and cloud providers.
Figure 1
Let's see what options of developer platforms are on the market. For example, Gimplet.IO is an open-source IDP that focuses primarily on operating Kubernetes deployments. Gimlet contains CLI to manage charts, deployments, secrets, and GitOps.
The Gimlet doesn't contain application configuration management which makes it partial IDP. I like the idea when IDP focuses on specific platforms. For example, Kubernetes. It makes it more stable than other platforms that try to cover all well-known platforms and cloud providers. Covering too many platforms can be suitable for marketing. However, having the proper support of multiple platforms and cloud providers requires significant effort as they have different APIs, standards, and even naming conventions.
Another good example is an Upbound. Upbound is a powerful IDP that allows you to deploy and build your application in the Kubernetes cluster. It supports GCP, AWS and the Goolge Cloud. I like how an Upbound team supports the documentation and provides architecture examples and architecture references. Below you can see Azure Reference Architecture for Kubernetes.
Figure 2
There are products like Backstage that made good progress in supporting different platforms. Backstage is an open-source platform made by Spotify and supported by Cloud Native Computing Foundation (CNCF). The Backstage is a platform that is intended to build developer portals. It includes the following components:
Backstage Software Catalog for managing microservices, libraries, data pipelines, websites, and ML models.
Backstage Software Templates for quickly creating new projects.
Backstage TechDocs for generating project documentation
The DevOpsBox is an example of a platform that supports numerous DevOps platforms, and Cloud Providers offer application configuration management, infrastructure orchestration, environment management, and adequate RBAC. However, many cloud providers and tools are still in the implementation phase ' for example, Azure and Google Cloud.
And last but not least is a Humanitec. It is a platform that connects multiple GitOps tools and Cloud platforms. I like the concept that Humanitec brings. For example, you can connect one or several Kubernetes clusters over Azure and AWS. Connect Datadog and Grafana. You can even reuse your existing Terraform and Pulumi scripts. You can operate all this infrastructure via CLI, API, and developer-focused UI. Developers can quickly provision Kubernetes clusters, spinup databases, and VM instances without writing pipelines, manifests, and Terraform scripts.
They don't have to wait for the DevOps team when they provide this one. However, DevOps specialists still should spend time on:
Selecting DevOps tools
Migrating existing scripts, pipelines, and manifests
Connect an environment
Figure 3
Humanitec also supports the PlatformCON community around IDP and DevOps topics.
Example of internal development platform
Many enterprise companies like Spotify are building their own IDP. They already have a massive DevOps code base that includes scripts and templates, pipelines for the most available platforms, and cloud providers.
A few years ago, I was working for an enterprise company with 500+ ongoing projects, and they needed to set up a team for the new project quickly. All projects need to provide infrastructure ASAP. The developer had no time to do DevOps stuff. The company asked to build Internal Developer Platform (IDP) to provision and manage infrastructure. The idea is simple. When a new project appears, the algorithm should be the following:
Onboard a development team.
The support engineers provision all required infrastructure.
Figure 4
As you can see, the architecture is primarily based on the Azure stack. However, you can easily extend it to use in AWS, GoogleCloud, or any other cloud provider. The IDP can provide the following resources and applications:
Azure DevTestLabs, VMs, and Extensions
Kubernetes Clusters (AKS)
Databases
App Service
A developer portal is a simple ReactJS application that uses an internal API. The API is based on Azure API management and contains several functions representing a single API call. For example, to trigger deployment or receive deployment status. The API layer is connected to the Azure DevOps API. The Azure DevOps contains YAML pipelines, Terraform, Bash, and PowerShell scripts.
Pros and Cons
An IDP provides you with the following benefits:
Developer's productivity. Developers can deliver the code and focus less on DevOps tasks.
DevOps teams freedom. An IDP reduces the infrastructure provisioning workload and focuses on building baseline configuration and templates.
IDP can bring value to the product. However, there are some drawbacks involving IDP. Below I've listed some of them:
Costs and complexity. Building a developer platform requires companies to provide additional resources to build, support, and keep it updated. We can partially solve this issue by buying the external development platform. I will describe a few existing platforms below.
Culture change. Involving a developer platform requires the development and DevOps team to change their culture and way of working. It may end up with the teams will lose time.
The list can be endless and may require its own article. In the article: Why companies that build internal dev platforms wouldn't do it again you can find a lot more points against building IDP internally.
Conclusion
Here I've tried to clarify what is an internal developer platform and how you and your organization can benefit from involving IDP. Also, I've added points about why using IDP may be not reasonable. Also, I've shared the architecture example that demonstrates how an organization can build its IDP and save time on managing several DevOps teams across the company.
Top comments (0)