DEV Community: Joonas Venäläinen

Crossplane: How do providers work

Joonas Venäläinen — Tue, 17 Oct 2023 07:16:35 +0000

Providers are the meat around Crossplane’s bones, and they are used to extend the capabilities of Crossplane. When Crossplane is installed, it doesn't have any capabilities to interact with external systems. A core Crossplane pod will only watch the following resources.

compositeresourcedefinitions.apiextensions.crossplane.io
compositionrevisions.apiextensions.crossplane.io         
compositions.apiextensions.crossplane.io                 
configurationrevisions.pkg.crossplane.io                 
configurations.pkg.crossplane.io                         
controllerconfigs.pkg.crossplane.io                      
locks.pkg.crossplane.io                                  
providerrevisions.pkg.crossplane.io                      
providers.pkg.crossplane.io                              
storeconfigs.secrets.crossplane.io

When you install a provider, a new pod is created to Crossplane's installation namespace. This pod is a Kubernetes Controller that watches the CRDs that are also installed as part of the provider package.

To find out what different kinds of providers are available, you can check the Upbound Marketplace and crossplane-contrib repository. For this series, we are going to work with the following providers:

Those GCP providers are installed from the provider-family-gcp package. These provider-family packages are special packages that allow you to install only the provider packages you need instead of installing everything, which would mean 343 CRDs if you install the provider-gcp package instead. Crossplane also states:

On average, 30 CRDs are used from Provider packages.

Looking at the average number, you would still have ~313 CRDs in the cluster that aren't used 🤯.

Install the providers

cat <<EOF | kubectl apply --filename=-
apiVersion: pkg.crossplane.io/v1
kind: Provider
metadata:
  name: provider-gcp-storage
spec:
  package: xpkg.upbound.io/upbound/provider-gcp-storage:v0.36.0
---
apiVersion: pkg.crossplane.io/v1
kind: Provider
metadata:
  name: provider-gcp-cloudplatform
spec:
  package: xpkg.upbound.io/upbound/provider-gcp-cloudplatform:v0.36.0
---
apiVersion: pkg.crossplane.io/v1
kind: Provider
metadata:
  name: provider-terraform
spec:
  package: xpkg.upbound.io/upbound/provider-terraform:v0.10.0
EOF

After a little while, you should see the providers installed and in a healthy state

kubectl get provider
---
NAME                          INSTALLED   HEALTHY   PACKAGE                                                      AGE
provider-gcp-cloudplatform    True        True      xpkg.upbound.io/upbound/provider-gcp-cloudplatform:v0.36.0   116s
provider-gcp-storage          True        True      xpkg.upbound.io/upbound/provider-gcp-storage:v0.36.0         116s
provider-terraform            True        True      xpkg.upbound.io/upbound/provider-terraform:v0.10.0           116s
upbound-provider-family-gcp   True        True      xpkg.upbound.io/upbound/provider-family-gcp:v0.37.0          107s

Now the providers are installed and ready, we need to set up the ProviderConfig, which configures the credentials for the provider to be able to interact with external systems, in this case, with Google Cloud. You can have multiple ProviderConfigs and reference them in managed resources using providerConfigRef. ProviderConfigs are cluster-scoped resources.

You can set up a ProviderConfig per tenant when you have a multi-tenant cluster. When creating compositions, you could patch the value of providerConfigRef in managed resources with a value of spec.claimRef.namespace, which points to the namespace where the XRC was created.

Every provider has their own individual settings available when it comes to ProviderConfig. For the GCP provider, you can find all the available configuration options here and for Terraform provider here. If you need to override Controller related settings eg. ServiceAccount you can use ControllerConfig for that.

In upcoming chapters, we will create resources in Google Cloud that involve creating a bucket, serviceaccount, iam-binding, and serviceaccountkey. Use the following to configure a new service account with needed permissions in GCP:

# GCP Project ID
PROJECT_ID=""

gcloud iam service-accounts create crossplane-sa-demo --display-name "Crossplane Service Account Demo"

gcloud projects add-iam-policy-binding $PROJECT_ID --member serviceAccount:crossplane-sa-demo@$PROJECT_ID.iam.gserviceaccount.com --role roles/storage.admin
gcloud projects add-iam-policy-binding $PROJECT_ID --member serviceAccount:crossplane-sa-demo@$PROJECT_ID.iam.gserviceaccount.com --role roles/iam.serviceAccountAdmin
gcloud projects add-iam-policy-binding $PROJECT_ID --member serviceAccount:crossplane-sa-demo@$PROJECT_ID.iam.gserviceaccount.com --role roles/iam.serviceAccountKeyAdmin
gcloud projects add-iam-policy-binding $PROJECT_ID --member serviceAccount:crossplane-sa-demo@$PROJECT_ID.iam.gserviceaccount.com --role roles/storage.iamMember

Create a service account key:

gcloud iam service-accounts keys create credentials.json --iam-account=crossplane-sa-demo@$PROJECT_ID.iam.gserviceaccount.com

Create a Kubernetes secret in crossplane-system namespace that contains the previously created credentials:

kubectl create secret generic gcp-creds --from-file=creds=./credentials.json -n crossplane-system

Create ProviderConfig that uses these credentials:

cat <<EOF | kubectl apply --filename=-
apiVersion: gcp.upbound.io/v1beta1
kind: ProviderConfig
metadata:
  name: default
spec:
  projectID: $PROJECT_ID
  credentials:
    source: Secret
    secretRef:
    name: gcp-creds
    namespace: crossplane-system
    key: creds
EOF

If you run this inside GKE, using the Workload Identity for authentication is much better. You can find detailed instructions for it here.

You can also read the secret from the filesystem using fs. This might come in handy in cases where you are leveraging, for example, Hashicorp Vault with Vault Agent sidecar to inject secrets to pods. Here is a quick example of how you would configure it without going into too much detail about how to work with Vault Agent Injector:

apiVersion: pkg.crossplane.io/v1alpha1
kind: ControllerConfig
metadata:
  name: gcp-config
spec:
  metadata:
    annotations:
    vault.hashicorp.com/agent-inject: "true"
    vault.hashicorp.com/role: "crossplane-providers"
    ...
---
apiVersion: pkg.crossplane.io/v1
kind: Provider
metadata:
  name: provider-gcp-storage
spec:
  package: xpkg.upbound.io/upbound/provider-gcp-storage:v0.36.0
  controllerConfigRef:
    name: gcp-config
---
apiVersion: gcp.upbound.io/v1beta1
kind: ProviderConfig
metadata:
  name: default
spec:
  projectID: $PROJECT_ID
  credentials:
    source: Filesystem
    fs:
      path: /vault/secrets/gcp-creds

Now we can quickly test that everything is working by creating a Bucket resource:

cat <<EOF | kubectl apply --filename=-
apiVersion: storage.gcp.upbound.io/v1beta1
kind: Bucket
metadata:
  name: ps-bucket-${RANDOM}
spec:
  forProvider:
    location: US
EOF

After a little while, you should see the bucket resource ready and synced:

kubectl get bucket
---
NAME                            READY   SYNCED   EXTERNAL-NAME                  AGE
bucket-crossplane-demo-30855    True    True     bucket-crossplane-demo-30855   15m

At this point, we are ready to start working with GCP using Crossplane. I will go through setting up the Terraform provider configs later in the series when it's time to start working with it.

Remember to delete the test bucket resource:

kubectl delete bucket <bucket_name>

The next chapter quickly reviews available configuration options for managed resources.

Crossplane: Streamline your infrastructure provisioning & management

Joonas Venäläinen — Tue, 17 Oct 2023 07:16:15 +0000

Crossplane is an extension to Kubernetes, which transforms your Kubernetes cluster into a universal control plane. It allows you to manage anything that has an API available with the help of provider packages. It's also fully extendable, so you can build your own providers to support your APIs. Everybody likes 🍕 so here is a post providers-101-ordering-pizza-with-kubernetes-and-crossplane which goes through on a high level how to build a provider that is capable of ordering pizza for you inside Kubernetes cluster.

Crossplane is often used to interact with cloud providers like Azure, AWS, and GCP. By using the Crossplane, you can bring your infrastructure management to Kubernetes. Another significant benefit of Crossplane is that it acts as a Kubernetes Controller, constantly monitoring the state of external resources. So, if someone would modify/delete the resources outside of Kubernetes, Crossplane would reconcile the resources.

By connecting Kubernetes and the cloud provider APIs with Kubernetes Custom Resource Definitions (CRDs), Crossplane can be used to enable a Kubernetes-native approach to managing cloud resources. We can for example define a Database custom resource which will then provision a database in the cloud provider that we have hooked Crossplane into. This makes it easier for developers to start consuming infrastructure resources by hiding the complexity behind Crossplane compositions. The Ops team can create and maintain these compositions and the XRDs which define how the resource will look like for the consumers. Without going into too much detail, we could have a custom resource Database that the developers can consume.

apiVersion: storage.polarsquad.com/v1alpha1
kind: Database
metadata:
  name: ps-demo-db-mysql
spec:
  parameters:
    name: ps-demo-db-mysql
    size: "small"

Here, we define the size as small. The Ops team could set small, medium, and large options for the size. Then, in compositions, patch these values to specific instance types depending on the cloud platform. This also enhances the experience for the developers as they don't have to know the specific instance types.

As the resources are native Kubernetes manifests, you can bundle them with the other manifests you use to deploy the application to the cluster. When the resources are created, Crossplane will create the connection secrets to the application namespace for pods to consume.

Overall, the Crossplane allows you to form your own Cloud Platform inside Kubernetes. With the help of compositions, you can build a self-service platform where developers can easily create resources on-demand when they need them without having to jump through hoops to get additional backing services.

Throughout the series, I will be using the terms XRD,XR, and XRS. Here is a quick overview of what they stand for.

XRD - Composite Resource Definition
XR - Composite Resource
XRC - Claim

Prerequisites:

Access to Google Cloud and gcloud cli
Kubernetes cluster eg. minikube
Helm
Access to Aiven

Aiven offers a free tier so you can create an account and use it to get through the tutorial.

Install Crossplane

helm repo add crossplane-stable https://charts.crossplane.io/stable

helm install crossplane --namespace crossplane-system --create-namespace crossplane-stable/crossplane

At this point, the Crossplane is ready, and the next step is to install the needed Providers that give the Crossplane capabilities to provision managed resources to external systems.

In this series, we will provision resources to Google Cloud using the Google Cloud providers and, later in the series, leverage Terraform provider to manage resources that don't have a Crossplane native provider available yet.

Links to each part of this series:

Crossplane: How do providers work