Platform Engineering On Kubernetes Part 3: Crossplane

In Platform Engineering, the whole goal, at a high level, is to make software delivery and engineering more efficient while separating concerns between engineers and ensuring that there’s an underlying platform to carry out all of the expected tasks.

In this case, that platform is Kubernetes, but what’s the task? Or rather, how can Kubernetes carry out the task for anything outside of the cluster itself?

In part 2 of the Platform Engineering On Kubernetes series, you learned all about Cluster API along with the overlap when it comes to Operators and Cluster API. In part 3, you’ll learn about one of the top tools that truly makes Kubernetes use itself as the Platform to deploy various resources, Crossplane.

Prerequisites

To follow along with the hands-on section, you should have access to Azure, AWS, or both. If you don’t, it would still be beneficial to read through the hands-on section as it’ll give you an idea of how Crossplane works from a declarative perspective.

Why Crossplane

In the opening of this blog post, there was mention that Kubernetes can be the platform, but the question then becomes “How does Kubernetes manage other resources?”. Out of the box, Kubernetes can’t. Kubernetes has a very specific job - orchestrating containers. However, the way Kubernetes was designed was to give engineers the ability to extend its capabilities. That’s why Operators exist. It’s why tools like ArgoCD, Istio, etc. can give you the ability to manage environments in a declarative fashion (using the provided APIs from those tools in YAML).

Where Crossplane comes into play is it has an Operator, much like ArgoCD and Istio, but the Crossplane Operator isn’t to manage Crossplane. It’s to manage other resources and services.

For example, you can use the Crossplane Operator within Kubernetes to create an S3 bucket in AWS from Kubernetes. Literally, you can write a Kubernetes Manifest that creates an AWS S3 bucket.

In part 2 of this blog post, you learned about Cluster API, which is managing Kubernetes with Kubernetes. Crossplane is managing everything else with Kubernetes.

Crossplane gives you the ability to truly utilize Kubernetes to do anything and everything for creating, managing, and updating resources/services outside of Kubernetes. It’s arguably the definition of Platform Engineering.

Crossplane Breakdown

Crossplane gives you a declarative method via Kubernetes to manage services, resources, and infrastructure outside of Kubernetes.

What this means is that you can do anything from creating an S3 bucket in AWS to creating virtual machines or AKS clusters. As long as the object is available in Crossplane, you can use it to create the resource/service/infrastructure.

For example, you can specify the API just like you would with any other Kubernetes Manifest

apiVersion: compute.azure.crossplane.io/v1beta1

Then, you can specify the object/kind. In this case, it’s an object to create an AKS cluster.

kind: AKSCluster

It doesn’t have to be a Kubernetes cluster you’re creating though. You can use the [network.azure.crossplane.io/v1beta1](http://network.azure.crossplane.io/v1beta1) API to create a Virtual Network in Azure.

In short, Crossplane allows you to create resources/services/infrastructure as you would in something like Terraform, except instead, you’d use a Kubernetes Manifest.

There are various Providers available, all of which can be found here.

Using Crossplane

Now that you know the “how” and “why” behind Crossplane, let’s dive into the hands-on piece and see how it all works underneath the hood.

The first part will be the overall configuration. Next, you’ll see how to run workloads on both Azure and AWS.

Configuration

First, add the Crossplane Helm repo.

helm repo add crossplane-stable https://charts.crossplane.io/stable

Ensure that the Helm Chart is up to date.

helm repo update

Lastly, install Crossplane. The Helm Chart for installation is straightforward in the sense that it isn’t mandatory to configure a lot of values.

helm install crossplane \
crossplane-stable/crossplane \
--namespace crossplane-system \
--create-namespace

AWS

Once Crossplane is installed, you can start going through the actual configuration of using it.

To use Crossplane, regardless of which Provider you’re using (Azure, AWS, GCP, etc.), you’ll need two key components:

1. The Provider itself
2. Credentials for authentication

First, set up the Provider. You’ll see that the below code configures the AWS provider.

cat <<EOF | kubectl apply -f -
apiVersion: pkg.crossplane.io/v1
kind: Provider
metadata:
  name: provider-aws
spec:
  package: xpkg.upbound.io/upbound/provider-aws-s3:v0.37.0
EOF

To ensure that the provider was installed properly, run the following command.

kubectl get providers

Next, it’s time to configure the credentials. Generate an access key and secret key from AWS IAM. Then, insert them into the below and save it to a file called creds.txt

[default]
aws_access_key_id = 
aws_secret_access_key =

Create a new Kubernetes secret with the IAM permissions.

kubectl create secret \
generic aws-secret \
-n crossplane-system \
--from-file=creds=./creds.txt

Create a Provider Config to customize Crossplane to use the secret you just created.

cat <<EOF | kubectl apply -f -
apiVersion: aws.upbound.io/v1beta1
kind: ProviderConfig
metadata:
  name: creds
spec:
  credentials:
    source: Secret
    secretRef:
      namespace: crossplane-system
      name: aws-secret
      key: creds
EOF

You’re now ready to use Crossplane. Below is a configuration to create an S3 bucket.

kubectl apply -f -
apiVersion: s3.aws.upbound.io/v1beta1
kind: Bucket
metadata:
  name: mjlbucket929211
spec:
  forProvider:
    region: us-east-1
  providerConfigRef:
    name: creds
EOF

Below is another configuration to create an EKS cluster.

apiVersion: eks.aws.upbound.io/v1beta1
kind: Cluster
metadata:
  name: ekscluster
  labels:
    example: "true"
spec:
  forProvider:
    region: us-east-1
    roleArnRef:
      name: role
    resourcesVpcConfig:
      endpointPrivateAccess: true
      endpointPublicAccess: true
      subnetIdRefs:
        - name: subnet1
        - name: subnet2
      securityGroupIdRefs:
        - name: cluster-sg
    version: "1.26"
  writeConnectionSecretToRef:
    name: cluster
    namespace: default
  providerConfigRef:
    name: provider-aws

Azure

When working with Azure, you’ll see that the setup and process is essentially the same.

First, add the Azure Provider.

cat <<EOF | kubectl apply -f -
apiVersion: pkg.crossplane.io/v1
kind: Provider
metadata:
  name: upbound-provider-azure
spec:
  package: xpkg.upbound.io/upbound/provider-azure:v0.29.0
EOF

Ensure that it was installed.

kubectl get providers

Generate RBAC credentials.

Run the following, copy the output, and save it to a file called azure.json

az ad sp create-for-rbac \
--sdk-auth \
--role Owner \
--scopes /subscriptions/your_sub_id

Create a Kubernetes Secret from the credentials above.

kubectl create secret \
generic azure-secret \
-n crossplane-system \
--from-file=creds=./azure.json

Add the secret to the provider config

cat <<EOF | kubectl apply -f -
apiVersion: azure.upbound.io/v1beta1
metadata:
  name: default
kind: ProviderConfig
spec:
  credentials:
    source: Secret
    secretRef:
      namespace: crossplane-system
      name: azure-secret
      key: creds
EOF

Start working with Crossplane! Below is an example of creating a new vNet.

cat <<EOF | kubectl create -f -
apiVersion: network.azure.upbound.io/v1beta1
kind: VirtualNetwork
metadata:
  name: vnet
spec:
  forProvider:
    addressSpace:
      - 10.0.0.0/16
    location: "US East"
    resourceGroupName: devrel-as-a-service
EOF

As you can see, working with Crossplane is a straightforward process, which makes your environment far easier to manage. You no longer have to have multiple tools and platforms to configure resources, services, and infrastructure. You only need one.

Comments

[MRostanski]

Nice introduction! However for AWS you should at least mention that using access keys instead of UseServiceAccount should be only for PoC purposes