To properly enjoy this article
This tutorial assumes you already followed the steps in part 1: Deploy AWS Resources using Crossplane on Kubernetes.
Also, this is the Github Repository we’ll be using:
https://github.com/MeteorOps/crossplane-aws-provider-bootstrap
What’s this article about?
In this article, we’ll cover a use-case that can benefit from Crossplane: full environment deployment.
This is a step-by-step guide with an example and a Git repository, so by the end of it, you should be able to deploy a sample env.
You can technically walkthrough the entire thing by "copy-paste" and everything should work. But, diving into the explanations with an extra 5–10 minutes will leave you with longer-term value.
Hope you enjoy!
What to expect from this article?
By the end of it, you’ll understand:
- How Crossplane can be used for full environment deployment
- How to deploy a sample app with AWS resources
What not to expect?
This article guides you through a simple application deployment, and not a full set of apps.
It also doesn’t go into using Crossplane in conjunction with Helm, but does cover important principles regarding it.
Why Crossplane for the Full Environment Use-Case?
When you want to deploy a full environment, it usually involves 3 layers:
- Infrastructure: Resources the application needs to run well
- Application: The programs built by the company to serve users
- Data: The data the application uses
But you already know that.
The thing is a tradition developed, and Crossplane sort of broke this tradition.
The tradition was this process: Build infrastructure, Deploy application on top.
How did Crossplane break this tradition?
The application deployment can now provision infrastructure required by the application.
Pull-Request Environments are also easier
By creating a namespace with all of the apps and the AWS resources required with Crossplane, the use-case of creating a full environment per Pull-Request as part of the CI becomes much easier.
That's a nice benefit of such setup for companies utilizing the feature-branch or Gitflow approaches.
A Traditional Full Env Example
To provision and deploy a full environment in the past, the process would generally look something like this:
- Provision VPC+EKS+... using Terraform
- Use Terraform to bootstrap the cluster with a CD tool (e.g., ArgoCD)
- ArgoCD looks at a repo that deploys all apps from there
- An application needs a new S3 Bucket, so the developer writes Terraform code for it
- The application gets removed after a while (but the bucket stays)
- Someone needs to remember that bucket was owned by that app and remove it from Terraform
A Crossplane Full Env Example
To provision and deploy a full environment with Crossplane the process is similar (we still need a Kubernetes Cluster to start with for the initial environment):
- Provision VPC+EKS+... using Terraform
- Deploy Crossplane’s prerequisites to the cluster with Terraform
- Add Crossplane resources to application Helm Charts (so they get their required infra upon deployment)
- Create a Crossplane manifest to deploy the Helm Charts + Some shared infra required by all apps
- When an application is removed, its AWS resources are gone with it
- When an entire environment is terminated, its AWS resources are gone with it
Crossplane in Helm vs. Helm in Crossplane
When using Crossplane alongside Helm, the question arises:
Should Helm apply the Crossplane code? Or, should Crossplane apply the Helm Charts?
I'm glad you asked it - the answer is both, depends when.
Reasons for Crossplane in Helm:
- Create or modify app-specific resources when that app is deployed
- Delete app-specific resources when that app is deleted
Reasons for Helm in Crossplane:
- Manage dependencies between resources and applications using Crossplane
- Create shared resources that are not owned by a single application
The Step-by-Step Guide
Deploy the simple application alongside a S3 bucket using a Crossplane Composite Application.
Before proceeding
Make sure you follow the steps in the 1st article (takes 3-minutes to just copy-paste the code snippets into your terminal and run the entire thing).
Deploy the Crossplane Kubernetes Provider
Prepare the AWS Credentials for the Application to be able to use AWS
Run the following oneliner to create the Secret containing the AWS credentials in the right format as required by the Application (the application will simply run aws s3 ls to show the bucket):
kubectl create secret generic aws-creds \
--from-literal=aws_access_key_id=$(grep -i aws_access_key_id creds | awk -F' = ' '{print $2}') \
--from-literal=aws_secret_access_key=$(grep -i aws_secret_access_key creds | awk -F' = ' '{print $2}')
Make sure it was created as expected by fetching the secret:
Deploy the Crossplane Kubernetes Provider resources using the k8s-provider-bootstrap.yaml file
kubectl apply -f k8s-provider-bootstrap.yaml
Make sure the provider was created and is ready before proceeding to the next steps:
kubectl get providers provider-kubernetes
You should see something like this:
Deploy the Crossplane Kubernetes Provider Configuration using the k8s-provider-conf.yaml file:
kubectl apply -f k8s-provider-conf.yaml
This is done separately as it needs to happen after the Provider resources were created.
This is where we tell the Crossplane Kubernetes Provider in which Kubernetes cluster it should operate when it’s creating resources.
Create a deployable unit of an App & AWS Resources using Crossplane
Here we do 3 things with 3 files:
The
composite-app-xrdfile:
Contains the CompositeResourceDefinition (XRD) for the K8sApplication by using the Composition of a K8s Deployment and S3 Bucket (described below)The
composite-app-compositionfile:
Contains the Composition definition which creates both the Kubernetes Deployment and the S3 BucketThe
composite-app-examplefile:
Calls the CompositeResource defined bycomposite-app-xrdfile
Crossplane Resources Files Breakdown & Creation
composite-app-xrd.yaml
~ K8sApplication CompositeResourceDefinition
This defines a composite resource for a Kubernetes application, with bucketName and bucketRegion fields in the spec. Users can claim this resource as K8sApplication.
The K8sApplication CompositeResource (XRD) accepts the bucketName & bucketRegion fields and uses them to create an S3 Bucket, and to create a K8s Deployment of a mock “service” that simply runs aws s3 ls to see the bucket.
~ Deploy the CompositeResourceDefinition (XRD)
kubectl apply -f composite-app-xrd.yaml
composite-app-composition.yaml
Defines a Composition of resources that can be created by a CompositeResource.
This is where we define the Composition that creates a combo of a Kubernetes Deployment with the mock “service” that runs aws s3 ls as well as the S3 bucket — The CompositeResource simply calls this resource.
~ Deploy the Composition
kubectl apply -f composite-app-composition.yaml
composite-app-example.yaml
Deploys the actual K8sApplication CompositeResource, and passes the details of the region in which the bucket should be created, and the name of the bucket (both are also passed to the Kubernetes Deployment as environment variables that helps it access the same bucket).
As mentioned above, the CompositeResource calls the Composition which creates the resources using the Crossplane providers.
Deploy the app by running the following command:
kubectl apply -f composite-app-example.yaml
Look at your pretty Application
Fetch the K8sApplication resource you’ve just created by running the below command obsessively until it’s marked as Healthy:
kubectl get K8sApplication
You should get something like this:
Print the logs of the application and see it fetching the AWS S3 Bucket:
kubectl logs -l app=awscli
# 2024-10-17 16:00:31 my-app-bucket-nqzhx-xzjcq
# 2024-10-17 16:00:50 my-app-bucket-nqzhx-xzjcq
Cleanup
kubectl delete -f composite-app-example.yaml
Recap
To briefly recap what you did here:
- Prepared Crossplane for deploying a mix of Kubernetes and AWS resources
- Defined the manifests required to deploy an app built of a Deployment and a S3 Bucket
- Sharpened your grasp on some Crossplane concepts
- Discussed some use-cases for which it’s useful
Hope you enjoyed this article, and if you are interested in another article about something related (or unrelated), please convince Michael it’s a good idea at michael@meteorops.com.
Disclaimer: In actual environments or production, it’s essential to fine-tune the permissions in the different manifests. Instead of using access keys and secret keys directly, consider implementing IAM Roles for Service Accounts (IRSA) to manage permissions more securely.
Top comments (1)
Awesome use-case, and pretty straight forward!