Overview of Crossplane and Crossplane-provider-aws

yelenary — Fri, 13 Jun 2025 19:46:03 +0000

Overview

Crossplane is an opensource project primarily developed and maintained by Upbound, a company focused on building tools for cloud-native infrastructure and application management. The project also has contributions from various individual developers and organizations within the cloud-native and Kubernetes community.

Essentially, it is a Kubernetes-native control plane that enables declarative infrastructure and application management via Custom Resource Definitions (CRDs). Its power comes from the provider system, which enables Crossplane to interface with external APIs (e.g., AWS, GCP, Azure, etc.).

This document describes how Crossplane and a Crossplane provider (e.g., provider-aws) are interconnected, their dependencies, and how data flows between components.

Dataflow diagram

Dataflow Steps Explained:

User → Kubernetes API

The user creates or applies a resource (like a CompositeResourceClaim or ManagedResource) using kubectl or a GitOps tool to the Kubernetes API server.

Kubernetes → Crossplane Core

Crossplane’s controllers (running in the cluster) watch for these resources, reconcile their state, and orchestrate the desired infrastructure.

Crossplane Core → Crossplane Provider

Crossplane core delegates concrete provisioning tasks to a Crossplane Provider (e.g., Provider-AWS) via custom resources (e.g., RDSInstance, S3Bucket).

Provider → AWS

The provider translates these Kubernetes custom resources into AWS API calls (using AWS SDK) to create/update/delete actual cloud resources.

AWS → Provider → Kubernetes

The provider monitors AWS for the resource’s real state and updates the status of the corresponding Kubernetes custom resource, closing the reconciliation loop.

Main components and roles

Crossplane:

*Key Components *

*1. Crossplane Core *

Responsible for:

Managing Provider, Configuration, and Composition resources.

Bootstrapping and orchestrating the lifecycle of provider controllers via Provider and ProviderRevision resources.

*2. Provider *

Custom resource of kind Provider (pkg.crossplane.io/v1).
Triggers installation and deployment of the provider controller.

Specifies the OCI image of a provider package (e.g., provider-aws:v0.50.0).

kubectl get provider.pkg 

NAME                  INSTALLED   HEALTHY   PACKAGE                                   AGE 

abc-provider   True        True      crossplane-contrib/provider-aws:v0.50.0   56d

apiVersion: pkg.crossplane.io/v1 
kind: Provider 
metadata: 
  annotations: 
    kustomize.toolkit.fluxcd.io/prune: disabled 
  labels: 
    app.kubernetes.io/instance: crossplane-provider 
    app.kubernetes.io/managed-by: Helm 
    app.kubernetes.io/name: crossplane-provider 
    helm.sh/chart: crossplane-provider-v0.10.0
    helm.toolkit.fluxcd.io/name: crossplane-provider 
  name: xxx-provider 
  namespace: crossplane-system 
spec: 
  package: crossplane-contrib/provider-aws:v0.50.0 
  runtimeConfigRef: 
    name: crossplane-provider-aws

*3. ProviderRevision *

Providerrevisions are typically not directly created or managed by users. Instead, theyare managed automatically when you install a Provider, upgrade Provider to a new version or Uninstall Provider

Created automatically by Crossplane when a Provider is applied.
Represents an exact, immutable snapshot of a provider version.
Crossplane manages only ACTIVE revision.

Responsible for:

Installing CRDs defined in the provider package.
Running the controller logic for reconciling managed resources.
Can be rolled back or rolled forward by changing the package version in the Provider.
Only one ProviderRevision can be ACTIVE at a time per Provider.
Old revisions are marked INACTIVE but retained for rollback if needed.
Updating the package field in a Provider creates a new ProviderRevision.
Deleting the Provider will uninstall the provider, delete the CRDs, and stop the controller

kubectl get providerrevision   
NAME                               HEALTHY   REVISION   IMAGE                                     STATE      DEP-FOUND   DEP-INSTALLED   AGE
aws-provider-6c3fd1826792   True      3          crossplane-contrib/provider-aws:v0.50.0   Active                                 16d
aws-provider-b4eafc5192c9   False     2          crossplanecontrib/provider-aws:v0.46.0    Inactive                               29d

*4. ProviderConfig *

Defines credentials and configuration needed by the provider to interact with external APIs.

Consumed by ManagedResources (e.g., RDSInstance, Bucket) via providerConfigRef.

Example:

apiVersion: aws.crossplane.io/v1beta1 
kind: ProviderConfig 
metadata: 
  name: aws-provider 
spec: 
  credentials: 
    source: Secret 
    secretRef: 
      namespace: crossplane-system 
      name: aws-creds 
      key: creds

*5. DeploymentRuntimeConfig *

An optional configuration resource that controls how provider controller pods (like crossplane-provider-aws) are deployed within the Kubernetes cluster.

It belongs to the pkg.crossplane.io API group and is used in conjunction with a Provider or Configuration resource to customize runtime settings such as:

resource limits (CPU/memory)
environment variables
volume mounts
labels and annotations
security contexts

*6. Managed Resources *

Represent real infrastructure (e.g., Bucket, RDSInstance, Network) modeled as Kubernetes CRs.

Managed by the provider controller.

Always reference a ProviderConfig.

Crossplane-provider-aws

After installing Crossplane, you install the AWS provider using a YAML manifest like:

apiVersion: pkg.crossplane.io/v1 
kind: Provider 
metadata: 
  name: provider-aws 
spec: 
  package: xpkg.upbound.io/crossplane-contrib/provider-aws:v0.50.0

This triggers Crossplane to:

Install the provider-aws OCI package

Crossplane fetches the image from Upbound’s OCI registry.

It spins up a new Deployment (in-cluster controller) for the provider.

kubectl get deployment -n crossplane-system 

NAME                               READY   UP-TO-DATE   AVAILABLE   AGE 

crossplane-provider-6c3fd1826792   1/1     1            1           16d 

crossplane                         1/1     1            1           618d 

crossplane-rbac-manager            1/1     1            1           618d

The provider’s controller pod installs:

Deployment (1 per provider): runs provider-aws controller manager
ProviderRevision object: tracks version and status
CustomResourceDefinitions (CRDs) for all supported AWS services E.g., s3buckets.aws.crossplane.io, rdsinstances.database.aws.crossplane.io, etc.

You can list all the CRDs installed with the provider by:

kubectl get crds | grep aws.crossplane.io

Dependency Flow

Crossplane Core is installed: installs built-in CRDs (e.g., Provider, Composition).
Provider resource is applied: triggers Crossplane to fetch the provider package from an OCI registry.
ProviderRevision is created: immutable snapshot of the provider package.
CRDs are installed by the revision (e.g., buckets.s3.aws.crossplane.io).
Controller Deployment is spun up: runs reconciliation logic for those CRDs.
User applies ProviderConfig: specifies cloud credentials.
User creates ManagedResource: such as Bucket, VPC, RDSInstance.
ManagedResource refers to ProviderConfig: to get credentials.
Provider controller reconciles the ManagedResource, calls external cloud API to provision/update/destroy real-world resources.

Step-by-Step: Configuring Internal YUM Repo Access on RPM-Based Hosts

yelenary — Thu, 05 Jun 2025 17:29:23 +0000

Background

In modern enterprise environments it's common to host private packages in an internal YUM repository. This guide walks through configuring a Rocky Linux 8/9 (or other RPM-based) host to securely access and install packages from a private YUM repository hosted in Google Artifact Registry.

✅ Prerequisites

A host running Rocky Linux 8 or 9
Access to a private YUM repo in Google Artifact Registry
A service account JSON key with the required permissions
yum or dnf installed

🛠 Step-by-Step Setup

🔹 Step 1: Import the GPG Key for Google Cloud RPM Packages

sudo rpm --import https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg

🔹 Step 2: Add the Artifact Registry Plugin Repository

Create a new repo file at /etc/yum.repos.d/artifact-registry-plugin.repo:

sudo tee /etc/yum.repos.d/artifact-registry-plugin.repo <<EOF
[ar-plugin]
name=Artifact Registry Plugin
baseurl=https://packages.cloud.google.com/yum/repos/dnf-plugin-artifact-registry-el9-stable
enabled=1
gpgcheck=1
EOF

📝 This enables your host to install the required plugin for accessing Google Artifact Registry.

🔹 Step 3: Update YUM and Install the Artifact Registry Plugin

sudo yum makecache
sudo yum install dnf-plugin-artifact-registry

🔹 Step 4: Configure the Artifact Registry Plugin Credentials

Create or edit the file /etc/dnf/plugins/artifact-registry.conf:

[main]
enabled=1
service_account_json = "/path/to/your/service-account.json"

⚠️ Replace /path/to/your/service-account.json with the actual path to your GCP service account key file.

🔹 Step 5: Add Your Internal YUM Repository

Create the file /etc/yum.repos.d/yum-private.repo with the following content:
sudo tee /etc/yum.repos.d/yum-private.repo <<EOF
[yum-private]
name=Internal YUM Repo
baseurl=https://<region>-yum.pkg.dev/path/to/repo
enabled=1
repo_gpgcheck=0
gpgcheck=0
EOF

🔁 Replace with your Google Cloud region (e.g., europe-west3) and /path/to/repo with your actual repository path inside Artifact Registry.

🔹 Step 6: Refresh the Package Cache

sudo yum makecache

This ensures the repo metadata is up to date and packages are available to install.

✅ Conclusion:

You’ve now configured your RPM-based host to securely pull packages from a private YUM repository hosted in Google Artifact Registry. This is especially useful in production or enterprise environments where package access must be controlled, reproducible, and secure.

🧠 Bonus Tip

To automate this process, consider:

Writing an Ansible role or bash script
Integrating this configuration into your Packer build if you're baking AMIs or VM images
Managing the credentials via Vault or GCP Secret Manager for security

DEV Community: yelenary