DEV Community: yelenary

Is CloudNativePG Ready to Replace RDS/Aurora for High-Traffic Databases?

yelenary — Mon, 29 Jun 2026 01:27:56 +0000

Is CloudNativePG Ready to Replace RDS/Aurora for High-Traffic Databases?

Short answer: Not for most teams. But for the right team, in the right conditions, it's a legitimate move.

I spent time digging into CloudNativePG (CNPG) — the CNCF-backed Postgres operator for Kubernetes — to answer a question I keep hearing from platform engineers: "Can we drop RDS/Aurora and run Postgres on EKS ourselves?"

Here's what I found, without the vendor spin.

The Evidence Problem

Before anything else: the public evidence base for CNPG at true high-traffic scale is thin.

Named enterprise adopters exist — GEICO Tech, HSBC, EDB's BigAnimal, IBM Cloud Paks. HSBC presented a real migration story at KubeCon EU 2026. The strongest concrete number comes from Mirakl: 450 Postgres clusters, 31 TB of data, 3,650 CPUs — impressive fleet scale.

But there is no published, independent benchmark of CNPG handling thousands of TPS on a single multi-TB database that explicitly migrated off Aurora or RDS. If you're evaluating this for a tier-1 OLTP workload, you're operating without a public proof point at that scale.

That's not a dealbreaker. But it's a flag.

Where CNPG Actually Falls Short Today

Failover is slower — and that's mostly Kubernetes, not CNPG

Aurora Multi-AZ typically fails over in ~30 seconds. RDS Multi-AZ clusters: under 35 seconds, with zero data loss.

The only public measured number for CNPG on node failure is a community report of 40–80 seconds — where the dominant term isn't CNPG's logic, it's Kubernetes' node-monitor-grace-period (40s on K8s 1.29–1.31, raised to 50s in 1.32+). After node detection, add EBS detach/reattach (20–90 seconds) before the new primary is writable.

Pod-level failures are faster. But losing a node — the failure mode that actually keeps you up at night — is gated by K8s infrastructure, not the operator.

RDS Proxy and Aurora also mask failover from clients by holding connections and re-routing. CNPG's PgBouncer Pooler doesn't queue connections across a failover the same way. Your clients see the blip.

No automatic primary fencing — by design

CNPG explicitly chose not to auto-fence an isolated primary. In a network partition, CNPG prevents split-brain at rejoin time rather than force-shutting the isolated primary immediately (the Patroni model).

With async replication, this is a real consistency risk. The mitigation is 3-node synchronous (quorum) replication across 3 AZs — which works, but adds write latency and means writes pause if a required standby goes unavailable.

Barman backup is mature, but you own it entirely

CNPG uses Barman Cloud for backups. It's solid. But there's a near-term migration required: the in-tree barmanObjectStore is deprecated as of CNPG 1.26 and removed in 1.28. New deployments need the plugin-barman-cloud sidecar architecture.

Documented failure modes in the wild: stalled WAL archiving filling the PGDATA volume and causing crashloops. The XAmzContentSHA256Mismatch bug after a routine image bump is the kind of incident that reminds you who's on-call when AWS isn't managing your backups.

Where CNPG Genuinely Wins

Cost at large scale — but the math is less obvious than it looks

Aurora I/O billing can be brutal. A busy OLTP workload at ~50,000 I/O/s is over $25,000/month in I/O charges alone under Standard pricing. Aurora I/O-Optimized helps when I/O exceeds 25% of your Aurora bill — but that's still you paying the AWS premium.

CNPG on EKS pays EC2 + EBS + S3. The savings are real. The catch: a senior engineer in Canada runs CAD 150k–250k/year fully loaded. Even a 0.25 FTE allocation for on-call, patching, upgrades, and restore drills is CAD 40–60k/year of fixed cost. At small/medium managed-DB spend, CNPG's infrastructure savings don't cover the labor. The crossover only happens when you're spending tens of thousands per month on RDS/Aurora — large fleets or I/O-heavy workloads — AND you already employ the expertise, making the marginal labor near-zero.

The unexpected win: Debezium / CDC workloads

This is the one area where the "default to managed services" recommendation actually flips.

Debezium depends on a logical replication slot surviving failover. Historically, on any self-managed HA Postgres, it didn't — which forced a full re-snapshot of a multi-TB database after every failover. Not acceptable at scale.

Aurora is the weakest of the three here. Its shared-storage replication does not preserve logical slots across failover. You re-snapshot.

CNPG ≥1.27 on PostgreSQL 17 natively synchronizes logical decoding slots across the HA cluster via synchronizeLogicalDecoding. Combined with CNPG's stable -rw service endpoint, Debezium reconnects after failover without re-snapshotting — the slot survives and the connection follows automatically. For a CDC-heavy shop, this is a legitimate reason to choose CNPG over Aurora.

Caveat: this is feature-existence, not a published production benchmark at scale. Validate it before trusting it.

Full Postgres control

RDS/Aurora restrict extensions to an allow-list and withhold true superuser. CNPG gives you any extension, any version, on your schedule. For teams blocked by RDS extension limitations or needing newer Postgres majors immediately — this is real.

GitOps-native operations

CNPG was designed for Kubernetes from day one. Declarative cluster specs, immutable images with SBOM provenance, native Prometheus metrics, PGAudit support. If your team lives in GitOps, CNPG fits that model in a way RDS/Aurora never will.

The Honest Decision Framework

Stay on Aurora / RDS if:

You don't have ≥2 engineers with deep overlapping Postgres + Kubernetes + storage/CSI expertise (bus factor matters — losing one person shouldn't be an existential risk)
Your managed-DB spend isn't large enough to justify the fixed operational overhead
Your RTO requirements are tight and you haven't benchmarked CNPG under your actual write load
You need compliance coverage you can point at (SOC, PCI, HIPAA eligibility) without building your own evidence chain

Consider CNPG if — and only if — at least 3 of these are true:

You already employ the Postgres + Kubernetes + storage expertise (marginal labor ≈ zero)
You're spending tens of thousands/month on managed DBs — fleet scale or I/O-heavy Aurora
You have a hard requirement managed services can't meet: extension control, version control, multi-cloud portability, data sovereignty, or CDC/Debezium slot failover
You can run 3 AZs with synchronous replication and accept the write-latency tradeoff
You've completed the in-tree → plugin Barman migration and have tested PITR at your actual data size

Before You Decide: Do This First

The public evidence gap is the biggest risk. Before any migration decision, run your own benchmark:

Stand up a 3-node CNPG cluster on EKS across 3 AZs. Drive it at your real write TPS. Measure pod-kill failover, node-kill failover (tuned node-monitor-grace-period + EBS reattach), and AZ-loss — under both async and sync replication. Compare against your Aurora/RDS SLOs.

If CNPG can't hit your RTO target under load, stop there. If it can, you have a data point nobody else seems to have published.

Bottom Line

CNPG is excellent software, backed by a serious community (CNCF Sandbox, Incubation applied for), and used in production at real enterprises. It is not vaporware.

But for a single high-traffic, multi-TB production database: the managed-service default holds. Aurora absorbs the 3am pages. CNPG hands them back to you — along with full control, which is either a feature or a burden depending entirely on your team's depth.

The clearest exception isn't a cost argument. It's CDC. If Debezium failing over without re-snapshotting is critical to your architecture, CNPG ≥1.27 on PG17 deserves a serious look.

I'm a Senior Staff DevOps/Platform Engineer working with Kubernetes, AWS, and Postgres at scale. What's your team's experience with CNPG in production? Drop it in the comments — there's a real gap in public production data here.

CloudNativePG #Kubernetes #PostgreSQL #DevOps #PlatformEngineering #AWS #EKS #Aurora #SRE #DatabaseEngineering

Overview of Crossplane and Crossplane-provider-aws

yelenary — Fri, 13 Jun 2025 19:46:03 +0000

Overview

Crossplane is an opensource project primarily developed and maintained by Upbound, a company focused on building tools for cloud-native infrastructure and application management. The project also has contributions from various individual developers and organizations within the cloud-native and Kubernetes community.

Essentially, it is a Kubernetes-native control plane that enables declarative infrastructure and application management via Custom Resource Definitions (CRDs). Its power comes from the provider system, which enables Crossplane to interface with external APIs (e.g., AWS, GCP, Azure, etc.).

This document describes how Crossplane and a Crossplane provider (e.g., provider-aws) are interconnected, their dependencies, and how data flows between components.

Dataflow diagram

Dataflow Steps Explained:

User → Kubernetes API

The user creates or applies a resource (like a CompositeResourceClaim or ManagedResource) using kubectl or a GitOps tool to the Kubernetes API server.

Kubernetes → Crossplane Core

Crossplane’s controllers (running in the cluster) watch for these resources, reconcile their state, and orchestrate the desired infrastructure.

Crossplane Core → Crossplane Provider

Crossplane core delegates concrete provisioning tasks to a Crossplane Provider (e.g., Provider-AWS) via custom resources (e.g., RDSInstance, S3Bucket).

Provider → AWS

The provider translates these Kubernetes custom resources into AWS API calls (using AWS SDK) to create/update/delete actual cloud resources.

AWS → Provider → Kubernetes

The provider monitors AWS for the resource’s real state and updates the status of the corresponding Kubernetes custom resource, closing the reconciliation loop.

Main components and roles

Crossplane:

*Key Components *

*1. Crossplane Core *

Responsible for:

Managing Provider, Configuration, and Composition resources.

Bootstrapping and orchestrating the lifecycle of provider controllers via Provider and ProviderRevision resources.

*2. Provider *

Custom resource of kind Provider (pkg.crossplane.io/v1).
Triggers installation and deployment of the provider controller.

Specifies the OCI image of a provider package (e.g., provider-aws:v0.50.0).

kubectl get provider.pkg 

NAME                  INSTALLED   HEALTHY   PACKAGE                                   AGE 

abc-provider   True        True      crossplane-contrib/provider-aws:v0.50.0   56d

apiVersion: pkg.crossplane.io/v1 
kind: Provider 
metadata: 
  annotations: 
    kustomize.toolkit.fluxcd.io/prune: disabled 
  labels: 
    app.kubernetes.io/instance: crossplane-provider 
    app.kubernetes.io/managed-by: Helm 
    app.kubernetes.io/name: crossplane-provider 
    helm.sh/chart: crossplane-provider-v0.10.0
    helm.toolkit.fluxcd.io/name: crossplane-provider 
  name: xxx-provider 
  namespace: crossplane-system 
spec: 
  package: crossplane-contrib/provider-aws:v0.50.0 
  runtimeConfigRef: 
    name: crossplane-provider-aws

*3. ProviderRevision *

Providerrevisions are typically not directly created or managed by users. Instead, theyare managed automatically when you install a Provider, upgrade Provider to a new version or Uninstall Provider

Created automatically by Crossplane when a Provider is applied.
Represents an exact, immutable snapshot of a provider version.
Crossplane manages only ACTIVE revision.

Responsible for:

Installing CRDs defined in the provider package.
Running the controller logic for reconciling managed resources.
Can be rolled back or rolled forward by changing the package version in the Provider.
Only one ProviderRevision can be ACTIVE at a time per Provider.
Old revisions are marked INACTIVE but retained for rollback if needed.
Updating the package field in a Provider creates a new ProviderRevision.
Deleting the Provider will uninstall the provider, delete the CRDs, and stop the controller

kubectl get providerrevision   
NAME                               HEALTHY   REVISION   IMAGE                                     STATE      DEP-FOUND   DEP-INSTALLED   AGE
aws-provider-6c3fd1826792   True      3          crossplane-contrib/provider-aws:v0.50.0   Active                                 16d
aws-provider-b4eafc5192c9   False     2          crossplanecontrib/provider-aws:v0.46.0    Inactive                               29d

*4. ProviderConfig *

Defines credentials and configuration needed by the provider to interact with external APIs.

Consumed by ManagedResources (e.g., RDSInstance, Bucket) via providerConfigRef.

Example:

apiVersion: aws.crossplane.io/v1beta1 
kind: ProviderConfig 
metadata: 
  name: aws-provider 
spec: 
  credentials: 
    source: Secret 
    secretRef: 
      namespace: crossplane-system 
      name: aws-creds 
      key: creds

*5. DeploymentRuntimeConfig *

An optional configuration resource that controls how provider controller pods (like crossplane-provider-aws) are deployed within the Kubernetes cluster.

It belongs to the pkg.crossplane.io API group and is used in conjunction with a Provider or Configuration resource to customize runtime settings such as:

resource limits (CPU/memory)
environment variables
volume mounts
labels and annotations
security contexts

*6. Managed Resources *

Represent real infrastructure (e.g., Bucket, RDSInstance, Network) modeled as Kubernetes CRs.

Managed by the provider controller.

Always reference a ProviderConfig.

Crossplane-provider-aws

After installing Crossplane, you install the AWS provider using a YAML manifest like:

apiVersion: pkg.crossplane.io/v1 
kind: Provider 
metadata: 
  name: provider-aws 
spec: 
  package: xpkg.upbound.io/crossplane-contrib/provider-aws:v0.50.0

This triggers Crossplane to:

Install the provider-aws OCI package

Crossplane fetches the image from Upbound’s OCI registry.

It spins up a new Deployment (in-cluster controller) for the provider.

kubectl get deployment -n crossplane-system 

NAME                               READY   UP-TO-DATE   AVAILABLE   AGE 

crossplane-provider-6c3fd1826792   1/1     1            1           16d 

crossplane                         1/1     1            1           618d 

crossplane-rbac-manager            1/1     1            1           618d

The provider’s controller pod installs:

Deployment (1 per provider): runs provider-aws controller manager
ProviderRevision object: tracks version and status
CustomResourceDefinitions (CRDs) for all supported AWS services E.g., s3buckets.aws.crossplane.io, rdsinstances.database.aws.crossplane.io, etc.

You can list all the CRDs installed with the provider by:

kubectl get crds | grep aws.crossplane.io

Dependency Flow

Crossplane Core is installed: installs built-in CRDs (e.g., Provider, Composition).
Provider resource is applied: triggers Crossplane to fetch the provider package from an OCI registry.
ProviderRevision is created: immutable snapshot of the provider package.
CRDs are installed by the revision (e.g., buckets.s3.aws.crossplane.io).
Controller Deployment is spun up: runs reconciliation logic for those CRDs.
User applies ProviderConfig: specifies cloud credentials.
User creates ManagedResource: such as Bucket, VPC, RDSInstance.
ManagedResource refers to ProviderConfig: to get credentials.
Provider controller reconciles the ManagedResource, calls external cloud API to provision/update/destroy real-world resources.

Step-by-Step: Configuring Internal YUM Repo Access on RPM-Based Hosts

yelenary — Thu, 05 Jun 2025 17:29:23 +0000

Background

In modern enterprise environments it's common to host private packages in an internal YUM repository. This guide walks through configuring a Rocky Linux 8/9 (or other RPM-based) host to securely access and install packages from a private YUM repository hosted in Google Artifact Registry.

✅ Prerequisites

A host running Rocky Linux 8 or 9
Access to a private YUM repo in Google Artifact Registry
A service account JSON key with the required permissions
yum or dnf installed

🛠 Step-by-Step Setup

🔹 Step 1: Import the GPG Key for Google Cloud RPM Packages

sudo rpm --import https://packages.cloud.google.com/yum/doc/rpm-package-key.gpg

🔹 Step 2: Add the Artifact Registry Plugin Repository

Create a new repo file at /etc/yum.repos.d/artifact-registry-plugin.repo:

sudo tee /etc/yum.repos.d/artifact-registry-plugin.repo <<EOF
[ar-plugin]
name=Artifact Registry Plugin
baseurl=https://packages.cloud.google.com/yum/repos/dnf-plugin-artifact-registry-el9-stable
enabled=1
gpgcheck=1
EOF

📝 This enables your host to install the required plugin for accessing Google Artifact Registry.

🔹 Step 3: Update YUM and Install the Artifact Registry Plugin

sudo yum makecache
sudo yum install dnf-plugin-artifact-registry

🔹 Step 4: Configure the Artifact Registry Plugin Credentials

Create or edit the file /etc/dnf/plugins/artifact-registry.conf:

[main]
enabled=1
service_account_json = "/path/to/your/service-account.json"

⚠️ Replace /path/to/your/service-account.json with the actual path to your GCP service account key file.

🔹 Step 5: Add Your Internal YUM Repository

Create the file /etc/yum.repos.d/yum-private.repo with the following content:
sudo tee /etc/yum.repos.d/yum-private.repo <<EOF
[yum-private]
name=Internal YUM Repo
baseurl=https://<region>-yum.pkg.dev/path/to/repo
enabled=1
repo_gpgcheck=0
gpgcheck=0
EOF

🔁 Replace with your Google Cloud region (e.g., europe-west3) and /path/to/repo with your actual repository path inside Artifact Registry.

🔹 Step 6: Refresh the Package Cache

sudo yum makecache

This ensures the repo metadata is up to date and packages are available to install.

✅ Conclusion:

You’ve now configured your RPM-based host to securely pull packages from a private YUM repository hosted in Google Artifact Registry. This is especially useful in production or enterprise environments where package access must be controlled, reproducible, and secure.

🧠 Bonus Tip

To automate this process, consider:

Writing an Ansible role or bash script
Integrating this configuration into your Packer build if you're baking AMIs or VM images
Managing the credentials via Vault or GCP Secret Manager for security