Building your first ROSA🌹 with Red Hat and AWS

“When life throws thorns, hunt for roses.” – Anonymous

When market trends, pessimistic forecasts, and Global economics throw companies and us developers thorns (on many levels), hunt for a Rosa (the Spanish word for 'Rose'). In this ever-changing market, looking for the most suitable solution can always make a difference between a successful business and a "what if we have done this differently". In this blog entry, I introduce a solution that combines the best of both worlds, a top-notch Cloud Services provider (and market leader) and the most complete Container Management platform offered by Red Hat, Red Hat OpenShift enterprise Kubernetes platform on AWS.

Table of Contents

1. Definition
2. What Makes Openshift Special
3. Architecture
4. Pre-requisites to create ROSA
5. ROSA Cluster Implementation
6. Delete ROSA Cluster

1. Definition

ROSA (Red Hat Openshift on AWS) is an example of a Platform service offered by Red Hat, as a sub-group of Red Hat Cloud Services.

Why is it important? it helps companies to spend more time building and deploying applications and less time managing infrastructure.

2. What makes Openshift special

To answer this, we should benchmark Red Hat Openshift to existing Cloud solutions. In this first blog entry, I will discuss about existing AWS solutions for Container Management such as AWS EKS (In my opinion, the closest AWS service that resembles ROSA):

Peerspot did a great job comparing both Products (Check detailed report here I will summarize some details I found relevant for this Blog entry.

Features

EKS:

• Scalability (Horizontal & Vertical), Observability and Performance.
• It helps to manage nodes and scalability in AWS.
• Blue-Green deployment strategy becomes easy.
• Faster solution to adopt on native applications.
• Embedded Cost Management tools.
• Federal Risk and Authorization Management Program (FedRamp) compliant.

Red Hat Openshift Container Platform:

• Rich stack in the software supply chain.
• Architecturally, it is the best solution for container-based applications.
• Dashboards provide excellent granular visibility of your cluster and pods.
• Fully automated upgrades, including Cluster life-cycle management.
• GitOps functionality allows developers to start working on applications right away.
• Red Hat SRE support enhances overall Customer service to improve cluster's availability.

Pricing and ROI

EKS:

• Eliminates data-security concerns.
• Pricing depends exclusively on the specific requirements.
• Functionalities can compensate sometimes the high price of the solution.

Red Hat Openshift Container Platform:

• Reduction in infrastructure and cluster management operational costs.
• Offers a centralized solution which also offers security.

Room for Improvement

EKS:

• Logging features need some improvement.
• Assign permissions to users still a tough task.
• EKS Security related documentation is hard to understand.

Red Hat Openshift Container Platform:

• Pretty steep learning curve.
• OpenShift Licenses are pretty expensive.
• Microservices Deployment can take over 10 minutes.
• Restricted support hours for lower-tier subscriptions.
• GitOps operator provided by Red Hat are behind latest trends.

Scalability & Performance

EKS:

• Granular control over your Kubernetes clusters.
• As only control plane is covered by Amazon, performance issues on the data plane side are hard to troubleshoot.

Red Hat Openshift Container Platform:

• Granular control over your Kubernetes clusters + Red Hat SRE monitoring.
• Performance and License related issues are hard to handle due to a non-centralized Ticket system.

Deployment and customer support

EKS:

• Initial setup is relatively easy, which a dependency on AWS account to start procedure.
• Terraform scripts or AWS CDK constructs.

Red Hat Openshift Container Platform:

• On the bare-metal side, it takes longer to install OpenShift because they are all physical nodes.
• The deployment involves steps like installation, configuration, and deploying common services on-premises.

When deciding which tool to use, I will consider factors such as project requirements, technical expertise, budget constraints, and long-term strategic objectives. Different industries have unique needs and paces, and Red Hat is certainly capitalizing on this diversity.

3. Architecture

The architecture of ROSA consists of several key components:

1. Control Plane: Managed by Red Hat, it includes the OpenShift API server, controller manager, scheduler, etcd, and other core services.
2. Worker Nodes: Deployed in your AWS account, running your containerized applications (Compute and Storage Volumes).
3. Infrastructure Nodes: Nodes where OpenShift components such as the ingress controller, image registry, and monitoring are deployed.
4. Networking: Utilizes AWS VPCs, subnets, security groups, and other networking services to manage communication.
5. Storage: Integrates with AWS storage services like EBS and S3 for persistent and object storage.
6. Identity and Access Management: Uses AWS IAM for permissions and OpenShift RBAC for fine-grained access control.

Red Hat OpenShift Service on AWS (ROSA) offers two cluster topologies:

1. Hosted Control Plane (HCP): In this topology, the control plane is managed and hosted in a Red Hat account, while the worker nodes are deployed within the customer's AWS account.

2. Classic: Both the control plane and the worker nodes are deployed within the customer's AWS account.

In the below chapters, I will explain Classic Architecture, leaving the HCP topology for future discussions. Find below AWS topology for Classic architecture. Reference: ROSA Architecture

4. Pre-requisites to create ROSA

Before you can create your first ROSA cluster, ensure the following:

• AWS Account and IAM User:

  • You need an AWS account with an IAM user.
  • Since you subscribe to ROSA through the AWS Marketplace, your IAM user must have AWS Marketplace permissions. If you lack these permissions, contact your AWS account administrator to grant you access.
  • For more details on troubleshooting ROSA enablement errors, review the documentation in the reference section.

• AWS Service Quotas:

  • Your AWS account must have sufficient AWS service quotas to create ROSA clusters.
  • Use the rosa command to verify these quotas.
  • Review the documentation in the reference section for a list of required quotas.

• Red Hat Account:

  • You need a Red Hat account to access the Hybrid Cloud Console.
  • The cluster creation process links your Red Hat account with your AWS account, allowing you to manage your ROSA clusters from the OpenShift Cluster Manager web interface.

How to Add OpenShift to Your AWS Account

Subscribing to ROSA through the AWS Marketplace is straightforward. Follow these steps to enable ROSA in your AWS account:

1. Log in to the AWS Management Console:

   • Visit AWS Management Console.

2. Navigate to the ROSA Service:

   • Go to Services > Containers > Red Hat OpenShift Service on AWS.

3. Get Started with ROSA:

   • Click Get started to reach the Verify ROSA prerequisites page.

4. Check Your Subscription Status:

   • If you see the "You previously enabled ROSA" checkmark, you are already subscribed.

5. Enable ROSA (if not already subscribed):

   • Select I agree to share my contact information with Red Hat.
   • Click Enable ROSA.

After following these steps, this should be the final result:

Install and Configure CLI

• Install the aws command on your system. The tool is available at https://aws.amazon.com/cli/.

• Run the aws configure command to provide your IAM user credentials and to select your AWS Region.
  

$ aws configure
AWS Access Key ID [None]: [Key]
AWS Secret Access Key [None]: [Secret]
Default region name [None]: us-east-1
Default output format [None]: <Enter>

• Download and install the ROSA CLI from Red Hat OpenShift Downloads.

• Execute the rosa login command to log in to your Red Hat account. This command will prompt you to generate an access token.
  

$ rosa login
To login to your Red Hat account, get an offline access token at https://console.redhat.com/openshift/token/rosa
? Copy the token and paste it here:

5. ROSA cluster implementation

The following steps will explain how to install a ROSA cluster using CLI. UI implementation is also available, but not discussed in this blog.

Create Account Roles

To create ROSA clusters, you must first set up specific IAM roles and policies in your AWS account. These roles grant the necessary permissions for the ROSA cluster creation process to create AWS resources, such as EC2 instances.

Steps:

1. Log in to your AWS and Red Hat accounts using aws configure and rosa login commands.
2. Run rosa create account-roles to create the IAM resources.
   • Use --mode auto to automate role and policy creation via the AWS API.
   • Add --yes to skip confirmation prompts.

Example:

$ rosa create account-roles --mode auto --yes
...output omitted...
I: Creating account roles
I: Creating roles using 'arn:aws:iam::...:user/[e-mail]-fqppg-admin'
I: Created role 'ManagedOpenShift-Installer-Role' ...
I: Created role 'ManagedOpenShift-ControlPlane-Role' ...
I: Created role 'ManagedOpenShift-Worker-Role' ...
I: Created role 'ManagedOpenShift-Support-Role' ...
I: To create a cluster with these roles, run the following command:
rosa create cluster --sts

Create a ROSA Cluster

Once your cloud environment is prepared, you can create a ROSA cluster.

To do this, open a command-line terminal and run rosa create cluster. This command starts the cluster creation process and exits immediately, allowing the installation to proceed unattended on AWS.

By default, rosa create cluster runs in interactive mode. You only need to specify the cluster name and can accept the default values suggested for other parameters.

$ rosa create cluster
I: Enabling interactive mode
? Cluster name: openshiftmarco
? Deploy cluster using AWS STS: Yes
W: In a future release STS will be the default mode.
W: --sts flag won't be necessary if you wish to use STS.
W: --non-sts/--mint-mode flag will be necessary if you do not wish to use STS.
? OpenShift version: 4.12.14
I: Using arn:...:role/ManagedOpenShift-Installer-Role for the Installer role
I: Using arn:...:role/ManagedOpenShift-ControlPlane-Role for the ControlPlane role
I: Using arn:...:role/ManagedOpenShift-Worker-Role for the Worker role
I: Using arn:...:role/ManagedOpenShift-Support-Role for the Support role
? External ID (optional): <Enter>
? Operator roles prefix: openshiftmarco-p5k3  1
? Multiple availability zones (optional): No
? AWS region: us-east-1
? PrivateLink cluster (optional): No
...output omitted...
I: Creating cluster 'openshiftmarco'
I: To create this cluster again in the future, you can run:  2
   rosa create cluster --cluster-name openshiftmarco --sts --role-arn arn:aws:iam::452954386616:role/ManagedOpenShift-Installer-Role --support-role-arn arn:aws:iam::452954386616:role/ManagedOpenShift-Support-Role --controlplane-iam-role arn:aws:iam::452954386616:role/ManagedOpenShift-ControlPlane-Role --worker-iam-role arn:aws:iam::452954386616:role/ManagedOpenShift-Worker-Role --operator-roles-prefix openshiftmarco-p5k3 --region us-east-1 --version 4.12.14 --compute-nodes 2 --compute-machine-type m5.xlarge --machine-cidr 10.0.0.0/16 --service-cidr 172.30.0.0/16 --pod-cidr 10.128.0.0/14 --host-prefix 23
I: To view a list of clusters and their status, run 'rosa list clusters'
I: Cluster 'openshiftmarco' has been created.
I: Once the cluster is installed you will need to add an Identity Provider before you can login into the cluster. See 'rosa create idp --help' for more information.
...output omitted...
I: Run the following commands to continue the cluster creation:  3

    rosa create operator-roles --cluster openshiftmarco
    rosa create oidc-provider --cluster openshiftmarco

I: To determine when your cluster is Ready, run 'rosa describe cluster -c openshiftmarco'.
I: To watch your cluster installation logs, run 'rosa logs install -c openshiftmarco --watch'.

A simplified, and more direct way to deploy a specific Red Hat Openshift cluster, defining above items + EC2 size will be:

rosa create cluster --cluster-name openshiftmarco --region us-east-1 --multi-az=false --compute-machine-type m5.2xlarge --replicas 2 --sts --mode auto

Monitor ROSA Cluster Creation Process

The rosa describe cluster --cluster [cluster_name] will show the deployment status.

$ rosa describe cluster --cluster mycluster
...output omitted...
State:      installing
...output omitted...

$ rosa describe cluster --cluster mycluster
...output omitted...
State:      ready
...output omitted...

Describe ROSA Cluster

Use the rosa describe cluster -c [cluster_name] to describe the cluster information.

>rosa describe cluster -c openshiftmarco
WARN: The current version (1.2.39) is not up to date with latest released version (1.2.40).
WARN: It is recommended that you update to the latest version.

Name:                       openshiftmarco
Domain Prefix:              openshiftmarco
Display Name:               openshiftmarco
ID:                         2bqn7jb8ts39iushkqantla77o3ic1sl
External ID:
Control Plane:              Customer Hosted
OpenShift Version:
Channel Group:              stable
DNS:                        Not ready
AWS Account:                615956341945
API URL:
Console URL:
Region:                     us-east-1
Multi-AZ:                   false

Nodes:
 - Control plane:           3
 - Infra:                   2
 - Compute:                 2
Network:
 - Type:                    OVNKubernetes
 - Service CIDR:            172.30.0.0/16
 - Machine CIDR:            10.0.0.0/16
 - Pod CIDR:                10.128.0.0/14
 - Host Prefix:             /23
EC2 Metadata Http Tokens:   optional
Role (STS) ARN:             arn:aws:iam::615956341945:role/ManagedOpenShift-Installer-Role
Support Role ARN:           arn:aws:iam::615956341945:role/ManagedOpenShift-Support-Role
Instance IAM Roles:
 - Control plane:           arn:aws:iam::615956341945:role/ManagedOpenShift-ControlPlane-Role
 - Worker:                  arn:aws:iam::615956341945:role/ManagedOpenShift-Worker-Role
Operator IAM Roles:
 - arn:aws:iam::615956341945:role/openshiftmarco-t2j5-openshift-cloud-network-config-controller-cl
 - arn:aws:iam::615956341945:role/openshiftmarco-t2j5-openshift-machine-api-aws-cloud-credentials
 - arn:aws:iam::615956341945:role/openshiftmarco-t2j5-openshift-cloud-credential-operator-cloud-cr
 - arn:aws:iam::615956341945:role/openshiftmarco-t2j5-openshift-image-registry-installer-cloud-cre
 - arn:aws:iam::615956341945:role/openshiftmarco-t2j5-openshift-ingress-operator-cloud-credentials
 - arn:aws:iam::615956341945:role/openshiftmarco-t2j5-openshift-cluster-csi-drivers-ebs-cloud-cred
Managed Policies:           No
State:                      waiting (OIDC Provider not found: operation error STS: AssumeRoleWithWebIdentity, https response error StatusCode: 400, RequestID: 0956a1b9-92dd-4270-b654-4143dc650624, InvalidIdentityToken: No OpenIDConnect provider found in your account for https://oidc.op1.openshiftapps.com/2bqn7jb8ts39iushkqantla77o3ic1sl)
Private:                    No
Delete Protection:          Disabled
Created:                    Jun 11 2024 03:17:42 UTC
User Workload Monitoring:   Enabled
Details Page:               https://[URL]
OIDC Endpoint URL:          https://[URL] (Classic)

>rosa create admin --cluster=openshiftmarco
WARN: The current version (1.2.39) is not up to date with latest released version (1.2.40).
WARN: It is recommended that you update to the latest version.
INFO: Admin account has been added to cluster 'openshiftmarco'.
INFO: Please securely store this generated password. If you lose this password you can delete and recreate the cluster admin user.
INFO: To login, run the following command:

   oc login https://api.openshiftmarco.b3b3.p1.openshiftapps.com:6443 --username cluster-admin --password 3HgZ3-wN495-RLc3v-7sLaU

INFO: It may take several minutes for this access to become active.

There you go! You have your brand-new Red Hat Openshift cluster available.

Let's check the AWS resources being created:

AWS EC2

AWS Route53

AWS Load Balancer

AWS EIP

6. Delete ROSA Cluster

Deleting ROSA cluster is even easier than creating one. Follow this simple steps:

1) Login Red Hat Hybrid Console and select your cluster. Then select the option "delete cluster"

2) Confirm the delete request by entering the cluster name

3) Confirm Resources are deleted from Red Hat Openshift Console