Installing Openshift Cluster on vSphere7

Contents

1. Pre-requisites
2. Generate Pull secret from Redhat
3. Creating openshift cluster
4. Fixing Internal Image registry
5. Deploy a sample nginx application

1. Pre-requisites

a) Connectivity to vCenter on port 443 from openshift network.

b) Connectivity to ESXi hosts on port 443 from openshift network.

c) Generate ssh keys (we can use the existing), this needs to be passed during cluster creation.

d) Working DHCP for openshift cluster nodes.

e) Two static ip's for API and Apps, for Step d.

f) DNS entry for "api.." and "*.apps.." .

In our case we mapped as below in our DNS,
api.openshift-test01.tanzu.local => 192.168.144.22
*.apps.openshift-test01.tanzu.local => 192.168.144.23

2. Generate Pull secret from Redhat

Lets get the Pull secret and also download the installer and client tools.

a) Register with https://console.redhat.com/openshift/ using your personal email or official email.

b) Once logged in, Click on Create Cluster.

c) Choose "Datacenter" and scroll down

d) Click on vSphere

e) Click on Automated installation

f) Download the Installer, Pull secret, and Command line tools.

3. Creating openshift cluster

We use a linux jumpserver which is in same network as openshift network to create the cluster so the installer can connect to API server to verify the installation without any dependencies on Firewall.

root@linux-vm-automation:~/openshift# ls -ltr
total 414864
-rw-r--r-- 1 root root      2783 May 30 17:30 pull-secret.txt
-rw-r--r-- 1 root root  59819571 May 30 17:31 openshift-client-linux.tar.gz
-rw-r--r-- 1 root root 364993703 May 30 17:31 openshift-install-linux.tar.gz
root@linux-vm-automation:~/openshift#
root@linux-vm-automation:~/openshift#
root@linux-vm-automation:~/openshift# tar -xvf openshift-install-linux.tar.gz
README.md
openshift-install
root@linux-vm-automation:~/openshift# ll
total 975252
drwxr-xr-x  2 root root       146 May 30 18:04 ./
drwx------ 22 root root      4096 May 30 18:02 ../
-rw-r--r--  1 root root  59819571 May 30 17:31 openshift-client-linux.tar.gz
-rwxr-xr-x  1 root root 573825024 May  9 18:10 openshift-install*
-rw-r--r--  1 root root 364993703 May 30 17:31 openshift-install-linux.tar.gz
-rw-r--r--  1 root root      2783 May 30 17:30 pull-secret.txt
-rw-r--r--  1 root root       706 May  9 18:10 README.md
root@linux-vm-automation:~/openshift#

Parameters we passed to the installer are below, so ensure you have the details ready.

a) ssh public key.
b) select vsphere as platform.
c) vcenter ip address.
d) vcenter username and password with required previleges.
e) datacenter.
f) datastore.
g) network.
h) VIP for API and Ingress.
i) Domain Name.
j) cluster name.
k) enter the pull secret that we copied from redhat console.

root@linux-vm-automation:~/openshift# ./openshift-install create cluster
? SSH Public Key /root/.ssh/id_rsa.pub
? Platform vsphere
? vCenter 172.17.22.118
? Username [e-mail]
? Password [? for help] *************
INFO Connecting to vCenter 172.17.22.118
INFO Defaulting to only available datacenter: vcenter-datacenter
? Cluster tenant-cluster
? Default Datastore SSD_Storage
? Network tenant43-ntw-72a59d1a-398e-4018-8dbd-5afa8ca60d40
? Virtual IP Address for API 192.168.144.22
? Virtual IP Address for Ingress 192.168.144.23
? Base Domain tanzu.local
? Cluster Name openshift-test01
? Pull Secret [? for help] ******************************************************************************************************************************************************************************************************************INFO Obtaining RHCOS image file from 'https://rhcos.mirror.openshift.com/art/storage/prod/streams/4.13-9.2/builds/413.92.202305021736-0/x86_64/rhcos-413.92.202305021736-0-vmware.x86_64.ova?sha256='
INFO The file was found in cache: /root/.cache/openshift-installer/image_cache/rhcos-413.92.202305021736-0-vmware.x86_64.ova. Reusing...
INFO Creating infrastructure resources...
INFO Waiting up to 20m0s (until 8:22AM) for the Kubernetes API at https://api.openshift-test01.tanzu.local:6443...
INFO API v1.26.3+b404935 up
INFO Waiting up to 30m0s (until 8:35AM) for bootstrapping to complete...
INFO Destroying the bootstrap resources...
INFO Waiting up to 40m0s (until 9:05AM) for the cluster at https://api.openshift-test01.tanzu.local:6443 to initialize...
INFO Checking to see if there is a route at openshift-console/console...
INFO Install complete!
INFO To access the cluster as the system:admin user when using 'oc', run 'export KUBECONFIG=/root/openshift/auth/kubeconfig'
INFO Access the OpenShift web-console here: https://console-openshift-console.apps.openshift-test01.tanzu.local
INFO Login to the console with user: "kubeadmin", and password: "c9T8a-ALwe9-ZU7D2-ENTDh"
INFO Time elapsed: 44m32s
root@linux-vm-automation:~/openshift#

So Cluster is created, lets login and verify.

Installer above provided the url and credentials to login

INFO To access the cluster as the system:admin user when using 'oc', run 'export KUBECONFIG=/root/openshift/auth/kubeconfig'
INFO Access the OpenShift web-console here: https://console-openshift-console.apps.openshift-test01.tanzu.local
INFO Login to the console with user: "kubeadmin", and password: "c9T8a-ALwe9-ZU7D2-ENTDh"
INFO Time elapsed: 44m32s

Login to redhat console if you see your cluster,

Access the cluster using oc or kubectl,

We already download oc tool "openshift-client-linux.tar.gz" from the redhat console, extract it and place it in /usr/local/bin/ or the location that you prefer.

export KUBECONFIG=/root/openshift/auth/kubeconfig

root@linux-vm-automation:~/openshift# oc get nodes
NAME                                    STATUS   ROLES                  AGE   VERSION
openshift-test01-pg8s9-master-0         Ready    control-plane,master   35m   v1.26.3+b404935
openshift-test01-pg8s9-master-1         Ready    control-plane,master   35m   v1.26.3+b404935
openshift-test01-pg8s9-master-2         Ready    control-plane,master   34m   v1.26.3+b404935
openshift-test01-pg8s9-worker-0-5c42f   Ready    worker                 14m   v1.26.3+b404935
openshift-test01-pg8s9-worker-0-djzl5   Ready    worker                 15m   v1.26.3+b404935
openshift-test01-pg8s9-worker-0-mtgzh   Ready    worker                 14m   v1.26.3+b404935
root@linux-vm-automation:~/openshift#

4. Fixing Internal Image registry

In vSphere environment, Openshift Internal Image registry won't be available since shareable stroage ReadWriteMany can't be created on vSphere storage.

If you try to create a pod with image pointing to internal image registry,

It will fail like below,

To Fix it, first create a PVC

root@linux-vm-automation:~/openshift# cat openshift-image-registry-pvc.yaml
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: image-registry-storage
  namespace: openshift-image-registry
spec:
  accessModes:
  - ReadWriteOnce
  resources:
    requests:
      storage: 100Gi
root@linux-vm-automation:~/openshift#

root@linux-vm-automation:~/openshift# oc create -f openshift-image-registry-pvc.yaml -n openshift-image-registry
persistentvolumeclaim/image-registry-storage created
root@linux-vm-automation:~/openshift#

Update the Registry CR spec with the following command.

oc edit configs.imageregistry.operator.openshift.io -n openshift-image-registry

Change spec.managementState from Removed to Managed.
Change spec.storage from {} to: claim: image-registry-storage

spec:
    managementState: Managed
storage:
      pvc:
        claim: image-registry-storage

After updating it should look like below,

spec:
    managementState: Managed
storage:
      pvc:
        claim: image-registry-storage

Once image registry pod is running fine, images from the internal image registry should be available.

The example which was not running earlier is running now,

5. Deploy a sample nginx application.

You should already be familiar on how to deploy a pod. Below we created a deployment using nginx image and created a service.

Lets create a route in openshift,

Note : This is not a Kubernetes object like Service.