DEV Community

Andy Tan
Andy Tan

Posted on

Deploy KubeSphere and Kubernetes on Amazon EC2 in All in One Mode

#aws #kubernetes #kubesphere #ec2

1 Cloud Instance Configuration

An All in One installation requires only one cloud host that meets the stated requirements.
Note: If the system has at least 8 CPU cores and 16 GB of memory, enabling all components is recommended. See the KubeSphere documentation on enabling pluggable components for details.
Operating system Configuration Security group Connection
CentOS Linux 7.9.2009 Core 8 CPU cores, 16 GB RAM, 100 GB disk Allow all traffic Remote SSH
Cloud instance requirements

  • The instance must be reachable through SSH.
  • The node must provide commands such as sudo, curl, openssl, and tar.

2 Connect to the Cloud Instance by SSH

BASH

C:\Users\xyb>ssh -i xybaws-ningxia-key.pem centos@<public-IP-address>
Enter fullscreen mode Exit fullscreen mode

3 Check the Operating System Version

BASH

[root@ip-172-31-6-245 ~]# cat /etc/redhat-release
CentOS Linux release 7.9.2009 (Core)
[root@ip-172-31-6-245 ~]# cat /etc/os-release
NAME="CentOS Linux"
VERSION="7 (Core)"
ID="centos"
ID_LIKE="rhel fedora"
VERSION_ID="7"
PRETTY_NAME="CentOS Linux 7 (Core)"
ANSI_COLOR="0;31"
CPE_NAME="cpe:/o:centos:centos:7"
HOME_URL="https://www.centos.org/"
BUG_REPORT_URL="https://bugs.centos.org/"

CENTOS_MANTISBT_PROJECT="CentOS-7"
CENTOS_MANTISBT_PROJECT_VERSION="7"
REDHAT_SUPPORT_PRODUCT="centos"
REDHAT_SUPPORT_PRODUCT_VERSION="7"
Enter fullscreen mode Exit fullscreen mode

4 Change the Hostname

BASH

$ sudo hostnamectl set-hostname kubesphere-master
$ sudo bash
$ sudo hostname
kubesphere-master
Enter fullscreen mode Exit fullscreen mode

5 Install Dependencies

BASH

$ sudo yum install -y socat conntrack ebtables ebtables ipset ipvsadm
Enter fullscreen mode Exit fullscreen mode

6 Configure the Security Group and DNS

  • Configure the cloud instance security group to allow all traffic and ports. This is suitable only for a test environment; in production, expose only the minimum required ports. If the network uses firewall rules or security groups, ensure that infrastructure components can communicate through the required ports. The original article recommends disabling the firewall for this test setup.
  • Ensure the DNS servers in /etc/resolv.conf are reachable; otherwise, cluster DNS may fail. BASH 
$ sudo vim /etc/resolv.conf
$ sudo cat /etc/resolv.conf
; generated by /usr/sbin/dhclient-script
search cn-northwest-1.compute.internal
nameserver 8.8.8.8
Enter fullscreen mode Exit fullscreen mode

7 Download KubeKey

The cluster must have an available container runtime. When KubeKey builds the cluster, it installs the latest Docker version by default. You can instead install Docker or another container runtime before creating the cluster.
KubeKey is a Go based installation tool that replaces the earlier Ansible based installer. It can install KubeSphere, Kubernetes, or both, providing a flexible and efficient setup path.
BASH

# If access to GitHub or Google APIs is restricted, set the correct download region first.
export KKZONE=cn

# Then run the following command to download KubeKey.
curl -sfL https://get-kk.kubesphere.io | VERSION=v3.0.13 sh -

# Finally, make the downloaded kk binary executable.
chmod +x kk

# Verify the download
# ll
total 112100
-rwxr-xr-x  1 root root 78973328 Nov  7 08:43 kk
drwxr-xr-x  3 root root       18 Nov 19 05:21 kubekey
-rw-r--r--  1 root root 35796470 Nov 19 05:17 kubekey-v3.0.13-linux-amd64.tar.gz
Enter fullscreen mode Exit fullscreen mode

8 Install KubeSphere and Kubernetes Together

Installation requires only one command. The command template is:
BASH

./kk create cluster [--with-kubernetes version] [--with-kubesphere version]
Enter fullscreen mode Exit fullscreen mode

To install Kubernetes and KubeSphere together, use commands such as the following:
BASH

./kk create cluster --with-kubernetes v1.26.12 --with-kubesphere v3.4.0

./kk create cluster --with-kubernetes v1.26.12 --with-kubesphere v3.4.0 --container-manager containerd -y
Enter fullscreen mode Exit fullscreen mode

After the command runs, KubeKey checks the installation environment and displays the results in a table. Enter yes to continue.
BASH

[root@kubesphere-master ~]# ./kk create cluster --with-kubernetes v1.22.12 --with-kubesphere v3.4.0


 _   __      _          _   __
| | / /     | |        | | / /
| |/ / _   _| |__   ___| |/ /  ___ _   _
|    \| | | | '_ \ / _ \    \ / _ \ | | |
| |\  \ |_| | |_) |  __/ |\  \  __/ |_| |
\_| \_/\__,_|_.__/ \___\_| \_/\___|\__, |
                                    __/ |
                                   |___/

05:28:23 UTC [GreetingsModule] Greetings
05:28:27 UTC message: [kubesphere-master]
Greetings, KubeKey!
05:28:27 UTC success: [kubesphere-master]
05:28:27 UTC [NodePreCheckModule] A pre-check on nodes
05:28:43 UTC success: [kubesphere-master]
05:28:43 UTC [ConfirmModule] Display confirmation form
+-------------------+------+------+---------+----------+-------+-------+---------+-----------+--------+--------+------------+------------+-------------+------------------+--------------+
| name              | sudo | curl | openssl | ebtables | socat | ipset | ipvsadm | conntrack | chrony | docker | containerd | nfs client | ceph client | glusterfs client | time         |
+-------------------+------+------+---------+----------+-------+-------+---------+-----------+--------+--------+------------+------------+-------------+------------------+--------------+
| kubesphere-master | y    | y    | y       | y        | y     | y     | y       | y         | y      |        |            | y          |             |                  | UTC 05:28:43 |
+-------------------+------+------+---------+----------+-------+-------+---------+-----------+--------+--------+------------+------------+-------------+------------------+--------------+

This is a simple check of your environment.
Before installation, ensure that your machines meet all requirements specified at
https://github.com/kubesphere/kubekey#requirements-and-recommendations

Continue this installation? [yes/no]: yes
Enter fullscreen mode Exit fullscreen mode


During installation, the following output indicates that the Kubernetes cluster has been installed successfully.
BASH

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

Alternatively, if you are the root user, you can run:

  export KUBECONFIG=/etc/kubernetes/admin.conf

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

You can now join any number of control-plane nodes by copying certificate authorities
and service account keys on each node and then running the following as root:

  kubeadm join lb.kubesphere.local:6443 --token qpnirw.gyfdxnceyth942i9 \
        --discovery-token-ca-cert-hash sha256:8f0739bdfa2fb5a6118b5b6d0e10985641aa7c903fd749e073caf718b350c282 \
        --control-plane

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join lb.kubesphere.local:6443 --token qpnirw.gyfdxnceyth942i9 \
        --discovery-token-ca-cert-hash sha256:8f0739bdfa2fb5a6118b5b6d0e10985641aa7c903fd749e073caf718b350c282
Enter fullscreen mode Exit fullscreen mode

The final output shown below indicates that the KubeSphere cluster has also been installed successfully.

Optional Install Kubernetes Separately

You may also omit the KubeSphere and Kubernetes version numbers.
BASH

./kk create cluster --with-kubernetes
Enter fullscreen mode Exit fullscreen mode

BASH

[root@kubesphere-master ~]# ./kk create cluster --with-kubernetes --with-kubesphere


 _   __      _          _   __           
| | / /     | |        | | / /           
| |/ / _   _| |__   ___| |/ /  ___ _   _ 
|    \| | | | '_ \ / _ \    \ / _ \ | | |
| |\  \ |_| | |_) |  __/ |\  \  __/ |_| |
\_| \_/\__,_|_.__/ \___\_| \_/\___|\__, |
                                    __/ |
                                   |___/

05:30:47 UTC [GreetingsModule] Greetings
05:30:47 UTC message: [kubesphere-master]
Greetings, KubeKey!
05:30:47 UTC success: [kubesphere-master]
05:30:47 UTC [NodePreCheckModule] A pre-check on nodes
05:30:48 UTC success: [kubesphere-master]
05:30:48 UTC [ConfirmModule] Display confirmation form
+-------------------+------+------+---------+----------+-------+-------+---------+-----------+--------+--------+------------+------------+-------------+------------------+--------------+
| name              | sudo | curl | openssl | ebtables | socat | ipset | ipvsadm | conntrack | chrony | docker | containerd | nfs client | ceph client | glusterfs client | time         |
+-------------------+------+------+---------+----------+-------+-------+---------+-----------+--------+--------+------------+------------+-------------+------------------+--------------+
| kubesphere-master | y    | y    | y       | y        | y     | y     | y       | y         | y      |        |            | y          |             |                  | UTC 05:30:48 |
+-------------------+------+------+---------+----------+-------+-------+---------+-----------+--------+--------+------------+------------+-------------+------------------+--------------+

This is a simple check of your environment.
Before installation, ensure that your machines meet all requirements specified at
https://github.com/kubesphere/kubekey#requirements-and-recommendations

Continue this installation? [yes/no]: yes
Enter fullscreen mode Exit fullscreen mode


The Kubernetes cluster is now installed. Use the following commands to check its status.
BASH

[root@kubesphere-master ~]# kubectl get nodes
NAME                STATUS   ROLES                         AGE     VERSION
kubesphere-master   Ready    control-plane,master,worker   2m14s   v1.23.10
[root@kubesphere-master ~]# kubectl get pod -A
NAMESPACE     NAME                                        READY   STATUS    RESTARTS   AGE
kube-system   calico-kube-controllers-74dbdc644f-rsn8n    1/1     Running   0          2m6s
kube-system   calico-node-p27nq                           1/1     Running   0          2m7s
kube-system   coredns-b7c47bcdc-45mzd                     1/1     Running   0          2m6s
kube-system   coredns-b7c47bcdc-5n5mc                     1/1     Running   0          2m6s
kube-system   kube-apiserver-kubesphere-master            1/1     Running   0          2m17s
kube-system   kube-controller-manager-kubesphere-master   1/1     Running   0          2m17s
kube-system   kube-proxy-xqsl4                            1/1     Running   0          2m7s
kube-system   kube-scheduler-kubesphere-master            1/1     Running   0          2m17s
kube-system   nodelocaldns-m6j69                          1/1     Running   0          2m7s
Enter fullscreen mode Exit fullscreen mode

Optional Install KubeSphere Separately

BASH

[root@kubesphere-master ~]# ./kk create cluster --with-kubesphere


 _   __      _          _   __           
| | / /     | |        | | / /           
| |/ / _   _| |__   ___| |/ /  ___ _   _ 
|    \| | | | '_ \ / _ \    \ / _ \ | | |
| |\  \ |_| | |_) |  __/ |\  \  __/ |_| |
\_| \_/\__,_|_.__/ \___\_| \_/\___|\__, |
                                    __/ |
                                   |___/

05:36:52 UTC [GreetingsModule] Greetings
05:36:53 UTC message: [kubesphere-master]
Greetings, KubeKey!
05:36:53 UTC success: [kubesphere-master]
05:36:53 UTC [NodePreCheckModule] A pre-check on nodes
05:36:53 UTC success: [kubesphere-master]
05:36:53 UTC [ConfirmModule] Display confirmation form
+-------------------+------+------+---------+----------+-------+-------+---------+-----------+--------+--------+------------+------------+-------------+------------------+--------------+
| name              | sudo | curl | openssl | ebtables | socat | ipset | ipvsadm | conntrack | chrony | docker | containerd | nfs client | ceph client | glusterfs client | time         |
+-------------------+------+------+---------+----------+-------+-------+---------+-----------+--------+--------+------------+------------+-------------+------------------+--------------+
| kubesphere-master | y    | y    | y       | y        | y     | y     | y       | y         | y      | 24.0.6 | v1.7.3     | y          |             |                  | UTC 05:36:53 |
+-------------------+------+------+---------+----------+-------+-------+---------+-----------+--------+--------+------------+------------+-------------+------------------+--------------+

This is a simple check of your environment.
Before installation, ensure that your machines meet all requirements specified at
https://github.com/kubesphere/kubekey#requirements-and-recommendations

Continue this installation? [yes/no]: yes
Enter fullscreen mode Exit fullscreen mode


BASH

#####################################################
###              Welcome to KubeSphere!           ###
#####################################################

Console: http://172.31.38.43:30880
Account: admin
Password: P@88w0rd
NOTES：
  1. After you log into the console, please check the
     monitoring status of service components in
     "Cluster Management". If any service is not
     ready, please wait patiently until all components 
     are up and running.
  2. Please change the default password after login.

#####################################################
https://kubesphere.io             2023-11-19 05:43:59
#####################################################
05:44:00 UTC success: [kubesphere-master]
05:44:00 UTC Pipeline[CreateClusterPipeline] execute successfully
Installation is complete.

Please check the result using the command:

        kubectl logs -n kubesphere-system $(kubectl get pod -n kubesphere-system -l 'app in (ks-install, ks-installer)' -o jsonpath='{.items[0].metadata.name}') -f
Enter fullscreen mode Exit fullscreen mode


BASH

[root@kubesphere-master ~]# kubectl get nodes -A
NAME                STATUS   ROLES                         AGE   VERSION
kubesphere-master   Ready    control-plane,master,worker   13m   v1.23.10
[root@kubesphere-master ~]# kubectl get pods -A
NAMESPACE                      NAME                                               READY   STATUS    RESTARTS   AGE
kube-system                    calico-kube-controllers-74dbdc644f-rsn8n           1/1     Running   0          13m
kube-system                    calico-node-p27nq                                  1/1     Running   0          13m
kube-system                    coredns-b7c47bcdc-45mzd                            1/1     Running   0          13m
kube-system                    coredns-b7c47bcdc-5n5mc                            1/1     Running   0          13m
kube-system                    kube-apiserver-kubesphere-master                   1/1     Running   0          13m
kube-system                    kube-controller-manager-kubesphere-master          1/1     Running   0          13m
kube-system                    kube-proxy-xqsl4                                   1/1     Running   0          13m
kube-system                    kube-scheduler-kubesphere-master                   1/1     Running   0          13m
kube-system                    nodelocaldns-m6j69                                 1/1     Running   0          13m
kube-system                    openebs-localpv-provisioner-69cbb769b8-cz2jf       1/1     Running   0          9m13s
kube-system                    snapshot-controller-0                              1/1     Running   0          8m28s
kubesphere-controls-system     default-http-backend-696d6bf54f-9rhx7              1/1     Running   0          7m33s
kubesphere-controls-system     kubectl-admin-b49cf5585-g59ww                      1/1     Running   0          3m9s
kubesphere-monitoring-system   alertmanager-main-0                                2/2     Running   0          5m53s
kubesphere-monitoring-system   kube-state-metrics-774ff789cf-pj4r6                3/3     Running   0          6m23s
kubesphere-monitoring-system   node-exporter-n6dmw                                2/2     Running   0          6m25s
kubesphere-monitoring-system   notification-manager-deployment-78ff4d6789-8qpgp   2/2     Running   0          5m16s
kubesphere-monitoring-system   notification-manager-operator-85d67fdc46-khmwv     2/2     Running   0          5m31s
kubesphere-monitoring-system   prometheus-k8s-0                                   2/2     Running   0          6m18s
kubesphere-monitoring-system   prometheus-operator-57c78bd7fb-dq7r7               2/2     Running   0          6m27s
kubesphere-system              ks-apiserver-76bf8bb556-hjq47                      1/1     Running   0          7m33s
kubesphere-system              ks-console-768dbcdf9c-bccbd                        1/1     Running   0          7m33s
kubesphere-system              ks-controller-manager-866bf75746-tfxd2             1/1     Running   0          7m33s
kubesphere-system              ks-installer-d949f8fcc-zh6pr                       1/1     Running   0          9m12s
Enter fullscreen mode Exit fullscreen mode

9 Verify the KubeSphere Installation

Run the following command to check the installation result.
BASH

kubectl logs -n kubesphere-system $(kubectl get pod -n kubesphere-system -l 'app in (ks-install, ks-installer)' -o jsonpath='{.items[0].metadata.name}') -f
Enter fullscreen mode Exit fullscreen mode

The output displays the web console IP address and port. The default NodePort is 30880. You can access the console at :30880 with the default credentials admin and P@88w0rd.

10 Sign In to the KubeSphere Console

Console: http://:30880
Account: admin
Password: P@88w0rd

At the first sign in, you are prompted to set a new password.

Open the KubeSphere console.


After signing in, review the status of each component under System Components. You can also run kubectl get pod --all-namespaces to check the health of KubeSphere components.

Optional Install the KubeSphere App Store

Graphical Method

  1. Sign in as admin. Open Platform Management in the upper left, then select Cluster Management.
  2. Select Custom Resource Definitions, search for clusterconfiguration, and open the matching result.
  3. Under Custom Resources, open the menu beside ks-installer and choose Edit YAML.
  4. Find openpitrix and change enabled from false to true. Select OK to save the configuration. YAML 
openpitrix:
  store:
    enabled: true # Change "false" to "true".
Enter fullscreen mode Exit fullscreen mode

Run the following command with kubectl to monitor installation progress.
BASH

kubectl logs -n kubesphere-system $(kubectl get pod -n kubesphere-system -l 'app in (ks-install, ks-installer)' -o jsonpath='{.items[0].metadata.name}') -f
Enter fullscreen mode Exit fullscreen mode

Command Line Method

BASH

[root@kubesphere-master ~]# kubectl edit cc ks-installer -n kubesphere-system
clusterconfiguration.installer.kubesphere.io/ks-installer edited
Enter fullscreen mode Exit fullscreen mode

Verify the Result
Open http://:30880/apps

11 Troubleshooting

Problem: required dependencies are missing.
BASH

[root@kubesphere-master ~]# ./kk create cluster --with-kubernetes v1.25.4 --with-kubesphere v3.4.0


 _   __      _          _   __           
| | / /     | |        | | / /           
| |/ / _   _| |__   ___| |/ /  ___ _   _ 
|    \| | | | '_ \ / _ \    \ / _ \ | | |
| |\  \ |_| | |_) |  __/ |\  \  __/ |_| |
\_| \_/\__,_|_.__/ \___\_| \_/\___|\__, |
                                    __/ |
                                   |___/

05:25:34 UTC [GreetingsModule] Greetings
05:25:35 UTC message: [kubesphere-master]
Greetings, KubeKey!
05:25:35 UTC success: [kubesphere-master]
05:25:35 UTC [NodePreCheckModule] A pre-check on nodes
05:25:35 UTC success: [kubesphere-master]
05:25:35 UTC [ConfirmModule] Display confirmation form
+-------------------+------+------+---------+----------+-------+-------+---------+-----------+--------+--------+------------+------------+-------------+------------------+--------------+
| name              | sudo | curl | openssl | ebtables | socat | ipset | ipvsadm | conntrack | chrony | docker | containerd | nfs client | ceph client | glusterfs client | time         |
+-------------------+------+------+---------+----------+-------+-------+---------+-----------+--------+--------+------------+------------+-------------+------------------+--------------+
| kubesphere-master | y    | y    | y       |          | y     |       |         | y         | y      |        |            | y          |             |                  | UTC 05:25:35 |
+-------------------+------+------+---------+----------+-------+-------+---------+-----------+--------+--------+------------+------------+-------------+------------------+--------------+

This is a simple check of your environment.
Before installation, ensure that your machines meet all requirements specified at
https://github.com/kubesphere/kubekey#requirements-and-recommendations

[Notice]
Incorrect runtime. Please specify a container runtime other than Docker to install Kubernetes v1.24 or later.
You can set "spec.kubernetes.containerManager" in the configuration file to "containerd" or add "--container-manager containerd" to the "./kk create cluster" command.
For more information, see:
https://github.com/kubesphere/kubekey/blob/master/docs/commands/kk-create-cluster.md
https://kubernetes.io/docs/setup/production-environment/container-runtimes/#container-runtimes
https://kubernetes.io/blog/2022/02/17/dockershim-faq/
Enter fullscreen mode Exit fullscreen mode

Solution: install the missing dependencies.
BASH

yum install -y ebtables ipset ipvsadm
Enter fullscreen mode Exit fullscreen mode

12 References

Install KubeSphere on Linux in All in One mode
KubeSphere App Store

Top comments (0)