Accessing Grafana via SSH Tunneling

Kiran Mova — Sat, 09 May 2020 13:31:17 +0000

Here is a quick guide "how-to" for configuring Prometheus and Grafana on Kubernetes Cluster that is behind firewalls and accessing the Grafana UI from a remote machine (your laptop at home) using SSH tunneling.

Setup Helm 3

You can use Helm 2 as well or skip this step if you already have helm installed.

Step 1: Install Helm 3

curl -fsSL -o get_helm.sh https://raw.githubusercontent.com/helm/helm/master/scripts/get-helm-3
chmod 700 get_helm.sh
./get_helm.sh

Step 2: Add Stable Charts

helm repo add stable https://kubernetes-charts.storage.googleapis.com/

Setup Prometheus and Grafana using Prometheus Operator

This is the easiest way to setup Prometheus and Grafana, and have the Grafana configured to use Prometheus as a data source.

Step 3: Install Prometheus Operator

In my case, the Kubernetes cluster is behind a firewall. I am configuring the Granfana to be accessible via NodePort, as I need to access the Grafana UI using ssh tunnel.

kubectl create namespace prometheus-operator
helm install prometheus-operator stable/prometheus-operator -n prometheus-operator --set prometheusOperator.createCustomResource=false,grafana.service.type=NodePort

Step 4: Verify

kubectl get pods -n prometheus-operator

The above commands should show that all promtheus operator, prometheus, node exporter and grafana pods are running.

NAME                                                     READY   STATUS    RESTARTS   AGE
alertmanager-prometheus-operator-alertmanager-0          2/2     Running   0          30m
prometheus-operator-grafana-cf6954699-5rcgl              2/2     Running   0          30m
prometheus-operator-kube-state-metrics-5fdcd78bc-sckjv   1/1     Running   0          30m
prometheus-operator-operator-5dd8f8f568-52qk8            2/2     Running   0          30m
prometheus-operator-prometheus-node-exporter-p8pm8       1/1     Running   0          30m
prometheus-operator-prometheus-node-exporter-trlhp       1/1     Running   0          30m
prometheus-operator-prometheus-node-exporter-wsm4n       1/1     Running   0          30m
prometheus-prometheus-operator-prometheus-0              3/3     Running   1          30m

kubectl get svc -n prometheus-operator

Note that Grafana alone is running on NodePort

NAME                                           TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)                      AGE
alertmanager-operated                          ClusterIP   None            <none>        9093/TCP,9094/TCP,9094/UDP   31m
prometheus-operated                            ClusterIP   None            <none>        9090/TCP                     30m
prometheus-operator-alertmanager               ClusterIP   10.102.104.48   <none>        9093/TCP                     31m
prometheus-operator-grafana                    NodePort    10.96.160.172   <none>        80:31409/TCP                 31m
prometheus-operator-kube-state-metrics         ClusterIP   10.105.92.154   <none>        8080/TCP                     31m
prometheus-operator-operator                   ClusterIP   10.99.15.245    <none>        8080/TCP,443/TCP             31m
prometheus-operator-prometheus                 ClusterIP   10.109.75.138   <none>        9090/TCP                     31m
prometheus-operator-prometheus-node-exporter   ClusterIP   10.98.128.115   <none>        9100/TCP                     31m

Configure SSH Tunnesl to access Grafana UI

You can skip this step, if you direct access to the Kubernetes Worker node IP from your machines.

Windows using PuTTY

Get the Kubernetes Worker Node IP and the Grafana Node Port.
Get the SSH server using which, Kubernetes Worker Node IP is accessible. Say this is Landing IP.
Configure the PuTTY as follows:
- Create a new Session with Landing IP, Landing Port
- Create a Connection -> SSH -> Tunnels
- Source Port = Grafana NodePort
- Destination = Kubernetes Worker Node IP:Grafana Node Port
- Open the PuTTY session. Enter SSH user name and passowrd for the Landing IP.
Now you can access Grafana UI at the following URL. Default login and password ( admin/prom-operator )

  http://localhost:<Grafana-Node-Port>/

Linux using SSH

Get the Kubernetes Worker Node IP and the Grafana Node Port.
Get the SSH server using which, Kubernetes Worker Node IP is accessible. Say this is Landing IP.
Open SSH tunnel using the following command.

  ssh -NL <Grafana-Node-Port>:<k8s-worker-node-IP>:<Grafana-Node-Port> <landing-machine-user>@<landing-machine-ip> -p <landing-machine-ssh-port>

Now you can access Grafana UI at http://localhost:<Grafana-Node-Port>/. Default login and password ( admin/prom-operator )

Verify Granafa Dashboard

Login to Granfa UI
Click on Settings -> Data Source. You must see a Default Prometheus data source for http://prometheus-operator-prometheus:9090/
Click on Dashboards -> Manage Dashboards. You must see a list of dashboards. Click on any of them like: kubernetes-compute-resources-cluster
You must see some colors like:

DEV Community: Kiran Mova