Monitoring HPE Morpheus HVM and HKS clusters and the appliance from your existing Grafana

Most teams I work with already have a Grafana that the operations people look at every morning. When HPE Morpheus Enterprise (or the VM Essentials ) arrives, the cluster and appliance health lives in the Morpheus UI, which means one more screen to check. I wanted the HVM clusters, HKS Kubernetes clusters and Morpheus appliance itself on the same Grafana as everything else, without installing an agent or writing an exporter.

The approach here only uses what is already there: the Morpheus REST API, a read-only service user, the Infinity data source plugin and, for pod-level graphs, the Prometheus that HKS already runs inside the cluster. If you have a Grafana and admin access to Morpheus Enterprise, you can follow this top to bottom in about an hour.

This is what you end up with:

What I tested with

• Morpheus Enterprise 9.0.2 with one HVM cluster, one HKS cluster, a VMware cloud and a VM Essentials cloud
• Grafana 13.2 with the Infinity plugin 4.0
• The kube-prometheus stack that HKS deploys in the monitoring namespace (only for the pods dashboard)

Older Grafana versions that run Infinity should work too, but I have not checked them.

Steps 1 to 3 are done in the Morpheus Enterprise UI (the VM Essentials manager has the same screens), logged in as an administrator. Steps 4 to 6 are done on the Grafana server.

1. Create a read-only role (Morpheus Enterprise)

Everything Grafana sees comes through one API user, so the role behind it decides what shows up. I start from an empty role rather than copying an existing one, because a copy tends to carry permissions nobody remembers to remove.

Go to Administration > Roles, click + Add and create a User Role named Grafana Reader. A short description helps the next person who finds it; leave the two multitenant boxes unticked.

Open the new role. Every entry in the list starts with Access set to None. Leave it that way and set only these entries to Read; the section is the collapsible heading they sit under:

Section	Name	Access
Operations	Activity	Read
Operations	Guidance	Read
Provisioning	Apps	Read
Infrastructure	Clouds	Read
Infrastructure	Clusters	Read
Infrastructure	Compute	Read
Admin	Health	Read
Monitoring	Monitoring	Read

On the Group Access tab set the groups you want to see (or all groups) to Read.

That is the whole role. The instance entries have no Read level, which is why the dashboard lists hosts and virtual machines through the Compute entry instead of instances. License usage has no read level either, so it is not on the dashboard.

2. Create the service user (Morpheus Enterprise)

Go to Administration > Users > + Add:

• Username: grafana-reader
• First and last name, email: anything that tells your colleagues what this account is for; I leave Receive Notifications unticked
• Roles: Grafana Reader and nothing else
• Password (under Credentials): a long one that satisfies your password policy

Keep the password somewhere safe. You only need it again when you renew the token.

3. Get a long-lived API token (Morpheus Enterprise)

A token issued through the default morph-api client expires after 30 days. I did not want the dashboards to go blank every month, so I created a separate OAuth client with a longer lifetime and only use it for this account.

Go to Administration > Settings > Clients and click + Add Client:

• Client ID: grafana
• Access Token Validity Interval (seconds): 31536000, which is one year
• Refresh Token Validity Interval (seconds): 31536000
• Client Secret and Redirect URL: leave empty

Pick a shorter lifetime if your security team prefers it; the only cost is renewing more often.

Now request a token for the service user with that client. This part is a shell command, so run it from any machine that reaches Morpheus, the Grafana server included:

MORPHEUS=https://morpheus.example.com

curl -sk -X POST "$MORPHEUS/oauth/token" \
  --data-urlencode grant_type=password \
  --data-urlencode scope=write \
  --data-urlencode client_id=grafana \
  --data-urlencode username=grafana-reader \
  --data-urlencode 'password=YOUR_PASSWORD'

The response contains access_token and expires_in. expires_in should be close to 31536000; if it says around 2592000, the request went through morph-api instead of your client.

Check that the token works and only reads:

TOKEN=paste-the-access-token-here

curl -sk -H "Authorization: Bearer $TOKEN" "$MORPHEUS/api/clusters?max=5" | head -c 300; echo
curl -sk -H "Authorization: Bearer $TOKEN" "$MORPHEUS/api/health" | head -c 300; echo

Both calls should return JSON with data. A 403 on /api/health means the Health entry is still at None.

4. Add the Infinity data source (Grafana server)

From here on everything happens in the Grafana UI. Install the plugin from Administration > Plugins and data > Plugins, search for Infinity and click Install.

Then go to Connections > Data sources > Add new data source > Infinity and fill in the fields below. The labels I use are from Infinity 4.0; I think they have moved around a little between versions, so look for the nearest match if yours differ.

• Main: name it Morpheus
• URL, Headers & Params: set Base URL to https://morpheus.example.com, with no path and no trailing slash
• Authentication: pick Bearer Token, paste the token from step 3, and add the same https://morpheus.example.com under Allowed hosts
• Network: if your Morpheus certificate is not signed by a CA that Grafana trusts, either switch on With CA Cert and paste the CA, or switch on Skip TLS Verify
• Security: the allowed host you added shows up here as well; nothing else to change

The Base URL matters more than it looks. The dashboard panels only contain paths such as /api/servers, and Infinity puts this URL in front of them. Use the address Grafana can reach: the load balancer name if you run Morpheus in HA, or the node name on a single-node install.

Click Save & test. The data source then shows up in the list with its URL.

5. Import the Morpheus Overview dashboard (Grafana server)

Download morpheus-overview.json from the repository:

https://github.com/emrbaykal/morpheus-k8/tree/main/grafana-morpheus/dashboards

In Grafana go to Dashboards > New > Import, upload the file and pick the Morpheus data source when it asks for Morpheus (Infinity). Nothing else needs editing.

The dashboard has two variables at the top. Cluster lists every cluster the service user can see, and one performance row is drawn for each selected cluster. Cloud does the same for the virtual machine tables. Both default to All; if a cluster or cloud has nothing interesting in it, just untick it.

A few things to know when you read it:

The CPU, memory, network and IOPS numbers for hosts and VMs are the last sample Morpheus holds, not a time series. They refresh every time the dashboard refreshes (every 5 minutes by default), which is fine for a status board but will not give you history. For history on the Kubernetes side, use the pods dashboard below.

The appliance section reads /api/health, so it shows the Morpheus application node you land on through the URL you configured. Behind a load balancer that can be a different node on each refresh.

The Monitoring section shows the checks and incidents from Morpheus monitoring. If you do not use Morpheus checks, those two tables stay empty.

6. Pod metrics from the HKS cluster (Grafana server, optional)

Morpheus does not keep usage per pod, so this part comes from Prometheus. You do not have to install anything for it: when Morpheus builds an HKS cluster it also deploys kube-prometheus into the monitoring namespace. You can see it with:

kubectl -n monitoring get svc prometheus-k8s

Where the Prometheus data source points depends on where your Grafana runs.

If Grafana runs inside the HKS cluster (ours does), the URL is simply http://prometheus-k8s.monitoring.svc:9090.

If Grafana runs somewhere else, the prometheus-k8s service is only reachable inside the cluster by default, so you need to publish it the same way you publish other services on that cluster: an ingress, a load balancer or a NodePort. Whichever you choose, keep in mind that Prometheus has no login of its own, so anyone who can reach that address can query it. Limit it to the Grafana server at the firewall.

In Grafana, add a Prometheus data source with the URL that fits your case, click Save & test, then import kubernetes-pods.json from the same repository folder and choose this data source for Prometheus.

The dashboard shows node CPU and memory, and per pod CPU, memory, network and PVC usage, with a namespace filter at the top. It uses node-exporter, kube-state-metrics and the kubelet metrics, all of which the HKS monitoring stack already collects. How far back you can look depends on the Prometheus retention; ours keeps one day.

What the token can see

The Grafana Reader role cannot change anything, but it does read what the API returns for clusters and hosts, and some of those responses include connection details such as the Kubernetes service token of a cluster or the agent key of a host. The dashboards do not display these fields, but a Grafana user who can edit panels or use Explore could query them through the Morpheus data source.

So treat this data source like a credential. Keep the Editor and Admin roles on this Grafana for people who could have Morpheus read access anyway, and if you use Grafana Enterprise or Cloud, restrict query permission on the data source.

Renewing the token

Put a reminder in your calendar a couple of weeks before the token expires. Renewing is step 3 again: request a new token with the same curl command, open the Morpheus data source in Grafana, paste the new token and click Save & test. The dashboards keep working without any other change.

If you ever need to cut Grafana off immediately, disable the grafana-reader user in Morpheus or delete the grafana client.

Checklist

• [ ] Role Grafana Reader with eight entries at Read and Read on the groups
• [ ] User grafana-reader with that role
• [ ] OAuth client grafana with the token lifetime you want
• [ ] curl with the token returns data from /api/clusters and /api/health
• [ ] Infinity data source Morpheus saved and tested, URL without a path
• [ ] morpheus-overview.json imported, Cluster and Cloud variables list your environment
• [ ] Optional: Prometheus data source tested and kubernetes-pods.json imported
• [ ] Calendar reminder for the token renewal

If a panel shows "No data", open it with Edit and look at the query inspector. A 401 means the token expired or was pasted wrong, a 403 points to a role entry that is still at None, and a connection error usually means the data source URL or the allowed hosts entry does not match the address Grafana uses.