Securing Solace Metrics: How to Use OAuth with solace-prometheus-exporter

Monitoring often gets less security attention than it deserves. While applications and APIs are protected with modern identity standards, metrics endpoints frequently rely on static credentials. For Solace environments exposed to Prometheus, this can turn monitoring into a weak spot within an otherwise well-secured system.

To address this, the community-driven solace-prometheus-exporter recently gained OAuth 2.0 support, enabling secure, token-based access to the Solace monitoring endpoints.

In this article, I'll explain why this matters, how it works, and how you can configure it in your own environment.

Why Authentication Matters for Metrics

Metrics contain sensitive operational data: queue backlogs, client sessions, system load, and detailed message flow patterns. Exposing these without secure authentication can reveal internal system behavior or traffic patterns to unauthorized parties.

OAuth 2.0 addresses these risks by providing:

• Use of short-lived tokens instead of long-lived static secrets
• Centralized identity control across environments
• Automatic token rotation and refresh without requiring service restarts or redeployment

This makes OAuth 2.0 a natural fit for production monitoring, particularly in Zero Trust or multi-cluster environments.

Introducing OAuth 2.0 Support in solace-prometheus-exporter

Starting from version 1.13.0, the solace-prometheus-exporter supports OAuth 2.0 using the Client Credentials Flow, when scraping metrics from the Solace SEMP API.

The exporter is now capable of:

• Requesting access tokens from an Identity Provider (IdP) using a Client ID and Secret.
• Automatically refreshing the tokens before expiration to maintain continuous scraping.
• Using the returned Bearer tokens in the HTTP Authorization header when calling the Solace SEMP API.

Architecture Overview of the OAuth-Based Setup

This setup consists of three main components:

• Identity Provider: Keycloak (issues the tokens)
• Resource Server: Solace Broker (validates tokens and provides metrics)
• Client: solace-prometheus-exporter (requests tokens and scrapes metrics)

Step-by-Step: Configuring OAuth for Solace Metrics

In this lab, we will generate our own CA and TLS certificates to keep everything self-contained. In a production environment, these certificates would typically be issued by an internal PKI or a trusted corporate CA.

Prerequisites

Before we start, ensure:

• Docker is installed and running.
• openssl is installed and available to generate the required TLS certificates.

Step 1: Prepare the Demo Infrastructure

OAuth in Solace requires TLS. To enable secure communication between containers using hostnames, we create a dedicated Docker bridge network and a local Certificate Authority (CA).

Certificates must include both the container hostname and localhost as Subject Alternative Names (SANs), allowing access from inside Docker and from the host.

Create a Docker Bridge Network

docker network create solace-net

Create a Root CA

# Generate private key for CA
openssl genrsa -out /tmp/rootCA.key 4096

# Create a self-signed Root CA
openssl req -x509 -new -nodes -key /tmp/rootCA.key -sha256 -days 365 \
  -out /tmp/rootCA.crt -subj "/CN=MyLocalCA"

Step 2: Identity Provider Setup (Keycloak)

In this example, we use Keycloak as our IdP, as it is open source and easily runnable via Docker.

Generate TLS Certificates for Keycloak

# OpenSSL config with SANs for Keycloak
echo '[req]
default_bits       = 4096
prompt             = no
default_md         = sha256
req_extensions     = req_ext
distinguished_name = dn

[dn]
CN = keycloak

[req_ext]
subjectAltName = @alt_names

[alt_names]
DNS.1 = keycloak
DNS.2 = localhost' > /tmp/keycloak-cert.conf

# Create CSR
openssl req -new -nodes -out /tmp/keycloak.csr -newkey rsa:4096 -keyout /tmp/keycloak.key -config /tmp/keycloak-cert.conf

# Sign CSR with the Root CA
openssl x509 -req -in /tmp/keycloak.csr -CA /tmp/rootCA.crt -CAkey /tmp/rootCA.key -CAcreateserial \
  -out /tmp/keycloak.crt -days 365 -sha256 -extfile /tmp/keycloak-cert.conf -extensions req_ext

Configure the Realm

We set up a Realm (acme-org) to represent our identity domain. Within this realm, we define:

• Client prometheus-exporter - The Token Requester
  This is the client application requesting the token. It uses the Client Secret (my-secret) to authenticate itself with Keycloak.

• Client solace - The Audience
  This represents the Resource Server (our Solace Broker) and is the expected Audience (aud) of the token. It tells Keycloak that the issued token is intended for use with Solace.

We rely on specific Mappers within Keycloak to inject the necessary claims into the JWT:

1. Audience (aud) - Where is this token valid?
   The oidc-audience-mapper ensures the issued JWT contains the claim "aud": "solace". This identifies the Solace Broker as the only intended recipient. If a token without this claim is presented, the broker will reject it, preventing tokens meant for other services from being misused here.

2. Scope (scope) - What permissions does the Client want?
   The scope specifies what actions the client is requesting, regardless of the client's assigned roles. In this example, we define a custom client scope called solace.admin. It tells the Solace broker that the exporter is requesting admin access for Solace. Also we assign this scope as a default scope for the client prometheus.exporter. Scopes are usually coarse-grained and represent what the client wants to do, not who the client is.

3. Roles/Groups (groups Claim) - Who is the client and what privileges does it have?
   The oidc-usermodel-realm-role-mapper maps the client's assigned roles (in this case, the default role default-roles-acme-org) into a claim named "groups". Solace uses this claim to map the identity to an internal access level (e.g., read-only). Usually you want to create a custom role for the Client to map the permissions on Solace. But for the blog, we stick to the default roles.
   

echo '{
  "realm": "acme-org",
  "enabled": true,
  "accessTokenLifespan": 3600,
  "clients": [
    {
      "clientId": "solace",
      "protocol": "openid-connect",
      "publicClient": false,
      "secret": "my-secret",
      "serviceAccountsEnabled": true,
      "directAccessGrantsEnabled": false,
      "standardFlowEnabled": false
    },
    {
      "clientId": "prometheus-exporter",
      "protocol": "openid-connect",
      "publicClient": false,
      "secret": "my-secret",
      "serviceAccountsEnabled": true,
      "directAccessGrantsEnabled": false,
      "standardFlowEnabled": false,
      "defaultClientScopes": ["solace.admin"],
      "protocolMappers": [
        {
          "name": "audience-mapper",
          "protocol": "openid-connect",
          "protocolMapper": "oidc-audience-mapper",
          "consentRequired": false,
          "config": {
            "claim.name": "aud",
            "jsonType.label": "String",
            "access.token.claim": "true",
            "id.token.claim": "false",
            "included.client.audience": "solace"
          }
        },
        {
          "name": "roles-mapper",
          "protocol": "openid-connect",
          "protocolMapper": "oidc-usermodel-realm-role-mapper",
          "consentRequired": false,
          "config": {
            "claim.name": "groups",
            "jsonType.label": "String",
            "access.token.claim": "true",
            "id.token.claim": "false",
            "multivalued": "true"
          }
        }
      ]
    }
  ],
  "clientScopes": [
    {
      "name": "solace.admin",
      "description": "Scope for Prometheus Exporter to read Solace metrics",
      "protocol": "openid-connect",
      "attributes": {
        "display.on.consent.screen": "false"
      }
    }
  ]
}' > /tmp/acme-org.json

Run Keycloak

docker run \
  --network solace-net \
  --platform linux/amd64 \
  -p 8080:8080 \
  -p 8443:8443 \
  -v /tmp/acme-org.json:/opt/keycloak/data/import/realm.json \
  -v /tmp/keycloak.key:/opt/keycloak/data/keycloak.key \
  -v /tmp/keycloak.crt:/opt/keycloak/data/keycloak.crt \
  -e KC_HOSTNAME_STRICT_HTTPS=false \
  -e KC_HOSTNAME_STRICT=false \
  -e KC_HOSTNAME=keycloak \
  -e KC_HTTPS_CERTIFICATE_FILE=/opt/keycloak/data/keycloak.crt \
  -e KC_HTTPS_CERTIFICATE_KEY_FILE=/opt/keycloak/data/keycloak.key \
  -e KEYCLOAK_ADMIN=admin \
  -e KEYCLOAK_ADMIN_PASSWORD=admin \
  --name keycloak \
  quay.io/keycloak/keycloak:26.4.7 \
  start-dev \
    --import-realm \
    --https-port=8443 \
    --hostname=keycloak

When you see the log entry [...] Listening on: http://0.0.0.0:8080 and https://0.0.0.0:8443, Keycloak is up and running and you can proceed with the next steps.

Validate Issued OAuth Tokens

We can now test the setup by fetching a token for the prometheus-exporter client with the required scope solace.admin.

curl --insecure -X POST "https://localhost:8443/realms/acme-org/protocol/openid-connect/token" \
  -d "grant_type=client_credentials" \
  -d "client_id=prometheus-exporter" \
  -d "client_secret=my-secret" \
  -d "scope=solace.admin"

The response contains a JSON Web Token (JWT) in the access_token field. When decoded, the payload confirms the configuration:

{
  "exp": 1766104964,
  "iat": 1766101364,
  "jti": "trrtcc:c50ac927-4c81-e22a-83b7-f36fad114c89",
  "iss": "https://keycloak:8443/realms/acme-org",
  "aud": "solace",
  "sub": "85ba915f-6c6f-4109-9704-dca2ff1057cd",
  "typ": "Bearer",
  "azp": "prometheus-exporter",
  "scope": "solace.admin",
  "groups": [
    "offline_access",
    "uma_authorization",
    "default-roles-acme-org"
  ]
}

Step 3: Configure Solace as the OAuth Resource Server

Generate TLS Certificates for Solace

# openssl config with san
echo '[req]
default_bits       = 4096
prompt             = no
default_md         = sha256
req_extensions     = req_ext
distinguished_name = dn

[dn]
CN = solace

[req_ext]
subjectAltName = @alt_names

[alt_names]
DNS.1 = solace
DNS.2 = localhost' > /tmp/solace-cert.conf

# create csr
openssl req -new -nodes -out /tmp/solace.csr -newkey rsa:4096 -keyout /tmp/solace.key -config /tmp/solace-cert.conf

# sign csr with ca:
openssl x509 -req -in /tmp/solace.csr -CA /tmp/rootCA.crt -CAkey /tmp/rootCA.key -CAcreateserial \
  -out /tmp/solace.crt -days 365 -sha256 -extfile /tmp/solace-cert.conf -extensions req_ext

cat /tmp/solace.key /tmp/solace.crt > /tmp/solace.pem

Start the Solace Broker

docker run \
  --network solace-net \
  --platform linux/amd64 \
  -p 8081:8080 \
  -p 1943:1943 \
  -v /tmp/solace.pem:/etc/solace_tls/solace.pem \
  --shm-size=2g \
  --env tls_servercertificate_filepath=/etc/solace_tls/solace.pem \
  --env username_admin_globalaccesslevel=admin \
  --env username_admin_password=admin \
  --name solace \
  solace/solace-pubsub-standard:10.25.12

Solace distinguishes between client users and management users.
For interacting with the SEMP API - which the Prometheus exporter uses to retrieve metrics - we need a management user, not a client user.

To enable OAuth for management access, we have to configure an OAuth profile at the broker level (outside of any Message VPN). This configuration is currently not available via the web UI, so it must be done via the SEMP API, either through the CLI or a direct REST call. Solace has a good documentation for this.

Since Solace needs to validate the signature of the JWT access tokens issued by Keycloak, we must upload the Root CA certificate which signed the Identity Provider's certificate (Keycloak). This establishes a trust relationship, allowing the broker to securely verify that the token truly came from our Keycloak server.

Establish Trust: Uploading the Root CA to Solace

# store CA in variable and replace newlines
ROOT_CA=$(awk '{printf "%s\\n", $0}' /tmp/rootCA.crt)

# upload domain CA
curl -X POST "http://localhost:8081/SEMP/v2/config/domainCertAuthorities" \
  -u "admin:admin" \
  -H "Content-Type: application/json" \
  -d "{
    \"certAuthorityName\": \"root_ca\",
    \"certContent\": \"$ROOT_CA\"
  }"

OAuth Profiles: Trusting the Identity Provider

An OAuth Profile represents the trust relationship between the Solace broker and an OAuth 2.0 / OpenID Connect IdP.

In simple terms, the OAuth profile answers the question:

"How do I verify that an incoming access token is valid, and what do I expect it to look like?"

An OAuth profile configures:

• Who issued the token: The issuer (iss claim) ensures the token comes from the expected IdP.
• How tokens are validated
  • Verifying the JWT signature using the IdP's JWKS endpoint
  • Optionally calling the token introspection endpoint
• Which token properties are required, For example:
  • Token type (JWT)
  • Audience (aud)
  • Required scopes
  • Required claims
• Which OAuth endpoints Solace can use
  • Authorization endpoint
  • Token endpoint
  • Introspection endpoint
  • JWKS endpoint
  • Userinfo endpoint
• What happens if no fine-grained authorization matches
  • Default global access level
  • Default Message VPN access level
• Which claim value grants access (accessLevelGroupsClaimName claim)

From Solace's perspective, the OAuth profile defines authentication rules:
If a token does not match these rules, access is denied before any permissions are evaluated.

Create the OAuth Profile with this:

curl -X POST "http://localhost:8081/SEMP/v2/config/oauthProfiles" \
  -u "admin:admin" \
  -H "Content-Type: application/json" \
  -d '{
    "accessLevelGroupsClaimName": "groups",
    "clientId": "solace",
    "clientSecret": "my-secret",
    "displayName": "keycloak_profile",
    "enabled": true,
    "endpointDiscovery": "https://keycloak:8443/realms/acme-org/.well-known/openid-configuration",
    "interactiveEnabled": false,
    "issuer": "https://keycloak:8443/realms/acme-org",
    "oauthProfileName": "keycloak_profile",
    "oauthRole": "resource-server",
    "resourceServerRequiredAudience": "solace",
    "resourceServerRequiredIssuer": "https://keycloak:8443/realms/acme-org",
    "resourceServerRequiredScope": "solace.admin",
    "resourceServerRequiredType": "JWT",
    "sempEnabled": true,
    "usernameClaimName": "azp"
  }'

OAuth Access Level Groups: Mapping Identity to Permissions

While the OAuth profile validates who you are, OAuth Access Level Groups determine what you are allowed to do.

This answers the question:

"Given this token, what permissions should this identity have inside Solace?"

Solace extracts the value from the configured claim (in our case, groups) and matches it against these groups.

Create the authorization group mapping:

curl -X POST "http://localhost:8081/SEMP/v2/config/oauthProfiles/keycloak_profile/accessLevelGroups" \
  -u "admin:admin" \
  -H "Content-Type: application/json" \
  -d '{
    "description": "Global read-only group",
    "globalAccessLevel": "read-only",
    "groupName": "default-roles-acme-org",
    "oauthProfileName": "keycloak_profile"
   }'

Step 4: Configure the Client (solace-prometheus-exporter)

To use OAuth in the exporter, set the following parameters.

Note: Since this is a demo environment using self-signed certs, SOLACE_SSL_VERIFY is set to false. In production, you would mount the Root CA and set this to true.

The exporter can be run using these parameters:

echo 'SOLACE_LISTEN_ADDR=0.0.0.0:9628
SOLACE_LISTEN_TLS=false
SOLACE_SCRAPE_URI=https://solace:1943
SOLACE_OAUTH_TOKEN_URL=https://keycloak:8443/realms/acme-org/protocol/openid-connect/token
SOLACE_OAUTH_CLIENT_ID=prometheus-exporter
SOLACE_OAUTH_CLIENT_SECRET=my-secret
SOLACE_OAUTH_CLIENT_SCOPE=solace.admin
SOLACE_DEFAULT_VPN=default
SOLACE_TIMEOUT=5s
SOLACE_SSL_VERIFY=false
PREFETCH_INTERVAL=0s
SOLACE_PARALLEL_SEMP_CONNECTIONS=1
' > /tmp/env

docker run \
  --network solace-net \
  --platform linux/amd64 \
  -p 9628:9628 \
  --env-file /tmp/env \
  --name exporter \
  solacecommunity/solace-prometheus-exporter:v1.13.0

Verify Access to Secured Metrics

Accessing a metrics endpoints in your browser should now return metrics. These are being securely fetched from Solace using a short-lived OAuth token.

Security Considerations & Best Practices

Securing metrics endpoints goes beyond enabling OAuth. Keep these best practices in mind:

• Enforce TLS: Always use trusted certificates. Self-signed certs are okay for demos but not for production.
• Least Privilege: Grant the exporter only the necessary scope and map identity groups carefully (read-only permissions).
• Short-Lived Tokens: Use short-lived access tokens and automatic refresh to minimize risk and operational overhead.
• Validate Token Claims: Ensure the broker validates the audience (aud), issuer (iss), and token type (JWT) for all requests.
• And as always: Observe access events and separate testing and production environments.

Following these practices ensures your metrics infrastructure is secure, reliable, and production-ready.

Conclusion

Monitoring endpoints are production APIs and should be treated as such.

This setup eliminates static secrets from monitoring, closing a commonly overlooked security gap. With OAuth in place, your monitoring endpoints are no longer a weak link, bringing you one step closer to a truly secure and scalable messaging environment.

This article focuses on securing communication between the exporter and the broker. Securing Prometheus scrape endpoints is a separate topic, but applying the same principles can further strengthen your overall monitoring infrastructure.