DEV Community

Adam Gardner
Adam Gardner

Posted on • Edited on

1 1 1

Hands On: Pull metrics into Kubernetes from anywhere and treat them generically with the Keptn Metrics Server

Metrics, Metrics, Everywhere

Metrics are everywhere because tool sprawl is real. If you work in anything beyond a startup, you'll most likely have multiple metric-storing backends. Multiple Prometheus instances, endless vendor solutions, homegrown databases etc.

What if...

What if you could just say "get me a metric called foo" and magically, it appeared?

That's precisely what the Keptn Metrics Server allows.

How the Keptn Metrics Server Works

Install Prometheus

The first thing you'll need, of course, is at least one backend to store metrics. So install Prometheus now:

helm repo add prometheus-community https://prometheus-community.github.io/helm-charts
helm repo update
helm install kube-prom-stack prometheus-community/prometheus
Enter fullscreen mode Exit fullscreen mode

Install Keptn

helm repo add keptn https://charts.lifecycle.keptn.sh
helm repo update
helm install keptn keptn/keptn -n keptn-system --create-namespace --wait
Enter fullscreen mode Exit fullscreen mode

Expose Prometheus and Get an Existing Metric

Now expose the Prometheus UI and get the name of a metric you would like to use.

Get the names of the services. Look for one called prometheus-server running on port 80 (that's the Prometheus user interface):

kubectl get svc
Enter fullscreen mode Exit fullscreen mode

Port forward to access the UI on port 8080:

kubectl -n default port-forward svc/prometheus-server 8080:80
Enter fullscreen mode Exit fullscreen mode

Go to http://localhost:8080 and in the search box, type prometheus_.

Choose a metric which has a non-zero value. For example:

prometheus_http_requests_total{code="200", handler="/-/ready", instance="localhost:9090", job="prometheus"}
Enter fullscreen mode Exit fullscreen mode

Make a note of that PromQL. You'll need it later.

Create KeptnMetricsProvider

A KeptnMetricsProvider is how you define where metrics are coming from. One KeptnMetricsProvider is created for each "backend".

So create one now. I'm deciding to call it local-prometheus:

keptnmetricsprovider.yml

---
apiVersion: metrics.keptn.sh/v1alpha3
kind: KeptnMetricsProvider
metadata:
  name: local-prometheus
  namespace: default
spec:
  type: prometheus
  targetServer: "http://prometheus-server.default.svc.cluster.local:80"
Enter fullscreen mode Exit fullscreen mode

Apply it:

kubectl apply -f keptnmetricsprovider.yml
Enter fullscreen mode Exit fullscreen mode

See all KeptnMetricsProviders:

kubectl get keptnmetricsproviders
Enter fullscreen mode Exit fullscreen mode

Create a KeptnMetric

A KeptnMetric is created to define what you want to retrieve, and from where (which leverages the KeptnMetricsProvider) and how often the metric should be retrieved (fetchIntervalSeconds).

The KeptnMetricsProvider and KeptnMetric must be in the same namespace.

keptnmetric.yml

---
apiVersion: metrics.keptn.sh/v1alpha3
kind: KeptnMetric
metadata:
  name: prometheus-http-requests-total
  namespace: default
spec:
  provider:
    name: local-prometheus
  query: 'prometheus_http_requests_total{code="200", handler="/-/ready", instance="localhost:9090", job="prometheus"}'
  fetchIntervalSeconds: 10
Enter fullscreen mode Exit fullscreen mode

Show all Keptn Metrics with:

kubectl -n default get keptnmetrics
Enter fullscreen mode Exit fullscreen mode

That's it. Use any metric generically regardless of backend

All of your metrics are now available on-cluster to use for whatever you need. Horizontal Pod Autoscaling, FinOps etc. etc.

Billboard image

The Next Generation Developer Platform

Coherence is the first Platform-as-a-Service you can control. Unlike "black-box" platforms that are opinionated about the infra you can deploy, Coherence is powered by CNC, the open-source IaC framework, which offers limitless customization.

Learn more

Top comments (0)

A Workflow Copilot. Tailored to You.

Pieces.app image

Our desktop app, with its intelligent copilot, streamlines coding by generating snippets, extracting code from screenshots, and accelerating problem-solving.

Read the docs

👋 Kindness is contagious

Discover a treasure trove of wisdom within this insightful piece, highly respected in the nurturing DEV Community enviroment. Developers, whether novice or expert, are encouraged to participate and add to our shared knowledge basin.

A simple "thank you" can illuminate someone's day. Express your appreciation in the comments section!

On DEV, sharing ideas smoothens our journey and strengthens our community ties. Learn something useful? Offering a quick thanks to the author is deeply appreciated.

Okay