DEV Community

SelimBF
SelimBF

Posted on Originally published at getruptura.dev

Predictive failure alerts in the OpenShift cluster Alertmanager

Prefer video? The 4-minute overview above covers the whole idea.

On OpenShift, platform alerts already have a home: the cluster Alertmanager in openshift-monitoring, with routes, silences and receivers that the platform team maintains. Any new alerting tool that ships its own notification system creates a second place to configure, a second place to silence, and a second pager.

Ruptura, an open-source operator that predicts workload failures, takes the other route: it sends its results to the Alertmanager you already run. Here is how that works on OpenShift, and what the alerts look like.

One resource, one channel

Ruptura is configured with a single custom resource, RupturaInstance. Pointing it at the cluster Alertmanager takes one channel:

apiVersion: ruptura.io/v1alpha1
kind: RupturaInstance
metadata:
  name: shop
  namespace: ruptura-demo
spec:
  level:
    scope: workload
    selector:
      matchLabels: {team: shop}
  alerting:
    channels:
      - name: cluster
        alertmanager:
          url: https://alertmanager-main.openshift-monitoring.svc:9094
          serviceAccountToken: true
Enter fullscreen mode Exit fullscreen mode

On OpenShift you do not even need to give it a Prometheus URL: without input.prometheus, the engine reads the cluster Thanos querier with its own service account.

What the operator does for you

With serviceAccountToken: true:

  • The operator binds the role monitoring-alertmanager-edit in openshift-monitoring to the engine's service account.
  • The engine sends that token only to the URL of this channel, and trusts the OpenShift service CA.

That is the whole integration. No extra receiver, no webhook bridge, no second dashboard. One rule to remember: never set serviceAccountToken on a channel that points to an Alertmanager outside the cluster.

What an alert looks like

Every minute, an instance sends its active failures to each channel. An alert ends 3 minutes after the failure stops being reported. Each alert carries labels you can route on:

Label Example
alertname RupturaSaturation, RupturaCrashLoop
severity critical or warning
namespace demo-shop
target demo-shop/Deployment/step-leak
kind predicted or detected
level workload

The summary annotation holds the recommended action, for example "Review the memory limit or restart the pod before it is killed", and state holds the failure state.

Why this matters in practice

Because the alerts land in the cluster Alertmanager, everything your team already set up applies to them:

  • Routes: send namespace="demo-shop" to the shop team, as you do for every other alert.
  • Silences: silence a noisy target during a maintenance window from the same console.
  • Inhibition: the standard OpenShift inhibition rule applies, so a critical alert hides warning alerts with the same alertname in the same namespace.
  • The kind label: route kind="predicted" to a channel the team reads during the day, and keep the pager for kind="detected". A prediction gives you time; a detection does not.

Try it

Install the operator:

oc apply -f https://raw.githubusercontent.com/benfradjselim/ruptura/main/install/operator.yaml
oc -n ruptura-system rollout status deploy/ruptura-operator
Enter fullscreen mode Exit fullscreen mode

Then create a RupturaInstance like the one above and check that it reaches Running:

oc get rupturainstance -A
Enter fullscreen mode Exit fullscreen mode

The full field reference is in the RupturaInstance docs, and the alerting options, including any non-OpenShift Alertmanager, are in Alerting.

Ruptura is open source (Apache 2.0) and still early, not yet recommended for production. If you run OpenShift, I would love your feedback on how the alerts fit your routing: become an early adopter.


Docs and getting started: getruptura.dev

Top comments (0)