DEV Community

Cover image for Dynatrace Managed alerting profiles and notification integrations
Span Observer
Span Observer

Posted on Originally published at span.observer

Dynatrace Managed alerting profiles and notification integrations

Dynatrace detects a problem, Davis identifies the root cause… and the notification lands in a shared mailbox that nobody reads at night. Or the opposite: every tiny degradation in a test environment wakes up the production on-call engineer. Either way, the value of automatic detection is lost on the last mile: alert routing. That is why Dynatrace Managed alerting profiles deserve as much care as detection itself.

In Dynatrace Managed, that last mile has its own quirks. The cluster runs inside your network: to reach an ITSM tool, a Teams webhook or an externally hosted paging service, the nodes (or a proxy) must be able to get out. The SMTP server is yours and must be configured in the CMC. And in a shared environment, routing must respect the split into management zones.

Here is how to combine alerting profiles and integrations so that every problem reaches the right team, at the right time, with the right level of urgency.

The model: problem, profile, integration

Three objects form a chain:

  1. The problem: opened by Davis when one or more events (unavailability, errors, slowdown, resource saturation, custom event) cross the thresholds. It has a severity, a scope of impacted entities and a duration.
  2. The alerting profile: a filter. It says « out of all problems, keep those with this severity, in this management zone, carrying this tag, open for at least N minutes ».
  3. The notification integration: a channel (email, webhook, ITSM, paging tool, chat) linked to an alerting profile. When a problem matches the profile, the notification is sent, then updated on resolution.

One profile can feed several integrations, and an environment can have many profiles. This separation is what makes fine-grained routing possible.

Designing your Dynatrace Managed alerting profiles

By scope

One profile per management zone of responsibility (prod-paiement, prod-catalogue…) is the foundation. It mirrors the segmentation already set up for access control (see users, groups and permissions).

By severity and delay

The most underrated setting is the delay before notification. An availability problem justifies an immediate alert; a performance degradation can wait a few minutes to avoid transient false positives.

Severity Suggested delay Channel
Availability 0 min On-call paging + ITSM
Errors 0 to 5 min ITSM + team chat
Slowdown 10 to 15 min Team chat
Resources 15 to 30 min Ops chat
Info / custom events As needed Email or webhook

These values are starting points to tune to your SLAs.

Managed-specific prerequisites

  • SMTP: in the CMC (Settings > Emails > SMTP server), choose the delivery method: SMTP server with Mission Control fallback (default), SMTP server only, or Mission Control only. On a cluster with no access to Mission Control, only your SMTP server (host, port, connection security, authentication, sender) can deliver email, including cluster event notifications.
  • Outbound traffic: webhooks are sent by the cluster. If your nodes have no direct internet access, configure a proxy in the CMC (Settings > Internet proxy) and allow the destinations on your firewalls.
  • Certificates: an internal tool exposed with a private certificate authority must be added to the cluster truststore, otherwise HTTPS calls will fail (a validation error shows up in the Server.*.*.log logs) (see SSL certificates and cluster domain).

Step-by-step tutorial

Step 1 — Check connectivity from a node

Before any configuration, test from every cluster node, taking any proxy into account:

curl -sv -x http://{proxy}:{port} -o /dev/null -w "%{http_code}\n" \
  https://{target-webhook-url}

nc -vz {smtp-server} 587
Enter fullscreen mode Exit fullscreen mode

Any HTTP response code, even a 4xx, proves the network path is open. A timeout points to a firewall block.

Step 2 — Configure SMTP in the CMC

In Settings > Emails > SMTP server, enter the SMTP server and send a test email to a mailbox you control. Check that it does not land in spam (SPF records of the sender domain).

Step 3 — Create the alerting profile

In the environment https://{your-domain}/e/{environment-id}, open Settings > Alerting > Problem alerting profiles and create the profile prod-paiement-astreinte: management zone prod-paiement, severity rules following the table above, and optionally a filter on the tier:front tag for availability problems.

Step 4 — Create a webhook integration

Create a custom webhook integration linked to the profile. The message body uses substitution variables (ID, title, state, impact, problem URL). Example payload:

{
  "source": "dynatrace-managed",
  "problemId": "{ProblemID}",
  "title": "{ProblemTitle}",
  "state": "{State}",
  "pid": "{PID}",
  "impact": "{ProblemImpact}",
  "impactedEntity": "{ImpactedEntity}",
  "tags": "{Tags}"
}
Enter fullscreen mode Exit fullscreen mode

The available variables (including {ProblemDetailsJSON} and {ImpactedEntities}) are listed on the configuration screen: use it as your reference for your version. Add the authentication header the target expects (Authorization: Bearer {target-token}) and send a test notification.

Step 5 — Automate through the API

Profiles and integrations are configuration objects available through the environment API. To list them:

export DT_ENV="https://{your-domain}/e/{environment-id}"
curl -s -H "Authorization: Api-Token {token}" \
  "$DT_ENV/api/v2/settings/objects?schemaIds=builtin:alerting.profile&fields=objectId,value" \
  | python3 -m json.tool
Enter fullscreen mode Exit fullscreen mode

The builtin:alerting.profile schema is documented for Managed; its exact content can be read through /api/v2/settings/schemas/builtin:alerting.profile. A profile accepts up to 100 severity rules, combined with OR. Version-control the definitions and deploy them by script to all your environments.

Step 6 — Test end to end

Trigger a controlled problem on a test host in the zone (for example, stopping a monitored process) and time it: problem opening, notification received, ticket created, resolution notification. That is the only test that counts.

Best practices

  • One profile, one audience: avoid catch-all profiles sent to everyone.
  • Use delays to filter noise rather than disabling detection.
  • ITSM as the source of truth: chat informs, the ticket keeps the record.
  • Test after every cluster update or every proxy and certificate change.
  • Monitor delivery failures: an integration failing silently is worse than no integration at all.
  • Secrets: store target-tool tokens in a vault and rotate them.

Frequently asked questions

Why are no email notifications sent from my Managed cluster?

Most often, the cluster has no access to Mission Control and the company SMTP server has not been configured in the CMC (Settings > Emails > SMTP server), or traffic to it is blocked. Enter host, port, TLS and authentication, test the port from every node, then send a test email.

How do you stop on-call being woken up by minor degradations?

Use the alerting profile’s scope and delay rather than switching detection off: one profile per production management zone, an immediate alert for availability, and a few minutes’ delay for slowdowns and resources.

A webhook fails although the configuration looks right: what should you check?

Webhooks are sent from the cluster nodes. Check from a node, with curl and any proxy, that the target responds, then that the certificate authority of an internal tool is trusted by the cluster.

Conclusion

Good alerting in Managed rests on a simple chain — problem, profile, integration — applied with discipline: profiles aligned with management zones, delays suited to each severity, and network paths checked from the cluster nodes. Once this foundation is in place and version-controlled through the API, each new team is onboarded in minutes, and on-call only receives what is worth waking up for.

span.observer is an independent community blog, not affiliated with Dynatrace LLC. Dynatrace is a trademark of Dynatrace LLC. Always check values (ports, versions, sizes) against the official documentation for your Managed version.


Originally published on span.observer. French version: https://span.observer/fr/alerting-profils-integrations/.

Top comments (0)