DEV Community

Cover image for Preparing your infrastructure: prerequisites and Dynatrace Managed cluster sizing
Span Observer
Span Observer

Posted on Originally published at span.observer

Preparing your infrastructure: prerequisites and Dynatrace Managed cluster sizing

#dynatrace #dynatracemanaged #infrastructure #linux

Most problems on a Dynatrace Managed cluster are decided before installation even starts: disks that are too slow, a data partition that is too small, a drifting clock or a forgotten firewall rule between two nodes. Once the cluster is in production and holds months of history, fixing these choices becomes slow and risky, which is why Dynatrace Managed cluster sizing has to happen up front.

This topic is specific to Managed because you supply the hardware or the virtual machines. There is no « more capacity » button: you have to anticipate the number of monitored hosts, the volume of user sessions, the retention you want and growth over two or three years.

This article offers a sizing method, the list of system prerequisites to validate and a checklist you can run on every future node.

The figures that drive Dynatrace Managed cluster sizing

A cluster’s size depends less on the number of servers than on the amount of data produced. The main drivers are:

  • The number of host units monitored in full-stack mode, plus the share of hosts on infrastructure-only monitoring.
  • The volume of transactions (service requests) captured, which feeds trace storage.
  • The number of user sessions (RUM), which makes Elasticsearch grow (the role of each store is covered in the article on Dynatrace Managed architecture).
  • The number of custom metrics and extensions, which weigh on Cassandra.
  • The retention periods configured per environment for each data type (see data retention and storage management).

Start by estimating these figures today and two years out, then add a margin of at least 30%.

Orders of magnitude per node

The official documentation offers several node profiles. The table below summarises the published profiles (capacity per node, minimum specifications); they are estimates to compare with your actual load:

Profile vCPU RAM Typical use
Micro 4 32 GB Up to 50 host units per node; proof of concept
Small 8 64 GB Up to 300 host units per node
Medium 16 128 GB Up to 600 host units per node
Large / XLarge 32 / 64 256 / 512 GB Up to 1,250 / 2,500 host units per node

For storage, plan for several hundred gigabytes to several terabytes per node, split between Cassandra, Elasticsearch and transactions. Disks must be fast and local or on a low-latency SAN: high-latency remote volumes (NFS, CIFS) are not supported for data, only for backups.

System prerequisites

Operating system

  • A 64-bit Linux distribution listed in the compatibility matrix for your version (x86-64 only: Red Hat, Rocky, Oracle Linux, SUSE, Ubuntu, Debian, Amazon Linux), on a dedicated host, physical or virtual but not in a container.
  • Root access for installation; the installer then creates the system user that runs the services.
  • Identical configuration on every node: same distribution, same kernel, same settings.

Network

  • A static IPv4 address for each node (IPv6 is not supported), with consistent forward and reverse DNS resolution.
  • Latency between nodes of 10 ms or less, over a private network; all nodes in the same time zone.
  • Inter-node ports open between all nodes and closed elsewhere; inbound HTTPS port open to agents, ActiveGates and users.
  • Outbound HTTPS access to Mission Control, unless you choose offline mode.

Time and storage

  • NTP synchronisation (chrony or equivalent) on every node: distributed databases do not tolerate clock drift well.
  • Dedicated volumes for /var/opt/dynatrace-managed and, ideally, for each of the stores.
  • A shared NFS export for backups, mounted at the same path on every node.

Step-by-step tutorial: validate a future node

Run this checklist on each server before installation and keep the results.

Step 1 — Check the system and resources 

cat /etc/os-release
uname -r
nproc
free -g
Enter fullscreen mode Exit fullscreen mode

Compare with the compatibility matrix and the chosen profile. All nodes must show the same values.

Step 2 — Check the disks 

lsblk -o NAME,SIZE,TYPE,ROTA,MOUNTPOINT
df -h /opt /var/opt
Enter fullscreen mode Exit fullscreen mode

A ROTA value of 0 means a non-rotational disk. Check that /var/opt/dynatrace-managed will sit on the intended volume.

Step 3 — Roughly measure disk performance 

dd if=/dev/zero of=/var/opt/ddtest bs=1M count=2048 oflag=direct
rm -f /var/opt/ddtest
Enter fullscreen mode Exit fullscreen mode

This test is basic; if you have dedicated benchmarking tools, use them to measure random write latency.

Step 4 — Check time and name resolution 

timedatectl
chronyc tracking
hostname -f
getent hosts {other-node-ip}
Enter fullscreen mode Exit fullscreen mode

Step 5 — Test connectivity between nodes 

ping -c 5 {other-node-ip}
curl -sv --connect-timeout 5 telnet://{other-node-ip}:{port} 2>&1 | grep -i connected
Enter fullscreen mode Exit fullscreen mode

Repeat for every inter-node port listed in the documentation for your version. Before installation, nothing is listening: the main point is to check that the firewall does not reject the connection (a « connection refused » is acceptable, a timeout is not).

Step 6 — Check outbound access and the backup share 

curl -sI --connect-timeout 10 https://{mission-control-url} | head -1
findmnt /mnt/{backup-share}
touch /mnt/{backup-share}/test && rm /mnt/{backup-share}/test
Enter fullscreen mode Exit fullscreen mode

If a corporate HTTP proxy is mandatory, note its settings: you will be asked for them during installation.

Step 7 — Check system limits 

ulimit -n
sysctl vm.max_map_count
swapon --show
Enter fullscreen mode Exit fullscreen mode

The installer adjusts some settings itself (it notably turns swap off and changes max_map_count and system limits), but it is better to know the initial state, especially if a hardening policy rewrites them on reboot.

Best practices

  • Size for the two-year target, not for the current estate.
  • Start with three identical nodes in production; add nodes rather than oversizing a single server.
  • Prefer local SSD or a fast SAN; avoid slow shared volumes for data.
  • Reserve resources for the VMs (no CPU or memory overcommit on the hypervisor).
  • Exclude data directories from antivirus scanning and from continuous system backups.
  • Automate the checklist (Ansible, a script) to guarantee identical nodes.

Frequently asked questions

How many nodes should a production Managed cluster have?

Three identical nodes are the recommended starting point. To gain capacity, add nodes of the same profile rather than oversizing a single server.

Can the data live on network storage?

Only if it offers low latency, such as a fast SAN. Cassandra copes poorly with slow network storage, so local SSD remains the safest choice. The NFS share is for backups only.

Does the cluster need Internet access?

Outbound HTTPS to Mission Control is expected for licensing and updates. If that is not possible, offline mode lets you do without it, at the cost of manual operations.

Conclusion

A well-prepared Managed cluster has identical nodes, disks that keep up with the load, a reliable clock and documented network flows. Spending a day validating these points saves weeks of remediation. With this checklist in hand, you are ready to install the first node.

span.observer is an independent community blog, not affiliated with Dynatrace LLC. Dynatrace is a trademark of Dynatrace LLC. Always check values (ports, versions, sizes) in the official documentation for your version of Managed.

Originally published on span.observer. French version: https://span.observer/fr/prerequis-dimensionnement-cluster-managed/.

Top comments (0)