I build a monitoring agent, which makes me the wrong person to tell you which one is light. So this post comes with a public harness and every raw sample, and you can check any number in it.
The question is simple: when you install a monitoring agent on a Linux server, what does it take from that server? Most of the comparison tables I found quote each vendor's own documentation. I wanted the agents installed side by side and measured the same way, so I did that for eleven of them.
The short version:
- Idle memory runs from 9 MB to 358 MB.
- The agent that uses the least memory uses the most CPU.
- Two agents sit at a low idle figure and return to about 120 MB at intervals.
- My own agent came fourth of eleven on memory. The benchmark also found a flaw in it, which I fixed before the published run.
- Twice the measurement itself was wrong, once because of something I did. Both are described below.
The setup
- Eleven agents, plus a control server with no agent.
- Hetzner Cloud CX23 in Helsinki: 2 shared vCPU, 4 GB of RAM, Ubuntu 26.04. One agent per server, each on a fresh image.
- Every agent is installed the way its vendor documents it, with default settings. Nothing is tuned, mine included.
- Each server also runs Docker with two idle containers, so agents that watch containers have containers to watch.
- A collector samples every 10 seconds from the cgroup of the agent's systemd units: CPU time, memory and disk bytes. Memory is PSS, summed from
smaps_rollup. Network bytes come from nftables counters matched on the cgroup. - The idle window is hours 1 to 24 after the install. The hour after that runs a fixed synthetic load on every vCPU.
- Each agent is measured three times, on three different servers. Every figure below is the median of the three runs.
Results
Sorted by idle memory. Lower is better in every column.
| Agent | Version | Idle memory (PSS), MB | Memory peak, MB | Idle CPU, % of one vCPU | Disk written per 24 h, MB | Network per 24 h, MB | On disk, MB |
|---|---|---|---|---|---|---|---|
| HetrixTools agent | 2.4.2 | 9.0 | 12.5 | 17.05 | 5.9 | 3.8 | 0.1 |
| Beszel agent | 0.21.1 | 17.9 | 118.1 | 0.17 | 0.0 | 3.8 | 10.8 |
| node_exporter | 1.12.1 | 24.5 | 10.6 | 0.02 | 0.0 | 19.7 | 23.9 |
| Monitorable agent (mine) | 1.7.0 | 45.8 | 18.2 | 0.09 | 0.0 | 8.1 | 57.3 |
| Pulse agent | 6.6.0 | 49.5 | 17.3 | 0.07 | 11.8 | 16.2 | 42.1 |
| Fivenines agent | 1.23.1 | 49.9 | 128.7 | 0.22 | 0.5 | 10.0 | 52.1 |
| Glances | 4.5.9 | 77.7 | 72.6 | 2.38 | 0.0 | 3929.7 | 28.8 |
| Telegraf | 1.40.3 | 180.2 | 59.0 | 0.21 | 0.0 | 17.3 | 330.6 |
| Datadog Agent | 7.85.2, 7.85.3 | 195.7 | 198.6 | 1.59 | 103.2 | 89.3 | 1619.3 |
| Netdata (self-hosted) | 2.12.3 | 357.1 | 308.0 | 3.81 | 138.7 | 0.0 | 794.5 |
| Netdata (claimed to Netdata Cloud) | 2.12.3 | 358.2 | 293.1 | 3.89 | 140.8 | 0.3 | 795.2 |
| No agent (the harness alone) | 12.6 | 16.1 | 0.07 | 10.5 | 0.0 | 0.1 |
Four things to know before reading the table:
- The last row is the control. It is what the harness itself costs. The harness runs in its own cgroup, so that cost is in no agent's row.
- Network is bytes sent for an agent that pushes its data, and bytes sent plus received for one that is polled.
- Glances' 3.9 GB is my own poller fetching its full API every 3 seconds, which is Glances' refresh period. Each reply is about 137 KB. Scale it to your own polling interval.
- A peak can be lower than the idle figure. PSS counts the pages of the agent's own binary, and the kernel may have charged those pages to another cgroup, such as the installer's.
CPU under load, install time and 45 row notes are on the full results page.
What stood out
The lightest on memory is the heaviest on CPU
The HetrixTools agent holds 9 MB, the lowest figure in the table. It also uses 17% of one vCPU at idle, the highest: between 12.6% and 17.4% across the three runs. On these servers it runs as a job that a systemd timer starts once a minute, and not as a long-lived daemon. The load average agrees with the CPU figure: an idle median of 0.22 to 0.30 on its servers against 0.05 to 0.13 on every other.
If you only compare memory, this agent wins. If you run it on a one-core VPS, a sixth of a core is a real cost.
A median hides the spikes
The Beszel agent's idle median is 17.9 MB, the second lowest. Its peak is 118 MB, and the peak is not only an install-time figure. In two of its three runs, its memory returned to 101 to 118 MB for one 10-second sample every 61 minutes.
The Fivenines agent shows something similar. It holds about 35 MB and rose to as much as 128 MB in single samples 16 minutes apart, during a few hours of two of its three runs.
I did not investigate what either agent does in those moments. If you are fitting an agent into a server with little free memory, read the peak column and not only the median.
The heavy agents do more
Netdata uses 357 MB, 3.8% of a vCPU and about 139 MB of disk writes a day. It also collects every second, stores the data on the server, and runs anomaly detection, all by default. An agent that forwards one sample a minute is a different kind of product. The table shows what the per-second local data costs; whether it is worth that is your decision.
The Datadog Agent uses 196 MB, writes 103 MB a day and takes 1.6 GB of disk.
node_exporter has the lowest CPU figure at 0.02%, but it only exposes metrics. You still run Prometheus somewhere to collect them.
Where mine landed
My agent is a custom build of the OpenTelemetry Collector. By default it collects host metrics, Docker containers, systemd services, SMART disk health and listening ports, and sends them once a minute.
It uses 45.8 MB, which puts it fourth. Three agents are lighter on memory: HetrixTools, Beszel and node_exporter. Beszel's idle figure is less than half of mine. On the other columns it uses 0.09% of a vCPU, sends 8.1 MB a day and writes nothing to disk while its deliveries succeed. One of its three runs wrote 0.3 MB, on a day when my own ingest endpoint was slow and seven batches waited long enough to be saved to disk.
What the benchmark did to my own agent
The first full run, from 31 October to 4 November, became the pilot. It showed that my agent, then at v1.5.0, wrote 115 MB a day to its on-disk export queue. It saved every batch to disk before sending it, even when the send succeeded seconds later. I had never measured that.
I fixed it in v1.7.0. A batch now stays in memory for up to 15 seconds and goes to disk only if it is still undelivered. The price is that a power cut can lose the last 15 seconds of data.
That left a fairness question. The rule of the benchmark is that every agent is measured at its current release, installed the way a reader would install it today. So I shipped the fix as a normal release for every user, and then measured every row again, not only mine. A benchmark-only configuration, or a redo of my row alone, would have broken the rule. The pilot's data, with the 115 MB figure, is still in the repository.
The pilot also found four defects in the harness:
- The collector missed processes in child cgroups, so one agent's row showed no processes at all.
- A sampler that restarted mid-run could write one timestamp twice, which doubled any figure summed across cgroups at that moment.
- The network counters included Docker bridge traffic.
- Ubuntu's unattended upgrades restarted some agents in the middle of a run.
All four were fixed before the second run, which took from 7 to 12 November.
Twice the measurement was wrong
The kernel billed an agent for someone else's disk writes
While checking the second run, I saw that one run of my agent and one run of Beszel's had each been charged about 16 MB of disk writes in their first seven to eight hours. The other runs of the same agents showed almost none.
The cause is in how cgroup v2 accounts for writeback. The kernel charges all writeback of one inode to a single cgroup. ext4 writes its metadata through the block device's own inode, about 2.8 MB an hour on an idle server of this type. Whichever cgroup owns that inode at the time gets the bill. On the live servers I saw it charged to a mount unit with no processes, to rsyslog, to the root cgroup and to Netdata's unit, where it reads as the agent's own disk writes. For the two runs above it is inferred from the rate and the size of the writes, because those servers had already been rebuilt.
I did not want to publish the figure as measured, and I did not want to correct it by inference. So the collector now records that stream with the tracepoint writeback:writeback_single_inode, and I measured again the eight runs it had affected or could have affected. Before the retest I committed a rule: a run measured again replaces the run it stands for, whatever it shows.
In the retest, my agent was charged nothing after its install in 25 hours, and Beszel's was charged 4 KB. The replaced runs stay published in the repository.
If you read per-service disk writes from io.stat on cgroup v2, the same thing can happen to your numbers.
I put traffic into a competitor's row
The results went live on 13 November. The Netdata Cloud row showed 0.6 MB of network a day, and all three of its servers had sent bursts of traffic at the same two moments. When I published, I did not know why.
Later that day I read Netdata's own access log on the three servers. It showed 53, 260 and 52 requests over the Netdata Cloud link, in those minutes and in no others. Almost all of them asked for chart data, and every one came from the same signed-in admin user, which was me. I had opened the Netdata Cloud dashboard while the run was in progress. The dashboard asks each agent for chart data over its cloud link, and that counts as the agent's traffic.
So for two days I published a competitor's network figure at twice its real value, and I was the cause.
I measured the three runs a third time, with nobody opening the dashboard, and read the access logs afterwards: no requests. Under the same rule the new runs replaced the old ones. The figure is 0.3 MB a day, and the page has shown it since 15 November.
Two rules came out of this. Nobody opens a vendor's interface while a run is in progress. And each agent's own logs are read before its server is rebuilt, because a rebuild would have destroyed this evidence.
What I still cannot explain
- CPU under load is lower than idle CPU in every row, the control included. No server recorded steal time. I have not found the cause. If you know it, please tell me in the comments.
- Servers on one plan are not identical. Two of the nine idled at 1.3 to 1.5 times the CPU time of the others. Their virtual CPUs report a different clock speed, so they are probably on another host type. Four rows have two of their three runs on those servers, and their row notes give the figure from a third server.
Limits
- One provider, one plan and one distribution. Other hardware or another kernel can move every figure.
- Three runs per agent is a small sample.
- Defaults only. A tuned configuration can be lighter or heavier.
- The agents do not collect the same things. The table shows what each one costs, not what you get for it.
- I make one of the agents.
Check it yourself
- The full table, the method and every row note: monitorable.io/benchmarks/agent-footprint-2026
- The harness, every 10-second sample and the run logs with every deviation: github.com/monitorable-io/agent-footprint-benchmark
- A recipe for measuring one agent on your own Ubuntu 26.04 VM in about an hour: Reproduce one agent in an hour
If you maintain one of these agents and think your row is wrong, open an issue on the repository. A run that had a problem gets measured again. I plan to repeat the whole benchmark next year.
I am the founder of Monitorable, a hosted server-monitoring service for Linux servers: one open-source OpenTelemetry agent per server, a dashboard with alerting, flat monthly pricing, and nothing to host.
Top comments (0)