I Measured eBPF vs /proc Monitoring Overhead — eBPF Is Cheaper, and the Gap Grows with Scale

TL;DR: eBPF reads structured kernel data in-kernel, so it never pays the text-serialization-and-parsing tax /proc charges on every scrape. In my Lima-VM runs, eBPF was cheaper in every scenario — /proc never came out ahead. For a couple of system-wide gauges (total CPU%, memory) the margin is small, within run-to-run noise. For per-process metrics — where polling must open and parse a file for every process each scrape — eBPF was ~4× cheaper at ~100 processes and ~17× at ~600, widening with the number of processes you track. How much you save scales with how much you collect.

The problem — monitoring's cost, and why eBPF reads it cheaper

Almost every production host runs a monitoring agent — node-exporter, cAdvisor, or something home-grown — that wakes up every few seconds, reads a batch of files under /proc, parses out the numbers, and ships them. It runs around the clock, on every node, and almost nobody measures what it costs.

eBPF reads the same data more efficiently: it pulls structured numbers straight from the kernel and skips the text serialization and parsing /proc forces on every scrape — so it is never more expensive. How much cheaper it is scales with how much you collect: a hair for one or two gauges, a lot for per-process metrics across many processes. I built both and measured them.

What you'll learn

• Why eBPF is the cheaper way to read kernel data (and never the more expensive one)
• Why the gap is tiny for system gauges but large for per-process metrics
• How to measure it fairly, including the kernel-side cost most benchmarks miss

Two patterns, two sizes of gap

System gauges — total CPU%, memory, a disk/net counter. /proc serves these from one or two small files (/proc/stat, /proc/meminfo). eBPF reads the same counters in-kernel and skips the text parse, so it's still a touch cheaper — but with only a file or two, the margin is tiny: within run-to-run noise on a single box.

Per-process metrics — CPU time per container / per PID. This is what agents collect for every workload, and it's where polling gets expensive: it must open / read / parse /proc/<pid>/stat for every process on the box, every scrape — O(number of processes), including the ones doing nothing at all.

The eBPF way for per-process: accumulate in-kernel, read cheaply

Instead of re-scanning every process each interval, attach to the kernel's sched_switch and add each task's on-CPU time to a per-PID map as it happens. User space then just drains the map once per interval:

• It only ever sees pids that actually ran — sleeping processes cost nothing (there's no file to open).
• The reader does O(active pids) map reads, not O(all pids) file parses.
• The one continuous cost is the sched_switch hook itself, which scales with the context-switch rate, not the process count.

This is eBPF's event-driven model paying off: accumulate continuously in-kernel, read cheaply — versus polling, which redoes the full scan every single time.

The experiment — how I measured

🔬 Honest disclosure about the environment. I ran this inside a Lima VM on an Apple Silicon Mac (arm64) — not on bare metal or a production node. A laptop VM is a noisy place to measure sub-1%-of-a-core overhead: hypervisor scheduling, host contention, and CPU-core migration inflate variance, and arm64 differs from a typical x86 production node. Treat the absolute numbers as direction and rough magnitude, not production figures. I report run-to-run variance so you can judge the noise; the relative result and how it scales are the robust parts.

Environment

• Host: Apple Silicon Mac · Guest: Lima (Apple Virtualization vz), Ubuntu 24.04, kernel 6.8 (aarch64), 2 vCPU / 4 GB
• uname -r: 6.8.0-124-generic

What I measured — per-process CPU (utime+stime) for all processes, at the same interval on both sides:

• /proc poller: each interval, scan /proc, open/read/parse every /proc/<pid>/stat.
• eBPF: sched_switch accumulates per-task on-CPU ns into a hash map; the reader drains it each interval.
• I also ran a system-gauge variant (/proc/stat vs a BPF-timer reading kernel counters) for contrast.

Fair eBPF accounting (the part most benchmarks miss) — sched_switch runs in the scheduler, charged to whatever task is switching, not to the reader. So I count reader-cgroup CPU (systemd CPUUsageNSec, ns precision) + the BPF program's own kernel time. Because sched_switch is a tracepoint, its CPU is captured by bpf_stats (run_time_ns) — I read that via the program fd and add it. Counting only the reader would undercount eBPF and unfairly favor it.

How I kept it fair — same metric and interval on both sides; 6–8 runs of 20 s each, interleaved/order-alternated, first run discarded as warm-up, reported as mean ± stddev; measured at two process counts (~100, and ~600 via spawned idle processes) to show how the gap scales.

Results (% of one core, Lima VM)

Scenario	/proc (mean ± sd)	eBPF reader+kernel (mean ± sd)	Δ	/proc ever cheaper?
System gauges (light)	~0.03%	~0.02%	~1.3× (eBPF lower)	No — gap within noise
Per-process, ~100 procs	0.30% ± 0.055%	0.076% ± 0.022%	3.9×	No
Per-process, ~600 procs	1.18% ± 0.13%	0.071% ± 0.004%	16.6×	No

Reading honestly: eBPF was cheaper in every scenario — in no run did /proc come out ahead. For light system gauges the margin is small enough to sit within run-to-run noise, so don't expect to feel it on one box. The per-process gap is large and reliable (outside noise, 0% steal) and grows with the number of processes — 3.9× at ~100, 16.6× at ~600 in my runs: /proc's cost scales with all processes, eBPF's stays roughly flat (only the active ones plus the context-switch rate). In absolute terms these are still small fractions of one core on a single box — eBPF's real payoff is at process density and fleet scale.

Show me the code (the core pattern)

The heart of the eBPF collector: on every context switch, add the outgoing task's on-CPU time to a per-PID map. A sketch of the shape, not the exact benchmark code — the full, runnable collector is in the repo.

// Sketch — accumulate per-PID on-CPU time on each context switch.
SEC("tracepoint/sched/sched_switch")
int on_switch(struct trace_event_raw_sched_switch *ctx)
{
    __u64 now = bpf_ktime_get_ns(), *last, *acc;
    __u32 zero = 0, prev = (__u32)ctx->prev_pid;

    last = bpf_map_lookup_elem(&last_ts, &zero);   // per-CPU "scheduled-in" timestamp
    if (last && *last) {
        __u64 dt = now - *last;
        acc = bpf_map_lookup_elem(&cpu_ns, &prev);
        if (acc) *acc += dt; else bpf_map_update_elem(&cpu_ns, &prev, &dt, BPF_ANY);
    }
    if (last) *last = now;
    return 0;
}
// user space: drain cpu_ns once per interval — O(active pids), no /proc files

👉 Full, runnable benchmark + both scenarios: hyperredstart/hello-ebpf → proc-vs-ebpf/

Where eBPF's edge is small — and where it's big

eBPF is cheaper either way; the question is by how much.

Smallest edge (real, but not worth a migration on its own):

• Only one or two system-wide gauges. /proc/stat//proc/meminfo are already cheap, so eBPF's lead sits within noise.
• Few processes / low density. The per-process gap is still a fraction of a core until processes pile up.

Biggest payoff:

• High pod/process density. The more processes, the more /proc files polling must scan each scrape, while eBPF only pays for the ones that ran — the gap widens exactly where it hurts.
• Per-process / per-container metrics at scale. This is the node-exporter/cAdvisor cost, multiplied across a fleet. At ~600 processes/node, eBPF saved ~1.1% of a core per node (1.18% → 0.07%); across 1,000 nodes that's ≈ 11 cores reclaimed.
• A caveat I can't skip: if your processes are busy (high context-switch rate), eBPF's sched_switch cost rises too. The win is biggest with many processes but moderate churn — which is what most container fleets look like.

In one line: eBPF is the cheaper model; measure your own collection volume to see how much you gain.

Takeaways

• eBPF reads kernel data without the text-parse tax — it's the more efficient model, and it was cheaper in every test (/proc never won).
• System gauges: eBPF's lead is small (within noise) — not worth migrating for on its own.
• Per-process at density: eBPF wins decisively — O(active pids) in-kernel vs O(all processes) of file I/O, and the gap grows with the number of processes.

Resources

• Full benchmark + both scenarios: hyperredstart/hello-ebpf → proc-vs-ebpf/
• BPF Performance Tools — Brendan Gregg
• ebpf.io

- 中文版（dev.to）：eBPF vs /proc 監控開銷——eBPF 且規模越大，差距越大

How do you currently measure your monitoring overhead? I'm genuinely curious what numbers others see — drop a comment.

This is part of a series on eBPF for traditional-industry / cloud-native systems. Next up, we get hands-on with C + libbpf to trace real syscalls.