High load average on your Linux server? Learn how to diagnose the real cause and fix it fast — with practical commands and clear steps.
You glance at your monitoring dashboard or run a quick top and see a load average of 14 on a 4-core server. Something is wrong — but load average alone doesn't tell you what. It's one of those metrics that looks alarming and yet gives you almost no actionable information on its own. Here's how to actually diagnose what's happening and get your server back to normal.
What Load Average Actually Means
Linux load average is a 1-, 5-, and 15-minute rolling average of the number of processes either running or waiting for CPU or I/O. The key word is 'waiting.' A load of 4.0 on a 4-core machine means your cores are fully saturated — but it doesn't tell you whether processes are waiting for the CPU or waiting on a slow disk. Those two problems have completely different fixes.
A quick rule of thumb: divide the load average by your CPU count. Run nproc to get your core count. If load / cores is above 1.0 for a sustained period, something needs attention. If it's above 2.0, something is actively broken.
Step 1 — Figure Out Whether It's CPU or I/O
Run top and immediately press the key 1 to expand per-CPU stats. Look at the %wa (wait) column. If CPU wait is consistently above 10–20%, your bottleneck is I/O, not CPU. If wait is low but %us (user) or %sy (system) is pegged, you have a CPU problem.
For a clearer picture of I/O, run iostat -xz 2 from the sysstat package. Watch the %util column for each disk. A disk sitting at 90–100% util is saturated. The await column shows average request wait time in milliseconds — anything above 20–30ms for a spinning disk or above a few milliseconds for SSD suggests a bottleneck.
- Install sysstat if needed:
sudo apt install sysstatorsudo yum install sysstat - Run:
iostat -xz 2 5— this samples every 2 seconds, 5 times - Run:
vmstat 2 5— check thebcolumn (blocked processes) andwa(I/O wait) - High
bcount with highwaconfirms I/O saturation
Step 2 — Find the Processes Responsible
Once you know whether it's CPU or I/O, find the culprits. For CPU, top sorted by CPU (default) shows you the worst offenders. For I/O, the picture is harder — top doesn't sort by I/O by default.
Use iotop -o to show only processes with active I/O. This is the fastest way to catch a runaway database vacuum, a backup job gone wrong, or a log rotation task chewing through disk. You'll need root: sudo iotop -o.
Another powerful option is pidstat -u -d 2 — it shows both CPU and I/O stats per process in one view, refreshing every 2 seconds. If you spot a PID causing trouble, get more detail with cat /proc/<PID>/status and ls -l /proc/<PID>/fd to see what files it has open.
- CPU culprits:
top(default), orps aux --sort=-%cpu | head -10 - I/O culprits:
sudo iotop -oorpidstat -d 2 - Identify what a process is doing:
strace -p <PID> -e trace=read,write -c - Check open files:
lsof -p <PID>
Step 3 — Common Causes and Their Fixes
Once you've identified the process, the fix depends on the pattern. Here are the most common scenarios:
- Runaway cron job or backup: check
crontab -land/etc/cron.d/. If a backup is hitting disk hard, consider scheduling it during off-peak hours or usingionice -c3 -p <PID>to lower its I/O priority without killing it. - Database doing heavy work: could be a missing index causing a full table scan, autovacuum (PostgreSQL), or a slow query. Check database slow query logs first before restarting anything.
- Log file explosion: a bug can cause an app to write millions of log lines per second. Check with
du -sh /var/log/*and watch for files growing in real time usingwatch -n1 ls -lh /var/log/myapp/. - Memory pressure causing swap: run
free -h. If swap usage is high and growing, the kernel is paging — which causes I/O wait even when your disk is healthy. The real fix is reducing memory usage or adding RAM, not tuning disk. - Fork bomb or process leak:
ps aux | wc -lto count processes. If it's in the thousands, you may have a process spawning out of control. Set limits in/etc/security/limits.conf(nproc) to prevent recurrence.
Step 4 — After You Fix It, Prevent It
A one-time spike is forgivable. A recurring spike that takes down production is a monitoring problem. Set up alerts on load average — but also on the underlying signals: CPU utilization, disk I/O wait, and memory usage. Alert on the combination, not just one number in isolation.
If you're running a small team without dedicated infrastructure staff, tools like Opservo can help by continuously watching these signals together and explaining in plain English what's happening — so you're not deciphering iostat output at 2am when something breaks. It can also flag patterns like a cron job that spikes load every hour before your users notice.
High load average is a symptom, not a diagnosis. The commands above — iostat, iotop, vmstat, pidstat — are your actual diagnostic tools. Get comfortable running them the moment you see a spike, and you'll cut your resolution time from hours to minutes.
Originally published on the Opservo blog — Opservo is the AI ops engineer for teams without an SRE. Free for 2 servers → https://getopservo.com/welcome
Top comments (0)