DEV Community

jun liu
jun liu

Posted on

Linux Server Feeling Slow? 7 Checks Before You Reboot

#debugging #infrastructure #linux #performance

Linux Server Feeling Slow? 7 Checks Before You Reboot

Every sysadmin has been there: 2 AM, an alert fires, the app is crawling, and your first instinct is to reboot the box. Reboots feel like a reset button, and sometimes they work. But they rarely fix the real problem. If a server slows down every Tuesday at the same hour, rebooting it every Tuesday is not a fix — it is a habit. Before you reach for reboot, run these seven checks. They take about five minutes and they will usually tell you what is actually wrong.

1. Load Average vs. CPU: Is It Busy or Blocked?

Start with uptime. It gives you the one-minute, five-minute, and fifteen-minute load averages.

uptime
# 02:14:33 up 41 days,  3:12,  2 users,  load average: 18.6, 17.2, 15.9
Enter fullscreen mode Exit fullscreen mode

Now compare the load to your core count with nproc. If load is 18 on a 4-core box, something is clearly overcommitted. But load alone does not tell you what is saturated. Open top and look at the %wa (I/O wait) line in the CPU summary.

top -n1 | head -12
Enter fullscreen mode Exit fullscreen mode
  • If %us and %sy are high, the CPUs are genuinely busy running code. Go hunt the process (check 4).
  • If %wa is high (say above 20%), the CPUs are mostly waiting on disk. Reboot will not help; the disk is the bottleneck.
  • If load is high but both CPU and I/O wait are low, the processes may be stuck in an uninterruptible sleep (D state) — often a hung NFS mount or a wedged storage driver.

2. Memory and Swap: Is Swap Actually Being Used?

free -h is quick and tells you a lot.

free -h
Enter fullscreen mode Exit fullscreen mode

A box with 3 GB of swap showing 2.9 GB used is swapping constantly, and every swap-in/swap-out cycle is slow. Reboot frees it temporarily, but if the workload does not fit in RAM, the problem comes back. Check whether the OOM killer has fired recently — that is a much stronger signal than swap usage.

dmesg | grep -i oom | tail -20
Enter fullscreen mode Exit fullscreen mode

If you see repeated Out of memory: Killed process, your app is leaking memory or the heap is misconfigured. The fix is to cap the process (systemd MemoryMax) or fix the leak, not to reboot.

3. Disk Space and Inodes: The Sneaky Full Filesystem

A full disk stops writes silently. A full inode table stops writes too, even when df -h shows plenty of space. Check both.

df -h
df -i
Enter fullscreen mode Exit fullscreen mode

If df -h shows /var at 98%, look for log growth. du -sh /var/log/* | sort -rh | head will reveal the offender. If df -i shows 100% inodes on some mount, you have millions of tiny files — usually a mail queue, a session store, or an app writing one file per request. Clean up the small files or raise the limit; rebooting does not delete them.

4. Find the Process: Top Consumers

Once you suspect CPU or memory pressure, stop guessing and identify the culprit.

top -o %CPU -n1
ps aux --sort=-%mem | head -15
Enter fullscreen mode Exit fullscreen mode

You will usually find one of these:

  • A runaway log writer: a cron job or app spitting a huge log. Check the log size and the write rate; if a process is writing megabytes per second, fix the log level or the loop behind it.
  • A stuck loop: a script with while true and no sleep, or a worker that crashes and restarts in a tight loop. systemctl list-units --state=failed shows the crash loop.
  • A memory leak: a service whose RSS climbs forever. Note the PID, run ps -o rss,vsz,cmd -p <pid>, and watch it over a few minutes. If RSS only ever grows, that is the leak.

5. Disk I/O: Who Is Hammering the Disks?

If %wa was high in check 1, find the process doing the damage.

iostat -x 1 5
iotop -o -n 3
Enter fullscreen mode Exit fullscreen mode

Look for a process with sustained read or write throughput. Common suspects are backup jobs, rsync runs, log rotation, or a database doing a big scan. If a nightly backup collides with the peak traffic window, the answer is scheduling — move the backup to a quiet hour. If it is an unplanned dd or a misbehaving agent, stop it and check its config. Reboot is pointless here because the job will simply start again on the next schedule.

6. Network Connections: Anomalous Connection Counts

A slow server is often a network problem, not a disk problem. Check the socket summary.

ss -s
Enter fullscreen mode Exit fullscreen mode

Compare the total and established counts to your baseline. A sudden jump to tens of thousands of connections usually means one of three things:

  • A half-open connection flood (SYN_RECV) — often a port scan or a SYN flood from outside. Check with ss -s | grep -i syn and consider rate limiting.
  • An application connection leak — a service that opens connections and never closes them. ss -tn state established | awk '{print $4}' | sort | uniq -c | sort -rn | head shows which local port is accumulating sockets.
  • A public internet scan hitting an exposed port. Confirm with ss -tn state established '( sport = :22 )' and decide whether you really need that port open.

7. System Logs: What Is the Kernel Complaining About?

The logs often tell the whole story before you even run the other checks.

journalctl -p err -n 50
Enter fullscreen mode Exit fullscreen mode

Look for repeated patterns, not one-off lines. A recurring I/O error or blk_update_request in dmesg means the disk is failing — that is a hardware replacement, not a reboot. Frequent task blocked for more than 120 seconds warnings point to a hung filesystem or an NFS timeout. If you see a cron job firing every minute and doing heavy work, that is your slowdown.

So, Should You Reboot?

Use this priority order:

  1. Check load, memory, and I/O first. They tell you which subsystem is the bottleneck.
  2. If it is a runaway process or a leak, kill or cap that process. That is a real fix.
  3. If it is disk or storage, reboot only buys you minutes. Replace the failing disk or free the space.
  4. If it is a scheduled job, reschedule it. No reboot involved.
  5. Only reboot when the kernel or a driver is genuinely stuck — for example, an unrecoverable D-state process or a wedged NFS mount that umount -l cannot clear.

A reboot resets every counter and clears every symptom, so it is a great diagnostic tool — but it is not a cure. If you reboot and the server slows down again a day later, the root cause is still there. Treat the cause, not the symptom. The box you save from a pointless 2 AM reboot may be the one that carries your busiest shift tomorrow.

Happy troubleshooting.

Tags: linux, devops, sysadmin, troubleshooting, tutorial

more on jiunotes · ops notes

Top comments (0)