<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: Opservo</title>
    <description>The latest articles on DEV Community by Opservo (@opservo).</description>
    <link>https://dev.to/opservo</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4086465%2F975a67ba-1959-4e5f-9528-b04026f4ae1b.png</url>
      <title>DEV Community: Opservo</title>
      <link>https://dev.to/opservo</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/opservo"/>
    <language>en</language>
    <item>
      <title>How to Monitor Swap Usage on a Linux Server Before It Causes Problems</title>
      <dc:creator>Opservo</dc:creator>
      <pubDate>Tue, 17 Nov 2026 14:18:11 +0000</pubDate>
      <link>https://dev.to/opservo/how-to-monitor-swap-usage-on-a-linux-server-before-it-causes-problems-1bcf</link>
      <guid>https://dev.to/opservo/how-to-monitor-swap-usage-on-a-linux-server-before-it-causes-problems-1bcf</guid>
      <description>&lt;p&gt;Your server is slow. SSH feels laggy, requests are timing out, and the logs are full of cryptic delays. You check CPU — fine. You check disk I/O — fine. Then you check memory and see it: swap usage is at 80% and climbing. By the time swap is doing heavy lifting, your application is already suffering. The trick is catching it before it gets there.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why Swap Usage Is a Warning Sign, Not a Feature
&lt;/h2&gt;

&lt;p&gt;Swap exists as a safety net — it lets the kernel move infrequently used memory pages to disk so active processes can keep running. That sounds helpful, but disk access (even on an NVMe SSD) is orders of magnitude slower than RAM. When your application starts hitting swap regularly, response times degrade visibly and unpredictably. Heavy swap activity — called thrashing — can make a server feel completely unresponsive even when it technically hasn't crashed. The OOM killer stepping in afterward is often the first obvious symptom teams notice, but the real damage happened minutes earlier.&lt;/p&gt;

&lt;h2&gt;
  
  
  The Commands You Actually Need
&lt;/h2&gt;

&lt;p&gt;Start with the basics. The &lt;code&gt;free -h&lt;/code&gt; command gives you a clean overview of physical memory and swap in human-readable units:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;free -h&lt;/p&gt;

&lt;p&gt;Check the Swap row. If 'used' is anything above zero, something has been pushed out of RAM. If it's above 20–30% of total swap, dig deeper immediately.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;For a real-time view of swap activity, &lt;code&gt;vmstat&lt;/code&gt; is more useful than a one-shot snapshot. Run it with an interval to see what's actually happening over time:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;vmstat 2 10&lt;/p&gt;

&lt;p&gt;Watch the 'si' (swap in) and 'so' (swap out) columns. Non-zero values here mean active swapping is happening right now. Even small, persistent numbers are a red flag.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;To find out which processes are responsible for swap consumption, use this one-liner that reads from &lt;code&gt;/proc&lt;/code&gt;:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;for pid in /proc/[0-9]*; do awk '/VmSwap/{print $2}' $pid/status 2&amp;gt;/dev/null | grep -v '^0$' | xargs -I{} echo "$pid: {} kB"; done | sort -t: -k2 -rn | head -20&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;This gives you a ranked list of which processes are sitting in swap, so you're not guessing. You can also use &lt;code&gt;smem&lt;/code&gt; if it's available on your distro — &lt;code&gt;smem -s swap -r&lt;/code&gt; does the same job with cleaner output.&lt;/p&gt;

&lt;h2&gt;
  
  
  Setting Thresholds That Actually Matter
&lt;/h2&gt;

&lt;p&gt;Not all swap usage is equally alarming. Context matters. A baseline of a few hundred megabytes of swap in use on a server with 32 GB RAM might be harmless — the kernel sometimes moves cold pages proactively. What you're really watching for are trends and activity rates, not just absolute numbers.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Swap used &amp;gt; 10% of total: worth noting, investigate if unexpected.&lt;/li&gt;
&lt;li&gt;Swap used &amp;gt; 25% of total: alert — memory pressure is real, find the cause.&lt;/li&gt;
&lt;li&gt;Swap used &amp;gt; 50% of total: urgent — performance is likely already degraded.&lt;/li&gt;
&lt;li&gt;si or so in vmstat consistently &amp;gt; 0: active swapping, fix this now regardless of percentage.&lt;/li&gt;
&lt;li&gt;Swap usage growing over hours without leveling off: a memory leak is probable.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;The swappiness kernel parameter (&lt;code&gt;/proc/sys/vm/swappiness&lt;/code&gt;, default 60) controls how aggressively the kernel offloads pages to swap. For application servers, lowering this to 10 or even 1 tells the kernel to exhaust RAM before touching swap. You can change it live with &lt;code&gt;sysctl vm.swappiness=10&lt;/code&gt; and make it permanent in &lt;code&gt;/etc/sysctl.conf&lt;/code&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  Building a Simple Alerting Loop
&lt;/h2&gt;

&lt;p&gt;If you're not running a full monitoring stack yet, a cron-based script can cover the basics. Drop something like this into your crontab to get an email when swap crosses a threshold:&lt;/p&gt;

&lt;blockquote&gt;
&lt;h1&gt;
  
  
  !/bin/bash
&lt;/h1&gt;

&lt;p&gt;SWAP_USED=$(free | awk '/Swap/{print $3}')&lt;br&gt;
SWAP_TOTAL=$(free | awk '/Swap/{print $2}')&lt;br&gt;
if [ "$SWAP_TOTAL" -gt 0 ]; then&lt;br&gt;
  PCT=$((SWAP_USED * 100 / SWAP_TOTAL))&lt;br&gt;
  if [ "$PCT" -gt 25 ]; then&lt;br&gt;
    echo "Swap at ${PCT}% on $(hostname)" | mail -s "Swap Alert" &lt;a href="mailto:you@example.com"&gt;you@example.com&lt;/a&gt;&lt;br&gt;
  fi&lt;br&gt;
fi&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;Run this every five minutes via cron. It's not sophisticated, but it will catch a swapping event before your users do. If you're already using Prometheus, the &lt;code&gt;node_exporter&lt;/code&gt; exposes &lt;code&gt;node_memory_SwapFree_bytes&lt;/code&gt; and &lt;code&gt;node_memory_SwapTotal_bytes&lt;/code&gt; — build an alert rule off those instead, and add a second rule on &lt;code&gt;node_vmstat_pswpin&lt;/code&gt; and &lt;code&gt;node_vmstat_pswpout&lt;/code&gt; to catch active swapping even when absolute usage looks low.&lt;/p&gt;

&lt;h2&gt;
  
  
  What to Do When Swap Is Already High
&lt;/h2&gt;

&lt;p&gt;If you arrive at a server mid-incident with high swap, your priority is identifying the offending process and deciding whether to kill it, restart it, or add RAM. Use the &lt;code&gt;/proc&lt;/code&gt; loop above to find the top swap consumers. If you have headroom in physical RAM, you can also run &lt;code&gt;swapoff -a &amp;amp;&amp;amp; swapon -a&lt;/code&gt; to force all pages back into RAM — but only attempt this if &lt;code&gt;free -h&lt;/code&gt; shows you have enough free memory to absorb them. Forcing a swapoff with insufficient free RAM will cause the OOM killer to start terminating processes, which is rarely what you want mid-incident.&lt;/p&gt;

&lt;p&gt;Long term, persistent swap usage almost always signals one of three things: a process with a memory leak, a workload that has outgrown the server's RAM, or a misconfigured application allocating more than it needs. Solving the symptom by adding swap space just delays the reckoning.&lt;/p&gt;

&lt;p&gt;If you want swap monitoring handled automatically without scripting it yourself, Opservo watches memory and swap trends continuously and will surface warnings in plain English before usage reaches a critical level — useful when you're running production without a dedicated ops person watching dashboards all day.&lt;/p&gt;

&lt;p&gt;Either way, the core habit is the same: treat non-zero swap activity as a signal worth investigating, not a normal background condition. Catching it early — when you still have time to think — is the difference between a quiet fix and a 2 a.m. outage.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;Invite the reader to connect their server to Opservo's free 2-server plan to get automatic swap and memory trend alerts without writing cron scripts. — one command installs a read-only agent and your first findings arrive in five minutes: &lt;a href="https://getopservo.com/signup?utm_source=blog&amp;amp;utm_medium=post&amp;amp;utm_campaign=auto-blog" rel="noopener noreferrer"&gt;start free&lt;/a&gt;. Related: &lt;a href="https://getopservo.com/guides/why-is-my-server-slow" rel="noopener noreferrer"&gt;guides why is my server slow&lt;/a&gt;.&lt;/p&gt;
&lt;/blockquote&gt;




&lt;p&gt;&lt;em&gt;Originally published on the &lt;a href="https://getopservo.com/blog/how-to-monitor-swap-usage-on-a-linux-server-before-it-causes-problems?utm_source=devto&amp;amp;utm_medium=syndication&amp;amp;utm_campaign=syndication" rel="noopener noreferrer"&gt;Opservo blog&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>linux</category>
      <category>devops</category>
      <category>monitoring</category>
      <category>operations</category>
    </item>
    <item>
      <title>SSL certificate expiry: a practical guide for small teams</title>
      <dc:creator>Opservo</dc:creator>
      <pubDate>Sun, 15 Nov 2026 09:36:06 +0000</pubDate>
      <link>https://dev.to/opservo/ssl-certificate-expiry-a-practical-guide-for-small-teams-1k25</link>
      <guid>https://dev.to/opservo/ssl-certificate-expiry-a-practical-guide-for-small-teams-1k25</guid>
      <description>&lt;p&gt;An expired SSL certificate takes your site offline for everyone, instantly, with no warning to users beyond a browser error screen. For small teams running production without a dedicated SRE, it is one of the most avoidable outages there is — and yet it still catches people out every week. This guide explains how SSL certificates expire, how to check them from the command line, what thresholds to alert on, and how to keep a watch across more than a handful of domains without it becoming a manual job.&lt;/p&gt;

&lt;h2&gt;
  
  
  How SSL certificate expiry actually works
&lt;/h2&gt;

&lt;p&gt;Every certificate is issued with a Not After date baked into it. When a browser or API client connects to your server, it checks that date before anything else. If today is past that date, the handshake fails and the connection is refused. There is no grace period. The certificate does not degrade; it simply stops working.&lt;/p&gt;

&lt;p&gt;Let's Encrypt certificates last 90 days. Certificates from commercial CAs typically last one year (since September 2020, browsers cap the maximum at 398 days). Both renew automatically in theory — Let's Encrypt via certbot or acme.sh, commercial certs via your CA's renewal tooling — but automation fails silently more often than teams expect: a cron job that stopped running, a firewall rule that blocks the ACME challenge, a certificate stored outside the renewal path, or a domain that moved to a new server without its cert following.&lt;/p&gt;

&lt;h2&gt;
  
  
  How to check SSL certificate expiry from the command line
&lt;/h2&gt;

&lt;p&gt;You do not need any special tool. OpenSSL is installed on almost every Linux server, and a single command gives you the full expiry date for any domain reachable from that machine.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="c"&gt;# Check expiry date for a domain&lt;/span&gt;
&lt;span class="nb"&gt;echo&lt;/span&gt; &lt;span class="o"&gt;|&lt;/span&gt; openssl s_client -connect example.com:443 -servername example.com 2&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;/dev/null \
  &lt;span class="o"&gt;|&lt;/span&gt; openssl x509 -noout -dates

&lt;span class="c"&gt;# Expected output:&lt;/span&gt;
&lt;span class="c"&gt;# notBefore=Apr  1 00:00:00 2025 GMT&lt;/span&gt;
&lt;span class="c"&gt;# notAfter=Jun 30 23:59:59 2025 GMT&lt;/span&gt;

&lt;span class="c"&gt;# Check how many days remain (Linux)&lt;/span&gt;
&lt;span class="nb"&gt;echo&lt;/span&gt; &lt;span class="o"&gt;|&lt;/span&gt; openssl s_client -connect example.com:443 -servername example.com 2&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;/dev/null \
  &lt;span class="o"&gt;|&lt;/span&gt; openssl x509 -noout -enddate \
  &lt;span class="o"&gt;|&lt;/span&gt; cut -d= -f2 \
  &lt;span class="o"&gt;|&lt;/span&gt; xargs -I&lt;span class="o"&gt;{&lt;/span&gt;&lt;span class="o"&gt;}&lt;/span&gt; date -d &lt;span class="o"&gt;{&lt;/span&gt;&lt;span class="o"&gt;}&lt;/span&gt; +%s \
  &lt;span class="o"&gt;|&lt;/span&gt; xargs -I&lt;span class="o"&gt;{&lt;/span&gt;&lt;span class="o"&gt;}&lt;/span&gt; bash -c &lt;span class="s1"&gt;'echo $(( ({} - $(date +%s)) / 86400 )) days remaining'&lt;/span&gt;

&lt;span class="c"&gt;# Check the cert served by nginx or Apache locally (no network needed)&lt;/span&gt;
openssl x509 -noout -dates -in /etc/letsencrypt/live/example.com/fullchain.pem

&lt;span class="c"&gt;# Quick check: does the cert chain include intermediates?&lt;/span&gt;
&lt;span class="nb"&gt;echo&lt;/span&gt; &lt;span class="o"&gt;|&lt;/span&gt; openssl s_client -connect example.com:443 -servername example.com 2&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;/dev/null \
  &lt;span class="o"&gt;|&lt;/span&gt; openssl x509 -noout -issuer -subject
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The -servername flag matters. It sends the SNI header so the server returns the correct certificate when multiple domains share one IP address. Without it, you may get the default certificate rather than the one you are checking.&lt;/p&gt;

&lt;p&gt;If you want to check a certificate file on disk rather than a live connection — useful after a renewal before you reload the web server — read it directly with openssl x509 -in as shown above. This confirms the file renewed correctly before nginx or Apache picks it up.&lt;/p&gt;

&lt;h2&gt;
  
  
  What to alert on: concrete thresholds
&lt;/h2&gt;

&lt;p&gt;Knowing your cert expires in 14 days is useful. Getting that information while you are still asleep is what saves you. These are the thresholds most teams settle on after one incident:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;30 days: warning alert. Renewal has not happened yet. Investigate why. For Let's Encrypt this should never trigger unless automation is broken.&lt;/li&gt;
&lt;li&gt;14 days: urgent alert. Treat this as a production issue. Manual renewal may be needed.&lt;/li&gt;
&lt;li&gt;7 days: critical alert. Page someone. At this point a renewal failure is very likely to cause an outage.&lt;/li&gt;
&lt;li&gt;0 days (expired): incident. The cert is already failing for end users.&lt;/li&gt;
&lt;/ul&gt;

&lt;blockquote&gt;
&lt;p&gt;Rule of thumb: if you are not getting a 30-day alert and your renewal automation is working, the system is healthy. If you do get a 30-day alert, do not dismiss it — that is the automation telling you it failed.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;For teams using Let's Encrypt, certbot renews at 30 days by default, so a 30-day alert firing means the renewal run failed or never ran. Check systemctl status certbot.timer and the renewal logs at /var/log/letsencrypt/letsencrypt.log before anything else.&lt;/p&gt;

&lt;h2&gt;
  
  
  Common reasons a certificate expires even with automation
&lt;/h2&gt;

&lt;p&gt;Automation does not eliminate expiry incidents; it just changes what goes wrong. The failures teams run into most often are:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;The ACME HTTP-01 challenge is blocked. A firewall rule, CDN setting, or redirect loop prevents Let's Encrypt from reaching /.well-known/acme-challenge/. The renewal runs, fails silently, and no one notices until the cert is two weeks from expiry.&lt;/li&gt;
&lt;li&gt;The domain moved. A site migrated to a new server or a different web root, but the cert (and the renewal hook) stayed on the old one.&lt;/li&gt;
&lt;li&gt;Post-renewal reload never ran. certbot renewed the files but the nginx or Apache reload in the deploy-hook failed, so the server is still serving the old expired cert from memory.&lt;/li&gt;
&lt;li&gt;Wildcard certs and DNS-01 challenges. These require DNS API access. Credentials rotate, IAM policies change, and the renewal quietly fails.&lt;/li&gt;
&lt;li&gt;Commercial certs managed manually. If someone bought a one-year cert and put a reminder in a calendar, that reminder gets missed or lands on a day when the right person is on leave.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;For the reload problem specifically: after any renewal, verify the live cert matches the file. Run the openssl s_client command against the live domain and compare the notAfter date to the file on disk. If they differ, the server has not picked up the new cert yet.&lt;/p&gt;

&lt;h2&gt;
  
  
  SSL certificate expiry monitoring across multiple domains
&lt;/h2&gt;

&lt;p&gt;Checking one domain with OpenSSL takes seconds. Checking twenty domains reliably, on a schedule, with alerts going to the right person — that is where manual checks break down. If you are evaluating how to handle this across a fleet, &lt;a href="https://getopservo.com/blog/monitor-ssl-certificate-expiry" rel="noopener noreferrer"&gt;SSL certificate expiry monitoring: get an alert before a cert expires&lt;/a&gt; covers the monitoring options in detail, including what a good alerting setup looks like for teams without a dedicated ops person.&lt;/p&gt;

&lt;p&gt;For teams deciding how to structure their overall monitoring setup, &lt;a href="https://getopservo.com/guides/best-server-monitoring-for-small-teams" rel="noopener noreferrer"&gt;what is the best server monitoring for small teams?&lt;/a&gt; puts SSL monitoring in context alongside uptime checks, disk alerts, and service health — which is usually the right way to think about it, since an SSL expiry is one of several things that can silently break production.&lt;/p&gt;

&lt;p&gt;If you are running a cron job to check certs, the basic pattern is: for each domain, run the openssl s_client command, extract the days remaining, and send an alert if the number is below your threshold. Store the results somewhere so you have a history. The fragile part is not the check itself but making sure the checker itself keeps running and that alert delivery does not silently break.&lt;/p&gt;

&lt;h2&gt;
  
  
  How Opservo handles SSL expiry monitoring
&lt;/h2&gt;

&lt;p&gt;Opservo auto-discovers nginx and Apache vhosts on your server and monitors their SSL expiry as part of the standard install. You get alerts at configurable thresholds without writing a cron job or maintaining a separate checker. The same agent also monitors uptime, service health, disk fill forecasts, and open ports, so SSL expiry sits alongside the other things that break production rather than in a separate dashboard. The install is one command and takes about 30 seconds:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;curl -sSL https://api.getopservo.com/install.sh &lt;span class="o"&gt;|&lt;/span&gt; sudo bash
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The free plan covers two servers with no card required, which is enough to see whether the auto-discovery picks up your vhosts and what the first SSL findings look like before you commit to anything.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;Invite the reader to try Opservo's free two-server plan to see SSL expiry auto-discovered alongside uptime and service health in a single agent install. — one command installs a read-only agent and your first findings arrive in five minutes: &lt;a href="https://getopservo.com/signup?utm_source=blog&amp;amp;utm_medium=post&amp;amp;utm_campaign=auto-blog" rel="noopener noreferrer"&gt;start free&lt;/a&gt;. Related: &lt;a href="https://getopservo.com/blog/monitor-ssl-certificate-expiry" rel="noopener noreferrer"&gt;blog monitor ssl certificate expiry&lt;/a&gt;.&lt;/p&gt;
&lt;/blockquote&gt;




&lt;p&gt;&lt;em&gt;Originally published on the &lt;a href="https://getopservo.com/blog/ssl-certificate-expiry-a-practical-guide-for-small-teams?utm_source=devto&amp;amp;utm_medium=syndication&amp;amp;utm_campaign=syndication" rel="noopener noreferrer"&gt;Opservo blog&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>linux</category>
      <category>devops</category>
      <category>monitoring</category>
      <category>operations</category>
    </item>
    <item>
      <title>Ubuntu unattended-upgrades: security patches without surprise reboots</title>
      <dc:creator>Opservo</dc:creator>
      <pubDate>Tue, 10 Nov 2026 14:11:08 +0000</pubDate>
      <link>https://dev.to/opservo/ubuntu-unattended-upgrades-security-patches-without-surprise-reboots-j38</link>
      <guid>https://dev.to/opservo/ubuntu-unattended-upgrades-security-patches-without-surprise-reboots-j38</guid>
      <description>&lt;p&gt;If you run Ubuntu servers without a dedicated ops team, you face a quiet dilemma: apply security patches manually and fall weeks behind, or turn on automatic updates and worry that a reboot will drop your app at 2 am. Ubuntu's unattended-upgrades package solves the first problem without forcing the second, but only if you configure it properly. This guide shows you exactly what to install, what to change, and what to watch so automatic patching stays under control.&lt;/p&gt;

&lt;h2&gt;
  
  
  What unattended-upgrades actually does
&lt;/h2&gt;

&lt;p&gt;unattended-upgrades is an apt wrapper that runs on a timer, checks a defined list of package origins, downloads and installs matching updates, and optionally cleans up old kernels. Out of the box on Ubuntu 20.04 and later it is installed but not always fully enabled. It applies only packages from the &lt;code&gt;Ubuntu:focal-security&lt;/code&gt; origin (or your release equivalent), not every pending update. That means your application packages stay pinned unless you explicitly add more origins.&lt;/p&gt;

&lt;p&gt;Reboots are never automatic unless you tell it so. The package can write a file to signal that a reboot is needed, but it will not reboot the machine until you configure a reboot time. That is the setting most people miss, and it is why their servers accumulate pending reboots for months.&lt;/p&gt;

&lt;h2&gt;
  
  
  Install and enable it
&lt;/h2&gt;

&lt;p&gt;First check whether the package is already installed and whether the apt-daily timers are running.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="c"&gt;# Check if already installed&lt;/span&gt;
dpkg -l unattended-upgrades

&lt;span class="c"&gt;# Check the systemd timers that drive it&lt;/span&gt;
systemctl status apt-daily.timer apt-daily-upgrade.timer

&lt;span class="c"&gt;# Install if missing&lt;/span&gt;
sudo apt install unattended-upgrades apt-listchanges -y

&lt;span class="c"&gt;# Enable and start&lt;/span&gt;
sudo dpkg-reconfigure -plow unattended-upgrades
&lt;span class="c"&gt;# Answer 'Yes' when prompted&lt;/span&gt;

&lt;span class="c"&gt;# Confirm the service file was written&lt;/span&gt;
cat /etc/apt/apt.conf.d/20auto-upgrades
&lt;span class="c"&gt;# Expected output:&lt;/span&gt;
&lt;span class="c"&gt;# APT::Periodic::Update-Package-Lists "1";&lt;/span&gt;
&lt;span class="c"&gt;# APT::Periodic::Unattended-Upgrade "1";&lt;/span&gt;

&lt;span class="c"&gt;# Run a dry-run to see what would be upgraded today&lt;/span&gt;
sudo unattended-upgrade --dry-run --debug 2&lt;span class="o"&gt;&amp;gt;&amp;amp;&lt;/span&gt;1 &lt;span class="o"&gt;|&lt;/span&gt; grep -E &lt;span class="s1"&gt;'Packages|upgradeable|error'&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The &lt;code&gt;--dry-run&lt;/code&gt; pass is worth running every time you change configuration. It prints which packages match, which are held back, and whether any errors occur, without touching anything.&lt;/p&gt;

&lt;h2&gt;
  
  
  The settings that actually matter
&lt;/h2&gt;

&lt;p&gt;The main configuration file is &lt;code&gt;/etc/apt/apt.conf.d/50unattended-upgrades&lt;/code&gt;. Open it and look for three sections.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Allowed origins.&lt;/strong&gt; The default block includes &lt;code&gt;${distro_id}:${distro_codename}-security&lt;/code&gt;. Leave that in. If you also want bug-fix updates from &lt;code&gt;-updates&lt;/code&gt;, uncomment that line too. Do not uncomment &lt;code&gt;-proposed&lt;/code&gt; or &lt;code&gt;-backports&lt;/code&gt; on production machines.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Packages to never touch.&lt;/strong&gt; The &lt;code&gt;Unattended-Upgrade::Package-Blacklist&lt;/code&gt; list accepts exact names and regular expressions. Add anything you deploy yourself and anything with a known migration risk, for example: &lt;code&gt;"mysql-server", "nginx", "php.*"&lt;/code&gt;. Held packages (those pinned with apt-mark hold) are automatically skipped regardless.&lt;/p&gt;

&lt;p&gt;&lt;strong&gt;Reboot behaviour.&lt;/strong&gt; Three settings control this:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;code&gt;Unattended-Upgrade::Automatic-Reboot "false";&lt;/code&gt; -- set to &lt;code&gt;"true"&lt;/code&gt; only if a scheduled reboot is acceptable.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;Unattended-Upgrade::Automatic-Reboot-WithUsers "false";&lt;/code&gt; -- when true, reboots even if users are logged in. Leave false.&lt;/li&gt;
&lt;li&gt;
&lt;code&gt;Unattended-Upgrade::Automatic-Reboot-Time "03:00";&lt;/code&gt; -- the time window used if automatic reboots are on. Pick your quietest hour.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;If you keep &lt;code&gt;Automatic-Reboot false&lt;/code&gt;, you are responsible for rebooting after a kernel or libc update. The file &lt;code&gt;/var/run/reboot-required&lt;/code&gt; appears when a reboot is pending. Checking for that file in a nightly alert is enough to stay on top of it.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;Rule of thumb: let unattended-upgrades handle security origins automatically; reboot on your schedule during a maintenance window rather than letting the machine decide.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;h2&gt;
  
  
  Reading the logs to confirm it is working
&lt;/h2&gt;

&lt;p&gt;unattended-upgrades writes to &lt;code&gt;/var/log/unattended-upgrades/unattended-upgrades.log&lt;/code&gt;. A successful run looks like this:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="c"&gt;# Tail the last 30 lines after the daily timer fires (~6 am by default)&lt;/span&gt;
sudo tail -30 /var/log/unattended-upgrades/unattended-upgrades.log

&lt;span class="c"&gt;# Lines to look for:&lt;/span&gt;
&lt;span class="c"&gt;# INFO Starting unattended upgrades script&lt;/span&gt;
&lt;span class="c"&gt;# INFO Packages that will be upgraded: libssl3 openssl&lt;/span&gt;
&lt;span class="c"&gt;# INFO All upgrades installed&lt;/span&gt;

&lt;span class="c"&gt;# If something failed:&lt;/span&gt;
sudo grep -i &lt;span class="s1"&gt;'error\|warning\|failed'&lt;/span&gt; /var/log/unattended-upgrades/unattended-upgrades.log &lt;span class="o"&gt;|&lt;/span&gt; tail -20

&lt;span class="c"&gt;# Check whether a reboot is currently pending&lt;/span&gt;
&lt;span class="nb"&gt;test&lt;/span&gt; -f /var/run/reboot-required &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; &lt;span class="nb"&gt;echo&lt;/span&gt; &lt;span class="s1"&gt;'REBOOT NEEDED'&lt;/span&gt; &lt;span class="o"&gt;||&lt;/span&gt; &lt;span class="nb"&gt;echo&lt;/span&gt; &lt;span class="s1"&gt;'No reboot pending'&lt;/span&gt;

&lt;span class="c"&gt;# See which packages triggered it&lt;/span&gt;
cat /var/run/reboot-required.pkgs 2&lt;span class="o"&gt;&amp;gt;&lt;/span&gt;/dev/null
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;If the log file does not exist or has not been written in the last 24 hours, the timer is probably not running. Check &lt;code&gt;systemctl list-timers apt-daily-upgrade.timer&lt;/code&gt; and look at the NEXT and LAST columns.&lt;/p&gt;

&lt;h2&gt;
  
  
  What to alert on
&lt;/h2&gt;

&lt;p&gt;Automatic patching creates a new class of thing to watch: silent failures. The package runs unattended, so you will not notice if it stops working until a CVE lands on an unpatched machine. Set these checks:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;
&lt;strong&gt;Reboot pending for more than 7 days.&lt;/strong&gt; &lt;code&gt;/var/run/reboot-required&lt;/code&gt; exists and its modification time is older than 7 days. Anything beyond that is a kernel update sitting unapplied.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Log file not updated in 48 hours.&lt;/strong&gt; If &lt;code&gt;/var/log/unattended-upgrades/unattended-upgrades.log&lt;/code&gt; has not changed in 48 hours, the daily timer has stalled.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;'error' or 'failed' in the log.&lt;/strong&gt; A grep on the log file at the end of each day catches broken apt locks, missing keys, or held dependencies that blocked the run.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Available security updates older than 24 hours.&lt;/strong&gt; Run &lt;code&gt;apt-get -s upgrade 2&amp;gt;/dev/null | grep -c '^Inst'&lt;/code&gt; to count pending upgrades. If this is non-zero and the package is supposed to be handling them, something is held back that you need to review.&lt;/li&gt;
&lt;li&gt;
&lt;strong&gt;Disk space below 20% on &lt;code&gt;/&lt;/code&gt;.&lt;/strong&gt; apt downloads and unpacks packages before removing the old version. A full disk is the most common cause of a mid-upgrade failure.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;These thresholds are conservative. A reboot pending for one day is fine; pending for two weeks is a risk. An upgrade log that stalled for 24 hours might just be a quiet patch day; 48 hours with no entry usually means a broken timer.&lt;/p&gt;

&lt;h2&gt;
  
  
  How Opservo watches this for you
&lt;/h2&gt;

&lt;p&gt;Checking five log files and modification times across a fleet of servers by hand does not scale. Opservo, the AI ops engineer for teams without an SRE, surfaces pending reboots, storage pressure and service failures as plain-English alerts with one-click fixes. Its Health Standard scores each server across Storage, Maintenance and Reliability and names the exact rule and current value behind every lost point, so you know whether a reboot-required file is new or has been sitting there for two weeks. The agent installs in about 30 seconds with one command and opens no inbound ports, so adding it to a production machine carries no exposure.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;Invite the reader to install the Opservo agent (one command, free for 2 servers, no card) to get automatic alerts for pending reboots, stalled upgrade timers, and low disk space across their fleet. — one command installs a read-only agent and your first findings arrive in five minutes: &lt;a href="https://getopservo.com/signup?utm_source=blog&amp;amp;utm_medium=post&amp;amp;utm_campaign=auto-blog" rel="noopener noreferrer"&gt;start free&lt;/a&gt;. Related: &lt;a href="https://getopservo.com/guides/how-to-check-disk-space-on-linux" rel="noopener noreferrer"&gt;guides how to check disk space on linux&lt;/a&gt;.&lt;/p&gt;
&lt;/blockquote&gt;




&lt;p&gt;&lt;em&gt;Originally published on the &lt;a href="https://getopservo.com/blog/ubuntu-unattended-upgrades-security-patches-without-surprise-reboots?utm_source=devto&amp;amp;utm_medium=syndication&amp;amp;utm_campaign=syndication" rel="noopener noreferrer"&gt;Opservo blog&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>linux</category>
      <category>devops</category>
      <category>monitoring</category>
      <category>operations</category>
    </item>
    <item>
      <title>How to monitor SSL certificate expiration</title>
      <dc:creator>Opservo</dc:creator>
      <pubDate>Sun, 08 Nov 2026 08:57:21 +0000</pubDate>
      <link>https://dev.to/opservo/how-to-monitor-ssl-certificate-expiration-4f83</link>
      <guid>https://dev.to/opservo/how-to-monitor-ssl-certificate-expiration-4f83</guid>
      <description>&lt;p&gt;An expired SSL certificate breaks your site for every visitor, kills API calls, and can take a Laravel app or e-commerce checkout completely offline in seconds. It happens to careful teams too, because certificates quietly tick down and the renewal reminder arrives in an inbox nobody checks. This guide shows you how to check SSL certificate expiration from the command line, what thresholds to alert on, and how to automate ssl expiry monitoring so the expiry date never sneaks past you.&lt;/p&gt;

&lt;h2&gt;
  
  
  How to check an SSL certificate expiration date manually
&lt;/h2&gt;

&lt;p&gt;The fastest check is a one-liner with openssl. It connects to the server on port 443 and prints the certificate's notAfter field.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="c"&gt;# Check expiry for a live domain&lt;/span&gt;
&lt;span class="nb"&gt;echo&lt;/span&gt; | openssl s_client &lt;span class="nt"&gt;-servername&lt;/span&gt; example.com &lt;span class="nt"&gt;-connect&lt;/span&gt; example.com:443 2&amp;gt;/dev/null &lt;span class="se"&gt;\&lt;/span&gt;
  | openssl x509 &lt;span class="nt"&gt;-noout&lt;/span&gt; &lt;span class="nt"&gt;-dates&lt;/span&gt;

&lt;span class="c"&gt;# Expected output:&lt;/span&gt;
&lt;span class="c"&gt;# notBefore=Apr  1 00:00:00 2024 GMT&lt;/span&gt;
&lt;span class="c"&gt;# notAfter=Apr  1 23:59:59 2025 GMT&lt;/span&gt;

&lt;span class="c"&gt;# Check days remaining (prints a number; negative means already expired)&lt;/span&gt;
&lt;span class="nb"&gt;echo&lt;/span&gt; | openssl s_client &lt;span class="nt"&gt;-servername&lt;/span&gt; example.com &lt;span class="nt"&gt;-connect&lt;/span&gt; example.com:443 2&amp;gt;/dev/null &lt;span class="se"&gt;\&lt;/span&gt;
  | openssl x509 &lt;span class="nt"&gt;-noout&lt;/span&gt; &lt;span class="nt"&gt;-enddate&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
  | &lt;span class="nb"&gt;awk&lt;/span&gt; &lt;span class="nt"&gt;-F&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="s1"&gt;'{print $2}'&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
  | xargs &lt;span class="nt"&gt;-I&lt;/span&gt;&lt;span class="o"&gt;{}&lt;/span&gt; &lt;span class="nb"&gt;date&lt;/span&gt; &lt;span class="nt"&gt;-d&lt;/span&gt; &lt;span class="s2"&gt;"{}"&lt;/span&gt; +%s &lt;span class="se"&gt;\&lt;/span&gt;
  | xargs &lt;span class="nt"&gt;-I&lt;/span&gt;&lt;span class="o"&gt;{}&lt;/span&gt; sh &lt;span class="nt"&gt;-c&lt;/span&gt; &lt;span class="s1"&gt;'echo $(( ({} - $(date +%s)) / 86400 )) days remaining'&lt;/span&gt;

&lt;span class="c"&gt;# Check a certificate file on disk instead&lt;/span&gt;
openssl x509 &lt;span class="nt"&gt;-noout&lt;/span&gt; &lt;span class="nt"&gt;-dates&lt;/span&gt; &lt;span class="nt"&gt;-in&lt;/span&gt; /etc/letsencrypt/live/example.com/cert.pem
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The -servername flag matters. Without it, openssl sends no SNI header and a server hosting multiple domains may return the wrong certificate. Always include it when you are checking a shared host or a CDN origin.&lt;/p&gt;

&lt;p&gt;If the command hangs, the host is likely blocking port 443 from your machine or the DNS does not resolve. Check with curl -v &lt;a href="https://example.com" rel="noopener noreferrer"&gt;https://example.com&lt;/a&gt; first and confirm the IP is reachable.&lt;/p&gt;

&lt;h2&gt;
  
  
  What to alert on: concrete thresholds for ssl certificate expiry monitoring
&lt;/h2&gt;

&lt;p&gt;There is no single right threshold, but these three levels cover most teams:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;30 days: send a Slack or email notice. This is plenty of time for a manual renewal or to investigate why auto-renew did not run.&lt;/li&gt;
&lt;li&gt;14 days: escalate the alert. If certbot or acme.sh has been failing silently, you still have time to fix it before users notice.&lt;/li&gt;
&lt;li&gt;3 days: wake someone up. At this point auto-renewal has clearly failed and you need a human to act immediately.&lt;/li&gt;
&lt;li&gt;0 days (or negative): the certificate is already expired. Treat this as an outage, not a warning.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;For internal APIs and staging environments the same thresholds apply. A staging cert expiring mid-sprint blocks every developer on the team, which is just as disruptive as a production outage.&lt;/p&gt;

&lt;p&gt;If you manage multiple sites, the manual check above does not scale. The next section covers how to run it across a fleet.&lt;/p&gt;

&lt;h2&gt;
  
  
  How to script ssl certificate monitoring for multiple domains
&lt;/h2&gt;

&lt;p&gt;A simple shell script lets you check every domain in a list and print any that expire within your chosen threshold. Save the list in a plain text file, one domain per line, and run the script from cron.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="c"&gt;#!/usr/bin/env bash&lt;/span&gt;
&lt;span class="c"&gt;# ssl-check.sh — alert when a cert expires within WARN_DAYS&lt;/span&gt;
&lt;span class="nv"&gt;WARN_DAYS&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;30
&lt;span class="nv"&gt;DOMAINS_FILE&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;"/etc/ssl-monitor/domains.txt"&lt;/span&gt;

&lt;span class="k"&gt;while &lt;/span&gt;&lt;span class="nv"&gt;IFS&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nb"&gt;read&lt;/span&gt; &lt;span class="nt"&gt;-r&lt;/span&gt; domain&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="k"&gt;do&lt;/span&gt;
  &lt;span class="o"&gt;[&lt;/span&gt; &lt;span class="nt"&gt;-z&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="nv"&gt;$domain&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt; &lt;span class="o"&gt;]&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; &lt;span class="k"&gt;continue
  &lt;/span&gt;&lt;span class="nv"&gt;expiry&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="si"&gt;$(&lt;/span&gt;&lt;span class="nb"&gt;echo&lt;/span&gt; | openssl s_client &lt;span class="nt"&gt;-servername&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="nv"&gt;$domain&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
    &lt;span class="nt"&gt;-connect&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="k"&gt;${&lt;/span&gt;&lt;span class="nv"&gt;domain&lt;/span&gt;&lt;span class="k"&gt;}&lt;/span&gt;&lt;span class="s2"&gt;:443"&lt;/span&gt; 2&amp;gt;/dev/null &lt;span class="se"&gt;\&lt;/span&gt;
    | openssl x509 &lt;span class="nt"&gt;-noout&lt;/span&gt; &lt;span class="nt"&gt;-enddate&lt;/span&gt; 2&amp;gt;/dev/null &lt;span class="se"&gt;\&lt;/span&gt;
    | &lt;span class="nb"&gt;cut&lt;/span&gt; &lt;span class="nt"&gt;-d&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nt"&gt;-f2&lt;/span&gt;&lt;span class="si"&gt;)&lt;/span&gt;

  &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="o"&gt;[&lt;/span&gt; &lt;span class="nt"&gt;-z&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="nv"&gt;$expiry&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt; &lt;span class="o"&gt;]&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="k"&gt;then
    &lt;/span&gt;&lt;span class="nb"&gt;echo&lt;/span&gt; &lt;span class="s2"&gt;"WARN: could not retrieve cert for &lt;/span&gt;&lt;span class="nv"&gt;$domain&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt;
    &lt;span class="k"&gt;continue
  fi

  &lt;/span&gt;&lt;span class="nv"&gt;expiry_epoch&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="si"&gt;$(&lt;/span&gt;&lt;span class="nb"&gt;date&lt;/span&gt; &lt;span class="nt"&gt;-d&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="nv"&gt;$expiry&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt; +%s 2&amp;gt;/dev/null&lt;span class="si"&gt;)&lt;/span&gt;
  &lt;span class="nv"&gt;now_epoch&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="si"&gt;$(&lt;/span&gt;&lt;span class="nb"&gt;date&lt;/span&gt; +%s&lt;span class="si"&gt;)&lt;/span&gt;
  &lt;span class="nv"&gt;days_left&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="k"&gt;$((&lt;/span&gt; &lt;span class="o"&gt;(&lt;/span&gt;expiry_epoch &lt;span class="o"&gt;-&lt;/span&gt; now_epoch&lt;span class="o"&gt;)&lt;/span&gt; &lt;span class="o"&gt;/&lt;/span&gt; &lt;span class="m"&gt;86400&lt;/span&gt; &lt;span class="k"&gt;))&lt;/span&gt;

  &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="o"&gt;[&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="nv"&gt;$days_left&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt; &lt;span class="nt"&gt;-le&lt;/span&gt; 0 &lt;span class="o"&gt;]&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="k"&gt;then
    &lt;/span&gt;&lt;span class="nb"&gt;echo&lt;/span&gt; &lt;span class="s2"&gt;"CRITICAL: &lt;/span&gt;&lt;span class="nv"&gt;$domain&lt;/span&gt;&lt;span class="s2"&gt; cert EXPIRED (&lt;/span&gt;&lt;span class="nv"&gt;$days_left&lt;/span&gt;&lt;span class="s2"&gt; days)"&lt;/span&gt;
  &lt;span class="k"&gt;elif&lt;/span&gt; &lt;span class="o"&gt;[&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="nv"&gt;$days_left&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt; &lt;span class="nt"&gt;-le&lt;/span&gt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="nv"&gt;$WARN_DAYS&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt; &lt;span class="o"&gt;]&lt;/span&gt;&lt;span class="p"&gt;;&lt;/span&gt; &lt;span class="k"&gt;then
    &lt;/span&gt;&lt;span class="nb"&gt;echo&lt;/span&gt; &lt;span class="s2"&gt;"WARN: &lt;/span&gt;&lt;span class="nv"&gt;$domain&lt;/span&gt;&lt;span class="s2"&gt; expires in &lt;/span&gt;&lt;span class="nv"&gt;$days_left&lt;/span&gt;&lt;span class="s2"&gt; days (&lt;/span&gt;&lt;span class="nv"&gt;$expiry&lt;/span&gt;&lt;span class="s2"&gt;)"&lt;/span&gt;
  &lt;span class="k"&gt;else
    &lt;/span&gt;&lt;span class="nb"&gt;echo&lt;/span&gt; &lt;span class="s2"&gt;"OK: &lt;/span&gt;&lt;span class="nv"&gt;$domain&lt;/span&gt;&lt;span class="s2"&gt; expires in &lt;/span&gt;&lt;span class="nv"&gt;$days_left&lt;/span&gt;&lt;span class="s2"&gt; days"&lt;/span&gt;
  &lt;span class="k"&gt;fi
done&lt;/span&gt; &amp;lt; &lt;span class="s2"&gt;"&lt;/span&gt;&lt;span class="nv"&gt;$DOMAINS_FILE&lt;/span&gt;&lt;span class="s2"&gt;"&lt;/span&gt;

&lt;span class="c"&gt;# Add to cron to run daily at 08:00 and email output:&lt;/span&gt;
&lt;span class="c"&gt;# 0 8 * * * /etc/ssl-monitor/ssl-check.sh | mail -s "SSL expiry report" ops@example.com&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;This works well for a handful of domains. Once you are managing ten or more servers, each potentially hosting dozens of vhosts, keeping the domains list accurate becomes its own maintenance burden. That is the point where a purpose-built ssl certificate expiry monitoring tool saves more time than it costs.&lt;/p&gt;

&lt;p&gt;If you are already thinking about broader server visibility, &lt;a href="https://getopservo.com/guides/monitor-multiple-linux-servers" rel="noopener noreferrer"&gt;monitoring multiple Linux servers from one place&lt;/a&gt; is a natural next step, especially when each server hosts its own set of vhosts and certificates.&lt;/p&gt;

&lt;h2&gt;
  
  
  Certificate expiry monitoring for Laravel and other app stacks
&lt;/h2&gt;

&lt;p&gt;Laravel apps often run behind nginx with multiple vhosts: one for the main application, one for an API subdomain, sometimes one for a separate admin panel. Each vhost has its own certificate, and they may renew on different schedules if you added subdomains at different times. An expired cert on the API subdomain will not affect the main site in a browser, but it will silently break every mobile client and third-party integration.&lt;/p&gt;

&lt;p&gt;The openssl command above works for any subdomain. Include every vhost hostname in your domains list, not just the apex domain. The same applies to Django apps on subdomains, Node services behind a reverse proxy, and any FastAPI or PHP app served through nginx or Apache.&lt;/p&gt;

&lt;p&gt;Laravel-specific monitoring covers more than just SSL, from queue workers to Horizon health. &lt;a href="https://getopservo.com/guides/monitor-laravel-in-production" rel="noopener noreferrer"&gt;Monitoring Laravel in production&lt;/a&gt; goes into the full picture if you want those checks alongside certificate expiry.&lt;/p&gt;

&lt;h2&gt;
  
  
  Common reasons ssl certificate expiration sneaks past you
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Certbot or acme.sh runs on a cron job that silently fails after a server rebuild or OS upgrade.&lt;/li&gt;
&lt;li&gt;The renewal email goes to a shared inbox or a former employee's address.&lt;/li&gt;
&lt;li&gt;A wildcard certificate renews correctly but a specific subdomain cert does not, because they are managed separately.&lt;/li&gt;
&lt;li&gt;A CDN or load balancer terminates TLS, so the origin cert expires unnoticed while the CDN cert stays valid.&lt;/li&gt;
&lt;li&gt;A staging or internal domain is excluded from monitoring because it "does not matter", until it does.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;For the CDN case, you need to check the cert at the CDN edge as well as the origin. Use the -connect flag to point openssl directly at the origin IP if the CDN masks it, or check both hostnames explicitly.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;Always check the certificate your the client actually sees, not the one on disk. A renewal can succeed locally but the web server may still be serving the old cached cert until nginx or Apache is reloaded.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;This is also why checking the live connection with openssl s_client matters more than checking the file in /etc/letsencrypt. If you forget to run systemctl reload nginx after certbot renew, the live cert stays expired even though the file is fresh.&lt;/p&gt;

&lt;h2&gt;
  
  
  Automating ssl expiry monitoring without managing scripts
&lt;/h2&gt;

&lt;p&gt;The script above works, but it adds something to maintain: you have to keep the domain list current, check the cron logs, and remember to add new vhosts when you deploy them. For teams without a dedicated SRE, that maintenance often slips. If you are thinking about how to cover all of this with less manual overhead, &lt;a href="https://getopservo.com/guides/monitor-linux-server-without-sre" rel="noopener noreferrer"&gt;monitoring a Linux server without an SRE&lt;/a&gt; covers the broader pattern.&lt;/p&gt;

&lt;p&gt;Opservo auto-discovers nginx and Apache vhosts from the server's config and tracks SSL expiry for each one without any domain list to maintain. It sends alerts at 30, 14 and 3 days, and you can add external HTTP or domain checks for domains hosted elsewhere. Every alert tells you the domain, the days remaining and which server it belongs to, so you know exactly what to renew and where.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;Invite the reader to add their servers to Opservo's free 2-server plan to get auto-discovered vhost SSL alerts without maintaining a domain list. — one command installs a read-only agent and your first findings arrive in five minutes: &lt;a href="https://getopservo.com/signup?utm_source=blog&amp;amp;utm_medium=post&amp;amp;utm_campaign=auto-blog" rel="noopener noreferrer"&gt;start free&lt;/a&gt;. Related: &lt;a href="https://getopservo.com/guides/monitor-laravel-in-production" rel="noopener noreferrer"&gt;guides monitor laravel in production&lt;/a&gt;.&lt;/p&gt;
&lt;/blockquote&gt;




&lt;p&gt;&lt;em&gt;Originally published on the &lt;a href="https://getopservo.com/blog/how-to-monitor-ssl-certificate-expiration?utm_source=devto&amp;amp;utm_medium=syndication&amp;amp;utm_campaign=syndication" rel="noopener noreferrer"&gt;Opservo blog&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>linux</category>
      <category>devops</category>
      <category>monitoring</category>
      <category>operations</category>
    </item>
    <item>
      <title>What an agency running 10 to 50 servers should actually monitor</title>
      <dc:creator>Opservo</dc:creator>
      <pubDate>Tue, 03 Nov 2026 14:08:15 +0000</pubDate>
      <link>https://dev.to/opservo/what-an-agency-running-10-to-50-servers-should-actually-monitor-jen</link>
      <guid>https://dev.to/opservo/what-an-agency-running-10-to-50-servers-should-actually-monitor-jen</guid>
      <description>&lt;p&gt;If you run a digital agency, servers are not your product. Your clients' products just happen to live on them. Somewhere between ten and fifty Linux boxes on AWS, DigitalOcean, Hetzner or an Indian VPS, running Laravel, WordPress, Node and Docker, looked after by developers who also have a sprint to finish.&lt;/p&gt;

&lt;p&gt;We are in the same position. Softuvo runs production for 18 companies and has no site reliability engineer. This is the list we wish someone had handed us: what to watch, what to skip, and how to make sure the alert reaches a person.&lt;/p&gt;

&lt;h2&gt;
  
  
  Start from what your clients notice
&lt;/h2&gt;

&lt;p&gt;Clients do not care about CPU graphs. In our experience they notice five things, and almost every outage we have had is one of them:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;The disk filled up. Logs that stopped rotating, backups nobody prunes, old Docker images. The site goes read-only or the database stops writing, usually at night.&lt;/li&gt;
&lt;li&gt;The SSL certificate expired. The browser shows a full-page warning and the client sees it before you do.&lt;/li&gt;
&lt;li&gt;A background service stopped. The site loads, but the Laravel queue worker, the PHP-FPM pool or a PM2 process died, so orders, emails or imports quietly stop.&lt;/li&gt;
&lt;li&gt;The database or Redis went away. Out of memory, out of connections, or simply stopped.&lt;/li&gt;
&lt;li&gt;The site is down from outside. DNS, the domain itself, or the load balancer, things you cannot see from inside the server.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  What to monitor on every server
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Disk usage and its trend. A threshold at 90% tells you too late. What you want is "at this rate /var is full on Thursday".&lt;/li&gt;
&lt;li&gt;Every service you depend on, by name. systemd units, Docker containers, PM2 apps, queue workers. "The server is up" says nothing about whether the worker is.&lt;/li&gt;
&lt;li&gt;SSL expiry for every domain on the box, not just the main one. Agencies collect vhosts.&lt;/li&gt;
&lt;li&gt;External checks from outside your network: HTTP for the site, a heartbeat for each cron job that matters (backups first), domain expiry.&lt;/li&gt;
&lt;li&gt;MySQL, PostgreSQL and Redis: reachable, memory, connections.&lt;/li&gt;
&lt;li&gt;The boring security basics: pending updates, open ports you did not expect, how SSH is configured. Not urgent today; the reason for an incident next quarter.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fq92g5jinezb96mivynmc.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fq92g5jinezb96mivynmc.png" alt="Opservo fleet view ranking servers by health score" width="800" height="479"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;p&gt;With 15 or 30 servers, the hard part is not collecting these numbers. It is knowing which server to look at first. That is why we rank servers by a single &lt;a href="https://getopservo.com/health-score" rel="noopener noreferrer"&gt;health score&lt;/a&gt; and, for every point a server loses, name the rule, the threshold, the current value and the fix.&lt;/p&gt;

&lt;h2&gt;
  
  
  What you can skip, for now
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Per-second dashboards nobody opens. If nobody looks at a graph until after the outage, it is not monitoring, it is archaeology.&lt;/li&gt;
&lt;li&gt;Distributed tracing and APM, unless you run many services calling each other. Most agency stacks are a monolith, a database and a queue. (Opservo does not do tracing either; if you need it, you need a different tool.)&lt;/li&gt;
&lt;li&gt;A 24/7 operations desk. You need one person paged reliably, and a second if they do not answer.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Make sure the alert lands
&lt;/h2&gt;

&lt;p&gt;Detection is the easy half. In September our own alerts went to a webhook that rejected every one of them, and we only found out weeks later. (&lt;a href="https://getopservo.com/blog/our-alerts-went-nowhere-for-59-days" rel="noopener noreferrer"&gt;The full story is on our blog&lt;/a&gt;.) Three rules we now follow:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Anything that pages goes to at least two places, for example Slack and email.&lt;/li&gt;
&lt;li&gt;Use an on-call rotation with escalation, so the 2 a.m. alert reaches whoever is on call that week, and the next person if they do not acknowledge.&lt;/li&gt;
&lt;li&gt;Once a month, send a test alert through every channel and check that a person saw it.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Keep each client separate
&lt;/h2&gt;

&lt;p&gt;Agencies have a problem SaaS teams do not: each client's servers, alerts and reports should stay with that client. Give every client its own workspace and its own public status page. A status page alone cuts down a lot of "is the site down?" emails.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F6liurwabosuj0vdh1dur.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F6liurwabosuj0vdh1dur.png" alt="Opservo uptime checks with a public status page" width="800" height="479"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  What it should cost
&lt;/h2&gt;

&lt;p&gt;Monitoring that is billed per metric, per log gigabyte or per feature tends to surprise you on the invoice, which is the last thing an agency needs on a fixed-fee client. Look for a flat per-server price you can pass on to a client.&lt;/p&gt;

&lt;p&gt;For reference, Opservo is free for 2 servers with no card. Growth is ₹999 per server per month. The Agency plan is ₹799 per server per month with a 10-server minimum and adds multi-tenant workspaces and white-label. Full details on the &lt;a href="https://getopservo.com/pricing" rel="noopener noreferrer"&gt;pricing page&lt;/a&gt;. If you already run Datadog and an SRE reads it every day, keep it; &lt;a href="https://getopservo.com/alternatives/datadog" rel="noopener noreferrer"&gt;here is how we compare&lt;/a&gt;, including where Datadog is the better choice.&lt;/p&gt;

&lt;h2&gt;
  
  
  Where to start this week
&lt;/h2&gt;

&lt;p&gt;Pick the server that scared you last. Put monitoring on it, set up two alert channels, and send a test alert. Opservo installs with one command in about 30 seconds and needs no configuration. The agent is read-only by default and opens no inbound ports, so it is safe to try on a client box. &lt;a href="https://getopservo.com/welcome" rel="noopener noreferrer"&gt;Start free at getopservo.com&lt;/a&gt;.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;Try Opservo free on 2 servers — one command installs a read-only agent and your first findings arrive in five minutes: &lt;a href="https://getopservo.com/signup?utm_source=blog&amp;amp;utm_medium=post&amp;amp;utm_campaign=auto-blog" rel="noopener noreferrer"&gt;start free&lt;/a&gt;.&lt;/p&gt;
&lt;/blockquote&gt;




&lt;p&gt;&lt;em&gt;Originally published on the &lt;a href="https://getopservo.com/blog/what-an-agency-should-monitor?utm_source=devto&amp;amp;utm_medium=syndication&amp;amp;utm_campaign=syndication" rel="noopener noreferrer"&gt;Opservo blog&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>linux</category>
      <category>devops</category>
      <category>monitoring</category>
      <category>guides</category>
    </item>
    <item>
      <title>How to monitor Docker memory usage</title>
      <dc:creator>Opservo</dc:creator>
      <pubDate>Sun, 01 Nov 2026 08:45:14 +0000</pubDate>
      <link>https://dev.to/opservo/how-to-monitor-docker-memory-usage-21b0</link>
      <guid>https://dev.to/opservo/how-to-monitor-docker-memory-usage-21b0</guid>
      <description>&lt;p&gt;A container that eats memory silently can OOM-kill your app at 3 am, restart your database mid-transaction, or starve every other service on the host. Most teams notice only after the crash. This guide walks through every useful way to watch Docker memory in real time and over time, what the numbers actually mean, and what thresholds are worth alerting on.&lt;/p&gt;

&lt;h2&gt;
  
  
  What docker stats shows you (and what it leaves out)
&lt;/h2&gt;

&lt;p&gt;Run &lt;code&gt;docker stats&lt;/code&gt; with no arguments and Docker prints a live table refreshed every second. The columns that matter for memory are MEM USAGE, LIMIT, and MEM %.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="c"&gt;# Live table, all running containers&lt;/span&gt;
docker stats

&lt;span class="c"&gt;# Example output:&lt;/span&gt;
&lt;span class="c"&gt;# CONTAINER ID   NAME         MEM USAGE / LIMIT     MEM %&lt;/span&gt;
&lt;span class="c"&gt;# a3f1b2c4d5e6   api-prod     412MiB / 2GiB         20.12%&lt;/span&gt;
&lt;span class="c"&gt;# 9e8d7c6b5a4f   redis        88.3MiB / 512MiB      17.24%&lt;/span&gt;
&lt;span class="c"&gt;# 1a2b3c4d5e6f   postgres     310MiB / 4GiB         7.57%&lt;/span&gt;

&lt;span class="c"&gt;# One snapshot (no live refresh), all containers&lt;/span&gt;
docker stats &lt;span class="nt"&gt;--no-stream&lt;/span&gt;

&lt;span class="c"&gt;# Sort by memory (highest first) using a format and sort&lt;/span&gt;
docker stats &lt;span class="nt"&gt;--no-stream&lt;/span&gt; &lt;span class="nt"&gt;--format&lt;/span&gt; &lt;span class="s2"&gt;"table {{.Name}}&lt;/span&gt;&lt;span class="se"&gt;\t&lt;/span&gt;&lt;span class="s2"&gt;{{.MemUsage}}&lt;/span&gt;&lt;span class="se"&gt;\t&lt;/span&gt;&lt;span class="s2"&gt;{{.MemPerc}}"&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
  | &lt;span class="nb"&gt;sort&lt;/span&gt; &lt;span class="nt"&gt;-t&lt;/span&gt;&lt;span class="s1"&gt;$'&lt;/span&gt;&lt;span class="se"&gt;\t&lt;/span&gt;&lt;span class="s1"&gt;'&lt;/span&gt; &lt;span class="nt"&gt;-k3&lt;/span&gt; &lt;span class="nt"&gt;-rh&lt;/span&gt;

&lt;span class="c"&gt;# Watch a single container&lt;/span&gt;
docker stats &lt;span class="nt"&gt;--no-stream&lt;/span&gt; my_container
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The &lt;code&gt;docker stats sort by memory&lt;/code&gt; trick above uses &lt;code&gt;--format&lt;/code&gt; to get tab-separated output and then sorts on the MEM % column. That is the fastest way to find your worst offender across a fleet of containers.&lt;/p&gt;

&lt;p&gt;One important gap: &lt;code&gt;docker stats&lt;/code&gt; shows RSS plus cache together as MEM USAGE. That can look higher than the real pressure. Cache memory is reclaimed under load, so the number the kernel cares about is RSS (resident set size) alone. More on reading the raw cgroup file below.&lt;/p&gt;

&lt;h2&gt;
  
  
  How to monitor Docker performance beyond the stats table
&lt;/h2&gt;

&lt;p&gt;Memory alone rarely tells the whole story. A container can sit at 80 % memory and be completely healthy, or it can sit at 40 % and be thrashing swap. Pair memory with two other signals to understand real performance:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;CPU throttling: check &lt;code&gt;docker stats&lt;/code&gt; CPU % against the container's CPU quota; if it is consistently at the quota ceiling, the app is CPU-starved not memory-starved.&lt;/li&gt;
&lt;li&gt;Page faults: high major page faults in &lt;code&gt;/sys/fs/cgroup/memory/docker/&amp;lt;id&amp;gt;/memory.stat&lt;/code&gt; mean the kernel is reading data from disk -- a sign of memory pressure even when the headline number looks low.&lt;/li&gt;
&lt;li&gt;Restart count: &lt;code&gt;docker inspect --format '{{.RestartCount}}' &amp;lt;name&amp;gt;&lt;/code&gt; tells you how many times the container has been killed and restarted; any non-zero value after a fresh deploy is a red flag.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;For a broader look at host-level pressure that Docker contributes to, the checklist in &lt;a href="https://getopservo.com/guides/why-is-my-server-slow" rel="noopener noreferrer"&gt;Why is my server slow? A 10-minute checklist&lt;/a&gt; walks through CPU, memory, disk, and network in one pass. It is worth keeping open alongside this guide.&lt;/p&gt;

&lt;p&gt;To watch per-core CPU across all containers on the host, open htop and press F2, then add the &lt;code&gt;CGROUP&lt;/code&gt; column so you can see which cgroup each process belongs to. That links a runaway process back to its container without needing container IDs.&lt;/p&gt;

&lt;h2&gt;
  
  
  Reading cgroup memory files directly
&lt;/h2&gt;

&lt;p&gt;The kernel exposes container memory through cgroup v1 or v2 files. These are the ground-truth numbers Docker stats reads from. Checking them directly is useful when you need scriptable output or when you want the RSS figure without cache.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="c"&gt;# Find the container's full ID&lt;/span&gt;
&lt;span class="nv"&gt;CID&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="si"&gt;$(&lt;/span&gt;docker inspect &lt;span class="nt"&gt;--format&lt;/span&gt; &lt;span class="s1"&gt;'{{.Id}}'&lt;/span&gt; my_container&lt;span class="si"&gt;)&lt;/span&gt;

&lt;span class="c"&gt;# --- cgroup v1 (most pre-2022 hosts) ---&lt;/span&gt;
&lt;span class="c"&gt;# Current usage in bytes&lt;/span&gt;
&lt;span class="nb"&gt;cat&lt;/span&gt; /sys/fs/cgroup/memory/docker/&lt;span class="k"&gt;${&lt;/span&gt;&lt;span class="nv"&gt;CID&lt;/span&gt;&lt;span class="k"&gt;}&lt;/span&gt;/memory.usage_in_bytes

&lt;span class="c"&gt;# Limit in bytes (-1 means no limit set)&lt;/span&gt;
&lt;span class="nb"&gt;cat&lt;/span&gt; /sys/fs/cgroup/memory/docker/&lt;span class="k"&gt;${&lt;/span&gt;&lt;span class="nv"&gt;CID&lt;/span&gt;&lt;span class="k"&gt;}&lt;/span&gt;/memory.limit_in_bytes

&lt;span class="c"&gt;# RSS only (excludes page cache)&lt;/span&gt;
&lt;span class="nb"&gt;grep&lt;/span&gt; ^rss /sys/fs/cgroup/memory/docker/&lt;span class="k"&gt;${&lt;/span&gt;&lt;span class="nv"&gt;CID&lt;/span&gt;&lt;span class="k"&gt;}&lt;/span&gt;/memory.stat
&lt;span class="c"&gt;# rss 423231488  &amp;lt;- bytes of real in-use RAM&lt;/span&gt;

&lt;span class="c"&gt;# OOM kill count (non-zero = container has been OOM-killed before)&lt;/span&gt;
&lt;span class="nb"&gt;cat&lt;/span&gt; /sys/fs/cgroup/memory/docker/&lt;span class="k"&gt;${&lt;/span&gt;&lt;span class="nv"&gt;CID&lt;/span&gt;&lt;span class="k"&gt;}&lt;/span&gt;/memory.oom_control
&lt;span class="c"&gt;# oom_kill_disable 0&lt;/span&gt;
&lt;span class="c"&gt;# under_oom 0&lt;/span&gt;
&lt;span class="c"&gt;# oom_kill 3      &amp;lt;- this container has been OOM-killed 3 times&lt;/span&gt;

&lt;span class="c"&gt;# --- cgroup v2 (Ubuntu 22.04+, Debian 12+, Amazon Linux 2023) ---&lt;/span&gt;
&lt;span class="nb"&gt;cat&lt;/span&gt; /sys/fs/cgroup/system.slice/docker-&lt;span class="k"&gt;${&lt;/span&gt;&lt;span class="nv"&gt;CID&lt;/span&gt;&lt;span class="k"&gt;}&lt;/span&gt;.scope/memory.current
&lt;span class="nb"&gt;cat&lt;/span&gt; /sys/fs/cgroup/system.slice/docker-&lt;span class="k"&gt;${&lt;/span&gt;&lt;span class="nv"&gt;CID&lt;/span&gt;&lt;span class="k"&gt;}&lt;/span&gt;.scope/memory.max
&lt;span class="nb"&gt;grep &lt;/span&gt;anon /sys/fs/cgroup/system.slice/docker-&lt;span class="k"&gt;${&lt;/span&gt;&lt;span class="nv"&gt;CID&lt;/span&gt;&lt;span class="k"&gt;}&lt;/span&gt;.scope/memory.stat
&lt;span class="c"&gt;# anon 423231488  &amp;lt;- equivalent of RSS in v2&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The &lt;code&gt;oom_kill&lt;/code&gt; counter is the most actionable number here. If it is greater than zero, the container has already been silently restarted by the kernel. Many teams never see these kills because Docker restarts the container immediately and the log shows nothing obvious.&lt;/p&gt;

&lt;h2&gt;
  
  
  Setting and checking memory limits
&lt;/h2&gt;

&lt;p&gt;A container without a memory limit borrows from the whole host. One leak takes everything down. Set limits in your Compose file or run command:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight yaml"&gt;&lt;code&gt;&lt;span class="c1"&gt;# In docker-compose.yml (Compose v3)&lt;/span&gt;
&lt;span class="na"&gt;services&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
  &lt;span class="na"&gt;api&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
    &lt;span class="na"&gt;image&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;my-api:latest&lt;/span&gt;
    &lt;span class="na"&gt;deploy&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
      &lt;span class="na"&gt;resources&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
        &lt;span class="na"&gt;limits&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
          &lt;span class="na"&gt;memory&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;512m&lt;/span&gt;
        &lt;span class="na"&gt;reservations&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt;
          &lt;span class="na"&gt;memory&lt;/span&gt;&lt;span class="pi"&gt;:&lt;/span&gt; &lt;span class="s"&gt;256m&lt;/span&gt;

&lt;span class="c1"&gt;# Or with docker run&lt;/span&gt;
&lt;span class="s"&gt;docker run -d --memory=512m --memory-reservation=256m my-api:latest&lt;/span&gt;

&lt;span class="c1"&gt;# Confirm the limit is applied&lt;/span&gt;
&lt;span class="s"&gt;docker inspect --format '{{.HostConfig.Memory}}' my_container&lt;/span&gt;
&lt;span class="c1"&gt;# 536870912   &amp;lt;- 512 MiB in bytes; 0 means no limit&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The &lt;code&gt;--memory-reservation&lt;/code&gt; (soft limit) lets the kernel reclaim from the container under host pressure without killing it. The hard &lt;code&gt;--memory&lt;/code&gt; limit is what triggers the OOM kill. Both are worth setting in production.&lt;/p&gt;

&lt;p&gt;If you run Laravel queue workers or Horizon inside a container, set a generous limit and watch it over a few days before tightening it. Laravel workers accumulate memory across jobs if &lt;code&gt;--max-memory&lt;/code&gt; is not set on the queue command. See &lt;a href="https://getopservo.com/guides/monitor-laravel-in-production" rel="noopener noreferrer"&gt;monitor Laravel in production&lt;/a&gt; for what to watch specifically.&lt;/p&gt;

&lt;p&gt;Redis in a container also deserves a hard limit. If Redis has no &lt;code&gt;maxmemory&lt;/code&gt; directive in its config and no cgroup limit, it will consume all available RAM on the host. Check current memory use with &lt;code&gt;docker exec my_redis redis-cli info memory | grep used_memory_human&lt;/code&gt;. More detail on Redis memory pressure is covered in &lt;a href="https://getopservo.com/blog/how-to-monitor-redis-memory-and-eviction" rel="noopener noreferrer"&gt;how to monitor Redis memory and eviction&lt;/a&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  Docker monitor logs: finding OOM and memory errors
&lt;/h2&gt;

&lt;p&gt;When a container OOM-kills, the event shows up in two places: the kernel ring buffer and the Docker daemon logs.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="c"&gt;# Kernel OOM events (look for 'Killed process' or 'oom-kill')&lt;/span&gt;
&lt;span class="nb"&gt;sudo &lt;/span&gt;dmesg | &lt;span class="nb"&gt;grep&lt;/span&gt; &lt;span class="nt"&gt;-iE&lt;/span&gt; &lt;span class="s1"&gt;'oom|killed process'&lt;/span&gt; | &lt;span class="nb"&gt;tail&lt;/span&gt; &lt;span class="nt"&gt;-20&lt;/span&gt;

&lt;span class="c"&gt;# Docker daemon log (systemd installs)&lt;/span&gt;
&lt;span class="nb"&gt;sudo &lt;/span&gt;journalctl &lt;span class="nt"&gt;-u&lt;/span&gt; docker.service | &lt;span class="nb"&gt;grep&lt;/span&gt; &lt;span class="nt"&gt;-iE&lt;/span&gt; &lt;span class="s1"&gt;'oom|kill'&lt;/span&gt; | &lt;span class="nb"&gt;tail&lt;/span&gt; &lt;span class="nt"&gt;-20&lt;/span&gt;

&lt;span class="c"&gt;# Container's own stdout/stderr since last restart&lt;/span&gt;
docker logs &lt;span class="nt"&gt;--since&lt;/span&gt; 1h my_container 2&amp;gt;&amp;amp;1 | &lt;span class="nb"&gt;tail&lt;/span&gt; &lt;span class="nt"&gt;-50&lt;/span&gt;

&lt;span class="c"&gt;# Follow logs in real time&lt;/span&gt;
docker logs &lt;span class="nt"&gt;-f&lt;/span&gt; my_container

&lt;span class="c"&gt;# Events stream: catch OOM kills as they happen&lt;/span&gt;
docker events &lt;span class="nt"&gt;--filter&lt;/span&gt; &lt;span class="nv"&gt;event&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;oom
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The &lt;code&gt;docker events --filter event=oom&lt;/code&gt; command is the cleanest way to watch for kills in real time. Pipe it to a log file or a monitoring agent. If you want to understand what else might be going wrong at the host level alongside these kills, the guide on &lt;a href="https://getopservo.com/blog/how-to-fix-high-memory-usage-on-a-linux-server" rel="noopener noreferrer"&gt;how to fix high memory usage on a Linux server&lt;/a&gt; covers identifying which process or container is responsible and whether swappiness is making things worse.&lt;/p&gt;

&lt;h2&gt;
  
  
  What to alert on: concrete thresholds for Docker memory
&lt;/h2&gt;

&lt;p&gt;Not every memory spike deserves a page. Use these thresholds as a starting point and adjust for your workload:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;MEM % above 85 % of the container limit for more than 5 minutes: warn. Above 95 % for more than 1 minute: page. At 100 % the kernel kills the process.&lt;/li&gt;
&lt;li&gt;OOM kill count increments: alert immediately, every time. This is a production incident whether or not the container recovered.&lt;/li&gt;
&lt;li&gt;Container restart count increases: alert on any new restart after the initial deploy period. Use &lt;code&gt;docker inspect --format '{{.RestartCount}}'&lt;/code&gt; in a cron or monitoring script.&lt;/li&gt;
&lt;li&gt;No memory limit set (&lt;code&gt;docker inspect&lt;/code&gt; returns 0): flag at deploy time, not at 3 am. Add a CI check that rejects images deployed without a memory limit.&lt;/li&gt;
&lt;li&gt;Host free memory below 10 %: even with per-container limits, the host can still be pressured if many containers are near their limits simultaneously. Check with &lt;code&gt;free -h&lt;/code&gt; and watch &lt;code&gt;MemAvailable&lt;/code&gt; in &lt;code&gt;/proc/meminfo&lt;/code&gt;.&lt;/li&gt;
&lt;/ul&gt;

&lt;blockquote&gt;
&lt;p&gt;Rule: if a container has no memory limit, it can silently consume the entire host. Set &lt;code&gt;--memory&lt;/code&gt; on every production container, even a generous one, so the kernel kills one service rather than the whole machine.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;h2&gt;
  
  
  Keeping this visible without running commands every day
&lt;/h2&gt;

&lt;p&gt;Running &lt;code&gt;docker stats&lt;/code&gt; by hand is fine for a one-off check. It does not scale to a fleet where you need to know which container drifted past its memory budget overnight, or which one OOM-killed twice this week while the on-call was asleep.&lt;/p&gt;

&lt;p&gt;Opservo auto-discovers every Docker container on your Linux servers and tracks memory usage, restart counts, and OOM events continuously. When a container approaches its limit or gets killed, you get a plain-English alert with the container name, the current memory figure, the limit, and a suggested fix, without writing a single PromQL query or Grafana dashboard. The agent is a single static Go binary under 10 MB, installs in one command, and is read-only by default.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;Invite the reader to install Opservo's free 2-server plan to get continuous container memory tracking and OOM alerts without running docker stats by hand. — one command installs a read-only agent and your first findings arrive in five minutes: &lt;a href="https://getopservo.com/signup?utm_source=blog&amp;amp;utm_medium=post&amp;amp;utm_campaign=auto-blog" rel="noopener noreferrer"&gt;start free&lt;/a&gt;. Related: &lt;a href="https://getopservo.com/blog/how-to-monitor-a-docker-container-s-cpu-and-memory-usage" rel="noopener noreferrer"&gt;blog how to monitor a docker container s cpu and memory usage&lt;/a&gt;.&lt;/p&gt;
&lt;/blockquote&gt;




&lt;p&gt;&lt;em&gt;Originally published on the &lt;a href="https://getopservo.com/blog/how-to-monitor-docker-memory-usage?utm_source=devto&amp;amp;utm_medium=syndication&amp;amp;utm_campaign=syndication" rel="noopener noreferrer"&gt;Opservo blog&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>linux</category>
      <category>devops</category>
      <category>monitoring</category>
      <category>operations</category>
    </item>
    <item>
      <title>When the person who knew your servers leaves: a five-day handover checklist</title>
      <dc:creator>Opservo</dc:creator>
      <pubDate>Tue, 27 Oct 2026 13:33:18 +0000</pubDate>
      <link>https://dev.to/opservo/when-the-person-who-knew-your-servers-leaves-a-five-day-handover-checklist-358l</link>
      <guid>https://dev.to/opservo/when-the-person-who-knew-your-servers-leaves-a-five-day-handover-checklist-358l</guid>
      <description>&lt;p&gt;Most agencies running 10 to 50 servers have one person who knows how they work. Which box runs which client. How to restart the queue when emails stop. Why one cron job runs at 3 a.m. and what breaks if it does not.&lt;/p&gt;

&lt;p&gt;Nobody plans for that person to leave, go on a long holiday or move to another team. When it happens, you usually get two weeks of notice and a document that was last updated a year ago.&lt;/p&gt;

&lt;p&gt;This is the checklist we would hand any agency in that situation. It takes five working days. Do it with the person while they are still there, and do every step from the servers themselves, not from memory.&lt;/p&gt;

&lt;h2&gt;
  
  
  Day 1: List what actually runs on every server
&lt;/h2&gt;

&lt;p&gt;For each server, write down the provider, the client it belongs to, who has root access, and what is running on it. Get the last part from the server, not from the person:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;systemctl list-units --type=service --state=running
docker ps
pm2 list
ls /etc/nginx/sites-enabled /etc/apache2/sites-enabled 2&amp;gt;/dev/null
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Anything on that output that nobody can explain goes on a list for day 5.&lt;/p&gt;

&lt;h2&gt;
  
  
  Day 2: Find everything that expires
&lt;/h2&gt;

&lt;p&gt;SSL certificates, domains, and any API keys or tokens with an end date. For certificates managed by Certbot, &lt;code&gt;certbot certificates&lt;/code&gt; lists them with their expiry dates. Put every date for the next 12 months in a shared calendar, with two people invited, not one.&lt;/p&gt;

&lt;h2&gt;
  
  
  Day 3: Find everything that runs on a schedule
&lt;/h2&gt;

&lt;p&gt;Scheduled jobs are where silent failures live, because nobody notices a job that stopped running until a client asks where their report went.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;crontab -l -u &amp;lt;each user&amp;gt;
ls /etc/cron.d
systemctl list-timers
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;For each job, note what it does and who would notice if it stopped. Those are the ones that need a check, not just a log line.&lt;/p&gt;

&lt;h2&gt;
  
  
  Day 4: Find out who gets told when something breaks
&lt;/h2&gt;

&lt;p&gt;List every alert channel: email addresses, Slack channels, webhooks, phone numbers. Then send a test alert through each one and confirm a person saw it. If any of them still point at the person who is leaving, change them today. We learned this one the hard way: for 59 days our own alerts went to a webhook that rejected every one of them, and nobody noticed.&lt;/p&gt;

&lt;h2&gt;
  
  
  Day 5: Write the three restart notes
&lt;/h2&gt;

&lt;p&gt;For each client, one page covering the three things that break most often in agency stacks: the queue workers, the web server and the database. What "broken" looks like, the command to check, the command to restart, and who to call if that does not work. Go through the day-1 "nobody can explain" list with the person before they leave.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why this goes out of date, and what to do about it
&lt;/h2&gt;

&lt;p&gt;A handover document is right on the day it is written. Then someone adds a container, a certificate is renewed by hand, a new cron job appears, and the document is wrong again. Re-run days 1 to 4 every month, or use something that reads the list from the servers for you.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fq92g5jinezb96mivynmc.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fq92g5jinezb96mivynmc.png" alt="Opservo fleet view: every server ranked by health score" width="800" height="479"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Where Opservo fits
&lt;/h2&gt;

&lt;p&gt;Opservo is built for exactly this gap. One command installs a read-only agent on each Linux server, with no inbound ports. Within minutes it lists what runs there from the server itself: systemd services, Docker containers, PM2 processes, nginx and Apache sites with their SSL expiry, MySQL, PostgreSQL and Redis, and Laravel, WordPress and Node apps. Cron heartbeats tell you when a scheduled job stops checking in. Every alert says in plain English what is wrong and what to do, so the person on call does not need to be the person who set it up, and any fix that runs is audited.&lt;/p&gt;

&lt;p&gt;It is free for 2 servers, no card. &lt;a href="https://getopservo.com/welcome" rel="noopener noreferrer"&gt;Start at getopservo.com/welcome&lt;/a&gt;, or see &lt;a href="https://getopservo.com/features" rel="noopener noreferrer"&gt;everything it monitors&lt;/a&gt;.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;Try Opservo free on 2 servers — one command installs a read-only agent and your first findings arrive in five minutes: &lt;a href="https://getopservo.com/signup?utm_source=blog&amp;amp;utm_medium=post&amp;amp;utm_campaign=auto-blog" rel="noopener noreferrer"&gt;start free&lt;/a&gt;.&lt;/p&gt;
&lt;/blockquote&gt;




&lt;p&gt;&lt;em&gt;Originally published on the &lt;a href="https://getopservo.com/blog/when-the-person-who-knew-your-servers-leaves-a-five-day-handover-check?utm_source=devto&amp;amp;utm_medium=syndication&amp;amp;utm_campaign=syndication" rel="noopener noreferrer"&gt;Opservo blog&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>linux</category>
      <category>devops</category>
      <category>monitoring</category>
      <category>agencies</category>
    </item>
    <item>
      <title>How to monitor network traffic on Linux</title>
      <dc:creator>Opservo</dc:creator>
      <pubDate>Sun, 25 Oct 2026 07:46:13 +0000</pubDate>
      <link>https://dev.to/opservo/how-to-monitor-network-traffic-on-linux-15db</link>
      <guid>https://dev.to/opservo/how-to-monitor-network-traffic-on-linux-15db</guid>
      <description>&lt;p&gt;Your server feels slow, but CPU and memory look fine. The culprit is often the network: a process hammering an external API, a container leaking connections, a misconfigured queue worker retrying thousands of failed jobs per minute. If you run a few Linux servers without a dedicated SRE, knowing which commands to reach for, and what numbers actually matter, saves you from chasing the wrong thing for an hour.&lt;/p&gt;

&lt;h2&gt;
  
  
  What you are actually trying to answer
&lt;/h2&gt;

&lt;p&gt;Network monitoring on Linux usually comes down to four questions: How much bandwidth is each interface using right now? Which process or connection is responsible? Is the volume trending in a direction that will cause a problem? And are any external dependencies (SSL certificates, DNS, upstream APIs) quietly failing? Those four questions need different tools, and most guides only cover the first one.&lt;/p&gt;

&lt;h2&gt;
  
  
  The tools worth knowing, and when to use each one
&lt;/h2&gt;

&lt;p&gt;Most Linux distributions ship with or can quickly install the following. You do not need all of them; knowing what each one is good at stops you from reaching for the wrong one.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;ss: the modern replacement for netstat. Shows every open socket, which process owns it, and its state. Fast and accurate. Use it first.&lt;/li&gt;
&lt;li&gt;ip -s link: reads interface counters directly from the kernel. Good for a quick bytes-in/bytes-out snapshot without installing anything.&lt;/li&gt;
&lt;li&gt;nload: a terminal dashboard showing per-interface throughput in real time, in and out, with a rolling average. Easy to read at a glance.&lt;/li&gt;
&lt;li&gt;iftop: shows bandwidth per connection pair (source IP to destination IP), sorted by volume. Useful for spotting a specific remote host consuming disproportionate bandwidth.&lt;/li&gt;
&lt;li&gt;nethogs: groups bandwidth by process rather than by connection. This is the tool to reach for when you know the interface is busy but not which process is responsible.&lt;/li&gt;
&lt;li&gt;tcpdump: raw packet capture. Overkill for everyday monitoring but essential for diagnosing TLS handshake failures, DNS misbehaviour, or unexpected traffic on a port.&lt;/li&gt;
&lt;li&gt;vnstat: logs interface traffic to a small database and lets you query hourly, daily, and monthly totals. Lightweight and persistent across reboots.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  A practical command sequence for diagnosing traffic in five minutes
&lt;/h2&gt;

&lt;p&gt;Run these in order. Each one narrows the search. The expected output comments explain what you are looking for.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="c"&gt;# 1. Check raw interface counters (no install needed)&lt;/span&gt;
ip &lt;span class="nt"&gt;-s&lt;/span&gt; &lt;span class="nb"&gt;link &lt;/span&gt;show eth0
&lt;span class="c"&gt;# Look for: RX errors &amp;gt; 0 or TX dropped &amp;gt; 0 -- these suggest a hardware or MTU issue&lt;/span&gt;

&lt;span class="c"&gt;# 2. See total throughput per interface in real time&lt;/span&gt;
&lt;span class="c"&gt;# Install: apt install nload   OR   yum install nload&lt;/span&gt;
nload eth0
&lt;span class="c"&gt;# Look for: Curr (current) vs Avg; spikes well above the average warrant investigation&lt;/span&gt;
&lt;span class="c"&gt;# Press right arrow to switch interfaces; q to quit&lt;/span&gt;

&lt;span class="c"&gt;# 3. Find which remote hosts are taking the most bandwidth&lt;/span&gt;
&lt;span class="c"&gt;# Install: apt install iftop   OR   yum install iftop&lt;/span&gt;
&lt;span class="nb"&gt;sudo &lt;/span&gt;iftop &lt;span class="nt"&gt;-i&lt;/span&gt; eth0 &lt;span class="nt"&gt;-n&lt;/span&gt;
&lt;span class="c"&gt;# -n suppresses DNS lookups so the display updates faster&lt;/span&gt;
&lt;span class="c"&gt;# Look for: any single host consuming &amp;gt; 50% of visible bandwidth&lt;/span&gt;
&lt;span class="c"&gt;# Press p to toggle ports; q to quit&lt;/span&gt;

&lt;span class="c"&gt;# 4. Find which local process owns the traffic&lt;/span&gt;
&lt;span class="c"&gt;# Install: apt install nethogs   OR   yum install nethogs&lt;/span&gt;
&lt;span class="nb"&gt;sudo &lt;/span&gt;nethogs eth0
&lt;span class="c"&gt;# Look for: a process with send+received KB/s far above the others&lt;/span&gt;
&lt;span class="c"&gt;# Press q to quit&lt;/span&gt;

&lt;span class="c"&gt;# 5. Check open connections and listening ports&lt;/span&gt;
ss &lt;span class="nt"&gt;-tulnp&lt;/span&gt;
&lt;span class="c"&gt;# -t TCP  -u UDP  -l listening  -n numeric  -p process&lt;/span&gt;
&lt;span class="c"&gt;# Look for: unexpected LISTEN entries on ports you did not open&lt;/span&gt;

&lt;span class="c"&gt;# 6. Count connections per remote IP (useful for spotting floods)&lt;/span&gt;
ss &lt;span class="nt"&gt;-n&lt;/span&gt; state established &lt;span class="s1"&gt;'( dport = :443 or dport = :80 )'&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
  | &lt;span class="nb"&gt;awk&lt;/span&gt; &lt;span class="s1"&gt;'{print $5}'&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
  | &lt;span class="nb"&gt;cut&lt;/span&gt; &lt;span class="nt"&gt;-d&lt;/span&gt;: &lt;span class="nt"&gt;-f1&lt;/span&gt; &lt;span class="se"&gt;\&lt;/span&gt;
  | &lt;span class="nb"&gt;sort&lt;/span&gt; | &lt;span class="nb"&gt;uniq&lt;/span&gt; &lt;span class="nt"&gt;-c&lt;/span&gt; | &lt;span class="nb"&gt;sort&lt;/span&gt; &lt;span class="nt"&gt;-rn&lt;/span&gt; | &lt;span class="nb"&gt;head&lt;/span&gt; &lt;span class="nt"&gt;-20&lt;/span&gt;
&lt;span class="c"&gt;# Look for: any single IP with &amp;gt; 100 simultaneous connections&lt;/span&gt;

&lt;span class="c"&gt;# 7. Check historical daily traffic totals&lt;/span&gt;
&lt;span class="c"&gt;# Install: apt install vnstat   OR   yum install vnstat&lt;/span&gt;
&lt;span class="c"&gt;# Then enable and start: systemctl enable --now vnstat&lt;/span&gt;
vnstat &lt;span class="nt"&gt;-d&lt;/span&gt;
&lt;span class="c"&gt;# Look for: days where RX or TX is 2-3x the usual daily average&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  What to alert on: concrete thresholds
&lt;/h2&gt;

&lt;p&gt;Tools give you numbers. Thresholds turn numbers into actionable signals. The right threshold depends on your server's role, but the following starting points work for most application servers.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Inbound bandwidth sustained above 80% of your provisioned link for more than five minutes: alert immediately. On a 1 Gbps port, that is 800 Mbps. On a shared 100 Mbps DigitalOcean droplet, it is 80 Mbps.&lt;/li&gt;
&lt;li&gt;TX errors or RX drops on an interface greater than zero over a five-minute window: worth a warning. A handful of drops is normal; a steady climb suggests a congested uplink or a misconfigured MTU.&lt;/li&gt;
&lt;li&gt;Established TCP connections to a single remote IP exceeding 200: investigate for a connection leak or an aggressive client. Web scraper traffic often looks like this.&lt;/li&gt;
&lt;li&gt;CLOSE_WAIT connections growing and not draining: the local application is not closing sockets properly. Check your application process first with nethogs.&lt;/li&gt;
&lt;li&gt;A new LISTEN port that was not there yesterday: treat as a security event until proven otherwise. ss -tulnp captures a snapshot; diff two snapshots taken 24 hours apart to spot changes.&lt;/li&gt;
&lt;li&gt;DNS resolution time exceeding 200 ms on repeated lookups: your upstream resolver may be failing over. dig +stats your-domain.com @8.8.8.8 shows the query time in the stats section.&lt;/li&gt;
&lt;/ul&gt;

&lt;blockquote&gt;
&lt;p&gt;Rule of thumb: before you add network capacity, check whether a single process or a single remote IP is responsible. Scaling the network when one misbehaving process is the cause wastes money and leaves the real problem in place.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;h2&gt;
  
  
  How to monitor containers, services, and SSL alongside network traffic
&lt;/h2&gt;

&lt;p&gt;Network traffic rarely fails in isolation. A spike in outbound traffic from a container often means a queue worker is retrying failed jobs at high frequency, or a deployment pushed bad code that is logging errors in a tight loop. Watching the network without watching what is generating the traffic leaves you guessing.&lt;/p&gt;

&lt;p&gt;For Docker containers, cgroup-level network counters are exposed in /sys/fs/cgroup/net_cls and through the Docker stats API. Running docker stats --no-stream gives you a per-container snapshot of network I/O alongside CPU and memory. If you run Laravel on a server and use Horizon or Supervisor to manage queue workers, a worker that is crash-looping will produce a burst of short-lived connections that look like a traffic spike in iftop. The guide on &lt;a href="https://getopservo.com/guides/monitor-docker-containers-on-a-vps" rel="noopener noreferrer"&gt;how to monitor containers on a VPS&lt;/a&gt; covers the cgroup approach in more detail.&lt;/p&gt;

&lt;p&gt;SSL certificate expiry is a network-adjacent problem that catches teams off guard. A certificate that expires quietly takes down HTTPS traffic with no process crash and no disk alert. You can check expiry from the command line with: echo | openssl s_client -connect yourdomain.com:443 -servername yourdomain.com 2&amp;gt;/dev/null | openssl x509 -noout -dates. The notAfter line tells you when it expires. Automate a check that alerts at 30 days and again at 7 days.&lt;/p&gt;

&lt;p&gt;DNS monitoring sits in the same category. A domain whose nameserver TTL has changed, or whose A record has been altered, will cause traffic to disappear rather than spike. A daily dig check against an authoritative nameserver and a trusted public resolver lets you spot divergence quickly.&lt;/p&gt;

&lt;h2&gt;
  
  
  When manual tools are not enough for a production fleet
&lt;/h2&gt;

&lt;p&gt;The commands above are exactly what you should run during an active incident. For ongoing visibility across multiple servers, you need something that runs those checks continuously, stores the trends, and tells you when a threshold is crossed, without you staying logged in. A linux server monitoring tool that auto-discovers listening ports, tracks connection counts over time, and correlates a network spike with a log burst saves significant investigation time.&lt;/p&gt;

&lt;p&gt;Opservo installs with a single command and auto-discovers listening ports, Docker containers, systemd services, and SSL certificate expiry on every server it monitors. When a network threshold is crossed, it correlates the spike with log entries from the same time window and surfaces a plain-English explanation of what changed. For Laravel monitoring, it watches Horizon and Supervisor workers and alerts if one stops or enters a crash loop, which is often the source of that unexpected outbound traffic surge. Remote actions are opt-in per server, so every fix command is approved and audited before it runs.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="c"&gt;# Install the Opservo agent (Linux only; single static binary under 10 MB)&lt;/span&gt;
curl &lt;span class="nt"&gt;-sSL&lt;/span&gt; https://api.getopservo.com/install.sh | &lt;span class="nb"&gt;sudo &lt;/span&gt;bash
&lt;span class="c"&gt;# No inbound ports opened; all traffic is outbound HTTPS&lt;/span&gt;
&lt;span class="c"&gt;# First findings appear within five minutes&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;blockquote&gt;
&lt;p&gt;Install the Opservo agent on up to two servers free (no card required) to get continuous network-threshold alerts, port-change detection, and SSL expiry monitoring without staying logged in. — one command installs a read-only agent and your first findings arrive in five minutes: &lt;a href="https://getopservo.com/signup?utm_source=blog&amp;amp;utm_medium=post&amp;amp;utm_campaign=auto-blog" rel="noopener noreferrer"&gt;start free&lt;/a&gt;. Related: &lt;a href="https://getopservo.com/guides/monitor-docker-containers-on-a-vps" rel="noopener noreferrer"&gt;guides monitor docker containers on a vps&lt;/a&gt;.&lt;/p&gt;
&lt;/blockquote&gt;




&lt;p&gt;&lt;em&gt;Originally published on the &lt;a href="https://getopservo.com/blog/how-to-monitor-network-traffic-on-linux?utm_source=devto&amp;amp;utm_medium=syndication&amp;amp;utm_campaign=syndication" rel="noopener noreferrer"&gt;Opservo blog&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>linux</category>
      <category>devops</category>
      <category>monitoring</category>
      <category>operations</category>
    </item>
    <item>
      <title>Our filesystem went read-only for 19 days. Nobody saw the warning.</title>
      <dc:creator>Opservo</dc:creator>
      <pubDate>Tue, 20 Oct 2026 12:58:41 +0000</pubDate>
      <link>https://dev.to/opservo/our-filesystem-went-read-only-for-19-days-nobody-saw-the-warning-58oj</link>
      <guid>https://dev.to/opservo/our-filesystem-went-read-only-for-19-days-nobody-saw-the-warning-58oj</guid>
      <description>&lt;p&gt;One of our servers fired a kernel filesystem-error alert. Nine days later, the OS remounted the filesystem read-only. Ten days after the first alert, a service on the same server entered a failed state. All three alerts sat unacknowledged for between 11 and 19 days. No alert channel had been configured for that server, so every notification went nowhere.&lt;/p&gt;

&lt;h2&gt;
  
  
  What the alerts actually said
&lt;/h2&gt;

&lt;p&gt;The first alert read: "Kernel reported filesystem errors -- schedule an fsck and check the underlying disk; back up first." It opened on 2026-09-10 and was notified once. It stayed open for about 11 days.&lt;/p&gt;

&lt;p&gt;8.6 days after the first alert, a second one opened: "A filesystem was remounted read-only after errors -- a read-only remount means the disk or filesystem hit errors. Check SMART health, run fsck on the affected mount, and expect a reboot." That alert ran for about 19 days before it closed. Also notified once, never acknowledged.&lt;/p&gt;

&lt;p&gt;10.8 days after the first alert, a third opened: "Service apport-autoreport is down." That one ran for about 17 days. Same story: one notification, no acknowledgement.&lt;/p&gt;

&lt;p&gt;We are not claiming these three alerts are causally linked. What we can say is that they appeared on the same server in that order, and none of them reached a human.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why a read-only remount is a serious signal
&lt;/h2&gt;

&lt;p&gt;When the Linux kernel detects filesystem errors it cannot recover from, it remounts the affected filesystem read-only as a last resort to stop further corruption. At that point, any process trying to write -- a database, a web server, a log daemon -- either fails silently or crashes. Applications may appear to be running while actually writing nothing.&lt;/p&gt;

&lt;p&gt;The kernel logs the event in the system journal. It also shows up in &lt;code&gt;dmesg&lt;/code&gt;. The problem is that neither of those surfaces alerts you unless something is watching.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;A read-only remount does not mean your data is safe. It means the kernel stopped accepting writes before things got worse. The underlying disk problem is still there.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;h2&gt;
  
  
  How to check your own servers right now
&lt;/h2&gt;

&lt;p&gt;Run these commands on any Linux server you care about. The first block checks for a current read-only mount; the second checks the kernel ring buffer for recent filesystem errors; the third runs a SMART health check on the underlying block device.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="c"&gt;# 1. Find any filesystem currently mounted read-only&lt;/span&gt;
&lt;span class="nb"&gt;cat&lt;/span&gt; /proc/mounts | &lt;span class="nb"&gt;awk&lt;/span&gt; &lt;span class="s1"&gt;'$4 ~ /(^|,)ro(,|$)/ {print $2, $3, $4}'&lt;/span&gt;

&lt;span class="c"&gt;# 2. Check the kernel log for filesystem errors in the last 500 lines&lt;/span&gt;
dmesg | &lt;span class="nb"&gt;tail&lt;/span&gt; &lt;span class="nt"&gt;-500&lt;/span&gt; | &lt;span class="nb"&gt;grep&lt;/span&gt; &lt;span class="nt"&gt;-iE&lt;/span&gt; &lt;span class="s1"&gt;'error|remount|read-only|EXT4-fs|XFS|I/O error'&lt;/span&gt;

&lt;span class="c"&gt;# 3. Check SMART health on your primary disk (replace sda with your device)&lt;/span&gt;
&lt;span class="nb"&gt;sudo &lt;/span&gt;smartctl &lt;span class="nt"&gt;-H&lt;/span&gt; /dev/sda
&lt;span class="c"&gt;# "SMART overall-health self-assessment test result: PASSED" is what you want&lt;/span&gt;
&lt;span class="c"&gt;# "FAILED" means the drive is reporting imminent failure&lt;/span&gt;

&lt;span class="c"&gt;# 4. Look for recent journal-level filesystem errors&lt;/span&gt;
journalctl &lt;span class="nt"&gt;-p&lt;/span&gt; err &lt;span class="nt"&gt;-b&lt;/span&gt; | &lt;span class="nb"&gt;grep&lt;/span&gt; &lt;span class="nt"&gt;-iE&lt;/span&gt; &lt;span class="s1"&gt;'filesystem|disk|I/O|remount|fsck'&lt;/span&gt;

&lt;span class="c"&gt;# 5. Check for any failed mount units&lt;/span&gt;
systemctl &lt;span class="nt"&gt;--failed&lt;/span&gt; &lt;span class="nt"&gt;--type&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;mount
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;If &lt;code&gt;dmesg&lt;/code&gt; shows lines like &lt;code&gt;EXT4-fs error (device sda1)&lt;/code&gt; or &lt;code&gt;XFS: corruption of in-memory data detected&lt;/code&gt;, stop writes to that filesystem if you can, take a backup immediately, and schedule an fsck on the unmounted device. Do not run fsck on a mounted filesystem.&lt;/p&gt;

&lt;h2&gt;
  
  
  What a SMART check actually tells you
&lt;/h2&gt;

&lt;p&gt;SMART (Self-Monitoring, Analysis and Reporting Technology) is built into every modern hard drive and SSD. The tool &lt;code&gt;smartctl&lt;/code&gt; reads it. Install it with &lt;code&gt;sudo apt install smartmontools&lt;/code&gt; or &lt;code&gt;sudo yum install smartmontools&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;The attributes that matter most for predicting failure are: &lt;code&gt;Reallocated_Sector_Ct&lt;/code&gt; (non-zero means the drive has already remapped bad sectors), &lt;code&gt;Current_Pending_Sector&lt;/code&gt; (sectors the drive has flagged as unstable), and &lt;code&gt;Offline_Uncorrectable&lt;/code&gt; (sectors the drive could not correct during offline scanning). Any non-zero value in those three columns deserves attention.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;&lt;span class="c"&gt;# Full attribute list; look for non-zero RAW_VALUE in columns 5, 197, 198&lt;/span&gt;
&lt;span class="nb"&gt;sudo &lt;/span&gt;smartctl &lt;span class="nt"&gt;-A&lt;/span&gt; /dev/sda | &lt;span class="nb"&gt;grep&lt;/span&gt; &lt;span class="nt"&gt;-E&lt;/span&gt; &lt;span class="s1"&gt;'Reallocated|Pending|Uncorrectable'&lt;/span&gt;

&lt;span class="c"&gt;# Example output you do NOT want to see:&lt;/span&gt;
&lt;span class="c"&gt;#   5  Reallocated_Sector_Ct   0x0033  100  100  036  Pre-fail  Always  -  17&lt;/span&gt;
&lt;span class="c"&gt;# 197  Current_Pending_Sector  0x0012  100  100  000  Old_age   Always  -  4&lt;/span&gt;
&lt;span class="c"&gt;# 198  Offline_Uncorrectable   0x0010  100  100  000  Old_age   Offline -  1&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;h2&gt;
  
  
  The real mistake: no alert channel on that server
&lt;/h2&gt;

&lt;p&gt;All three alerts were generated and each was notified once. The first alert's detail even said "back up first". But we had not configured an alert channel for that server, so each notification had nowhere to go. Zero channels were configured, and no person acknowledged any of the three alerts.&lt;/p&gt;

&lt;p&gt;We have written about this failure mode before in "Our alerts went nowhere for 59 days. Here is what we changed." This is a different server, a different set of alerts, and the same root cause: alert channel not configured.&lt;/p&gt;

&lt;p&gt;The lesson is not about disk health specifically. It is that any server without a configured alert destination is a server you are monitoring in name only. Go through your fleet and verify that every server has at least one channel -- email is enough -- and that a test alert actually arrives.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;Monitoring without a delivery channel is a log you never read. The alert count does not matter; the delivery count does.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;h2&gt;
  
  
  How Opservo surfaces this
&lt;/h2&gt;

&lt;p&gt;Opservo watches for kernel filesystem errors and read-only remounts as health-category alerts, separate from lower-priority recommendations. When a server has no alert channel configured, the dashboard flags it explicitly so you can see the gap before an incident reveals it. The health score docks points for both the filesystem condition and the missing notification path, so a server in this state cannot quietly score well.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;Try Opservo free on 2 servers — one command installs a read-only agent and your first findings arrive in five minutes: &lt;a href="https://getopservo.com/signup?utm_source=blog&amp;amp;utm_medium=post&amp;amp;utm_campaign=auto-blog" rel="noopener noreferrer"&gt;start free&lt;/a&gt;.&lt;/p&gt;
&lt;/blockquote&gt;




&lt;p&gt;&lt;em&gt;Originally published on the &lt;a href="https://getopservo.com/blog/our-filesystem-went-read-only-for-19-days-nobody-saw-the-warning?utm_source=devto&amp;amp;utm_medium=syndication&amp;amp;utm_campaign=syndication" rel="noopener noreferrer"&gt;Opservo blog&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>linux</category>
      <category>devops</category>
      <category>monitoring</category>
      <category>operations</category>
    </item>
    <item>
      <title>Why our server health score is not AI</title>
      <dc:creator>Opservo</dc:creator>
      <pubDate>Sun, 18 Oct 2026 07:35:47 +0000</pubDate>
      <link>https://dev.to/opservo/why-our-server-health-score-is-not-ai-3g0g</link>
      <guid>https://dev.to/opservo/why-our-server-health-score-is-not-ai-3g0g</guid>
      <description>&lt;p&gt;Opservo uses AI in a lot of places. It explains what is wrong with a server in plain English, clusters log lines into patterns, forecasts when a disk will fill, and answers questions in a chat grounded in live data from the server. One thing it deliberately does not use AI for is the health score.&lt;/p&gt;

&lt;h2&gt;
  
  
  The problem with a score you cannot question
&lt;/h2&gt;

&lt;p&gt;Most monitoring tools that show a single "health" number do not tell you how they got it. If a server drops from 92 to 71 overnight, you are left guessing: was it the disk, the certificate, a service that restarted? A number you cannot question is a number you learn to ignore.&lt;/p&gt;

&lt;p&gt;We run 40+ production servers at Softuvo, mostly looked after by developers doing ops on the side. For them, a score is only useful if it answers the next question on its own: what lost the points, and what do I do about it?&lt;/p&gt;

&lt;h2&gt;
  
  
  What the Opservo Health Standard does instead
&lt;/h2&gt;

&lt;p&gt;Every server gets a score from 0 to 100, built from seven areas: Performance, Security, Storage, Reliability, Maintenance, Network and Configuration. The score comes from a fixed set of rules, not a model. Two things follow from that:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;It is deterministic. The same server in the same state always gets the same score. Nothing about the score depends on how a model felt that day.&lt;/li&gt;
&lt;li&gt;Every lost point is explained. Each deduction names four things: the rule, the threshold, the current value, and the fix. If you disagree with a deduction, you know exactly what you are disagreeing with.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fn3xsuwzrwjvs91nlv404.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fn3xsuwzrwjvs91nlv404.png" alt="An Opservo server health score with each lost point showing its rule, threshold, current value and fix" width="800" height="479"&gt;&lt;/a&gt;&lt;/p&gt;

&lt;h2&gt;
  
  
  Where the AI comes in
&lt;/h2&gt;

&lt;p&gt;The score tells you what. The AI helps with why and what next. When a server loses points, Opservo explains in plain English what changed and why it matters, and when the fix is known you can apply it with one click or a policy you approved in advance. The agent is read-only by default, and remote actions are opt-in per server and audited.&lt;/p&gt;

&lt;p&gt;Keeping the two apart means the AI can be wrong in a sentence without making the score wrong. You can always fall back to the rule, the threshold and the value.&lt;/p&gt;

&lt;h2&gt;
  
  
  Three questions to ask of any health score
&lt;/h2&gt;

&lt;ul&gt;
&lt;li&gt;Can it show you every point it took away? If not, you cannot tell a real problem from noise.&lt;/li&gt;
&lt;li&gt;Does the same state give the same number? If the score moves when nothing on the server changed, you will stop trusting it.&lt;/li&gt;
&lt;li&gt;Does each deduction come with a fix? A score that only grades you is a report card. A score that tells you what to do next is a to-do list.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Try it on your worst server
&lt;/h2&gt;

&lt;p&gt;The full standard is public at &lt;a href="https://getopservo.com/health-score" rel="noopener noreferrer"&gt;getopservo.com/health-score&lt;/a&gt;. Opservo is the AI ops engineer for teams without an SRE: plain-English server monitoring with one-click fixes. Free for 2 servers. &lt;a href="https://getopservo.com/welcome" rel="noopener noreferrer"&gt;Start at getopservo.com/welcome&lt;/a&gt;.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;Try Opservo free on 2 servers — one command installs a read-only agent and your first findings arrive in five minutes: &lt;a href="https://getopservo.com/signup?utm_source=blog&amp;amp;utm_medium=post&amp;amp;utm_campaign=auto-blog" rel="noopener noreferrer"&gt;start free&lt;/a&gt;.&lt;/p&gt;
&lt;/blockquote&gt;




&lt;p&gt;&lt;em&gt;Originally published on the &lt;a href="https://getopservo.com/blog/why-our-server-health-score-is-not-ai?utm_source=devto&amp;amp;utm_medium=syndication&amp;amp;utm_campaign=syndication" rel="noopener noreferrer"&gt;Opservo blog&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>linux</category>
      <category>devops</category>
      <category>monitoring</category>
      <category>engineering</category>
    </item>
    <item>
      <title>How to Monitor Disk I/O Latency on a Linux Server and Find the Bottleneck</title>
      <dc:creator>Opservo</dc:creator>
      <pubDate>Sun, 18 Oct 2026 06:37:17 +0000</pubDate>
      <link>https://dev.to/opservo/how-to-monitor-disk-io-latency-on-a-linux-server-and-find-the-bottleneck-20nn</link>
      <guid>https://dev.to/opservo/how-to-monitor-disk-io-latency-on-a-linux-server-and-find-the-bottleneck-20nn</guid>
      <description>&lt;p&gt;Your app feels sluggish, queries are timing out, and top shows CPU is fine. Nine times out of ten the culprit is disk I/O — but knowing that doesn't tell you which disk, which process, or whether it's a hardware problem or just one runaway job hammering the filesystem. This guide walks through exactly how to measure I/O latency on a Linux server and narrow it down to the real bottleneck.&lt;/p&gt;

&lt;h2&gt;
  
  
  Start With iostat — Your First View Into Disk Performance
&lt;/h2&gt;

&lt;p&gt;The fastest way to get a snapshot of disk activity is iostat, part of the sysstat package. If it's not installed, run: sudo apt install sysstat (Debian/Ubuntu) or sudo yum install sysstat (RHEL/CentOS).&lt;/p&gt;

&lt;p&gt;Run it with extended stats and a refresh interval: iostat -xz 2 10 — that gives you extended metrics, hides idle devices, refreshes every 2 seconds, and runs 10 times. The columns that matter most for latency are:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;await — average time (ms) for I/O requests to complete, including queue wait. This is your headline latency number.&lt;/li&gt;
&lt;li&gt;r_await / w_await — separate read and write latency, so you can tell if it's reads or writes causing pain.&lt;/li&gt;
&lt;li&gt;svctm — deprecated, ignore it.&lt;/li&gt;
&lt;li&gt;%util — how busy the device is. Near 100% means it's saturated, but on SSDs and NVMe drives this can be misleading because they handle parallel I/O.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;A healthy NVMe SSD should show await below 1 ms. A SATA SSD might run 1–5 ms. A spinning disk is acceptable around 5–20 ms but anything above that consistently signals a problem. If you see await spiking to 100 ms or more, something is seriously wrong.&lt;/p&gt;

&lt;h2&gt;
  
  
  Use iotop to Find the Offending Process
&lt;/h2&gt;

&lt;p&gt;Once you know which disk is struggling, the next question is: who is hammering it? iotop shows per-process I/O in real time, similar to top for CPU. Install it with sudo apt install iotop, then run: sudo iotop -o — the -o flag shows only processes that are actively doing I/O, keeping the output clean.&lt;/p&gt;

&lt;p&gt;Look for the DISK READ and DISK WRITE columns. A process reading or writing hundreds of megabytes per second when it shouldn't be is your bottleneck. Note the PID, then cross-reference with ps aux or lsof -p  to see what files it's accessing.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;Pro tip: if iotop shows I/O is spread evenly across many processes rather than one dominant culprit, the problem is likely at the storage layer itself — a failing drive, a full disk cache, or a RAID controller under pressure.&lt;/p&gt;
&lt;/blockquote&gt;

&lt;h2&gt;
  
  
  Go Deeper With blktrace and biolatency
&lt;/h2&gt;

&lt;p&gt;If iostat and iotop point to a disk but not a clear cause, you need to look at the I/O stack more granularly. Two tools are especially useful here.&lt;/p&gt;

&lt;p&gt;blktrace captures every block I/O event on a device. Run sudo blktrace -d /dev/sda -o trace for 10–15 seconds, then stop it with Ctrl+C and analyse with blkparse: blkparse -i trace.blktrace.0 | tail -50. You'll see queue times vs. service times broken down by I/O type. This is verbose but reveals whether latency is happening in the kernel queue or at the hardware level.&lt;/p&gt;

&lt;p&gt;If you're running a modern kernel (4.7+) with BPF tools installed, biolatency from the BCC toolkit is far more ergonomic: sudo biolatency -d /dev/sda 1 10. It prints a histogram of I/O completion times, showing you the distribution rather than just averages. Averages hide bimodal latency — where most I/Os complete fast but a small percentage take seconds and drag down your p99 response times.&lt;/p&gt;

&lt;h2&gt;
  
  
  Check for Filesystem and Mount-Level Issues
&lt;/h2&gt;

&lt;p&gt;Sometimes the disk hardware is fine but the filesystem is the bottleneck. A few things to check:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Disk fullness — run df -h. Linux filesystems degrade badly above ~90% capacity. Inodes can also be exhausted separately: df -i.&lt;/li&gt;
&lt;li&gt;Dirty page writeback — cat /proc/meminfo | grep -i dirty. If dirty pages are consistently high (gigabytes), the kernel is accumulating writes faster than it can flush them. Tune vm.dirty_ratio and vm.dirty_background_ratio in /etc/sysctl.conf.&lt;/li&gt;
&lt;li&gt;Mount options — check /proc/mounts. If a database volume is mounted without noatime, every read triggers a metadata write. Add noatime to the mount options in /etc/fstab and remount.&lt;/li&gt;
&lt;li&gt;Filesystem errors — sudo dmesg | grep -i 'error|EXT4|XFS|I/O error'. Hardware errors show up here before drives fully fail.&lt;/li&gt;
&lt;/ul&gt;

&lt;h2&gt;
  
  
  Build a Baseline and Set Alerts Before It Gets Bad
&lt;/h2&gt;

&lt;p&gt;Reactive troubleshooting is fine, but the real goal is catching latency before it becomes an outage. Run iostat -xz 1 60 &amp;gt; baseline-$(date +%F).txt during normal operating hours and save the output. Now you have a reference for what normal looks like — which makes future spikes obvious rather than ambiguous.&lt;/p&gt;

&lt;p&gt;You can wire iostat into basic alerting by parsing await in a cron job or a shell script that pushes to Slack when it crosses a threshold. For something more automatic, Opservo continuously tracks I/O metrics across your servers, surfaces latency anomalies in plain English, and can correlate a slow-disk event with the specific process responsible — without you having to remember which command to run at 2 AM.&lt;/p&gt;

&lt;p&gt;Whether you choose to script your own monitoring or use a tool, the key habit is the same: don't wait for users to report slowness. Disk latency tends to creep up gradually — a drive entering the early stages of failure, a cron job that gets bigger every week, a log directory nobody ever cleans up. Measure it regularly and you'll almost always see the problem coming.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;Invite the reader to connect their server to Opservo's free 2-server plan so I/O latency and process-level disk activity are tracked automatically without keeping a cron script alive. — one command installs a read-only agent and your first findings arrive in five minutes: &lt;a href="https://getopservo.com/signup?utm_source=blog&amp;amp;utm_medium=post&amp;amp;utm_campaign=auto-blog" rel="noopener noreferrer"&gt;start free&lt;/a&gt;. Related: &lt;a href="https://getopservo.com/guides/why-is-my-server-slow" rel="noopener noreferrer"&gt;guides why is my server slow&lt;/a&gt;.&lt;/p&gt;
&lt;/blockquote&gt;




&lt;p&gt;&lt;em&gt;Originally published on the &lt;a href="https://getopservo.com/blog/how-to-monitor-disk-i-o-latency-on-a-linux-server-and-find-the-bottlen?utm_source=devto&amp;amp;utm_medium=syndication&amp;amp;utm_campaign=syndication" rel="noopener noreferrer"&gt;Opservo blog&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>linux</category>
      <category>devops</category>
      <category>monitoring</category>
      <category>guides</category>
    </item>
    <item>
      <title>How to Monitor and Alert on Redis Memory Fragmentation Before It Causes Latency Spikes</title>
      <dc:creator>Opservo</dc:creator>
      <pubDate>Sat, 10 Oct 2026 15:36:24 +0000</pubDate>
      <link>https://dev.to/opservo/how-to-monitor-and-alert-on-redis-memory-fragmentation-before-it-causes-latency-spikes-3227</link>
      <guid>https://dev.to/opservo/how-to-monitor-and-alert-on-redis-memory-fragmentation-before-it-causes-latency-spikes-3227</guid>
      <description>&lt;p&gt;Your Redis instance looks healthy — low CPU, plenty of free memory, stable connection count. Then latency spikes out of nowhere, and you spend an hour debugging before realizing the culprit was memory fragmentation. It's one of the most common Redis performance issues that teams miss because it doesn't show up in the obvious metrics. Here's how to catch it before it bites you.&lt;/p&gt;

&lt;h2&gt;
  
  
  What Redis Memory Fragmentation Actually Is
&lt;/h2&gt;

&lt;p&gt;Redis allocates memory in chunks through its memory allocator (jemalloc by default). Over time, as keys are created, updated, and deleted at varying sizes, gaps form between those chunks — memory the OS has handed to Redis but Redis isn't actively using for data. The ratio between the memory Redis holds from the OS versus the memory Redis actually uses for data is your fragmentation ratio. A ratio of 1.0 means perfect efficiency. Anything significantly above 1.5 is where problems start, and above 2.0 you'll almost certainly see latency impact.&lt;/p&gt;

&lt;p&gt;Fragmentation hurts performance because the allocator has to work harder to find contiguous memory regions, and in extreme cases Redis will trigger active defragmentation, which competes with your normal workload. The insidious part: your application sees slow responses but your system-level memory graphs look fine.&lt;/p&gt;

&lt;h2&gt;
  
  
  The Metrics You Need to Watch
&lt;/h2&gt;

&lt;p&gt;Everything you need lives inside Redis INFO output. Run this against your instance:&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;redis-cli -h 127.0.0.1 -p 6379 INFO memory&lt;/p&gt;
&lt;/blockquote&gt;

&lt;p&gt;The key fields to extract:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;used_memory — bytes Redis allocated for data&lt;/li&gt;
&lt;li&gt;used_memory_rss — bytes the OS reports Redis is using (resident set size)&lt;/li&gt;
&lt;li&gt;mem_fragmentation_ratio — the ratio of rss to used_memory; this is your primary signal&lt;/li&gt;
&lt;li&gt;mem_fragmentation_bytes — absolute bytes wasted, useful for context&lt;/li&gt;
&lt;li&gt;active_defrag_running — 1 means Redis is actively defragmenting right now&lt;/li&gt;
&lt;li&gt;allocator_frag_ratio — fragmentation inside the allocator itself, separate from OS-level fragmentation&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Don't rely on mem_fragmentation_ratio alone. A ratio of 1.8 on an instance using 500 MB of data is very different from 1.8 on an instance using 50 GB. Always look at mem_fragmentation_bytes alongside it — if you're wasting 200 MB, that's a real problem; if you're wasting 4 MB, it's noise.&lt;/p&gt;

&lt;h2&gt;
  
  
  Setting Meaningful Alert Thresholds
&lt;/h2&gt;

&lt;p&gt;Generic thresholds don't work well here because workloads differ. That said, sensible starting points are:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;mem_fragmentation_ratio &amp;gt; 1.5 — warning; begin watching closely&lt;/li&gt;
&lt;li&gt;mem_fragmentation_ratio &amp;gt; 2.0 — critical; investigate immediately&lt;/li&gt;
&lt;li&gt;mem_fragmentation_bytes &amp;gt; 500 MB — critical regardless of ratio&lt;/li&gt;
&lt;li&gt;active_defrag_running = 1 for more than a few minutes — worth noting; it means Redis already decided things were bad enough to act&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;If you're using Prometheus with the redis_exporter, you can alert on these directly. A Prometheus alerting rule for the critical case looks like this:&lt;/p&gt;

&lt;blockquote&gt;
&lt;ul&gt;
&lt;li&gt;alert: RedisHighFragmentation
expr: redis_memory_used_rss_bytes / redis_memory_used_bytes &amp;gt; 2.0
for: 5m
labels:
severity: critical
annotations:
summary: "Redis memory fragmentation ratio above 2.0 for 5 minutes"&lt;/li&gt;
&lt;/ul&gt;
&lt;/blockquote&gt;

&lt;p&gt;The for: 5m clause matters. Fragmentation naturally spikes briefly during bulk deletes or expiry storms, so you want to alert on sustained high fragmentation, not momentary blips.&lt;/p&gt;

&lt;h2&gt;
  
  
  How to Reduce Fragmentation When It Gets Bad
&lt;/h2&gt;

&lt;p&gt;Once you've caught fragmentation early, you have a few options depending on severity:&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;Enable active defragmentation — if you're on Redis 4.0+, you can turn this on at runtime: redis-cli CONFIG SET activedefrag yes. Redis will gradually repack memory in the background. Set active-defrag-ignore-bytes 100mb and active-defrag-enabled yes in your config to make it permanent.&lt;/li&gt;
&lt;li&gt;Restart Redis — the nuclear option, but sometimes the fastest path. Redis will reload from an RDB or AOF snapshot and start with a clean memory layout. Only practical if you have replication set up and can failover cleanly.&lt;/li&gt;
&lt;li&gt;Tune your key expiry and eviction policy — fragmentation often worsens with volatile-lru or allkeys-lru eviction under memory pressure. If you're evicting constantly, reconsider your maxmemory setting or data model.&lt;/li&gt;
&lt;li&gt;Reduce key churn for large values — frequent updates to large string or hash values are a primary driver. Where possible, batch updates or restructure data to avoid constantly reallocating big chunks.&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Check whether active defrag is already enabled before you turn it on: redis-cli CONFIG GET activedefrag. If it's already yes and fragmentation is still climbing, you likely have a structural data problem rather than a tuning one.&lt;/p&gt;

&lt;h2&gt;
  
  
  Making This Easier to Catch Automatically
&lt;/h2&gt;

&lt;p&gt;The manual approach — periodically running INFO memory and checking the numbers — works fine until you have more than one Redis instance or a team that doesn't watch dashboards closely. Most latency incidents caused by fragmentation are preventable, but only if someone notices the ratio creeping up over hours or days before it becomes critical.&lt;/p&gt;

&lt;p&gt;This is exactly where a tool like Opservo helps — it continuously monitors Redis memory metrics, surfaces fragmentation trends in plain English, and can alert your team before the ratio crosses critical thresholds rather than after latency has already degraded. Instead of digging through INFO output during an incident, you get an early warning with context.&lt;/p&gt;

&lt;p&gt;Whether you go the manual route or automate it, the core habit is the same: treat mem_fragmentation_ratio as a first-class metric alongside hit rate and latency. It's not exotic telemetry — it's one of the most reliable early warning signals Redis gives you, and it's sitting there in INFO memory waiting to be used.&lt;/p&gt;

&lt;blockquote&gt;
&lt;p&gt;Point readers to Opservo's free 2-server plan as the easiest way to get continuous fragmentation alerting without building a Prometheus stack. — one command installs a read-only agent and your first findings arrive in five minutes: &lt;a href="https://getopservo.com/signup?utm_source=blog&amp;amp;utm_medium=post&amp;amp;utm_campaign=auto-blog" rel="noopener noreferrer"&gt;start free&lt;/a&gt;. Related: &lt;a href="https://getopservo.com/blog/how-to-monitor-redis-memory-and-eviction" rel="noopener noreferrer"&gt;blog how to monitor redis memory and eviction&lt;/a&gt;.&lt;/p&gt;
&lt;/blockquote&gt;




&lt;p&gt;&lt;em&gt;Originally published on the &lt;a href="https://getopservo.com/blog/how-to-monitor-and-alert-on-redis-memory-fragmentation-before-it-cause?utm_source=devto&amp;amp;utm_medium=syndication&amp;amp;utm_campaign=syndication" rel="noopener noreferrer"&gt;Opservo blog&lt;/a&gt;.&lt;/em&gt;&lt;/p&gt;

</description>
      <category>linux</category>
      <category>devops</category>
      <category>monitoring</category>
      <category>operations</category>
    </item>
  </channel>
</rss>
