DEV Community

Cover image for When Your Server Runs Out of Disk Space: Prevention, Detection, and Recovery
Deploynix
Deploynix

Posted on • Originally published at deploynix.io

When Your Server Runs Out of Disk Space: Prevention, Detection, and Recovery

#laravel #deploynix #diskspace #troubleshooting

A full disk is one of the most destructive server failures. It does not happen gradually from the application's perspective — everything works fine until the moment it does not. Then, in rapid succession: log writes fail, database transactions abort, PHP sessions cannot be saved, and your application crashes with cryptic errors that point everywhere except the actual cause.

The worst part is that disk space exhaustion is entirely preventable. Yet it remains one of the most common causes of production outages, especially on cloud servers with modest 25-40 GB SSDs.

This guide covers the three phases of dealing with disk space: preventing it from filling up, detecting when it is getting close, and recovering when it is already full.

What Fills Up Your Disk

On a typical Deploynix-managed Laravel server, disk consumption comes from five sources, roughly in order of how sneakily they grow:

1. Application Logs

Laravel's default logging configuration writes to storage/logs/laravel.log. In production, a busy application can generate hundreds of megabytes of log data per day. Error logs are especially voluminous — a single unhandled exception with a full stack trace can be several kilobytes, and if the error occurs on every request, the log file grows by gigabytes per hour.

The daily log channel helps by creating a new file each day (laravel-2026-03-18.log), but without a retention limit, those daily files accumulate indefinitely.

2. Old Deployment Releases

Deploynix uses a symlinked release structure for zero-downtime deployments. Each deployment creates a new release directory containing a full copy of your application code (minus shared directories like storage and vendor). If you deploy five times per day and keep releases indefinitely, each 50 MB release adds up quickly: 250 MB per day, 7.5 GB per month.

Deploynix automatically removes old releases beyond the configured retention count. The default is typically 5 releases. If your deployment configuration retains more, or if the cleanup step fails silently, releases accumulate.

3. Database Dumps and Backups

Manual database dumps left on the server are common disk consumers. Running mysqldump to debug an issue and forgetting to delete the output file can leave a multi-gigabyte file sitting in the home directory.

Deploynix's automated backups send dumps directly to offsite storage (S3, Wasabi, DigitalOcean Spaces), so they do not consume local disk. But if you run manual backups or if a backup job fails after creating the dump but before uploading it, the local file remains.

4. System Logs and Package Cache

Ubuntu's system logs (/var/log/syslog, /var/log/auth.log, journal logs), apt package cache (/var/cache/apt), and old kernel versions consume disk space that your application has no visibility into.

journalctl logs can grow particularly large if systemd journal size limits are not configured. On a server with multiple daemons (Nginx, PHP-FPM, MySQL, Supervisor), journal logs accumulate from all services.

5. MySQL Binary Logs and Temporary Files

MySQL generates binary logs for replication and point-in-time recovery. If binary logging is enabled (the default in MySQL 8.x) and the expire_logs_days setting is generous, binary logs can consume gigabytes.

MySQL also creates temporary files for complex queries (sorts, joins, temporary tables). These are usually cleaned up automatically, but if a query is killed mid-execution, temporary files may be orphaned.

Prevention: Stop the Disk From Filling Up

Configure Log Rotation for Laravel

Switch from the single log channel to daily with retention:

// config/logging.php
'channels' => [
    'daily' => [
        'driver' => 'daily',
        'path' => storage_path('logs/laravel.log'),
        'level' => 'info',
        'days' => 7,
    ],
],
Enter fullscreen mode Exit fullscreen mode

Setting days to 7 keeps one week of logs and automatically deletes older files. For high-traffic applications, consider setting the level to warning or error in production to reduce log volume.

Deploynix configures system-level log rotation via logrotate during server provisioning. This handles Nginx access logs, error logs, and MySQL logs. But your Laravel application logs are managed by Laravel itself, so configure the days parameter in your logging config.

Configure Deployment Release Retention

In Deploynix, the number of retained releases is configurable per site. Set this to 3-5 releases. This gives you enough rollback points without consuming excessive disk space.

Each release is a full copy of your application code. If your repository is 50 MB, five releases consume 250 MB. That is manageable on a 25 GB disk but adds up if you keep 20 releases.

Clean Up Package Cache Regularly

Add a cron job through Deploynix to periodically clean the apt cache:

apt-get clean
Enter fullscreen mode Exit fullscreen mode

This removes downloaded .deb packages that are no longer needed. On a server that has received several system updates, this can free gigabytes.

Configure MySQL Binary Log Expiration

In MySQL 8.x, set the binary log expiration:

SET GLOBAL binlog_expire_logs_seconds = 259200;
Enter fullscreen mode Exit fullscreen mode

This keeps three days of binary logs (259,200 seconds). If you do not use replication or point-in-time recovery, you can reduce this further or disable binary logging entirely:

SET GLOBAL log_bin = OFF;
Enter fullscreen mode Exit fullscreen mode

Note: disabling binary logging requires a MySQL restart and should be done during a maintenance window.

Set Journal Size Limits

Configure systemd journal to limit disk usage:

journalctl --vacuum-size=500M
Enter fullscreen mode Exit fullscreen mode

This immediately reduces journal logs to 500 MB. To make it permanent, edit /etc/systemd/journald.conf:

[Journal]
SystemMaxUse=500M
Enter fullscreen mode Exit fullscreen mode

Detection: Know Before It Is Full

Deploynix Health Alerts

Configure disk usage alerts in Deploynix's monitoring settings. Set the alert threshold to 80%. This gives you a buffer to investigate and clean up before reaching 100%.

Deploynix's real-time monitoring tracks disk usage alongside CPU and memory, so you can see trends over time. A disk that grows from 60% to 75% in a week tells you that you have roughly two weeks before it is full.

Application-Level Monitoring

Add a scheduled task that checks disk space and alerts your team:

// routes/console.php or a scheduled command
Schedule::call(function () {
    $freeSpaceBytes = disk_free_space('/');
    $totalSpaceBytes = disk_total_space('/');
    $usedPercent = 100 - ($freeSpaceBytes / $totalSpaceBytes * 100);

    if ($usedPercent > 85) {
        Log::warning('Disk usage is at ' . round($usedPercent) . '%');
        // Send notification to operations team
    }
})->hourly();
Enter fullscreen mode Exit fullscreen mode

External Monitoring

If you use external monitoring tools (UptimeRobot, BetterStack), some offer server monitoring agents that track disk space independently of your application. This provides a safety net — if your application crashes due to disk exhaustion, external monitoring still alerts you.

Recovery: Your Disk Is Already Full

When the disk is at 100%, your application is likely already experiencing failures. Here is the triage process, ordered by impact and speed.

Step 1: Identify the Largest Consumers

SSH into the server through Deploynix's web terminal and identify what is using space:

du -sh /var/log/*
du -sh /home/deploynix/*/releases/*
du -sh /home/deploynix/*/storage/logs/*
du -sh /var/lib/mysql/
du -sh /tmp/*
Enter fullscreen mode Exit fullscreen mode

This gives you an immediate picture of where the space went.

Step 2: Quick Wins — Delete Safely Removable Files

Old Laravel logs:

find /home/deploynix/*/storage/logs -name "*.log" -mtime +1 -delete
Enter fullscreen mode Exit fullscreen mode

This deletes log files older than one day. You can adjust the -mtime value based on how much space you need to recover.

Old deployment releases:

Identify releases that are not the current or previous release:

ls -lt /home/deploynix/your-site/releases/
Enter fullscreen mode Exit fullscreen mode

Delete the oldest releases, keeping the current one (symlinked from current) and one previous release for rollback:

rm -rf /home/deploynix/your-site/releases/20260115_120000
Enter fullscreen mode Exit fullscreen mode

Apt package cache:

apt-get clean
Enter fullscreen mode Exit fullscreen mode

Journal logs:

journalctl --vacuum-size=200M
Enter fullscreen mode Exit fullscreen mode

Temporary files:

rm -rf /tmp/laravel-*
Enter fullscreen mode Exit fullscreen mode

Step 3: Address the Root Cause

After freeing immediate space, identify why the disk filled up in the first place:

  • If logs were the culprit, configure rotation (see Prevention section)
  • If old releases, reduce the retention count in Deploynix
  • If database dumps, set up a cron job to clean them or move to offsite-only backups
  • If MySQL binary logs, configure expiration

Step 4: Restart Services

After freeing space, services that failed due to disk exhaustion may need a restart:

systemctl restart php8.4-fpm
systemctl restart nginx
systemctl restart mysql
Enter fullscreen mode Exit fullscreen mode

In Deploynix, use the server dashboard to restart these services. Check the application is functioning correctly by loading a few pages and verifying database connectivity.

Step 5: Verify Recovery 

df -h /
Enter fullscreen mode Exit fullscreen mode

You should see reasonable free space (at least 20%). If the disk is still nearly full after cleanup, you may need to upgrade to a larger server or add block storage through your cloud provider.

Long-Term Disk Management Strategy

The 80/20 Rule for Disk Space

Keep your disk usage below 80% at all times. This provides:

  • Room for traffic spikes that generate more logs
  • Room for deployments that create new release directories
  • Room for MySQL temporary tables during complex queries
  • Room for system updates that download packages before installing

Offsite Everything

Any data that can live offsite should live offsite:

  • Database backups: Deploynix sends these to S3/Wasabi/Spaces
  • Application logs: Consider shipping to an external logging service (Papertrail, Logtail)
  • User uploads: Store in S3 or similar object storage, not on the server's local disk

Automated Cleanup Cron Jobs

Set up a weekly cleanup cron job through Deploynix:

# Clean old Laravel logs
find /home/deploynix/*/storage/logs -name "*.log" -mtime +7 -delete

# Clean apt cache
apt-get clean -y

# Clean old temporary files
find /tmp -type f -mtime +7 -delete
Enter fullscreen mode Exit fullscreen mode

This runs automatically and prevents gradual accumulation.

Disk Usage Tracking

Monitor your disk usage trend, not just the current value. A disk at 50% that was at 40% last week tells a different story than a disk at 50% that has been steady for months.

Deploynix's monitoring dashboard shows this trend. If you see consistent growth, calculate when the disk will be full and plan your response — either cleanup, configuration change, or server upgrade — before it becomes an emergency.

When to Upgrade Instead of Clean

Sometimes the answer is not better cleanup but more disk. Signs you need a larger disk:

  • Your database is legitimately large (multiple gigabytes) and growing
  • Your application stores files locally that cannot move to object storage
  • You need to keep more than 7 days of logs for compliance reasons
  • Deployment artifacts (node_modules, vendor) are large and you deploy frequently

Cloud providers make disk upgrades straightforward. Hetzner, DigitalOcean, Vultr, and Linode all support resizing volumes or upgrading to plans with more storage. Deploynix manages the application layer, so a disk resize on the infrastructure side does not require any changes in your deployment configuration.

The Nuclear Option: Server Rebuild

In rare cases, a server's disk is so cluttered with accumulated debris that cleanup is impractical. Deploynix makes rebuilding straightforward:

  1. Provision a new server with a larger plan
  2. Connect your Git repository
  3. Restore your database from the most recent Deploynix backup
  4. Configure your environment variables
  5. Deploy
  6. Update DNS to point to the new server
  7. Decommission the old server

This process takes 15-30 minutes and gives you a clean server with a known-good state. It is faster than forensically cleaning a server with years of accumulated files.

Conclusion

Disk space exhaustion is a predictable, preventable failure. The combination of log rotation, release retention limits, offsite backups, and monitoring alerts eliminates the surprise element entirely.

Configure Deploynix's health alerts to warn you at 80% disk usage. Set up log rotation for your Laravel application. Keep release retention at 3-5 deployments. Ship backups and logs offsite. These four actions prevent the vast majority of disk-related outages.

When it does happen — and eventually, it happens to everyone — the recovery process is systematic: identify the largest consumers, delete safely removable files, address the root cause, and restart affected services. With Deploynix's web terminal and monitoring dashboard, you can execute this entire process from your browser in minutes.

Top comments (0)