Automating Storage Utilization Monitoring on a Private Cloud - Part 1

Automating Storage Utilization Monitoring on a Private Cloud - Part 1
Monitoring is one of those things that’s easy to take for granted until it fails, and in this case, it was the missing piece in our private cloud setup.
A few years ago, I worked with a team that managed a private cloud infrastructure. One of our goals was to monitor the core performance metrics of each server: CPU, RAM, and storage utilization.
We already had reliable API endpoints that returned CPU and memory usage, but there was a gap: no API existed to track storage usage per instance. And that was a serious issue, because storage metrics were crucial for capacity planning and incident response.
So, I had to find a way to automatically collect and report disk usage for every server that was provisioned without relying on manual checks or post-deployment scripts.

Identifying the Challenge
Without an API for disk metrics, we had partial visibility at best. The team could see how much compute power was being used, but not how much storage was left on each server.
We needed a lightweight, secure, and automated way to collect disk usage data and feed it back into our monitoring dashboard.
After considering a few options, I decided to design a solution that would hook directly into the instance provisioning process itself, using Cloud-Init.

Solution Design and Implementation
The final solution was implemented through a Cloud-Init script and structured into three main components:

1. Disk_Usage User
   The Cloud-Init script began by creating a dedicated user account named disk_usage. This user handled the background process responsible for collecting disk utilization data.
   To maintain system security and comply with the principle of least privilege, I ensured this user had only the minimal permissions required to execute the monitoring task. This isolated the disk usage process from other system operations, maintaining both security and stability.

2. Disk Usage Collection Script
   Next, I wrote a Bash script that gathered real-time disk utilization metrics from each server.
   Initially, this script worked, but there was a challenge: it collected data without identifying which server the metrics came from. We couldn’t map the usage data back to the specific instance that generated it, which made it useless for visualization or analysis.
   To solve this, I collaborated with our backend team to expose a custom API endpoint that returned the unique server identifier (Server ID).
   I then modified the script to include this Server ID in its output, ensuring every metric could be accurately traced back to its source instance.

3. Data Ingestion and Automation
   With data now correctly attributed, we needed a reliable way to push it to our monitoring dashboard.
   We created an API endpoint that accepted POST requests from the Bash script. Each server would send its latest disk usage metrics, along with its Server ID, at regular intervals. The backend then processed this data and made it available for visualization on the dashboard.
   To ensure the process ran automatically, I configured a cron job on each instance to execute the Bash script periodically. This eliminated any manual intervention, keeping the data flow consistent and real-time.
   Results and Impact
   By the end of the project, we had built a robust and automated monitoring solution that ran seamlessly in the background.
   The system continuously collected disk usage data from every instance and displayed it on the monitoring dashboard, giving the team full visibility into storage health across the private cloud.
   This implementation led to measurable improvements:
   Reduced Mean Time to Detect (MTTD) issues by 25%, thanks to early detection of disk saturation.

Eliminated manual storage checks, freeing up the team’s time for higher-priority work.

Improved infrastructure observability, allowing proactive scaling and faster incident resolution.

What Came Next
Following the success of this implementation, I adapted the same solution for Windows servers using PowerShell.
That version was more complex due to Windows’ different automation and permission models, but the same core principle applied: collect and report system metrics automatically, securely, and in real-time.
I’ll share that implementation in my next post.
Key Takeaway
Monitoring shouldn’t be reactive. By embedding telemetry collection directly into the provisioning workflow, you can design observability into your infrastructure from day one, and save your team hours of troubleshooting later.

Photo credit: www.e-spincorp.com