DEV Community

Boris Kl
Boris Kl

Posted on

Nobody checks the VPS between deploys — until it's mining crypto for someone else.

A friend runs an agency. Six people, forty-something client sites, each on its own small VPS. One Tuesday a client emails asking why their server is "running hot." Load average through the roof. Fan noise from a box that's supposed to just serve a WordPress site. My friend logs in, runs top, and there it is: a process with a random name, eating every core, phoning home to an IP nobody knows. A miner. It had been there, quietly, for three weeks.

Nobody had touched that server in three weeks. That's exactly the problem.

The gap between "deployed" and "watched"

Most small ops setups have decent coverage for one moment: the deploy. CI runs, the app boots, maybe a health check pings once. Then everyone moves on to the next client, the next ticket. What happens on day 12, or day 40, when nobody's looking? Nothing. Nothing gets checked, because nothing triggered a check. The server just sits there, open to whatever the internet throws at it. Until a client notices the fan. Or the CPU bill. Or worse.

Scale that across an agency running dozens of servers and the truth gets ugly fast: most of that fleet is watched by nobody, most of the time. Not because anyone's careless. Because there was never a cheap way to watch it all the time.

Full security monitoring is the wrong first step

The instinct is to reach for a proper monitoring agent. Install a daemon, ship logs somewhere, wire up a SIEM. That's real work, real cost, real upkeep. For a five-person agency running forty small VPS, it's way too much for the problem at hand. Nobody's rolling out a full security stack just to catch "someone added a cron job."

But you don't need the whole stack to catch the miner. You need one small thing: notice when something changes that shouldn't have.

Drift, not intrusion detection

Think about what actually shifts on a server between deploys, when nothing legitimate is going on. New listening ports. New processes that weren't there last time. New or edited cron entries. New user accounts. A real deploy changes some of this too — but on purpose, in a known, expected way. An intrusion changes it in a way nobody asked for.

So the check isn't "is this server secure." That's a much bigger question. The check is smaller: is this server the same shape it was yesterday? If not, was that change supposed to happen? That's drift detection, and it's cheaper than it sounds. No agent daemon required.

A basic version is just three snapshots and a diff:

# collect_state.sh — run via cron, once per hour
{
  echo "### ports"
  ss -tulpn
  echo "### processes"
  ps -eo pid,user,cmd --sort=pid
  echo "### cron"
  for u in $(cut -f1 -d: /etc/passwd); do crontab -u "$u" -l 2>/dev/null; done
} > /var/log/state/snapshot-$(date +%s).txt
Enter fullscreen mode Exit fullscreen mode
# compare against the last snapshot, alert on anything new
diff "$PREV_SNAPSHOT" "$LATEST_SNAPSHOT" | grep '^>' > /tmp/new_lines.txt
if [ -s /tmp/new_lines.txt ]; then
  curl -s -X POST "$WEBHOOK_URL" -d "New on $(hostname): $(cat /tmp/new_lines.txt)"
fi
Enter fullscreen mode Exit fullscreen mode

That's it. No agent on the box, nothing running in the background, just a cron job and a diff. It won't catch a careful attacker who cleans up after themselves, and it's not a substitute for patching or a firewall. But it catches exactly the case above: something new showed up and stayed, because nobody was ever going to notice a strange process name on a server they hadn't opened in three weeks.

Where this earns its keep

The value isn't in any single alert. It's the fact that a server nobody has logged into for a month still gets checked every hour, automatically, for free. For an agency carrying dozens of client boxes, that's the gap between "we'll find out when the client complains" and "we already know." The miner in my friend's story would have shown up in the first diff, the same hour it started. Not three weeks later, in a support ticket about fan noise.

I build this kind of lightweight, agentless drift-checking for people who manage a stack of client VPS and don't have a security team. The whole point is that it costs almost nothing to run and doesn't need anyone babysitting it.

So: if someone SSH'd into one of your servers right now and left a new process running quietly in the background, would anything tell you? Or would you find out the way my friend did?

Top comments (0)