After the ex-employee key incident, I stopped trusting my own assumptions about who was actually connecting to what. So for one month, I logged every SSH session across three environments: a 6-person startup, a 40-person mid-size company, and my own infrastructure.
The numbers were worse than I expected.
23% of connections came from credentials nobody could immediately identify.
Not malicious, mostly. Old CI/CD service keys still hanging around. A contractor's access that IT thought was revoked. One senior engineer connecting through a personal laptop key he'd generated two jobs ago and never removed from the authorized_keys file.
The average time between "this person left the team" and "their SSH access was actually cut" was 11 days.
At the 40-person company, offboarding meant filing a ticket, waiting for infra to manually SSH into each box, and removing the key by hand. Eleven days is eleven days of a former employee, or worse, whoever now controls their laptop or email, having standing access to production.
Nobody could answer "who connected to the payments database last week" without grepping raw auth logs across a dozen hosts.
Not because they didn't care. Because the tooling made it genuinely hard. Bash history is local and unreliable. Centralized logging existed but nobody had built the query for "SSH sessions grouped by host and user" because nobody had needed it until they suddenly did, mid-incident.
Here's what actually fixed it, in order of impact:
1. One source of truth for who has access to what. Not a spreadsheet. Not "ask in the #infra channel." A system where access is a first-class object you can list, not a side effect of someone having a key file somewhere.
2. Revocation as a single action, not a per-host chore. If cutting access takes more than one click, it will eventually take more than one day. Every day past zero is risk you're carrying for no reason.
3. An audit log that answers the question before someone asks it. Not "technically the data exists in syslog somewhere." A log you'd actually query during an incident, at 2am, without needing to remember which host has which retention policy.
None of this is exotic. It's the same access-control discipline every team already applies to production databases and cloud consoles. SSH just tends to be the one thing that's still running on trust and tribal memory, because it's the layer underneath everything else and nobody wants to touch it.
I ended up building this into SecurSSH after hitting the same wall repeatedly, team vaults instead of shared keys, role-based access that revokes instantly, and an audit log that's actually queryable. Not the point of this post though, the audit itself is worth doing regardless of what tool you use afterward.
If you run the same kind of audit on your own infra, even informally for a week, I'd genuinely like to know what you find. My guess is the offboarding gap is universal.
Top comments (0)