We passed the tabletop exercise on backups for years. Offsite copies, tested restores, documented retention. The thing we never examined was how the backup system authenticated, and the answer was that its service account was a member of a group with administrative rights on effectively every server it touched, because that was the simplest way to make agentless backups work when it was installed in 2016.
I only looked properly after reading an incident report from another organisation in our sector. Their attacker did not encrypt anything for days. They got ordinary user access, moved laterally, found the backup server, used its credentials to reach everything else, deleted the backup catalogue and the offsite copies, and only then started encrypting. The restore plan was excellent and completely irrelevant, because the restore plan lived inside the blast radius.
Our setup was the same shape. A single highly privileged account, stored on a Windows box that was domain-joined like any other, with network reach to every system in the estate by design. We had built a perfect pivot point and then written "backups: tested" on the risk register.
Fixing it took most of a quarter and none of it was clever. Per-tier service accounts with only the rights they actually need instead of one account with all of them. Backup infrastructure outside the production identity domain, so compromising production does not hand you the backups. Immutable retention on the copies that matter, so a valid administrative credential cannot shorten a retention period or delete a snapshot. Alerting on retention changes and catalogue deletions, which turns out to be one of the highest-signal alerts we have, because legitimate reasons to delete backups are extremely rare.
And one restore test per quarter performed under the assumption that the production directory service is untrustworthy, because if you cannot authenticate to your restore tooling during a domain compromise, you cannot restore during a domain compromise.
The general lesson I took is that we test our controls against failure and almost never against malice. A disk dies and the backup works. That tells you nothing about what happens when the thing attacking you is inside the account your backup runs as.
– Serguey Shinder
Top comments (0)