The certificate on our public website came within two days of expiring, which is how we found out that the registrar account holding our main domain was in the personal name of a contractor who left in 2019, billed to a card that stopped working the year after, with renewal notices going to an address that forwarded to a mailbox nobody had read since.
It was recoverable, with a week of identity paperwork and a supportive registrar. The interesting part was what we found when we went looking for the same pattern elsewhere.
The root account of one cloud subscription was registered to a former employee, with the second factor on a phone number that had been reassigned. Two software tenants had a single global administrator, and in both cases that person had moved to another company. The account that publishes our mobile application belonged to a developer rather than to the organisation. A monitoring service the platform team depended on was being expensed monthly, at forty pounds, by an engineer who had set it up during an incident three years earlier. And our secondary domain, the one used for staff email during a migration, was hosted at a provider nobody could name.
None of this was misconduct. Every one of them started with somebody needing something on a Friday, and with the plain fact that paying for it personally takes five minutes while raising a new supplier takes six weeks. The process punished the correct behaviour, so people routed around it.
What makes these accounts different from ordinary shadow spending is that they are not systems, so nobody thinks of them as infrastructure. They sit underneath everything, they are invisible on architecture diagrams, and they fail in a way that cannot be fixed by engineering. When the domain lapses, no amount of resilience in the estate helps.
We keep a register now of the foundational registrations: domains, DNS, certificates, cloud root accounts, code signing, app stores, and the handful of services that would stop the company. Each is held in an organisational identity, notifications go to a monitored shared mailbox, billing goes through accounts payable, and expiry dates are watched centrally. Registrar locks are on and auto-renewal is confirmed annually.
The most fragile things we owned were never servers. They were accounts.
– Serguey Shinder
Top comments (0)