Our staff accounts still begin life in the Active Directory we have run since 2009. A small server at head office runs Microsoft Entra Connect, which copies every change from there to our Microsoft 365 tenant about every thirty minutes: new starters, changed names, new passwords and accounts that have been disabled. It is one virtual machine with two processors, and in seven years nobody had needed to think about it.
In March we moved our virtual machines onto new hosts. That server came back up attached to a network that did not reach anything. Nothing visible broke. Email, Teams and every cloud application carried on, because they live in Microsoft's data centres, and the accounts in them simply stopped changing. Our monitoring checked that the server was switched on, and it was.
It took us three weeks to notice, and the evidence had been reaching the service desk the whole time in pieces. New starters who could log on to their PC but not to Teams. Depot staff whose password resets worked on the counter PC and not on their phone. A manager whose new surname had not reached her email. Each ticket was handled on its own, and several were fixed by hand in the cloud, which made the gap harder to see.
What finally joined them up was a depot manager who rang to say that a forklift driver dismissed the week before was still posting in the depot's Teams channel from his own phone. His account had been disabled on the day he left, in Active Directory, and that change had never left the building. When we counted, nineteen people who had left in those three weeks still had working cloud accounts, two of them with access to our finance system through single sign on. We found no sign that anyone but the driver had used one.
The server is now treated as one of the most important machines we own, alongside the domain controllers. A second server at our other site runs in staging mode, holding the same configuration and ready to take over within an hour. The health of synchronisation feeds our main monitoring, and the on call engineer is paged if no change has reached the cloud for two hours. The leaver process also blocks sign in and revokes sessions directly in the cloud, so it no longer depends on a copy arriving later.
Moving to the cloud did not take that server away. It made it invisible, because everything it feeds keeps working when it stops. Find the machines on your own premises that your cloud still depends on, and look after them as if everything ran on them, because it does.
– Serguey Shinder
Top comments (0)