There is a stage every automation enthusiast passes through where the goal becomes automating everything, end to end, no human in the loop. I was there once. Then I watched a fully automated remediation script confidently make a minor incident into a major one, because the situation was slightly outside what the author imagined, and the script had no idea it was wrong. It just kept executing with perfect discipline in the wrong direction.
The lesson I took was not to automate less. It was to be deliberate about where automation ends and judgment begins. Automate the boring, repetitive, well-understood path, the ninety percent of cases that look the same every time. That is where machines beat humans easily, because they do not get bored, tired, or sloppy on the fiftieth deployment of the day.
But keep a manual escape hatch, and keep it tested. Every automated system should have a clear, documented way for a human to take over when reality does not match the script's assumptions. The dangerous automation is not the one that fails loudly. It is the one that succeeds at doing the wrong thing, quietly, faster than any human could have. If your remediation can take drastic action, it needs a circuit breaker and a human confirmation for the irreversible steps.
I now design automation with two questions in mind. What does this do when its assumptions are true, which is the easy part everyone thinks about. And what does it do when its assumptions are false, which is the part that separates automation you can sleep next to from automation that will page you with a disaster it caused.
The other trap is automation that becomes a black box. If nobody on the team understands what the pipeline actually does anymore, you have not saved labor, you have deferred it to the worst possible moment. Good automation is legible. A new team member should be able to read it and predict its behavior, including its failure behavior.
The mature position is not "automate everything" and it is not "trust nothing." It is knowing which decisions are safe to hand to a machine and which ones still deserve a human who can be held accountable for the outcome.
– Serguey Shinder
Top comments (0)