TL;DR: My production service survived four separate waves of losing SSH access without going down once. What failed, four times, was the only door I had into the machine. Uptime protects the service. A second door protects your ability to run it — and for a one-person operation, the second is the one that matters.
Four times, my management access to my own production machine died. Four times, the service itself was fine. The cause was embarrassingly mundane: the outbound proxy my SSH traffic rides through kept cutting port 22. The machine was healthy, the containers were healthy, users would have noticed nothing. I noticed everything, because SSH was the only way in — and when your only door jams, a healthy house and a burning one look exactly the same from the sidewalk.
That is the failure mode nobody's uptime dashboard measures. The industry sells availability for the service: more regions, more replicas, more nines. Almost nobody sells availability for the operator. And for an operation like mine — a personal AI system run for exactly one principal, me, with no SRE team on call — the operator is the scarcest resource in the stack. An agent that cannot be reached, inspected, or restarted by its owner is not autonomous. It is just unsupervised.
First, learn to tell a dead service from a dead path
The most expensive mistake in those four waves was not technical. It was diagnostic. Early on, a failed SSH connection read as "the service is down," and that misreading invites the worst responses: frantic restarts, rolled-back deploys, incident language for an incident that does not exist.
The fix was a rule, and it is now load-bearing in my operation: never diagnose from the door that just failed. My service exposes a public health endpoint, and during every SSH outage it kept answering 200 — status ok, database up. That independent signal, on a completely different path from SSH, is what let us say with evidence, not hope: the path is broken, the service is alive. Log it, do not escalate it, and do not let a proxy's mood masquerade as your product's health.
Any system an agent operates needs that split: one signal for "is it alive," carried on infrastructure that shares no fate with the channel you manage it through. If your health check and your management access die together, you do not have monitoring. You have a second opinion from the same witness.
A second door is three unglamorous pieces, tested end to end
So we built a second door: a management channel that does not use SSH at all. In my cloud, that is the provider's run-command service — an API that dispatches a command to an agent running on the instance and returns the result. No port 22, no proxy, no shared fate with the thing it replaces.
Building it taught me why most "backup access plans" are fiction. The channel only worked when three separate pieces lined up: the instance agent's run-command plugin actually enabled, an identity policy scoped so the API is authorized to dispatch to that instance, and the execution user on the machine holding the group permissions to inspect the containers — which only took effect after the agent was restarted. Miss any one and the channel exists on paper and fails in practice. My favorite scar: for a while the plugin's polling returned 404 authorization errors, which looks exactly like a dead daemon and is actually a missing IAM grant. We nearly debugged the wrong layer for hours.
And the door only counted as built when it was walked through: a command dispatched through the API, executed on the machine, returning a real container health result, execution status successful. End to end, on a quiet day — not assembled for the first time during the next outage. An untested fallback is not a fallback. It is a hypothesis you will be debugging at the worst possible moment, through the door that is already jammed.
Uptime is a property of services. Access is a property of operators.
Here is the claim I will defend: for agent-run systems, the binding constraint is not whether the service stays up. It is whether the operator — human or AI — retains an independent way to observe it, diagnose it, and act on it when the primary path fails. Replicas do not help you reach a machine you cannot reach. A status page does not restart a container. Every layer of redundancy teams stack on the user-facing side is worthless in the exact incidents where the management plane is the thing that broke.
My SSH still flakes. The proxy still has opinions about port 22. None of that pages me anymore, because it no longer decides whether I can run my own system. Build the health signal that shares no fate with your access. Build the second door, all three pieces of it, and walk through it before you need it. Your AI doesn't need more uptime. It needs a second door.
— Jiahui Miao, founder of vertciti, building a personal AI operating system for one.
Top comments (0)