Local models are having a moment. Ollama on a workstation, an open-weight model behind your own endpoint, a self-hosted coding agent — you run them for good reasons: privacy, cost, no rate limits, works on a plane. And once a model is already handling your prompts, the obvious next step is to point it at your infrastructure: let it tail a log, restart a service, run the migration.
Here's the trap that rides along with it: "it's local, so it's safe" is not true. Running the model on your own hardware keeps your prompts off someone else's servers. It does nothing about what happens when that model gets a shell.
Why "local" doesn't make server access safe
Two things stay exactly as dangerous whether the model runs in a datacenter or on the box under your desk.
The key is still a bearer credential. Drop id_ed25519 into the agent's environment so it can run ssh itself, and whoever — or whatever — holds those bytes is you, on every host that trusts the key. Running the model locally doesn't change that; it just moves the copy to a machine you happen to own. And you still can't take it back: once the key has passed through the model's context — an env var, a mounted file, a tool call, a transcript — you can no longer prove it didn't end up somewhere it shouldn't. The only honest answer to "did the model see my key?" is to rotate it on every host.
A local model still gets confused. Prompt injection doesn't need a cloud model. A poisoned log line, a malicious filename, a booby-trapped tool result, an over-eager plan — a local model acts on all of them just as readily as a hosted one. If anything the risk is worse, because people trust local setups more, wire them up with fewer guardrails, and hand them broader access, precisely because "it's on my machine." That's the wrong instinct: the model's judgement isn't better because the weights are local.
So the goal is the same as for any agent — let the model do the work without ever holding the credential, and make every action scoped, visible and reversible.
Three ways to do it badly
Paste the raw key. The direct route, and the worst: no scope, no expiry, and the irreversibility above. The key that deploys to staging can usually also reach the production database two hops over.
Mint a long-lived token. A deploy token or "automation" credential that expires never. It feels tidier than a key, but it's the same bearer problem with extra steps. If revoking something means remembering it exists, it'll outlive the experiment that created it.
Give it its own root login and walk away. The right instinct — a separate identity — done the wrong way. Drop a public key into authorized_keys for a bot user and you've built access with no audit trail: nothing distinguishes the model's commands from anyone else's, and when something breaks at 2 a.m. you're reconstructing its session from bash history and vibes.
The common thread: the model holds a standing credential, and observability is an afterthought.
The pattern that works: a custodian, not a copy
Flip it. The model should never hold the credential at all.
- The model asks; a custodian signs. A broker holds the keys (or mints short-lived certificates) and authenticates on the model's behalf. Compromise the model's context and you get the ability to request actions through the broker — not the ability to impersonate you from anywhere.
- Scope is explicit. The model reaches only the hosts you've listed. A new host is a new decision, not a default.
- Commands are policed. Per host, decide whether the model gets a full shell, an allowlist of commands, or nothing — so "restart nginx" is allowed on the web tier and "anything at all" never is.
- A human can watch — live. Not just in the post-mortem, and with the record distinguishing "the model did this" from "I did this."
- Revocation is a toggle, not a rotation. Because nothing was ever shared, turning access off costs nothing.
None of this is exotic — it's roughly how certificate-based SSH already works on serious infra teams. The catch has always been that wiring it up yourself is a project, so people skip to one of the bad options above.
Wiring it to a local model
MCP — the Model Context Protocol — is the clean way to do this, and it's client-agnostic: any MCP-capable client can drive it, so it doesn't matter whether your model is Claude, an open-weight model behind Ollama, or something you host yourself.
- Run your local model through an MCP-capable client (a coding agent or chat client that speaks MCP).
- Point it at an MCP server that fronts SSH — one that authenticates for the model rather than handing it a key.
- Turn on only the hosts you want, and set each host's command policy (full / allowlist / blocked).
- Keep the session in view and let it write to an audit log, so every command the model runs is attributable and replayable.
The model gets tools — open a session, run a command, read or write a file — and the custodian does the authenticating. Your key never enters the model's environment, on your machine or anywhere else.
How Termalin does it
Termalin is an SSH client with a built-in MCP server, so this pattern is the default rather than a project.
-
On your machine, register the bundled local server with your agent (
claude mcp add termalin -- <path>/termalin-mcp, or the equivalent for any MCP client). The model reaches only the hosts you enable — agent access is off by default — and authentication goes through Termalin's key custodian: you unlock your keys once, Termalin signs on the model's behalf, and no key file is ever there for the model to read. - Per host, you set the policy — full access, an allowlist of commands, or blocked — so a local model that gets confused can't run something you never authorized on that box.
- You watch it happen. Agent sessions run as live terminal tabs; the watch grid mirrors them, and the tiles a model is driving glow. Every command is marked in the session recording — output only, never your keystrokes — and written to an audit log with the device and IP it came from.
- No app running? Point the model at Termalin's hosted MCP endpoint with an API key that's scoped to specific servers, carries an expiry, is command-policed, and authenticates each run with a short-lived certificate — so even a leaked key can't reach an unlisted host or run outside its allowed commands.
Either way, the property you wanted holds: your local model can operate your servers, and it has never seen a key.
Start with one boring host
Don't begin with production. Enroll a low-stakes box — a staging server, a toy VPS — set it to an allowlist, and give the model a real chore: tail the log until the error shows, fix the config, restart the service. Keep the grid open while it works. What you learn in the first hour — how it behaves, where it hesitates, what it does with ambiguity — tells you whether the second host gets enrolled, and with how much rope.
That's the quiet payoff of the custodian model: you expand one host at a time, because no step you take is one you can't take back — no matter where the model is running.
Termalin is a free, cross-platform SSH client with a built-in MCP server, a key custodian and per-host agent policy — download it, or read how it handles keys safely.
Top comments (10)
Good framing. I’m building Nexus Shell’s local Agent Bridge and arrived at a similar “custodian, not credential copy” boundary. One caution: an allowlist around shell strings is weaker than it looks. If an allowed command can invoke a shell or interpreter, accept metacharacters, or follow attacker-controlled paths, it can become full shell access.
I’d treat full-shell as full access; use structured argv or typed tools for narrow actions; bind approval to the agent identity, session, host, and exact operation; re-check authorization immediately before execution; and terminate existing sessions on revoke. Also scrub tool output before returning it to the model, because secrets can leak through command results even when the SSH key never does.
A visible terminal and audit trail are excellent for observability, but they should not be the policy boundary themselves.
This is the sharpest critique of the allowlist model, and it's fair - a string allowlist is a speed bump, not a wall. The moment an "allowed" command can reach a shell, take metacharacters, or follow a path you don't control, the allowlist is decorative. The real boundary is typed operations / structured argv, where "restart nginx" is a tool, not a string you pattern-match - execve, never system(), and reject anything that would re-enter a shell.
The output-scrubbing point is the one people skip. Custody keeps the key out of the model's context, but a cat of an env file or a stray token in a log line leaks through the result - a separate control from the credential path, and one that has to live at the boundary too.
And "audit trail shouldn't be the policy boundary" - completely. The watch grid and log exist so a human can catch and revoke; they don't prevent anything. Observability and enforcement are different jobs; conflating them is how you feel safe while the policy is a string match.
Honest read on where we are: per-command allowlist is the coarse first line, and everything you listed - structured argv, approval bound to identity+session+host+operation, re-check at exec, kill-on-revoke, output redaction - is the direction that actually holds. What's your Agent Bridge doing at the typed-tool vs raw-command boundary? Curious how you draw the line between a narrow typed action and "the user genuinely needs a shell."
No-new-work plus honest-about-the-rest is the right place to land - I'd rather a tool tell me "the remote PID may still be running" than pretend a client-side cancel reached across the wire.
That residual is the genuinely hard part, and I don't think it's solvable purely client-side. Closing the channel drops your stdin/stdout to the process, but the process was reparented to init (or its own session leader) the moment it was backgrounded or the shell got a controlling-tty; the kernel on the far end doesn't care that your TCP session went away. A real kill needs server-side cooperation - a control master you can signal, a session-scoped cgroup you can freeze/kill, or at minimum a PID you recorded at launch so someone can go SIGTERM it. Most setups have none of that, which is why the human watching the live session ends up being the actual backstop rather than a nice-to-have.
So the honest capability ladder probably ends with a rung you can't reach alone: "revoke that also kills in-flight remote work" is a server-side feature the client can request but not guarantee. Worth labeling it that way so nobody trusts revoke to do more than it can.
Today it is deliberately a hybrid, and the raw side is still a real escape hatch.
The typed surface is MCP tools for connections, terminals, SFTP, keys, monitors, and session-log metadata. Those give me structured arguments, size caps, destructive/read-only hints, atomic file writes, and a place to add operation-specific policy later. But the bridge also exposes
run_command,send_text, and headlessexec_command.run_commandrejects embedded newlines, but it still hands a shell a command string;send_textcan drive an interactive shell. So I do not describe the current bridge as typed-only or shell-safe.The present trust boundary is: local Unix socket, same-UID peer check, server-derived/signature-aware client identity, bridge off by default, first-tool-call consent for that Agent, visible tabs for interactive terminal work, and an audit log. Credentials are resolved inside the app and never returned. That is useful custody and observability, but the consent is currently Agent-level, not host+operation-level. Revoking a remembered Agent makes future calls ask again; it does not yet kill a command already running. Audit arguments redact password/passphrase/file-content fields, but arbitrary stdout/stderr and terminal output are not universally secret-scrubbed.
The line I am leaning toward is capability escalation rather than pretending raw shell can be made narrow:
cat .env.I would keep the shell because real incident work eventually needs it, but make crossing into it explicit and temporary. The mistake would be exposing a raw string and then calling it a typed operation because it arrived inside JSON.
The hybrid honesty is the right move - the failure mode across this whole space is the one you name at the end: laundering a raw string through JSON and calling it a typed operation.
Your escalation ladder matches where we've landed from the client side, two notes:
On the allowlist rung: after this thread I'd downgrade it from boundary to brake. String-matching a command is not an operation grant - typed tools with host+operation scope are the honest version; the allowlist survives as friction, not as the security claim.
On revoke (4): semantics for long-running mutating commands are nastier than they look. Killing a terraform apply or apt upgrade mid-flight because a grant expired can do more damage than letting it finish. Our current answer is that the human watching the live session makes that call. Are you planning revoke = kill active calls immediately, or revoke = no-new-work + a bounded drain window? The second feels right for infra work, but it means a revoked agent still holds a live shell for the drain period - which is its own hazard.
And (5) is the unsolved one for everyone: custody stops key exfiltration, not cat .env. Redaction that actually works probably has to be entropy/pattern-based on the output stream, and that's full of false positives in exactly the places ops output lives.
That is exactly the trap. I checked the current implementation before replying: revoke is no-new-work, not kill. It removes the durable approval and connection-level decisions, so the next
tools/callmust be authorized again, but a call that has already passed the consent gate is allowed to finish. Disabling the bridge closes the client connections and cancels request wrappers, but I deliberately do not claim that this safely rolls back or even reliably terminates the remote process; a command started in a visible terminal may still be running there.I do not have a bounded drain model yet. The direction I prefer is to separate control-plane revoke from operator abort: block new calls immediately, show any in-flight work in the UI/audit log, and let the human choose finish, interrupt, or close the terminal. For non-terminal exec, a time budget plus explicit soft/hard termination can be useful, but automatic kill-on-revoke still feels wrong because cancellation is not rollback.
The unresolved part is exactly the authority leak you called out. One promising rule is: after revocation, the client gets no further output or follow-up capability, while the remote work remains visible and controllable by the human. That is closer to fail-closed without pretending that killing
terraform applyis a safe undo.That matches where we landed - revoke as no-new-work, and honest that it doesn't reap the remote process.
The residual is the genuinely hard part, and I don't think it's solvable purely client-side. Closing the SSH channel tears down the client end, but the remote PID only dies if it's still in the session's process group and didn't detach - nohup / setsid / systemd-run --scope walks right out of that. A real kill needs server-side cooperation: a control-master channel you can send a signal through, or a session-scoped cgroup/systemd scope you can stop by name. Most setups have neither.
Which is why I treat the human watching the live session as the actual backstop, not the revoke button. Revoke stops the next thing; a person stops the current thing. Are you planning any server-side hook, or keeping the boundary client-only and leaning on visibility?
For now I’m keeping the boundary client-only and being explicit about the limitation. Nexus Shell can revoke durable approval or stop the local bridge, which blocks new calls and disconnects the MCP client, but neither action claims to reap a process that has already started on the remote host.
The constraint is exactly the one you described: a general SSH client cannot assume systemd, root, or a helper daemon on every server. So today the backstop is the visible terminal, manual takeover, and an explicit session close when the operator decides interruption is safer — with the caveat that closing the session still is not a guaranteed kill for a detached process.
I can see value in an opt-in server-side hook later, perhaps a user-configured wrapper or named scope on hosts they control. I would keep that separate from authorization revocation, though: revoke should continue to mean “no new work,” while stopping current work should be an explicit, host-specific operation with its own audit trail. Otherwise the UI would imply a cancellation guarantee the client cannot actually provide.
No-new-work + let approved calls finish is the honest model, and I think refusing to claim reliable remote termination is the right call rather than a gap. The residual is the genuinely hard part: closing the client channel doesn't reap the PID. Once a command is executing on the box, killing it needs server-side cooperation you usually don't have - a control channel, a session-scoped cgroup, or something that can signal the process group. SSH gives you none of that after the fact; the exec already forked.
Which is why we treat the human on the live session as the actual backstop for the drain window, not revoke itself. Revoke bounds future authority cleanly; the in-flight command is a physics problem, not an authz one. The only real kill I trust is server-side (systemd scope you can stop, a wrapper that runs under a reapable cgroup) - and that's a deploy-time decision, not something the client can retrofit.
Do you expose the still-running session anywhere after revoke, so an operator can go find and stop it manually? That handoff - 'authority is gone but this is still alive over here' - seems like the piece that's easy to drop.
That separation is the clean version - revoke = no new work, stop-current-work = an explicit, host-specific, audited operation. Collapsing them is where most designs go wrong: killing a running process becomes a side effect of a policy change instead of a decision someone made and signed for.
Framing "stop" as its own audited op is right beyond the kill mechanics too - "I'm ending this deploy now" is a deliberate act that deserves its own log line, separate from "this agent's grant expired." The opt-in wrapper / named scope is the pragmatic handle: on hosts you control, launching agent work under a systemd-run --scope (or a cgroup you can name) gives you a real stop button without pretending every server has one. Everywhere else, honest is honest - the visible terminal and manual takeover, nothing more claimed.
Good thread. Clearest I've seen the boundary drawn.