SSH quoting ate a byte of my API key: 'Invalid API key' wasn't the tool

On 2026-09-24 I hit the same kind of bug twice in one day. Both times a secret was moved from one place to another by hand, and both times the thing that broke it was the move, not the destination. The first one cost me the most, because it looked like somebody else's fault.

The key that was one character short

I was setting up the OpenCode CLI as a scripting and coding agent on my Proxmox host. It uses the same subscription API key my AI agent container already has, so the plan was simple: read the key out of the container's environment file and write it into OpenCode's auth file on the host.

The container is reached with pct exec on the Proxmox host, over SSH, and I wanted the bare value without the quotes it might carry. So the copy went through nested SSH, pct exec and a tr -d to strip quote characters — several layers of shell, each with its own idea of what a quote is.

OpenCode started, then answered with "Invalid API key" and hung.

That looked like OpenCode's problem. A new tool, a provider I'd only just configured, a subscription endpoint whose path I had already been caught out by once before. Every instinct said "config issue" or "the CLI is broken". It was neither.

The key in the original file was 67 characters. The copy was 66. One character had been eaten somewhere in the quoting, and a key that is one character short is simply a wrong key. The server's message was exactly right; I was looking for the fault in the wrong place.

Why nested quoting eats bytes

Every hop is a separate parser:

1. my local shell reads the command line
2. SSH hands a string to the remote shell, which parses it again
3. pct exec runs a command inside the container, sometimes through another bash -c, which parses it a third time
4. tr -d with a set built out of escaped quote characters is a fourth interpretation of what was typed

At each level, quotes, backslashes and dollar signs can be consumed, doubled or expanded. When you are also deliberately deleting quote characters with tr, the set of characters you are deleting is itself built through three layers of escaping. If one layer reads it differently from the way you intended, tr deletes something you didn't mean it to, and nothing reports an error. The command succeeds and writes a plausible-looking string.

That's the nasty part. A corrupted secret doesn't look corrupted. It's still a long run of random characters. You can't eyeball 66 characters and notice one is missing.

The fix: move the secret as data

I stopped treating the key as text to be typed through shells and treated it as data:

• Copy with a program that reads the file, not with a shell pipeline re-quoting the value. A few lines of Python that open the source file, parse the line and write the destination don't give any shell a chance to reinterpret the bytes.
• Verify by length and fingerprint. Compare the character count and the first few hex characters of the SHA-256 of both copies. If both match, the copy is exact. And neither value is ever printed.

The fix was never in OpenCode. It was a clean copy.

One caveat I noted for myself: this created a second on-disk copy of that key, which goes against my own one-home-per-secret rule. The source of truth is still the secrets manager, and the two copies have to be rotated together. That is a known debt, not a solved problem.

The second one: an install prompt that got the wrong thing pasted in

The same afternoon I was installing a read-only machine identity for my secrets manager, Infisical, so that the agents that only read secrets would stop using a read-write login.

I wrote a small install script. It asked for the client ID, then for the client secret (hidden), wrote them to a mode-600 file, and then proved the identity worked: it read two known secrets, then tried to write a probe secret, which had to be refused. If the write succeeded, the script deleted the probe and the file.

Run interactively from my terminal, it failed twice:

• The first time, the label from the Infisical UI came along with the ID — the pasted text started with "O client ID:" rather than being just the ID.
• Another time, the client ID was pasted into the secret prompt.

Copying from a web page picks up neighbouring text, and a pasted value often carries a trailing newline that can answer the next prompt blank before you've pasted anything into it. A hidden prompt then gives you no visual cue that you've got the wrong thing.

Making the script defensive

I hardened the script against exactly those failures:

• Extract the ID by shape. A client ID is a UUID, so the script runs a regex for the 8-4-4-4-12 hex pattern over whatever was pasted and keeps only the match. Whatever label came along with it is thrown away.
• Reject an ID-shaped secret. If the "secret" contains a UUID or the word "client", it's the ID pasted twice, and the script says so and asks again.
• Drain the input buffer before the secret prompt, so a stray newline from the previous paste can't answer it blank.
• Report lengths, not values. The script prints "got client ID (N chars) and secret (N chars)", which is enough to spot an obviously wrong paste without showing anything.

Those checks were worth adding. But what actually got it installed was taking the terminal out of the loop.

The route that worked: hand it over through the vault

I created the read-only identity in the Infisical UI, and stored its client ID and secret in Infisical itself, in a folder. Then, using the existing read-write identity, the values were fetched straight into a mode-600 temporary file, and the install script ran in file mode: it read the two values from the file, shredded it, and carried on.

No clipboard, no prompt, no shell quoting. The values went from one API to one file to one script as data.

The script's own checks passed: both test reads worked, and the write probe was refused with a 403 — "not allowed to create on secrets". The read-only identity went live at 16:23 that day, and the agents' secret readers now use it; only the sync and rotation jobs keep read-write access.

What I'd take from it

1. "Invalid API key" usually means the key is invalid. Before blaming the tool, check the bytes you gave it.
2. Every layer of shell quoting is a chance to lose a byte. Nested SSH plus pct exec plus tr is three or four parsers deep. It will eventually go wrong, and it won't tell you.
3. Move secrets as data. Read files with a program, pass values on stdin or through a file with tight permissions, or let the secrets manager be the handoff. Don't retype or re-quote them.
4. Verify by length and fingerprint. Character count plus a SHA-256 prefix proves an exact copy without printing anything.
5. If a human must paste, validate the shape. IDs and secrets usually look different; a script can tell them apart better than a tired person at a hidden prompt.

One character out of 67 isn't much. It was still enough to send me looking for the fault in the wrong tool.

---

🤖 Drafted with AI assistance from my own homelab notes, logs and repos, then reviewed and edited before publishing.