Writing Files Over SSH: How a base64 Newline Bug Was Silently Corrupting Data

When implementing SSH-based file writes in a Flask dashboard, I ran into a bug where only certain remote nodes would fail. The culprit turned out to be a subtle behavior of the base64 command I had taken for granted.

---

What I Was Building

I run multiple Linux nodes on a home LAN, managed through a Flask-based admin dashboard. One feature writes configuration files (auth-profiles.json) to each node. Local nodes get direct writes, but remote nodes require SSH.

My original implementation looked like this:

import base64, subprocess

content = json.dumps(auth_data, indent=2)
encoded = base64.b64encode(content.encode()).decode()

ssh_cmd = f"echo '{encoded}' | base64 -d > /path/to/auth-profiles.json"
subprocess.run(["ssh", f"user@{host}", ssh_cmd], ...)

Looks clean, right? It wasn't.

---

Symptom: Only Specific Nodes Fail

During testing, the local node (infra) worked fine. Remote nodes like joe and work-a would fail intermittently. Errors included base64: invalid input, empty output files, and corrupted JSON.

Different environments? SSH config issue? After chasing these red herrings, I found the real cause was on the Python side.

---

Root Cause: Shell Argument Limits + base64 Defaults

Python's base64.b64encode() returns a byte string with no newlines. But passing a long string to echo '...' over SSH creates two problems:

1. Shell argument length limits — longer JSON means longer base64 strings, hitting shell-dependent limits for quoted strings
2. Shell quoting edge cases — special characters within the encoded string can confuse the remote shell's quoting rules

The nodes where JSON was larger failed more often. The pattern became clear: the longer the encoded string, the more likely shell interpretation would break it.

---

Fix: Stream File Content via stdin

There's a simpler approach that avoids both quoting and base64 entirely: pipe file content directly to SSH's stdin.

import json, subprocess

def write_auth_profiles(host: str, path: str, auth_data: dict) -> bool:
    content = json.dumps(auth_data, indent=2).encode()

    if host == "localhost":
        with open(path, "w") as f:
            json.dump(auth_data, f, indent=2)
        return True

    # Stream content via stdin — no quoting, no base64
    result = subprocess.run(
        ["ssh", f"user@{host}", f"cat > {path}"],
        input=content,
        capture_output=True,
    )
    return result.returncode == 0

The key is input=content in subprocess.run. Python passes the byte string directly to the process's stdin. On the SSH side, cat > /path/to/file writes stdin straight to disk.

No quoting. No base64. No newline issues.

---

Before / After

	Old (base64 echo)	New (stdin cat)
Command complexity	High (escape hell)	Low (one command)
Length limit	Yes (shell-dependent)	None
Special char safety	Weak	Strong
Debuggability	Low	High

---

Lessons Learned

When writing files over SSH, the echo '...' | base64 -d pattern looks simple but hides multiple failure modes:

1. Long JSON (hundreds of bytes+) strains shell argument handling
2. Node environment differences (bash version, shell config) cause inconsistent behavior
3. Hard to debug — SSH commands with long encoded strings produce unreadable logs

subprocess.run(input=...) + cat > file is simpler and more robust. Unless base64 encoding is actually required, reach for stdin piping first.

---

Bonus: Found 15 More Instances of the Same Pattern

After fixing the bug, a grep through server.py revealed 15 other places using the same echo '...' | base64 -d pattern for bot config writes. Consolidating them into one function reduced the blast radius of future changes.

Technical debt: find it, fix it, consolidate it.

---

Tags: #Python #SSH #Flask #DevOps #Linux #BugFix #SRE