I Logged Checkmarks for 48 Hours. The Server Encoding Was ASCII.

Have you ever watched a boring logging line explode only after the code left your laptop? I spent forty-eight hours chasing a UnicodeEncodeError that never once appeared on my own machine. The suite stayed green locally, the remote traceback named an ascii codec, and I kept deleting the wrong characters. This notebook is the order of operations I will follow the next time encodings drift.

Hour 0: a checkmark that only failed somewhere else

I had added a tiny status logger so a long pytest run would show progress without dumping every assertion. One of those lines printed a checkmark, a dash, and a short human sentence about the step that had just finished. On my Ubuntu laptop the output looked friendly, almost decorative, and encodings never crossed my mind. Why would they, when the same file had been round-tripping through git without drama for weeks?

Then I asked a coding assistant to run the same tests on a second machine. Disclosure: This article was prepared as part of MonkeyCode's product outreach. I was using MonkeyCode's free model access and the free server option as another interpreter, not as a substitute for a pinned CI image. The traceback was immediate, ugly, and strangely confident about a codec I had not chosen.

UnicodeEncodeError: 'ascii' codec can't encode character '\u2713' in position 0: ordinal not in range(128)

Have you stared at that message long enough to start blaming the wrong layer in the stack? I did, for most of a working day, before I printed a single encoding field from sys.

What I tried first, in the wrong order

Hours 0–8: I blamed the character

I replaced the checkmark with "OK" and reran a single test, which of course passed on the remote process. Then another helper printed an em dash that I had copied from a sentence in the spec. Then a fixture wrote a temporary CSV whose header contained naïve because the sample file came from a real ticket. Each cosmetic fix uncovered the next non-ASCII byte like a bad game of whack-a-mole.

• I grepped the repo for emoji, en dashes, and anything that was not plain ASCII punctuation.
• I switched the progress logger from print to logging.info and expected the traceback to vanish.
• I added errors="replace" on one open() call and felt briefly, incorrectly clever.
• I did not print sys.stdout.encoding until hour twenty-something, which still embarrasses me.

Logging did not save me. Do you know why that substitution failed so quietly? The default StreamHandler still encodes through the same stdout encoding unless you hand it a different stream or a custom formatter with an explicit codec.

Hours 8–16: I blamed pytest capture

I ran pytest -s because capture has lied to me before, especially around sockets that write mixed text and binary frames. The error moved from the captured section into the live output and kept the same codec name, which should have been the clue. I then ran pytest --capture=no -vv on one file and only proved the failure was not an artifact of file-descriptor redirection. Pytest was innocent, and I was stalling by rerunning flags I already understood.

pytest -s tests/test_status_logger.py
pytest --capture=no -vv tests/test_status_logger.py::test_progress_line
python -c "import sys; print(repr(sys.stdout.encoding))"

That last command is the one I should have typed at minute ten. I typed it at hour eighteen, after rewriting comments.

Hours 16–24: I blamed Docker, then a library

The remote command was not even running inside a container I owned, which did not stop me from rebuilding a local image. I set LANG=C.UTF-8 in that Dockerfile, watched the image pass, and learned nothing about the process I was actually debugging. I also opened a private argument with rich, click, and a vendor SDK that had never printed the checkmark. None of those packages appeared in the traceback, yet I still read their issues like they owed me an apology.

The fingerprint I wish I had run immediately

Here is the artifact I now keep beside the tests. It is a boring script on purpose, because the bug was boring and environmental. It would have ended the hunt on hour one if I had treated the remote interpreter as a foreign locale instead of my laptop with extra latency.

# env_fingerprint.py
"""Print the encodings this process will actually use. Run before the suite."""
from __future__ import annotations

import locale
import os
import sys
from pathlib import Path


def main() -> None:
    preferred = locale.getpreferredencoding(False)
    loc = locale.getlocale()
    rows = [
        ("python", sys.version.split()[0]),
        ("executable", sys.executable),
        ("platform", sys.platform),
        ("LANG", os.environ.get("LANG", "<unset>")),
        ("LC_ALL", os.environ.get("LC_ALL", "<unset>")),
        ("LC_CTYPE", os.environ.get("LC_CTYPE", "<unset>")),
        ("PYTHONIOENCODING", os.environ.get("PYTHONIOENCODING", "<unset>")),
        ("PYTHONUTF8", os.environ.get("PYTHONUTF8", "<unset>")),
        ("locale.getlocale", repr(loc)),
        ("preferredencoding", preferred),
        ("filesystemencoding", sys.getfilesystemencoding()),
        ("stdout.encoding", getattr(sys.stdout, "encoding", None)),
        ("stderr.encoding", getattr(sys.stderr, "encoding", None)),
        ("utf8_mode", getattr(sys.flags, "utf8_mode", None)),
    ]
    width = max(len(key) for key, _ in rows)
    for key, value in rows:
        print(f"{key:<{width}}  {value}")

    probe = "✓ café — naïve"
    try:
        sys.stdout.write(probe + "\n")
        sys.stdout.flush()
        print("stdout_probe  ok")
    except UnicodeEncodeError as exc:
        print("stdout_probe  FAILED:", exc, file=sys.stderr)
        raise SystemExit(2)

    sample = Path("fingerprint-sample.txt")
    try:
        # Omitting encoding follows the locale unless UTF-8 mode is already on.
        sample.write_text(probe + "\n")
        print("write_text    ok")
    except UnicodeEncodeError as exc:
        print("write_text    FAILED:", exc, file=sys.stderr)
        raise SystemExit(3)
    finally:
        if sample.exists():
            sample.unlink()


if __name__ == "__main__":
    main()

Run it locally and in the remote process with the same Python major version, then diff the rows instead of the test names.

python env_fingerprint.py
python -c "import sys, locale; print(sys.stdout.encoding); print(sys.flags.utf8_mode); print(locale.getpreferredencoding(False))"
locale
locale charmap
env | grep -E '^(LANG|LC_|PYTHON)' | sort

On my laptop the preferred encoding was utf-8 and stdout.encoding matched it without any extra exports. On the remote process LANG was C, locale.getlocale() returned (None, None), and stdout.encoding reported US-ASCII. That single row explained every checkmark I had deleted and every em dash I had treated as a content bug.

Would you have checked PYTHONUTF8 before rewriting log lines into baby talk? I would not have, and that reluctance is the entire lesson of this notebook.

A pytest guard that fails before the suite pretends to pass

Deleting every non-ASCII character made the suite green and made the next encoding bug silent. The following teammate will paste a ticket title containing an en dash, and the fire returns in a file you stopped reading. I now fail fast when the process cannot round-trip a tiny probe string that already exists in our fixtures.

# tests/conftest.py
"""Fail collection if this interpreter cannot encode characters we already ship."""
from __future__ import annotations

import os
import sys

import pytest

PROBE = "✓ café — naïve"


def pytest_sessionstart(session: pytest.Session) -> None:
    encoding = getattr(sys.stdout, "encoding", None) or ""
    if os.environ.get("ALLOW_ASCII_STDOUT") == "1":
        return
    try:
        PROBE.encode(encoding)
    except (LookupError, UnicodeEncodeError) as exc:
        pytest.exit(
            f"stdout encoding {encoding!r} cannot represent test output ({exc}). "
            "Export PYTHONUTF8=1 or PYTHONIOENCODING=utf-8, or set "
            "ALLOW_ASCII_STDOUT=1 for a truly ASCII-only run.",
            returncode=2,
        )

The shell I now export in remote sessions looks like this, and I run the fingerprint before pytest so a red X has a locale attached.

export PYTHONUTF8=1
export PYTHONIOENCODING=utf-8
export LANG=C.UTF-8
python env_fingerprint.py
pytest -q

PYTHONUTF8=1 forces UTF-8 mode without waiting for login scripts to define a UTF-8 locale, which remote images often skip. PYTHONIOENCODING=utf-8 is the narrower lever when you only want stdin, stdout, and stderr changed for that process. I set both because I am tired of relearning the difference at two in the morning, not because both levers are required on every host.

Decision table: the failures I actually recorded

Symptom	What I thought it was	What it actually was	Repeatable check
UnicodeEncodeError: 'ascii' codec on print	A bad character in my logger	LANG=C, stdout ASCII	python -c "import sys; print(sys.stdout.encoding)"
Logs look fine in a UTF-8 terminal, die in pytest	pytest capture bug	the test process inherited POSIX locale	fingerprint stdout.encoding
Path.write_text(text) raises, write_bytes does not	a corrupt fixture file	text mode used locale encoding	locale.getpreferredencoding(False)
Only the remote run fails	flaky test order	different LANG / LC_ALL	locale versus local locale
Replacing emoji "fixes" the job	a content problem	you hid the encoding drift	probe string in conftest.py
rich or click suspected	a dependency regression	they inherit stdout encoding	traceback module names

Notice that none of these rows are about model quality or prompt wording. An assistant can draft a clean patch and still execute it inside a POSIX locale that your laptop has not used in years. Have you been grading the generated code when you should have been grading LANG?

What broke when I "fixed" the wrong layer

Rewriting logs to pure ASCII made the suite green and taught the next change nothing about the process. Forcing errors="replace" turned checkmarks into question marks and hid a truncated display name in a fixture dump I later trusted. Setting LANG inside one child subprocess did not change pytest's parent interpreter, which is the process that actually prints session banners.

I also learned that Path.write_text(probe) without encoding="utf-8" is not the portable shortcut I had been treating it as in reviews. On a UTF-8 laptop the omission is invisible and survives code review because nothing explodes. On LANG=C the same call is a landmine that only remote jobs step on. The explicit encoding="utf-8" argument is not a style nit; it is a contract about the bytes you meant to store.

from pathlib import Path

path = Path("status.txt")
# Portable: the file is UTF-8 even when LANG=C and UTF-8 mode is off.
path.write_text("✓ done\n", encoding="utf-8")

# Host-dependent: follows locale encoding unless PYTHONUTF8=1 is already set.
path.write_text("✓ done\n")

What I would repeat next time

1. Fingerprint encodings before the first message that asks why a remote run is different from laptop pytest.
2. Pin PYTHONUTF8=1 in the remote session, then rerun the failing test without rewriting log lines.
3. Add the pytest session guard so ASCII locales fail at collection, not at a random decorative print.
4. Keep encoding="utf-8" on every text open() and Path.write_text() that ships in the repository.
5. Treat web transcripts as UTF-8 renderers, never as evidence of the child process locale.

Would I still bother with a second, less cozy interpreter for this class of bug? Yes, because a machine that boots into LANG=C will volunteer failures my laptop politely hides. When I need that second interpreter, I rerun the fingerprint on MonkeyCode's free server after the free model drafts a change, then I trust the script more than the transcript.

Limitations, and who should skip this

This workflow assumes you control the process environment and that UTF-8 is the right on-disk contract for the files you own. It will fight you if you ingest legacy bytes that are actually Latin-1, CP1252, or Shift-JIS and you need the locale to stay honest about those files. Do not export PYTHONUTF8=1 on a pipeline whose job is to round-trip those bytes unchanged, because you will silently recode data that was never UTF-8.

Skip the pytest guard if you maintain an ASCII-only embedded target and non-ASCII output would be a real product defect rather than an operator inconvenience. In that case keep ALLOW_ASCII_STDOUT=1 and ban the characters in lint so the failure stays in review, not in a remote locale surprise. Also skip the drama if your remote is already a fully specified container with C.UTF-8 baked into the image; the fingerprint should still run, but you probably will not get a forty-eight hour plot out of it.

I am not claiming any remote host keeps a particular locale, quota, or image after the next rebuild, including hosts I used for this notebook. Locales drift when base images change and when someone copies a sudo command without copying ENV LANG. That is why the script is the artifact, not a screenshot of one passing run and not a promise about anyone else's shell.

If you steal one command from these notes, steal the fingerprint and run it wherever your tests actually execute. I keep that script next to conftest.py now, and I run it before I ask any assistant to interpret a red X that my laptop cannot reproduce.