Lock Collation Before You Merge a Generated Concat Step

The merge request is green on the author's laptop. CI is not.

bundle.json has the wrong checksum, and the job log still claims every fragment was included. The generated pack.sh concatenates fragments/*.json. That script came from an agent session on a remote Linux box. You are about to debug a flake that is not a flake.

Collation did this. Path order is a build input. Treat it like one.

Read the runner locale before you reread the chat

Open the CI log first. Print the environment the job actually had, not the environment the agent described.

locale
python3 -c 'import locale, sys; print(sys.platform); print(locale.getlocale())'
echo "LANG=${LANG-}"
echo "LC_ALL=${LC_ALL-}"
echo "LC_COLLATE=${LC_COLLATE-}"

Print the same block on the machine that produced the patch. Do not argue from memory. en_US.UTF-8 and C do not sort the same byte strings.

A concat step that uses an unquoted glob is a sort step in disguise. The shell expands fragments/*.json in collation order. Change the locale and you change the bundle. A checksum test then fails even though every file is present.

Mixed machines make this look like CI noise

You review on macOS. The agent wrote the script on Linux. Many hosted runners look closer to C.UTF-8 than to a desktop GUI locale. None of those hosts is broken. They are simply different inputs.

Generated code makes the gap worse. The model copies a glob because the glob is short. Short is not deterministic. You want an explicit file list, or LC_ALL=C plus a documented sort, or both.

Names that differ only by case sit in the same bucket. Linux will keep Auth.json and auth.json as two files. Default macOS APFS may not. An agent on a Linux box can create both. Your laptop then reports one path, and CI reports two. That is not a flaky hasher. That is a filesystem contract bug sitting next to the collation bug.

Recreate the mismatch in a throwaway directory

Work somewhere disposable. Treat the next script as a demonstration template, not as a result you already measured.

# proposal: demo_collation.sh — unexecuted template
set -euo pipefail
rm -rf /tmp/collation-demo
mkdir -p /tmp/collation-demo/fragments
cd /tmp/collation-demo

printf '{"k":1}\n' > fragments/a.json
printf '{"k":2}\n' > fragments/B.json
printf '{"k":3}\n' > fragments/c.json

pack() {
  local out="$1"
  : > "$out"
  for f in fragments/*.json; do
    cat "$f" >> "$out"
    printf '\n' >> "$out"
  done
}

LC_ALL=en_US.UTF-8 pack bundle-utf8.json || LC_ALL=C.UTF-8 pack bundle-utf8.json
LC_ALL=C pack bundle-c.json

sha256sum bundle-utf8.json bundle-c.json
echo "desktop-like order:"
LC_ALL=en_US.UTF-8 bash -lc 'printf "%s\n" fragments/*.json' || true
echo "C order:"
LC_ALL=C bash -lc 'printf "%s\n" fragments/*.json'

Run it where both locales exist. If locale -a does not list en_US.UTF-8, do not install extra locales on a production runner just to imitate a laptop. Pin the script instead.

Watch the two checksums. If they differ, you have the bug in miniature. If they match, keep the uppercase file. C locale puts B.json before a.json. Many UTF-8 locales do not. That single swap is enough to poison a concatenated OpenAPI file, a SQL migration chain, or a changelog blob.

Make order a reviewed artifact

Do not set a locale in CI and hope the next generated script obeys it. Put the contract in the file that ships.

# proposal: pack.sh — manifest first, POSIX collation as belt-and-suspenders
#!/usr/bin/env bash
set -euo pipefail
export LC_ALL=C
export LANG=C

root="$(cd "$(dirname "$0")" && pwd)"
list="$root/fragments.manifest"
out="$root/bundle.json"

if [[ ! -f "$list" ]]; then
  echo "missing $list" >&2
  exit 2
fi

: > "$out"
while IFS= read -r rel || [[ -n "$rel" ]]; do
  [[ -z "$rel" || "$rel" == \#* ]] && continue
  path="$root/$rel"
  if [[ ! -f "$path" ]]; then
    echo "missing fragment $path" >&2
    exit 3
  fi
  cat "$path" >> "$out"
  printf '\n' >> "$out"
done < "$list"

Keep fragments.manifest in git. One relative path per line. Comments allowed. An agent may still draft that list. You still review the order as a product decision, because schema fragments and migrations are ordered documents.

If a manifest is truly impractical, sort with a stable key instead of trusting the glob:

# proposal: fallback only for generated snapshots, not schema history
export LC_ALL=C
mapfile -t files < <(find "$root/fragments" -name '*.json' -type f | LC_ALL=C sort)

find plus sort beats a raw glob. It still loses to a reviewed list. Use the fallback for snapshot dumps. Do not use it for anything whose sequence is load-bearing.

Gate the tree, not the transcript

Add a check that never needs the original agent host. It reads your scripts and your paths.

#!/usr/bin/env python3
"""proposal: collation_gate.py — template, not a shipped metric."""
from __future__ import annotations

import os
import re
import subprocess
import sys
from pathlib import Path

UNQUOTED_GLOB = re.compile(r"(?<![\"'\w])([A-Za-z0-9_./-]*\*[A-Za-z0-9_./-]*)")
SKIP_DIRS = {".git", ".venv", "node_modules", "__pycache__", ".mypy_cache"}


def tracked_files(root: Path) -> list[Path]:
    proc = subprocess.run(
        ["git", "-C", str(root), "ls-files", "-z"],
        check=True,
        capture_output=True,
    )
    out: list[Path] = []
    for raw in proc.stdout.split(b"\0"):
        if not raw:
            continue
        out.append(root / Path(raw.decode()))
    return out


def bash_glob(dir_path: Path, lc_all: str) -> list[str]:
    env = os.environ.copy()
    env["LC_ALL"] = lc_all
    env["LANG"] = lc_all
    quoted = dir_path.as_posix().replace("'", "'\\''")
    proc = subprocess.run(
        ["bash", "-lc", f"printf '%s\\n' '{quoted}'/*"],
        check=False,
        capture_output=True,
        text=True,
        env=env,
    )
    return [line for line in proc.stdout.splitlines() if line]


def collation_problems(root: Path) -> list[str]:
    hits: list[str] = []
    dirs = sorted({path.parent for path in tracked_files(root) if path.is_file()})
    current = os.environ.get("LC_ALL") or os.environ.get("LANG") or "C"
    for directory in dirs:
        if SKIP_DIRS.intersection(directory.relative_to(root).parts):
            continue
        c_order = bash_glob(directory, "C")
        here = bash_glob(directory, current)
        if c_order != here and len(c_order) > 1:
            rel = directory.relative_to(root).as_posix() or "."
            hits.append(f"collation drift in {rel}/ under LC_ALL={current!r}")
    return hits


def glob_problems(root: Path) -> list[str]:
    hits: list[str] = []
    for path in tracked_files(root):
        if path.suffix not in {".sh", ".bash", ".zsh"}:
            continue
        text = path.read_text(encoding="utf-8", errors="replace")
        for index, line in enumerate(text.splitlines(), 1):
            stripped = line.split("#", 1)[0]
            if "LC_ALL=C" in text and "sort" in stripped:
                continue
            match = UNQUOTED_GLOB.search(stripped)
            if match:
                rel = path.relative_to(root).as_posix()
                hits.append(f"unquoted glob {rel}:{index}: {match.group(1)}")
    return hits


def case_collisions(root: Path) -> list[str]:
    seen: dict[str, str] = {}
    hits: list[str] = []
    for path in tracked_files(root):
        rel = path.relative_to(root).as_posix()
        key = rel.lower()
        if key in seen and seen[key] != rel:
            hits.append(f"case collision {seen[key]} vs {rel}")
        else:
            seen[key] = rel
    return hits


def main() -> int:
    root = Path(sys.argv[1] if len(sys.argv) > 1 else ".").resolve()
    problems = collation_problems(root) + glob_problems(root) + case_collisions(root)
    if not problems:
        print("collation_gate: ok")
        return 0
    print("collation_gate: fail")
    for item in problems:
        print(f"- {item}")
    return 2


if __name__ == "__main__":
    raise SystemExit(main())

Run it on a clean checkout. Pass the repo root if you are not already there.

# proposal: local invocation
python3 collation_gate.py .

Wire the same command into GitLab CI so the gate does not depend on a laptop locale. Keep the image boring.

# proposal: .gitlab-ci.yml fragment — unexecuted template
collation_gate:
  image: python:3.12-bookworm
  script:
    - locale
    - python3 collation_gate.py .

The job should fail closed. A missing fragments.manifest is a product defect, not a warning you scroll past.

Decide with a table, then stop debating hosts

What you observed	Safe to merge?	What to require
Unquoted glob, no LC_ALL=C, checksum test only	No	Manifest or POSIX sort in the script
Glob plus export LC_ALL=C and a CI locale dump	Maybe	Human review of file order
Reviewed fragments.manifest committed next to pack.sh	Yes, as order	Still run the checksum in CI
Two tracked paths differ only by case	No	Rename on a case-sensitive clone
Laptop green, CI red, locales never printed	No	Print locale on both sides first
Agent said "all files included" and showed stdout	No	Compare git paths, not chat

"Maybe" still needs a reviewer. Tables do not merge requests.

A spare Linux shell helps. Its locale is not the contract

If you work on macOS and the runner is Linux, you need one case-sensitive shell to see the real tree. That is the whole reason to use a remote box for this check.

MonkeyCode offers free model access and a free server option, which is enough to draft pack.sh and then rerun the concat under Linux. Disclosure: This article was prepared as part of MonkeyCode's product outreach. Use that server to print locale, run collation_gate.py, and catch Auth.json versus auth.json. Do not use it as the definition of production order.

Copy the locale dump into the merge request. Leave the generated script behind only after the manifest exists. The free box is a rehearsal stage. The committed list is the show.

Numbered path from red CI to a mergeable script

1. Print locale in CI and on the machine that wrote pack.sh.
2. Recreate the two-locale concat in a throwaway directory.
3. Replace the glob with fragments.manifest or with find | LC_ALL=C sort.
4. Commit the manifest next to the script.
5. Run python3 collation_gate.py . on a case-sensitive clone.
6. Fail the pipeline if the gate exits 2.
7. Review the order as a behavior change, then merge.

Skip step 7 and you only moved the bug from the glob into a file nobody read.

What this gate does not prove

A stable sort is not a schema review. A manifest can still concatenate the wrong documents in a confident order. Checksums do not tell you the JSON is valid.

LC_ALL=C is also the wrong hammer for user-facing sort. If the product must sort names for German or Swedish readers, do that in application code with an explicit locale. Do not inherit it from a build host.

Skip this workflow when the artifact is a locale-aware UI list. Skip it when you cannot run a POSIX shell. Skip it for regulated trees you must not send to a third-party host. Solo spikes can live with a glob for an afternoon. Ordered migrations cannot.

Clocks, containers, and uname will not save you here. The file list will.

What belongs in the merge request

Paste the locale dump. Paste the manifest diff. Paste the gate output.

If the answer is a screenshot of an agent saying the bundle looks complete, reject the request. Completeness is not order. Order is data. Put that data in git before anyone concatenates another file.