Testing an AI LaTeX Repair Loop With Real Compiler Failures

Figure 1: TeX64’s current Axiom documentation, captured on 2026-08-31. A guarded edit is written first and then shown as an applied diff for review or Undo. The screenshot supports the product surface; the compiler fixture is published separately.

Figure 2: A Fermion-owned TeX64 test workspace captured on 2026-03-10. The file tree and preview show real .tex, .log, and PDF artifacts in the application. This frame does not show a complete before-and-after repair trace or prove that the agent selected the intended root.

An agent can give an excellent explanation of Undefined control sequence without changing a single byte. It can also patch the wrong main.tex, compile a different root successfully, and report victory. Both outcomes look convincing in a chat transcript. Neither repairs the document.

The test target for a LaTeX repair loop should be an artifact, not an answer: the intended source changed within a declared boundary, the same root was rebuilt, and the expected PDF exists. When repair fails, the system should stop with the first useful evidence rather than improvise indefinitely.

This article describes the contract used in TeX64’s current compiler path. The implementation is product-specific, but the state machine and fixtures apply to any agent that edits buildable documents.

Give compilation its own capability

The model-facing tool is compile_document, not run_shell(command). That decision removes many tasks from the agent’s authority, including arbitrary file discovery, network utilities, and unrelated process execution. The compile tool accepts a document and optional engine, resolves it inside the workspace, invokes the existing build service, and returns structured issues with a bounded log.

A simplified interface looks like this:

type CompileInput = {
  mainFile?: string;
  engine?: "pdflatex" | "xelatex" | "lualatex" | "uplatex";
};

type CompileResult = {
  status: "success" | "failed";
  targetFile: string;
  issues: Array<{ file?: string; line?: number; message: string }>;
  logExcerpt: string;
  pdfPath?: string;
};

The exact type is illustrative; the important constraints are the named target, bounded output, and explicit artifact. Dumping an entire log into the model context wastes tokens and can bury the first actionable error. Truncating without parsing is also unsafe because the relevant file and line may be lost. Preserve structured issues, then include enough surrounding log to identify the tool and package involved.

Root selection is part of correctness

Many repositories contain more than one main.tex: a thesis, a slide deck, supplementary notes, and minimal examples. “Find the first file named main” is not a valid build policy.

The repair loop resolves in this order:

1. an explicitly requested document inside the workspace;
2. otherwise the active .tex file captured for this turn, followed by the conversation’s document main file;
3. a valid % !TEX root declared by that selected candidate;
4. only when there is no candidate, the workspace root metadata or main.tex.

It must not silently fall back to an unrelated workspace root after the selected document fails. One real-build fixture makes this visible: the workspace-level main.tex contains an undefined command, while documents/notes/main.tex is valid and active. The test passes only if documents/notes/main.pdf is created. A mocked “compiler called” assertion would miss this entire class of false success.

Separate source repair from environment repair

A missing package and a missing brace both produce failed builds, but the safe actions differ.

For source failures, the agent may read the relevant region and use bounded line, section, or patch tools. Model-facing writes cannot remove protected LaTeX structure such as \documentclass, \begin{document}, \end{document}, title structure, or bibliography navigation. Another deterministic guard rejects a newly introduced adjacent duplicate block, a common failure when a model inserts a replacement next to the original instead of replacing it.

For environment failures, editing the source may be wrong. If the log says File 'booktabs.sty' not found, the build path can attempt package recovery only when all of these conditions hold:

• the TeX distribution is managed by the application;
• the normalized basename passes the filename and extension allow-list;
• tlmgr search identifies an exact listed-path basename match;
• the install targets the managed tree, not a system TeX installation;
• the retry budget has not been spent.

TeX Live’s upstream package manager is documented in the official tlmgr manual. The application should not parse a vague substring and install the first similarly named package. It should also leave a user-managed MacTeX, TeX Live, or other distribution unchanged.

Make the retry budget explicit

An unbounded “compile, ask model, edit, repeat” loop can spend money and damage a source file without increasing information. A small state machine is easier to reason about:

DIRTY
  -> COMPILE
       -> SUCCESS -> VERIFIED
       -> MISSING_PACKAGE
            -> exact managed recovery available?
                 yes -> install next package -> COMPILE
                 no  -> STOP_WITH_EVIDENCE
       -> SOURCE_FAILURE
            -> bounded edit -> DIRTY
       -> SAME_FAILURE_WITHOUT_NEW_EVIDENCE
            -> STOP_WITH_EVIDENCE

After the initial failure, TeX64 allows at most four managed recovery rounds for the same exact target. Each successful round resolves an exact missing filename, excludes packages already installed during this invocation, installs only through the managed tlmgr, and rebuilds the same document. If the same unresolved package appears again, exact resolution yields no new package and the loop stops. A newly exposed different package may consume the next recovery round.

The agent loop also records whether a successful model-triggered compile already cleared the dirty state. If the model edits a file and then finishes without compiling, a deterministic final compile runs once. If compilation already succeeded after the latest write, the settlement path does not build the same document again.

Test failures by category, not by anecdote

A useful fixture family should contain at least these cases:

Fixture	Expected repair boundary	Required assertion
Undefined command in source	Small source edit	Original removed or corrected; no duplicate block
Missing .sty in managed TeX	Exact package recovery, at most four managed rounds	Already-installed packages excluded; same root rebuilt
Missing .sty in system TeX	No mutation of system environment	Stop with package evidence
Unmatched document structure	Guarded source edit	Protected boundaries remain
Valid nested active root	No repair required	PDF appears beside the nested root
Broken workspace root, valid active root	Correct target selection	Broken unrelated root is not compiled
Same error after retry	No further blind retries	Stable failure result and bounded log
External symlink target	No traversal	Build is refused outside the workspace

Store four artifacts for every fixture: initial source, first compiler output, applied patch or environment action, and final build result. For successful cases, retain the PDF checksum or at least assert that a real non-empty PDF was produced. For failures, assert the exact stopping reason. A screenshot of a green status badge is not a test record.

The public axiom-repair trace records the selected nested root, first compiler failure, bounded one-file edit, successful rebuild, and PDF hash. The evidence manifest binds the published artifact hashes. The separate axiom-review trace records that the guarded edit was saved before the applied-diff card, Done left the saved bytes unchanged, and Undo restored the original source and rebuilt it.

What successful compilation does not prove

A PDF can compile while containing an incorrect proof, a false citation, a broken cross-reference hidden by warning settings, or layout that is unusable on the intended page size. The repair loop validates a narrower statement: the declared document builds in the declared environment after the recorded action.

Visual review, domain review, and warning policy belong to additional gates. Combining them into a single “AI fixed it” score makes failures hard to diagnose.

The same restraint applies to product claims. We do not claim that this path repairs every LaTeX error or that users never need to read a log. The TeX64 Axiom documentation is the specific product reference; the engineering value in this article is the bounded capability, explicit state machine, and reproducible fixture family.

If a case reports success without the intended PDF, or package recovery touches an unmanaged TeX tree, include the fixture, selected root, engine, and first bounded log when reporting it through TeX64 support. A repair system improves when failures become permanent tests, not when its success message becomes more confident.