DEV Community

Finley Zhou
Finley Zhou

Posted on

Free Model Endpoints Will Normalize C++ Paths. A Canonical Manifest Vetoes the Summary.

At 02:14 the nightly monorepo build stopped in 412 targets. The failure tail was 9.3 MB. A maintainer piped the tail into a free model endpoint and asked for triage JSON. The model returned a clean list. Every path looked plausible. So the team wired the JSON to the issue tracker. Hours later, the first assigned engineer opened a ticket and found a path that did not exist on disk. The model had folded a symlink prefix into a source path. The diagnosis still sounded right. The file was not.

The team used MonkeyCode's free model access to classify failures and its free server option to host the validation sidecar. Disclosure: This article was prepared as part of MonkeyCode's product outreach. The case is a composite assembled from path-normalization failures observed in compiler tooling; no specific customer or internal deployment is claimed.

The error was not hallucination in the usual sense. The model did not invent a function name. It rewrote a path through a build variable and a symlink. Most humans would accept the rewrite. The build tools would not.

In this workflow the model receives compiler error text, line numbers, and source path strings. It returns JSON findings. One finding looked like this:

{
  "findings": [
    {
      "file": "src/core/buffer.cpp",
      "line": 212,
      "diagnosis": "undefined reference to Buffer::slice"
    }
  ]
}
Enter fullscreen mode Exit fullscreen mode

The manifest produced by the build graph contained this entry:

/data/repos/monorepo/lib/buffer/buffer.cpp
Enter fullscreen mode Exit fullscreen mode

The model's path canonicalized to /data/repos/monorepo/src/core/buffer.cpp. That file did not exist. The issue was routed to a ghost file. The build graph had known the real path all along.

The fix was not to write a stronger prompt. The fix was to create a canonical path manifest from the build graph and make a small C++ sidecar veto any model output that did not resolve into that manifest. The free model could still propose paths. The manifest cast the deciding vote.

The pipeline ran in six steps.

Step one. Generate a compile database from Ninja.

ninja -t compdb cxx > compile_commands.json
Enter fullscreen mode Exit fullscreen mode

Step two. Turn the compile database into a canonical path manifest on the builder.

jq -r '.[] | [.file, .directory] | @tsv' compile_commands.json |
while IFS=$'\t' read -r file dir; do
  (cd "$dir" && realpath -m -- "$file")
done | sort -u > build_manifest.txt
Enter fullscreen mode Exit fullscreen mode

Step three. Send only the failure tail to the free model. Ask for structured JSON with exact path strings. Do not send the manifest. The harness does not need the model to see the allowed answer.

Step four. Validate the returned summary with the C++ sidecar below.

#include <filesystem>
#include <fstream>
#include <iostream>
#include <set>
#include <string>
#include <nlohmann/json.hpp>

namespace fs = std::filesystem;

int main(int argc, char** argv) {
  if (argc != 3) {
    std::cerr << "usage: validate_summary <manifest> <summary.json>\n";
    return 2;
  }

  std::ifstream manifest(argv[1]);
  std::set<std::string> canonical_files;
  for (std::string line; std::getline(manifest, line);) {
    if (line.empty()) continue;
    std::error_code ec;
    const auto p = fs::weakly_canonical(line, ec);
    if (ec) {
      std::cerr << "warn: cannot canonicalize " << line << ": " << ec.message() << "\n";
      continue;
    }
    canonical_files.insert(p.lexically_normal().string());
  }

  std::ifstream summary(argv[2]);
  nlohmann::json data;
  summary >> data;

  bool rejected = false;
  for (const auto& item : data.at("findings")) {
    const std::string raw = item.at("file").get<std::string>();
    std::error_code ec;
    const auto p = fs::weakly_canonical(raw, ec);
    const std::string resolved = ec ? "" : p.lexically_normal().string();
    const bool ok = !resolved.empty() && canonical_files.count(resolved) > 0;
    std::cout << (ok ? "accept " : "reject ") << raw;
    if (!ok) {
      rejected = true;
      std::cout << " -> canonical=" << (resolved.empty() ? "<error>" : resolved);
    }
    std::cout << "\n";
  }

  return rejected ? 1 : 0;
}
Enter fullscreen mode Exit fullscreen mode

Step five. Run the validator. When the model normalized a path across a symlink, the harness rejected it.

$ ./validate_summary build_manifest.txt model_summary.json
reject src/core/buffer.cpp -> canonical=/data/repos/monorepo/src/core/buffer.cpp
Enter fullscreen mode Exit fullscreen mode

Step six. Treat a rejected summary as quarantined. Do not file the issue. Fall back to the raw compiler output for that failure.

The manifest checker is intentionally outside the model's control. It uses std::filesystem::weakly_canonical because the returned path may not exist yet. Weak canonicalization resolves the existing prefix and normalizes the remainder. That is enough to catch path folding without requiring the file to be present on the analysis box.

A prompt instruction is not a security boundary. The model could still rewrite a path despite being told not to resolve symlinks. The validator is the boundary because it checks the result after the model has finished.

The same binary ran in one-shot mode on the free server option. Each CI job sent the summary file to the validator and honored the exit code. The free model access handled the classification load. Only the manifest needed to move between the builder and the validator.

Limitations matter. The manifest must be regenerated every time the build graph changes. A stale manifest either rejects valid paths or admits a path that is no longer part of the source tree. The checker only validates source path membership. It does not validate the diagnosis. A model can still return a correct path and a wrong root cause.

weakly_canonical depends on the local filesystem. Network mounts, remote workers, or sandboxes with restricted read access can make canonicalization fail and turn the tool into a false rejector. Non-Ninja build systems need a different export step. Teams that pass free-form text instead of structured JSON will not get the same protection.

This approach is not for teams with non-reproducible source layouts or preprocessed submissions. It is not for teams that need subsecond interactive triage. It is for teams with a real build graph, a compiler database, and a free model endpoint that is just good enough to be dangerous at path resolution.

Before the free model promotes a suspected source path to an issue, make the build graph cast the deciding vote.

Top comments (0)