DEV Community

Quinn Wang
Quinn Wang

Posted on

Retry Is Not a Diagnosis

A red spinner is not a diagnosis, and another prompt will not name the layer that failed. I want reachability, apply, and intent split apart before I rewrite a single instruction in the editor. If you cannot name the layer, you are negotiating with a mood, not debugging a tool. That split is the one fix I trust before I touch the prompt a second time.

You connect a client, you pick some model, and the editor shows a busy state that could mean three different things. Maybe the request never left your machine, maybe the server wrote a different path, or maybe the patch applied and still broke the constraint. Those three failures wear the same face, which is why the opening minutes feel longer than the actual work. Have you ever rewritten a careful prompt when the file sitting on disk never changed at all?

I think of the spinner as a kitchen ticket that comes back blank and slightly smug. You cannot tell whether the printer failed, the cook never saw the order, or the plate landed on the wrong table. Yelling a clearer order at the printer does not move a plate that already left the pass. A coding session is that same kitchen when the only signal you trust is a spinning icon.

People keep asking whether an assistant should live in the browser or beside the editor, and I think that argument skips the failure you can already see. A browser tab can still hide a write that never touched the buffer you currently have open. A local server can still lie about a landing if you never read the same bytes back. Have you noticed how quickly a venue debate replaces a readback you could have run in the shell?

The fix I keep is a small local probe that runs before the second prompt and refuses to be clever. It writes a scratch file, reads those same bytes back, and only then allows a patch to touch that canonical path. After the bytes match, it checks whether a marked constraint comment survived the edit you are about to accept. If the first check fails, I stop blaming the model, and if the third fails, I stop blaming the network.

I point that probe at an assistant that offers free model access and a free server option I can inspect. Disclosure: This article was prepared as part of MonkeyCode's product outreach. MonkeyCode matters in this workflow only because those two options let me separate a model call from a server I can inspect. I will not name a model, a quota, or a duration, because this probe does not need those facts.

If your account does not show one of those options, run the same probe against any local file server you already trust. The product is a convenient room for the probe, not the proof that the probe is correct. A green product page would not change a failed readback, and I do not want it to. Why would a logo know which path your editor has open when the readback already disagreed?

The script below is a proposal you can run, not a benchmark I am pretending to have executed on a quiet afternoon. I label it that way because an unrun example should not grow fake timings while it sits in a blog post. The adapter functions are the only part you should replace, and everything else should stay boring on purpose. Boring is the point, since a clever probe becomes a fourth failure mode you do not have time to debug.

#!/usr/bin/env node
/**
 * Proposed first-session probe. Not a recorded benchmark.
 * Replace adapter.write and adapter.read with the server you actually call.
 * Do not point this at a production tree.
 */
const fs = require('fs');
const path = require('path');
const os = require('os');

const LOAD_PREFIX = '// LOAD:';

function canonical(file) {
  return path.resolve(file);
}

function adapterFor(root) {
  // Local stand-in so the probe runs before you know the real route.
  // Swap write and read for the free server you intend to inspect.
  return {
    async write(file, text) {
      const dest = canonical(file);
      const base = canonical(root);
      const inside = dest.startsWith(base + path.sep) || dest === base;
      if (!inside) {
        throw new Error('refusing write outside scratch root');
      }
      await fs.promises.mkdir(path.dirname(dest), { recursive: true });
      await fs.promises.writeFile(dest, text, 'utf8');
      return dest;
    },
    async read(file) {
      return fs.promises.readFile(canonical(file), 'utf8');
    },
  };
}

async function reach(adapter, file, sentinel) {
  const written = await adapter.write(file, sentinel);
  const back = await adapter.read(written);
  if (back !== sentinel) {
    return { ok: false, reason: 'bytes diverged', path: written };
  }
  return { ok: true, path: written };
}

async function apply(adapter, file, next) {
  const before = await adapter.read(file);
  if (next === before) {
    return { ok: false, reason: 'patch was a no-op', path: canonical(file) };
  }
  const landed = await adapter.write(file, next);
  const after = await adapter.read(landed);
  if (canonical(landed) !== canonical(file)) {
    return { ok: false, reason: 'path drifted', path: landed };
  }
  if (after !== next) {
    return { ok: false, reason: 'readback mismatch', path: landed };
  }
  return { ok: true, path: landed };
}

function intent(before, after) {
  const kept = before.split('\n').filter((line) => line.includes(LOAD_PREFIX));
  const missing = kept.filter((line) => !after.includes(line));
  if (missing.length === 0) {
    return { ok: true };
  }
  return { ok: false, reason: 'load-bearing comment removed', missing };
}

async function main() {
  const root = await fs.promises.mkdtemp(path.join(os.tmpdir(), 'probe-'));
  const file = path.join(root, 'sample.js');
  const adapter = adapterFor(root);
  const sentinel = [
    '// LOAD: timeout must stay under the caller budget',
    'function budget(ms) { return ms > 0; }',
    '',
  ].join('\n');

  const reached = await reach(adapter, file, sentinel);
  if (!reached.ok) {
    console.error('REACH fail', reached);
    process.exit(2);
  }

  let proposal = sentinel.replace(
    'function budget(ms) { return ms > 0; }',
    'function budget(ms) { return Number(ms) > 0; }'
  );
  if (process.env.FAIL_APPLY === '1') {
    proposal = sentinel;
  }
  if (process.env.FAIL_INTENT === '1') {
    proposal = proposal
      .split('\n')
      .filter((line) => !line.includes(LOAD_PREFIX))
      .join('\n');
  }

  const applied = await apply(adapter, file, proposal);
  if (!applied.ok) {
    console.error('APPLY fail', applied);
    process.exit(3);
  }

  const after = await adapter.read(file);
  const judged = intent(sentinel, after);
  if (!judged.ok) {
    console.error('INTENT fail', judged);
    process.exit(4);
  }

  console.log(JSON.stringify({
    reach: reached,
    apply: applied,
    intent: judged,
  }, null, 2));
}

main().catch((err) => {
  console.error('REACH fail', err.message);
  process.exit(2);
});
Enter fullscreen mode Exit fullscreen mode

The reach check writes a sentinel and demands the same bytes from the same absolute path, not from a friendly relative alias. Relative paths can feel successful while the editor is still showing a different copy of the file. The apply check rejects a patch that claims success without changing that sentinel path, because a no-op is not a landing. Would you merge a diff that only edited a twin file in a temporary directory you never opened?

The intent check is the part that connects this probe to a problem older than any assistant. A patch can be reachable, applied, and still wrong if it deletes the comment that carried the real constraint. I mark those lines with a stable prefix and treat deletion as a failed apply, not as a style improvement. A shorter function is not automatically a clearer one, and a vanished warning is not a cleanup.

Why would a second prompt know more than the first if you never checked the path? Create a scratch directory, drop the script beside it, and point the adapter at the server you mean to test. Run it once before you ask the model for anything, so a dead socket cannot disguise itself as a bad prompt. Run it again after the model proposes a patch, and feed that proposal through the intent check before you accept it.

Save the listing as probe.mjs, then run the three commands below and read the exit code before you read the prose of the error. The happy path of this stand-in should exit zero, the no-op patch should exit three, and a deleted load line should exit four. Those codes are a teaching device for the local adapter, not a contract from any hosted product. If your real server uses other status values, map them inside the adapter and leave the three names intact.

mkdir -p "$HOME/tmp/session-probe"
cp probe.mjs "$HOME/tmp/session-probe/probe.mjs"
cd "$HOME/tmp/session-probe"
node probe.mjs
echo "happy_exit=$?"
FAIL_APPLY=1 node probe.mjs
echo "apply_exit=$?"
FAIL_INTENT=1 node probe.mjs
echo "intent_exit=$?"
node -e 'const path=require("path"); console.log(path.resolve("sample.js")); console.log(path.resolve("/tmp/other/sample.js"));'
Enter fullscreen mode Exit fullscreen mode

A reach failure means you should fix the server path, the port, or the credential before you edit a single word of the prompt. An apply failure means the write went somewhere else or nowhere, so retrying the model is just a more expensive shrug. An intent failure means the model was reachable and the file changed, but the constraint you marked did not survive. Only that third case deserves a prompt revision, and even then I change the constraint note before I change the tone.

This probe will not catch a subtle logic bug that preserves the comment and still computes the wrong result. It will not prove that a free option remains free next week, and it will not rank one model against another. It assumes you can isolate a scratch directory and that the server resolves the same path the same way twice. If those assumptions fail, the script is a liar, and you should not promote its green line into a team policy.

You should skip this approach if the work needs retention, SSO, or a private path you have not verified. You should also skip it if you are tuning kernels, measuring latency, or writing a compliance report that needs numbers. I am not giving you those numbers, and borrowing this page as evidence would be a sloppy kind of certainty. A solo debugging session is the right size, and a regulated production change is the wrong room entirely.

So the opening friction is not that the assistant is mysterious, it is that three failures share one face. Split them with a probe you can read, then spend your judgment on the layer that actually broke. If free model access and a free server option would let you try that split on a scratch directory, start there. Wire the adapter to the real write path, and leave the second prompt waiting until the layer has a name.

Top comments (0)