The Manifest Was Valid. My AI Agent Was Still Running Yesterday's Plan.

A publishing run can pass every schema check and still execute the wrong plan. In my case, the cause was one stale JSON file in the publishing agent.

The failure looked impossible at first. The agent printed the correct task ID. The browser opened the correct platform. Every validation check returned green. Yet the run kept using yesterday's article title, yesterday's image count, and yesterday's publish target. Nothing crashed, so the logs offered no obvious starting point.

The bug was not in the browser automation. It was not a race in the state machine. It was not even a malformed manifest. The file was syntactically valid, passed its schema check, and contained values that had once been correct. My loader had answered the wrong question: "Can I parse this file?" What I needed to ask was: "Does this file belong to this run?"

This post is the design I now use to keep task manifests fresh, attributable, and safe before an agent performs any side effect.

The failure in numbers

A simplified version of the diagnostic log looked deceptively healthy:

task_id: devto-20260801-02
manifest: C:\agent\runtime\publish.json
schema: valid
editor: ready
article_title: How I Fixed an MCP Timeout
expected_title: The Manifest Was Valid...

Five checks passed. One value was wrong. Because my log grouped "manifest loaded" and "manifest belongs to this task" into the same green status, the mismatch disappeared inside a successful-looking startup sequence.

The stale file had survived from a prior run. A cleanup step had failed quietly when another process still held the file. The next run found the path, parsed the JSON, and moved on. The loader treated existence as freshness.

That assumption is common in agent pipelines because manifests feel like configuration. In practice, they are messages between stages. A planner writes intent; a worker reads it later. Once you see the file as a message, identity and age become part of correctness.

Why modification time was not enough

My first fix checked the file's modification time. It looked reasonable:

from pathlib import Path
from time import time

def is_recent(path: Path, max_age_seconds: int = 900) -> bool:
    age = time() - path.stat().st_mtime
    return 0 <= age <= max_age_seconds

This stopped a manifest from last week. It did not stop a manifest from ten minutes ago that belonged to a different task. It also created edge cases after clock changes, file restoration, and copy operations that preserved timestamps.

Freshness is not one dimension. At minimum, the loader needs to verify identity, creation context, expected target, and age. Time is useful as a final expiry rule, not as the primary proof that two stages are talking about the same run.

Put identity inside the manifest

The useful fix began by assigning each run an immutable ID before any content or browser work started. Every artifact created by that run carries the same ID.

{
  "schema_version": 2,
  "run_id": "20260801T174210Z-6f3a9c2e",
  "created_at": "2026-08-01T17:42:10Z",
  "producer": "topic-and-draft@2.4.1",
  "target": "hashnode",
  "content_sha256": "f6d1b8...91c4",
  "title": "The Manifest Was Valid. My AI Agent Was Still Running Yesterday's Plan.",
  "body_path": "runtime/20260801T174210Z-6f3a9c2e/article.md"
}

The worker receives the expected run_id from its caller. It never discovers the active run by scanning for the newest file. "Newest" is a guess; a run ID is a contract.

I also stopped reusing one global publish.json. Each run gets its own directory. That decision removes most accidental cross-run collisions and makes cleanup safer because the target is narrow and explicit.

Verify the manifest before opening the browser

The order matters. My old pipeline launched the browser, navigated to the editor, and then read the manifest. When validation failed, it left a half-started browser session and sometimes an autosaved blank draft.

Now the worker performs a pure preflight first:

from dataclasses import dataclass
from datetime import datetime, timezone
from hashlib import sha256
from pathlib import Path
import json

@dataclass(frozen=True)
class PublishIntent:
    run_id: str
    target: str
    title: str
    body_path: Path

def load_intent(manifest_path: Path, expected_run_id: str) -> PublishIntent:
    raw = json.loads(manifest_path.read_text(encoding="utf-8"))

    if raw.get("schema_version") != 2:
        raise ValueError("unsupported manifest schema")
    if raw.get("run_id") != expected_run_id:
        raise ValueError("manifest belongs to another run")
    if raw.get("target") != "hashnode":
        raise ValueError("wrong publish target")

    created = datetime.fromisoformat(raw["created_at"].replace("Z", "+00:00"))
    age = (datetime.now(timezone.utc) - created).total_seconds()
    if age < 0 or age > 1800:
        raise ValueError("manifest expired")

    body_path = Path(raw["body_path"]).resolve()
    body = body_path.read_bytes()
    if sha256(body).hexdigest() != raw["content_sha256"]:
        raise ValueError("article content changed after planning")

    return PublishIntent(raw["run_id"], raw["target"], raw["title"], body_path)

Only after this function returns does the pipeline start a browser session. That one ordering change removed orphan tabs, empty drafts, and misleading screenshots from failed preflights.

Write manifests atomically

Identity checks do not help if the worker can read a file while the planner is still writing it. JSON parsers usually catch truncated files, but a partially updated file can still be valid JSON if the writer overwrites fields in place.

The planner now writes to a sibling temporary file, flushes it, and replaces the final path in one operation:

import json
import os
from pathlib import Path

def atomic_json_write(path: Path, payload: dict) -> None:
    temp = path.with_suffix(path.suffix + ".tmp")
    with temp.open("w", encoding="utf-8", newline="\n") as handle:
        json.dump(payload, handle, ensure_ascii=False, indent=2)
        handle.flush()
        os.fsync(handle.fileno())
    os.replace(temp, path)

The worker never watches the temporary filename. It waits for the final path, then validates the entire contract once. There is no polling loop that repeatedly reads a moving target.

On Windows, os.replace can fail when another process holds the destination. I do not hide that with ten retries. The planner makes one short retry after a small randomized delay, then stops with the exact path and owning stage in the error. Repeated retries turn a clear lock problem into a long, expensive mystery.

Bind browser actions to the same run

The manifest can be correct at startup and still become detached from later browser work. A worker might resume an old editor tab or continue after the user switches drafts.

I bind the run identity to browser state in two places. First, the title and target are checked immediately after injection. Second, the worker writes a local checkpoint only after the editor reflects the expected values.

{
  "run_id": "20260801T174210Z-6f3a9c2e",
  "state": "editor_verified",
  "target": "hashnode",
  "observed_title": "The Manifest Was Valid. My AI Agent Was Still Running Yesterday's Plan.",
  "body_word_count": 1638,
  "checked_at": "2026-08-01T17:44:03Z"
}

The publish step requires that checkpoint and checks the run_id again. This is not a second source of truth. It is evidence that the intended content reached the intended interface before a destructive action.

If the title, body count, target, or run ID differs, the worker stops before clicking Publish. It does not try to repair the page by appending missing text or switching to another open tab.

Reject implicit recovery paths

The worst version of this bug was made possible by "helpful" fallbacks. If the requested manifest was missing, my loader searched the runtime folder for the newest JSON file. If the article body path was absent, it searched for the newest Markdown file. Those branches kept demos moving and made production runs untraceable.

I removed both. The worker now has one input path and one expected run ID. A missing file is a missing file. A mismatched ID is a hard failure. A stale file is expired. Those errors are cheap because they happen before network calls or browser actions.

This is the important design principle: recovery should restore the same intended run, not quietly select a different run that happens to have usable files.

Test the contract, not only the parser

My original tests proved that valid JSON could be read. The new tests prove that invalid relationships are rejected.

import pytest

def test_rejects_manifest_from_previous_run(tmp_path):
    manifest = make_manifest(tmp_path, run_id="run-old")
    with pytest.raises(ValueError, match="another run"):
        load_intent(manifest, expected_run_id="run-new")

def test_rejects_body_modified_after_planning(tmp_path):
    manifest = make_manifest(tmp_path, run_id="run-1")
    body = tmp_path / "article.md"
    body.write_text("changed after the hash", encoding="utf-8")
    with pytest.raises(ValueError, match="content changed"):
        load_intent(manifest, expected_run_id="run-1")

def test_rejects_wrong_platform(tmp_path):
    manifest = make_manifest(tmp_path, run_id="run-1", target="devto")
    with pytest.raises(ValueError, match="wrong publish target"):
        load_intent(manifest, expected_run_id="run-1")

I also keep one integration test that writes a manifest atomically, loads it, injects a local fake editor, and verifies a checkpoint. It never publishes. The production smoke test is the real public URL check after a controlled run.

What changed after repeated failure tests

I tested this version across successful publishes, deliberate validation failures, locked files, killed worker processes, and edited article bodies. The important result was not that every run published. Some were supposed to fail.

The result was that every run stopped in the correct state. A stale manifest failed before browser launch. A modified body failed before editor injection. A locked destination failed in the planner with a named path. A killed worker left a run-scoped directory that could be inspected without contaminating the next run.

The preflight stays local and completes before any network or browser work begins. More importantly, failed runs became much easier to understand. The logs no longer said only "manifest valid." They said which contract was checked and which run owned the evidence.

The compact rule set

If your agent passes work between stages through files, the safe pattern is small:

1. Create the run ID before producing artifacts.
2. Put that ID, the target, creation time, schema version, and content hash in the manifest.
3. Store artifacts in a directory named for that run.
4. Write the manifest atomically.
5. Validate everything before opening a browser or calling an external API.
6. Check the run ID again at the pre-publish checkpoint.
7. Never recover by selecting the newest unrelated artifact.

My stale manifest was valid JSON. That fact had almost no operational value. Correct agent automation depends on provenance: who produced this input, for which run, for which target, and whether the content still matches the plan.

Once the loader started answering those questions, yesterday's plan finally stayed in yesterday's folder.

What about you?

Have you ever had a valid manifest point an agent at the wrong run?

---

Originally published on Dispatch.