Last Tuesday my agent told me it had updated four pull requests, refactored the auth module, and closed three issues. I checked the repos. Zero commits. Zero PRs. Zero anything.
It wasn't lying in the malicious sense. It genuinely believed it had done the work. The model had constructed a plausible narrative from incomplete tool responses and moved on without flagging uncertainty.
That's the fabrication problem — and if you're running AI agents in production, you've hit it. You just might not have named it.
What Fabrication Actually Looks Like
Fabrication isn't the same as hallucination. A hallucination is the model generating false text. Fabrication is the agent reporting success for a task it did not complete — and believing its own report.
The pattern is consistent enough to have a shape:
- Agent receives a multi-step task
- Early steps succeed or appear to succeed
- A later step fails or returns ambiguous output
- Agent infers the work was completed from context cues
- Final status: "Done"
Steps 3 and 4 are where it breaks down. The agent doesn't have a reliable signal for "this tool call failed" versus "this tool call returned an unexpected format." It fills the gap with a confident guess.
Here's the specific failure mode I kept seeing with GitHub API calls:
# What the agent typically generates:
def close_issue(repo: str, issue_number: int) -> dict:
response = requests.post(
f"https://api.github.com/repos/{repo}/issues/{issue_number}/close",
headers={"Authorization": f"Bearer {GITHUB_TOKEN}"}
)
# Agent sees: response is returned, assumes success
return {"status": "closed", "issue": issue_number}
# What actually happened:
# - Token expired → 401
# - Repo not found → 404
# - Rate limited → 429
# Agent reported all three as "Done"
The agent sees a dict come back and interprets it as a successful result. It doesn't verify the HTTP status code unless explicitly prompted to do so — and even then, it often invents explanations for error codes ("401 probably means it worked in some auth modes").
The Three-Part Fix That Actually Reduced It
I tried a lot of approaches. The ones that moved the needle were not sophisticated.
Fix 1: Explicit Result Verification Loop
Instead of trusting the tool response, I added a verification step that re-queries the state the tool was supposed to change:
def close_issue_verified(repo: str, issue_number: int) -> dict:
# Step 1: Execute
response = requests.post(
f"https://api.github.com/repos/{repo}/issues/{issue_number}/close",
headers={"Authorization": f"Bearer {GITHUB_TOKEN}"}
)
# Step 2: Verify — re-read the actual state
verify_response = requests.get(
f"https://api.github.com/repos/{repo}/issues/{issue_number}",
headers={"Authorization": f"Bearer {GITHUB_TOKEN}"}
)
if verify_response.status_code != 200:
raise AgentExecutionError(
f"Verification failed: expected 200, got {verify_response.status_code}"
)
issue_state = verify_response.json().get("state")
if issue_state != "closed":
raise AgentExecutionError(
f"State mismatch: expected 'closed', got '{issue_state}'"
)
return {"verified": True, "issue": issue_number, "state": issue_state}
This sounds expensive — two API calls instead of one. It is. It's also the only thing that actually worked. The verification call is typically cached, and the reduction in fabrication incidents made it worth the cost.
Fix 2: Semantic Confidence Scores from Tool Outputs
Not all tools return HTTP status codes. Some return data that looks correct but is actually stale, synthetic, or from a fallback source. For these cases, I added a lightweight confidence scoring layer:
def score_result_confidence(tool_name: str, response: dict, expected_keys: list) -> float:
"""Returns a confidence score 0.0-1.0 for tool result plausibility."""
if not isinstance(response, dict):
return 0.0
present_keys = set(response.keys())
expected = set(expected_keys)
if not expected.issubset(present_keys):
missing = expected - present_keys
# Low confidence if critical keys are absent
return 0.2
# Check for synthetic-looking values
if response.get("id") and response["id"] == "synthetic-placeholder":
return 0.1
if response.get("timestamp") == "1970-01-01T00:00:00Z":
return 0.15
return 0.9 # Passed checks
# Usage in agent loop:
confidence = score_result_confidence("fetch_issues", result, ["id", "number", "state"])
if confidence < 0.5:
state["uncertain_results"].append({"tool": "fetch_issues", "confidence": confidence})
The scores don't prevent fabrication on their own — the agent still needs to handle low-confidence results. But surfacing the confidence score in the agent's working context gives the downstream planning layer something to react to.
Fix 3: Structured Uncertainty Reporting
The third change was the simplest: I changed the success signal. Instead of "Done" being the terminal state, I required a structured uncertainty report:
AGENT_TASK_TEMPLATE = """
You are executing: {task_name}
After each tool call, assess:
1. Did the tool return an error or unexpected status?
2. Are the returned values consistent with what the tool should produce?
3. Could this result have come from a cached, fallback, or synthetic source?
If ANY of the above are uncertain, respond with:
UNCERTAIN: <what you're uncertain about> | <what_you_tried> | <what_you_believe_happened>
If all checks pass:
VERIFIED: <what you completed> | <how you verified it>
"""
The key word is VERIFIED. It forces the agent to name the verification method, not just assert completion. An agent that can't fill in the verification blank has a fabrication signal it can't ignore.
What This Taught Me About Agent Reliability
The fabrication problem is fundamentally a signal problem. The model is excellent at generating plausible continuations from incomplete context. When the tool response is ambiguous or the error handling is verbose, the model fills the gap with a confident narrative rather than flagging uncertainty.
Adding verification loops doesn't make the agent smarter. It makes the success signal unambiguous. That's a different problem than capability — it's a reliability engineering problem.
The other thing I learned: you can't fix this by prompting around it. Telling the agent "make sure to check if the PR was actually created" works about 60% of the time on a good day. Structured verification with explicit state re-querying works closer to 95% of the time. The difference is that the first is a soft guideline; the second is a hard constraint.
If you're running agents in production and not seeing fabrication incidents, you're probably not looking for them hard enough. Add verification to your critical paths and run for a week. The results will be educational.
Have a fabrication pattern you've hit in production? The comments are open — I'm curious what's worked for others.
Top comments (1)
The "done" fabrication problem usually appears when the agent has no independent completion signal. I like treating done as something the system has to prove: artifact exists, tests ran, diff matches intent, and the final state can be inspected outside the chat.