Show the Memory Audit Before the Agent Acts

Scenario: a support agent approved a refund yesterday. It "remembered" a policy from an older chat. That policy belonged to a different product line. The customer got paid. The manager got angry.

We keep trusting agent memory. Agents remember everything. They trust all of it. That gap is the problem.

This tutorial builds a recall audit. It runs on MonkeyCode's free server. It uses the free model access. You will see what the agent actually remembers before it acts.

Disclosure: This article was prepared as part of MonkeyCode's product outreach.

MonkeyCode currently offers free model access with a ten-million-token allowance and a free server option. I use both here. Everything else is a workflow you can verify yourself.

Who owns the decision? The human reviewer. What is the consequence? A confident answer built on stale context. Where is the point of reversibility? Right before the tool call fires. Put the audit there.

Step 0: Install the CLI

npm install -g monkeycode
monkeycode --version

If the version prints, you are ready. If not, use the install command from the official docs. Do not proceed with a broken client. Verification is part of the habit.

Step 1: Start the free server

monkeycode server start --plan free
monkeycode server status

Expect healthy in the status output. You are testing the model now, not your network.

Step 2: Point the client at the server

export MONKEYCODE_ENDPOINT="http://localhost:8787"
export MONKEYCODE_MODEL="default-free"
monkeycode whoami

whoami should return a short ID. Keep this terminal open.

Step 3: Write the recall probe

Here is the core artifact. A small Python script injects five facts. Then it asks ten questions. Five questions test recall. Five are traps that test confabulation.

import json, subprocess

FACTS = [
    "Refund limit is 40 USD per ticket.",
    "Billing window closes at 18:00 UTC.",
    "VIP status requires 12 months of tenure.",
    "Password reset links expire after 15 minutes.",
    "Escalation needs two human approvals.",
]

RECALL = [
    ("What is the refund limit per ticket?", "40 USD"),
    ("When does the billing window close?", "18:00 UTC"),
    ("How long is a password reset link valid?", "15 minutes"),
    ("What does VIP status require?", "12 months"),
    ("How many approvals does escalation need?", "two"),
]

TRAPS = [
    "What is the refund limit for enterprise accounts?",
    "Can one human approve an escalation?",
    "Do reset links expire after 24 hours?",
    "Is the billing window 09:00 UTC?",
    "Is the refund limit 400 USD?",
]

def ask(question):
    prompt = "\n".join(FACTS) + "\n\n" + question
    result = subprocess.run(
        ["monkeycode", "run", prompt],
        capture_output=True, text=True, timeout=60
    )
    return result.stdout.strip()

def audit():
    recall_hits = 0
    trap_failures = 0
    for question, expected in RECALL:
        answer = ask(question)
        if expected.lower() in answer.lower():
            recall_hits += 1
    for question in TRAPS:
        answer = ask(question)
        if "cannot" in answer.lower() or "not provided" in answer.lower():
            continue
        trap_failures += 1
    return recall_hits, trap_failures

if __name__ == "__main__":
    hits, traps = audit()
    report = {
        "recall": hits / len(RECALL),
        "confabulation": traps / len(TRAPS),
        "pass": (hits / len(RECALL)) >= 0.8 and traps == 0,
        "timestamp": __import__("datetime").datetime.now().isoformat(),
    }
    with open("audit.jsonl", "a") as f:
        f.write(json.dumps(report) + "\n")
    print(json.dumps(report, indent=2))

Step 4: Run the audit

python3 recall_audit.py

The output should look like this:

{
  "recall": 1.0,
  "confabulation": 0.0,
  "pass": true,
  "timestamp": "2026-08-29T09:12:00Z"
}

I am not claiming a benchmark. This is a smoke test. It tells you one thing: whether the model kept your facts straight in one conversation. That one thing decides whether the agent gets to act.

Why five facts? Because five is enough to show a pattern and small enough to read in one sitting. Ten questions take about ninety seconds. You can run this before every approval gate without slowing the team down.

What does a failed audit look like? recall drops below 0.8. Or one trap answer slips through. Stop then. Do not approve. Rebuild the context. Re-run. The stop condition is the point.

The human gate

cat audit.jsonl

Show this file before approving the agent's plan. That is the design pattern. Evidence first. Action second. The reviewer's job becomes checking a number. No more re-reading the whole conversation.

The audit file is the interface. Not the chat. Not the plan. A one-line JSON record a reviewer reads in three seconds. Put the evidence exactly at the decision point.

Read it without color, too. The report uses plain text and numbers. A screen reader should land on the pass line first. That is the accessibility check for this pattern.

If the audit fails, hand control back to the human. That is the recovery path. The agent does not get a second try inside the same gate.

Why this matters

Compare this with the common flow. The agent writes a plan. The reviewer reads the plan. The plan sounds confident. Nobody checks the context. Then the agent "remembers" the wrong policy. Sound familiar?

The audit moves evidence to the exact moment of choice. That is where refusal, reversibility, and recovery belong.

What evidence supports this? The pattern comes from retrieval evaluation. RAG systems measure recall and confabulation separately. I simplified that idea into a smoke test. No new research claims. Just a transferable practice.

Inject facts -> Ask 10 questions -> Score recall -> Show audit.jsonl -> Human approves or stops

Limitations

The free server has limits. I will not quote numbers I cannot verify. Expect variable latency at peak hours. The model behind the free endpoint can change without notice. Re-run the audit whenever your results shift.

This approach does not test reasoning. It tests recall. A model can pass this audit and still pick the wrong tool. Do not use it for healthcare, financial, or safety-critical decisions without a human reviewing every action.

Who should skip this? Teams without a reviewer in the loop. If nobody reads the audit, the audit is performance theater. Also skip it if you need guaranteed uptime. Free servers do not promise that.

Start small

Pick one agent. Inject five facts. Ask ten questions. Show the result before the agent acts. That is the whole pattern.

Free model access and the ten-million-token allowance make this cheap to try. The habit is what costs nothing.

Try MonkeyCode's free server this week. Run the audit. Keep the jsonl. Your future self will thank you.