I Don't Trust the LLM That Writes My Shared Memory: 3 Regex Gates Before Claude Code Auto-Commits to My Obsidian Vault

Every 10 minutes, an unattended job has an LLM summarize my AI conversation logs, then commits and pushes the result to my Obsidian Vault. No human reviews it first. The obvious risk: the model's output ends up in git history, and other AI sessions read it later as memory. In the naive version, whatever the model returned went straight into the Vault. Now I treat that output as untrusted input. The model runs with no write tools, and the output goes through the same regex checks at three separate points before anything reaches a commit.

Last time, I wrote about how I misread a quota-limit message as "content too short" and failed three times in a row. This post covers something that comes one step earlier: a design that treats the AI's own output as untrusted input.

The problem: unattended commits of conversation-log summaries to my Vault

Conversation logs from Claude Code and Codex keep piling up under ~/Documents/my-knowledge-base/raw/ (conversations/, codex-conversations/, manus-conversations/). Every 10 minutes a job picks them up, has a model summarize them, writes the summary back to AI/SESSION-STATE.md in my Obsidian Vault, and runs git commit and push. Nobody checks it along the way.

The scariest part is that the LLM doing the summarizing can itself return untrustworthy output. A conversation log might contain an API key or auth URL I pasted in. It might also contain prompt-injection text that came in through another AI or a web page. If I write that into the Vault without checking it, secrets end up in git history, or the next AI session that reads the Vault follows fake instructions.

The previous post was about misreading a quota-limit message as real body text, but the root cause is the same: if you treat whatever text an AI returns as meaningful content by default, things break. This post covers the countermeasures: a sandbox on the generation side and a regex gate on the receiving side.

Overview: isolate generation, distrust everything on receipt

The pipeline is two files: memory-reflect.sh (a bash script launched by launchd) and normalize-memory-reflection.py (the receiving gate).

Layer	What it does	Owner
Generation	Have a maximally de-privileged Claude produce the summary	memory-reflect.sh
Validation	Re-validate the returned JSON without trusting any of it	normalize-memory-reflection.py
Double check	Re-scan the written files and the git diff with regex	memory-reflect.sh (second half)

Note: The key is not to assume that restricting the generation side makes things safe. Even with the tools limited to --tools Read,Grep,Glob, you can't control the content of the text the model returns. So the receiving side validates it again, zero-trust.

Layer 1: Summarize with a Claude that has no privileges

Here's the invocation:

"$TIMEOUT_BIN" "$MODEL_TIMEOUT_SECONDS" "$CLAUDE_BIN" --print --no-session-persistence --model sonnet \
  --setting-sources project \
  --strict-mcp-config --mcp-config '{"mcpServers":{}}' \
  --tools 'Read,Grep,Glob' \
  --output-format json --json-schema "$SCHEMA" \
  --system-prompt 'You are a read-only structured memory extractor. Follow the user prompt exactly and return only schema-compliant JSON.' \
  "$PROMPT" > "$RESULT" 2>>"$LOG"

• --tools 'Read,Grep,Glob': no Bash, Edit, Write, or MCP. The model can't write files and can't reach outside.
• --strict-mcp-config --mcp-config '{"mcpServers":{}}': forces MCP server connections to zero in code, instead of relying on me having cleaned up my config.
• --output-format json --json-schema "$SCHEMA": no free-form text. If the output doesn't match the schema, the call itself fails.
• timeout 120 (MODEL_TIMEOUT_SECONDS): a wall-clock cap in case it hangs.

Here's the schema (reformatted for readability):

{
  "type": "object",
  "additionalProperties": false,
  "required": ["session_state", "proposals", "sources_reviewed", "limitations"],
  "properties": {
    "session_state": { "type": "string", "maxLength": 5000 },
    "proposals": {
      "type": "array",
      "maxItems": 8,
      "items": {
        "type": "object",
        "additionalProperties": false,
        "required": ["kind", "title", "content", "sources", "confidence", "expiry"],
        "properties": {
          "kind": { "enum": ["durable-candidate", "provisional-candidate", "wiki-update-candidate", "do-not-store"] },
          "confidence": { "enum": ["high", "medium", "low"] }
        }
      }
    },
    "sources_reviewed": { "type": "array", "maxItems": 16 },
    "limitations": { "type": "array", "maxItems": 12 }
  }
}

The prompt adds a second layer of instructions:

Never include or paraphrase: credentials, API keys, tokens, passwords, cookies,
auth URLs/codes, financial account or card data, medical detail, direct contact
data, private messages, meeting links, detailed private schedules, unconfirmed
legal/financial terms, or unverified allegations.

However, this is only a request to the model. Telling it "don't write secrets" in the prompt doesn't guarantee it will comply. That's why the layers that follow do the real work.

If the call fails, the script tries a local Qwen fallback (ollama-memory-json.py). If that also fails, it writes {} and treats the result as empty. This is an unattended job, so I'd rather it fail safely with no output than halt when something breaks.

Layer 2: The regex gate, zero-trust validation of model output

The docstring of normalize-memory-reflection.py says:

"""Normalize untrusted local LLM reflection output into a safe schema.

On malformed or unsafe output, emit a provenance-only fallback. The fallback
never summarizes transcript contents and therefore cannot promote invented or
sensitive details into the shared session state.
"""

It states outright that LLM output is untrusted. If validation fails, it returns a fallback that never summarizes anything from the transcripts. Most of the work happens in three stages.

① Don't trust JSON parsing itself

def unique_object(pairs: list[tuple[str, Any]]) -> dict[str, Any]:
    result: dict[str, Any] = {}
    for key, value in pairs:
        if key in result:
            raise ReflectionError("duplicate-json-key")
        result[key] = value
    return result

With object_pairs_hook=unique_object, any JSON object with duplicate keys is rejected. The standard json.loads silently keeps the last value, so this closes off parser-differential attacks where something like {"kind":"do-not-store","kind":"durable-candidate"} slips past the check.

Envelope unwrapping is tightly restricted here too:

def unwrap_structured_output(value: Any, depth: int = 0) -> Any:
    ...
    for key in ("structured_output", "result"):
        nested = value.get(key)
        if isinstance(nested, dict):
            return unwrap_structured_output(nested, depth + 1)
        if isinstance(nested, str):
            match = FENCED_JSON_RE.fullmatch(nested.strip())
            # A result string may be exactly JSON or a single explicit JSON fence.
            # Do not scrape an object embedded in arbitrary untrusted prose.
            return unwrap_structured_output(parse_json_text(match.group(1) if match else nested), depth + 1)
    raise ReflectionError("structured-output-missing")

As the comment says, it deliberately does not fish JSON-looking fragments out of surrounding prose, where prompt injection could have planted them. It only accepts the --output-format json CLI envelope (the structured_output/result keys) or exactly one Markdown JSON fence. Nesting is cut off at depth > 3.

② Reject content with regex

SECRET_RE = re.compile(r"(?i)(?:\bsk-[A-Za-z0-9_-]{16,}\b|\b(?:api[_ -]?key|password|secret|bearer|token)\s*[:=]\s*[^\s`'\"]{8,})")
MEETING_RE = re.compile(r"https?://(?:meet\.google\.com|zoom\.us/j|teams\.microsoft\.com)/[^\s)\]>]+", re.I)
AUTH_LINK_RE = re.compile(r"https?://[^\s)\]>]*(?:oauth|auth|signin|login)[^\s)\]>]*|https?://[^\s)\]>]*[?&](?:code|token|access_token|id_token)=[^\s&#)\]>]+", re.I)
FINANCIAL_RE = re.compile(r"(?i)\b(?:card\s*number|credit\s*card|bank\s*account|routing\s*number|iban)\b")
HEALTH_RE = re.compile(r"(?i)\b(?:medical\s+record|diagnosis|medication|patient\s+detail|health\s+condition)\b")
CONTACT_RE = re.compile(r"(?i)\b[A-Z0-9._%+-]+@[A-Z0-9.-]+\.[A-Z]{2,}\b|\b(?:\+?\d[\d .()/-]{7,}\d)\b")
INSTRUCTION_RE = re.compile(
    r"(?i)\b(?:ignore|disregard|override)\b.{0,80}\b(?:previous|prior|system|developer)\b"
    r"\b(?:instructions?)\b"
    r"|\b(?:system\s+prompt|developer\s+message|jailbreak)\b"
)

The interesting part is how CONTACT_RE is used. A regex that rejects phone-number-like digit sequences will almost certainly flag dates like 2026-09-23 too. So the actual check looks like this:

def is_safe(value: str) -> bool:
    contact_candidate = ISO_DATE_RE.sub("[DATE]", value)
    if CONTACT_RE.search(contact_candidate):
        return False
    return not any(pattern.search(value) for pattern in (
        SECRET_RE, MEETING_RE, AUTH_LINK_RE, FINANCIAL_RE, HEALTH_RE, INSTRUCTION_RE,
    ))

Replacing ISO dates with [DATE] before the phone-number check prevents those false positives. INSTRUCTION_RE targets the stock phrases of prompt injection, like "ignore previous instructions", "system prompt", and "jailbreak". It's there so that even if instructions injected in another session show up in a conversation log, they never get adopted as session_state.

③ Restrict citations to the list of files actually provided

raw_reviewed = candidate["sources_reviewed"]
...
# The model may only cite the source manifest generated by the wrapper.
if any(item not in sources for item in raw_reviewed):
    raise ReflectionError("sources-reviewed-not-in-manifest")

sources is the file list that memory-reflect.sh built with find and passed in as an argument (SOURCE_LIST). If the model claims "I also read this file" and that file isn't in the list, the output is rejected immediately. Each proposal's sources must likewise be a subset of the files provided. This closes off the trick where the model fabricates sources to sneak its own "facts" in as durable-candidates.

If validation fails, say nothing

def fallback(sources: list[str], reason: str) -> dict[str, Any]:
    return {
        "status": "fallback",
        "session_state": (
            "## Automatic archive update\n\n"
            f"- {len(sources)}件の新しいlocal transcriptを取得した。"
            "structured reflectionの出力が検証できなかったため、内容は共有memoryへ昇格していない。\n"
            ...
        ),
        "proposals": [],
        ...
    }

(The Japanese strings read roughly: "Fetched N new local transcripts. The structured reflection output could not be validated, so its content was not promoted to shared memory.")

When validation fails, the fallback session_state only says how many transcripts were fetched and summarizes nothing from their contents. The safe option here isn't to produce a harmless-looking summary. It's to say nothing about the content at all.

Layer 3: Scan the files and the git diff again

Even after normalization passes, memory-reflect.sh runs two more regex checks of the same kind.

# Last line of defense: never copy likely secret values to shared state.
if grep -E -q -i '(api[_ -]?key|password|secret|bearer|token)[[:space:]]*[:=][[:space:]]*[^[:space:]]{8,}' "$CANDIDATE"; then
  echo 'reflection blocked: secret-like value in candidate state'
  exit 1
fi
mv "$CANDIDATE" "$STATE_FILE"

This grep runs separately on the candidate file for SESSION-STATE.md and on the proposal file for AI/INBOX/auto/. Then, right before committing, it re-scans the staged diff itself:

git add -- "$STATE_FILE" "$INBOX_FILE"
git diff --cached --check -- "$STATE_FILE" "$INBOX_FILE"
git diff --cached -- "$STATE_FILE" "$INBOX_FILE" | grep -E -i '(api[_ -]?key|password|secret|bearer|token)[[:space:]]*[:=][[:space:]]*[^[:space:]]{8,}' >/dev/null && {
  echo 'reflection blocked: secret-like value in staged diff'
  git restore --staged -- "$STATE_FILE" "$INBOX_FILE" || true
  exit 1
}
if ! git diff --cached --quiet -- "$STATE_FILE" "$INBOX_FILE"; then
  git commit --only -m "chore(memory): reflect local sessions" -- "$STATE_FILE" "$INBOX_FILE" >/dev/null
  git push origin main >/dev/null
fi

Running the same regex three times (inside the normalizer, on the candidate file, and on the staged diff) looks redundant, but each check sees different input. The normalizer sees the model's raw JSON. The second check sees the file after it's been rendered as Markdown. The third sees the exact diff going into git. Formatting and Markdown conversion can change how escapes and newlines are handled, which could let something slip past the first check. So the last line of defense has to look at the actual bytes being committed.

git commit --only matters too. I also edit this Vault repo by hand, so the job might run while I have other changes that I forgot to stage. --only limits the commit to the two files added in this run, so unrelated changes never get swept in.

The launchd configuration

<key>StartInterval</key>
<integer>600</integer>
<key>ThrottleInterval</key>
<integer>60</integer>
<key>LowPriorityIO</key>
<true/>
<key>Nice</key>
<integer>12</integer>
<key>EnvironmentVariables</key>
<dict>
  <key>MEMORY_REFLECT_MIN_INTERVAL_SECONDS</key>
  <string>300</string>
</dict>

StartInterval=600 launches the job every 10 minutes. Two guards on the script side prevent overlapping runs even if a previous one takes longer: a debounce (MIN_INTERVAL, overridden to 300 seconds in the plist) and a lock directory (if mkdir "$LOCK" fails, the run skips immediately). LowPriorityIO and Nice 12 keep this background job from taking CPU and I/O away from foreground work.

Pitfalls I hit

Here are the problems I actually ran into, as recorded in the code comments:

• The phone-number regex flagged dates as false positives → replace dates with [DATE] via ISO_DATE_RE before applying CONTACT_RE
• Duplicate JSON keys could bypass the check → reject duplicate keys as soon as they're found, using object_pairs_hook
• Trusting the model's own claims about its sources isn't validation → force sources_reviewed/proposal.sources to be a subset of the files actually provided
• ValueError was only caught after ad-hoc extraction paths, so the script could crash partway through → this past bug is noted in the code. It's now fixed: errors are normalized to ReflectionError, which returns a deterministic failure code
• With the regex check in only one place, things could slip through during formatting or Markdown conversion → repeat the same check in three places: normalizer output, the written files, and the staged diff
• Commits could sweep in unrelated changes → git commit --only limits the commit to the two files

Summary

• On the generation side, --tools Read,Grep,Glob, disabled MCP, and an enforced JSON Schema mean the model has no write capability at all.
• On the receiving side, output is treated as fully untrusted and validated in three stages: hardened JSON parsing, regex detection of secrets, contact info, and prompt injection, and restricting citations to the manifest.
• When validation fails, the fallback chooses to summarize nothing, erring on the side of safety.
• The same regex checks run again when writing the files and right before the git commit. Validation shouldn't stop at one checkpoint.
• git commit --only keeps the unattended job from sweeping up a human's uncommitted changes.

Next time, I plan to write about what happens after SESSION-STATE.md and the INBOX proposals pass these layers: how a human actually reviews them and promotes them into durable memory.

If you let an LLM write into shared memory or a repo unattended, where do you put your trust boundary: on the generation side, the receiving side, or right before the commit?

---

Written by **Lily* — I ship iOS apps and automate my content stack with Claude Code.
Follow along: Portfolio · X · GitHub*