Prompt Injection in Assistant Tools Is a Boundary Bug

In one recent Reddit thread, a Claude user said the assistant surfaced internal-looking Apify and Notion tool JSON, then warned about prompt injection after the user pasted job-board text. Treat that as a third-party report, not a vendor-confirmed root cause. Still, the symptom is the lesson: if hidden tool state can show up in chat, hidden state is close enough to the model, UI, logs, or memory path to deserve a boundary review.

The reported incident is useful as a boundary-design signal, not as vendor-confirmed root cause.

Prompt injection in assistant tools is not mainly an input-filtering problem. It is a context-boundary problem. The practical fix starts by making every piece of assistant state carry three labels before it reaches a model or a tool: source, visibility, and authority.

That sounds boring. Good. Boring boundaries are easier to test than vibes.

Prompt injection in assistant tools is a boundary bug

Hidden tool or context leakage happens when internal control state, tool schema, tool output, memory, or client-only metadata crosses into model-visible content or user-visible output without an explicit source, visibility, and authority boundary.

That definition matters because "the prompt" is too small a word for modern assistants. A tool-using assistant may combine user text, browser content, retrieved files, memory summaries, tool manifests, tool outputs, guardrail annotations, system instructions, signed file URLs, and UI metadata before the user sees one response.

The Reddit report is a good opening signal because it is messy. It includes user-visible internal-looking state and a warning that may have been a false positive. The point is not "this one vendor leaked exactly X for exactly Y reason." The point is that visible boundary confusion is a security smell.

In a production assistant, the same class of confusion can appear in four places:

• A model sees implementation metadata and treats it as instruction.
• A user sees internal state and loses trust in the product boundary.
• A tool receives stale or injected arguments.
• A memory system stores a contaminated summary for later turns.

Once you look at prompt injection through that lens, a regex over user input starts to feel like a smoke alarm in a building with no fire doors.

Stop calling every string "the prompt"

A model does not care where a string came from; your application has to.

OWASP's LLM01 guidance describes prompt injection as direct or indirect input that alters model behavior, including content that may be imperceptible to humans but still parsed by the model. That is the part assistant builders should keep in their heads: the model does not care whether a string came from a chat box, a README, a website, a tool result, or a hidden field.

Your application has to care.

Plane	Examples	Boundary failure
User-visible data	Chat text, pasted job posts, README files, uploaded PDFs	Data gets treated as privileged instruction
Model-visible control	System prompts, developer instructions, tool descriptions, guardrail policy text	Hidden control state leaks into normal output
Tool and action surface	Shell commands, browser actions, MCP tools, file writes, external sends	Untrusted or stale text triggers a real action
Client-only UI state	Signed URLs, widget metadata, progress state, internal JSON, _meta fields	Implementation detail enters chat, logs, model context, or memory

The table is simple because the failure is simple: a string crosses a lane without being reclassified.

The hard cases are not the obvious malicious prompts. They are ordinary product states that accidentally become model-visible. A progress object contains _meta.resourceUri. A file preview carries a signed URL. A tool result includes a debug blob. A memory summary says "the user prefers automatic setup recovery" after one rushed conversation. None of these is evil. Each one can become unsafe if it graduates into instruction or tool authority.

This is the operational rule I use: every context block needs a source, a visibility level, and an authority level before it can move.

A boundary-first assistant keeps user data, model control, tool action, and client-only UI state in separate lanes.

Memory makes continuity useful and state larger

Memory is valuable because it reduces restart cost. OpenAI describes memory as shared context that helps future conversations start from preferences, projects, and constraints rather than from scratch, while also naming freshness, relevance, staleness, correctness, and scale as engineering challenges.

That is not an argument against memory. It is an argument for classifying memory as state.

Here is a concrete failure mode. A developer asks an assistant to fix a Node repo. In a previous project, the user said, "If setup fails, run npm install and try again." A memory system compresses that into "user prefers automatic setup recovery." Weeks later, the assistant opens a different repo where install scripts are not trusted. If that memory has no source, freshness, or authority label, the product has a quiet policy bug.

The same problem shows up with project preferences:

• "Always use production data for realistic tests" might be fine in one sandbox and reckless in another.
• "Send status updates to the team channel" might be useful for a demo and a data leak in a customer workspace.
• "Ignore linter warnings for generated code" might be acceptable in one package and fatal in a shared library.

Memory improves continuity by expanding state. Expanded state needs labels.

Tools turn boundary mistakes into actions

With a chat-only assistant, prompt injection can produce a bad answer. With tools, the blast radius changes.

The Model Context Protocol security guidance names implementation risks such as confused deputy problems, token passthrough, SSRF, session hijacking, local server compromise, OAuth validation, and scope minimization. Those are not copywriting concerns. They are what happens when a language model sits near credentials, network access, files, and long-lived sessions.

The 0DIN Claude Code proof of concept is a sharper example. The researcher described a normal-looking repo flow where setup recovery and external lookup chained into compromise. Do not copy that attack path into your app or your tests. Take the design lesson instead: routine developer actions become dangerous when untrusted project content can steer trusted tools.

This is why "detect prompt injection" is the wrong primary promise. Detection helps. It is a tripwire. The boundary is enforced by what content may enter model context and what a tool is allowed to do after the model proposes an action.

For tool-calling assistants, I would rather block one helpful but ambiguous action than silently convert untrusted text into shell, network, credential, file-write, or external-send authority.

A small context contract you can test

Here is a compact TypeScript pattern you can adapt. It does not solve prompt injection. It does something narrower and more useful: it prevents the wrong category of text from reaching the wrong authority level.

Save this as context-boundary.ts and run it with tsx context-boundary.ts.

type Source = "user" | "external" | "tool" | "memory" | "system" | "ui";
type Visibility = "user_visible" | "model_visible" | "tool_only" | "client_only";
type Authority = "data" | "instruction" | "tool_request" | "privileged";

type ContextBlock = {
  id: string;
  source: Source;
  visibility: Visibility;
  authority: Authority;
  content: string;
  createdAt: string;
  ttlSeconds?: number;
};

type Finding = {
  level: "error" | "warn";
  blockId: string;
  message: string;
};

const INTERNAL_PATTERNS = [
  /\btool_schema\b/i,
  /\bfunction_call\b/i,
  /\bApify\b/i,
  /\bNotion\b/i,
  /\b_meta\b/i,
  /\bsystem prompt\b/i,
  /\bBearer\s+[A-Za-z0-9._-]+/i,
  /https:\/\/[^\s]+X-Amz-Signature=/i,
];

function isExpired(block: ContextBlock, now = Date.now()): boolean {
  if (!block.ttlSeconds) return false;
  const created = Date.parse(block.createdAt);
  return Number.isFinite(created) && now - created > block.ttlSeconds * 1000;
}

function validateForModel(blocks: ContextBlock[]): Finding[] {
  const findings: Finding[] = [];

  for (const block of blocks) {
    if (block.visibility === "client_only") {
      findings.push({
        level: "error",
        blockId: block.id,
        message: "Client-only state must never enter model-visible context.",
      });
    }

    if (block.source === "user" || block.source === "external") {
      const leakedInternal = INTERNAL_PATTERNS.some((pattern) =>
        pattern.test(block.content),
      );

      if (leakedInternal) {
        findings.push({
          level: "error",
          blockId: block.id,
          message: "Untrusted content contains internal-looking tool or secret material.",
        });
      }
    }

    if (block.source === "memory" && !block.ttlSeconds) {
      findings.push({
        level: "warn",
        blockId: block.id,
        message: "Memory block has no freshness metadata.",
      });
    }

    if (block.source === "memory" && isExpired(block)) {
      findings.push({
        level: "warn",
        blockId: block.id,
        message: "Memory block is stale and should be re-confirmed.",
      });
    }

    if (block.authority === "privileged" && block.source !== "system") {
      findings.push({
        level: "error",
        blockId: block.id,
        message: "Only system-owned blocks may carry privileged authority.",
      });
    }
  }

  return findings;
}

function buildModelContext(blocks: ContextBlock[]): string {
  const findings = validateForModel(blocks);
  const errors = findings.filter((finding) => finding.level === "error");

  if (errors.length > 0) {
    throw new Error(errors.map((error) => `${error.blockId}: ${error.message}`).join("\n"));
  }

  return blocks
    .filter((block) => block.visibility === "model_visible")
    .map((block) => `[${block.source}:${block.authority}] ${block.content}`)
    .join("\n\n");
}

type ToolCall = {
  name: string;
  args: Record<string, unknown>;
  action: "read" | "shell" | "network" | "credential" | "file_write" | "external_send";
  reason: string;
};

const APPROVAL_REQUIRED = new Set<ToolCall["action"]>([
  "shell",
  "network",
  "credential",
  "file_write",
  "external_send",
]);

function gateToolCall(call: ToolCall, currentUserGoal: string) {
  const reasonMentionsGoal = call.reason
    .toLowerCase()
    .includes(currentUserGoal.toLowerCase());

  if (!reasonMentionsGoal) {
    return {
      allowed: false,
      approvalRequired: false,
      log: `Blocked ${call.name}: reason does not match current user goal.`,
    };
  }

  if (APPROVAL_REQUIRED.has(call.action)) {
    return {
      allowed: false,
      approvalRequired: true,
      log: `Approval required for ${call.name}: ${call.action} action.`,
    };
  }

  return {
    allowed: true,
    approvalRequired: false,
    log: `Allowed ${call.name}: read-only action matches current user goal.`,
  };
}

const blocks: ContextBlock[] = [
  {
    id: "user-1",
    source: "user",
    visibility: "model_visible",
    authority: "data",
    content: "Review this README and summarize setup risks.",
    createdAt: new Date().toISOString(),
  },
  {
    id: "ui-1",
    source: "ui",
    visibility: "client_only",
    authority: "data",
    content: "https://files.example.test/private?X-Amz-Signature=demo",
    createdAt: new Date().toISOString(),
  },
];

try {
  console.log(buildModelContext(blocks));
} catch (error) {
  console.error(String(error));
}

console.log(
  gateToolCall(
    {
      name: "readFile",
      args: { path: "README.md" },
      action: "read",
      reason: "Review this README and summarize setup risks.",
    },
    "Review this README",
  ),
);

The sample is intentionally conservative:

• client_only blocks fail before model submission.
• User and external content cannot smuggle internal-looking tool material.
• Memory without freshness metadata gets flagged.
• Privileged authority is reserved for system-owned blocks.
• Tool calls are checked against the current user goal before approval policy runs.

For a more complete review checklist around access, shared context, files, credentials, and human review, this public page on workflow guardrails is a useful companion. The important part is not the checklist format. It is forcing every action path to say what state it trusts.

If you want a tiny test, add this below the sample and make it part of CI:

function assertNoLeakage(text: string) {
  const leaked = INTERNAL_PATTERNS.filter((pattern) => pattern.test(text));
  if (leaked.length > 0) {
    throw new Error(`Model-visible text contains ${leaked.length} internal pattern(s).`);
  }
}

assertNoLeakage("[user:data] Summarize README setup risks.");

This test will miss real attacks. That is fine. Its job is not omniscience. Its job is to catch category mistakes early: signed URLs in model text, internal tool names in user-originated blocks, and metadata that was supposed to stay in the client.

What this does not solve

A boundary contract reduces blast radius. It does not make prompt injection disappear.

Control	Best for	Not best for	Tradeoff
Source, visibility, and authority labels	Preventing state from crossing lanes silently	Detecting every malicious phrase	Requires plumbing through every context builder
Guardrail classifiers	Catching suspicious content before or after model calls	Replacing permission design	False positives can block useful work
Scope-minimized tools	Reducing damage when a tool is misused	Complex workflows that need broad access	More prompts and approval steps
Human approval gates	Shell, network, credential, file-write, and external-send actions	Low-risk read-only operations	Review fatigue if everything needs approval
Memory freshness metadata	Preventing stale preferences from acting like current policy	Perfectly understanding user intent	Older context may need re-confirmation

The strongest objection is fair: prompt injection is not solved. So do not sell "solved." Sell smaller blast radius, clearer state movement, and better logs when something crosses a line.

My rule is plain: if a field is client-only, privileged, stale, or untrusted, it should not silently become instruction. That rule will not catch every hostile page, poisoned README, or bad memory summary. It will make the failure easier to find, easier to review, and less likely to become a tool action before a human notices.