Sentinel V5: Security Boundaries in Agentic Systems

Checkpoint v5 — Sentinel security boundary checkpoint: A–K complete

Sentinel commit: 5e27e265eeb (master, after MR-K !83). This document and all measurement inputs live outside
the repository. It records what was measured and, with equal weight, what was not. Nothing here is a
statement that Sentinel is secure, or that any other product is insecure. "36/36" below means 36 fixtures the
team wrote pass on the code the team wrote; it is a regression contract, not a security rating.

1. What the series is

Four layers, each changed by its own set of MRs, each measured separately, each approved separately:

layer	MRs	what it decides	what it does NOT decide
Detection	A–H (!73–!80)	whether a text span looks like an agent-directed instruction	nothing is authorised or denied by a label
Channel	I (!81)	which channel a file is scanned on (prose = all data region)	not what the file means
Provenance	J (!82)	where the operating context may come from; fail-closed if it is not clean	no sanitisation of the system role — refusal, not repair
Language-aware region	K (!83)	which parts of a .py file are data (docstrings, comments, strings)	no new rule, weight or threshold

Method for every MR: fixture → change → measurement → approval → next MR. Every change was first dry-run
outside the repo against the same corpus, and the corpus was extended with clean controls chosen against
the hypothesis before the change was accepted.

2. Frozen numbers

checkpoint	agent payloads detected	clean controls flagged	verbatim egress of a payload to the stub LLM	P3 instruction in system role
v1 baseline	30/118	0/36	19	not measured
v2 (A–H)	100/118	0/72	51	60/60
v3 + I	100/118 (label only, +30 suspect→hostile)	0/72	26	60/60
v4 + J	100/118	0/72	26	0/60
v5 + K	100/118 (labels identical to v4)	0/90 (72 + 18 adversarial Python-docstring controls)	3	0/60

Constant across all checkpoints: gitleaks 0/118 and semgrep 0/118 on agent payloads (out of their scope, not a
failure); 0 network calls, 0 commits, 0 payload persistence, 0/780 memory-carry rows; two consecutive runs
byte-identical; benchmark pack 36/36, 25 adversarial, 2 documented gaps, 32 stub LLM calls.

3. Verified (measured, reproducible with the external harness)

• Detection. 100/118 agent-directed payloads labelled (synthetic 90 %, external 68 %). 0/90 clean controls flagged, including 18 Python docstring controls written with imperative language, embedded Markdown/SQL/shell, verbatim system prompts, role-play, transcripts with "role": "system", and Authorization: Bearer <token> documentation. 18 misses catalogued by mechanism (pointer 3, long-range context 4, narrative authority 5, third-person demonstrations 3, invoice identity 3); 15/18 are not lexical.
• Channel boundary (I). Prose files scanned on their natural channel; 30 payloads moved suspect → hostile and are neutralised before egress; no rule changed; 0 control changed.
• Provenance boundary (J). Operating context is read only from an explicit source, never from cwd (70 → 0 reads of a repo-shipped SENTINEL_CONTEXT.md); a non-clean context stops the run before any prompt exists (70/70 SKIP context-untrusted, 0 prompts); clean twins go through verbatim (70/70).
• Python data-region handling (K). Docstrings, # comments and strings of .py/.pyi files are data regions. Effect is exactly the 26 predicted payload files (suspect → hostile); the real lexer agrees with the dry-run's virtual lexer on 90/90 .py files; perimeter labels unchanged; 0/90 controls changed.
• Deterministic output. No model in the InjectionGate path; two runs of the full harness byte-identical; rule spans identical before/after K (only the region flag moves).
• No fixture drift. Fixtures 1–35 unchanged by I, J and K; each MR added exactly one fixture (34, 35, 36).
• Isolation. 0 network calls, 0 commits, 0 persisted payload text, 0 real secret values in prompts across all runs (secrets removed from the environment before each run, outbound calls intercepted).

4. Not verified (explicitly out of the evidence)

• Production-world effectiveness. Every number above comes from a shadow Executor with a stub LLM on a corpus of 210 files (118 payloads, 90 controls, 2 positive controls). No production traffic, no real model, no real repository under decision was measured.
• decision_changed. Whether a payload that does reach a real model changes its decision was never measured. Sentinel's containment claim is "the span did not reach the model", not "the model was not influenced".
• Real-world false-positive rate. 0/90 is the FP rate on controls the team wrote to be hard. It is not an estimate of the FP rate on arbitrary repositories; the clean corpus is small and synthetic-heavy.
• Customer demand, marketplace conversion, cost/benefit. Not part of this series; no data.
• The 2 documented gaps in the benchmark pack remain: injection_from_documentation and injection_from_api_response — the label is produced, but no Executor path consumes those channels yet.
• 3 extensionless egress cases. The three external puzzle files without a file extension still reach the stub model verbatim as suspect (channel code by conservative default). Content sniffing was deliberately not implemented; this is a documented boundary, not a scheduled fix.
• Suspect-goes-verbatim is policy. Everything below HOSTILE_AT = 0.6 is forwarded unchanged. Whether that is the right policy needs a decision_changed experiment with a non-stub model.
• Pointer / URL following. The Executor never follows either, so those variants were measured as "not followed", not as "safe when followed". A consumer that does follow has a different boundary.
• Scanner scope. The production repository scanner reads libs/shared/*.js only (6/40 hazmat carriers); the shadow harness feeds every file directly to Executor.processFile. Coverage of a real checkout is therefore narrower than the harness suggests.
• Coverage of other languages. K covers Python only. Docstring/comment syntax of every other non-C-family language (Ruby, shell, YAML, Go raw strings, …) is still scanned by the default lexer.
• Semantic robustness. Rules are narrow lexical conjunctions with NFKC + leet canonicalisation. Paraphrase, translation, typo variants ("pevious instructons") and split-across-files instructions are catalogued misses.

5. How to read this checkpoint

• Labels are observational. InjectionGate never authorises, blocks or decides; the only consumers are the egress copy (hostile spans neutralised) and the trusted-context stop.
• Detection recall, containment and provenance are three different numbers and moved in three different MRs. Quoting one as evidence for another is a misreading of this series.
• A future change that adds a rule, changes a weight or threshold, or extends a lexer must repeat the cycle (dry run on this corpus, adversarial controls, 0 FP gate) — the 0/90 result does not transfer to it.

6. State at close

• A–K complete and merged; master 5e27e265eeb, local checkout clean.
• No implementation in progress. Candidates that would each need their own approval: decision_changed experiment (non-deterministic, costly, different benchmark type); routing documentation/api_response channels through the Executor (closes the 2 gaps); nothing recommended for the 3 extensionless files.