<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: shenn667</title>
    <description>The latest articles on DEV Community by shenn667 (@shenn667).</description>
    <link>https://dev.to/shenn667</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4252668%2F484541f9-6529-4482-83d2-4d44dcd9b509.png</url>
      <title>DEV Community: shenn667</title>
      <link>https://dev.to/shenn667</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/shenn667"/>
    <language>en</language>
    <item>
      <title>Zero-Dependency Verified Agents: Cited or Refused, Gated in CI</title>
      <dc:creator>shenn667</dc:creator>
      <pubDate>Wed, 18 Nov 2026 23:06:03 +0000</pubDate>
      <link>https://dev.to/shenn667/zero-dependency-verified-agents-cited-or-refused-gated-in-ci-3fnh</link>
      <guid>https://dev.to/shenn667/zero-dependency-verified-agents-cited-or-refused-gated-in-ci-3fnh</guid>
      <description>&lt;p&gt;Hallucination is not a model bug. It is a design default.&lt;/p&gt;

&lt;p&gt;Most agent stacks treat the language model as an oracle: prompt in, prose out, no provenance, no refusal contract, no CI gate that can say no. When the model does not know, it still answers â€” fluently, confidently, and wrong. That is not a failure of scale. It is what happens when three responsibilities are left implicit: remembering what was already decided, proving where an answer comes from, and blocking bad output before it ships.&lt;/p&gt;

&lt;p&gt;This post describes a boring alternative that works: three small, zero-dependency Python packages that together make an agent whose answers are either cited or refused, and whose safety properties are enforced by tests and CI gates rather than by hope. Each package does one job. Combined â€” as demonstrated by the reference &lt;code&gt;claude-verified-agent&lt;/code&gt; â€” they form an end-to-end loop: record context, retrieve evidence, gate delivery.&lt;/p&gt;

&lt;p&gt;The three bricks are &lt;a href="https://pypi.org/project/agent-context-bridge/" rel="noopener noreferrer"&gt;&lt;code&gt;agent-context-bridge&lt;/code&gt;&lt;/a&gt; (inter-agent memory, v0.3.2 on PyPI), &lt;a href="https://pypi.org/project/cited-rag/" rel="noopener noreferrer"&gt;&lt;code&gt;cited-rag&lt;/code&gt;&lt;/a&gt; (cite-or-refuse retrieval, v0.1.0 on PyPI), and &lt;a href="https://pypi.org/project/llm-eval-ci/" rel="noopener noreferrer"&gt;&lt;code&gt;llm-eval-ci&lt;/code&gt;&lt;/a&gt; (deterministic CI gates, v0.1.0 on PyPI). All are stdlib-only, offline-first, and installable from PyPI. The reference integration &lt;code&gt;claude-verified-agent&lt;/code&gt; itself is &lt;strong&gt;not&lt;/strong&gt; on PyPI â€” it is a &lt;a href="https://github.com/shenn667/claude-verified-agent" rel="noopener noreferrer"&gt;GitHub reference repo&lt;/a&gt; whose CI installs the three pillars from PyPI at pinned versions, falling back to a portable stdlib path when the network is unavailable. Every number below is measured, with method and limits stated, and every claim points at a file you can re-run.&lt;/p&gt;

&lt;h2&gt;
  
  
  1. Shared memory: stop re-reading the repo at every agent switch
&lt;/h2&gt;

&lt;p&gt;Every AI coding tool starts blind. You settle an auth design with one agent, switch to another, and the new agent re-reads the whole repository to rediscover what you already decided. That re-reading is the token tax of multi-agent work, and it grows linearly with codebase size.&lt;/p&gt;

&lt;p&gt;&lt;a href="https://github.com/shenn667/agent-context-bridge" rel="noopener noreferrer"&gt;Agent Context Bridge&lt;/a&gt; (ACB) replaces the dump with a bounded handoff. One append-only store (&lt;code&gt;.agent_bridge/records.jsonl&lt;/code&gt; plus a human-readable &lt;code&gt;SHARED_CONTEXT.md&lt;/code&gt;) records decisions once via &lt;code&gt;acb record&lt;/code&gt;. At switch time, &lt;code&gt;acb inject --target &amp;lt;hermes|claude|opencode|codex&amp;gt;&lt;/code&gt; replays only the N most recent records (default 5) in the target agent's native format, or &lt;code&gt;acb search "query"&lt;/code&gt; returns only matching hits newest-first. A readonly MCP server (&lt;code&gt;acb-mcp&lt;/code&gt;, stdio) exposes exactly two tools â€” &lt;code&gt;acb_search&lt;/code&gt; and &lt;code&gt;acb_status&lt;/code&gt; â€” both strictly read-only; the destructive &lt;code&gt;prune&lt;/code&gt; command stays CLI-only with interactive confirmation, and unknown tool names fail closed with &lt;code&gt;-32601&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;The measured effect, on the published v0.3.0 benchmark over a 30-record store: naive re-read of 7 source files (60,768 chars, ~15,192 tokens at chars/4) versus bounded &lt;code&gt;acb inject&lt;/code&gt; (793 chars, ~198 tokens). That is (15,192 âˆ’ 198) / 15,192 = &lt;strong&gt;98.7% fewer tokens&lt;/strong&gt;. The same run reports âˆ’95.7% versus re-reading decision docs (4,555 â†’ 198 tokens) and âˆ’90.5% versus forwarding the full shared-context file (2,078 â†’ 198 tokens).&lt;/p&gt;

&lt;p&gt;Methodology, stated honestly: tokens are approximated as chars/4, measured from real files rather than estimated, and the baseline is "what a blind agent re-reads at switch." The key property is structural, not a one-off: &lt;code&gt;inject&lt;/code&gt; cost stays flat as the store grows because it is bounded to the last 5 entries, while naive re-reading grows linearly. &lt;code&gt;search&lt;/code&gt; costs only the hits. Limits: the exact percentage moves with the tokenizer (chars/4 is an approximation), with store size, and with the baseline you compare against â€” which is why the report publishes all three baselines, not just the flattering one. The v0.3.2 suite passes &lt;strong&gt;92/92 tests&lt;/strong&gt; (re-run 2026-11-18: 92 passed in ~26 s), including live CLI and live MCP stdio end-to-end sessions.&lt;/p&gt;

&lt;p&gt;Since the benchmark the package has hardened its distribution surface without changing the measured loop: &lt;code&gt;server.json&lt;/code&gt; validates against the official &lt;a href="https://registry.modelcontextprotocol.io/" rel="noopener noreferrer"&gt;MCP Registry&lt;/a&gt; schema under the name &lt;code&gt;io.github.shenn667/agent-context-bridge&lt;/code&gt; (v0.3.2), Dependabot (pip, weekly) and CodeQL (Python) workflows are configured, and the &lt;a href="https://glama.ai/mcp/servers/shenn667/agent-context-bridge" rel="noopener noreferrer"&gt;Glama Discovery&lt;/a&gt; score is reported at 92%. A submission PR to &lt;code&gt;awesome-mcp-servers&lt;/code&gt; (#19807) is open and reported mergeable.&lt;/p&gt;

&lt;h2&gt;
  
  
  2. Cited answers: verbatim or nothing
&lt;/h2&gt;

&lt;p&gt;Retrieval-augmented generation usually returns prose with decorative citations â€” plausible-looking references that may not match any source text. &lt;a href="https://github.com/shenn667/cited-rag" rel="noopener noreferrer"&gt;&lt;code&gt;cited-rag&lt;/code&gt;&lt;/a&gt; inverts the contract: the answer is extractive and verbatim, every sentence carries an exact &lt;code&gt;[file:Lstart-Lend]&lt;/code&gt; offset, and anything the corpus cannot support produces &lt;code&gt;Insufficient evidence&lt;/code&gt; with zero citations and exit 0. There is no "helpful" middle ground where the model improvises.&lt;/p&gt;

&lt;p&gt;Under the hood it is deliberately unfashionable: BM25 plus sublinear TF-IDF cosine, fused by Reciprocal Rank Fusion (k=60, rank-only so incompatible score scales never mix), followed by a verbatim rerank. No embeddings, no vector server, no network, no API key. An opt-in &lt;code&gt;--llm&lt;/code&gt; adapter exists but is off by default; the default path never touches the network.&lt;/p&gt;

&lt;p&gt;On the bundled corpus (12 docs, &lt;code&gt;eval/golden.jsonl&lt;/code&gt; with 52 questions: 40 answerable plus 12 off-topic), &lt;code&gt;cited-rag eval&lt;/code&gt; reports &lt;strong&gt;Recall@5 = 1.00, MRR = 0.93, correct-refusal = 1.00 (12/12)&lt;/strong&gt; â€” re-run 2026-11-18 with identical output. Definitions matter: Recall@5 is the fraction of answerable questions whose expected document appears in the top-5 retrieved chunk files; MRR is the mean of 1/rank of the first relevant chunk over answerable questions only; correct-refusal is the fraction of off-topic questions the full ask pipeline (search + generate + score gate) refuses. The release gates demand Recall@5 â‰¥ 0.80, MRR â‰¥ 0.60, refusal â‰¥ 0.90 â€” all cleared with margin, and the bundled pytest contract (&lt;code&gt;tests/test_eval.py&lt;/code&gt;) asserts exactly those thresholds on the real corpus so regressions fail loudly. The v0.1.0 QA run additionally verified 3/3 cited sentences verbatim against source offsets with fidelity 1.00, 12/12 off-corpus refusals with no invented citation, and a green suite re-run at &lt;strong&gt;61 passed&lt;/strong&gt;.&lt;/p&gt;

&lt;p&gt;Honest limits: this is a 12-document, 12-chunk technical corpus about the retriever itself â€” a favorable, self-descriptive domain where vocabulary overlap is high. Recall@5 of 1.00 means "the right document is always in the top 5 on this set," not "the agent is always right." MRR of 0.93 means the first relevant chunk usually ranks first, averaged over 40 answerable questions. Small n, narrow domain, no adversarial paraphrase set. The value of the number is not generality â€” it is reproducibility: one command re-derives it deterministically, and the gate blocks release if it drops.&lt;/p&gt;

&lt;h2&gt;
  
  
  3. CI gates: the safety net that can say no
&lt;/h2&gt;

&lt;p&gt;Even cited answers need a last-mile check: does the produced text leak PII or obey an injected instruction? &lt;a href="https://github.com/shenn667/llm-eval-ci" rel="noopener noreferrer"&gt;&lt;code&gt;llm-eval-ci&lt;/code&gt;&lt;/a&gt; is a stdlib-only runner that turns that question into a blocking CI exit code. Cases are JSONL (&lt;code&gt;id&lt;/code&gt;, &lt;code&gt;prompt&lt;/code&gt;, &lt;code&gt;checks[]&lt;/code&gt;, optional &lt;code&gt;max_tokens&lt;/code&gt; / &lt;code&gt;max_latency_ms&lt;/code&gt; / &lt;code&gt;max_cost_usd&lt;/code&gt;); eleven pure checks (&lt;code&gt;contains&lt;/code&gt;, &lt;code&gt;not_contains&lt;/code&gt;, &lt;code&gt;regex&lt;/code&gt;, &lt;code&gt;json_valid&lt;/code&gt;, &lt;code&gt;json_schema_lite&lt;/code&gt;, &lt;code&gt;refusal&lt;/code&gt;, &lt;code&gt;budget_tokens&lt;/code&gt;, &lt;code&gt;budget_latency&lt;/code&gt;, &lt;code&gt;budget_cost&lt;/code&gt;, &lt;code&gt;no_pii_leak&lt;/code&gt;, &lt;code&gt;no_injection_obey&lt;/code&gt;) plus budgets, with strict fail-closed semantics: unknown check names fail the case instead of crashing the runner, malformed budgets fail, absent usage fails any budget check, and PII values are always masked (&lt;code&gt;***&lt;/code&gt;) in reports. Output is &lt;code&gt;report.md&lt;/code&gt; plus machine-readable &lt;code&gt;report.json&lt;/code&gt; (&lt;code&gt;{version, cases: [{id, status, failed_checks[], latency_ms, usage}]}&lt;/code&gt;), and any red case exits 1.&lt;/p&gt;

&lt;p&gt;The semantics are pinned by &lt;code&gt;SPEC_CASES.md&lt;/code&gt; â€” the normative contract shared by dev, security fixtures, and QA â€” so "what does this gate mean" has exactly one answer. The v0.1.0 QA run is the kind of evidence CI people respect: &lt;strong&gt;146 tests passed&lt;/strong&gt; (re-run 2026-11-18: 146 passed), 100% coverage over all 13 check modules, &lt;code&gt;pytest --llm-eval&lt;/code&gt; additivity proven (N cases = N tests), offline install verified in a bare venv with &lt;code&gt;--no-index&lt;/code&gt;, demo CLI in ~0.2 s, exit-1 behavior confirmed in both directions (red suites exit 1, an all-green suite exits 0), and budget checks verified individually (tokens 500 &amp;gt; 100 fails, 300 ms &amp;gt; 50 ms fails, $0.05 &amp;gt; $0.01 fails, missing usage fails closed).&lt;/p&gt;

&lt;p&gt;The reference integration, &lt;a href="https://github.com/shenn667/claude-verified-agent" rel="noopener noreferrer"&gt;&lt;code&gt;claude-verified-agent&lt;/code&gt;&lt;/a&gt;, wires all three pillars into one auditable pipeline â€” record context (ACB), ask with citations (cited-rag), gate delivery (llm-eval-ci) â€” and subjects itself to both layers of proof: &lt;strong&gt;14/14 contract tests&lt;/strong&gt; (exact verbatim citations, off-corpus refusal with zero citations, gate blocks on hostile PII/injection strings, injection-laced questions never obeyed, determinism, JSON contract, memory recording, backend auditability) passing identically on the pillar path and on the portable stdlib-fallback path (re-run 2026-11-18: 14 passed), plus &lt;strong&gt;3 blocking &lt;code&gt;llm-eval&lt;/code&gt; suites&lt;/strong&gt; (&lt;code&gt;non_regression.json&lt;/code&gt;, &lt;code&gt;schema.json&lt;/code&gt;, &lt;code&gt;tripwires.json&lt;/code&gt;) whose red gates fail the build via &lt;code&gt;make gates&lt;/code&gt; and the GitHub Actions &lt;code&gt;pytest + gates&lt;/code&gt; jobs. Its CI installs the pillars from PyPI at pinned versions (&lt;code&gt;agent-context-bridge==0.3.0&lt;/code&gt;, &lt;code&gt;cited-rag==0.1.0&lt;/code&gt;, &lt;code&gt;llm-eval-ci==0.1.0&lt;/code&gt;) and degrades to the stdlib fallback only if the install fails â€” so the green build proves the PyPI artifacts, not just the local checkout. The shipped &lt;code&gt;report.json&lt;/code&gt;/&lt;code&gt;report.md&lt;/code&gt; show the tripwire suite 2/2 passed; the red path (leaked PII or obeyed injection fails, exit 1) is proven in pytest. Limits, again stated plainly: the verified-agent suites are small by design (tripwires, not a red-team corpus), and the package itself ships via GitHub, not PyPI.&lt;/p&gt;

&lt;h2&gt;
  
  
  Quickstart: three commands
&lt;/h2&gt;



&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;pip install agent-context-bridge cited-rag llm-eval-ci
acb init &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; acb record --topic &lt;span class="s2"&gt;"decision"&lt;/span&gt; --details &lt;span class="s2"&gt;"why, in one line"&lt;/span&gt;
cited-rag &lt;span class="nb"&gt;eval&lt;/span&gt; --golden eval/golden.jsonl &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; llm-eval run evals/tripwires.json
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;That is the whole loop: record a decision, verify retrieval quality against the golden set, run the safety tripwires as a blocking gate. Then explore the reference wiring â€” it is &lt;strong&gt;not&lt;/strong&gt; on PyPI, clone it from GitHub:&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;git clone https://github.com/shenn667/claude-verified-agent
&lt;span class="nb"&gt;cd&lt;/span&gt; claude-verified-agent &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; make demo &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; make &lt;span class="nb"&gt;test&lt;/span&gt; &lt;span class="o"&gt;&amp;amp;&amp;amp;&lt;/span&gt; make gates
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Its CI resolves the same three PyPI pins.&lt;/p&gt;

&lt;h2&gt;
  
  
  Build on it
&lt;/h2&gt;

&lt;p&gt;Hallucination stops being mysterious once refusal and gating are designed in. If your agent cannot point at the line it quoted, it should say so; if its output trips a PII or injection check, CI should say no. Small, offline, auditable pieces compose better than one opaque oracle.&lt;/p&gt;

&lt;ul&gt;
&lt;li&gt;agent-context-bridge v0.3.2 â€” &lt;a href="https://pypi.org/project/agent-context-bridge/" rel="noopener noreferrer"&gt;PyPI&lt;/a&gt; Â· &lt;a href="https://github.com/shenn667/agent-context-bridge" rel="noopener noreferrer"&gt;GitHub&lt;/a&gt; Â· MCP Registry &lt;code&gt;io.github.shenn667/agent-context-bridge&lt;/code&gt; (&lt;a href="https://registry.modelcontextprotocol.io/" rel="noopener noreferrer"&gt;registry&lt;/a&gt;) Â· &lt;a href="https://glama.ai/mcp/servers/shenn667/agent-context-bridge" rel="noopener noreferrer"&gt;Glama&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;cited-rag v0.1.0 â€” &lt;a href="https://pypi.org/project/cited-rag/" rel="noopener noreferrer"&gt;PyPI&lt;/a&gt; Â· &lt;a href="https://github.com/shenn667/cited-rag" rel="noopener noreferrer"&gt;GitHub&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;llm-eval-ci v0.1.0 â€” &lt;a href="https://pypi.org/project/llm-eval-ci/" rel="noopener noreferrer"&gt;PyPI&lt;/a&gt; Â· &lt;a href="https://github.com/shenn667/llm-eval-ci" rel="noopener noreferrer"&gt;GitHub&lt;/a&gt;
&lt;/li&gt;
&lt;li&gt;claude-verified-agent (reference integration, GitHub only, not on PyPI) â€” &lt;a href="https://github.com/shenn667/claude-verified-agent" rel="noopener noreferrer"&gt;GitHub&lt;/a&gt;
&lt;/li&gt;
&lt;/ul&gt;

&lt;p&gt;Clone one, run its QA commands verbatim, and check the numbers yourself. That reproducibility is the entire point.&lt;/p&gt;

</description>
      <category>python</category>
      <category>mcp</category>
      <category>ai</category>
      <category>opensource</category>
    </item>
  </channel>
</rss>
