I spent August building PHAGE, an immune system for fleets of AI agents. It
vaccinates them: it writes prompt-injection payloads, fires them at its own
agents, watches what lands, revokes the tool that got abused, and remembers the
signature so the next mutation of that attack never fires at all.
The number I want to open with has nothing to do with any of that.
It is 1 out of 70 versus 50 out of 70 — and it is a measurement of how the
wording of a system prompt changes whether a model agrees to help.
The component that writes the attacks
PHAGE has a component called VACCINATOR whose job is to author injection
payloads tailored to a specific target agent. If the target is a supplier-relations
bot with send_email and read_contacts in its tool list, VACCINATOR writes
something that tries to abuse exactly those two tools. It calls Gemini 3.5 Flash
to do the tailoring.
Somewhere in the middle of the build I rewrote VACCINATOR's system prompt to
describe the task more accurately — to say plainly that the output would be fired
at a live agent. The refusal rate went through the roof. I reverted the wording,
kept building, and wrote a note to myself that "seven of seven archetypes flip."
That note turned out to be wrong, and finding out how wrong is the most useful
thing I did all month.
Measuring it properly
I built a two-wording matrix. Same seven attack archetypes, same target, same
temperature of 0.7, ten repetitions each. The only thing that varies is the
system prompt wording. The script monkeypatches the module global rather than
editing the constant, so neither wording can contaminate the other — and it runs
strictly sequentially, because threads sharing that global would execute cells
under the wrong wording and misattribute refusals silently.
140 logical calls. About 28 minutes. The results:
Committed wording: 1 refusal out of 70. Rate 0.014.
Accurate wording: 50 refusals out of 70. Rate 0.714.
So the effect is real and it is large. But "seven of seven" does not reproduce.
Five of seven archetypes flip.
And here is the part I did not expect. The effect is not a rate at all. It is
categorical. Five archetypes refused ten times out of ten. Two refused zero
times out of ten. Nothing landed anywhere in between — no archetype came back
at four, or six, or eight.
The two that never flip are instruction-override and
indirect-injection-readonly. They are the two whose payloads imply no state
change and no exfiltration. Everything that asks the target agent to send
something, write something, or leak something flips. Everything that only asks it
to behave differently does not.
I do not think this is a bug, and I do not think it is a jailbreak. It reads to me
like the model is evaluating the described consequence, not the described task —
and that the wording change moved the consequence into view rather than moving
the task out of it. That is a distinction worth more study than a hackathon
allows.
What I can say without hedging: the wording I shipped is not the wording that
describes what the code does. That is disclosed in the repository, in the
write-up, and in the demo video, because a finding that only flatters the project
is not a finding.
What PHAGE actually is
Five components, one local Python process, built on Google's Agent Development
Kit:
MARROW — the fleet orchestrator that runs the cycle.
VACCINATOR — authors and paraphrases payloads via Gemini 3.5 Flash.
ARCHIVIST — semantic memory, backed by Vertex AI Agent Engine's Memory Bank.
SENTINEL — reads OpenTelemetry spans and decides whether an attack landed.
MACROPHAGE — revokes the abused tool, in place, process-wide.
The target fleet is four deliberately vulnerable agents for a small cooperative:
order intake, supplier relay, stock keeper, quote bot. Each has a real tool list
and each is exploitable through it.
The cycle runs in this order, and the third step is the one that matters:
VACCINATOR tailors a payload → ARCHIVIST checks it against Memory Bank before
anything is fired → if it is recognized, the payload is never fired at all →
otherwise it fires at the live target → SENTINEL triages the resulting spans →
if it landed, MACROPHAGE revokes the exploited tool → ARCHIVIST records the
signature.
That pre-fire gate is the whole idea. On first exposure an attack lands, gets
contained, and gets remembered. On second exposure a mutated variant of the
same attack — different words, same intent — is caught by similarity search
before it ever reaches the agent. The demo shows both passes back to back: the
first takes about 73 seconds of real execution, the second about 2.6.
Making the recognition claim falsifiable
"Similarity search catches mutations" is easy to say and easy to fool yourself
about. The obvious failure mode is that the store recognizes paraphrases of
attacks it has already seen and nothing else — which would be memorization
dressed up as generalization.
So the evaluation holds out an entire archetype at a time. Eight folds: train the
signature store on seven archetypes, test on the eighth, rotate. Across the pooled
folds — 175 positives, 60 negatives — that gives AUC 0.9727, with a true
positive rate of 1.00 and a false positive rate of 0.1833 at the adopted
threshold of 0.59.
The false positive rate is the honest weak spot. Eleven of sixty negatives get
flagged. In a real fleet that is a lot of false alarms, and I would not ship this
threshold to production without more work on the negative set.
Three probe distances anchor the scale: byte-identical text sits at 0.386, a
paraphrase with the same intent at 0.529, an unrelated topic at 0.875. The
threshold at 0.59 sits between paraphrase and unrelated, which is where you want
it if the claim is "catches rewordings, ignores everything else."
The cost tier
SENTINEL triages every trace. Traces are the highest-volume thing in the system,
so triage is where the money goes. It runs Gemma 4 26B as a cheap first pass and
only escalates to Gemini 3.5 Flash when Gemma comes back ambiguous.
I have to be honest that this is currently an argument rather than a
measurement. Both calls go through a raw client that ADK does not instrument, so
there are no spans and no token counts on disk for either path. The cascade is
built and it works; the claim that it saves money is untested. Instrumenting it
is the first thing on the list after this deadline.
What is not wired
The original design mapped every immune function onto a Google Cloud primitive.
Several of those are enabled on the project and called by nothing: agent
registry, agent identity, gateway routing, managed inline guardrails. The
architecture diagram in the README deliberately omits them rather than implying
integration that does not exist, and there is a section listing them by name.
There is one gap I would fix first if I had another week: PHAGE writes none of
its own judgements to disk. No verdict, no tier, no containment action, no
recognition distance. Every span the target agents produce is recorded, but the
immune system's own decisions live only in stdout. An immune system with no
memory of what it decided is a strange thing to have built, and I noticed it too
late to fix properly.
What I would tell someone starting one of these
Write down the numbers with the command that produced them, in the same commit.
Every figure in my README maps to a committed artifact, and the one time I
skipped that — the "seven of seven" note — is the one time I was wrong.
Assume your own notes are wrong until a grep confirms them. In one working
session I found five wrong premises in my own build briefs: a file that did not
exist, a citation to a line number that had drifted, a git revert that never
happened. All five were caught by checking before editing, and none would have
been caught by reading carefully.
And when a measurement contradicts the story you were going to tell, run it ten
more times before you decide which one is wrong.
I created this piece of content for the purposes of entering the All Things
Agentic Hackathon. PHAGE is open source; the repository includes the evaluation
artifacts, the refusal-rate raw data, and a read-only dashboard that renders both.
https://phage-dashboard-680106551305.us-central1.run.app/
https://youtu.be/bNjJdOjum9k
BLANK-242
/
phage
An immune system for agent fleets. PHAGE attacks its own agents, learns what a landed exploit looks like, and recognises the same attack reworded — in about two seconds, before it is ever sent.
PHAGE — an immune system for AI agent fleets
PHAGE continuously inoculates the AI agents an organization deploys with tailored prompt-injection and tool-poisoning payloads, quarantines the ones that fail, and remembers every attack signature so a repeat attack is neutralized on recognition rather than re-analysis. It is built for an organization deploying agents with no security staff.
The biological metaphor is the architecture, not decoration — every platform primitive maps to an immune function.
Google Cloud All Things Agentic Hackathon — Fortified Enterprise Fleet Built entirely within the submission window with AI assistance (permitted).
Live dashboard → https://phage-dashboard-680106551305.us-central1.run.app The read-only evaluation and trace viewer, open to anyone — no Google account no sign-in. It serves the committed evaluation artifacts and a redacted copy of the span database; it fires no agent, calls no model, and writes nothing.
The five components
| Component | Immune role | What it does | Form |
|---|---|---|---|
| MARROW |
Top comments (0)