A hospital doesn't fail all at once. It fails one missed signal at a time: a nurse working past the point it's safe to keep going, a supply room running low on something nobody reordered, a credential that quietly lapsed, two schedulers double-booking the same operating room because their systems don't talk to each other. None of it trips an alarm. Each one stays invisible until it becomes a patient-safety incident, and by then it's too late to have caught it early.
That's the gap we built Prudently to close for the All Things Agentic Hackathon's Fortified Enterprise Fleet track: eight specialist AI agents, deployed independently on Google Cloud, reading a hospital's live operating data around the clock and acting the instant something crosses a line. It decides when to raise something on its own. It never decides, on its own, that something should happen.
Here's what that took to build, what broke along the way, and what we'd tell another team building an autonomous agent system.
The shape of the fleet
Eight agents, each an expert in exactly one thing.
Coordinator. The only agent a person talks to directly. It never acts on its own; it routes each request to the right specialist through a security checkpoint first.
Shift Allocation. Tracks clinician hours against safe fatigue limits, proposes reassignments before exhaustion becomes a mistake on the floor.
Inventory Management. Watches stock levels against reorder points, using actual usage rate rather than a fixed shelf count.
Supply Chain Resiliency. Picks a vendor and drafts the purchase order when Inventory flags a shortage.
HR. Tracks credential expirations, finds per-diem coverage when Shift Allocation runs out of people.
Chaos & Continuity. The resilience specialist. It knows 24 hospital emergency scenarios (mass-casualty surges, outbreaks, power and water outages, staff walkouts, cyberattacks) and re-checks all 24 against live data every monitoring cycle, unprompted.
Surgical Scheduling. Catches operating-room conflicts, keeps patient data encrypted at rest.
Medical Representative. The one specialist that talks to the outside world, over an agent-to-agent hop to its own separately hosted service, not a same-process shortcut.
Underneath all eight sits a background watch that never sleeps. On its own schedule, every 90 seconds in production, it re-reads the hospital's live state and diffs it against what it saw last time. It only speaks up when something got worse. A problem that hasn't moved since the last check doesn't get flagged again; being bad because it was already bad ninety seconds ago isn't news.
Autonomy, with a hard boundary
The interesting design decision wasn't making the agents smart. It was deciding exactly where their authority stops.
Every agent can decide when to raise something. No agent can decide, on its own, that something with a real-world consequence should happen. Contacting a vendor, notifying staff, reordering stock, reallocating a shift: all of it lands as a pending approval with a yes or no link, and nothing happens until a person clicks it.
That split, deciding when versus deciding whether, mattered more than any amount of prompt engineering. We didn't try to make a model trustworthy enough to act alone. We made the boundary trustworthy instead, and let the model push as hard as it wants inside it.
Security as a prerequisite, not a feature
Once agents can call each other, coordination stops being the hard problem. Authorization takes over. We built the security gateway, a registry check, a policy check, a trace on every internal call, before we finished the coordination logic between agents. Retrofitting who's allowed to call whom onto working agent-to-agent calls is a much worse day than designing for it up front.
Two other pieces of that same discipline:
Field-level encryption. Patient-identity fields (name, date of birth, contact info) are encrypted with a cloud-managed key before they're written to the database. Two roles in the whole system can decrypt them. Everyone else, including other agents, sees ciphertext.
Two-pass prompt-injection screening. Every inbound vendor message is screened before any model reads it, and screened again on whatever the model pulls out of it. In our own testing, a paraphrased injection attempt slipped past the first pass and got caught by the second, during an actual test run, not a slide.
What broke
The bugs were, without exception, more interesting than the features.
A deployment that lied. More than once, a deploy reported success while still quietly serving an old, broken build for a short window afterward. The failure wasn't the deploy itself. It was everything underneath it that stayed wrong for a few more calls.
A silent autonomy killer. A single misconfigured region setting for the fleet's shared memory store broke the entire autonomy pipeline, with no error visible anywhere a person would normally look. Nothing crashed. It just quietly stopped remembering anything.
A fatigue-tracking bug that erased weeks of data. A record-keying bug meant a clinician's fatigue history was overwriting itself into a single day's record instead of accumulating. The feature had probably never worked correctly until the numbers stopped adding up.
26 emails in five minutes. A depletion-rate bug applied a full day's inventory usage on every short monitoring cycle instead of a small fraction of one. Every tracked item crossed into critical stock within about five minutes of each other, and 26 reorder-approval emails went out before anyone caught it. The tell wasn't a crash; it was the live inventory numbers looking wrong.
A "that's not possible" that wasn't true. Early on we wrote down, as a conclusion, that giving each agent its own cloud identity wasn't possible on the platform, because the CLI tool we were using didn't expose that option. The tool was the limitation, not the platform. Reading what that tool actually sent to Google's API surfaced a working configuration the command-line flag never exposed. All eight agents went from sharing one identity to each carrying its own, individually revocable.
What we'd tell another team
Autonomy and authority are different problems. Conflating "the agent can decide when" with "the agent can decide whether" is the risk, not the model's judgment.
Design the security boundary before the coordination logic. Multi-agent systems are an authorization problem before they're an orchestration problem.
Shared memory between agents is a correctness risk, not a convenience. An agent that can read another specialist's memory can also reason from assumptions it was never meant to have. Scoping memory per-agent limits what each agent can even get wrong.
React to change, not to state. An agent that only speaks up when something is different can run unattended indefinitely without becoming noise a person tunes out. That property matters more in production than it ever does in a demo.
The stack
| Service / model | Purpose |
|---|---|
| Vertex AI Agent Engine | Hosts all 8 reasoning engines |
| Vertex AI Memory Bank | Per-agent scoped long-term memory |
| Gemini | Agent reasoning, tool calls |
| Agent Development Kit (ADK) | Multi-agent orchestration framework |
| Cloud Run | API + dashboard hosting |
| Firestore | Live fleet state, single source of truth |
| Model Armor | Prompt-injection screening |
| Cloud Trace | Distributed tracing, per agent turn |
| Cloud Logging | Structured audit log |
| Pub/Sub | Async messaging |
| Cloud KMS | Field-level PII encryption |
| Firebase Auth | Manager sign-in |
| Agent2Agent (A2A) Protocol | Cross-service hop to Medical Representative |
Built for the All Things Agentic Hackathon, Fortified Enterprise Fleet track.
Top comments (0)